Unravelled Development

Loading a Data Warehouse Dimension with RabbitMQ and SSIS

In this post I’m going to take a break from the actual development of custom components for SSIS and show how the RabbitMQ Source that I have created so far could be used in an ETL package to load a data warehouse. The Scenario The scenario for this post is a company that sells cool stuff over the internet, they have a website where people can signup and registered users can place orders.

Part 4 - Add a Custom UI to the RabbitMQ Source

A lot of the content in this article is similar to what I did in Part 2, so if you haven’t read it I suggest you go back and read it now :) The next part of this series will take a look at implementing a custom UI for configuration of the RabbitMQ Source component. The main reference for this entry can be found on MSDN. A shout out and my thanks goes to Tillmann Eitelberg (Web | Twitter) who pointed me in the right direction when I was trying to figure out how to get a list of all the connection managers in a package.

Part 3 - Building a Custom Source Component

So far in this series I have shown how to create a custom connection manager and also how to create a user friendly interface to configure and test it. In this post I’m going to outline the steps I went through to create a custom source component for SSIS. By the end of this post I will show messages that are in a RabbitMQ queue being consumed by an SSIS package.

Part 1 - Building a Custom Connection Manager

Following on from my introductory post, here I will be explaining how to develop a Custom Connection Manager for RabbitMQ. This connection manager will then be used in future posts by a custom Source and Destination component. What is a Connection Manager for? From the MSDN documentation: “Integration Services uses connection managers to encapsulate the information needed to connect to an external data source. […] If the connection managers and external data sources supported by Integration Services do not entirely meet your requirements, you can create a custom connection manager.

Part 2 - Adding a Custom UI to the Connection Manager

In my previous post I explained how to create a custom connection manager for SSIS to connect to RabbitMQ. In this post I’m going to extend this further by adding a custom UI where the connection can be configured and tested. Preparing the Project The first step is to add a few more references: Microsoft.SqlServer.Dts.Design Microsoft.SqlServer.DTSPipelineWrap Microsoft.SqlServer.DTSRuntimeWrap Microsoft.SqlServer.PipelineHost These assemblies can be found in C:\Program Files (x86)\Microsoft SQL Server\110\SDK\Assemblies

Building Custom SSIS Components

Introduction I’ve been using SSIS for a long time and have always been curious about developing custom sources and destinations. After having a bit of a look through the MDSN documentation ( last night I thought I would give it a go. So over the next couple of blog posts I’m going to attempt to create a custom Connection Manager, Source and Destination for RabbitMQ. RabbitMQ is a widely used message broker built on the Advanced Message Queuing Protocol (AMQP).