blogs.conchango.com

Quite often you may want to process data from multiple files all at the same time. There are a number of options available to you if you want to do this. 1. Have a seperate source adapter for each file and amalgamate them together with the Union All component. Advantages of this approach: This works well if the files are of ...

In January of this year I posted instructions and some code to show how one could use the Recordset destination in conjunction with a source script component to negate the need for a raw file to pass data between data-flows in the same package. I didn't understand why anyone would want to do this given that raw files are much easier to ...

"I have a file that is a list of orders. Each record in the file contains the order number and the item that has been ordered along with information about that item (e.g. Quantity ordered). I want to split the file into an OrderHeader and an OrderDetail table with RI between those tables. How do I do that?" I have seen the above ...

Update below There has been, and still is, quite alot of conjecture in Conchango as to whether one should use SSIS data-flows or stored procedures for ETL. Traditionally we have implemented ETL solutions using stored procedures. DTS was used but as little more than a workflow engine, occasionally using the data-pump for importing data from ...

It is a very common scenario to be able to load two tables in the same data-flow that have a foreign key relationship between them. The question of how to do this gets raised very often on the SSIS forum so I figured I'd try and address it here. Basically, its not possible to do this without using a workaround. There are two possible workarounds ...

Short one this. HarshB asked here: http://blogs.conchango.com/jamiethomson/archive/2006/01/04/2540.aspx#2670 if you need to deploy raw files along with your packages in order to use them. Happily the answer is no - if you set DelayValidation=TRUE on the data-flow that consumes data from the raw file then you won't need to deploy it. Of ...

I thought it would be worth publishing a list of guidelines that I see as SSIS development best practices. These are my own opinions and are based upon my experience of using SSIS over the past 18 months. I am not saying you should take them as gospel but these are generally tried and tested methods and if nothing else should serve as a basis ...

Many moons ago on 8th February 2005 I posted a blog entry explaining how one could navigate over a recordset that is stored in an Object variable, created there by a Recordset destination. That blog entry is here: http://blogs.conchango.com/jamiethomson/archive/2005/02/08/960.aspx As it transpired that blog entry became fairly redundant as ...

The following question was posed on the SSIS forum recently: I have a table with 25 Million records. I want to transfer the records from the table to flat files. One flat file can contain only 1 Million record. So I need to generate 25 flat files with 1 Million in each file. Whats the best way to accomplish this task using SSIS. On the surface ...

Just lately I've been breaking the cardinal rule of blogging: ''Thou shalt always blog''. Well the reason is simply that I've been heads-down on my current project and: a) Haven't had anything to blog about and b) Haven't got time to anyway!   Well b) is still true but a) isn't so I'm back blogging! This is all to do with using ...