Welcome to blogs.conchango.com

I have been having an email back-and-forth with Jon Jaques which began with us discussing various limitations regarding BLOB support within the OLE DB Command and ended with Jon’s blog entry SSIS and the Perils of Embedded SQL where he discusses the drawbacks of the Execute SQL Task – namely that the Execute SQL Task has no understanding of the ...

Someone emailed me today asking: I have an SSIS package that outputs data that can’t be inserted into destination tables into flat log files [JT: I presume he means that error rows are redirected to a flat file] What is the easiest way to then after processing has completed: A) Check to see whether there has been any data that could not be ...

Microsoft have published some TPC benchmarks for SQL Server 2008 which you can read here. Of most interest to me was the final paragraph which stated: ETL Performance Application Company Result ETL Performance Unisys Load 1TB of data in less than 30 minutes using ETL tools 32-proc (64-core) server [ES7000] World record ...

Jorg Klein has just written a useful blog entry about the SSIS Lookup component entitled "SSIS – Lookup Transformation is case sensitive". Jorg correctly observes that:A lookup transformation uses full caching by default. This means that the first thing it does on execution, is loading all the lookup data in its cache. When this is ...

Yesterday I received an email from a SSIS developer asking questions about some peculiarities of SSIS. One of the points she raised was thus: I found i spent alot of time trying to figure out how to get rid of un-needed fields in the flow.  Not all transforms  allow you to change the output columns.  This is sort of an organizing ...

I believe it can be very useful to share success stories and case studies, particularly when a seemingly insurmountable problem has been overcome so I'm going to enjoy providing you with a SSIS success story right now. A guy going by the name Andy (he didn't give his surname) was having trouble using SSIS to pump the contents of a 5.6m ...

Did you know that it is possible to set expressions on components within the dataflow? Expressions on data-flow components was virtually the very last feature that was added to SSIS 2005 prior to RTM. All components CAN support expressions on their custom properties, the component developer decides whether or not it WILL by setting ...

  This is a post I've been meaning to write for some time but as you may have noticed my blogging output has been pretty lax lately. During my activity on the SSIS forum I've noticed that much of the content is in regard to the dataflow task and that's not a surprise given that its the most useful tool in the SSIS box and also ...

17 days ago I released my Normaliser Component but embarrassingly it turns out there was a small bug. Any null values in the sorted columns on the component input would cause the component to throw a nasty unhandled error. I've fixed the bug in v1.1 which you can get from here: You will need to uninstall the previous version (via Control ...

Scene Setting Its a common scenario when doing data integration that one has to process datasets containing redundant data. In many cases this is when data is received in flat files and all data is put into a single file for ease of transport. Perhaps this is better explained by way of illustration; below is a fragment of a file containing order ...