blogs.conchango.com

Microsoft have published some TPC benchmarks for SQL Server 2008 which you can read here. Of most interest to me was the final paragraph which stated: ETL Performance Application Company Result ETL Performance Unisys Load 1TB of data in less than 30 minutes using ETL tools 32-proc (64-core) server [ES7000] World record ...

Jorg Klein has just written a useful blog entry about the SSIS Lookup component entitled "SSIS – Lookup Transformation is case sensitive". Jorg correctly observes that:A lookup transformation uses full caching by default. This means that the first thing it does on execution, is loading all the lookup data in its cache. When this is ...

Yesterday I received an email from a SSIS developer asking questions about some peculiarities of SSIS. One of the points she raised was thus: I found i spent alot of time trying to figure out how to get rid of un-needed fields in the flow.  Not all transforms  allow you to change the output columns.  This is sort of an organizing ...

I believe it can be very useful to share success stories and case studies, particularly when a seemingly insurmountable problem has been overcome so I'm going to enjoy providing you with a SSIS success story right now. A guy going by the name Andy (he didn't give his surname) was having trouble using SSIS to pump the contents of a 5.6m ...

Did you know that it is possible to set expressions on components within the dataflow? Expressions on data-flow components was virtually the very last feature that was added to SSIS 2005 prior to RTM. All components CAN support expressions on their custom properties, the component developer decides whether or not it WILL by setting ...

  This is a post I've been meaning to write for some time but as you may have noticed my blogging output has been pretty lax lately. During my activity on the SSIS forum I've noticed that much of the content is in regard to the dataflow task and that's not a surprise given that its the most useful tool in the SSIS box and also ...

17 days ago I released my Normaliser Component but embarrassingly it turns out there was a small bug. Any null values in the sorted columns on the component input would cause the component to throw a nasty unhandled error. I've fixed the bug in v1.1 which you can get from here: SSIS Normaliser Component v1.1.zip Compressed (zipped) ...

Scene Setting Its a common scenario when doing data integration that one has to process datasets containing redundant data. In many cases this is when data is received in flat files and all data is put into a single file for ease of transport. Perhaps this is better explained by way of illustration; below is a fragment of a file containing order ...

I've found a bug in SSIS today (confirmed by Bob Bojanic from the SSIS team) but it shouldn't have any cause for alarm. Its a very obscure circumstance that causes it and isn't too much of a problem either. However, I thought it would be useful to share the details in case anyone else runs into it. If you never develop custom ...

If you are building asynchronous custom components for SSIS then at some point you will need to use the various Set*() methods of the PipelineBuffer class to populate data into the output buffer. There are a lot of these set methods but the documentation is pretty lax about which you should use depending on the DataType of the column ...