blogs.conchango.com

Browse by Tags

• Ab Initio
• best practice
• BizTalk
• Checkpoints
• Conchango
• configurations
• connection manager
• Datareader source adapter
• Demo
• DTS
• EngineThreads
• ExclusionGroup
• Expressions
• Federated Querying
• Foreach Loop
• fuzzy
• fuzzy logic
• Informatica
• lookup
• Microsoft
• Nugget
• ole db source adapter
• pipeline component
• raw files
• regex
• regular expressions
• RetainSameConnection
• Scripting
• SQL
• SQL Server
• SQL Server 2005
• sqL server 2008
• SSIS
• template
• T-SQL
• Variables
• video

• SSIS Nugget: Conditionally email a file containing error rows

  Someone emailed me today asking: I have an SSIS package that outputs data that can’t be inserted into destination tables into flat log files [JT: I presume he means that error rows are redirected to a flat file] What is the easiest way to then after processing Read More... Posted 11 July 2008 20:02 by jamie.thomson | 6 Comments Filed under: SQL Server Integration Services, SQL Server, SSIS, Dataflow Task, Expressions, Demo, Nugget

• SSIS: SQL 2008 Benchmarks

  Microsoft have published some TPC benchmarks for SQL Server 2008 which you can read here . Of most interest to me was the final paragraph which stated: ETL Performance Application Company Result ETL Performance Unisys Load 1TB of data in less than 30 Read More... Posted 27 February 2008 20:01 by jamie.thomson | 6 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task

• SSIS: Case-sensitivity in Lookup component

  Jorg Klein has just written a useful blog entry about the SSIS Lookup component entitled "SSIS – Lookup Transformation is case sensitive" . Jorg correctly observes that: A lookup transformation uses full caching by default. This means Read More... Posted 12 February 2008 16:59 by jamie.thomson | 7 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task

• SSIS: Hiding columns in the pipeline

  Yesterday I received an email from a SSIS developer asking questions about some peculiarities of SSIS. One of the points she raised was thus: I found i spent alot of time trying to figure out how to get rid of un-needed fields in the flow. Not all transforms Read More... Posted 16 January 2008 02:12 by jamie.thomson | 2 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task

• SSIS: A performance tuning success story

  I believe it can be very useful to share success stories and case studies, particularly when a seemingly insurmountable problem has been overcome so I'm going to enjoy providing you with a SSIS success story right now. A guy going by the name Andy Read More... Posted 18 December 2007 19:56 by jamie.thomson | 3 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task

• SSIS: Dataflow expressions

  Did you know that it is possible to set expressions on components within the dataflow? Expressions on data-flow components was virtually the very last feature that was added to SSIS 2005 prior to RTM. All components CAN support expressions on their custom Read More... Posted 09 October 2007 05:12 by jamie.thomson | 7 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task, Expressions

• SSIS: Dataflow mechanics

  This is a post I've been meaning to write for some time but as you may have noticed my blogging output has been pretty lax lately. During my activity on the SSIS forum I've noticed that much of the content is in regard to the dataflow task and Read More... Posted 09 October 2007 04:39 by jamie.thomson | 24 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task

• SSIS: Normaliser Component bug fix

  17 days ago I released my Normaliser Component but embarrassingly it turns out there was a small bug. Any null values in the sorted columns on the component input would cause the component to throw a nasty unhandled error. I've fixed the bug in v1.1 Read More... Posted 25 September 2007 05:44 by jamie.thomson | 7 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task

• SSIS: The Normaliser Component

  Scene Setting Its a common scenario when doing data integration that one has to process datasets containing redundant data. In many cases this is when data is received in flat files and all data is put into a single file for ease of transport. Perhaps Read More... Posted 07 September 2007 20:16 by jamie.thomson | 9 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task, video

• SSIS: Obscure bug when developing custom components.

  I've found a bug in SSIS today ( confirmed by Bob Bojanic from the SSIS team) but it shouldn't have any cause for alarm. Its a very obscure circumstance that causes it and isn't too much of a problem either. However, I thought it would be Read More... Posted 05 September 2007 21:26 by jamie.thomson | 1 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task

• SSIS: PipelineBuffer.Set*() methods

  If you are building asynchronous custom components for SSIS then at some point you will need to use the various Set*() methods of the PipelineBuffer class to populate data into the output buffer. There are a lot of these set methods but the documentation Read More... Posted 05 September 2007 08:31 by jamie.thomson | 1 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task

• SSIS: Big improvements to Lookup in SQL Server 2008

  I awoke this morning to find my inbox full of automated replies from msftconn@microsoft.com thus indicating that someone on the SSIS team has been busy updating many of my submissions to Connect . Generally it was the usual "Closed, Won't Fix" update Read More... Posted 21 August 2007 20:25 by jamie.thomson | 8 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task, sqL server 2008, lookup

• Informatica -vs- SSIS pipeline throughput

  A long time ago my colleague The Galloping Data Architect and I wrote a paper entitled " A Comparison of Enterprise Class Data Integration Tools. SQL Server Integration Services for the Informatica Customer ". You can get it from here and read what I Read More... Posted 21 August 2007 03:23 by jamie.thomson | 7 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task, Informatica, Ab Initio

• SSIS Nugget: Output the number of processed rows

  Here's a handy little tip for when you are building custom destination components or script destination components that enables you to log the number of processed rows. Have you ever seen that handy little message that appears in your logs that tells Read More... Posted 03 July 2007 20:30 by jamie.thomson | 3 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task, Scripting, Nugget Attachment(s): JT OutputtingRowsProcessed.zip

• SSIS: A great article on performance tuning the data-flow

  Just a quick post to point you to an article that gives tips on performance tuning the SSIS data-flow. It was written by Kirk Haselden, the bloke who just happened to be the manager of the development team that built the thing. Here's the link: http://www.simple-talk.com/sql/sql-server-2005/sql-server-2005-ssis-tuning-the-dataflow-task/ Read More... Posted 08 May 2007 00:10 by jamie.thomson | 3 Comments Filed under: SQL Server Integration Services, SSIS, Dataflow Task