Welcome to blogs.conchango.com Sign in | Join | Help

Welcome to blogs.conchango.com

SSIS Junkie

SSIS: Comparing methods of sourcing surrogate keys for a fact table

Back in February I posted a blog entry telling you about an article I had written at SQLServerCentral.com called "The New ETL Paradigm". Happily it turns out that at least one person has downloaded the example package and had a play with it, that person being Marco Russo.

Part of the article explained a method for obtaining dimensional surrogate keys for insertion into a fact table using MERGE JOIN transformations. In another seperate article on SQLIS.com I have advocated using the LOOKUP transformation to do effectively the same thing.

I don't know if Marco has read that second article but he took this approach of using LOOKUP transformations and adapted my package to do some proper stress testing of the 2 approaches. He put 600000 rows through the pipeline and found overwhelmingly that using LOOKUPs is preferable to MERGE JOINs, primarily because the former is less resource intensive. Less resources invariably means quicker execution times which is after all what we're ultimately all after.

Marco has talked about his experiences here and its well worth a read. He had some issues around dealing with unknown dimension members which led to Marco, Michael Barrett Johnson and myself having a short discussion about the different approaches to dealing with this problem in the comments section of the blog.

-Jamie

 

Published 03 April 2005 11:49 by jamie.thomson

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

TrackBack said:

Business logic and data integration with SQL Server Integration Services
April 4, 2005 11:35
 

TrackBack said:

Handling Data Integrity Issues in Analysis Services 2005
April 6, 2005 08:46
 

Gaane said:

Your site is realy very interesting.
September 17, 2005 10:23
 

Paromita Sen said:

Hi Jamie,

I was just wondering if I can achieve both the operations  updation(if the record is there) and insertion(if the record is not there) of data on destination by implementing lookup on my reference table.

Thank you.
April 26, 2006 19:42
 

jamie.thomson said:

Paromita,
Yes, you can do that. This: http://www.sqlis.com/default.aspx?311 explains how.
-Jamie
April 26, 2006 22:30

Leave a Comment

(required) 
(optional)
(required) 
Submit

This Blog

Syndication

Powered by Community Server (Personal Edition), by Telligent Systems