blogs.conchango.com

welcome to the conchango blogging site
Welcome to blogs.conchango.com Sign in | Join | Help
in Search

SSIS Junkie

Conchango are busy and need top level Technical Architects for Microsoft & Open Source platforms in and around London. Interested? Email me or send me a message

SSIS: The steep SSIS learning curve!

As anyone who has been using SQL Server Integration Services (SSIS) for a while will know there are lots and lots of knobs within a package that can be tweaked to alter behaviour. Many of these aren't obvious and the steep learning curve that many people attribute to SSIS education is, in my opinion, down to knowing about these little nuances that can affect your package development. I'm thinking of things like:

  • The need to set DelayValidation=TRUE on any data-flow that contains a Raw File Source Adapter.
  • Sometimes when developing you need to set DisableEventHandlers=TRUE in order to unit test some functionality. It is easy to forget to set it back to FALSE.
  • When accessing some relational data sources (particularly Oracle) you need to set AlwaysUseDefaultCodePage=TRUE in order to suppress a warning.
  • Setting BypassPrepare=TRUE on the Execute SQL Task is sometimes a pre-requisite to your SQL statement being validated
  • In order for a package that uses checkpoints to restart from the previously failing task (rather than the subsequent task), the task in question needs to have FailPackageOnFailure=TRUE.
  • Packages that have ProtectionLevel='DontSaveSensitive' need to use configurations in order for passwords to be used at execution-time
  • Columns in the data-flow that are not used will cause warnings at execution-time. Those warnings are useful, but annoying.
  • You need to manually set IsSorted=TRUE if you know that data in a data-path is sorted and there is no SORT component in the data-path.
  • Setting IsSorted=TRUE will not sort the data for you
  • [Can you think of any more? There must be loads]

These aren't things that are generally documented, they are just things that you pick up through using the product.

 

I'm interested in ways that these problems can be solved. I have a few ideas myself but first of all I'd like to know the opinion of the community. Is it worth trying to solve these problems or should we just accept that this is just the way things are with SSIS and we have to get on with it? If we should try and solve it, how should we go about it? Should the SSIS team be building automatic checks into the Designer?

Let me know in the comments section below.

 -Jamie

 

Published 20 April 2007 22:03 by jamie.thomson

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

Kory said:

The Oracle Default code page warning also appears in the lookup component if you use an oracle source as a reference table.  However, I don't know any place to suppress the warning on this component...

April 21, 2007 15:30
 

Bart said:

More annoying stuff:

- your SSIS expression in a Conditional Split or Derived Column has to be on a single line. This can get rather unreadable, as below expression shows. It just figures out whether Fld1 and Fld 2 are different:

(ISNULL(Fld1)&&!ISNULL(Fld2))||(!ISNULL(Fld1)&&ISNULL(Fld2))!!(!ISNULL(Fld1)&&!ISNULL(Fld2)&&Fld1 != Fld2)

- You can't change the scope of a variable. It's not too complicated to delete and recreate it, but it's annoying.

April 23, 2007 12:34
 

Greg Galloway said:

We haven't tackled any SSIS stuff with the BIDS Helper project yet, but I'm wondering if these would be worth checking:

http://www.codeplex.com/bidshelper/WorkItem/View.aspx?WorkItemId=9800

April 23, 2007 16:26
 

jamie.thomson said:

Greg,

Interesting stuff, and definately along the lines of what I was thinking. I'm hoping to announce something similar very soon (there, I've committed to it now so I HAVE to do it).

Gotta say - I love Codeplex :)

cheers

Jamie

April 23, 2007 16:36
 

jamie.thomson said:

Bart,

I wholeheartedly agree and have suggested this to Microsoft here: https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=226188

Feel free to vote and add a comment.

And also raise a connect submission regarding being able to change the scope of a variable.

-Jamie

April 23, 2007 16:46
 

John Welch said:

I think some of these are checks that could be included in the SQL Server Best Practices Analyzer. It seems to be a little lacking when it comes to the SSIS side.

Others, I think could be remedied by the community through things like BIDSHelper or other open source projects.

How about an option to sort the packages alphabetically? Without having to hand-edit the project file?

April 27, 2007 02:21
 

jamie.thomson said:

John,

I think there's great potential for a BIDS add-in to do this sort of stuff.

As for sorting packages, check this out: http://www.sqldbatips.com/showarticle.asp?ID=78

-Jamie

April 30, 2007 16:54
 

Dora said:

There is no way to perform ISNUMERIC test in expressions. One way of accomplishing it is to try to convert it to DT_I4 and if it succeeds its numeric and if it fails it is not!!! isnt that something so easy to implement in a functionin expressions ?? Thats annoying!

Dora.

May 2, 2007 19:45
 

Ted Pin said:

Jamie, that BIDS add-in is GREAT! Goodbye annoying package XML editing!

March 17, 2008 19:19
 

Dave Neeley said:

Yes! Sorted SSIS packages in BIDS! Thanks for the link!

May 2, 2008 17:29
 

Gabriel said:

Sorting SSIS Packages:

Modify the [msdb].[dbo].[sp_dts_listpackages] stored procedure to include  

order by name

at the end

May 28, 2008 21:29

Leave a Comment

(required) 
(optional)
(required) 
Submit

This Blog

Syndication

News

Powered by Community Server (Personal Edition), by Telligent Systems