blogs.conchango.com

welcome to the conchango blogging site
Welcome to blogs.conchango.com Sign in | Join | Help
in Search

SSIS Junkie

Conchango are busy and need talented consultants in and around London. Interested? Email me or send me a message

SSIS: But it used to work in DTS (7) - Capture a value from inside the data pump into a variable

Here is the next in my series of posts explaining how common tasks in DTS can now be implemented in SSIS.


Capture a value from inside the data pump into a variable

SSIS enables us to write a value to a variable using the script component. Often when people do this they attempt it in the main data processing method of the component. Unfortunately that is not possible as it is not considered efficient to write to a variable when processing each incoming row due to the "last one wins" effect. Instead, writing to a variable is done in the PostExecute() method. Here is a simple example of using a synchronous script component to write the number of rows passing through the component to a variable. You will notice that the variable write is done in PostExecute():

Imports System
Imports
System.Data

Imports
System.Math

Imports
Microsoft.SqlServer.Dts.Pipeline.Wrapper

Imports
Microsoft.SqlServer.Dts.Runtime.Wrapper


Public
Class ScriptMain
  Inherits
UserComponent
  'Declare a variable that is local to the script component code

  Dim i As Integer

  Public Overrides Sub PreExecute()
    'Initialise the variable

    i = 0
    MyBase
.PreExecute()
  End Sub

  'Input0_ProcessInputRow is called on each incoming row
  Public Overrides Sub Input0_ProcessInputRow(ByVal Row As
Input0Buffer)
    'Increment the variable

    i += 1
  End Sub

  Public Overrides Sub PostExecute()
    Dim vars As
IDTSVariables90
    'Lock a variable so that we can write to it

    Me.VariableDispenser.LockOneForWrite("RowCount"
, vars)
    'Write the value to the variable

    vars(0).Value = i
    'Release the lock

    vars.Unlock() 
    MyBase
.PostExecute()
  End
Sub
End
Class

There are other ways of accomplishing this as you will see if you click here.

 

 

Published 18 July 2006 12:30 by jamie.thomson

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

SSIS Junkie said:

I've said on numerous occasions, in slightly different ways, that making the move from DTS to SSIS

January 11, 2007 21:48
 

Dev said:

Hi Jamie

I have a table with Id as primary key. It is not specified as an identity column.I am using data flow task to migrate data from source to destination table. I need to insert a unique value for Id (datatype int) in the destination table.

One way I can think of is using a variable..setting its value to 0, increment and assign its value to the column...this needs to be done for each row...its not very efficient and also i don't how to go about doing it in SSIS :(

My task is to insert unique value (int value preferably in ascending order) for the primary key.

It would be really helpful if you could suggest me a way to accomplish this.

~Dev

June 11, 2007 11:42
 

jamie.thomson said:

June 11, 2007 16:31
 

Dev said:

Hi Jamie

Thanks a lot for the articles...it specified exactly what i wanted.

~Dev

June 12, 2007 09:51
 

Angel said:

Hi Jamie,

I wanna create automatic data transforation task like what we could before.

Here is a simple script:

intLoop = 1

For Each Item In aDestinationColList

Set oTransform = oDataPump.Transformations.New("DTSPump.DataPumpTransformCopy")

oTransform.Name = "Transform" & intLoop

oTransform.TransformFlags = 1 + 2+ 4+8+16 + 32

SourceItem = Trim(aSourceColList(intLoop - 1))

DestinationItem = Trim(Item)

oTransform.SourceColumns.AddColumn SourceItem, intLoop

oTransform.DestinationColumns.AddColumn DestinationItem, intLoop

intLoop = intLoop + 1

oDataPump.Transformations.Add oTransform

Next

Set oTransform = oDataPump.Transformations.New("DTSPump.DataPumpTransformCopy")

oTransform.Name = "TransformZone"

oTransform.TransformFlags = 1 + 2+4+8+16 + 32

Do you know how we can create this  in SSIS?

June 18, 2007 16:55
 

jamie.thomson said:

Angel,

Do you mean you want to add a transformation at execution-time? You can't do that I'm afraid.

-Jamie

June 18, 2007 17:09
 

Angel said:

You mean no way :( But we could do at DTS and now we can not convert this script to the SSIS script, right? I need a create auto-create data pump tasks. Can we do with another way?

June 19, 2007 06:12
 

jamie.thomson said:

June 19, 2007 07:24
 

Dylan said:

jamie.thomson

re The approach you linked to above for "Dev" , the links no longer work.

Where can I find this info?

August 30, 2007 01:23
 

jamie.thomson said:

Dylan,

I think its a temporary problem.

-Jamie

August 30, 2007 03:27
 

PatKat said:

What if ur PostExecute() Method is not getting called itself. I'm facing this issue in one of my codes which is as follows:-

   Public Overrides Sub PostExecute()

       'MsgBox("In first script...")

       'Me.ReadWriteVariables("maxcnt").Value = cnt

       'Me.Variables.maxcnt = cnt

       'vars(0).Value = cnt

       'MsgBox("variable is " & Me.Variables.maxcnt)

       Dim vars As IDTSVariables90 = Nothing

       Me.VariableDispenser.LockOneForWrite("maxcnt", vars)

       'Me.VariableDispenser.GetVariables(vars)

       vars(0).Value = cnt

       vars.Unlock()

       MyBase.PostExecute()

   End Sub

I am not sure what the issue is out here!!!!

October 19, 2007 13:11
 

Saurabh said:

Hi Everyone,

I am facing the lock problem when I am trying to write a variable. I have used exactly the same code as suggested by Jamie in this post. When I run the Data Flow task I get the following exception HRESULT: 0xC001404D

I know this has got something to do with Variable Locking but I am not able to figure out the reason or the resolution.

Thanks,

Saurabh

November 23, 2007 08:26
 

Phillip said:

Saurabh,

I had the same problem but it works if I in the PostExecute section remove everything and just use it like this:

Public Overrides Sub PostExecute()

       'Write the value to the variable

       Variables.UnknownKst = text

       MyBase.PostExecute()

   End Sub

//phillip

February 27, 2008 16:00

Leave a Comment

(required) 
(optional)
(required) 
Submit

This Blog

Syndication

News

Powered by Community Server (Personal Edition), by Telligent Systems