Microsoft SQL Server 2000 Data Transformation Services- P11

The PrecedenceBasisproperty is set to the execution result,and the Valueproperty indicates that failure is the result that is to trigger the precedence.L ISTING 24.1 This ActiveX Script

collection with the Addmethod The PrecedenceBasisproperty is set to the execution result,and the Valueproperty indicates that failure is the result that is to trigger the precedence.

L ISTING 24.1 This ActiveX Script Creates a New On Failure Precedence Constraint

Option Explicit Function Main() Dim pkg, stpSource, stpDestination, prc Set pkg = DTSGlobalVariables.Parent Set stpSource = pkg.Steps(“stpBulkInsert”) Set stpDestination = pkg.Steps(“stpInsertCustomer”) Set prc = stpDestination.PrecedenceConstraints.New(stpSource.Name) prc.PrecedenceBasis = DTSStepPrecedenceBasis_ExecResult

prc.Value = DTSStepExecResult_Failure stpDestination.PrecedenceConstraints.Add prc Main = DTSTaskExecResult_Success

The Execution Status of the Step

Steps execute when all of their precedence constraints have been met and the step has a status

ofDTSStepExecStat_Waiting If the step has any other status, it will not be executed.You can change the execution of a step by setting the ExecutionStatusproperty of the Stepobject in code:

• If a step has already been executed and you then set the ExecutionStatusproperty to

DTSStepExecStat_Waiting, the step will be executed again

exe-cuted The status that makes the most sense to use for this purpose is

NOTE

Chapter 16, “Writing Scripts for an ActiveX Script Task,” has an example of setting up a loop

in a DTS package

Threads and Priority of Execution

DTS is a multithreaded application Many tasks can be executed simultaneously, each one withits own separate thread

Package Level Thread Execution Parameters

There are two thread execution properties for the DTS package as a whole—Priority class andLimit the maximum number of tasks executed in parallel These properties can be set on theGeneral tab of the DTS Package Properties dialog, as shown in Figure 24.4 You can open thisdialog by choosing Properties on the Package menu when no objects are currently selected inthe Design Sheet

The Priority Class

This setting determines the Microsoft Win32 process priority class for the DTS package when

it is executed The possible values are low, normal, and high

In the object model, this setting is the PackagePriorityClassproperty of the Packageobject

The three allowed settings for this property are displayed in Table 24.3

The Maximum Number of Tasks Executed in Parallel

This setting limits the number of steps that are allowed to execute concurrently on separatethreads The default value for this setting is four

This setting can affect the performance of a DTS package Raising this value can increase thespeed of a package’s execution, especially when multiple processors are available More stepscan be executed simultaneously, as long as each step has had its precedence constraints satis-fied But if this value is raised too high, package execution can be slowed because of excessiveswitching between threads

In code, this setting is the MaxConcurrentStepsproperty of the Packageobject

Step Level Thread Execution Parameters

There are six settings in the Execution group on the Options tab of the Workflow Propertiesdialog:

• Task priority

• Execute on main package thread

• Close connection on completion

• DSO rowset provider

• Disable this step

• Fail package on step failure

Task Priority

The task priority gives a precise level of control over the execution priority of an individualtask The package priority class sets the overall thread priority class to low, normal, or high

The task priority sets the relative thread priority within each of the three priority classes

five constants that can be used for this property are shown in Table 24.4

T ABLE 24.4 Constants Used for the RelativePriority Property of the Step Object

Execute on Main Package Thread

DTS normally spawns separate threads to execute different steps of the package This settingchanges that behavior for one particular step by forcing it to be executed on the main packagethread

These are the situations where it is necessary to execute a process on the main package thread:

• If the data provider is not free-threaded and does not support parallel execution of tasks

This is true for the Microsoft Jet OLE DB Provider, as well as the providers for Excel,dBase, Paradox, and HTML files If more than one task is being executed with one ofthese providers at the same time, they should all be executed on the main packagethread

• If you are using custom tasks that have been created with Visual Basic

• If you are executing a package from Visual Basic

• If you want to debug multiple ActiveX Scripts with the script debugger provided with

TheExecuteInMainThreadproperty of the Stepobject implements this option This is aboolean property with a default value of FALSE.

Close Connection on Completion

The default behavior for opening and closing data connections is as follows:

• Do not open a connection until it is needed by a task

• Do not close an open connection until the package completes its execution

This default behavior is usually the most efficient because it minimizes the number of timesthat data connections have to be established

The Close connection on completion option allows you to override the default behavior for aparticular step by closing all of the step’s connections when the step is finished

There are several reasons to consider using this option:

• Some data providers have better performance if connections are not left open

• If there are many connections in a package and inadequate memory resources, closingthe connections could conserve memory and improve overall performance

• You have to close a connection before dynamically modifying it For example, you not change the file being used in a text file data source if that connection is open.You can set this option in code with the CloseConnectionproperty of the Stepobject This is

can-a boolecan-an property with can-a defcan-ault vcan-alue of FALSE

DSO Rowset Provider

This option allows a DTS package to return a recordset

This option is the IsPackageDSORowsetproperty of the Stepobject, a boolean value with adefault value of FALSE It is discussed in Chapter 23, “The DTS Package and Its Properties.”

Disable This Step

When you choose this option, you block the execution of this step when the package is cuted As discussed in the section on precedence constraints, you can specify another task to

exe-Errors can be generated if several tasks are being executed simultaneously using data sources or custom tasks that do not support parallel execution Use the Execute on main thread option to avoid those problems.

CAUTION

run if, and only if, a particular task is disabled You do this by using the DTSStepExecStat_

Inactiveconstant for the Valueproperty and the DTSStepPrecedenceBasis_ExecStatus

PrecedenceConstraintobject

Fail Package on Step Failure

When this option is selected, the package is terminated with a failure if this step fails Bydefault, this option is not selected so that a step failure does not cause the package to fail

the only extended property of this object It is a boolean property with a default value of

If both FailOnError and FailPackageOnError are set to FALSE (their default settings), all the steps in the package can fail and the package will still complete successfully

NOTE

Transactions in DTS Packages

Some or all of the tasks in a DTS package can be joined together into transactions If you use atransaction in a package, an error in one task will cause all the data changes made by othertasks in the transaction to be rolled back If you do not use transactions, data modificationsremain that have already been completed, even if an error occurs that causes a task or theentire package to terminate with an error

A package can have many transactions, but only one of them can be in effect at a time

Whether or not data modifications are successfully rolled back when a transaction failsdepends on the transactional support of the OLE DB provider

The transaction properties are set in two places—for the package as a whole and for each task

Transaction Properties Set at the Package Level

You can set the three package transaction properties on the Advanced tab of the DTS PackageProperties dialog This tab is shown in Figure 24.5

This option is the UseTransactionproperty of the Packageobject It is a boolean propertywith a default value of True

Commit On Successful Package Completion

If this option is selected and a transaction is in effect, that transaction will be committed matically when the execution of the package is completed

auto-If this option is set to FALSE, a transaction that is in progress when the package completes itsexecution will be rolled back

TheAutoCommitTransactionproperty of the Packageobject sets this option This is a booleanproperty with a default of True

Transaction Isolation Level

The transaction isolation level can be set to one of five levels in the DTS Designer These fivelevels are assigned using eight constants for the corresponding property, the

TransactionIsolationLevelproperty of the Packageobject

Here are some definitions of terms used in defining transaction isolation levels:

• Dirty read—Reading data that has been changed by another user, even though thatchange hasn’t been committed and might still be rolled back

• Non-repeatable read—Reading data that might be updated by another user before youread it again

• Phantom read—You read a set of data and then another user adds data to that set Whenyou read it again, you see the phantoms, the new records that have been added

DTSIsoLevel_Chaos

Chaos is not implemented in Transact-SQL

• Value: 16

• ANSI SQL-92 Isolation Level 0

• At this isolation level, two different users may update the same data at the same time

(That’s why it’s called chaos.) This level is only appropriate for a database in single-usermode

DTSIsoLevel_ReadUncommitted

• Value: 256

• ANSI SQL-92 Isolation Level 1

• Allows dirty reads, non-repeatable reads, and phantom reads

• This isolation level is useful for running complex decision support queries on data that isbeing updated No locks are taken or honored when reading data at this isolation level

DTSIsoLevel_ReadCommitted

• Value: 4096

• ANSI SQL-92 Isolation Level 2

• This is the default transaction isolation level in SQL Server and in DTS

• Does not allow dirty reads.

• Allows non-repeatable reads and phantom reads

• You are not allowed to read data modifications that have not been committed

DTSIsoLevel_RepeatableRead

Equivalent to Transact-SQL SET TRANSACTION ISOLATION LEVEL REPEATABLE READ

• Value: 65536

• ANSI SQL-92 Isolation Level 3

• Does not allow dirty reads or non-repeatable reads

• Allows phantom reads

• If you start a transaction in this isolation level and you read some data, you are teed that the data you have read will not be changed until your transaction ends

guaran-DTSIsoLevel_Serializable

Equivalent to Transact-SQL SET TRANSACTION ISOLATION LEVEL SERIALIZABLE

• Value: 1048576

• ANSI SQL-92 Isolation Level 4

• Equivalent to constant DTSIsoLevel_Isolated

• Does not allow dirty reads, non-repeatable reads, and phantom reads

• Provides total isolation for the data being used in the transaction You cannot read anydata that other users have locked No other users can change or update the data No otherusers are allowed to put any locks on the data, including shared locks Other users arenot allowed to add new records to record sets that have been viewed by any queries inyour transaction

Transaction Settings for the Steps

You configure a step’s participation in a transaction on the Options tab of the WorkflowProperties dialog (see Figure 24.6)

Join Transaction If Present

If you select this option, the step will participate in the current transaction if the package’sUseTransactionproperty is set to TRUE If the UseTransactionproperty is set to FALSE, therequest to participate in a connection will be ignored

There can be only one transaction active in a package at a time If one is currently active, thisstep will join it If a transaction is not active, this step will start a new one

In code, this option is implemented as the JoinTransactionIfPresentproperty of the Stepobject This is a boolean property with a default value of FALSE

F IGURE 24.6

You have three choices regarding a step’s involvement with a transaction.

Commit Transaction on Successful Completion of This Step

If this option is selected, the current transaction will be committed if this step completes cessfully All the data modifications made in this step, and in previous steps included in thetransaction, will be committed

suc-You can select this option without choosing to have the step join the transaction

After this step is completed and the transaction is committed, the next step that is set to join atransaction will start a new transaction

Rollback Transaction on Failure

If this option is selected, the current transaction will be rolled back if this step fails All thedata modifications made in this step, and in previous steps included in the transaction, will berolled back

You can select this option without choosing to have the step join the transaction

If this step fails and the transaction is rolled back, the next step that is set to join a transactionwill start a new transaction.

If this option is not selected, this step is included in a transaction, and the step fails, the action will continue without being committed or rolled back

trans-This option is the RollbackFailureproperty of the Stepobject

Participation in Transactions by Connections and Tasks

Data connections and tasks have to be able to participate in distributed transactions, or elsethey are not allowed to join DTS transactions

You can commit or roll back a DTS transaction based on the success or failure of any task Thetask does not have to participate in the transaction for it to trigger the commit or the rollback.Here are the tasks that can participate in DTS transactions:

• Copy SQL Server Objects

• File Transfer Protocol

NOTE

Connections using the following providers can participate in DTS transactions:

• The Microsoft OLE DB provider for SQL Server

If data is modified in a non-supported data connection as a part of a DTS transaction, a time error will be generated and the task and the package will fail You can use a non-sup-ported data connection as the source for a transformation without generating an error, however

run-Some examples of data sources that do not support DTS transactions include

• Microsoft Excel 2000 worksheets

• Microsoft Access 2000 tables

• Text files

A Transaction with Steps Executed in Sequence

The easiest way to structure a transaction in a DTS package is to set all the steps to execute insequence, as shown in Figure 24.7 The DTS package with this transaction is on the CD in afile called SerialTransaction.dts

Here’s what you have to do to set up this simple transaction:

1 Select Use transactions and Commit on successful package completion on the Advancedtab of the DTS Package Properties dialog These are the default choices

2 Select Join transaction if present and Rollback on failure on the Options tab of theWorkflow Properties dialog for all of the steps Do not select Commit transaction on suc-cessful completion of this step for any of the steps

3 Set precedence constraints so that the tasks are executed sequentially Use On Successprecedence

As soon as an error occurs in this package, all the data modifications that have been previouslymade will be rolled back The On Success precedence constraints will prevent any more stepsfrom being executed

You can set all the properties for this transaction programmatically by executing the script inListing 24.2 as an ActiveX Script task This script sets the precedence constraints to the appro-priate type, but it does not create any new constraints

L ISTING 24.2 A VBScript to Set the Properties for a Simple Transaction

Option Explicit Function Main Dim pkg, stp, tsk, prc Set pkg = DTSGlobalVariables.Parent pkg.UseTransaction = True

pkg.AutoCommitTransaction = True For Each stp in pkg.Steps Set tsk = pkg.Tasks(stp.TaskName) Select Case tsk.CustomTaskID Case “DTSDataPumpTask”, “DTSExecuteSQLTask”, _

“DTSDataDrivenQueryTask”, “DTSBulkInsertTask”, _

“DTSExecutePackageTask”, “DTSMessageQueueTask”

stp.JoinTransactionIfPresent = True stp.RollbackFailure = True

stp.CommitSuccess = False

For Each prc in stp.PrecedenceConstraints prc.PrecedenceBasis = DTSStepPrecedenceBasis_ExecResult prc.Value = DTSStepExecResult_Success

Next

End Select

Next

Main = DTSTaskExecResult_Success End Function

If you have some tasks that you want to run after the transaction, you can use the VBScriptcode in Listing 24.3 in a Workflow ActiveX script The code checks if any of the steps in thetransaction have failed or if all the steps in the transaction have been executed The task is exe-cuted when one of these conditions is met

L ISTING 24.3 A Workflow ActiveX Script That Can Be Used to Watch for the Completion

of a Transaction

Option Explicit Function Main()

Dim pkg, stp Set pkg = DTSGlobalVariables.Parent

Main = DTSStepScriptResult_ExecuteTask

For Each stp In pkg.Steps

If stp.JoinTransactionIfPresent = True Then

If stp.ExecutionResult = DTSStepExecResult_Failure And _ stp.ExecutionStatus = DTSStepExecStat_Completed Then Main = DTSStepScriptResult_ExecuteTask

Exit For End If

If stp.ExecutionStatus = DTSStepExecStat_Waiting Then Main = DTSStepScriptResult_RetryLater

Exit For End If End If Next

‘Initialize or increment variable used for timing out

If IsEmpty(DTSGlobalVariables(“lCounter”).Value) Then DTSGlobalVariables(“lCounter”).Value = 1

Else DTSGlobalVariables(“lCounter”).Value = _ DTSGlobalVariables(“lCounter”).Value + 1 End If

‘Time out if we’ve run the Workflow script too many times

IF DTSGlobalVariables(“lCounter”).Value > 1000 Then Main = DTSStepScriptResult_DontExecuteTask End If

End Function

A Transaction with Steps Executed in Parallel

Sometimes, for better performance, you may want to execute several steps in a transaction inparallel Figure 24.8 shows a package with steps executing in parallel The DTS package withthis transaction is on the CD in a file called ParallelTransaction.dts

If you use parallel execution in a transaction, you have to be aware of these issues:

• If you have two connections to the same instance of SQL Server 2000, one of them isbeing used, and a task using the second is set to join the transaction, the package willfail One way to avoid this error is to use a single connection for each SQL Server 2000that is accessed by your package Only one task can use a connection at a time, so thetasks will actually be executed serially even if they’re set to execute in parallel If thereare two connections to separate instances of SQL Server 2000, they can be successfullyexecuted in parallel

• It can be hard to prevent new transactions from starting after a transaction failure Forexample, in Figure 24.8 a transaction could start and be joined by the first tasks on each

of the three branches of the programmatic flow If From Pubs fails and the other twosteps are completed successfully, the data modifications made in all three steps will berolled back The problem is that a new transaction will be started with the ReportEmployees tasks To avoid this problem, you can select Fail package on first error on theLogging tab of the Package Properties dialog

L ISTING 24.3 Continued

F IGURE 24.8

You can use a transaction with steps executed in parallel.

There may be times when you want to execute tasks in parallel and you don’t want to fail thepackage on the first error For example, you might want to send a message or do some otherprocessing after the transaction has failed

You can use an ActiveX Script task to commit or roll back a transaction after explicitly ing on the completion status of all the steps in the transaction To implement this strategy, dothe following:

check-1 Do not select Fail package on first error on the Logging tab of the Package Properties

dialog

2 Do not select Rollback on failure or Commit transaction on successful completion of this

step for any of the steps in the transaction

3 Create a new ActiveX Script task to be executed after all the tasks of the transaction havebeen completed Figure 24.9 shows the package from Figure 24.8 with the ActiveXScript task added

4 Do not select Join transaction if present for the ActiveX Script task Select both Rollback

transaction on failure and Commit transaction on successful completion of this step

F IGURE 24.9

You can use an ActiveX Script task to commit or roll back a transaction executed in parallel.

5 Set all the precedence constraints to execute on completion

6 The script for the ActiveX Script task is shown in Listing 24.4 This script assumes there

is only one transaction in the package If there were more than one, you would have toreference each step in the transaction explicitly, rather than looping through all the stepsand examining all that were participating in a transaction

L ISTING 24.4 Code for an ActiveX Script Task to Commit or Roll Back a Transaction

Option Explicit Function Main() Dim pkg, stp Set pkg = DTSGlobalVariables.Parent Main = DTSTaskExecResult_Success For Each stp In pkg.Steps

If stp.JoinTransactionIfPresent = True Then

If stp.ExecutionResult = DTSStepExecResult_Failure Then Main = DTSTaskExecResult_Failure

Exit For End If

Transactions Involving More Than One DTS Package

You can create a DTS transaction that includes steps from more than one package by doingthese two things:

1 Call another DTS package with the Execute Package task

2 Select Join transaction if present for the step associated with that Execute Package task

The transaction in the child package is called an inherited transaction The events in the child

package can cause the transaction to be committed or rolled back, but not in the same way as anormal DTS transaction:

• Select Join transaction if present for all steps in the child package that you want toinclude in the transaction Data modifications will be committed immediately for stepsthat don’t have this option selected

• Rollback on failure and Commit transaction on successful completion of this step have

no effect when selected for a step in an inherited transaction

• If Rollback on failure is selected for the Execute Package task in the parent package andthe child package terminates with failure, the whole transaction is rolled back If youwant to include all the steps of the child package in the transaction, select Join transac-tion if present for all the steps and select the option to fail the package on first error

• If Commit transaction on successful completion of this step is selected for the ExecutePackage task in the parent package, and the child package terminates successfully, thewhole transaction is committed

• You can call another package from a child package that is participating in a transaction

so that steps in additional packages are included in the transaction

If you execute a package using the OLE Automation stored procedures in an Execute SQLtask, using DTSRun in an Execute Process task, or using COM in an ActiveX Script task, youcannot include the called package in the DTS transaction.

Workflow ActiveX Scripts

A Workflow ActiveX Script is run at the beginning of a step’s execution, before the task ciated with that step is executed The main purpose of the Workflow ActiveX Script is to deter-mine whether or not the task should be executed

asso-You choose a Workflow ActiveX Script by selecting the Use ActiveX script box on the bottom

of the Options tab of the Workflow Properties dialog (see Figure 24.10) You write the script byclicking the Properties button beside that check box The Properties button opens the WorkflowActiveX Script Properties dialog, shown in Figure 24.11

F IGURE 24.10

You choose a Workflow ActiveX Script on the Options tab of the Workflow Properties dialog.

The Workflow ActiveX Script Properties dialog is identical to the ActiveX Script TaskProperties dialog, except that it doesn’t have a box to enter a description for the task

Script Result Constants

There are three script result constants that are used with workflow ActiveX scripts, each ofwhich is described in this section These values are assigned as the return value of the entryfunction:

Main = DTSStepScriptResult_ExecuteTask

• The task associated with this step is not executed when the workflow script is completed.

The execution method of the task is called again later in the execution of the package

When the step is retried, the workflow script is again executed before the task

Using the Script Results for Looping

You can use the Retry Later Script Result to create a loop in the flow of the DTS Package Youcould use this loop to check for the existence of a file, or to open a recordset to check on thestate of data in a table You could also use a loop to wait for the completion of other tasks, asshown in Listing 24.3:

1 You may want to use a Global Variable to serve as a counter Increment the GlobalVariable each time the workflow script is run

2 If the condition has not been met, the workflow script returns the Retry Later ScriptResult

3 If everything is ready for the task to be run, the script returns the Execute Task ScriptResult

4 You may want to terminate the loop with a return value of Don’t Execute Task if theGlobal Variable reaches a certain value without the condition being met

Step Properties for the Workflow Script

The workflow scripts are implemented with the following properties of the Stepobject:

• FunctionName—Specifies the entry function of the script The default function name is

Main.

• ActiveXScript—The text of the workflow ActiveX Script

• ScriptLanguage—The scripting language used for the script

• AddGlobalVariables—Determines whether or not global variables can be used in thisworkflow script The default value is True

The following Stepobject properties have already been discussed:

• AddGlobalVariables, ActiveXScript, ScriptLanguage, and FunctionNamewere ered in the section “Workflow ActiveX Scripts.”

cov-• CloseConnection, DisableStep, ExecuteInMainThread, and RelativePrioritywerecovered in the section “Threads and Priority of Execution.”

• IsPackageDSORowsetwas covered in Chapter 23, “The DTS Package and Its Properties”

• JoinTransactionIfPresent, CommitSuccess, and RollbackFailurewere covered in thesection “Transactions in DTS Packages.”

• NameandTaskNamewere covered in the section “Steps.”

• ExecutionResultandExecutionStatuswere covered in the section “PrecedenceConstraints.”

The remaining step properties are discussed in this section

StartTime , FinishTime , and ExecutionTime

StartTimeandFinishTimeare read-only properties with a date data type ExecutionTimeis aread-only integer, giving the duration of the task in seconds

Description

TheDescriptionproperty on the Stepobject cannot be set in the DTS Designer, but it can beset in code The description appears on the Options tab of the Workflow Properties dialog

Parent

TheParentproperty of the Stepobject returns the Stepscollection

Creating a Step Using Visual Basic

of the step can be set with this procedure

Chapter 24 on the book’s CD, with files CreateStep.vbp, CreateStep.frm, CreateStep.frx, andCreateStep.bas This project creates a new package and associates the step with a DynamicProperties task

L ISTING 24.5 Code That Creates a Step and Sets All Its Properties

Public Function fctCreateStep( _ pkg As DTS.Package2, _ tsk As DTS.Task, _ Optional sBaseName As String = “NewStep”, _ Optional sActiveXScript As String = “”, _ Optional bAddGlobalVariables As Boolean = True, _ Optional bCloseConnection As Boolean = False, _ Optional bCommitSuccess As Boolean = False, _ Optional sDescription As String = “New Step”, _ Optional bDisableStep As Boolean = False, _ Optional bExecuteInMainThread As Boolean = False, _

Optional sFunctionName As String = “”, _ Optional bIsPackageDSORowset As Boolean = False, _ Optional bJoinTransactionIfPresent As Boolean = False, _ Optional dtsRelativePriority As DTSStepRelativePriority = 3, _ Optional bRollbackFailure As Boolean = False, _

Optional sScriptLanguage As String = “VBScript”, _ Optional bFailPackageOnError As Boolean = False)

On Error GoTo ProcErr Dim stp As DTS.Step2

‘Check to see if the selected Base name is unique sBaseName = fctFindUniqueBaseName(pkg, sBaseName)

‘Create step for task Set stp = pkg.Steps.New With stp

.Name = “stp” & sBaseName Description = sBaseName TaskName = tsk.Name ActiveXScript = sActiveXScript AddGlobalVariables = bAddGlobalVariables CloseConnection = bCloseConnection CommitSuccess = bCommitSuccess DisableStep = bDisableStep ExecuteInMainThread = bExecuteInMainThread FunctionName = sFunctionName

.IsPackageDSORowset = bIsPackageDSORowset JoinTransactionIfPresent = bJoinTransactionIfPresent RelativePriority = dtsRelativePriority

.RollbackFailure = bRollbackFailure ScriptLanguage = sScriptLanguage FailPackageOnError = bFailPackageOnError End With

pkg.Steps.Add stp fctCreateStep = stp.Name Set stp = Nothing ProcExit:

Exit Function

L ISTING 24.5 Continued