Hands-On Microsoft SQL Server 2008 Integration Services part 28 pdf

You saw that this utility generates a command line that can be copied to a file and used with the CommandFile option in DTExec, or pasted directly to the Command Prompt window when runni

2 4 8 H a n d s - O n M i c r o s o f t S Q L S e r v e r 2 0 0 8 I n t e g r a t i o n S e r v i c e s

7 Move on to the Reporting page This page has Console events and Console logging boxes, which can be configured with the numerous given options You can use the Reporting page to set the options for the package by selecting the events and the information about the running package to report to the console Console events specify the events and types of messages to report, whereas Console logging specifies the information that you want written to the log when the selected event occurs

8 In the Logging page, you can tell the Package Execution utility which log providers to use at run time by specifying the log providers and the connection string for connecting to a log The options for log providers are:

SSIS Log Provider For Windows Event Log c

SSIS Log Provider For XML Files c

SSIS Log Provider For Text Files c

SSIS Log Provider For SQL Server c

SSIS Log Provider For SQL Server Profiler c

Click in the field under the Log Provider column to choose from one of the available log providers Then you can specify the connection string in the Connection String field

9 The Set Values page is probably the one you will be using most often when you want to update property values of packages, executables, connections, variables, and log providers at run time This useful feature makes running and testing packages easier You can modify the property values by typing in the modified value and the path to the property The path can be specified using a backslash (\) before the container, the period (.) before the property, and including a collection member within the brackets

10. In the Verification page, you can set the attributes that must be met before allowing a package to run For example, you can set the attributes to run only signed packages Other options are to verify the package for the build number, package ID number, or version ID number

11. Finally, the Command Line page allows you to review and optionally edit the command (see Figure 6-4) that has been generated by the options you selected in the various pages

Select the Edit The Command Line Manually radio button to make changes in the command line If you accidentally make mistakes here and want to undo your changes, you can do so by selecting the Restore The Original Options radio button Please note that when you select options in the other pages of the Package Execution utility, the command line gets updated; however, this is not the case if you make changes to the command line—i.e., these changes don’t get reflected in the options

of relevant pages Also, these changes are not validated for the same reason

12. Now click Execute to run the package, and you will see the progress of the running package in Package Execution Progress window

In this exercise, you learned about the Package Execution utility that you can run from

within the SQL Server Management Studio or from the command prompt You also

learned various options and how to set values for the properties at run time You saw

that this utility generates a command line that can be copied to a file and used with the

CommandFile option in DTExec, or pasted directly to the Command Prompt window

when running a package using DTExec The DTExecUI user interface provides the

facility to select several options that you find difficult to write syntactically correct and

create command lines that can be passed on to the DTExec command prompt tool

The DTExecUI and DTExec utilities have a couple of differences worth noting

First, DTExecUI is a GUI-based tool that helps you create command lines easily, but

lacks the ability to automate the execution of packages On the other hand, DTExec is

a handy and powerful command prompt tool that is a bit difficult to program, especially

Figure 6-4 Command generated by the Package Execution utility

2 5 0 H a n d s - O n M i c r o s o f t S Q L S e r v e r 2 0 0 8 I n t e g r a t i o n S e r v i c e s

if you are not deploying packages very often, but once the command line is created, it can automate running of packages with the help of the SQL Server Agent service So you will use DTExecUI when you want to run a package one off or manually and also when you want to write a complex command line for running a package automatically using DTExec and the SQL Server Agent The second difference that can cause much grief on a 64-bit server is that the DTExecUI is a 32-bit only utility, whereas DTExec comes in both 64-bit and 32-bit flavors You might get conflicting results if you’re running your packages using different versions of the utilities

DTExec Utility

SSIS provides the command prompt utility DTExec to configure and execute SSIS packages; it replaces the DTSRun utility provided in Data Transformation Services

of SQL Server 2000 DTExec is a powerful utility that lets you access packages saved

in the SQL Server, on the file system or in the SSIS Package Store It allows you to configure connections, checkpoints, variables, logging, and reporting for the package you want to run

With DTExec, you can run packages on the local server unless you use the SQL Server Agent or custom-build application to run DTExec commands from a remote computer The SQL Server Agent allows you to automate the execution of your SSIS packages using the DTExec utility You can select the Operating System (CmdExec) Agent subsystem, while adding a step to the job and use the DTExec command directly

in the command area As mentioned earlier, you can use the DTExecUI utility to create

a command line with all the options you require and use it directly with DTExec

By default, the DTExec utility is located in the <drive>:\Program Files\Microsoft SQL Server\100\DTS\Binn folder DTExec comes in both 32-bit and 64-bit versions,

so on a 64-bit server if you install 32-bit utilities, a 32-bit version of DTExec may also exist in the <drive>:\Program Files(x86)\Microsoft SQL Server\100\DTS\Binn folder

In such a case, the environment variable PATH contains the 32-bit path before the 64-bit path, making 32-bit DTExec run by default from a command prompt However, the SQL Server Agent doesn’t use this path statement; rather, it uses a registry setting

to identify the DTExec location and will run the 64-bit version of DTExec Make sure you test your packages against the correct version of DTExec before creating SQL Server Agent jobs with this utility You can, though, avoid this situation by modifying the order of paths in the PATH environment variable

The basic syntax for the command is shown next; the parameters are specified as option-value pairs The options are specified beginning either with a minus sign (–)

or a slash (/) and are not case-sensitive, barring passwords

DTExec /option [value] [/option [value]]

When the DTExec utility is executed, it passes through different phases, depending

upon the parameters specified in the command line, and in the end, it can return an

exit code as well Let’s study these phases and the parameters that decide the behavior

of DTExec

Command Sourcing Phase

This is the initialization phase in which the DTExec utility reads the parameters—i.e.,

the list of options and arguments specified If you want to see only the help on the

commands using a /? or /H option, for example, DTExec will run for this phase only

and all the subsequent phases will be skipped If you specify an argument along with

help options, DTExec will display help for that argument

DTExec /H DTS

/Rem is another option that is parsed in this phase Using /Rem, you can include

remarks on the command prompt or more appropriately in command files These

remarks are discarded during the command sourcing phase

In the command sourcing phase, if the /CommandFile (/com) option is used,

DTExec opens up the file specified with the /CommandFile option and reads all the

options from the file until EOF is reached You can either use the complete option

name or use the short form of the option shown in parentheses with each option The

file specified with /CommandFile option is a text file that contains additional DTExec

command options

Here’s an example: Open a blank text file in Notepad and type /DTS “\msdb\

mailing opportunities” at the beginning of the file Save this as C:\dtscmds.txt Then

go to a command prompt and type this command:

DTExec /com "C:\dtscmds.txt"

You will see that the Mailing Opportunities package has executed successfully

DTExec doesn’t put any restriction on how you write command options—i.e., on the

same line or on different lines in the command file

Package Load Phase

After passing through the first phase, DTExec goes on to load the package from the

storage specified by the /SQL, /FILE, or /DTS option

To load a package stored in the MSDB database of the SQL Server, type the

following command using the /SQL (/SQ) option:

DTExec /SQL "Mailing Opportunities"

2 5 2 H a n d s - O n M i c r o s o f t S Q L S e r v e r 2 0 0 8 I n t e g r a t i o n S e r v i c e s

The /File (/F) option loads a package saved in the file system The file path can be specified in either Universal Naming Convention (UNC) format or as a local path

DTExec /F "C:\SSIS\Packages\Mailing Opportunities.dtsx"

Don’t forget to include the file extension when using the /File option

Finally, to load a package saved in the SSIS Package Store, you will use the /DTS (/DT) option

DTExec /DTS "\MSDB\Mailing Opportunities"

You can use only one of these three options, as they are mutually exclusive Using these options together will cause the package to fail You can also use the following supportive options:

/User

c can be used with the /SQL option when the package is protected by SQL Server Authentication If you omit the /User option, Windows Authentication

is used It is recommended that you use Windows Authentication whenever possible

/Password

c (/P) option is used with the /User option If you omit the /Password option, a blank password will be used

/Decrypt

c (/De) is used to load the package that has been encrypted with a password encryption; this option is used to set the decryption password

/Server

c (/Ser) is used to specify the server from which to retrieve the package when the /SQL or /DTS option is specified If you omit the /Server option when using /SQL or /DTS, the local server default instance will be used to retrieve the package

The following command will run the package stored at the RemoteServer:

DTExec /SQL "Mailing Opportunities" /Ser RemoteServer

Configuration Phase

After the package is loaded, DTExec goes on to process rest of the options that set flags, variables, or properties; verify the package versioning; and specify the logging and reporting These options are as follows:

The

c /CheckPointing (/CheckP) option is used when you want to include

checkpoints during package execution The valid values are on or off The default

is on when the option is specified without a value The checkpoints enable a failed

package to rerun from the point of failure For example, if you are loading multiple tables in a data warehouse and your package fails halfway through, then using

checkpoints will enable this failed package to rerun from the failed task position

and save the processing for already imported tables To achieve this, the DTExec

utility records the current position in a file called CheckFile

The

c /CheckFile (/CheckF) option is used by DTExec to specify the checkpoint

file so that a failed package can be restarted from the point of failure The filename

specified with the /CheckFile option is used to update the value of the

CheckPointFileName property of the package

The

c /ConfigFile (/Conf) option is used at run time when you want to set

different configuration values than what were specified at design time The

run-time values can be stored in an XML file and loaded before executing the package

using the /ConfigFile option

The

c /Connection (/Conn) option can be used to change the connection

managers at run time You have to specify the connection manager name or ID

used in the package along with the connection string to which you want to change

The

c /ConsoleLog (/Cons) option can be used to display the specified log

entries to the console (see Figure 6-5) You can specify the columns you want

to see by using the displayoptions parameter, for which the values include N for

Name, C for Computer, O for Operator, S for Source Name, G for Source GUID,

X for Execution GUID, M for Message, and T for time If you do not specify

Figure 6-5 Use of the /ConsoleLog option displays the execution log on the screen

2 5 4 H a n d s - O n M i c r o s o f t S Q L S e r v e r 2 0 0 8 I n t e g r a t i o n S e r v i c e s

the option parameters, no column is displayed, and if you specify the option parameters, all the values for the selected columns will be displayed that you may want to limit For limiting the display of log entries on the console, you use

extended options that are called list_options The list_options allow you to log only

the sources that are specified with parameter (I) for inclusion or only the sources that are not explicitly excluded with parameter (E) using a list

The

c /Logger (/L)option lets you choose one or more log providers to be used at run time by specifying an ID and a connection string to establish a connection to the destination store for log entries The options of the log providers are as follows: SSIS Log Provider for Text Files

c SSIS Log Provider for SQL Profiler c

SSIS Log Provider for SQL Server c

SSIS Log Provider for Windows Event Log c

SSIS Log Provider for XML Files c

The

c /Reporting (/Rep) option works as follows: By default, DTExec will report errors, warnings, and progress of the events when the package is executed However, you can specify the types of messages to report using the /Reporting option You can choose from the options listed in the following table:

Option Symbol and Name Description

N – No Reporting A mutually exclusive option with all other options and should be

specified alone

E – Errors Errors are reported

W – Warnings Warnings are reported

I – Information Informational messages are reported

C – Custom Events Custom events are reported

D – Data Flow Task Events Events related to Data Flow task are reported

P – Progress Progress of the execution of package is reported

V – Verbose Reporting A mutually exclusive option with all other options and should be

specified alone

The

c /Restart (/Res) option specifies a new value for the CheckpointUsage property on the package The possible parameters are shown next:

Deny Sets CheckpointUsage property to DTSCU_NEVER.

Force (Default) Sets CheckpointUsage property to DTSCU_ALWAYS

ifPossible Sets CheckpointUsage property to DTSCU_IFEXISTS

The

c /Set option is used to modify the configuration values of variables, properties,

containers, log providers, Foreach enumerators, or connections at run time Both

the property path and the value are to be specified according to specific syntax rules

The syntax uses a backslash (\) as a container separator, a period (.) to delimit

properties, square brackets ([]) to specify collection members, and a semicolon (;)

to separate the property path and its value

The

c /MaxConcurrent (/M) option limits the number of executables running

in parallel on your machine You can specify a positive non-zero integer value or

–1 A value of –1 means the maximum concurrently running executables (total

number of processors on the computer plus two) are allowed

The

c /Sum (/Su) option is an optional switch showing a counter that contains the

number of rows that will be passed on to the next component

The

c /VerifyBuild (/VerifyB) option works like this: During the

development stage when you will be making lots of changes to your package, you

may want to make sure that the package is executed with a stable version of the

build while you can carry on the development work without jeopardizing the

normal running of the package You can specify the build numbers you want to

run and verify your package against those numbers In case of a mismatch, the

package will fail to run The /VerifyBuild option has three arguments, major,

minor, and build, which can be specified in three forms: major or major; minor or

major; minor; build.

The

c /VerifyPackageID (/VerifyP) option is similar to the /VerifyBuild

option; you can specify the package ID with this option to let it verify against the

package GUID

The

c /VerifySigned (/VerifyS) option is a security option to check that the

package is signed If the package is not signed or the signature is not valid, the

execution of the package will fail

The

c /VerifyVersionID (VerifyV) option verifies the version ID of a

package to be executed against the version ID you specify with this option

2 5 6 H a n d s - O n M i c r o s o f t S Q L S e r v e r 2 0 0 8 I n t e g r a t i o n S e r v i c e s

Validation and Execution Phase

In this phase, the package is validated and then finally run However, you can also choose only to validate the package without running it

The

c /Validate (/Va) option lets you validate only without running the package The

c /WarnAsError (/W) option, specified along with the /Validate option, can go one step further by letting the package treat the warnings as errors and will fail if a warning occurs during validation phase

The

c /Dump errorcode option can create the debug dump files when one or more specified events—error, warning, or information—occur while the package

is running You can specify the events by specifying the error codes, separated by a semicolon (;), that you want to use to invoke the system to create the debug dump files Integration Services creates a binary debug dump file with an extension .mdmp and a text-formatted debug dump file with an extension tmp; by default,

it stores these files in the <drive>:\Program Files\Microsoft SQL Server\100\ Shared\ErrorDumps folder

The

c /DumpOnError option works like this: If your package has been failing and you want to debug the failures, you can create debug dump files You can use this option if you want to create the debug dump files, mdmp and tmp, only when an error occurs while the package is running

Finally, the DTExec utility can return an exit code depending upon the state of the package loading and execution The exit codes are values from 0 through 6 that can be interpreted as follows:

Exit Code Description

0 The package was executed successfully

1 The package failed to execute

3 The user canceled execution of the package

4 The package could not be found at the specified location

5 The package could not be loaded

6 The command line could not be interpreted due to syntax or semantic errors

SQL Server Agent

The SQL Server Agent can automate the running of SSIS packages SQL Server Agent jobs can have one or more steps, each running a different package and a bit of work that the agent can do for you The jobs can also be scheduled to run at specific times using extensive SQL Server Agent scheduling capabilities

One thing worth mentioning here is that Integration Services is a server component

and not a redistributable component, as was its predecessor DTS 2000 This means

that you can’t install SSIS (as redistributable software) on a client machine and execute

SSIS packages as you could have done in the past with DTS 2000 Also, SSIS packages will always run locally even when they load a package from a remote server; hence,

you can’t run a package (outside the design environment BIDS) on a computer where

Integration Services is not installed However, there will be instances where users or

applications need to run packages while Integration Services is not installed on the

local computer In those instances the SQL Server Agent (or any other scheduling

agent) comes in handy, as it allows you to schedule and automate the execution of SSIS

packages not only on local servers but also on the remote servers You can start the

jobs on remote servers using system stored procedure sp_start_job or SMO The only

drawback of this method is that after you’ve created jobs on the SQL Server Agent,

you can’t configure the jobs at run time; that is, you can’t provide parameters or other

options at run time that you can do with DTExec utility

Hands-On: Automating Running an

SSIS Package with SQL Server Agent

In this exercise, you will learn how to use the SQL Server Agent service to create and

automate jobs for running SSIS packages

Method

We will use the SQL Server Agent in SQL Server Management Studio to

Create a new job and add a package (step) to the job

c

Create a schedule to execute the package

c

View the job history after running

c

For this exercise, we will again use the Mailing Opportunities package as a test

package

Exercise (Create a New Job and add a Package to the Job)

You will be creating an SQL Server Agent job and then will add an SSIS package to

a step in the job here

1 Start SQL Server Management Studio and connect to the database engine Once

connected, verify that the SQL Server Agent service is started If not, start the

SQL Server Agent service