Microsoft SQL Server 2000 Data Transformation Services- P8

• When to Use the Transfer Databases and• Creating the Tasks and Setting Their • The Transfer Master Stored Procedures... The five tasks described in this chapter are the ones used by th

The File Transfer Protocol (FTP) task allows you to retrieve files from network or Internetsites.

The lack of FTP integration was a weakness of DTS in SQL Server 7.0 It didn’t prevent DTS from being used, but it reduced the convenience of the DTS development envi- ronment You had to coordinate your FTP processes with the execution of your DTS packages, ensuring that the files were moved to the proper directories before the DTS package execution began.

Now, with SQL Server 2000, FTP has become another step in the DTS process.

NOTE

When to Use the File Transfer Protocol (FTP) Task

The FTP task can be used to move files or directories in your local network It can also beused to get files from remote Internet sites It cannot be used to send files to Internet sites

It would be more convenient if the FTP task could be used to send files to Internet sites as well as getting files There are tools that make an FTP site appear like a Windows directory One of these tool, WebDrive ( www.webdrive.com ) is included on the book’s CD With WebDrive or a similar tool you can use the FTP task to send files

to remote Internet sites.

TIP

When you’re using the FTP task, it’s important to recognize the possibilities for integrationwith the Dynamic Properties task and with the many ways you can use global variables Youcan use the Dynamic Properties task to change the source, the destination, and the filesinvolved with the FTP You can set these values with global variables that have themselvesbeen set with DTSRun, by another DTS package, by an Execute SQL task, or with an ActiveXScript task

There are times when an ActiveX Script task is more convenient than an FTP task for movingfiles around your local network See the discussion of using the FileSystemObject for file anddirectory manipulation in Chapter 16, “Writing Scripts for an ActiveX Script Task.”

Creating the Task and Setting Its Properties

You can create the FTP task with the Package Designer or with code The last section of thischapter has an example of creating the task with code

The FTP Site tab of the File Transfer Protocol Task Properties dialog is shown in Figure 14.1

This is the place where you enter information about the site from which you want to FTP files

The FTP task allows you to move files from a local network or from an Internet site.

The primary choice you make on the FTP Site tab is the type of source location You canchoose either an Internet site or a network directory This choice is implemented in code bysetting the SourceLocationproperty to one of the DTSFTPSourceLocationconstants:

• 0—DTSFTPSourceLocation_InternetSite(The default choice)

• 1—DTSFTPSourceLocation_DirectoryMost of the other properties you set on the FTP Site fill in the details about the source of theFTP transfer:

• SourceSite—Must be used for an Internet source The property is set to an FTP sitename, such as ftp.mcp.com

• SourceUserName—“Anonymous” is used as the default, which is the standard usernamefor making a read-only connection to an Internet FTP site

• SourcePassword—The email address of the user is often used when connecting as

F IGURE 14.2

Choose files to transfer on the FTP Transformation tab.

These selections are implemented by the SourceFileNameproperty This property is a colon-delimited string that contains the filenames, file paths, and file sizes in bytes The stringlooks like this:

semi-“‘FileOne.dat’;’ftp.mcp.com’;’1234’;’FileTwo.dat’;’ftp.mcp.com’;

➥’4312’;’FileThre.dat’;’ftp.mcp.com’;’314’;”

If you specify the site in the SourceSiteproperty, you do not need to include the second meter Also, when you are creating this task programmatically, you do not need to specify thesize of the file You do still need to include the spaces for these values, however

para-This string is equivalent to the preceding one if ftp.mcp.comis assigned to the SourceSite

property:

“‘FileOne.dat’;’’;’’;’FileTwo.dat’;’’;’’;’FileThre.dat’;’’;’’;”

The only other choice you have on the FTP Transformation tab is a check box for specifyingwhether or not files with the same name should be overwritten This choice is implementedwith the NonOverwritableproperty This property has a default value of TRUE, which meansthat files are not overwritten

Creating the Task in Visual Basic

I have created a Visual Basic procedure,fctCreateFTPTask, which creates a step, a task, and acustom task for an FTP task All the properties of the task can be set with this procedure Youcan find the code for it in the directory for Chapter 14 on the book’s CD as a Visual BasicProject, with files CreateFTPTask.vbp, CreateFTPTask.frm, and CreateFTPTask.bas

The code for fctCreateFTPTaskis shown in Listing 14.1 The procedure needs some utilityfunctions that are included with the code listings on the CD The project requires references tothe Microsoft DTSPackage Object Library and the Microsoft DTS Custom Tasks ObjectLibrary

L ISTING 14.1 The Visual Basic Code to Create an FTP Task Option Explicit

Public Function fctCreateFTPTask( _ pkg As DTS.Package2, _

Optional sBaseName As String = “FTPTask”, _ Optional sDestSite As String = “C:\Temp”, _ Optional lNumRetriesOnSource As Long = 0, _ Optional sSourceFileName As String = “”, _ Optional lSourceLocation As Long = 0, _ Optional sSourcePassword As String = “”, _ Optional sSourceSite As String = “”, _ Optional sSourceUserName As String = “anonymous”, _ Optional bNonOverwritable As Boolean = True) As String

On Error GoTo ProcErr Dim stp As DTS.Step2 Dim tsk As DTS.Task Dim cus As DTSCustTasks.DTSFTPTask

‘Check to see if the selected Base name is unique sBaseName = fctFindUniqueBaseName(pkg, sBaseName)

‘Create task and custom task Set tsk = pkg.Tasks.New(“DTSFTPTask”) Set cus = tsk.CustomTask

With cus Name = “tsk” & sBaseName Description = sBaseName NonOverwritable = bNonOverwritable

If sDestSite <> “” Then DestSite = sDestSite End If

If sSourceFileName <> “” Then SourceFilename = sSourceFileName End If

If sSourceSite <> “” Then SourceSite = sSourceSite End If

.SourceLocation = lSourceLocation

If SourceLocation = 0 Then

If sSourcePassword <> “” Then SourcePassword = sSourcePassword End If

If sSourceUserName <> “” Then SourceUsername = sSourceUserName End If

.NumRetriesOnSource = lNumRetriesOnSource End If

End With pkg.Tasks.Add tsk

L ISTING 14.1 Continued

‘Create step for task Set stp = pkg.Steps.New With stp

.Name = “stp” & sBaseName Description = sBaseName TaskName = tsk.Name End With

pkg.Steps.Add stp fctCreateFTPTask = stp.Name Set tsk = Nothing

Set cus = Nothing Set stp = Nothing ProcExit:

Exit Function ProcErr:

MsgBox Err.Number & “ - “ & Err.Description fctCreateFTPTask = “”

GoTo ProcExit End Function

• When to Use the Transfer Databases and

• Creating the Tasks and Setting Their

• The Transfer Master Stored Procedures

The five tasks described in this chapter are the ones used by the Copy Database Wizard tomove databases and associated meta data from one SQL Server to a separate SQL Server 2000.It’s important to be able to move meta data along with the transfer of databases SQL Serverstores most of the meta data needed for database manipulation inside each individual database,but there is a significant amount of meta data that is stored in the Master and Msdb systemdatabases.

Centralized meta data storage makes it possible for the meta data to be used by all the bases on a server But the centralized meta data becomes a problem when you move an indi-vidual database to a new server Unless you include all the needed meta data, the database willnot operate properly on its new server

data-Each of the four additional transfer tasks involves the movement of a particular kind of data:

• Logins, stored in master

• System stored procedures, stored in master

• Error messages, stored in master

• Jobs, stored in msdbThe most common difficulty I have seen in moving databases is getting all the logins movedproperly But all the meta data is important Stored procedures, scheduled jobs, and batchprocesses can all fail if the proper meta data is missing

When to Use the Transfer Databases and Other Transfer Tasks

The five transfer tasks are designed for two purposes:

• The specific purpose of upgrading a SQL Server 7.0 database to SQL Server 2000

• The more general purpose of moving a database and associated meta data between base servers

data-You can only use databases on SQL Server 7.0 or SQL Server 2000 as the source for thesetransfer tasks The destination must be SQL Server 2000 One or more databases can beincluded in a database transfer For each included database, you can choose to copy it or move it

You cannot resolve most naming conflicts in the process of using these tasks You need toresolve any conflicts before setting up these tasks, with the exception of conflicts in the nam-ing of database storage files

You can include or exclude individual items of meta data, such as particular stored procedures,logins, messages, and jobs You cannot exclude any of the objects included in the databasesbeing transferred If you just want to transfer some objects in a database, you should considerusing the Copy SQL Server Objects task If you want to modify the data as it is being moved,you should consider using one of the transformation tasks.

These five tasks are the ones to use when you want to move or copy one or more whole bases

data-Creating the Tasks and Setting Their Properties

The primary method of creating these tasks is to use the Copy Database Wizard, which isdescribed in Chapter 25 You can also create the tasks in the Package Designer, although this isnot as convenient as using the wizard

It is possible to create these tasks in code, but the object model for these tasks is not as welldocumented as the object model for the other DTS tasks Most of the properties for theseobjects are not displayed with Disconnected Edit The last section of this chapter shows how tocreate the basic tasks in code

The Source and the Destination for the Tasks

All five of the transfer tasks have similar properties for the source and destination Figure 15.1shows the Source tab for the Transfer Databases task

All five transfer tasks have the same tabs for entering source and destination server information.

The transfer tasks do not use DTS connections Instead, the connection information must beentered for both source and destination in the task’s properties dialog

The Transfer Database Task

The Databases tab of the Transfer Databases dialog, shown in Figure 15.2, is the place whereyou choose which databases to copy and which databases to move

This is one of the reasons why using the wizard is appealing You enter the source and destination information once, and it is used for all of the tasks.

NOTE

F IGURE 15.2

The Transfer Databases dialog shows you which databases you are allowed to copy or move.

If you move a database, that database won’t be available on the source server after the step is executed.

The database files on the source server are not removed, however Whether you copy

or move a database with the Copy Databases task, after the task is executed there will be a copy of all the database files on both the source and the destination You could use the FileSystemObject in an ActiveX task to delete the source database files after the databases have been moved

NOTE

You are not allowed to transfer a database if

• The destination has a database with the same name

• The database is involved in replication

• The source is a SQL Server in Windows 2000 and the destination is a SQL Server inWindows 98.

• The database is unavailable because it is marked inaccessible, loading, offline, ing, suspect, or in Emergency Mode

recover-Figure 15.3 shows the File Locations tab You can change the destination file or directory ifthere are problems with filename conflicts You can also move the files to a different directory

if there is inadequate space to copy the database to the default data location

You can modify the destination files for the databases while you are creating the task.

The Transfer Logins Task

You have two basic options on the Logins tab of the Transfer Logins dialog:

• Include all server logins detected at package runtime

• Include logins for selected databases

The other meta data transfer tasks have similar options

If you choose to include logins for selected databases only, the list of choices is enabled, asshown in Figure 15.4

When you’re transferring databases, it’s sometimes reasonable to include only the logins that are being used in those databases After all, if certain logins aren’t using those databases on the old server, why should those logins need to be transferred?

NOTE

F IGURE 15.4

You can choose whether to transfer all of the logins or only some of them.

The Transfer Jobs Task

Figure 15.5 shows the Jobs tab of the Transfer Msdb Jobs dialog after the choice has beenmade to include selected jobs

On the other hand, when a server has a group of user logins, it’s only a matter of time until more of those users are going to be given access to those particular data- bases If you transfer all the logins, you will have an easier time giving database access permissions in the future.

F IGURE 15.5

You can choose which jobs to include or exclude in the transfer.

Choosing to include only certain jobs probably makes sense, unless you are moving all thedatabases on a server If a job involves a database that is not being moved, that job will fail if it

is executed on a different server

If you want to view details of the jobs you are considering transferring, they are listed in theEnterprise Manager tree under the Management\SQL Server Agent node

The Transfer Master Stored Procedures Task

Figure 15.6 shows the Stored Procedures tab of the Transfer Master Stored Procedures dialogafter the choice has been made to select individual stored procedures

You choose which of the Master database’s stored procedures to transfer.

If you create a stored procedure in the Master database and it is named with an sp_ prefix, thatstored procedure can be executed from all the databases on a server as if it were local A storedprocedure created like this is called a system stored procedure

There are many system stored procedures that are included with SQL Server These proceduresshouldn’t cause any problems when a database is being transferred because they exist on allSQL Servers But if users have defined their own customized system stored procedures, therecan be problems System stored procedures can be referenced from within a database’s ownstored procedures An error will be generated if they don’t exist on the local server

In my opinion, it’s usually best to transfer all the stored procedures They won’t hurt anything, even if they’re never used.

The only reason for following another strategy is if there are naming conflicts between stored procedures on separate servers If you have different stored proce- dures that have the same names, you will have to carefully examine how you can sep- arate the two procedures and make them unique

TIP

The Transfer Error Messages Task

Figure 15.7 shows the Error Messages tab of the Transfer Error Messages dialog after thechoice has been made to select specific error messages

F IGURE 15.7

The error messages and numbers are displayed as you select the ones to transfer.

SQL Server stores all its error messages in a table called sysmessages in the master database.All the error numbers under 50000 are reserved for Microsoft’s use Error numbers startingwith 50000 are user-defined error messages

You can create user-defined error messages to send customized, application-specific error sages back to client applications Many developers create a separate set of error messages foreach of their database applications When used, these messages become an essential part of thedatabase application

mes-Moving user-defined error messages from one server to another can be a problem if the same error number is used for two separate messages Careful modifications will have to be made to the code to make sure the wrong error message isn’t called

I have seen database developers who have used the numbers in the 50000 to 51000 range for their user-defined error messages Using those numbers brings a high risk

of error number conflicts.

I have seen other developers who use a 4-digit prefix for their database applications, which is assigned as a unique value for their organization, and another 4-digit value for each particular error So altogether, the error numbers all have 8 digits This sys- tem reduces the chance of conflict in error numbers

TIP

Creating the Tasks in Visual Basic

I have created a Visual Basic procedure,fctCreateTransferDatabaseTask, which creates astep, a task, and a custom task for a Transfer Databases task Only the properties exposed inDisconnected Edit are set by this procedure

NOTE

I have also created functions that create the other four tasks discussed in this chapter You canfind the code for these procedures in the directory for Chapter 15 on the book’s CD as a VisualBasic Project, with files CreateTransferDatabaseTask.frm, CreateTransferDatabaseTask.bas,and CreateTransferDatabaseTask.vbp

The code for fctCreateTransferDatabaseTaskis shown in Listing 15.1 The procedure needssome utility functions that are included with the code listings on the CD The project requiresreferences to the Microsoft DTSPackage Object Library and the OMWCustomTasks 1.0 TypeLibrary

L ISTING 15.1 The Visual Basic Code to Create a Transfer Databases Task Public Function fctCreateTransferDatabaseTask( _

pkg As DTS.Package2, _ Optional sBaseName As String = “TransferDatabaseTask” _ ) As String

On Error GoTo ProcErr Dim stp As DTS.Step2 Dim tsk As DTS.Task Dim cus As OMWTransferDatabases

‘Check to see if the selected Base name is unique sBaseName = fctFindUniqueBaseName(pkg, sBaseName)

‘Create task and custom task Set tsk = pkg.Tasks.New(“OMWCustomTasks.OMWTransferDatabases”) Set cus = tsk.CustomTask

With cus Name = “tsk” & sBaseName Description = sBaseName End With

pkg.Tasks.Add tsk

‘Create step for task Set stp = pkg.Steps.New With stp

.Name = “stp” & sBaseName Description = sBaseName TaskName = tsk.Name End With

pkg.Steps.Add stp fctCreateTransferDatabaseTask = stp.Name Set tsk = Nothing

Set cus = Nothing Set stp = Nothing ProcExit:

Exit Function ProcErr:

MsgBox Err.Number & “ - “ & Err.Description fctCreateTransferDatabaseTask = “”

GoTo ProcExit End Function

Conclusion

The Transfer Databases task and the other transfer tasks are useful when you’re moving datafrom one server to another These tasks cannot be manipulated in code as easily as most of theother tasks because their properties and methods are not documented or displayed in

Disconnected Edit

The easiest way to use these tasks is through the Copy Database Wizard

L ISTING 15.1 Continued

IN THIS PART

IV

• Creating an ActiveX Script Task in

The ActiveX Script task allows you to include a block of scripting code anywhere in your DTSpackage.

Much of the information about how to use ActiveX scripts in DTS is presented in Chapter 7,

“Writing ActiveX Scripts for a Transform Data Task.” Chapter 27, “Handling Errors in aPackage and Its Transformations,” tells how to debug ActiveX scripts

Most of the code for this chapter has been put into one DTS package This package is stored in

a file called ScriptSample.dts You can find it on the book’s CD

When to Use an ActiveX Script Task

You can use an ActiveX Script task to accomplish a variety of programmatic goals:

• Read or modify the properties of DTS objects

• Set the values of global variables

• Read, write, create, delete, and move files and directories

• Open and manipulate ADO recordsets

• Manipulate a disconnected ADO recordset created as the output of an Execute SQL task

• Write customized log records to the DTS task log

• Execute other DTS packages

• Access COM objects

Most of these things can also be done with other DTS tasks, of course It’s usually better to usethe other DTS tasks when you can because each one assists in automating the developmentprocess Here are some of the alternatives to consider:

• Use the Dynamic Properties task to modify the properties of DTS objects or set the ues of global variables Use the ActiveX Script task to do this when you have to usesome programmatic logic to determine the value you are going to assign

val-• Use the FTP task to move files Use the ActiveX script task for other file and directorymanipulation

• Use the Transform Data task to retrieve data, rather than opening ADO recordsets in anActiveX script

• Use the Execute Package task to execute other DTS packages Consider executing a DTSpackage from an ActiveX script when you want to load it, modify its properties, and thenexecute it

Creating an ActiveX Script Task

You can create an ActiveX Script task in the DTS Designer or in code The last section of thischapter shows how to create the task in code

Figure 16.1 shows the ActiveX Script Task Properties dialog The interface is very similar tothe one used for creating ActiveX transformation scripts In fact, the object browser lists theresult constants for the Data Pump and the Data Driven Query next to the return constants forthe ActiveX Script, even though only the ActiveX Script return constants can be used here.

You can write your scripts in the ActiveX Script Task Properties dialog or load the scripts from a file.

The ActiveX Script task has very few properties:

• NameandDescription

• ActiveXScript—The full text of the ActiveX script

• ScriptLanguage—The scripting language used, such as VBScript or JScript You can setthis property on the Language tab in the ActiveX Script Task Properties dialog

• FunctionName—The name of the entry function for the script This is the function that iscalled by the package when the task is executed, and the function that returns theExecution Result to the package The default value is Main You can change this on theLanguage tab in the ActiveX Script Task Properties dialog

• AddGlobalVariables—Whether or not the package’s global variables can be accessedfrom inside the script The default is TRUE, and you cannot change this value in the userinterface

There are only two possible result values from the entry function of a script used in an ActiveXScript task The value returned determines whether the task is marked as a success or a failure:

• DTSTaskExecResult_Success—The task is a success

• DTSTaskExecResult_Failure—The task failed

Dynamically Modifying DTS Properties

Just about the only way to change properties during package execution in SQL Server 7.0 was

to use the ActiveX Script task In SQL Server 2000, it’s easier to use the Dynamic Propertiestask for much of this property manipulation

But there are still many situations in which you need to modify properties and the DynamicProperties task cannot do the job Here are a few examples:

• You want to construct the text of a message for a Send Mail task, with information aboutexecution times and the number of records processed

• You want to modify the name of the table in the FROM clause in the SQL Statement of

an Execute SQL task You want to set the name of this table with a parameter in theDTSRun command

• You want to read the name of a file in a directory and set a text file connection to thatparticular name

You can use ActiveX scripts to reference the properties of almost all the objects in DTS, andyou can change almost all the read/write properties All the code samples in the following sec-tions are in one ActiveX Script task in the ScriptSample package

Referencing a Package

First, you have to obtain a reference to the Package object before you can obtain a reference toany of the objects within a package The easiest way to get that reference is by using theParentproperty of the DTSGlobalVariablescollection:

Dim pkg Set pkg = DTSGlobalVariables.Parent MsgBox pkg.Name

Referencing a Connection

You reference a connection by naming a particular member of the package’sConnectionscollection:

Dim con Set con = pkg.Connections(“conPubs”) MsgBox con.DataSource

You can also gain a reference to the connection by using the ordinal number of that connection

in the Connectionscollection This method is unreliable, though, because if any connection ismodified, its ordinal number is changed to the last in the collection:

Set con = pkg.Connections(2) ‘You shouldn’t do this!

For the Connectionscollection, and almost all other collections in the DTS object model, youcan loop through all the objects using For Nextsyntax Remember that in VBScript, youalways use the word Nextby itself:

For Each con in pkg.Connections msgbox con.Name

Next

Referencing a Global Variable

DTS exposes the package’sGlobalVariablescollection in a special way as the

DTSGlobalVariablescollection You can reference DTSGlobalVariablesdirectly, without firstreferencing the Packageobject This is the easiest and most common way of referencing aglobal variable:

Msgbox DTSGlobalVariables(“TopSellingProduct”).Value

You can also reference a GlobalVariablethe same way you reference a connection, by ing the specific member of the package’sGlobalVariablescollection:

nam-Dim gv Set gv = pkg.GlobalVariables (“TopSellingProduct”) Msgbox gv.Value

Referencing Steps, Tasks, and Custom Tasks

You can reference steps and tasks as members of the package’s collections:

Dim stp, tsk Set stp = pkg.Steps(“stpLoadAuthors”) Msgbox stp.ExecutionStatus

Set tsk = pkg.Tasks(“tskLoadAuthors”) Msgbox tsk.Description

The description of a task is displayed in the Package Designer’s user interface The name ofthe task is not displayed, except in Disconnected Edit The name of the step is displayed on thesecond tab of the Workflow Properties dialog You can gain a reference to the task object with-out knowing its name, if you know the name of the step:

Dim stp, tsk Set stp = pkg.Steps(“stpLoadAuthors”) Set tsk = pkg.Tasks(stp.TaskName)After you have established a reference to the task, you can then reference the custom task:Dim cus

Set cus = tsk.CustomTaskMost of the properties of DTS tasks are specific to the custom task that is being used You canreference these specific properties as properties of the CustomTask object or as members of theProperties collections of the Task object You cannot reference any custom task specific proper-ties directly as properties of the Task object:

Dim pkg, tsk, cus Set pkg = DTSGLobalVariables.Parent Set tsk = pkg.Tasks(“tskUpdateAuthors”) ‘An Execute SQL task Set cus = tsk.CustomTask

Msgbox cus.SQLStatement ‘One way to reference custom task properties.

Msgbox tsk.Properties(“SQLStatement”).Value ‘The other way.

Msgbox tsk.SQLStatement ‘This Will Not Work!

Referencing the Collections and Objects in a Transform Data Task

You can continue in a similar way through the object hierarchy The transformation tasks aremore complex than most of the other tasks because there are collections inside of collections.Here’s how you could retrieve the Sizeproperty of one of the Source Columns:

Dim pkg, tsk, cus, trn, col Set pkg = DTSGlobalVariables.Parent Set tsk = pkg.Tasks(“tskLoadAuthors”) Set cus = tsk.CustomTask

Set trn = cus.Transformations(1) Set col = trn.SourceColumns(“au_id”) Msgbox col.Size

Referencing the DTS Application Object

You can also reference the DTS Applicationobject and all its collections from inside a age The Applicationobject is referenced independently of the package You have to use theCreateObjectmethod to initialize it: