Hands-On Microsoft SQL Server 2008 Integration Services part 19 docx

The variable passed by the Foreach Loop Container was used to set the source filename in the File System task, which was configured to move files from a dynamic source to the hard-coded

Exercise (Configure File System Task)

In this final part, you will configure the File System task to move the downloaded zipped files from the C:\SSIS\downloads folder to the C:\SSIS\downloads\Archive folder This task will move the files one by one with each iteration of the Foreach Loop

Container It will use the variable User::fname, populated by Foreach Loop Container,

to determine the source filename

8 Drag and drop the File System task from the Toolbox within the Enumerating Files Container

9 Double-click the File System Task icon to open the File System Task Editor Select Move File in the Operation field In the General area on the right pane, fill

in the following details:

Figure 5-6 Configuring the Foreach Loop Container to enumerate fully qualified filenames

Name Archive downloaded files

Description This task copies downloaded files from the ‘downloads’ folder to the ‘Archive’

folder

10. In the Source Connection section, set IsSourcePathVariable to True

11. Click in the SourceVariable field and then click the down arrow to see the

drop-down list Choose User::fname as shown in Figure 5-7

12. In the Destination Connection section, verify that IsDestinationPathVariable is

set to False

Figure 5-7 Configuring the File System task for moving files

13. Click in the DestinationConnection field and then click the down arrow to see the drop-down list Choose the <New Connection…> to open File Connection Manager Editor In the Usage type field, select Existing Folder and type

C:\ SSIS\downloads\Archive in the Folder field Note that a File Connection

Manager named Archive has been created

14. The OverwriteDestination field allows you to overwrite the files with the same name at destination folder Be mindful while configuring this option in the production environment Leave it set at the default value of False Click OK to close the File System Task Editor

15. Now that your package is ready to be run, press f5 on the keyboard to run the package and notice how the Enumerating Files Container changes from yellow followed by Archive Downloaded Files task changing from yellow to green This cycle is repeated twice before both the objects stop processing and turn green

to declare success of the operation Each time Archive Downloaded Files task changes color from yellow to green, one file has been moved Stop debugging the package by pressing shift-f5

16. Run Windows Explorer to check that the files have been moved from C:\SSIS\ downloads folder to the Archive subfolder in this directory

17. Press ctrl-shift-s to save all the files in this solution and then choose File | Close Project

Review

You have configured the Foreach Loop Container to enumerate over files in a folder and pass the filenames via a variable to the File System task The variable passed by the Foreach Loop Container was used to set the source filename in the File System task, which was configured to move files from a dynamic source to the hard-coded destination Archive folder In this exercise, you have seen the functionality provided

by SSIS components to run in synchronization, where one component was reading the files one by one and passing the information to the other component that was moving those files to a different folder as it receives the filenames from the parent container

Web Service Task

You can read data from a Web Service method and write that data to a variable or a file using the Web Service task For example, you can obtain a list of postal codes from the local postal company, write it to a flat file using the Web Service task, and then do the lookup against this postal codes file to clean or standardize your data at loading time Web Service task uses the HTTP Connection Manager to connect to the web service HTTP Connection Manager specifies the server URL, user credentials, optional client certificate details, time-out length, proxy settings, and so on

The Web Service Description Language (WSDL) is an XML-based language used

for defining web services in a WSDL file, which lists the methods that the web service

offers, the input parameters that the methods require, the responses that the methods

return, and how to communicate with the web service Thus, a web service requires a

WSDL file to get details of settings to communicate with another web service The

HTTP Connection Manager can specify in the Server URL field a web site URL or

a WSDL file URL If you specify the WSDL file URL in the Server URL field, the

computer can download the WSDL file automatically However, if you are specifying

the web site URL, you must copy the WSDL file to the local computer

XML Task

Whenever you are working with XML data, you will be most likely using the XML

task to perform operations on the XML documents This task is designed to work

with the XML documents from the workflow point of view, whereas if you want to

bring XML data, i.e., the content of an XML document in the data flow, to apply

transformations, you will be using the XML Source adapter while configuring your

Data Flow task The XML Source adapter is available in the Data Flow Sources section

in the Toolbox when you’re working with the Data Flow task on the Data Flow panel

Using the XML task, you can perform the following operations on XML

documents:

1 Retrieve XML documents and dynamically modify those documents at run time

2 Select a segment of the data from the XML document using XPath expressions

similar to how you select data using an SQL query against database tables

3 Transform an XML document using XSLT (extensible stylesheet language

transformations) style sheets and make it compatible with your application

database

4 Merge multiple documents to make one comprehensive document at run time

and use it to create reports

5 Validate an XML document against the specified schema definition

6 Compare an XML document against another XML document

The XML task can automatically retrieve a source XML document from a specified

location To perform this operation, the XML task can use a File Connection Manager,

though you can directly enter XML data in the task or specify a variable to access the XML

file If the XML task is configured to use a File Connection Manager, the connection

string specified inside the File Connection Manager provides the information of the

path of the XML file; however, if the XML task is configured to use a variable, the

specified variable contains the path to the XML document At run time, other processes

or tasks in the package can dynamically populate this variable Like the retrieval process

of XML documents, the XML task can save the result set after applying the defined operation to a variable or file By now, you can guess that to write to a file, the XML task will be using a File Connection Manager

The XML Task Editor has a dynamic configuration interface that changes depending upon the type of operation you choose to apply to the XML documents Following are the descriptions of these configuration areas

Input Section

As mentioned, the XML task can retrieve the source document that is specified under the Input section in the XML Task Editor You can choose from three available SourceType options: Direct Input allows you to type in XML data directly in the Source field; File Connection allows you to specify a File Connection Manager in the Source field; and Variable allows you to specify a variable name in the Source field

Second Operand Section

This section defines the second document required for the operation to be performed The type of second document depends on the type of operation For example, the second document type will be an XML document if you are merging two documents, while the second document will be an XSD (XML Schema Definition) document if you are trying to validate an XML document against an XSD schema Again, like the Input section, you can choose between the three types—Direct input, File connection, and Variable—in the SecondOperandType field and, based on your choice, specify the document details in the SecondOperand field

Output Section

In this section, you specify whether you want to save the results of the operation performed by running the XML task You can save the results to a variable or a file by using the File Connection Manager to specify the destination file You can also choose

to overwrite the destination

Operation Options Section

This section is dynamic and changes with the option selection For example, for a Diff operation, this section will change to the Diff Options section (see Figure 5-8), and for Merge operation, this will become the Merge Options section with its specific fields relevant to the operation The two operations XSLT and Patch do not have this section at all

The XML task has six predefined operations for you to use The configuration

layout of the options changes as soon as you select a different operation

Validate

You can validate the XML document against a Document Type Definition (DTD) or

XML Schema Definition (XSD) schema The XML document you want to validate is

specified in the Input section in the Editor, and the schema document is specified in the

Second Operand section The type of schema document depends upon what you specify

for ValidationType—XSD or DTD With either type of ValidationType, you can

choose to fail the operation on a validation failure in the FailOnValidationFail field

Figure 5-8 The XML Task Editor

You can perform XSL transformations on the XML documents using XSLT style sheets The Second Operand should contain the reference to the XSLT document, which you can type directly into the field or specify by using either the File Connection Manager or a variable

XPATH

Using this operation, you can perform XPATH queries and evaluations on the XML document The Second Operand should contain a reference to the second XML document, which you can type directly into the field or specify by using either the File Connection Manager or a variable You can select the type of XPATH operation in the XPathOperation field The XPathOperation field provides three options

Evaluation

Node list

Values

c Return the results in a concatenated string for text values of all the selected nodes

Merge

Using this operation, you can merge two XML documents This operation adds the contents of the document specified in the Second Operand section into the source document The operation can specify a merge location within the base document One thing to note here is that the XML task merges only the documents that have Unicode encoding To determine whether your documents are using Unicode encoding, open the XML document with any editor or using Notepad and look at the beginning

of the document to find [encoding="UTF-8"] in the declaration statement UTF-8 indicates the 8-bit Unicode encoding

Diff

Using this operation you can compare the source XML document to the document specified in the Second Operand section and write the differences to an XML

document called a Diffgram document The Diff operation provides a number of

options to customize this comparison:

DiffAlgorithm

between comparison algorithm to be fast or precise The Auto option lets the Diff operation decide whether to select a fast or precise comparison based on the size

of the documents being compared

IgnoreNamespaces

identifier) of an element and its attribute names are compared

IgnorePrefixes

compared

IgnoreXMLDeclaration

compared

IgnoreOrderOfChildElements

and this option specifies whether the order of child elements is compared.

IgnoreWhiteSpaces

IgnoreProcessingInstructions

compared

IgnoreDTD

FailOnDifference

e.g., an XML document fails to validate according to the validation schema

SaveDiffGram

Patch

Using this operation, you can apply the Diffgram document you saved earlier in the

package during the Diff operation to an XML document By doing this, you actually

create a new XML document that includes the contents of the Diffgram document

created earlier by the Diff operation

Execute SQL Task

The Execute SQL task is the main workhorse task to run SQL statements or stored

procedures and uses the power of the underlying relational database If you have used

DTS, you may have used this task Typically in DTS, once you have loaded data

into a database and you apply transformations using the Execute SQL task These

transformations vary from generating salutations to lookup transformations, deriving

columns, or applying business rules using the SQL Server relational engine The design

philosophy used in SQL Server 2008 Integration Services allow you to perform many

of these tasks during the loading phase while data is still in memory, thereby increasing

performance by reducing the repeated and inefficient transformations that require data

to be staged or involved read/write operations on hard disks The power of the Execute

SQL task is still available in SSIS in a more usable form by providing ability to use variables, to create expressions over the properties of the task, and to return a result set

to the control flow that can be used to populate a variable

Using the Execute SQL task, you can perform workflow tasks such as create, alter, drop, or truncate tables or views You can run a stored procedure and store the result

to a variable to be used later in the package You can use it to run either a single SQL statement or multiple SQL statements, parameterized SQL statements, and save the rowset returned from the query in to a variable You have already used this task in the “Using System Variables to Create Custom Logs” Hands-On exercise in Chapter 3, where you used a parameterized SQL statement, and in the “Contacting Opportunities” Hands-On exercise in Chapter 4, where you saved the resulting rowset

to a variable, which then got enumerated over by a Foreach Loop Container

If you scroll to the Maintenance Plan Tasks in the Control Flow Toolbox, you will see a similar task, the Execute T-SQL Statement task The Execute T-SQL Statement task has a more simple interface than the Execute SQL task and is focused on

performing maintenance tasks on SQL Server databases using T-SQL It doesn’t give you any facility to run parameterized queries and direct the result set to the work flow, whereas the Execute SQL task has a more complex interface and is designed for use in

a relatively complex workflow where you need to use SQL Statements against not only the SQL Server but a variety of sources, deal with variables, run parameterized queries,

or direct the result set to the data flow

Keep this task open in front of you and try various selections as we go through each option, as the task contains dynamic fields that change depending upon the choices you make in certain fields

The Execute SQL Task Editor includes General, Parameter Mapping, Result Set, and Expressions pages

General Page

In this page, you define a Name and Description for the task under the General section

In the Options section, you can specify a TimeOut value in seconds for the query to run before timing out By default, the TimeOut value is 0 (zero), indicating an infinite time The CodePage field allows you to specify the code page value

In the Result Set section, you choose one of four options based upon the result set returned by the SQL statement you specify in this task Based on the type of SQL statement—i.e., whether it is a SELECT statement or INSERT/UPDATE/DELETE statement—the result set may or may not be returned Also, the result set may contain

zero rows, one row, or many rows, and the following four options in the Execute SQL

Task Editor ResultSet field allow you to configure them:

None

statement that returns the result set containing zero rows

Single Row

row in the result set

Full result set

XML

c When the SQL statement returns a result set in the XML format

In the SQL Statement section ConnectionType field, options are the EXCEL,

OLE DB, ODBC, ADO, ADO.NET, and SQLMOBILE connection manager types,

used for connecting to a data source Depending on the type of connection manager

you’ve chosen, the Connection field provides a drop-down list of already configured

connection managers of the same type or provides a <New Connection…> option to

let you add a connection manager of the appropriate type The interface provided by

the <New Connection…> option changes to match your selection of the connection

manager specified in the ConnectionType field

Depending on the data source type, you can write a query using an SQL statement in

the dialect the specified data source can parse Further, you can specify the source from

where the SQL statement can be read for execution in the SQLSourceType field The

selection of the source in the SQLSourceType field changes the next field dynamically

(which is coupled to it) to match the SQLSourceType choice The options available in

the SQLSourceType field and how it affects the coupled field are explained here:

Direct input

changes the coupled field to SQLStatement, which provides an interface in which

to type your query

File connection

choose this option to enable the Execute SQL task to get the SQL statements

from the file Selecting this option changes the coupled field to FileConnection,

which allows you to specify a File Connection Manager to connect to an existing

file containing SQL statements

Variable

variable This option changes the coupled field to the SourceVariable field, which

provides a drop-down list of all the system and user variables