Microsoft SQL Server 2000 Data Transformation Services- P15

//Destination column type is 8-byte real.*pdAverage = double cint; for j = 0; j < m_iDecimals; j++ { *pdAverage = *pdAverage / 10; } memcpypDestData, pdAverage, *pulLength; Adding the P

//Destination column type is 8-byte real.

*pdAverage = (double) cint;

for (j = 0; j < m_iDecimals; j++) {

*pdAverage = *pdAverage / 10;

} memcpy(pDestData, pdAverage, *pulLength);

Adding the Project’s Error Definitions

If you want to return specific error codes for conditions that occur in your custom tion, add them in the *.idl file following this line:

transforma-import “ocidl.idl”;

The code that defines the errors for the AvgXform task is shown in Listing 32.5

L ISTING 32.5 The Error Definitions for the Custom Task // -Error codes for the Average Transform - typedef [helpstring(

“Error codes generated by the average transform”)]

enum DTSAvgTransformError { DTSAvgXform_Error_WRONG_NUM_DEST_COLS = 0x80041001, DTSAvgXform_Error_NOT_CORRECT_TYPE = 0x80041002

Extending the Power of DTS

P ART VI

676

L ISTING 32.4 Continued

Registering the Custom Transformation

The ATL COM wizard creates the code to register the custom transformation during the buildprocess You can register the transformation on other computers by using the regsvr32

Build a custom transformation when you need the best possible performance, or when you areusing a particular ActiveX transformation script repeatedly

Creating a Custom Transformation with VC++

AcquireConnection method, 122 activating debugging, 55

Active Template Library, see ATL

ActiveX Script task, 40, 50

constants, 42creating, 346-348, 362-363Data Driven Query task, 214, 221-222debugging, 558-561

destination columns, 202empty strings, 203-204errors, 555

executing, 182-183failure precedence constraints, 476full outer joins, 206

global variables, 196-197lookups, 199-200null values, 203-204objects, 42

Package Object Browser, 47, 49programmatic identifier, 622properties, 172, 347, 496strings, 202-203

Transform Data task, 260-263transformations, 152, 172, 183-187Workflow, 494, 495

680

adding, 624

assignments, 370data, 89-90, 220fields, 277-278, 280global variables, 347,496

joins, 91lookups, 200objects, 29, 317-320properties to customtransformations,665-667records, 201

ADO recordset, 355-356 AllowIdentityInsert property, 158, 217 Analysis Services Processing task, 40,

44, 93, 413-417, 420-421

analyzing

errors, 554-555Internet clickstreamdata, 39-40

API calls, 361 appending

files, 152hierarchical rowsets,249

Application object, 28

creating, 446hierarchy, 625-628log files, 21properties, 626referencing, 350

architecture, DTS ages, 8, 10

pack-archiving data, 38, 60

arguments, DTSRun utility, 20-21

AS keyword rowsets, 249

assignments

constants, 374data files, 373Dynamic Propertiestask, 367-368,370-371environment variables,374

global variables, 372INI files, 373-374SQL queries, 372

asynchronous processing, 392 ATL transformation, 662-663, 677 Attributes field (dimension tables), 85-87

AutoCommit Transaction property,

465, 482 automating

filters, 415-416global variables, 55

avoiding dimension tables, 94-98

B

batch files, 228, 427 bcp command line utility, 268, 427, 585 binary files, 427

bindings table, 214 browsing meta data,

591, 593 bulk copy, 426 Bulk Insert task, 50, 268

bcp command lineutility, 268connections, 102creating, 270, 287-290data, 89

delimited text files, 271destination files, 270executing, 26fixed-length text files,271

format files, 52,271-276programmatic identifier,622

properties

Check Constraints, 281-282

Code Page, 284-285 Data File Type, 285 Enable Identity Insert, 283-284 Keep Nulls, 282-283 Maximum Errors, 286

Rows, 286 Sorted Data, 284

selecting, 269-270Transform Data task,268

business analysis data, 64-69, 589

Commit On option

681

C

cache, DTS packages, 56

calculating values, 37 Called Package, 385, 386

calling

functions, 535wizards, 504-507

capturing rowsets, 299-300

case sensitivity, 197-198 Catalog property, 108 categories, objects, 612 CCLine property, 407 cdw command-line utility, 505

changing Server property, 368 chaos, 483 Check Constraints property, 281-282 CheckingDurationIn Secs property, 656 CheckingIntervalInSecs property, 656

child packages, 378-383 child rowsets, 249 cleansing data, 89, 91 clearing objects, 317-320

clickstream data, 39-40, 60

CloseConnection property, 496 closing connections, 122-123, 480 clustering data, 67 code

events, 42libraries, 530pages, 284-285wizards, 506-507writing phases, 228

Batch Complete, 228

On Batch Complete, 240-241

On Insert Failure/Success,

228, 239-240

On Transform Failure, 228 Post Source Data,

228, 241-242 Pre Source, 228 Pump Complete,

228, 242 Row Transform, 228

collation, 314 collections

Add method, 624Count property, 624CustomTask interface,639-640

Insert method, 624Item method, 624methods, 625New method, 624Package2 object, 445Parallel Data Pumptask, 252

Parent property, 466,624

Properties, 625referencing, 350transformations,146-147

columns, 159

creating, 167-169destination, 139-141,143

identity, 283lineage, 600-601mappings, 519-520names, 575-577ordinal numbers,575-577prediction queries, 419properties, 159-160referencing, 183rowsets, 249scripts, 187source, 139-141, 143Transform Data task,138

COM objects

accessing, 27creating, 243references, 546

command-line utilities

cdw, 505DTSQiz, 505DTSRun, 53, 427osql, 427

wizards, 505-506

CommandProperties property, 294 commands, 21-22 CommandTimeout property, 294 Commit On option, 482

Commit Transaction option

682

Commit Transaction option, 485

CommitSuccess property, 496 comparing transfor- mations, 573-574 Component Object

Model, see COM

components, 370 configuration

Connection object,113-115

packages, 479-481transactions, 484

Connected property, 112

ConnectImmediate property, 112 Connection object, 27

Bulk Insert task, 102Called Package, 385closing, 122-123, 480collections, 109, 624creating, 512-513, 633data, 37, 102, 109-110DTS Designer, 10,103-104

Execute SQL task, 102methods, 122

modifying, 121-122OLAP data source, 119OLE DB providers, 102,114-115, 119, 121opening, 123Oracle DB Provider,121

packages, 628-631Parallel Data Pump task,102

properties, 104-114, 294referencing, 348-349Server property, 368text files, 115-119transactions, 486-487Transform Data task,102

troubleshooting, 490trusted, 20

Visual Basic, 104,114-115

consolidation, data, 36-37

constants

ActiveX scripts, 42Data Driven Query task,219-220

Dynamic Propertiestask, 374

constraints

copying, 316disabling, 281enabling, 281failure precedence, 476monitoring, 281-282precedence, 471-476

conversion

binary files to text, 427Visual Basic toVBScript, 360-361

Copy Column mation, 149, 168-169, 573

transfor-Copy Database Wizard, 33-34, 45, 52

calling, 504-507databases, 502-503,508-509

DTS packages, 511-512location, 508-509objects, 510-511packages, 17

Copy SQL Server Objects task

collation, 314CopySchema, 313data, 313-314databases, 310-311DropDestinationObjectsFirst, 313

executing, 26IncludeDependencies,313

objects, 312-315performance, 310programmatic identifier,622

properties, 317script file directories,316-317

user default options,315-316

Visual Basic, 320-323

copying

collation, 314commands, 21constraints, 316data, 313-314databases, 52, 310-311,316

FOREIGN keys, 316indexes, 316

logins, 316objects, 314-315, 502permissions, 316PRIMARY keys, 316tables, 516

triggers, 316

custom tasks

683

Copying in ActiveX Script transformation, 573

CopySchema property, 313

Count property, 624 covering indexes, 585 Create Process task, 50 CreateCustomToolTip method, 645

CreateProcess object, 428

creating

ActiveX Script task,346-348, 362-363Analysis ServicesProcessing task,420-421Application object, 446Bulk Insert task, 270,287-290

columns, 167-169COM object, 243connections, 512-513,553

constants, 374Copy Column transfor-mation, 168-169cubes, 656-657custom transformations,664

Data Driven Query task,216-219, 223-226data files, 7-8, 111, 373Data Mining PredictionQuery task, 421-423destination columns,159-160

DTS Designer, 103-104,534

Dynamic Propertiestask, 375-376environment variables,374

Execute Process task,407-408, 428-429Execute SQL task,292-294, 306-308files, 151-152FTP task, 327-331global variables, 55,194-197, 372INI files, 373-374local variables, 192LocalCubeTask, 657lookups, 199-200loops, 352-355, 542Message Queue task,400-403

multiphase data pump,243

objects, 31, 312-313,360

packages, 31-32,497-499, 613Parallel Data Pumptask, 254-257parameter queries, 217precedence constraints,473-474

Read File tion, 170-171source columns,159-160SQL queries, 372-373steps, 470

transforma-subordinate dimensions,98

tasks, 11-12, 335, 536,620-621, 634-635templates, 32-33Transfer Databases task,341-342

Transform Data task,128-129, 160-165, 201transformation scripts,185

transformations,167-169UDL files, 111utility packages,384-386variables, 360Write File transforma-tion, 170-171

CreationDate property, 466

CreatorComputerName property, 466

CRM (Customer Relationship Management), 60 cubes

creating, 656-657fact tables, 84processing, 413-414updating, 414

CurrentPhase property, 232

CurrentSource property, 232 custom tasks

creating, 634-635Description property,639

custom tasks

684

errors, 650-652events, 650-652Execute method,640-645interface, 636-637log files, 650-652Name property, 637-638Properties collection,639-640

referencing, 635,653-654registering, 652-653threads, 653VBScript, 654-655Visual Basic, 653-655

custom tions, 152

transforma-creating, 664error definitions, 676PreValidateSchemamethod, 667-669ProcessPhase method,672-676

properties, 665-667registering, 677starting, 664-665ValidateSchemamethod, 669-671

Customer Relationship Management (CRM), 60

CustomTask property, 615-616, 619

CustomTaskUI face, 645-649

inter-D

data

adding, 89-90, 220Analysis ServicesProcessing task, 93archiving, 38, 60business analysis, 64-69cleansing, 89, 91clustering, 67connections, 37consolidation, 36-37copying, 313-314Data Mining PredictionQuery task, 93decision trees, 67destination tables,277-278duplication, 37Dynamic Propertiestask, 370-371, 373enterprise data flow, 61errors, 554-555exporting, 427, 596history, 68

homogenization, 37Import/Export Wizard,519-520

Information Model, 598Internet, 60

loading, 569-572marts, 37-38merging, 206messages, 396-399mining, 45, 67-68, 93,

419, 421-423missing, 205modifying, 201, 503moving, 60

OLTP systems, 68

processing, 502queries, 253, 514-515reading, 483

returning, 464-465sources, 102, 191, 419,460

storage, 60, 72, 74, 76summarization, 37transactions, 63-64transformations, 7-8, 49,61-63, 228

types, 311warehousing, 37-38Web clickstream, 69

see also Meta Data

Services

Data Driven Query task

ActiveX scripts, 214bindings table, 214connections, 102creating, 216-219,223-226

executing, 26hierarchy, 617parameter queries,214-216

performance, 215, 582programmatic identifier,622

source queriesevents,214

tables, 220-222transformation status,219-220

Data File Type erty, 285

prop-Data Lineage Fields field, 82, 85 Data Link files, 109-111

copying, 52, 310-311,316

creating, 341-342destination, 311, 508information models,598-599

location, 508-509moving, 502-503objects, 310properties, 419queries, 294replication, 585repository, 598source, 311, 508tables, 310transferring, 336-337transformations, 599upgrading, 36, 502, 504users, 311

databases servers

Bulk Insert taskexecuting, 26Copy SQL ServerObjects task, 26Execute SQL task, 26Transfer Databases task,26

DataDrivenQueryTask object, 218

DataPumpOptions property, 158 DataPumpTask object,

136, 612 DataSource property,

108, 415 DataType property,

159, 197 DateTime in ActiveX Script transformation, 573

DateTime String formation, 148-149,

trans-171, 573

DB providers, 102 debugging, 55

ActiveX scripts,558-561Enterprise Manager,561-563

Package Manager,561-563

Decimals property, 666-667

decision trees, 67 declaring variables, 192-193, 446, 450, 531 decompression, text files, 427

default settings

configuration, 311Transformation Flagsdialog, 145

deleting

DTS packages, 21messages, 399packages, 443

queries, 218records, 221rows, 518tasks, 645

delimited text files, 271 Description property

ActiveX Script task, 347Connection object, 112CustomTask interface,

637, 639Data Mining PredictionQuery, 419

Dynamic Propertiestask, 367

Execute Package task,380

Execute Process task,428

Package object, 368,436-438, 465PackageInfo object, 445Send Mail task, 407Step object, 497Task object, 619Transform Data task,129-130

DesignerSettings erty, 626

prop-destination files, 116

Bulk Insert task, 270collections, 252columns, 139-141, 143creating, 159-160databases, 311, 508error rows, 49Import/Export Wizard,516-518

objects, 312-313referencing, 183

destination files

686

rows, 517-518source files, 276-281strings, 370

Transfer Databases task,335

Transform Data task,137-139

values, 202

DestTranslateChar property, 317 DestUseTransaction property, 317 development errors, 554

Dimension Keys field (fact tables), 82 dimension tables, 79

fields, 85loading, 90-92records, 84-85size, 84subordinate, 92, 95-98updates, 94-98

directories

manipulating, 357moving, 326script files, 316-317

dirty read, 483 disabling

constraints, 281steps, 480, 496

Disconnected Edit tool, 29-30, 54

displaying

commands, 21objects, 318-320packages, 457-458

dividing, see splitting

documentation, age connections, 628-631

pack-DropDestination ObjectsFirst, 313 DTS packages (Data Transformation Services), 16, 40, 42, 460

Application hierarchy,

613, 625-628architecture, 8, 10Browser, 591bulk copy, 426caches, 56connections, 10,103-104data transformations,7-8

deleting, 21Execute Package task,

42, 44Execute Process task,407-408, 426-429executing, 23-24, 367,511-512, 521FTP task, 42Import/Export Wizard,34-36, 504

Information Model, 599logs, 56

Message Queue task,

42, 44multiphase data pump,228-230

passwords, 21performance, 122-123precedence constraints,473-474

Query Designer,132-133replication, 521reports, 406retrieving, 14saving, 14, 521, 523scheduling, 24-25,511-512, 521-522Send Mail task, 406steps, 534

storing, 14-15structured storage file,15

templates, 54Transact-SQL code,549-552

Transformation Statusvalue, 188

versions, 16, 467Visual Basic, 15, 53

see also packages

DTSIsoLevel tions, 483-484 DTSLineage variables, 602

transac-DTSMQMessage object, 395

DTSMQType properties, 394

DTSPackageDSO, 460 DTSRun utility, 427

arguments, 20-21Dynamic Propertiestask, 366

Execute Process task,378

packages, 22-23Windows, 53

687

DTSStepScriptResult scripts, 495

DTSTransformPhaseInfo object, 232-233

DTSTransformStat ues, 188-191, 219-220 DTSWiz command-line utility, 505

val-duplication, data, 37 DWGuide tool, 599-600 Dynamic Properties task, 40, 42-43, 326

assignment, 367-368,370-371

constants, 374creating, 375-376data files, 373DTSRun utility, 366environment variables,374

files, 366global variables, 372INI files, 373-374object models, 28-29packages, 366programmatic identifier,623

properties, 367-370queries, 372-373statements, 366

dynamic values, 383

E

E-commerce, 60 Edit method, 645, 647

editing

objects, 29-30overwriting, 31properties, 29-30

email 406 empty strings, 203-204 enabling

constraints, 281lineage variables,602-604multiphase data pump,228-230

properties, 283triggers, 282

encryption

commands, 22packages, 443-445parameters, 20

enterprise data flow, 61

Enterprise Manager

packages, 17debugging, 561-563wizards, 505

entrance functions, 244-245

entry function, 361 EnumPackageInfos method, 447 EnumPackageLineages method, 608

EnumPackageLogRecor

ds method, 458 EnumStepLogRecords method, 458

EnumTaskLogRecords method, 458

environment variables, 374

ErrorIfFileNotFound property, 170 ErrorIfReceiveMessage TimeOut property, 400

errors

ActiveX, debugging,558-561

administration tion, 558

notifica-custom tasks, 650-652definitions, 676development, 554Enterprise Manager,561-563

format files, 275generating, 49-50global variables, 195handling, 154-156, 361,540-541, 548-549, 554Import/Export Wizard,523

incorrect data, 554-555local variables, 193lookups, 557maximum, 286

On Transform Failuresubphase, 238-239Package Manager,561-563

packages, 481raising, 555-557records, 557-558rows, 49, 556storing, 340transactions, 558troubleshooting, 557

688

events

code, 42custom tasks, 650-652handling, 538-540

ExceptionFileName property, 154 exceptions in rows, 191 Execute method

CustomTask interface,640-645

Lookup objects, 200

Execute Package task, 40

creating, 379-380DTS packages, 42, 44executing, 26

global variables,380-383NestedExecutionLevelproperty, 383-384packages, 378, 380programmatic identifier,622

properties, 380RepositoryDatabaseName, 380ServerUserName, 380utility functions, 378,384-386

Visual Basic, 387-389

Execute Process task

bcp commands, 427bulk copy, 426creating, 407-408,428-429

DTS packages, 426, 427DTSRun utility, 378executing, 26

programmatic identifier,622

properties, 426, 428Visual Basic, 429-430

Execute SQL task, 40, 50

connections, 102creating, 292-294,306-308

data, 90executing, 26input parameters,294-296output, 51output parameters,296-298

programmatic identifier,622

properties, 294queries, 50recordsets, 51remote servers, 301-306rowsets, 299-300SQL statements,300-301stored procedures, 301

ExecuteCommandBy

ID method, 506 ExecuteInMainThread property, 480, 496 executing

Batch Complete phase,233

bcp commands, 427Bulk Insert task, 26Called Package, 385Copy SQL ServerObjects task, 26

Data Driven Query task,26

DTS packages, 23-24,

367, 427, 511-512,

521, 549-552DTSRun utility, 22-23Execute Package task,

26, 379-380context, 25-26FTP processes, 427Import/Export Wizard,523

Insert Failure/Successphase, 231, 233

On Batch Completephase, 231packages, 16-22,458-460, 477-480,542-545

Parallel Data Pump task,26

Post Row Transformphase, 231

Post Source data phase,

231, 233Pre Source phase, 230,233

Pump Complete phase,

231, 233queries, 220remote servers, 378Row Transform phase,

230, 233scripts, 182-183, 186Send Mail task, 406steps, 471, 476-477text file decompression,427

transaction steps,487-490

FTP task

689

Transform Data task, 26Transform Failurephase, 231, 233

execution context, 25-26

ExecutionResult property, 497 ExecutionStatus property, 477, 497 ExecutionTime property, 497 Explicit Global Variables tool, 55, 465 exporting

data, 427, 596XML, 40

extended objects, 618, 622

failing packages, 481, 555

FailPackageOnError property, 466, 555 FailPackageOnLog Failure property, 466

FailPackageOnTimeout property, 428

failure precedence straints, 476

con-Fast Load, 570 FastLoadOptions prop- erty, 157-159, 217, 570

fetching

buffer size, 156, 578rows, 189

fields

adding, 277-278, 280dimension tables, 85fact tables, 82-84sorting, 278-279updating, 220

File Transfer Protocol

task, see FTP task

FileAttachments property, 407 FilePath property, 170 files

accessing, 360appending, 152creating, 151-152finding, 655-656format, 271-276location, 508-509manipulating, 357moving, 151, 326naming, 380, 509, 656packages, 442-443,450-451

text, 271

filters

automating, 415-416messages, 398-399package log files, 457

Find File task, 655-656 FinishTime property, 497

FirstRow property, 156 fixed values, 383 fixed-length text files, 271

Flags parameter, 160, 439-440, 442

flattened mode, 253 Folder property, 656 For Next loop, 360 FOR XML statement,

40, 464-465 ForceBlobsInMemory property, 147 ForceSourceBlobs Buffered property, 147

FOREIGN keys, 316 format files

auto-generating, 52Bulk Insert task,271-272error messages, 275generating, 274-276structure, 272-274troubleshooting, 275

freestanding icon, 173-176

FTP task, 44

creating, 327-331data, 89

directories, 326DTS packages, 42Dynamics Propertiestask, 326

files, 326

FTP task

690

processes, 427programmatic identifier,623

properties, 327-329source location, 327

full cube updating, 414 full outer joins, 206 FunctionEntry property, 172

FunctionName erty, 347, 496 functions

prop-calling, 535entry, 361Pump Complete dataphase, 234-235user-defined, 57

G

generating

error files, 49-50format files, 52,274-276

Get function, 666-667 GetDefaultProfileName method, 408

GetDTSVersionInfo method, 467 GetExpandedProcess CommandLine, 428 GetObjectForTransfer method, 317

GetRootItem method, 506

GetSQLNamespace Object method, 506

GetUIInfo method, 645 global variables, 27, 624

case sensitivity, 197-198child packages, 380-383creating, 55, 194-197Dynamic Propertiestask, 372

messages, 397-399methods, 198packages, 21, 194Pre Source phase,235-237referencing, 194, 196,349

sending, 392values, 53, 346, 368,378

GoTo command, 360 GROUP BY clause, 584

H

handling

errors, 361, 540-541,554

events, 538-540

headers, packages, 531 Help method, 645, 649 hierarchical rowsets, 249-250

child rowsets, 249keywords, 249mode, 252-253Parallel Data Pumptask, 251

parent rowsets, 249queries, 251

shape language, 249subsets, 249transformations, 248

hierarchy

Data Driven Query task,617

dimension tables, 86DTS Application object,625-628

objects, 613-618Parallel Data Pump task,617

Task object, 614Transformation object,617

history of data, 68 HOLAP (Hybrid OLAP), 76

homogenization of data, 37

HTML files, 102

I

icons, freestanding, 173-176

ID property, 104-107 identity columns, 283, 518

Import/Export Wizard,

502, 568

calling, 504-507column mappings,519-520

connections, 512-513data, 503, 519-520databases, 504