sql server 2005 bible

About the AuthorPaul Nielsen is a hands-on database developer, Microsoft SQL Server MVP, author, and trainer specializing in data architecture and Microsoft SQL Server technologies.. Bes

SQL Server ™

2005 Bible

Paul Nielsen

SQL Server ™

2005 Bible

SQL Server ™

2005 Bible

Paul Nielsen

LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND THE AUTHOR MAKE NO REPRESENTATIONS

OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE CONTENTS OF THIS WORK AND SPECIFICALLY DISCLAIM ALL WARRANTIES, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE NO WARRANTY MAY BE CREATED OR EXTENDED BY SALES OR PROMOTIONAL MATERIALS THE ADVICE AND STRATEGIES CONTAINED HEREIN MAY NOT BE SUITABLE FOR EVERY SITUATION THIS WORK IS SOLD WITH THE UNDERSTANDING THAT THE PUBLISHER IS NOT ENGAGED IN RENDERING LEGAL, ACCOUNTING,

OR OTHER PROFESSIONAL SERVICES IF PROFESSIONAL ASSISTANCE IS REQUIRED, THE SERVICES OF A COMPETENT PROFESSIONAL PERSON SHOULD BE SOUGHT NEITHER THE PUBLISHER NOR THE AUTHOR SHALL BE LIABLE FOR DAMAGES ARISING HEREFROM THE FACT THAT AN ORGANIZATION OR WEBSITE IS REFERRED TO IN THIS WORK AS A CITATION AND/OR A POTENTIAL SOURCE OF FURTHER INFORMATION DOES NOT MEAN THAT THE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMATION THE ORGANIZATION OR WEBSITE MAY PROVIDE

OR RECOMMENDATIONS IT MAY MAKE FURTHER, READERS SHOULD BE AWARE THAT INTERNET WEBSITES LISTED

IN THIS WORK MAY HAVE CHANGED OR DISAPPEARED BETWEEN WHEN THIS WORK WAS WRITTEN AND WHEN IT

Copyright © 2007 by Wiley Publishing, Inc., Indianapolis, Indiana

Published by Wiley Publishing, Inc., Indianapolis, Indiana

Published simultaneously in Canada

Trademarks: Wiley and related trade dress are registered trademarks of Wiley Publishing, Inc., in the United States and

other countries, and may not be used without written permission All other trademarks are the property of their respectiveowners Wiley Publishing, Inc., is not associated with any product or vendor mentioned in this book

Wiley also publishes its books in a variety of electronic formats Some content that appears in print may not be available inelectronic books

For Edie, my bride, who is so beautiful to me

Technical Editors

John Mueller Kevin Lloyd Chris Shaw

Quality Control Technician

John Greenough

Media Development Specialists

Angela Denny Kit Malone Travis Silvers

Media Development Coordinator

Laura Atkinson

Proofreading and Indexing

Techbooks

About the Author

Paul Nielsen is a hands-on database developer, Microsoft SQL Server MVP, author, and

trainer specializing in data architecture and Microsoft SQL Server technologies Besides ing several certifications, he was the Design-SME (subject matter expert) for the MicrosoftOfficial Course 2784: Tuning and Optimizing Queries Using Microsoft SQL Server 2005.Paul has been developing datacentric solutions since 1982, and was the Enterprise DataArchitect for Compassion International, a SQL Server instructor with Learning Tree, the technical editor for a database magazine, and a U S Navy Submariner (Data Systems Tech)

hold-He serves on the PASS (Professional Association for SQL Server) Board of Directors, is active

in the Colorado area SQL Server user groups, and organizes the Colorado PASS Camp Forrecreation, Paul enjoys scuba diving, playing guitar, and hiking/biking in the Front Rangemountains of Colorado Paul regularly offers public and private workshops in SQL Serverdevelopment and data architecture, including the popular Advanced Design and Optimizationcourse For more information visit his website at www.SQLServerBible.com

About the Co-Authors and Contributors

Hilary Cotter, Microsoft SQL Server MVP and author of Chapter 13, “Using Full-Text Search,”

and Chapter 39, “Replicating Data,” is director of Text Mining and Database Strategy forRelevantNoise, which indexes blogs for business intelligence He received a degree inmechanical engineering from the University of Toronto, and subsequently studied economics

at the University of Calgary and computer science at the University of California (Berkeley)

He and his beautiful wife, Miriam, have five children ranging in age from 13 months to sixyears Hilary Cotter has a companion volume to this text on merge replication, which is in the works for 2005

Hilary has worked for Microsoft, Merrill Lynch, UBS-Paine Webber, MetLife, VoiceStream,Tiffany & Co., Pacific Bell, Cahners, Novartis, Petro-Canada, Associated Press, and Johnsonand Johnson You can find Hilary on the Microsoft newsgroups or contact him at hilary.cotter@gmail.com

Monte Holyfield, author of Chapter 46, “Authoring Reports with Reporting Services,” and

Chapter 47, “Administering Reports with Reporting Services,” is the founder and manager ofMJH Software Solutions LLC (www.mjhsoftware.com) He has over 10 years of experiencedeveloping custom software and has served in many capacities including solution architect,principal consultant, mentor, and project lead He is a Microsoft Certified Professional andhas successfully leveraged the Microsoft.NET technologies to deliver outstanding solutions to

a diverse group of clients Monte offers a unique combination of technical, analytical, andleadership skills and shares a contagious “can do” attitude When Monte is not building soft-ware, he can most likely be found with his family doing something active such as riding dirtbikes, skiing, windsurfing, or just enjoying the incredible Colorado outdoors

Kevin Lloyd, Technical Editor, is an experienced SQL Server Developer and Data Architect,

Microsoft MCP, consultant and primary member of Data Solutions, LLC He specializes indata architecture and application integration, data modeling, data quality, and performancetuning Besides working with SQL Server, he develops C# XML Web Services for SQL Mobile

replication He also enjoys mentoring others in SQL and writing Windows Mobile tions Kevin has been developing data-centric solutions since 1996 in finance, e-commerce,and government, and has worked or consulted with organizations such as MicrosoftCorporation; Frontier Airlines, Inc.; Compassion International, Inc.; and Keane, Inc He can bereached at kevin@kevndeb.com When not writing code, he enjoys woodworking, spendingtime with his family, and traveling.

applica-George Mudrak, author of chapter 4, “Installing SQL Server 2005,” Chapter 5, “Client Software

Connectivity,” and Chapter 54, “Designing High-Performance Data Access Providers,” works

as an Enterprise Data Architect for Compassion International in Colorado Springs, CO He hasbeen doing full life-cycle programming in a variety of languages for over 12 years, with hismost recent development activities using Microsoft’s NET technologies He is currently work-ing within the Enterprise Information Architecture and Management (EIM/EIA) realm and hasworked at start-ups as well as major corporations When he’s not working, George enjoysspending time with his family, working out, and pursuing hobbies

John Mueller, primary technical reviewer and co-author of Chapter 34, “Configuring SQL

Server,” Chapter 35, “Transferring Databases,” Chapter 37, “Maintaining the Database,”Chapter 38, “Automating Database Maintenance with SQL Server Agent,” and Chapter 41,

“Administering SQL Server Express,” is a freelance author and technical editor He has writing

in his blood, having produced 69 books and over 300 articles to date, with the topics thatrange from networking to artificial intelligence and from database management to heads-down programming Some of his current books include a Windows power optimization book,

a book on NET security, and books on Amazon Web Services, Google Web Services, and eBayWeb Services He’s currently editor of the NET electronic newsletter for Pinnacle Publishing.See www.freeenewsletters.com for more information When John isn’t working at the com-puter, he is likely in his workshop An avid candle maker and woodworker, he can be foundworking at a lathe or putting the finishing touches on a bookcase His website is

www.mwt.net/~jmueller

Darren Shaffer, Microsoft Compact Framework MVP, author of Chapter 26, “Developing for

SQL Server Mobile,” is the Chief Architect for NET Solutions at Connected Innovation LLC(www.connectedinnovation.com) Darren is responsible for the design and development ofover 25 Compact Framework solutions for Fortune 1000 clients in the past four years Darren

is a frequent speaker at Global Microsoft Events such as MEDC and TechEd as well as a tributor to MSDN, emphasizing the importance of the NET Compact Framework,

con-SQL CE/con-SQL Mobile, and teaching mobile development best practices Darren moderates the MSDN SQL Mobile forum, and authors a well-traveled blog for NET junkies at www.dotnetjunkies.com/WebLog/darrenshaffer/default.aspx A West Point graduate andretired Army Telecommunications Officer, Darren now lives in Colorado with his wife anddaughter

Chris Shaw, technical editor and author of chapter 52, “Providing High Availability” is President

of SQL on Call in Colorado Springs, Colorado, for the past 5 years Chris has been working atPremiere Global Services He has also worked with Yellow Pages Inc, Ford Fairlane MotorCredit, AdminiQuest, Wells Fargo and AT&T Wireless Chris has published a number of Articlesfor a number of magazines including SQL Server Magazine and SQL Standard Magazine

He is a member of Rocky Mountain Microsoft Community Insiders group, is the president of theColorado Springs SQL Server Users Group, and is one of the organizers of the Colorado PASSCamp He has also been a guest speaker to a number of other user groups, is a regular speaker

at PASS Summits and SQL Connection, and presents web casts with SSWUG; recent tions include Replication SQL Server 2005, Performance Metrics, and Crisis Management

presenta-Andrew Watt, Microsoft SQL Server MVP, wrote Chapter 33, “InfoPath and SQL Server 2005.”

Andrew Watt is an independent consultant and experienced computer book author He hasbeen a Microsoft MVP since 2004

Michael White has focused on database development and administration since 1992.

Concentrating on Microsoft’s Business Intelligence (BI) tools and applications since 2000, hehas architected and implemented large warehousing and Analysis Services applications, aswell as nontraditional applications of BI tools After many years in corporate IT and consult-ing, Mike currently works as a software engineer for Wall Street On Demand He is a strongadvocate for the underused BI toolset; currently leads the Boulder (Colorado) SQL ServerUser Group and is one of the organizers of the Colorado PASS Camp Mike wrote Chapter 42,

“ETL with Integration Services,” Chapter 43, “Business Intelligence with Analysis Services,”

Chapter 44, “Data Mining with Analysis Services,” Chapter 45, “Programming MDX Queries,”

and Chapter 48, “Analyzing Data with Excel and Data Analyzer.”

Bill Wunder, Microsoft SQL Server MVP, wrote Chapter 27, “Programming CLR Assemblies

within SQL Server,” Chapter 29, “Persisting Custom Data Types,” and Chapter 30,

“Programming with ADO.NET 2.0.” Bill’s introduction to set based technology began when hewas an AS/400 developer in 1993 The query processor at the time was built by an Australiancompany, LANSA The 4GL language was called Relational Database Markup Language (RDML)

It was an innovative product written in RPG that included commands like SELECT, INSERT,UPDATE and DELETE Shortly after that introduction he became the DBA for the company’ssales force automation application The database had just been upgraded to SQL Server 4.2.1running on the Novell OS He has been working with SQL Server since that time He begansharing his scripts on the Internet in 1995 He has published over 100 SQL Server related arti-cles at various web sites He has been a speaker at swynk.com’s Back Office AdministratorsConferences and Professional Association for SQL Server (PASS) conferences in the US andEurope He founded and has been a frequent speaker at the Boulder SQL Server Users Group

He received Microsoft’s MVP award for his contributions to the SQL Server community in 2004,

2005 and 2006 Today Bill is a working DBA on the Dutch Antilles island of Curaçao Readersare invited to contact him at bwunder@yahoo.com with any questions or comments

Contents at a Glance

Foreword xxxvii

Acknowledgments xxxix

Introduction xli Part I: Laying the Foundation 1

Chapter 1: The Information Architecture Principle 3

Chapter 2: Relational Database Modeling 25

Chapter 3: Exploring SQL Server 2005 Architecture 47

Chapter 4: Installing SQL Server 2005 69

Chapter 5: Client Software Connectivity 91

Chapter 6: Using Management Studio 99

Part II: Manipulating Data with Select 115

Chapter 7: Understanding Basic Query Flow 117

Chapter 8: Using Expressions and Scalar Functions 141

Chapter 9: Merging Data with Joins and Unions 171

Chapter 10: Including Data with Subqueries and CTEs 199

Chapter 11: Aggregating Data 221

Chapter 12: Navigating Hierarchical Data 243

Chapter 13: Using Full-Text Search 257

Chapter 14: Creating Views 279

Chapter 15: Working with Distributed Queries 295

Chapter 16: Modifying Data 317

Part III: Developing with SQL Server 353

Chapter 17: Implementing the Physical Database Schema 355

Chapter 18: Programming with Transact-SQL 397

Chapter 19: Performing Bulk Operations 427

Chapter 20: Kill the Cursor! 431

Chapter 21: Developing Stored Procedures 449

Chapter 22: Building User-Defined Functions 471

Chapter 23: Implementing Triggers 483

Chapter 24: Exploring Advanced T-SQL Solutions 499

Chapter 25: Creating Extensibility with a Data Abstraction Layer 539

Chapter 26: Developing for SQL Server Everywhere 549

Chapter 27: Programming CLR Assemblies within SQL Server 589

Chapter 28: Queueing Data with Service Broker 617

Chapter 29: Persisting Custom Data Types 623

Chapter 30: Programming with ADO.NET 2.0 643

Chapter 31: Using XML, XPath, and XQuery 677

Chapter 32: Building an SOA Data Store with Web Services 689

Chapter 33: InfoPath and SQL Server 2005 695

PART IV: Enterprise Data Management 703

Chapter 34: Configuring SQL Server 705

Chapter 35: Transferring Databases 741

Chapter 36: Recovery Planning 751

Chapter 37: Maintaining the Database 775

Chapter 38: Automating Database Maintenance with SQL Server Agent 795

Chapter 39: Replicating Data 813

Chapter 40: Securing Databases 857

Chapter 41: Administering SQL Server Express 889

Part V: Business Intelligence 897

Chapter 42: ETL with Integration Services 899

Chapter 43: Business Intelligence with Analysis Services 935

Chapter 44: Data Mining with Analysis Services 973

Chapter 45: Programming MDX Queries 989

Chapter 46: Authoring Reports with Reporting Services 1005

Chapter 47: Administering Reports with Reporting Services 1035

Chapter 48: Analyzing Data with Excel and Data Analyzer 1051

Part VI: Optimization Strategies 1063

Chapter 49: Measuring Performance 1065

Chapter 50: Query Analysis and Index Tuning 1081

Chapter 51: Managing Transactions, Locking, and Blocking 1105

Chapter 52: Providing High Availability 1141

Chapter 53: Scaling Very Large Databases 1153

Chapter 54: Designing High-Performance Data Access Providers 1173

Part VII: Appendixes 1197

Appendix A: SQL Server 2005 Specifications 1199

Appendix B: Sample Databases 1205

Index 1213

Foreword xxxvii

Acknowledgments xxxix

Introduction xli Part I: Laying the Foundation 1 Chapter 1: The Information Architecture Principle 3

Simplicity vs Complexity 4

Complexity 4

Simplicity 5

The Usability Objective 5

Suitability of Design 6

Data Store Configurations 6

Master Data Stores Design Styles 8

Data Integrity 11

Entity Integrity 12

Domain Integrity 12

Referential Integrity 12

User-Defined Integrity 12

Transactional Integrity 13

Transactional Faults 14

Isolation Levels 15

The Value of Null 15

Performance 16

Design 16

Set-Based Processing 16

Indexing 17

Partitioning 17

Caching 17

Availability 18

Redundancy 18

Recovery 19

Extensibility 19

Abstraction Layer 19

Generalization 20

Security 20

Restricted Access 20

Information Ownership 20

Audit Trails 21

Optimization Theory and SQL Server 21

Schema Design 21

Queries 22

Indexing 23

Concurrency 23

Advanced Scalability 23

Summary 24

Chapter 2: Relational Database Modeling 25

Modeling Reality 26

Visible Entities 27

Every Tuple (Row) Is an Island 27

Primary Keys 27

Tables, Rows, Columns 28

Identifying Multiple Entities 29

Multiple Objects 29

Relationships Between Objects 29

Organizing or Grouping Objects 30

Consistent Lookup Values 30

Complex Objects 30

Relational Patterns 30

Secondary Entities and Foreign Keys 31

Relationship Cardinality 31

Relationship Optionality 32

Data-Model Diagramming 33

One-to-Many Relationships 33

One-to-One Relationships 34

Supertype/Subtype Relationships 34

Many-to-Many Relationships 35

Category Entities 36

Reflexive Relationships 37

Normalization 38

Entity/Attribute Design Principles 39

Normal Forms 39

Simplicity and Normalization 40

First Normal Form (1NF) 40

The Second Normal Form (2NF) 41

The Third Normal Form (3NF) 42

The Boyce-Codd Normal Form (BCNF) 44

The Fourth Normal Form (4NF) 44

The Fifth Normal Form (5NF) 44

Relational Algebra 45

Summary 46

Chapter 3: Exploring SQL Server 2005 Architecture 47

Data Access Architectures 49

The Client/Server Database Model 49

N-Tier Design 51

Service-Oriented Architecture 52

SQL Server Services 52

Relational Engine 52

Transact-SQL 54

Visual Studio and the CLR 54

Service Broker 55

Replication Services 55

Full-Text Search 56

Notification Services 56

Server Management Objects 56

SQL Server Agent 56

Distributed Transaction Coordinator (DTC) 57

SQL Mail 57

Business Intelligence Services 57

Integration Services 57

Reporting Services 58

Analysis Services 59

SQL Server 2005 Editions 59

Enterprise (Developer) Edition 60

Standard Edition 60

Workgroup Edition 61

SQL Server Express Edition 61

SQL Server Everywhere Edition 62

SQL Server Tools and Components 62

SQL Server Management Studio 62

SQL Server Configuration Manager 63

Surface Area Configuration Tool 63

Business Intelligence Development Studio 64

SQL Integrated Help 64

SQL Profiler 65

Performance Monitor 65

Database Tuning Advisor 65

Command-Line Utilities: SQLCmd, Bulk Copy 65

SQL Server 2005 Feature Pack 66

AdventureWorks 66

Exploring the Metadata 66

System Databases 67

Metadata Views 67

Summary 68

Chapter 4: Installing SQL Server 2005 69

Planning Your Installation 69

Operating System 69

Security Accounts 69

Authentication Mode 72

Server Instances 73

Hardware Recommendations 74

Dedicated Server 74

Copious Memory 74

Using Multiple CPUs 74

Disk-Drive Subsystems 75

RAID Disk Subsystems 75

Network Performance 76

Performing the Installation 76

Attended Installations 76

Unattended Installations 78

Remote Installations 78

Installing in a Clustered Environment 79

Installing Multiple Instances 79

Testing the Installation 79

Upgrading from Previous Versions 80

SQL Server 2005 Upgrade Advisor Tool 80

Upgrading from SQL Server 2000 82

Migrating to SQL Server 82

Migration Assistant 82

Schema Conversion 83

Data Migration 83

Business Logic Conversion 83

Validation and Integration 85

Configuring the Surface Area of SQL Server 85

Surface Area Configuration Tool 86

Command Prompt Utilities 89

Removing SQL Server 89

Summary 90

Chapter 5: Client Software Connectivity 91

Enabling Server Connectivity 91

Server Configuration Manager 91

SQL Native Client Connectivity (SNAC) 92

SQL Server Native Client Features 93

Requirements 93

Database Mirroring 94

Asynchronous Operations 94

Multiple Active Result Sets (MARS) 95

XML Data Types 95

User-Defined Types 95

Large Value Types 95

Handling Expired Passwords 96

Snapshot Isolation 96

Summary 97

Chapter 6: Using Management Studio 99

Organizing the Interface 100

Window Placement 101

The Context Menu 102

The Summary Page 102

Registered Servers 103

Object Explorer 103

Navigating the Tree 104

Filtering Object Explorer 105

The Table Designer 106

Building Database Diagrams 107

The Query Designer 108

Using the Query Editor 110

Connecting to a Server 110

Opening a sql File 110

Executing SQL Batches 111

Shortcuts and Bookmarks 111

Viewing Query Execution Plans 112

Using the Solution Explorer 113

Introducing the Templates 114

Summary 114

Part II: Manipulating Data with Select 115

Chapter 7: Understanding Basic Query Flow 117

Understanding Query Flow 117

Syntactical Flow of the Query Statement 118

A Graphical View of the Query Statement 118

Logical Flow of the Query Statement 119

Physical Flow of the Query Statement 120

From Clause Data Sources 121

Possible Data Sources 121

Named Ranges 121

[Table Name] 122

Four-Part Table Names 122

Where Conditions 122

Using the Between Search Condition 123

Using the In Search Condition 126

Using the Like Search Condition 127

Multiple Where Conditions 129

Select Where 130

Ordering the Result Set 131

Specifying the Order by Using Column Names 132

Specifying the Order by Using Expressions 133

Specifying the Order by Using Column Aliases 134

Specifying the Order by Using Column Ordinal Position 134

Order by and Collation 134

Select Distinct 136

Ranking 138

Top 138

The With Ties Option 139

Summary 140

Chapter 8: Using Expressions and Scalar Functions 141

Building Expressions 141

Operators 143

Bitwise Operators 143

Case Expressions 146

Simple Case 146

Boolean Case 147

Working with Nulls 148

Testing for Null 148

Handling Nulls 149

Scalar Functions 154

User Information Functions 155

Data-Time Functions 155

String Functions 158

Soundex Functions 162

Data-Type Conversion Functions 165

Server Environment Information 168

Summary 169

Chapter 9: Merging Data with Joins and Unions 171

Using Joins 172

Inner Joins 174

Creating Inner Joins within SQL Code 175

Number of Rows Returned 176

ANSI SQL 89 Joins 178

Multiple Table Joins 178

Outer Joins 180

Outer Joins and Optional Foreign Keys 182

Full Outer Joins 183

Placing the Conditions within Outer Joins 185

Self-Joins 187

Cross (Unrestricted) Joins 189

Exotic Joins 190

Θ(theta) Joins 190

Multiple-Condition Joins 191

Non-Key Joins 191

Set Difference 192

Using Unions 195

Intersection Union 197

Difference Union/Except 197

Summary 198

Chapter 10: Including Data with Subqueries and CTEs 199

Methods and Locations 199

Simple Subqueries 201

Common Table Expressions 202

Using Scalar Subqueries 203

Using Subqueries as Lists 205

Using Subqueries as Tables 209

Correlated Subqueries 211

Relational Division 215

Relational Division with a Remainder 216

Exact Relational Division 218

Summary 220

Chapter 11: Aggregating Data 221

Simple Aggregations 221

Basic Aggregations 222

Beginning Statistics 224

Grouping within a Result Set 225

Simple Groupings 227

Aggravating Queries 228

Generating Totals 232

Rollup Subtotals 232

Cube Queries 233

Computing Aggregates 234

Building Crosstab Queries 236

Fixed-Column Crosstab Queries 237

Dynamic Crosstab Queries 240

Summary 242

Chapter 12: Navigating Hierarchical Data 243

Adjacency List Schema Patterns 243

Basic Adjacency List Pattern 244

Adjacency List Variations 245

Navigating the Adjacency List 248

Using a Standard select Statement 248

Using a Recursive Cursor 249

Using a Set-Based Solution 251

Using a User-Defined Function 252

Using Recursive Common Table Expressions 254

Summary 255

Chapter 13: Using Full-Text Search 257

Configuring Full-Text Search Catalogs 258

Creating a Catalog with the Wizard 259

Creating a Catalog with T-SQL Code 260

Pushing Data to the Full-Text Index 261

Maintaining a Catalog with Management Studio 262

Maintaining a Catalog in T-SQL Code 263

Noise Files 264

Word Searches 265

The Contains Function 265

ContainsTable 266

Advanced Search Options 267

Multiple Word Searches 267

Searches with Wildcards 269

Phrase Searches 269

Word-Proximity Searches 270

Word-Inflection Searches 270

Thesaurus Searches 271

Variable-Word-Weight Searches 272

Fuzzy Searches 273

Freetext 274

FreetextTable 274

Binary Object Indexing 275

Performance 277

Summary 277

Chapter 14: Creating Views 279

Why Use Views? 279

Working with Views 281

Creating Views Using Management Studio 281

Creating Views with DDL Code 283

Order By and Views 284

View Restrictions 284

Executing Views 285

Locking Down the View 285

Protecting the Data 285

Protecting the View 287

Updating through Views 288

Nesting Views 290

Using Synonyms 292Summary 293

Chapter 15: Working with Distributed Queries 295

Distributed Query Concepts 295Accessing a Local SQL Server Database 297Linking to External Data Sources 298Linking with Management Studio 298Linking with T-SQL 300Linking with Non–SQL Server Data Sources 303Developing Distributed Queries 305Distributed Queries and Management Studio 306Distributed Views 306Local-Distributed Queries 306Pass-Through Distributed Queries 310Distributed Transactions 312Distributed Transaction Coordinator 312Developing Distributed Transactions 313Monitoring Distributed Transactions 314Summary 315

Chapter 16: Modifying Data 317

Inserting Data 318Inserting One Row of Values 319Inserting a Result Set from Select 321Inserting the Result Set from a Stored Procedure 322Creating a Default Row 324Creating a Table While Inserting Data 325Updating Data 327Updating a Single Table 328Performing Global Search and Replaces 329Referencing Multiple Tables While Updating Data 329Deleting Data 333Referencing Multiple Tables While Deleting 333Cascading Deletes 334Alternatives to Physically Deleting Data 335Returning Modified Data 336Returning Data from an Insert 336Returning Data from an Update 337Returning Data from a Delete 337Returning Data into a @Table Variable 337Potential Data-Modification Obstacles 338Data Type/Length Obstacles 339Primary Key Obstacles 339Foreign Key Obstacles 343Unique Index Obstacles 344Null and Default Obstacles 345Check Constraint Obstacles 345Instead of Trigger Obstacles 346After Trigger Obstacles 347Calculated Columns 348Non-Updateable View Obstacles 349

Views with the with check option Obstacles 349Security Obstacles 350Summary 351

Chapter 17: Implementing the Physical Database Schema 355

Designing the Physical Database Schema 356Physical Schema Design Options 356Refining the Data Patterns 357Designing for Performance 357Designing for Extensibility 358Responsible Denormalization 358Creating the Database 359The Create DDL Command 359Database-File Concepts 361Configuring File Growth 362Using Multiple Files 363Planning Multiple Filegroups 366Creating Tables 367Designing Tables Using Management Studio 367Working with SQL Scripts 369Schemas 370Table and Column Names 371Filegroups 372Creating Primary Keys 372Primary Keys 373Creating Foreign Keys 377Creating User-Data Columns 381Column Data Types 382Calculated Columns 385Column Constraints and Defaults 385Data Catalog 390DDL Triggers 391Creating and Altering DDL Triggers 392EventData() 393Enabling and Disabling DDL Triggers 394Summary 395

Chapter 18: Programming with Transact-SQL 397

Transact-SQL Fundamentals 398T-SQL Batches 398T-SQL Formatting 399Debugging T-SQL 401Variables 401Variable Default and Scope 402Using the Set and Select Commands 402Conditional Select 403Using Variables within SQL Queries 404Multiple Assignment Variables 404

Procedural Flow 405

If 405While 406Goto 407Examining SQL Server with Code 408sp_help 408Global Variables 409Temporary Tables and Table Variables 411Local Temporary Tables 411Global Temporary Tables 412Table Variables 412Dynamic SQL 413Executing Dynamic SQL 413sp_excecuteSQL 414Developing Dynamic SQL Code 414Error Handling 416Try Catch 416Legacy @@Error Global Variable 418

@@RowCount Global Variable 419Raiserror 420Catch Block 424T-SQL Fatal Errors 424Summary 425

Chapter 19: Performing Bulk Operations 427

Bulk Insert 428Bulk Insert Options 429BCP 430Summary 430

Chapter 20: Kill the Cursor! 431

Anatomy of a Cursor 431The Five Steps to Cursoring 432Managing the Cursor 432Update Cursors 434Cursor Scope 434Cursors and Transactions 435Cursor Strategies 435Complex-Logic Solutions 436The Logic Code 437SQL-92 Cursor with Stored Procedure 439Fast-Forward Cursor with Stored Procedure 439Fast-Forward Cursor and User-Defined Function 440Update Cursor with Stored Procedure 441Update Query with User-Defined Function 441Multiple Queries 442Query with Case Expression 443Performance Analysis 444Denormalizing a List Example 445Summary 447

Chapter 21: Developing Stored Procedures 449

Managing Stored Procedures 450Create, Alter, and Drop 450Returning a Record Set 451Compiling Stored Procedures 451Stored Procedure Encryption 452System Stored Procedures 453Passing Data to Stored Procedures 453Input Parameters 453Parameter Defaults 454Returning Data from Stored Procedures 455Output Parameters 455Using the Return Command 457Path and Scope of Returning Data 457Using Stored Procedures within Queries 458Executing Remote Stored Procedures 459The Complete Stored Procedure 460The pGetPrice Stored Procedure 460The pOrder_AddNew Stored Procedure 462The pOrder_AddItem Stored Procedure 466Adding an Order 468Summary 469

Chapter 22: Building User-Defined Functions 471

Scalar Functions 472Creating a Scalar Function 473Calling a Scalar Function 475Creating Functions with Schema Binding 475Inline Table-Valued Functions 476Creating an Inline Table-Valued Function 476Calling an Inline Table-Valued Function 476Using Parameters 477Correlated User Defined Functions 478Multi-Statement Table-Valued Functions 480Creating a Multi-Statement Table-Valued Function 480Calling the Function 481Summary 481

Chapter 23: Implementing Triggers 483

Trigger Basics 483Transaction Flow 484Creating Triggers 485After Triggers 486Instead of Triggers 487Trigger Limitations 488Disabling Triggers 488Listing Triggers 489Triggers and Security 489Working with the Transaction 490Determining the Updated Columns 490Inserted and Deleted Logical Tables 491Developing Multi-Row Enabled–Triggers 492

Multiple-Trigger Interaction 494Trigger Organization 494Nested Triggers 495Recursive Triggers 495Instead of and After Triggers 497Multiple after Triggers 497Summary 498

Chapter 24: Exploring Advanced T-SQL Solutions 499

Complex Business Rule Validation 500Complex Referential Integrity 501Row-Level Custom Security 503The Security Table 504The Security-Check Stored Procedure 511The Security-Check Function 512Using the NT Login 513The Security-Check Trigger 515Auditing Data Changes 516The Audit Table 516The Fixed Audit Trail Trigger 517Rolling Back from the Audit Trail 520Auditing Complications 521The Dynamic Audit-Trail Trigger and Procedure 522Transaction-Aggregation Handling 528The Inventory-Transaction Trigger 529The Inventory Trigger 531Logically Deleting Data 532Logical Delete Triggers 533Undeleting a Logically Deleted Row 534Filtering Out Logically Deleted Rows 535Cascading Logical Deletes 535Degrees of Inactivity 536Archiving Data 536Summary 537

Chapter 25: Creating Extensibility with a Data Abstraction Layer 539

The AddNew Stored Procedure 540The Fetch Stored Procedure 542The Update Stored Procedure 543Update with RowVersion 543Minimal-Update 545The Delete Stored Procedure 547Summary 548

Chapter 26: Developing for SQL Server Everywhere 549

An Overview of SQL Server 2005 Everywhere Edition 550History 550Concepts 551What’s New in SQL Server 2005 Everywhere Edition 555Getting Started with SQL Everywhere 556Installing SQL Everywhere 556Query Analyzer 3.0 559

Creating a SQL Everywhere Database 562Upgrading an Existing SQL CE 2.0 Database 575Synchronizing Data 576Remote Data Access 577Merge Replication 578Web Services 579Packaging and Deployment 580Creating the Database Programmatically 581Deploying a Starter Database with the Mobile Application 581Creating the Database Dynamically through Merge Replication 581Other Approaches 581Security 581Password Protection 582Encryption 582Secure Data Synchronization 582Tuning, Maintenance, and Administration 583Measuring and Improving Query Performance 583SQL Everywhere Maintenance 584Repairing a Damaged SQL Everywhere Database 585Keeping Merge Replication Healthy 586More Information 587Summary 587

Chapter 27: Programming CLR Assemblies within SQL Server 589

.NET Framework Crash Course 590Assemblies 591App Domains 592Overview of the CLR SQL Server Types 595CLR Integration NET Type Attributes 596Shared Features of SQL Server CLR Types 597Data Types 598.NET Methods That Support CLR Integration 601T-SQL CLR DDL Commands and Catalog Views 602Assemblies 603Database Objects 603Building Database Types with Visual Studio 2005 604Creating a CLR Project 604Coding a CLR Stored Procedure 608CLR Functions 611Using the CLR versus Using T-SQL 613T-SQL Is Not Going Away 614Summary 615

Chapter 28: Queueing Data with Service Broker 617

Configuring a Message Queue 617Working with Dialogs 618Sending a Message to the Queue 619Receiving a Message 619Monitoring Service Broker 621Summary 622

Chapter 29: Persisting Custom Data Types 623

Creating CLR Integration User-Defined Types 625Satisfying the Requirements 626Coding a CLR UDT with Visual Studio 628Testing and Debugging the UDT 635Performance Considerations 635CLR Integration UDT Deployment Details 637Strongly Named Assemblies and the GAC 637Creating Strongly Named NET Assemblies 638Maintaining the UDT 640Summary 640

Chapter 30: Programming with ADO.NET 2.0 643

An Overview of ADO.NET 644ADO 645The ADO Object Model 649ADO.NET 658ADO.NET in Visual Studio 2005 668Server Explorer 669Debugging ADO.NET 669Application Tracing 670Application Building Basics 671Connecting to SQL Server 671Data Adapters 672DataReaders and Recordsets 672Streams 673Asynchronous Execution 673Using a Single Database Value 674Data Modification 674Binding to Controls 675Summary 675

Chapter 31: Using XML, XPath, and XQuery 677

XML Data Type 678Casting and Converting 678XML Type Limitations 678XML Schema Collections 679XML Indexes 680Querying XML Data 680XPATH 680FLWOR Queries 681Merging XQuery with Select 681Decomposing XML SQL Server 682Reading XML into SQL Server 683Creating XML with SQL Server 2005 685Summary 687

Chapter 32: Building an SOA Data Store with Web Services 689

HTTP Listening 690About HTTP.sys 690Implicit Endpoints 691Explicit Endpoints 691WSDL 692Endpoint Security 693Summary 693

Chapter 33: InfoPath and SQL Server 2005 695

Overview of InfoPath 2003 695Automatic Data Validation 696Offline Form Completion 696Conditional Formatting 696Security in InfoPath 2003 697InfoPath Object Model 698Scripting and NET Code 698Note 698Creating a Form Template 698Other Features of InfoPath 702Summary 702

Chapter 34: Configuring SQL Server 705

Setting the Options 705Configuring the Server 705Configuring the Database 708Configuring the Connection 709Configuration Options 711Displaying the Advanced Options 711Start/Stop Configuration Properties 712Memory-Configuration Properties 714Processor-Configuration Properties 720Security-Configuration Properties 723Connection-Configuration Properties 725Server-Configuration Properties 728Index-Configuration Properties 730Configuring Database Auto Options 731Cursor-Configuration Properties 733SQL ANSI–Configuration Properties 734Trigger Configuration Properties 737Database-State-Configuration Properties 737Recovery-Configuration Properties 739Summary 740

Chapter 35: Transferring Databases 741

Copy Database Wizard 742Working with SQL Script 746Detaching and Attaching 748Summary 750

Chapter 36: Recovery Planning 751

Recovery Concepts 752Recovery Models 753Simple Recovery Model 754The Full Recovery Model 754Bulk-Logged Recovery Model 756Setting the Recovery Model 757Modifying Recovery Models 757Backing Up the Database 758Backup Destination 758Backup Rotation 758Performing Backup with Management Studio 759Backing Up the Database with Code 760Verifying the Backup with Code 762Working with the Transaction Log 762Inside the Transaction Log 762Backing Up the Transaction Log 764Truncating the Log 764The Transaction Log and Simple Recovery Model 765Recovery Operations 765Detecting the Problem 765Recovery Sequences 766Performing the Restore with Management Studio 767Restoring with T-SQL Code 769System Databases Recovery 772Master Database 773MSDB System Database 773Performing a Complete Recovery 774Summary 774

Chapter 37: Maintaining the Database 775

DBCC Commands 775Database Integrity 776Index Maintenance 780Database File Size 784Miscellaneous DBCC Commands 787Managing Database Maintenance 788Planning Database Maintenance 788Maintenance Plan Wizard 788Command-Line Maintenance 793Monitoring Database Maintenance 794Summary 794

Chapter 38: Automating Database Maintenance with SQL Server Agent 795

Setting Up SQL Server Agent 795Understanding Alerts, Operators, and Jobs 799

Managing Operators 799Managing Alerts 800Creating User-Defined Errors 801Creating an Alert 801Managing Jobs 804Creating a Job Category 805Creating a Job Definition 806Setting Up the Job Steps 807Configuring a Job Schedule 810Handling Completion-, Success-, and Failure-Notification Messages 810Summary 811

Chapter 39: Replicating Data 813

Why Replicate Data? 813Fault Tolerance/Disaster Recovery 813Application Requirements 815Performance Gains 815Data Distribution 815Comparing Options for Distributing Data 816The Microsoft Model 817Publisher 818Subscriber 818Distributor 818Central Publisher 818Central Subscriber 818Republishing 819Peer-to-Peer 819Article 819Push Subscriptions 820Pull Subscriptions 820Replication Types 820Snapshot Replication 821Snapshot Replication with Immediate Updating 821Snapshot Replication with Queued Updating 822Snapshot Replication with Immediate Updating and Queued Failover 822Transactional Replication 822Snapshot Agent 823Log Reader Agent 823Distribution Agent 823Peer-to-Peer Replication 824Bi-directional Transactional Replication 824Transactional Replication with Immediate Updating 824Transactional Replication with Queued Updating 824Transactional Replication with Immediate Updating and Queued Failover 825Transactional Replication over the Internet 825Merge Replication 825Merge Replication and SQL CE or SQL Mobile Subscribers 827Merge Replication over the Internet 828New in SQL 2005 Replication 828Restartable Snapshots 828Oracle Publishing 828Very Tight Security 828Peer-to-Peer Replication Model 828

Replicating All DDL 829Replicating Full-Text Indexes 829Allowing Anonymous Subscriptions for All Publications 829Logical Records in Merge Replication 829Precomputed Partitions 829Updates to Unique Keys 830Custom Conflict Handling through SQL RMO 830Numerous Performance Improvements 830Latency Tokens 830Transactional Parallelism 830Download Only Articles 830Replication Monitor 831Merge Replication over HTTPS 832Merge Replication Performance and Scalability Improvements 832SQL RMO 832Simplified Wizards 833Initializing a Subscriber 833Configuring Replication 834Using a Local Distributor 834Using a Remote Distributor 834Creating Snapshot Replication Publications 835Creating Transactional Replication Publications 837Creating Bi-directional Transactional Replication Publications 841Creating Oracle Publications 842Creating Peer-to-Peer Replication Publications 843Creating Merge Replication Publications 844Creating Subscriptions 846Creating Web Synchronizing Subscriptions 848Monitoring Your Replication Solution 852Agent Profiles 852All Subscriptions 853Warnings and Agents 853Modifying Replication Agent Properties 854Tracer Tokens 854Replication Performance 855Replication Troubleshooting 856Summary 856

Chapter 40: Securing Databases 857

Security Concepts 858Server-Level Security 859Database-Level Security 859Object Ownership 859Windows Security 860Windows Security 860SQL Server Login 860Server Security 861SQL Server Authentication Mode 861Windows Authentication 862SQL Server Logins 866Server Roles 867Database Security 869Guest Logins 869Granting Access to the Database 869

Fixed Database Roles 871Securables Permissions 872Application Roles 873Object Security 873Object Permissions 874Standard Database Roles 875Object Security and Management Studio 877

A Sample Security Model Example 878C2-Level Security 880Views and Security 880Cryptography 881Introduction to Cryptography 881The SQL Server-Crypto Hierarchy 882Encrypting with a Passphrase 882Encrypting with a Symmetric Key 884Preventing SQL Injection 886Appending Malicious Code 886

Or 1=1 886Password? What Password? 887Prevention 887Summary 887

Chapter 41: Administering SQL Server Express 889

Installing SQL Server Express 890Using Command-Line Parameters to Install SSE Silently 891Using INI Files to Install SSE 894SQL Server Management Studio Express Edition 895Summary 896

Chapter 42: ETL with Integration Services 899

Design Environment 900Connection Managers 902Variables 902Configuring Elements 904Event Handlers 907Executing a Package in Development 908Integration Services Package Elements 909Connection Managers 909Control Flow Elements 912Data Flow Components 918Maintainable and Manageable Packages 928Logging 929Package Configurations 930Checkpoint Restart 931Deploying Packages 932Installing Packages 932Executing Packages 932Summary 933

Chapter 43: Business Intelligence with Analysis Services 935

Data Warehousing 936Star Schema 936Consistency 937Loading Data 938Analysis Services Quick Start 939Analysis Services Architecture 940Unified Dimensional Model 941Server 941Client 942Building a Database 942Business Intelligence Development Studio 942Data Sources 943Data Source View 944Creating a Cube 947Dimensions 948Dimension Designer 948Changing Data in Dimensions 954Beyond Regular Dimensions 955Dimension Refinements 957Cubes 959Cube Structure 960Dimension Usage 962KPIs 963Actions 964Partitions 964Perspectives 966Data Storage 967SQL Server Notifications 968Client-Initiated Notifications 968Scheduled Polling Notifications 968Data Integrity 969Null Processing 969Unknown Member 970Error Configuration 971Summary 972

Chapter 44: Data Mining with Analysis Services 973

The Data Mining Process 974Modeling with Analysis Services 975Algorithms 982Decision Trees 982Linear Regression 983Clustering 983Sequence Clustering 984Neural Network 985Logistic Regression 986Naive Bayes 986Association Rules 986Time Series 987OLAP Integration 987Summary 988

Chapter 45: Programming MDX Queries 989

Basic Select Query 990Cube Addressing 990Dimension Structure 991Basic SELECT Statement 992Advanced Select Query 996Subcubes 997WITH Clause 997Dimension Considerations 1001MDX Scripting 1002Calculated Members and Named Sets 1003Adding Business Intelligence 1003Summary 1004

Chapter 46: Authoring Reports with Reporting Services 1005

Anatomy of a Report 1005Report Definition Language (RDL) 1005Data Sources 1006Reporting Services Data Sets 1008Query Parameters and Report Parameters 1008Report Content and Layout 1009The Report Authoring Process 1010Creating a Reporting Services Project in Visual Studio 2005 1010Steps to Create a Report 1011Using the Report Wizard to Create Reports 1011Authoring a Report from Scratch 1011Working with Data 1014Working with SQL in the Report Designer 1014Using Query Parameters to Select and Filter Data 1015Adding Calculated Fields to a Data Set 1018Working with XML Data Sources 1019Working with Expressions 1020Designing the Report Layout 1023Design Basics 1023Using the Table and Matrix Property Pages 1026Grouping and Sorting Data in a Table and Matrix 1027Illustrating Data with Charts 1030Adding a Pie Chart to a Report 1032Summary 1032

Chapter 47: Administering Reports with Reporting Services 1035

Deploying Reporting Services Reports 1036Deploying Reports Using Visual Studio 2005 1036Deploying Reports Using the Report Manager 1038Deploying Reports Programmatically Using the Reporting

Services Web Service 1039Configuring Reporting Services Using the Report Manager 1039Configuring Reporting Services Site Settings 1039Administering Security 1041Working with Linked Reports 1044Creating Linked Reports 1044

Leveraging the Power of Subscriptions 1045Creating a Data-Driven Subscription 1046Summary 1049

Chapter 48: Analyzing Data with Excel and Data Analyzer 1051

Excel PivotTables 1052Multidimensional Connections 1053Relational Connection 1054PivotTable Construction 1056PivotChart Construction 1058Excel Data Ranges 1059Data Analyzer 1060Creating a View 1060Formatting a View 1061Summary 1062

Chapter 49: Measuring Performance 1065

Measuring Accuracy 1065Using Performance Monitor 1066System Monitor 1066Counter Logs 1069Using SQL Server Profiler 1071Defining a New Trace 1071Selecting Events 1072Filtering Events 1073Organizing Columns 1073Using the Profiler Trace 1073Integrating Performance Monitor Data 1074Using SQL Trace 1075Using Transact-SQL 1076Using Dynamic Management Views 1076Using GetDate() 1076Using Statistics 1077The Database Performance KPI 1077Repeatable Performance Testing 1078Production Performance Data 1079Scalability Performance Load Testing 1080Summary 1080

Chapter 50: Query Analysis and Index Tuning 1081

A Holistic Approach to Index Tuning 1081Indexing 1082Index Basics 1082Creating Indexes 1084Index Options 1087Creating Base Indexes 1090Query Analysis 1090Viewing Query Execution Plans 1091Using the Showplans 1092Interpreting the Query Execution Plan 1092

Index Tuning 1095Missing Indexes 1095Bookmark Lookup 1097Optimizable SARGs 1098Index Selectivity 1098Reusing Query Execution Plans 1099

A Comprehensive Index Strategy 1100Using the Database Engine Tuning Advisor 1102Summary 1103

Chapter 51: Managing Transactions, Locking, and Blocking 1105

Transactional Basics 1106Transactional Integrity 1107The ACID Properties 1107Transactional Faults 1108Isolation Levels 1113Transaction-Log Architecture 1115Transaction Log Sequence 1115Transaction-Log Recovery 1119Understanding SQL Server Locking 1119Lock Granularity 1120Lock Mode 1120Lock Duration 1123Monitoring Locking and Blocking 1123Using Profiler 1124Controlling SQL Server Locking 1126Setting the Connection Isolation Level 1126Using Database Snapshot Isolation 1127Using Locking Hints 1129Index-Level Locking Restrictions 1130Controlling Lock Timeouts 1131Evaluating Database Concurrency Performance 1131Application Locks 1131Deadlocks 1133Creating a Deadlock 1133Automatic Deadlock Detection 1135Handling Deadlocks 1136Minimizing Deadlocks 1136Application Locking Design 1136Implementing Optimistic Locking 1136Lost Updates 1137Transaction Performance Strategies 1139Summary 1140

Chapter 52: Providing High Availability 1141

Availability Testing 1142Warm-Standby Availability 1142Log Shipping 1143Shipping the Users 1146Returning to the Original Primary Server 1146Failover Servers and Clustering 1147Failover SQL Server Installs 1147Configuring 1148

Database Mirroring 1149Prerequisites 1150Configuring 1150Architecting the Environment 1151Summary 1152

Chapter 53: Scaling Very Large Databases 1153

Optimization Theory and Scalability 1154Scaling Out the Platform 1154Scaling Up the Solution 1156Partitioned Tables and Indexes 1157Creating the Partition Function 1158Creating Partition Schemes 1159Creating the Partition Table 1160Querying Partition Tables 1162Altering Partition Tables 1164Switching Tables 1164Rolling Partitions 1167Indexing Partitioned Tables 1167Removing Partitioning 1168Working with Indexed Views 1168Indexed Views and Queries 1170Updating Indexed Views 1171Summary 1171

Chapter 54: Designing High-Performance Data Access Providers 1173

Data Access Concepts 1174Why Good Data Access Matters 1174Defining Requirements 1174Data Access Objects 1175How the DAO Scenario Works 1175The Cons of the DAO Pattern 1178Factories 1179The Pros of the Factory Pattern 1182The Cons of the Factory Pattern 1182Data Providers 1182How a Data Provider Works 1185Pros of the Data Provider Pattern 1194Cons of the Data Provider Pattern 1194Summary 1194

Appendix A: SQL Server 2005 Specifications 1199 Appendix B: Sample Databases 1205

Index 1213

Ifirst meet Paul a few PASS Summits ago and we now serve together on the PASS Board ofDirectors We share a passion for the SQL Server community and that’s what I want to writeabout Paul and I want to extend an invitation to you to visit your local PASS chapter, attend aPASS camp, and join us at the annual Global Summit, my favorite week of the year I can guar-antee that if you get involved with PASS you’ll get more out of it than you put in This is agreat community with plenty of good will and a common desire to help each other grow ourskills Here you’ll meet MVPs and authors, fellow DBAs and database developers, theMicrosoft SQL Server development team, and friends So come, get involved, speak up, andlet your voice be heard

Wayne Synder, PASS Director of Vendor ProgramsManaging Consultant, Mariner