OReilly SQL cookbook dec 2005 ISBN 0596009763

Considerthat both complex and simple solutions are provided, and thatsolutions for five different vendors are available when a common solution does not exist.. The non-SQL programmer Per

By Anthony Molinaro

Publisher: O'Reilly Pub Date: December 2005 Print ISBN-10: 0-596-00976-3 Print ISBN-13: 978-0-59-600976-2 Pages: 628

Table of Contents | Index

You know the rudiments of the SQL query language, yet you feel you aren't taking full advantage of SQL's expressive power You'd like to learn how to do more work with SQL inside the database before pushing data across the network to your applications You'd like

to take your SQL skills to the next level.

Let's face it, SQL is a deceptively simple language to learn, and many database developers

never go far beyond the simple statement: SELECT FROM WHERE But there is so much more you can do with the language In the SQL Cookbook, experienced SQL developer

Anthony Molinaro shares his favorite SQL techniques and features You'll learn about:

Window functions, arguably the most significant enhancement to SQL in the past decade If you're not using these, you're missing out

Powerful, database-specific features such as SQL Server's PIVOT and UNPIVOT operators, Oracle's MODEL clause, and PostgreSQL's very useful GENERATE_SERIES function

Pivoting rows into columns, reverse-pivoting columns into rows, using pivoting to facilitate inter-row calculations, and double-pivoting a result set

to please Anthony's credo is: "When it comes down to it, we all go to work, we all have bills to pay, and we all want to go home at a reasonable time and enjoy what's still

available of our days." The SQL Cookbook moves quickly from problem to solution, saving

you time each step of the way.

By Anthony Molinaro

Publisher: O'Reilly Pub Date: December 2005 Print ISBN-10: 0-596-00976-3 Print ISBN-13: 978-0-59-600976-2 Pages: 628

Printed in the United States of America

Published by O'Reilly Media, Inc., 1005 Gravenstein HighwayNorth, Sebastopol, CA 95472

O'Reilly books may be purchased for educational, business, orsales promotional use Online editions are also available for

Many of the designations used by manufacturers and sellers to

While every precaution has been taken in the preparation of thisbook, the publisher and author assume no responsibility for

errors or omissions, or for damages resulting from the use ofthe information contained herein

ISBN: 0-596-00976-3

[M]

To my mom:

You're the best! Thank you for everything.

SQL is the language in the database world If you're developing

for or reporting from relational databases, your ability to putdata into a database and then get it back out again ultimatelycomes down to your knowledge of SQL Yet many practitionersuse SQL in a perfunctory manner, and are unaware of the power

at their disposal This book aims to change all that, by openingyour eyes to what SQL can really do for you

The book you're holding in your hands is a cookbook It's a

collection of common SQL problems and their solutions that Ihope you'll find helpful in your day-to-day work Recipes arecategorized into chapters of related topics When faced with anew SQL problem that you haven't solved before, find the

chapter that best seems to apply, skim through the recipe titles,and hopefully you will find a solution, or at least inspiration for

a solution

More than 150 recipes are available in this 600-plus page book,and I've only scratched the surface of what can be done usingSQL The number of different SQL solutions available for solvingour daily programming problems is eclipsed only by the number

of problems we need to solve You won't find all possible

problems covered in this book Indeed, such coverage would beimpossible You will, however, find many common problems andtheir solutions And in those solutions lie techniques that you'lllearn how to expand upon and apply to other, new problemsthat I never thought to cover

worthy SQL recipes If you come across a good or clever SQL solution

My publisher and I are constantly on the lookout for new, cookbook-to a problem, consider sharing it; consider sending it in for inclusion in the next edition of this book See "Comments and Questions" for our contact information.

Queries, queries, queries My goal from the beginning of thisproject has not been so much to write a "SQL Cookbook" as towrite a "Query Cookbook." I've aimed to create a book

comprised of queries ranging from the relatively easy to therelatively difficult in hopes the reader will grasp the techniquesbehind those queries and use them to solve his own particularbusiness problems I hope to pass on many of the SQL

programming techniques I've used in my career so that you,the reader, will take them, learn from them, and eventuallyimprove upon them; through this cycle we all benefit Beingable to retrieve data from a database seems so simple, yet inthe world of Information Technology (IT) it's crucial that theoperation of data retrieval be done as efficiently as possible.Techniques for efficient data retrieval should be shared so that

we can all be efficient and help each other improve

Consider for a moment the outstanding contribution to

mathematics by Georg Cantor, who was the first to realize thevast benefit of studying sets of elements (studying the set itselfrather than its constituents) At first, Cantor's work wasn't

accepted by many of his peers In time, though, it was not onlyaccepted, but set theory is now considered the foundation ofmathematics! More importantly, however, it was not throughCantor's work alone that set theory became what it is today;rather, by sharing his ideas, others such as Ernst Zermelo,

Gottlob Frege, Abraham Fraenkel, Thoralf Skolem, Kurt Gödel,and John von Neumann developed and improved the theory.Such sharing not only provided everyone with a better

understanding of the theory, it made for a better set theorythan was first conceived

Ultimately, the goal of this book is to give you, the reader, aglimpse of what can be done using SQL outside of what is

considered the typical SQL problem domain SQL has come avery long way in the last ten years Problems typically solvedusing a procedural language such as C or JAVA can now be

solved directly in SQL, but many developers are simply unaware

of this fact This book is to help make you aware

Now, before you take what I just said the wrong way, let mestate that I am a firm believer in, "If it ain't broke, don't fix it."For example, let's say you have a particular business problem

to solve, and you currently use SQL to simply retrieve your datawhile applying your complex business logic using a languageother than SQL If your code works and performance is

acceptable, then great I am in no way suggesting that you

scrap your code for a SQL-only solution; I only ask that youopen your mind and realize that the SQL you programmed with

in 1995 is not the same SQL being used in 2005 Today's SQLcan do so much more

This text is unique in that the target audience is wide, but thequality of the material presented is not compromised Considerthat both complex and simple solutions are provided, and thatsolutions for five different vendors are available when a

common solution does not exist The target audience is indeedwide:

The SQL novice

Perhaps you have just purchased a text on learning SQL, oryou are fresh into your first semester of a required

database course and you want to supplement your new

knowledge with some challenging real world examples

Maybe you've seen a query that magically transforms rows

to columns, or that parses a serialized string into a resultset The recipes in this book explain techniques for

performing these seemingly impossible queries

The non-SQL programmer

Perhaps your background is in another language and you'vebeen thrown into the fire at your current job and are

expected to support complex SQL written by someone else.The recipes shown in this book, particularly in the later

chapters, break down complex queries and provide a gentlewalk-through to help you understand complex code that youmay have inherited

The SQL journeyman

to an answer in a single query! Allow me to again state that

I have no intention of trying to force-feed my ideas to analready experienced practitioner Instead, consider this book

as a way to update your skill set if you haven't caught on tosome of the newer additions to the SQL language

The SQL expert

Undoubtedly you've seen these recipes before, and you

probably have your own variations Why, then, is this bookuseful to you? Perhaps you've been a SQL expert on oneplatform your whole career, say, SQL Server, and now wish

to learn Oracle Perhaps you've only ever used MySQL, andyou wonder what the same solutions in PostgreSQL wouldlook like This text covers different relational database

management systems (RDBMSs) and displays their

solutions side by side Here's your chance to expand yourknowledge base

Be sure to read this preface thoroughly It contains necessarybackground and other information that you might otherwisemiss if you dive into individual recipes The section on "Platformand Version" tells you what RDBMSs this book covers Pay

special attention to "Tables Used in This Book," so that you

become familiar with the example tables used in most of therecipes You'll also find important coding and font conventions

in "Conventions Used in This Book." All these sections comelater in this preface

Use common sense! Realize that it is impossible to write abook that provides a solution to every possible business

techniques required to solve your own specific problems Ifyou find yourself saying, "Great, this recipe works for thisparticular data set, but mine is different and thus the recipedoesn't work quite correctly," that's expected In that case,try to find commonality between the data in the book andyour data Break down the book's query to its simplest formand add complexity as you go All queries start with

SELECT…FROM…, so in their simplest form, all queries arethe same If you add complexity as you go, "building" a

query one step, one function, one join at a time, you willnot only understand how those constructs change the resultset, but you will see how the recipe is different from whatyou actually need And from there you can modify the

Don't be afraid to experiment Be creative! Feel free to usetechniques different from what I've used I make it a point

to use many of the functions supplied by the different

vendors in this book, and often there are several other

functions that may work as well as the one I've chosen touse in a particular recipe Feel free to plug your own

variations into the recipes of this book

Newer does not always mean better If you're not using

mean your code is not as efficient as it can be There aremany cases in which traditional SQL solutions are as good

or better than any new solution Please keep this in mind,particularly in the Appendix B, Rozenshtein Revisited After

reading this book, you should not come away with the ideathat you need to update or change all your existing code.Instead, only realize there are many new and extremelyefficient features of SQL available now that were not

Program defensively when necessary In an effort to makethe queries in this book as terse as humanly possible

without obscuring their meaning, I've removed many

"defensive measures" from the recipes For example,

consider a query computing a running total for a number ofemployee salaries It could be the case that you have

declared the column of type VARCHAR and are (sadly)

storing a mix of numeric and string data in one field You'llfind the running total recipe in this book does not check forsuch a case (and it will fail as the function SUM doesn't

know what to do with character data), so if you have thistype of "data" ("problem" is a more accurate description),you will need to code around it or (hopefully) fix your data,

design practices as the mixing of character and numericdata in the same column The idea is to focus on the

technique; once you understand the technique,

sidestepping such problems is trivial

Repetition is the key The best way to master the recipes inthis book is to sit down and code them When it comes tocode, reading is fine, but actually coding is even better Youmust read to understand why things are done a certain way,but only by coding will you be able to create these queriesyourself

Be advised that many of the examples in this book are

contrived The problems are not contrived They are real

However, I've built all examples around a small set of tablescontaining employee data I've done that to help you get

familiar with the example data, so that, having become familiarwith the data, you can focus on the technique that each recipeillustrates You might look at a specific problem and think: "Iwould never need to do that with employee data." But try tolook past the example data in those cases and focus on the

technique that I'm illustrating The techniques are useful Mycolleagues and I use them daily We think you will too

Due to constraints on time and book size, it isn't possible for asingle book to provide solutions for all the possible SQL

Chapter 4, however, does provide recipes for common

problems involving the insertion, updating, and deleting ofdata

XML

It is my strong opinion that XML recipes do not belong in abook on SQL Storing XML documents in relational

databases is becoming increasingly popular, and each

RDBMS has their own extensions and tools for retrievingand manipulating such data XML manipulation often

involves code that is procedural and thus outside the scope

of this book Recent developments such as XQUERY

represent completely separate topics from SQL and belong

in their own book (or books)

Until a language more suitable for dealing with objects

comes along, I am strongly against using object-orientedfeatures and designs in relational databases At the presenttime, the object-oriented features available from some

These sort of theoretical discussions have their place, butnot in a book centered on delivering SQL solutions to real-life problems To solve real-life problems, you simply have

to work with the tools available to you at the time You have

to deal with what you have, not what you wish you had

If you wish to learn more about theory, any ofChris Date's "Relational Database Writings"

books would be a good start You might alsopick up a copy of his most recent book,

Database in Depth (O'Reilly).

Vendor Politics

This text provides solutions for five different RDBMSs It isonly natural to want to know which vendor's solution is

"best" or "fastest." There is plenty of information that each

only solutions, fine As I mentioned before, I am not here totell you to turn all your code upside down If what you have

otherwise achieve using standard SQL If you prefer ANSI-is strictly ANSI and it works for you, great When it comesdown to it, we all go to work, we all have bills to pay, and

we all want to go home at a reasonable time and enjoy

what's still left of our days So, I'm not suggesting that

ANSI-only is wrong Do what works and is best for you But,

I want to make clear that if you're looking for ANSI-onlysolutions, you should look elsewhere

Legacy Politics

The recipes in this text make use of the newest featuresavailable at the time of writing If you are using old versions

of the RDBMSs that I cover, many of my solutions will

simply not work for you Technology does not stand still,and neither should you If you need older solutions, you'llfind that many of the SQL texts available from years past

This book is divided into 14 chapters and 2 appendices:

Chapter 1, Retrieving Records, introduces very simple

queries Examples include how to use a WHERE clause torestrict rows from your result set, providing aliases for

columns in your result set, using an inline view to referencealiased columns, using simple conditional logic, limiting thenumber of rows returned by a query, returning random

records, and finding NULL values Most of the examples arevery simple, but some of them appear in more complex

complexity ranging from simple, single-column ordering, toordering by substrings, to ordering based on conditionalexpressions

Chapter 3, Working with Multiple Tables, introduces recipes

for combining data from multiple tables If you are new toSQL or are a bit rusty on joins, I strongly recommend youread this chapter before reading Chapter 5 and later Joiningtables is what SQL is all about; you must understand joins

to be successful Examples in this chapter include

performing both inner and outer joins, identifying Cartesianproductions, basic set operations (set difference, union,

intersection), and the effects of joins on aggregate

functions

for inserting, updating, and deleting data, respectively Most

of the examples are very straightforward (perhaps evenpedestrian) Nevertheless, operations such as inserting rowsinto one table from another table, the use of correlated

subqueries in updates, an understanding of the effects ofNULLs, and knowledge of new features such as multi-tableinserts and the MERGE command are extremely useful foryour toolbox

Chapter 5, Metadata Queries, introduces recipes for getting

at your database metadata It's often very useful to find theindexes, constraints, and tables in your schema The simplerecipes here allow you to gain information about your

schema Additionally, "dynamic" SQL examples are shownhere as well, i.e., SQL generated by SQL

Chapter 6, Working with Strings, introduces recipes for

manipulating strings SQL is not known for its string parsingcapabilities, but with a little creativity (usually involving

specific functions, you can accomplish quite a bit This

Cartesian products) along with the vast array of vendor-chapter is where the book begins to get interesting Some

of the more interesting examples include counting the

occurrences of a character in a string, creating delimitedlists from table rows, converting delimited lists and stringsinto rows, and separating numeric and character data from

dealing with dates Being able to perform simple date

arithmetic is crucial to everyday tasks Examples includedetermining the number of business days between two

dates, calculating the difference between two dates in

different units of time (day, month, year, etc.), and countingoccurrences of days in a month

Chapter 9, Date Manipulation, is the second of the two

chapters dealing with dates In this chapter you will findrecipes for some of the most common date operations youwill encounter in a typical work day Examples include

returning all days in a year, finding leap years, finding firstand last days of a month, creating a calendar, and filling inmissing dates for a range of dates

Chapter 10, Working with Ranges, introduces recipes for

identifying values in ranges, and for creating ranges of

values Examples include automatically generating a

sequence of rows, filling in missing numeric values for arange of values, locating the beginning and end of a range

of values, and locating consecutive values

Chapter 11, Advanced Searching, introduces recipes that

are crucial for everyday development and yet sometimesslip through the cracks These recipes are not any moredifficult than others, yet I see many developers making veryinefficient attempts at solving the problems these recipessolve Examples from this chapter include finding knightvalues, paginating through a result set, skipping rows from

reports), creating buckets or groups of data, creating

histograms, calculating simple and complete subtotals,

performing aggregations over a moving window of rows,and grouping rows based on given units of time

Chapter 13, Hierarchical Queries, introduces hierarchical

recipes Regardless of how your data is modeled, at somepoint you will be asked to format data such that it

represents a tree or parent-child relationship This chapterprovides recipes accomplishing these tasks Creating tree-structured result sets can be cumbersome with traditionalSQL, so vendor-supplied functions are particularly useful inthis chapter Examples include expressing a parent-childrelationship, traversing a hierarchy from root to leaf, androlling up a hierarchy

Chapter 14, Odds 'n' Ends, is a collection of miscellaneous

recipes that didn't seem to fit into any other problem

domain, but that nevertheless are interesting and useful.This chapter is different from the rest in that it focuses onvendor-spe-cific solutions only This is the only chapter ofthe book where each recipe highlights only one vendor Thereasons are twofold: first, this chapter was meant to serve

as more of a fun, geeky chapter Second, some recipes existonly to highlight a vendor-specific function that has no

equivalent in the other RDBMSs (examples include SQL

Server's PIVOT/UNPIVOT operators and Oracle's MODELclause) In some cases, though, you'll be able to easily

tweak a solution provided in this chapter to work for a

platform not covered in the recipe

Appendix A, Window Function Refresher, is a window

function refresher along with a solid discussion of groups inSQL Window functions are new to most, so it is appropriate

my experience I have noticed that the use of GROUP BY inqueries is a source of confusion for many developers Thischapter defines exactly what a SQL group is, and then

proceeds to use various queries as proofs to validate thatdefinition The chapter then goes into the effects of NULLs

on groups, aggregates, and partitions Lastly, you'll finddiscussion on the more obscure and yet extremely powerfulsyntax of the window function's OVER clause (i.e., the

SQL is a moving target Vendors are constantly pumping newfeatures and functionality into their products Thus you shouldknow up front which versions of the various platforms wereused in the preparation of this text:

DB2 v.8

Oracle Database 10g (with the exception of a handful of recipes, the solutions will work for Oracle8i Database and Oracle9i Database as well)

PostgreSQL 8

SQL Server 2005

MySQL 5

7369 SMITH CLERK 7902 17-DEC-1980 800 20

7499 ALLEN SALESMAN 7698 20-FEB-1981 1600 300 30

7521 WARD SALESMAN 7698 22-FEB-1981 1250 500 30

7566 JONES MANAGER 7839 02-APR-1981 2975 20

Any other tables are specific to particular recipes and chapters,and will be introduced in the text when appropriate.

I use a number of typographical and coding conventions in thisbook Take time to become familiar with them Doing so willenhance your understanding of the text Coding conventions inparticular are important, because I can't discuss them anew foreach recipe in the book Instead, I list the important

Constant width bold

Indicates user input in examples showing an interaction

Indicates a tip, suggestion, or general note.

Coding Conventions

My preference for case in SQL statements is to always use

lowercase, for both keywords and user-specified identifiers Forexample:

Your preference may be otherwise For example, many prefer touppercase SQL keywords Use whatever coding style you prefer,

or whatever your project requires

Despite my use of lowercase in code examples, I consistentlyuppercase SQL keywords and identifiers in the text I do this tomake those items stand out as something other than regularprose For example:

The preceding query represents a SELECT against the EMPtable

subquery in the FROM clause While column aliases are a

valuable tool for writing self-documenting code, aliases on inlineviews (for most recipes in this book) are simply formalities

They are typically given trivial names such as X, Y, Z, TMP1,and TMP2 In cases where I feel a better alias will provide moreunderstanding, I do so

You will notice that the SQL in the SOLUTION section of the

recipes is typically numbered, for example:

2 from emp

The number is not part of the syntax; I have included it so I canreference parts of the query by number in the discussion

section

This book is here to help you get your job done In general, youmay use the code in this book in your programs and

documentation You do not need to contact O'Reilly for

permission unless you're reproducing a significant portion of thecode For example, writing a program that uses several chunks

of code from this book does not require permission Selling or

distributing a CD-ROM of examples from O'Reilly books does

require permission Answering a question by citing this bookand quoting example code does not require permission

Incorporating a significant amount of example code from this

book into your product's documentation does require

permission

We appreciate, but do not require, attribution An attributionusually includes the title, author, publisher, and ISBN For

example: SQL Cookbook, by Anthony Molinaro Copyright 2006

O'Reilly Media, Inc., 0-596-00976-3

If you feel your use of code examples falls outside fair use orthe permission given above, feel free to contact us at

permissions@oreilly.com

We have tested and verified the information in this book to thebest of our ability, but you may find that features have changed

available for public view there) You can access this page at:

http://www.oreilly.com/catalog/sqlckbk

When you see a Safari® Enabled icon on the cover ofyour favorite technology book, it means the book is availableonline through the O'Reilly Network Safari Bookshelf

Safari offers a solution that's better than e-books It's a virtuallibrary that lets you easily search thousands of top technologybooks, cut and paste code samples, download chapters, andfind quick answers when you need the most accurate, currentinformation Try it for free at http://safari.oreilly.com