How to measure anything finding the value of intangibles in business, second edition

Building up from simple concepts to illustrate the hands-on yet intuitively easy application of advanced statistical techniques, How to Measure Anything, Second Edition reveals the powe

REVISED, EXPANDED &

SIMPLIFIED

(continued on back fl ap)

assertion is the key to solving many problems in business and life in general The myth that certain things can’t be measured

is a signifi cant drain on our nation’s economy, public welfare, the environment, and even national security In fact, the chances are good that some part

of your life or your professional responsibilities is greatly harmed by a lack of measurement—by you, your fi rm, or even your government.

Building up from simple concepts to illustrate the hands-on yet intuitively easy application of

advanced statistical techniques, How to Measure Anything, Second Edition reveals the power of

measurement in our understanding of business and the world at large This insightful and engaging book shows you how to measure those things in your business that, until now, you may have considered “immeasurable,” including technology ROI, organizational fl exibility, customer satis- faction, and technology risk Offering examples that will get you to attempt measurements—even when it seems impossible—this book provides you

with the substantive steps for measuring anything,

especially uncertainty and risk

Don’t wait—take a look inside and fi nd out:

• The three reasons why things may seem

immeasurable but are not

• Inspirational examples of where seemingly impossible measurements were resolved with suprisingly simple methods

• How computing the value of information will show that you probably have been measuring all the wrong things

instruments

• How you can use the Internet as an

instrument of measurement

A complete resource with case studies and a robust

accompanying Web site providing downloadable

spreadsheet-based examples, How to Measure

Anything, Second Edition illustrates how author

Douglas Hubbard—creator of Applied Information

Economics—has used his approach across various

industries You’ll learn how any problem, no

matter how diffi cult, ill-defi ned, or uncertain,

can lend itself to measurement using proven

methods Straightforward and easy-to-follow,

this is the resource you’ll turn to again and

again—beyond measure.

the inventor of Applied Information

Economics (AIE), a measurement

methodology that has been used in

IT portfolios, entertainment media,

military logistics, R&D portfolios, and many more

areas where big decisions are based on factors

that seem diffi cult or impossible to measure He

is an internationally recognized expert in metrics,

decision analysis, and risk management, and

is a popular speaker at numerous conferences

He has written articles for InformationWeek, CIO

Enterprise, Architecture Boston, Analytics, and OR/MS

Today, and is also the author of The Failure of Risk

Management: Why It’s Broken and How to Fix It.

ISBN: 978-0-470-53939-2

techniques in several fi elds for several years For the fi rst time, somebody wrote together all these concerns on one canvas that is at the same time accessible to a broad audience and applicable by specialists This book is a

must for students and experts in the fi eld of analysis (in general) and decision-making.”

—Dr Johan Braet, University of Antwerp, Faculty of Applied Economics,

Risk Management and Innovation

“Doug Hubbard’s book is a marvelous tutorial on how to defi ne sound metrics to justify and manage complex programs It is a must-read for anyone concerned about mitigating the risks involved with capital planning,

investment decisions, and program management.”

—Jim Flyzik, former Government CIO, White House Technology Advisor and

CIO magazine Hall of Fame Inductee

“I love this book Douglas Hubbard helps us create a path to know the answer to almost any question, in

business, in science, or in life How to Measure Anything provides just the tools most of us need to measure

anything better, to gain that insight, to make progress, and to succeed.”

—Peter Tippett, PhD, MD, Chief Technology Offi cer, CyberTrust, and inventor, fi rst antivirus software

“Interestingly written and full of case studies and rich examples, Hubbard’s book is a valuable resource for those who routinely make decisions involving uncertainty This book is readable and quite entertaining, and even

those who consider themselves averse to statistics may fi nd it highly approachable.”

—Strategic Finance

“This book is remarkable in its range of measurement applications and its clarity of style A must-read for every

professional who has ever exclaimed, ‘Sure, that concept is important, but can we measure it?’”

—Dr Jack Stenner, cofounder and CEO of MetaMetrics, Inc

“Hubbard has made a career of fi nding ways to measure things that other folks thought were immeasurable

Quality? The value of telecommuting? The benefi ts of greater IT security? Public image? He says it can be done—

and without breaking the bank If you’d like to fare better in the project-approval wars, take a look at this book.”

—ComputerWorld

“I use this book as a primary reference for my measurement class at MIT The students love it because it provides practical advice that can be applied to a variety of scenarios, from aerospace and defense, healthcare, politics, etc.”

—Ricardo Valerdi, PhD, Lecturer, MIT

Praise for the first edition—The bestselling Business Math book two years in a row!

SECOND

EDITION

iv

How to Measure Anything

Finding the Value of

Copyright
C 2010 by Douglas W Hubbard All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or

transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the

1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923,

(978) 750-8400, fax (978) 646-8600, or on the web at www.copyright.com Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011,

fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose No warranty may be created or extended by sales representatives or written sales materials The advice and strategies contained herein may not be suitable for your situation You should consult with a professional where appropriate Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at

(800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002 Wiley also publishes its books in a variety of electronic formats Some content that appears in print may not be available in electronic books For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Hubbard, Douglas W.,

1962-How to measure anything : finding the value of “intangibles” in business /

Douglas W Hubbard – 2nd ed.

I dedicate this book to the people who are my inspirations for so many things: to my wife, Janet, and to our children, Evan, Madeleine, and Steven, who show every potential for

being Renaissance people.

I also would like to dedicate this book to the military men and women of the United States, so many of whom I know personally I’ve been out of the Army National Guard for many years, but I hope my efforts at improving battlefield logistics for the U.S Marines by using better measurements

have improved their effectiveness and safety.

iii

iv

CHAPTER 2 An Intuitive Measurement Habit: Eratosthenes,

How an Ancient Greek Measured the Size of Earth 10

Experiments: Not Just for Adults 13Notes on What to Learn from Eratosthenes,

CHAPTER 3 The Illusion of Intangibles:

Economic Objections to Measurement 35The Broader Objection to the Usefulness

v

Ethical Objections to Measurement 39Toward a Universal Approach to Measurement 41

Getting the Language Right: What “Uncertainty”

Examples of Clarification: Lessons for Businessfrom, of All Places, Government 51

CHAPTER 5 Calibrated Estimates: How Much Do You Know Now? 57

Further Improvements on Calibration 64Conceptual Obstacles to Calibration 65

Real Risk Analysis: The Monte Carlo 81

An Example of the Monte Carlo Method and Risk 82Tools and Other Resources for Monte

The Risk Paradox and the Need for Better

The Chance of Being Wrong and the Cost ofBeing Wrong: Expected Opportunity Loss 100The Value of Information for Ranges 103The Imperfect World: The Value of Partial

The Epiphany Equation: How the Value ofInformation Changes Everything 110Summarizing Uncertainty, Risk, and Information

Value: The First Measurements 114

SECTION III MEASUREMENT METHODS 117

CHAPTER 8 The Transition: From What to Measure to

Choose and Design the Instrument 136

CHAPTER 9 Sampling Reality: How Observing Some Things

Building an Intuition for Random Sampling: The

A Little about Little Samples: A Beer Brewer’s

Statistical Significance: A Matter of Degree 145

The Easiest Sample Statistics Ever 150

A Biased Sample of Sampling Methods 153

Seeing Relationships in the Data: AnIntroduction to Regression Modeling 169One Thing We Haven’t Discussed—and Why 174

Using Your Natural Bayesian Instinct 181Heterogeneous Benchmarking: A “Brand

Bayesian Inversion for Ranges: An Overview 190

Bayesian Inversion for Ranges: The Details 193

CHAPTER 11 Preference and Attitudes: The Softer Side

Maximization versus Purely Subjective Trade-offs 218

CHAPTER 12 The Ultimate Measurement Instrument:

CHAPTER 13 New Measurement Instruments for Management 251

The Twenty-First-Century Tracker: Keeping Tabs

Measuring the World: The Internet as an Instrument 254Prediction Markets: A Dynamic Aggregation

CHAPTER 14 A Universal Measurement Method: Applied

Bringing the Pieces Together 266Case: The Value of the System that Monitors

Case: Forecasting Fuel for the Marine Corps 275Ideas for Getting Started: A Few Final Examples 281

x

Alot has happened since the first edition of this book was released in

2007 First, my publisher and I found out that a book with the title How

to Measure Anything apparently sparks interest For three years, the book

has consistently been the single best seller in Amazon’s math for businesscategory Interest shows no sign of slowing and, in fact, registrations on thebook’s supplementary Web site (www.howtomeasureanything.com) showthat the interest is growing across many industries and countries It wassuccessful enough that I could pitch my second book idea to my editor.The 2008 financial crisis occurred just as I was finishing my second

book, The Failure of Risk Management: Why It’s Broken and How to Fix It I

started writing that book because I felt that the topic of risk, which I couldspend only one chapter on in this book, merited much more space I arguedthat a lot of the most popular methods used in risk assessments and riskmanagement don’t stand up to the bright light of scientific scrutiny And Iwasn’t just talking about the financial industry I started writing the bookwell before the financial crisis started I wanted to make it just as relevant

to another Katrina or 9/11 as to a financial crisis

I’ve also written several more articles, and the combined research fromthem, my second book, and comments from readers on the book’s Website gave me plenty of new material to add to this second edition Butthe basic message is still the same I wrote this book to correct a costlymyth that permeates many organizations today: that certain things can’t bemeasured This widely held belief is a significant drain on the economy,public welfare, the environment, and even national security “Intangibles”such as the value of quality, employee morale, or even the economic impact

of cleaner water are frequently part of some critical business or governmentpolicy decision Often an important decision requires better knowledge ofthe alleged intangible, but when an executive believes something to beimmeasurable, attempts to measure it will not even be considered

As a result, decisions are less informed than they could be The chance

of error increases Resources are misallocated, good ideas are rejected, andbad ideas are accepted Money is wasted In some cases life and health are

xi

put in jeopardy The belief that some things—even very important things—might be impossible to measure is sand in the gears of the entire economy.All important decision makers could benefit from learning that any-thing they really need to know is measurable However, in a democ-racy and a free enterprise economy, voters and consumers count amongthese “important decision makers.” Chances are your decisions in somepart of your life or your professional responsibilities would be improved

by better measurement And it’s virtually certain that your life has already

been affected—negatively—by the lack of measurement in someone else’s

decisions

I’ve made a career out of measuring the sorts of things many thoughtwere immeasurable I first started to notice the need for better measure-ment in 1988, shortly after I started working for Coopers & Lybrand as abrand-new MBA in the management consulting practice I was surprised

at how often clients dismissed a critical quantity—something that wouldaffect a major new investment or policy decision—as completely beyondmeasurement Statistics and quantitative methods courses were still fresh in

my mind In some cases, when someone called something “immeasurable,”

I would remember a specific example where it was actually measured Ibegan to suspect any claim of immeasurability as possibly premature, and

I would do research to confirm or refute the claim Time after time, I keptfinding that the allegedly immeasurable thing was already measured by anacademic or perhaps professionals in another industry

At the same time, I was noticing that books about quantitative ods didn’t focus on making the case that everything is measurable Theyalso did not focus on making the material accessible to the people whoreally needed it They start with the assumption that the reader already be-lieves something to be measurable, and it is just a matter of executing theappropriate algorithm And these books tended to assume that the reader’sobjective was a level of rigor that would suffice for publication in a scientificjournal—not merely a decrease in uncertainty about some critical decisionwith a method a nonstatistician could understand

meth-In 1995, after years of these observations, I decided that a market isted for better measurements for managers I pulled together methods fromseveral fields to create a solution The wide variety of measurement-relatedprojects I had since 1995 allowed me to fine-tune this method Not only wasevery alleged immeasurable turning out not to be so, the most intractable

ex-“intangibles” were often being measured by surprisingly simple methods Itwas time to challenge the persistent belief that important quantities werebeyond measurement

In the course of writing this book, I felt as if I were exposing a bigsecret and that once the secret was out, perhaps a lot of things would bedifferent I even imagined it would be a small “scientific revolution” of sorts

for managers—a distant cousin of the methods of “scientific management”introduced a century ago by Frederick Taylor This material should be evenmore relevant than Taylor’s methods turned out to be for twenty-first-centurymanagers Whereas scientific management originally focused on optimizinglabor processes, we now need to optimize measurements for managementdecisions Formal methods for measuring those things management usuallyignores have barely reached the level of alchemy We need to move fromalchemy to the equivalent of chemistry and physics.

The publisher and I considered several titles All the titles consideredstarted with “How to Measure Anything” but weren’t always followed by

“Finding the Value of Intangibles in Business.” I give a seminar called “How

to Measure Anything, But Only What You Need To.” Since the methods inthis book include computing the economic value of measurement (so that

we know where to spend our measurement efforts), it seemed particularlyappropriate We also considered “How to Measure Anything: Valuing Intan-gibles in Business, Government, and Technology” since there are so manytechnology and government examples in this book alongside the general

business examples But the title chosen, How to Measure Anything: Finding

the Value of “Intangibles” in Business, seemed to grab the right audience

and convey the point of the book without necessarily excluding much ofwhat the book is about

The book is organized into four sections The chapters and sectionsshould be read in order because the first three sections rely on instructionsfrom the earlier sections Section One makes the case that everything ismeasurable and offers some examples that should inspire readers to at-tempt measurements even when it seems impossible It contains the basicphilosophy of the entire book, so, if you don’t read anything else, read thissection In particular, the specific definition of measurement discussed inthis section is critical to correctly understand the rest of the book

Section Two begins to get into more specific substance about how tomeasure things—specifically uncertainty, risk, and the value of information.These are not only measurements in their own right but, in the approachI’m proposing, prerequisites to all measurements Readers will learn how

to measure their own subjective uncertainty with “calibrated probabilityassessments” and how to use that information to compute risk and thevalue of additional measurements It is critical to understand these conceptsbefore moving on to the next section

Section Three deals with how to reduce uncertainty by various methods

of observation, including random sampling and controlled experiments Itprovides some shortcuts for quick approximations when possible It alsodiscusses methods to improve measurements by treating each observation

as updating and marginally reducing a previous state of uncertainty It views some material that readers may have seen in first-semester statistics

re-courses, but it is written specifically to build on the methods discussed inSection Two Some of the more elaborate discussions on regression model-ing and controlled experiments could be skimmed over or studied in detail,depending on the needs of the reader.

Section Four is an eclectic collection of interesting measurement tions and case examples It discusses methods for measuring such things aspreferences, values, flexibility, and quality It covers some new or obscuremeasurement instruments, including calibrated human judges or even theInternet It summarizes and pulls together the approaches covered in therest of the book with detailed discussions of two case studies and otherexamples

solu-In Chapter 1, I suggest a challenge for readers, and I will reinforce thatchallenge by mentioning it here Write down one or more measurementchallenges you have in home life or work, then read this book with thespecific objective of finding a way to measure them If those measurementsinfluence a decision of any significance, then the cost of the book and thetime to study it will be paid back manyfold

So many contributed to the content of this book through their suggestions,reviews, and as sources of information about interesting measurementsolutions In no particular order, I would like to thank these people:

Freeman Dyson Pat Plunkett Robyn Dawes

Peter Tippett Art Koines Jay Edward RussoBarry Nussbaum Terry Kunneman Reed Augliere

Skip Bailey Luis Torres Linda Rosa

Chuck McKay Ray Epich Robin Hansen

Ray Gilbert Dominic Schilt Mary Lunz

Henry Schaffer Jeff Bryan Andrew Oswald

Leo Champion Peter Schay George EberstadtTom Bakewell Betty Koleson Grether

Bill Beaver Arkalgud Ramaprasad David Todd WilsonJulianna Hale Harry Epstein Emile Servan-SchreiberJames Hammitt Rick Melberth Bruce Law

Michael Brown Gunther Eyesenbach Michael HodgsonSebastian Gheorghiu Johan Braet Moshe Kravitz

Jim Flyzik Jack Stenner Michael Gordon-Smith

Special thanks to Dominic Schilt at Riverpoint Group LLC, who saw theopportunities with this approach back in 1995 and has given so much sup-port since then And thanks to all of my blog readers who have contributedideas for this second edition

xv

xvi

SECTION I

Measurement: The Solution Exists

1

2

CHAPTER 1

Intangibles and the Challenge

When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express

it in numbers, your knowledge is of a meager and unsatisfactory kind;

it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of science.

—Lord Kelvin, British physicist and member of

the House of Lords, 1824–1907

Anything can be measured If a thing can be observed in any way at all,

it lends itself to some type of measurement method No matter how

“fuzzy” the measurement is, it’s still a measurement if it tells you morethan you knew before And those very things most likely to be seen asimmeasurable are, virtually always, solved by relatively simple measurementmethods

As the title of this book indicates, we will discuss how to find thevalue of those things often called “intangibles” in business There are twocommon understandings of the word “intangible.” It is routinely applied tothings that are literally not tangible (i.e., not touchable, solid objects) yetare widely considered to be measurable Things like time, budget, patentownership, and so on are good examples of things that you cannot touchbut yet are measured In fact, there is a well-established industry aroundmeasuring so-called intangibles such as copyright and trademark valuation.But the word “intangible” has also come to mean utterly immeasurable inany way at all, directly or indirectly It is in this context that I argue thatintangibles do not exist

You’ve heard of “intangibles” in your own organization—things thatpresumably defy measurement of any type The presumption of immea-surability is, in fact, so strong that no attempt is even made to make anyobservations that might tell you something—anything—about the alleged

3

immeasurable that you might be surprised to learn You may have run intoone or more of these real-life examples of so-called intangibles:

Management effectiveness

The forecasted revenues of a new product

The public health impact of a new government environmental policy

The productivity of research

The “flexibility” to create new products

The value of information

The risk of bankruptcy

The chance of a given political party winning the White House

The risk of failure of an information technology (IT) project

Quality

Public image

Each of these examples can very well be relevant to some major decision

an organization must make It could even be the single most importantimpact of an expensive new initiative in either business or governmentpolicy Yet in most organizations, because the specific “intangible” wasassumed to be immeasurable, the decision was not nearly as informed as itcould have been

One place I’ve seen this many times is in the “steering committees”that review proposed investments and decide which to accept or reject Theproposed investments may be related to IT, new product research and devel-opment, major real estate development, or advertising campaigns In somecases, the committees were categorically rejecting any investment where thebenefits were primarily “soft” ones Important factors with names like “im-proved word-of-mouth advertising,” “reduced strategic risk,” or “premiumbrand positioning” were being ignored in the evaluation process becausethey were considered immeasurable It’s not as if the idea was being rejectedsimply because the person proposing it hadn’t measured the benefit (a validobjection to a proposal); rather it was believed that the benefit couldn’t pos-sibly be measured—ever Consequently, some of the most important strate-gic proposals were being overlooked in favor of minor cost-savings ideassimply because everyone knew how to measure some things and didn’tknow how to measure others Equally disturbing, many major investmentswere approved with no basis for measuring whether they ever worked at all.The fact of the matter is that some organizations have succeeded inanalyzing and measuring all of the previously listed items, using methodsthat are probably less complicated than you would think The purpose ofthis book is to show organizations two things:

1 Intangibles that appear to be completely intractable can be measured

2 This measurement can be done in a way that is economically justified

To accomplish these goals, this book will address some common conceptions about intangibles, describe a “universal approach” to show how

mis-to go about measuring an “intangible,” and provide some interesting ods for particular problems Throughout, I have attempted to include someexamples (some of which I hope the reader finds inspirational) of howpeople have tackled some of the most difficult measurements there are.Without compromising substance, this book also attempts to make some

meth-of the more seemingly esoteric statistics around measurement as simple asthey can be Whenever possible, math is converted into simpler charts,tables, and procedures Some of the methods are so much simpler thanwhat is taught in the typical introductory statistics courses that we might beable to overcome many phobias about the use of quantitative measurementmethods Readers do not need any advanced training in any mathematicalmethods at all They just need some aptitude for clearly defining problems.Readers are encouraged to use this book’s Web site at www.howtomeasureanything.com The site offers a library of downloadablespreadsheets for many of the more detailed calculations shown in this book.There also are additional learning aids, examples, and a discussion boardfor questions about the book or measurement challenges in general Thesite also provides a way for me to discuss new technologies or techniquesthat were not available when this book was printed

Yes, I Mean Anything

I have one recommendation for a useful exercise to try When readingthrough the chapters, write down those things you believe are immeasurable

or, at least, you are not sure how to measure After reading this book, mygoal is that you are able to identify methods for measuring each and everyone of them And don’t hold back We will be talking about measuring suchseemingly immeasurable things as the number of fish in the ocean, the value

of a happy marriage, and even the value of a human life Whether you want

to measure phenomena related to business, government, education, art, oranything else, the methods herein apply

With a title like How to Measure Anything, anything less than a

multi-volume text would be sure to leave out something My objective does notinclude every area of physical science or economics, especially where mea-surements are well developed Those disciplines have measurement meth-ods for a variety of interesting problems, and the professionals in those disci-plines are already much less inclined even to apply the label “intangible” tosomething they are curious about The focus here is on measurements thatare relevant—even critical—to major organizational decisions and yet don’tseem to lend themselves to an obvious and practical measurement solution

If I do not mention your specific measurement problem by name, don’tconclude that methods relevant to that issue aren’t being covered The ap-

proach I will talk about applies to any uncertainty that has some relevance

to your firm, your community, even your personal life This extrapolationshould not be difficult When you studied arithmetic in elementary school,you may not have covered the solution to 347 times 79 in particular but youknew that the same procedures applied to any combination of numbers andoperations So, if your problem happens to be something that isn’t specifi-cally analyzed in this book—such as measuring the value of better productlabeling laws, the quality of a movie script, or effectiveness of motivationalseminars—don’t be dismayed Just read the entire book and apply the stepsdescribed Your immeasurable will turn out to be entirely measurable

3 Therefore, management needs a method to analyze options for reducinguncertainty about decisions

Perhaps you think the first two points are too obvious to make Butwhile it may seem obvious, few management consultants, performancemetrics experts, or even statisticians approach the problem with the explicitpurpose of supporting defined decisions Even if they had that squarely inmind, the last point, at a minimum, is where a lot of business measurementmethods fall short

It is very useful to see measurement as a type of optimization problemfor reducing uncertainty Upon reading the first edition of this book, a busi-ness school professor remarked that he thought I had written a book aboutthe somewhat esoteric field called “decision analysis” and disguised it under

a title about measurement so that people from business and governmentwould read it That wasn’t my intention when I set out, but I think he hitthe nail on the head Measurement is about supporting decisions, and thereare even several decisions to make within measurements themselves

If the decision in question is highly uncertain and has significant sequences if it turns out wrong, then measurements that reduce uncertainty

con-about it have a high value Nobody should care con-about measuring something

if it doesn’t inform a significant bet of some kind Likewise, if measurementswere free, obvious, and instantaneous, we would have no dilemma aboutwhat, how, or even whether to measure

Granted, a measurement might also be taken because it has its ownmarket value (e.g., results of a consumer survey) or because it is simplysatisfying a curiosity or will be entertaining (e.g., academic research aboutthe evolution of clay pottery) But the methods we discuss in the decision-focused approach to measurement should be useful on those occasions, too

If a measurement is not informing your decisions, it could still be informingthe decisions of others who are willing to pay for the information And

if you are an academic curious about what really happened to the woolymammoth, then, again, I believe this book will have some bearing on howyou set up the problem

From here on out, this book addresses three broad issues: why nothing

is really immeasurable, how to set up and define any measurement problem,and how to use powerful and practical measurement methods to resolve theproblem The next two chapters of this book build the argument for the firstpoint: that you can really measure anything Chapters 4 through 7 set upthe measurement problem by answering questions from the point of view

of supporting specific decisions We have to answer the question “What isthe real problem/decision/dilemma?” underlying the desired measurement

We also have to answer the question “What about that problem really needs

to be measured and by how much (to what degree of accuracy/precision)?”These questions frame the problem in terms of the primary decision themeasurement is meant to resolve and the “microdecisions” that need to bemade within the measurement process itself

The remainder of the book combines this approach with powerful andpractical empirical methods to reduce uncertainty—some basic, some moreadvanced The final chapter pulls it all together into a solution and describeshow that solution has been applied to real-world problems Since this ap-proach can apply to anything, the details might sometimes get complicated.But it is much less complicated than many other initiatives organizationsroutinely commit to doing I know, because I’ve helped many organizations

apply these methods to the really complicated problems: venture capital, IT

portfolios, measuring training, improving homeland security, and more

In fact, measurements that are useful are often much simpler than ple first suspect I make this point in Chapter 2 by showing how three cleverindividuals measured things that were previously thought to be difficult orimpossible to measure

peo-8

a skill would look like It’s revealing, however, to find out that so many

of the best examples seem to be from outside of business In fact, thisbook will borrow heavily from outside of business to reveal measurementmethods that can be applied to business

Here are just a few people who, while they weren’t working on surement within business, can teach business people quite a lot about what

mea-an intuitive feel for qumea-antitative investigation should look like

In ancient Greece, a man estimated the circumference of Earth by ing at the different lengths of shadows in different cities at noon and

look-by applying some simple geometry

A Nobel Prize–winning physicist taught his students how to estimate byestimating the number of piano tuners in Chicago

A nine-year-old girl set up an experiment that debunked the growingmedical practice of “therapeutic touch” and, two years later, became the

youngest person ever to be published in the Journal of the American

Medical Association (JAMA).

You may have heard of these individuals, or maybe just one or two

of them Even if you vaguely remember something about them, it is worthreviewing each in the context of the others None of these people ever meteach other personally (none lived at the same time), but each showed an

9

ability to size up a measurement problem and identify quick and simpleobservations that have revealing results They were able to estimate un-knowns quickly by using simple observations It is important to contrasttheir approach with what you might typically see in a business setting Thecharacters in these examples are or were real people named Eratosthenes,Enrico, and Emily.

How an Ancient Greek Measured the Size of Earth

Our first mentor of measurement did something that was probably thought

by many in his day to be impossible An ancient Greek named Eratosthenes(ca 276–194B.C.) made the first recorded measurement of the circumference

of Earth If he sounds familiar, it might be because he is mentioned in manyhigh school trigonometry and geometry textbooks

Eratosthenes didn’t use accurate survey equipment, and he certainlydidn’t have lasers and satellites He didn’t even embark on a risky andprobably lifelong attempt at circumnavigating Earth Instead, while in theLibrary of Alexandria, he read that a certain deep well in Syene, a city insouthern Egypt, would have its bottom entirely lit by the noon sun one day ayear This meant the sun must be directly overhead at that point in time But

he also observed that at the same time, vertical objects in Alexandria (almoststraight north of Syene) cast a shadow This meant Alexandria received sun-light at a slightly different angle at the same time Eratosthenes recognizedthat he could use this information to assess the curvature of Earth

He observed that the shadows in Alexandria at noon at that time of yearmade an angle that was equal to an arc of one-fiftieth of a circle Therefore,

if the distance between Syene and Alexandria was one-fiftieth of an arc, thecircumference of Earth must be 50 times that distance Modern attempts toreplicate Eratosthenes’s calculations vary by exactly how much the angleswere, conversions from ancient units of measure, and the exact distancesbetween the ancient cities, but typical results put his answer within 3%

of the actual value.1 Eratosthenes’s calculation was a huge improvementover previous knowledge, and his error was less than the error modernscientists had just a few decades ago for the size and age of the universe.Even 1,700 years later, Columbus was apparently unaware of or ignoredEratosthenes’s result; his estimate was fully 25% short (This is one of thereasons Columbus thought he might be in India, not another large, interven-ing landmass where I reside.) In fact, a more accurate measurement thanEratosthenes’s would not be available for another 300 years after Columbus

By then, two Frenchmen, armed with the finest survey equipment available

in late-eighteenth-century France, numerous staff, and a significant grant,finally were able to do better than Eratosthenes.2

Here is the lesson for business: Eratosthenes made what might seem

an impossible measurement by making a clever calculation on some simpleobservations When I ask participants in my measurement and risk analysisseminars how they would make this estimate without modern tools, theyusually identify one of the “hard ways” to do it (e.g., circumnavigation)

But Eratosthenes, in fact, may not have even left the vicinity of the library to

make this calculation One set of observations that would have answeredthis question would have been very difficult to make, but his measurementwas based on other, simpler observations He wrung more information out

of the few facts he could confirm instead of assuming the hard way wasthe only way

Estimating: Be Like Fermi

Another person from outside business who might inspire measurementswithin business is Enrico Fermi (1901–1954), a physicist who won the NobelPrize in physics in 1938 He had a well-developed knack for intuitive, evencasual-sounding measurements

One renowned example of his measurement skills was demonstrated atthe first detonation of the atom bomb, the Trinity Test site, on July 16, 1945,where he was one of the atomic scientists observing the blast from basecamp While other scientists were making final adjustments to instrumentsused to measure the yield of the blast, Fermi was making confetti out of

a page of notebook paper As the wind from the initial blast wave began

to blow through the camp, he slowly dribbled the confetti into the air,observing how far back it was scattered by the blast (taking the farthestscattered pieces as being the peak of the pressure wave) Fermi concludedthat the yield must be greater than 10 kilotons This would have beennews, since other initial observers of the blast did not know that lowerlimit Could the observed blast be less than 5 kilotons? Less than 2? Theseanswers were not obvious at first (As it was the first atomic blast on theplanet, nobody had much of an eye for these things.) After much analysis ofthe instrument readings, the final yield estimate was determined to be 18.6kilotons Like Eratosthenes, Fermi was aware of a rule relating one simpleobservation—the scattering of confetti in the wind—to a quantity he wanted

to measure

The value of quick estimates was something Fermi was familiar withthroughout his career He was famous for teaching his students skills toapproximate fanciful-sounding quantities that, at first glance, they mightpresume they knew nothing about The best-known example of such

a “Fermi question” was Fermi asking his students to estimate the ber of piano tuners in Chicago His students—science and engineering

num-majors—would begin by saying that they could not possibly know anythingabout such a quantity Of course, some solutions would be to simply do acount of every piano tuner perhaps by looking up advertisements, checkingwith a licensing agency of some sort, and so on But Fermi was trying toteach his students how to solve problems where the ability to confirm theresults would not be so easy He wanted them to figure out that they knew

something about the quantity in question.

Fermi would start by asking them to estimate other things about pianosand piano tuners that, while still uncertain, might seem easier to estimate.These included the current population of Chicago (a little over 3 million

in the 1930s to 1950s), the average number of people per household (2 or3), the share of households with regularly tuned pianos (not more than 1

in 10 but not less than 1 in 30), the required frequency of tuning (perhaps

1 a year, on average), how many pianos a tuner could tune in a day (4 or

5, including travel time), and how many days a year the turner works (say,

250 or so) The result would be computed:

Tuners in Chicago= Population/people per household

× percentage of households with tuned pianos

× tunings per year/

(tunings per tuner per day× workdays per year)Depending on which specific values you chose, you would probablyget answers in the range of 20 to 200, with something around 50 being fairlycommon When this number was compared to the actual number (whichFermi might get from the phone directory or a guild list), it was always closer

to the true value than the students would have guessed This may seem like

a very wide range, but consider the improvement this was from the “Howcould we possibly even guess?” attitude his students often started with.This approach to solving a Fermi question is known as a Fermi decom-position or Fermi solution This method helped to estimate the uncertainquantity but also gave the estimator a basis for seeing where uncertaintyabout the quantity came from Was the big uncertainty about the share ofhouseholds that had tuned pianos, how often a piano needed to be tuned,how many pianos can a tuner tune in a day, or something else? The biggestsource of uncertainty would point toward a measurement that would reducethe uncertainty the most

Technically, a Fermi decomposition is not yet quite a measurement It

is not based on new observations (As we will see later, this is central to themeaning of the word “measurement.”) It is really more of an assessment ofwhat you already know about a problem in such a way that it can get you

in the ballpark The lesson for business is to avoid the quagmire that tainty is impenetrable and beyond analysis Instead of being overwhelmed

uncer-by the apparent uncertainty in such a problem, start to ask what things

about it you do know As we will see later, assessing what you currently

know about a quantity is a very important step for measurement of thosethings that do not seem as if you can measure them at all

A Fermi Decomposition for a New Business

Chuck McKay, with Wizard of Ads, encourages companies to useFermi questions to estimate the market size for a product in a givenarea An insurance agent once asked Chuck to evaluate an opportunity

to open a new office in Wichita Falls, Texas, for an insurance carrierthat currently had no local presence there Is there room for anothercarrier in this market? To test the feasibility of this business proposition,McKay answered a few Fermi questions with some Internet searches.Like Fermi, McKay started with the big population questions andproceeded from there

According to City-Data.com, there were 62,172 cars in WichitaFalls According to the Insurance Information Institute, the averageautomobile insurance annual premium in the state of Texas was

$837.40 McKay assumed that almost all cars have insurance, since it

is mandatory, so the gross insurance revenue in town was $52,062,833each year The agent knew the average commission rate was 12%,

so the total commission pool was $6,247,540 per year According toSwitchboard.com, there were 38 insurance agencies in town, a numberthat is very close to what was reported in Yellowbook.com When thecommission pool is divided by those 38 agencies, the average agencycommissions are $164,409 per year

This market was probably getting tight since City-Data.com alsoshowed the population of Wichita Falls fell from 104,197 in 2000 to99,846 in 2005 Furthermore, a few of the bigger firms probably wrotethe majority of the business, so the revenue would be even less thanthat—and all this before taking out office overhead

McKay’s conclusion: A new insurance agency with a new brand

in town didn’t have a good chance of being very profitable, and theagent should pass on the opportunity

(Note: These are all exact numbers But soon we will discuss how

to do the same kind of analysis when all you have are inexact ranges.)

Experiments: Not Just for Adults

Another person who seemed to have a knack for measuring her worldwas Emily Rosa Although Emily published one of her measurements in

the JAMA, she did not have a PhD or even a high school diploma At the

time she conducted the measurement, Emily was a 9-year-old working on

an idea for her fourth-grade science fair project She was just 11 years oldwhen her research was published, making her the youngest person ever tohave research published in the prestigious medical journal and perhaps theyoungest in any major, peer-reviewed scientific journal

In 1996, Emily saw her mother, Linda, watching a videotape on a ing industry called “therapeutic touch,” a controversial method of treatingailments by manipulating the patients’ “energy fields.” While the patient laystill, a therapist would move his or her hands just inches away from thepatient’s body to detect and remove “undesirable energies,” which presum-ably caused various illnesses Emily suggested to her mother that she might

grow-be able to conduct an experiment on such a claim Linda, who was a nurseand a long-standing member of the National Council Against Health Fraud(NCAHF), gave Emily some advice on the method

Emily initially recruited 21 therapists for her science fair experiment.The test involved Emily and the therapist sitting on opposite sides of atable A cardboard screen separated them, blocking each from the view

of the other The screen had holes cut out at the bottom through whichthe therapist would place her hands, palms up, and out of sight Emilywould flip a coin and, based on the result, place her hand four to fiveinches over the therapist’s left or right hand (This distance was marked

on the screen so that Emily’s hand would be a consistent distance fromthe therapist’s hand.) The therapists, unable to see Emily, would have todetermine whether she was holding her hand over their left or right hand

by feeling for her energy field Emily reported her results at the science fairand got a blue ribbon—just as everyone else did

Linda mentioned Emily’s experiment to Dr Stephen Barrett, whom sheknew from the NCAHF Barrett, intrigued by both the simplicity of themethod and the initial findings, then mentioned it to the producers of the

TV show Scientific American Frontiers shown on the Public Broadcasting

System In 1997, the producers shot an episode on Emily’s experimentalmethod Emily managed to convince 7 of the original 21 therapists to takethe experiment again for the taping of the show She now had a total of

28 separate tests, each with 10 opportunities for the therapist to guess thecorrect hand

This made a total of 280 individual attempts by 21 separate therapists(14 had 10 attempts each while another 7 had 20 attempts each) to feelEmily’s energy field They correctly identified the position of Emily’s handjust 44% of the time Left to chance alone, they should get about 50%right with a 95% confidence interval of+/− 6% (If you flipped 280 coins,there is a 95% chance that between 44% and 56% would be heads.) Sothe therapists may have been a bit unlucky (since they ended up on the

bottom end of the range), but their results are not out of bounds of whatcould be explained by chance alone In other words, people “uncertified”

in therapeutic touch—you or I—could have just guessed and done as well

as or better than the therapists

With these results, Linda and Emily thought the work might be worthy

of publication In April 1998, Emily, then 11 years old, had her experiment

published in the JAMA That earned her a place in the Guinness Book of

World Records as the youngest person ever to have research published in a

major scientific journal and a $1,000 award from the James Randi EducationalFoundation

James Randi, retired magician and renowned skeptic, set up this dation for investigating paranormal claims scientifically (He advised Emily

foun-on some issues of experimental protocol.) Randi created the $1 millifoun-on

“Randi Prize” for anyone who can scientifically prove extrasensory tion (ESP), clairvoyance, dowsing, and the like Randi dislikes labeling hisefforts as “debunking” paranormal claims since he just assesses the claimwith scientific objectivity But since hundreds of applicants have been un-able to claim the prize by passing simple scientific tests of their paranormalclaims, debunking has been the net effect Even before Emily’s experimentwas published, Randi was also interested in therapeutic touch and was try-ing to test it But, unlike Emily, he managed to recruit only one therapistwho would agree to an objective test—and that person failed

percep-After these results were published, therapeutic touch proponents stated

a variety of objections to the experimental method, claiming it proved ing Some stated that the distance of the energy field was really one to threeinches, not the four or five inches Emily used in her experiment.3 Othersstated that the energy field was fluid, not static, and Emily’s unmoving handwas an unfair test (despite the fact that patients usually lie still during their

noth-“treatment”).4 None of this surprises Randi “People always have excusesafterward,” he says “But prior to the experiment every one of the therapistswere asked if they agreed with the conditions of the experiment Not onlydid they agree, but they felt confident they would do well.” Of course, thebest refutation of Emily’s results would simply be to set up a controlled,

valid experiment that conclusively proves therapeutic touch does work No

such refutation has yet been offered

Randi has run into retroactive excuses to explain failures to demonstrateparanormal skills so often that he has added another small demonstration tohis tests Prior to taking the test, Randi has subjects sign an affidavit statingthat they agreed to the conditions of the test, that they would later offer noobjections to the test, and that, in fact, they expected to do well under thestated conditions At that point Randi hands them a sealed envelope Afterthe test, when they attempt to reject the outcome as poor experimentaldesign, he asks them to open the envelope The letter in the envelope

simply states “You have agreed that the conditions were optimum and thatyou would offer no excuses after the test You have now offered thoseexcuses.” Randi observes, “They find this extremely annoying.”

Emily’s example provides more than one lesson for business First, eventouchy-feely-sounding things like “employee empowerment,” “creativity,” or

“strategic alignment” must have observable consequences if they matter atall I’m not saying that such things are “paranormal,” but the same rulesapply

Second, Emily’s experiment demonstrated the effectiveness of simplemethods routinely used in scientific inquiry, such as a controlled experiment,sampling (even a small sample), randomization, and using a type of “blind”

to avoid bias from the test subject or researcher These simple elements can

be combined to allow us to observe and measure a variety of phenomena.Also, Emily showed that useful levels of experimentation can be un-derstood by even a child on a small budget Linda Rosa said she spentjust $10 on the experiment Emily could have constructed a much moreelaborate clinical trial of the effects of this method using test groups andcontrol groups to test how much therapeutic touch improves health But shedidn’t have to do that because she simply asked a more basic question If

the therapists can do what they claimed, then they must, Emily reasoned, at

least be able to feel the energy field If they can’t do that (and it is a basic

as-sumption of the claimed benefits), then everything about therapeutic touch

is in doubt She could have found a way to spend much more if she had,say, the budget of one of the smaller clinical studies in medical research.But she determined all she needed with more than adequate accuracy Bycomparison, how many of your performance metrics methods could getpublished in a scientific journal?

Emily’s example shows us how simple methods can produce a usefulresult Her experiment was far less elaborate than most others published inthe journal, but the simplicity of the experiment was actually considered apoint in favor of the strength of its findings According to George Lundberg,

the editor of the journal, JAMA’s statisticians “were amazed by its simplicity

and by the clarity of its results.”5

Perhaps you are thinking that Emily is a rare child prodigy Even asadults, most of us would be hard-pressed to imagine such a clever solution

to a measurement problem like this According to Emily herself, nothingcould be further from the truth At the writing of this second edition, EmilyRosa was working on her last semester for a bachelor’s degree in psychology

at the University of Colorado–Denver She volunteered that she had earned arelatively modest 3.2 GPA and describes herself as average Still, she does en-counter those who expect anyone who has published research at the age of

11 to have unusual talents “It’s been hard for me,” she says, “because somepeople think I’m a rocket scientist and they are disappointed to find out that

I’m so average.” Having talked to her, I suspect she is a little too modest, buther example does prove what can be done by most managers if they tried.

I have at times heard that “more advanced” measurements like trolled experiments should be avoided because upper management won’tunderstand them This seems to assume that all upper management reallydoes succumb to the Dilbert Principle (cartoonist Scott Adam’s tongue-in-cheek rule that states that only the least competent get promoted).6

con-In my experience, upper management will understand it just fine, if youexplain it well

Emily, explain it to them, please

Example: Mitre Information Infrastructure

An interesting business example of how a business might measure an

“intangible” by first testing if it exists at all is the case of the MitreInformation Infrastructure (MII) This system was developed in the late1990s by Mitre Corporation, a not-for-profit that provides federal agen-cies with consulting on system engineering and information technology.MII was a corporate knowledge base that spanned insular departments

to improve collaboration

In 2000, CIO magazine wrote a case study about MII The

maga-zine’s method for this sort of thing is to have a staff writer do all theheavy lifting for the case study itself and then to ask an outside expert

to write an accompanying opinion column called “Critical Analysis.”The magazine often asked me to write the opinion column when thecase was anything about value, measurement, risk, and so on, and Iwas asked to do so for the MII case

The “Critical Analysis” column is meant to offer some balance in thecase study since companies talking about some new initiative are likely

to paint a pretty rosy picture The article quotes Al Grasso, the chiefinformation officer (CIO) at the time: “Our most important gain can’t

be as easily measured—the quality and innovation in our solutionsthat become realizable when you have all this information at yourfingertips.” However, in the opinion column, I suggested one fairlyeasy measure of “quality and innovation”:

If MII really improves the quality of deliverables, then it should affect customer perceptions and ultimately revenue.7 Simply ask a random sample of customers to rank the quality of some pre-MII and post-MII deliverables (make sure they don’t know which

(continued )

be able to tell that there is any difference? If the relevant judges (i.e.,

the customers) can’t tell, in a blind test, that post-MII research is “higherquality” or “more innovative” than pre-MII research, then MII shouldn’thave any bearing on customer satisfaction or, for that matter, revenue

If, however, they can tell the difference, then you can worry about thenext question: whether the revenue improved enough to be worth theinvestment of over $7 million by 2000 Like everything else, if Mitre’squality and innovation benefits could not be detected, then they don’tmatter I’m told by current and former Mitre employees that my columncreated a lot of debate However, they were not aware of any suchattempt actually to measure quality and innovation Remember, the

CIO said this would be the most important gain of MII, and it went

unmeasured

Notes on What to Learn from Eratosthenes, Enrico, and Emily

Taken together, Eratosthenes, Enrico, and Emily show us something verydifferent from what we are typically exposed to in business Executives oftensay, “We can’t even begin to guess at something like that.” They dwell adinfinitum on the overwhelming uncertainties Instead of making any attempt

at measurement, they prefer to be stunned into inactivity by the apparentdifficulty in dealing with these uncertainties Fermi might say, “Yes, there

are a lot of things you don’t know, but what do you know?”

Other managers might object: “There is no way to measure that thingwithout spending millions of dollars.” As a result, they opt not to engage in

a smaller study—even though the costs might be very reasonable—becausesuch a study would have more error than a larger one Yet perhaps eventhis uncertainty reduction might be worth millions, depending on the sizeand frequency of the decision it is meant to support Eratosthenes and Emilymight point out that useful observations can tell you something you didn’tknow before—even on a budget—if you approach the topic with just a littlemore creativity and less defeatism

Eratosthenes, Enrico, and Emily inspire us in different ways thenes had no way of computing the error on his estimate, since statisticalmethods for assessing uncertainty would not be around for two more mil-lennia However, if he would have had a way to compute uncertainty,the uncertainties in measuring distances between cities and exact angles ofshadows might have easily accounted for his relatively small error Fortu-nately, we do have those tools available to us The concept of measurement

Eratos-as “uncertainty reduction” and not necessarily the elimination of uncertainty

is a central theme of this book

We learn a related but different lesson from Enrico Fermi Since he won

a Nobel Prize, it’s safe to assume that Fermi was an especially proficient perimental and theoretical physicist But the example of his Fermi questionshowed, even for non–Nobel Prize winners, how we can estimate thingsthat, at first, seem too difficult even to attempt to estimate Although his in-sight on advanced experimental methods of all sorts would be enlightening,

ex-I find that the reason intangibles seem intangible is almost never for lack

of the most sophisticated measurement methods Usually things that seemimmeasurable in business reveal themselves to much simpler methods ofobservation, once we learn to see through the illusion of immeasurability

In this context, Fermi’s value to us is in how we determine our current state

of knowledge about a thing as a precursor to further measurement.Unlike Fermi’s example, Emily’s example is not so much about initialestimation since her experiment made no prior assumptions about howprobable the therapeutic touch claims were Nor was her experiment aboutusing a clever calculation instead of infeasible observations, like Eratos-thenes Her calculation was merely based on standard sampling methodsand did not itself require a leap of insight like Eratosthenes’s simple geome-try calculation But Emily does demonstrate that useful observations are notnecessarily complex, expensive, or even, as is sometimes claimed, beyondthe comprehension of upper management even for ephemeral concepts liketouch therapy (or strategic alignment, employee empowerment, improvedcommunication, etc.)

And as useful as these lessons are, we will build even further on thelessons of Eratosthenes, Enrico, and Emily We will learn ways to assessyour current uncertainty about a quantity that improve on Fermi’s methods,some sampling methods that are in some ways even simpler than whatEmily used, and simple methods that would have allowed even Eratosthenes

to improve on his estimate of the size of a world that nobody had yettraveled

Given examples like this, we have to wonder why anyone ever believessomething to be beyond measurement There are only a few arguments forbelieving something to be immeasurable In the next chapter, we will discusswhy each of these arguments is flawed

1 M Lial and C Miller, Trigonometry, 3rd ed (Chicago: Scott, Foresman, 1988).

2 Two Frenchmen, Pierre-Franc¸ois-Andr´e M´echain and Jean-Baptiste-Joseph, lated Earth’s circumference over a seven-year period during the French Revolution

calcu-on a commissicalcu-on to define a standard for the meter (The meter was originallydefined to be one 10-millionth of the distance from the equator to the pole.)

3 Letter to the Editor, New York Times, April 7, 1998.

4 “Therapeutic Touch: Fact or Fiction?” Nurse Week, June 7, 1998.

5 “A Child’s Paper Poses a Medical Challenge” New York Times, April 1, 1998.

6 Scott Adams, The Dilbert Principle (New York: Harper Business, 1996).

7 Although a not-for-profit, Mitre still has to keep operations running by generatingrevenue through consulting billed to federal agencies

8 Doug Hubbard, “Critical Analysis” column accompanying “An Audit Trail,” CIO,

May 1, 2000

CHAPTER 3

The Illusion of Intangibles:

Why Immeasurables Aren’t

There are just three reasons why people think that something can’t bemeasured Each of these three reasons is actually based on miscon-ceptions about different aspects of measurement I will call them concept,object, and method

1 Concept of measurement The definition of measurement itself is widely

misunderstood If one understands what “measurement” actually means,

a lot more things become measurable

2 Object of measurement The thing being measured is not well defined.

Sloppy and ambiguous language gets in the way of measurement

3 Methods of measurement Many procedures of empirical observation

are not well known If people were familiar with some of these basicmethods, it would become apparent that many things thought to beimmeasurable are not only measurable but may already have beenmeasured

A good way to remember these three common misconceptions is by

using a mnemonic like “howtomeasureanything.com,” where the c, o, and

m in “.com” stand for concept, object, and method Once we learn that these

three objections are misunderstandings of one sort or another, it becomesapparent that everything really is measurable

In addition to these reasons why something can’t be measured, thereare also three common reasons why something “shouldn’t” be measured.The reasons often given for why something “shouldn’t” be measured are:

1 The economic objection to measurement (i.e., any measurement would

be too expensive)

21

2 The general objection to the usefulness and meaningfulness of statistics(i.e., “You can prove anything with statistics.”)

3 The ethical objection (i.e., we shouldn’t measure it because it would beimmoral to measure it)

Unlike the concept, object, and method list, these three objections don’treally argue that a measurement is impossible, just that it is not cost effec-tive, that measurements in general are meaningless, or that it is morallyobjectionable to measure it I will show that only the economic objectionhas any potential merit, but even that one is overused

The Concept of Measurement

As far as the propositions of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality.

—Albert Einstein

Although this may seem a paradox, all exact science is based on the idea

of approximation If a man tells you he knows a thing exactly, then you can be safe in inferring that you are speaking to an inexact man.

—Bertrand Russell (1873-1970), British mathematician

and philosopherFor those who believe something to be immeasurable, the concept of

measurement, or rather the misconception of it, is probably the most

impor-tant obstacle to overcome If we incorrectly think that measurement meansmeeting some nearly unachievable standard of certainty, then few thingswill seem measurable I routinely ask those who attend my seminars orconference lectures what they think “measurement” means (It’s interesting

to see how much thought this provokes among people who are actually

in charge of some measurement initiative in their organization.) I usuallyget answers like “to quantify something,” “to compute an exact value,” “toreduce to a single number,” or “to choose a representative amount,” and

so on Implicit or explicit in all of these answers is that measurement iscertainty—an exact quantity with no room for error If that was really whatthe term means, then, indeed, very few things would be measurable.But when scientists, actuaries, or statisticians perform a measurement,they seem to be using a different de facto definition In their special fields,each of these professions has learned the need for a precise use of certainwords sometimes very different from how the general public uses a word.Consequently, members of these professions usually are much less confused