Longitudinal Study of EJournal Use at a US Chemistry Library

Over 1.8 million chemistry and science article views or downloads are projected to be accessed by UIUC users in 2010 based on data gathered so far this year.. This 2010 study examines a

A Longitudinal Study of E-Journal Use at a U.S Academic Chemistry Library:

Are we measuring what matters and counting what counts?

By

Tina E Chrzastowski Chemistry Librarian University of Illinois at Urbana-Champaign

Urbana, Illinois USA

Submitted July 1, 2010

“Measuring usage and understanding users!

E-resources statistics and what they teach us.”

The University of Illinois at Urbana-Champaign (UIUC) Chemistry Library, an early adopter of electronic journals, began collecting e-use data biannually in 2000 A decade later, our

longitudinal e-journal use data show that our users were also early adopters who migrated

completely from print to e-journals within 6 years During this decade, e-journal use exploded as the Chemistry Library purchased additional content and e-access became the user-preferred format After years of growth in collection size and use, e-use nudged up less than 2% between

2006 and 2008; however, use jumped again in 2010, increasing 9.3% between 2008 and 2010 Over 1.8 million chemistry and science article views or downloads are projected to be accessed

by UIUC users in 2010 based on data gathered so far this year These data tell us a lot about our users’ needs, but they also inform cost/use ratios, publisher value, and local citation utilization reports Although enlightening and useful, beyond these broad-brush trend data, what lies ahead

in e-journal measurement and analysis? Over 1.8 million chemistry and science views or PDF downloads per year indicate that we have moved beyond user acceptance to a new model

However, the next decade will require new measurement tools and a different perspective to help libraries determine user needs and expectations and how to meet them with diminishing budgets

Introduction

Journal use analysis at the University of Illinois at Urbana-Champaign (UIUC) Chemistry Library was published in 1991 (Chrzastowski) and 1993 (Chrzastowski and Olesko) looking at print use only A 2003 study (Chrzastowski) focused on the migration of users from print to electronic journal use, documenting the leap users made from the established print format to the new electronic format This 2010 study examines a decade (2000-2010) of e-journal use at the UIUC Chemistry Library to look for trends, to benchmark the effectiveness of the collection over time, and to compare publisher use data with other use-related data available in order to develop

a better understanding of what “use” of an e-journal article means The decade’s statistics also demonstrate the impact on the library as we moved to the digital age

In most science disciplines, journals comprise the subject budget’s largest expenditure This is especially true for chemistry, the discipline lucky enough to host the most expensive journals published (Henderson and Bosch) Pricing fluctuations, in part based on inflation, are annual

events that must be balanced with any new money awarded to the fund And since very little new money is finding its way to libraries these days, the result is an evaluation of the collection to determine what cuts must be made or if funds from other material formats might be moved to cover the serial budget Serial cancellation decisions, when necessary, are optimally based on data that include cost and local use by title

For the purposes of this study, “cost/use ratio” is used over the term “cost-per-use.” The term per-use implies a very exact number that is comparable to other per-use data But cost-per-use for a weekly journal cannot fairly be compared to a quarterly journal Another example is trying to compare a journal that has been published for over 20 years in many volumes to a fairly new journal with only a few volumes In order to value each use, the total number of volumes must be taken into consideration to establish a true “cost-per-use.” A much broader term is

“cost/use ratio.” This phrase implies a more general formula that is closer to the actual valuations being made when dividing the number of annual uses into the annual cost of a journal

Ultimately, the cost/use ratio of a journal can help determine if an annual subscription is

warranted or if it is more cost effective to purchase that journal’s articles individually from a document supplier For this collection of research-level chemistry journals, the cutoff is based on the cost of a single document delivery article for that journal For example, if the cost of getting

an article from a specific journal from the British Library’s Inside Web (including copyright fees and delivery within 48 hours) is $88.00, then the cost/use ratio for that same journal should be below $88.00 in order to remain part of the subscribed collection This type of valuation is not exact, but for the purposes of determining the relative cost/use ratio for over 800 journals, this process is a way to quickly determine those journals highly cost effective to own and those not pulling their weight The bottom of the ranked listing is where the most attention will be paid during a cancellation project, and this methodology can easily determine high and low cost/use ratios

The importance of acquiring, analyzing, and relying on local cost/use journal data cannot be overstated Only local data matter because other libraries’ data have no local relevance In the United States, automobile industry manufacturers must report the mileage range a car can be

expected to produce, and these data are always given with the caveat “your mileage may differ” due to the vast types of driving conditions and drivers that will effect the result The same is true for determining the use of library materials – those using/driving the product will ultimately determine use and cost/use ratios Each user/driver is operating under a unique set of research demands, searching skills, needs, time limitations, and subject parameters Therefore, the local group of researchers, all with unique needs, together create the local use that guides the overall collection in a specific discipline

Setting

The UIUC is a premiere research institution located in the Midwest United States The School of Chemical Sciences is a top-ten, nationally ranked institution comprising the Departments of Chemistry and Chemical and Biomolecular Engineering In addition to collecting and supporting chemistry and chemical engineering, the Chemistry Library is also the home for biochemistry materials Table 1 shows the number of faculty, staff, and graduate students in these subject areas biannually for the past 10 years, the study period for this research (American Chemical Society) Few changes have occurred in the population over the last decade, although recent budget crises have led to a declining number of graduate students and staff over the period The relative stability of these numbers shows that very few changes in our user population have taken place over time, and any increase in e-journal use during the past decade is not due to an increase in the number of users

E-journal use data collection began in 2000 based on the print journal collection then housed in the UIUC Chemistry Library The expansive nature of this collection meant that broad science

journals such as Nature and Science would be included as well as top journals in physics, a smattering in biology, and Annual Reviews in all science disciplines In order to continue to

measure the same collection over time, these journals have remained part of the use study to

2010 However, this means that non-Chemistry users campus-wide are measured, too, because

we cannot track individual users to isolate use by Chemistry affiliates only Although this greatly expands the user population, it remains essentially the same from 2000 to 2010 No large

increases in user populations campus-wide are discernable during the study period Therefore, these are valid comparisons over time for the e-journal use but not valid compared to print

measurement As noted later in this paper, print and e-journal use statistics are not comparable

due to a number of factors

Table 1 Estimated number of primary chemistry e-journals users at UIUC (2001-2009) (American

Chemical Society) Not included in this table are secondary users such as food scientists, plant scientists, ecologists, etc

University of Illinois at Urbana-Champaign

Methodology

As with any longitudinal study, research methods have a way of changing over time due to a

number of factors, including technological changes (LOTUS 1-2-3 and 5.25” floppy disks no

longer being available), new standards implemented (Project COUNTER), and physical changes (print journals no longer on the shelf cannot be counted) Longitudinal studies therefore result in data that may not precisely match the collection methods of previous studies, even those

conducted just a few years earlier This is especially true with e-journal use data over the past

decade In 2000, when e-journal use statistics were just beginning to be made available to

libraries, few options were available Because technologies now allow for more detail and

Project COUNTER has established a standard, publishers have made a number of changes in

data reporting since 2000 For example, the American Chemical Society has changed their

statistical reporting for e-journal use twice in the past ten years, once mid-month, making it

challenging to accurately compare data between these different systems with their different

definitions

Another issue with e-journal data collection is that in early years statistics are broad-brush with vague definitions, such as “number of article requests,” and often publishers simply could not

provide data at all In 2000, the first year we attempted to acquire e-journal use data from

publishers, only 15% of titles had corresponding use statistics available Further complicating usage comparisons by title, publishers often choose to separate “backfile” (archive) use and

“frontfile” (current) use For the purposes of this study, we have, where possible, combined front- and backfile use under a single title

Because of these discrepancies, the methodology for collecting e-journal use data has had to adapt to new definitions and categories of use over time In 2000, the standard was to report the number of times a user from a selected institution attempted to access an article Within a few years, this changed so that the type of download (HTML or PDF) was reported as well as views

of Table of Contents and Abstracts Over the past decade, definitions of “use” have become more specific and therefore more precise, which has led to better, more precise counting and, therefore, less broad-brush counts even before COUNTER standards were adopted Currently, when available, we use COUNTER Journal reports 1 and 3: “Number of Successful Full-Text Article Requests by Month and Journal.” Report 3 adds data for the viewing of Table of Contents and Abstracts Although these “viewing” numbers are smaller when compared to the PDF and HTML download statistics, this report most closely replicates data provided beginning in 2000

However, as made clear by the many challenges facing longitudinal data collection during times

of change, journal use data are not perfectly matched across the decade of data collection and should be seen as best estimates that can help to identify trends

E-journal use data are collected from publishers’ Web sites for January, February, and March of each study year These months do not represent our busiest periods (which are mid-to-late

semester, i.e November and April) However, the methodology for these studies is remnant from the first (1988) study that recorded print use for six months, January to June The difficulty of this length of study led to a shortened version beginning in 1993, and since that time data have been collected for three months and doubled Those data are doubled again when compared to annual cost in order to establish a cost/use ratio The three month period chosen, although not our busiest time, may still result in a somewhat higher estimate of use than the actual use, but again, the numbers are best estimates and useful for comparing within the study and identifying trends Unfortunately, UIUC is not yet SUSHI compliant but is implementing Verde very soon It is

hoped that by the 2012 study, collecting and analyzing use data will be much easier,

incorporating COUNTER use data, SUSHI delivery, and local cost data from Verde

Table 2 Success rate for acquiring e-journal use data for individual titles from publishers in 2008 and 2010 The

“No Statistics Available” category includes Open Access journals.

Acquiring E-Journal Use Statistics 2008 2010

Table 2 shows the success rate for acquiring use data in 2008 and 2010 Although nearly 100 more journals have use data in 2010 than in 2008, not all titles are able to provide use data The

42 titles with “no statistics available” in 2010 are mostly open access journals; and the publishers

of these journals neither keep nor provide use statistics

Print use is shown with e-journal use in Table 3 to provide a baseline of journal use prior to electronic access These data show when our users switched to e-journals as their primary

method for accessing journal articles The methodology employed for measuring print journals is found in Chrzastowski (1991), Chrzastowski and Olesko (1997), and Chrzastowski (2003) Although they are both reported in Table 3, print journal use and e-journal use are not

comparable Print use is measured when a bound volume or an unbound issue is “used,” for example checked out, taken off the shelf, moved to a cart, or photocopied The print use

methodology cannot show how many articles within an issue or a volume were used, whereas e-journal use is specific to the exact article and type of view or download The convenience factor alone will cause e-journals to be used many times more than print, and combined with the more precise use measurement, e-use would be expected to exceed print many times over However, data on print use, although not comparable to electronic use, inform those data by providing another, distinct baseline measurement

Identifying the cost of a journal subscription is often more difficult than acquiring use data Journal subscriptions are purchased in “Big Deals,” through consortia, in series packages, tacked

on free with another subscription, and, sad to say, sometimes we have no idea how we are getting

a journal To determine pricing data, journal titles are first checked locally on subject-fund spreadsheets and against lists of titles purchased in package deals by publisher Ebsco is another source used to identify our local, annual pricing, and when every effort to find the price we paid fails, we use the standard price from the Ebsco database, which may or may not include our discounts As with use data, journal prices are best estimates that lead to identifying trends in cost/use ratios for each title we purchase When the database of titles is populated with use and cost information, the formula to calculate the cost/use ratio is simple: annual cost is divided by annual use and ranked by cost/use ratio with the most cost effective titles at the top of the list

Purposes for Data Collection and Analysis

Although recent technologies, including the Project COUNTER standards and access to more publishers’ data, have made the job of collecting, organizing, and analyzing e-journal use data easier over time, this process is still very time-consuming, difficult, and more than a little messy, and it results in an inexact product Even Project COUNTER data require local adjustments and the additional time of adding cost data Why continue to do it? The primary reason is

accountability These data provide the information needed to weigh price increases and to

estimate the local worth of the access price The resulting cost/use ratios are extremely important for justifying a very high-priced collection of journals These data provide welcome information when collection decisions must be made that reflect local use while maximizing collection dollars

The resulting product, in our case a ranked listing of cost/use ratios for the over 800 titles in the Chemistry Library’s e-journal collection, is extremely valuable The top titles rarely fluctuate from year to year, so most of the attention is focused at the bottom of the list These are titles that have little or no use Why? It may not be as simple as saying no one used them because they were not needed Are they indexed in SciFinder Web, PubMed, Web of Science, or Google? Was our access impeded or turned off during the year? Did the journal undergo changes in editorship, content, or focus? Is the country of origin undergoing strife or unrest leading to an interruption in publication? These questions are applied to titles that do not meet the Inside Web document delivery cost comparison discussed previously (a single article from that journal to be delivered

in 48 hours, a cost/use ratio averaging $80-$100), that is, use of this journal might be better

purchased on demand than subscribed to annually

Are there other data sets that can explicate journal use for a defined group of researchers? The

inherent problems with acquiring cost and use data have led to researching other available data

including Local Journal Utilization Reports and SFX linking data These e-measures also have

limitations but can verify information in collection evaluation and answer related questions about

journal use and use of indexes by specific database These products are discussed further in the

Results section of this paper

Results: What Do the Data Show?

Table 3 shows longitudinal journal use data for the UIUC Chemistry Library from 1988 to 2010

E-journal use data collection began in 2000, although only 15% of titles had publisher-supplied

use data at that time In 2010, that number has grown to 84% compliance, but this is still not the

100% one would expect By late 2006, when the library was remodeled and moved to a new,

smaller location, print bound journals duplicated in electronic format were moved off-site to a

storage facility and are paged if needed However, as Table 3 shows, by 2006 print journal use

was a mere fraction of total journal use, and this trend was firmly established as early as 2002

These longitudinal data, although imperfect, clearly show that the migration to e-journals by

chemists at UIUC was swift and decisive

Table 3 Print and e-journal use 1988-2010 for the UIUC Chemistry Library Print use data collection ceased in

2008 as many journals were moved off-site Print use data include ILL use of the collection (Chrzastowski, 2003)

Data are for six months; multiply by 2 for annual use

Print

Total use 31,703 46,984 42,490 44,650 95,824 343,644 680,581 849,288 863,620 943,066

There are a number of reasons why e-journal use increased dramatically between 2000 and 2006

As previously mentioned, statistical reporting by publishers did not reach a critical mass until

after 2002, and more publisher data increased journal use counts Table 4 shows the number of journal titles in each study period and the percentage of publisher-produced use data we could access for each study year These data, and the reasons for no data available in Table 2, suggest that we may never reach 100% compliance, especially with the increase in open access journals

Table 4 The number of e-journals in each study, the number of titles for which data were available and the percentage of data available for that study period.

Total Number of Chemistry

The period between 2000 and 2006 was also a time of major investment in backfiles and new electronic content at this library, as well as a time of improvement in journal Web sites,

bandwidth, and local networking speeds, which all helped to increase use as e-journals began to show promise in saving researchers’ time and energy Also increasing during this time were the number of open access journals and the prominence of Google and Google Scholar Convenient, free, and reliable indexing combined with more advanced UIUC linking capabilities (SFX) increased e-journal traffic significantly until the plateau that began in 2006 and continued in

2008

The usage plateau that took place between 2006 and 2008 was short-lived and looks to be partly attributable to a plateau in the number of titles in each study In contrast, use rose by 9.3% in

2010 (compared to 2008) This rise may seem to be attributable to an increase in the number of journals in the study, increasing from 836 titles in 2008 to 959 titles in 2010 (Table 4) However, looking closer at the data for individual journals and publishers shows that online use of the major journals did, in fact, increase

The Table 4 data also reflect that we are not at 100% compliance in obtaining publisher-based use data Although most major publishers (28 of 30 in the 2010 study) are providing data in COUNTER format, there are still small, often single-journal publishers that don’t provide use data at all, in any format Since we are more likely to cancel a journal with no statistics than low