Data Matters for User Service: Acquiring & Assessing the Impact of NYPL’s Money Matters Grant-Funded Initiative

Evaluating Money MattersGoal: To create a corps of frontline staff equipped with the skills & confidence to meet the growing need among their patrons to access timely, accurate & trus

Data Matters for User Service: Acquiring & Assessing the

Impact of NYPL’s Money Matters

Grant-Funded Initiative

Marie L Radford

Chair, Department of Library & Information Science

& Associate Professor Rutgers University, NJ

Kristin McDonough

Director, Science, Industry, and Business Library,

The New York Public Library

Library Research Round Table American Library Association Conference

Chicago, Illinois June 27 th -July 2 nd , 2013

The New Normal

Agencies Mandated to

Develop Consumer

Education Programs

Pre-Training Survey

of Branch Staff

 Why

 How

 What We Found

Grants: Acquisition &

Execution

Not about the money…

but the meaning

Opportunity lookout

Chat before

submission

Pitch & pivot

Define scope

Manage RFP process

Negotiate and collaborate with contractor

Engage internal stakeholders

Evaluating Money Matters

Goal: To create a corps of frontline staff

equipped with the skills & confidence to meet

the growing need among their patrons to access timely, accurate & trustworthy information

about personal finance.

Evaluation

IMLS requirement

RFP by NYPL to recruit outside evaluator

Measure MM’s success in meeting goals for:

Individual workshops (content & format)

Instructors

Overall program

Money Matters Pro Program

Staff Training Portal

http:// bit.ly/NYPLMoneyMatters

Evaluation Design

Mixed-Methods

Quantitative & Qualitative

Instruments designed with NYPL input

Surveys

Pre-Training Survey

Immediate Post-Training Survey

Final Post-Training Survey

Observation

Focus Groups

Website Analytics

Mix of paper & online (Survey Monkey)

Staff cooperation vital

Understand need & purpose

Paper successful for individual session evals (100% return rates)

Online useful for Pre & Post surveys

Incentive for Final Survey (gift card raffle)

Money Matters

Workshops

Workshop Titles (3 core, 8

electives, … 10 eLearning Modules)

Banking (core)

Retirement Planning (core)

Credit & Debt 1 (core)

Credit & Debt 2

Identity Theft

Investing 1

Investing 2

Paying for College

Income Tax

Insurance

Living on the Financial Edge

Immediate Post-Training

Survey Results

Completed at end of each training workshop

Nearly 100% return rate (administered on-ground)

Uniformly excellent results (means 5.0 to 4.0)

Quality of instruction

Content/scope

Activities/scenarios

Resources/handouts

Likert scale = Strongly Agree (5) to Strongly Disagree (1)

Immediate Post-Training

Survey Results

“Before this workshop, how prepared were you to answer patron questions?

 3.0 to 2.3 (mean, on several PF topics)

“After completing this workshop how well prepared are you to answer patron

questions?”

 5.0 to 4.0 (mean, on same PF topics)

Reported all workshops practical, relevant,

enjoyable, excellent instructors, & useful in daily interactions with NYPL patrons.

Survey Results: Pre-Training (n=

90, 69%)

vs Final Post-Training (n= 95,

73%)

Confidence in knowledge & understanding

Personal Finance

2.6 (Pre-Training mean) rose to 3.9 (Post-Training mean)

Adequate training in locating quality PF resources

2.7 (Pre) rose to 4.2 (Post)

Adequate training in evaluating PF resources

2.7 (Pre) rose to 4.1 (Post) Would welcome PF questions

2.8 (Pre) rose to 3.8 (Post)

Staff Voices on Money Matters Program

“Suggested to a single mom that starting to

save now for her child's college education is

beneficial because of the time value of money.”

“Helped a patron who was a grandfather find information on how to pay his granddaughter's college costs and how to ask financial

questions.”

“Helping immigrants at the adult learning

center set up bank accounts and helping them

to improve their financial knowledge.”

“I understand investment company information and reports better since I'm more comfortable with financial terms.”

 Critical factors in successful MM program

Clear vision of need & goals

Talented/qualified instructors

NYPL involvement in content & design

Staff cooperation in evaluation

Feedback & adjustments

Early observation

Mid-point Focus Groups

Interim reports

Acknowledgements &

Contact

This is one of the outcomes from the project

Money Matters Financial Education Program

Funded by the Institute for Museum and

Library Services (IMLS) and NYPL

Special thanks to: Marzena Ermler, T.J

Woods, Gretchen Smith, Brandy McNeil,

Barbara O’Neill, & Carole Glade

Staff training portal Money Matters Pro

http://bit.ly/NYPLMoneyMatters

http://www.nypl.org/moneymatters