Since the introduction of the Skills for Life strategy in 2001, the evidence base on ALN has expanded greatly and now includes national data on ALN levels and needs from the 2003 Skills
conducting bibliographical searches and retrieving documents
Bibliographic searches to identify literature on ALN evaluation and ALN research were conducted by the NFER Centre for Information and Reviews NFER information specialists mapped database keywords to the review questions and finalised the search strategy in collaboration with the SQW and NRDC review teams The keyword sets were designed to cover concepts for each of the six themes, and a population-of-interest term set was included in all searches to ensure comprehensive coverage.
2.4 Three types of sources were searched to ensure thorough coverage of the evidence base within the available time:
a range of general bibliographic databases
publication and references lists compiled by subject experts and earlier literature reviews
Bibliographic database searches were limited to 2004–2011 for most sources, with a trial of the British Education Index covering 2000–2011 The trial produced such a large volume of hits that the team decided to set the start date at 2004 for other databases to ensure manageability while still capturing literature related to the 2003 Skills for Life survey To avoid overlooking important research from 2000–2004, an expert panel identified significant publications as well as essential pre-2000 sources.
Bibliographic databases and organisational websites included in the search are listed in Tables 2-1 and 2-2, with the search conducted using main keywords and the publications, research, and policy sections of each website browsed as appropriate to identify relevant material.
AEI is Australia’s largest source of education information covering reports, books, journal articles, online resources, conference papers and book chapters searched via Dialog Datastar 16/03/11
BEI provides information on research, policy and practice in education and training in the UK
Sources include over 300 journals, mostly published in the
UK, plus other material including reports, series and conference papers searched via Dialog Datastar 04/03/11
The free collections search interface of the BEI (formerly the British Education Internet
The Resource Catalogue provides access to a range of freely available internet resources, as well as records for the most recently indexed journal articles not yet included in the full BEI subscription database, with the search conducted on 17/03/11.
ERIC, sponsored by the U.S Department of Education, is the world’s largest education database, with coverage that includes research documents, journal articles, technical reports, program descriptions, evaluations, and curricula materials, and is searchable via Dialog Datastar (as of March 15, 2011).
Elsevier is the world’s leading publisher of science and health information searched via Elsevier 14/03/11
Emerald Emerald is a long established publisher with over 200 titles in the fields of management, information science and engineering searched via Emerald 14/03/11
Oxford Economic Papers is a general journal publishing papers in a wide range of areas in theoretical and applied economics searched via Oxford Journals 14/03/11
PsycINFO PsycINFO contains references to the psychological literature including articles from over 1,300 journals in psychology and related fields, chapters and books, dissertations and technical reports searched via Ovid SP 17/03/11
Social Policy and Practice is a bibliographic database of abstracts covering evidence-based social policy, public health, social services, and mental health and community health The content is primarily drawn from the UK, with supplementary material from the USA and Europe.
Searches were carried out across the descriptors, heading word, title and abstract fields, to enable retrieval of terms both as keywords and free text searched via Ovid SP 17/03/11
SSRN, the Social Science Research Network, is dedicated to the rapid worldwide dissemination of social science research and is organized as a collection of specialized research networks, each focused on a distinct social science discipline.
Table 2-2: Websites of key organisations searched
Department for Business, Innovation and
Skills (BIS) http://www.bis.gov.uk/
Confederation of British Industry (CBI) http://www.cbi.org.uk/
CfBT Education Trust http://www.cfbt.com/
Department for Education (DfE) http://www.education.gov.uk/
Excellence Gateway (Learning and Skills
Improvement Service) http://www.excellencegateway.org.uk/
Improvement and Development) http://www.idea.gov.uk/
International Labour Organization (ILO) http://www.ilo.org/
National Audit Office (NAO) http://www.nao.org.uk/
National Literacy Trust http://www.literacytrust.org.uk/
Organisation for Economic Co-operation and Development (OECD) http://www.oecd.org/
Scottish Government http://www.scotland.gov.uk/
Trades Union Congress (TUC) http://www.tuc.org.uk/
United Nations Educational, Scientific and Cultural Organization (UNESCO) http://www.unesco.org.uk/
US Commission for Skills http://www.skillscommission.org/
US Office of Management and Budget http://www.whitehouse.gov/omb/
World Bank http://www.worldbank.org/
Three bibliographic databases produced more than 1,000 items in total, with the Education Resources Information Center (ERIC) contributing more than 4,000 items NFER information specialists then conducted a manual screening of these results and shortlisted shorter item lists for consideration by the SQW and NRDC review teams Items were selected based on relevance to the research questions, alignment with one or more cross-cutting considerations, and an assessment of study quality by design—ranging from randomized controlled trials and other experimental studies to quasi-experimental, longitudinal, case study, ethnographic, and grounded theory research—and fidelity to the research methods appropriate to each design.
From the shortlisted results, the SQW and NRDC review team built a comprehensive database of unique source references, aggregating more than 1,600 entries Each record includes bibliographic data about the source and details on how it was identified—whether via a bibliographic database name, a specific website, or an expert recommendation—and notes which of the research review’s six themes the source pertains to.
2.9 The NRDC team reviewed each listed source and graded its importance as none, low, medium, or high Approximately 650 items were excluded from further review during this process, mainly because they were not relevant to the research review due to the subject, cohort (for example, school children), or geography (for example, adult literacy in developing countries) studied Sources prioritised as ‘low’.
Approximately 435 items were included, most of which centered on theoretical discussions and frameworks Other items were deemed peripheral to the main content by the review’s six themes Exclusions also applied to articles published in non-peer-reviewed publications (for example, sector-specific magazines), to sources where retrieving the publication was not cost- or time-effective, and to minor or interim study publications where more detailed or final reports were available elsewhere.
Many items labeled as medium or high priority were reviewed through online retrieval and a brief consultation of each source The medium-priority group—approximately 325 items—included sources for which reliable summaries were already available from sources graded as high priority; publications highly relevant to the subject matter but published before Skills for Life; small-scale studies with less robust findings and potentially limited applicability in other contexts; items with a strong focus on pedagogy and practice; and high-quality sources that addressed narrow themes (such as the learning persistence of women domestic workers, the use of Individualised Learning Plans, and Virtual Learning Environments in Scotland) High-priority sources were deemed highly relevant—focused on adults, literacy and numeracy, skills, learning, and interventions and their impact—and of high quality; the vast majority of these high-priority sources were already familiar to the research teams due to their subject-matter expertise.
2.11 Once sources had been provisionally graded, senior members of the SQW and
NRDC teams reviewed all high- and medium-priority items, incorporating notes entered in the database by the first-level review team to produce a slightly revised dataset The Expert Panel then evaluated the high-priority items, recommending additions or deletions as appropriate Reviewers also identified additional sources through team work and by citing literature reviews embedded in the process The final list comprises 175 high-priority items For many of these sources, the review team drew on reliable summaries or syntheses prepared for earlier literature reviews.
populating the review templates
A small team from SQW and NRDC reviewed the sources and prepared a template for each item, which was stored using QSR NVivo qualitative research software The template captured essential information, including the full bibliographic reference, the abstract, the research questions and methods, and the sources of evidence In the main body of the template, reviewers recorded findings pertinent to the six themes of the study, including sub-themes under 'quality and effectiveness of provision'.
‘learning outcomes’, and to the four cross-cutting considerations The template was also used to record the full bibliographic reference for any significant findings drawn from other sources.
analysing the review templates and drafting the report
Once templates for all sources under review were completed, they were themselves reviewed to draft the findings The analysis was carried out by a four-person team from SQW and NRDC—Michael Frearson and Stuart Johnson from SQW, and John Vorhaus and Jennifer Litster from NRDC Members of the Expert Panel were also invited to comment on an early draft of the review.
Outcomes of learning: economic returns
This chapter synthesizes recent quantitative evidence on the economic returns to improving literacy and numeracy, highlighting impacts on earnings and employment, training and promotion trajectories, and the development of job-specific and job-seeking skills It also analyzes the costs and benefits to employers and to the Exchequer associated with lower or higher levels of basic skills.
Research on the economic returns to learning is well developed, characterized by a large body of well-designed, rigorous econometric studies The evidence is conclusive that literacy and numeracy skills—when acquired—have a positive impact on earnings and employment Evidence on the returns to attaining basic skills in adulthood is less abundant, but the majority of studies indicate a positive effect on labour market outcomes.
(Patrignani and Conlon, 2011, is one example of this)
The National Child Development Study (NCDS) and the 1970 British Cohort Study (BCS70) are invaluable sources for exploring the relationship between economic and social outcomes and levels of ALN However, because only a small number of individuals in each birth cohort have taken ALN courses, these studies provide limited evidence on the effects of adult basic skills interventions Consequently, use of these sources is kept to a minimum in this chapter.
3.4 Econometric evidence is complex; whilst technicalities are kept to a minimum, they cannot be altogether removed without also making it impossible to assess the main findings
3.5 Beyond a number of general messages expert opinion is often divided, and this chapter therefore includes details from sources that do not always point towards the same conclusion
3.6 Owing to the complexity of the evidence, this section includes a tabular summary of evidence on wage and employment returns to gaining or improving literacy and numeracy in adulthood
Table 3-1: Map of UK evidence – attaining skills in adulthood
Impact of attaining literacy skills on earnings
Impact of attaining numeracy skills on earnings
Impact of attaining literacy skills on employment
Impact of attaining numeracy skills on employment
Metcalf et al, 2009 No No Inconclusive Inconclusive High levels of attrition
Yes Yes Yes: men Yes: women
Yes: men Yes: men n/a Yes: men
Yes Yes Yes Yes Consistent results from two data sets
Employment effects small/ statistically insignificant
BIS, 2011 Yes Yes Yes Yes McIntosh
Literacy and numeracy not distinguished
3.7 BIS (2011) reported on the economic value of government-funded qualifications delivered by the post-19 FE sector Because wage-return estimates vary and the literature offers conflicting evidence, McIntosh (2010) identified a set of premia that are as consistent as possible across approaches The model assumes that wage returns from Skills for Life rise with higher prior qualification levels, specifying a 3% wage premium for individuals with no prior qualifications and a 5% premium for those previously qualified to Levels 1–5.
3.8 Metcalf et al (2009) conducted a longitudinal study of literacy and numeracy learners taking Skills for Life courses in FE colleges leading to qualifications in
From 2002/03 to 2006, learners who took Skills for Life courses were compared with a similar group with low or no literacy or numeracy qualifications who did not participate in such courses; over Wave 1 to Wave 4, the learners’ net annual earnings rose by £560, compared with £542 for non-learners, a small difference that was not statistically significant Consequently, the study did not find evidence of wage returns to college-based literacy and numeracy courses; however, this does not prove that there are no wage returns Statistical significance is hard to establish due to sample attrition, potential bias from sample loss, and substantial fluctuations in earnings over the measurement period.
Using ILR attainment data from 2002/03–2005/06 and annual earnings data from 2003/04–2009/10, London Economics (Patrignani and Conlon, 2011) found sizeable earnings returns to Skills for Life qualifications at Level 1 In the short‑to‑medium term, literacy learning aims yielded higher earnings gains than numeracy aims (over 7% in the first three years post completion vs about 3% for numeracy), while numeracy returns increased more gradually from about 3% in year one to roughly 12% by year six For Level 2, numeracy qualifications produced immediate and persistent gains (about 1.7% immediately after attainment rising to over 13.7% after seven years) Level 2 literacy qualifications showed an initial earnings dip (around -2.5% for the first four years) before returning to positive effects, with gains of between 2.7% and 8.8% in the fifth to seventh years Attaining both Level 2 literacy and numeracy yielded a positive return after an initial first‑year dip, with earnings gains around 10% in years six and seven.
3.10 Evidence from Bynner and Parsons (2006) drawing from the two British cohort studies points to a strong relationship between poor basic skills and income
Compared with women who had a poor grasp of literacy or numeracy at both age
21 and 34, women who had improved their literacy or numeracy by age 34 were more likely to be generally better off and to have savings and investments
Men who improved their literacy or numeracy by age 34 were more likely to own their own home and less likely to rely on state benefits or to have borrowed money from a friend, a family member, or other sources, compared with men who had a poor grasp of literacy or numeracy at both ages 21 and 34.
Machin (2001) assessed the economic impact of skill improvements between age 16 and 37 using skill measurements taken at age 37 from the National Child Development Study (NCDS) Individuals who reported that their skills had improved generally earned more than those who did not believe their skills had improved For example, males who claimed to have improved their numeracy skills earned about 3% more than those who did not report such improvement.
Women who claimed to have improved their numeracy skills earned 11% more than those who made no such claim
Focusing on the wage effect of having Level 1 skills relative to below this level, numeracy skills appear to exert a greater effect on earnings than literacy skills However, when examining the change in skills from age 16 to 37, literacy skills have a larger wage impact, at least for men, particularly if the individual began at the upper end of the skills distribution at age 16.
The results imply that for adults with very low basic skills, boosting numeracy to at least Level 1 yields larger earnings gains than boosting literacy, while those with a stronger baseline see the greatest wage gains from further improving literacy rather than numeracy Consequently, attaining an acceptable numeracy level is key, whereas literacy benefits continue to rise with ongoing skill development (at least for males).
Using data from the National Child Development Study (NCDS) and the International Adult Literacy Survey (IALS), McIntosh and Vignoles (2001) estimated the potential earnings gains from boosting adults’ literacy and numeracy skills Their model controls for a range of factors that could be correlated with literacy and numeracy, including educational attainment, social class and background, childhood type of schooling, parental interest in education, regional differences, and baseline reading and mathematics ability on entering school When these controls are accounted for, the study estimates the wage premium associated with attaining higher literacy and numeracy levels.
Numeracy skills at Level 1 or above unlock a wage premium of about 8–10%, while Level 1 literacy skills confer a smaller premium of around 2–6% Two notable features stand out from these results: first, the two data sets—comprising distinct groups of workers of different ages—produce strikingly consistent findings; second, numeracy exerts a large impact on earnings, even after controlling for a wide range of other factors that might influence wages.
In the United States, Reder (2010) documented strong links between literacy proficiency and earnings among high school dropouts, a finding that replicates Tyler et al (2004b) and extends to the entire population of dropouts, not just GED examinees Analyses from the Longitudinal Study of Adult Learning (LSAL) show that dropouts can improve their basic skills after leaving school The initial level of literacy influences both the starting earnings and the rate of earnings growth over time, implying that literacy affects the economic returns on work experience, potentially by shaping access to continuing and postsecondary education or workplace training, and by influencing what individuals are able to learn and accumulate through work experience.
Literacy growth rates among individuals shape earnings, influencing both the starting level of earnings and the potential pace of earnings growth This association warrants further analysis, because literacy progress may reflect omitted variables that drive both literacy advancement and earnings trajectories—such as non-cognitive skills, motivation, and dispositional attributes.