output and efficiency in the production of business and economics majors

University of Nebraska, 2007 Advisor: William Walstad A linear education production function was estimated to identify the educational factors that contribute to the production and reten

Presented to the Faculty of

The Graduate College at the University of Nebraska

In Partial Fulfillment of Requirements

For the Degree of Doctor of Philosophy

Major: Economics

Under the Supervision of Professor William B Walstad

Lincoln, Nebraska

May, 2007

3263485 2007

Copyright 2007 by Asarta, Carlos

UMI Microform Copyright

All rights reserved This microform edition is protected against unauthorized copying under Title 17, United States Code.

ProQuest Information and Learning Company

300 North Zeeb Road P.O Box 1346 Ann Arbor, MI 48106-1346 All rights reserved.

by ProQuest Information and Learning Company

Carlos Asarta, Ph.D

University of Nebraska, 2007 Advisor: William Walstad

A linear education production function was estimated to identify the educational factors that contribute to the production and retention of the core business knowledge and basic academic abilities of graduating seniors at the college level The data set used in this study was comprehensive and included information on the standardized test scores, demographic characteristics, ability levels, transfer status, major areas of study and core business course performance of 689 graduating seniors from the College of Business Administration (CBA) at the University of Nebraska-Lincoln (UNL)

The production and retention of core business knowledge was influenced by a number of demographic, ability and transfer variables Male students outperformed females in all four Major Field Test in Business (MFT-B) models, suggesting that gender

is a significant factor in the production of core business knowledge Other significant demographic factors included the age, ethnicity/race and nationality of graduating seniors Entry SAT scores and core GPAs were highly significant in explaining the production of core business knowledge, while the transfer of core business courses from outside institutions negatively influenced the performance of students on the MFT-B Economics major were the only students to exhibit a positive and significant MFT-B point advantage, while marketing students were the only major to score significantly lower than their business peers The performance of students in the Principles of

significantly higher MFT-B scores The transfer of Statistics, Principles of Accounting II and Business Law was detrimental to the production of core business knowledge Finally, all majors but economics were less efficient at retaining core business knowledge when they transferred at least one core business course from an outside institution

The basic academic abilities of graduating seniors were unrelated to a student’s age or gender White students, however, tended to exhibit significantly higher exit ability levels than students from other races/ethnicities A student’s nationality and entry SAT scores were not found to significantly improve his/her basic academic abilities Student performance in non-core courses, however, consistently explained student scores on the Collegiate Learning Assessment (CLA) test The performance of students in Principles of Macroeconomics and Principles of Marketing positively influenced their exit academic abilities, while the transfer of the Business Law course offered at UNL was the only course transfer to influence the basic academic abilities of graduating seniors in a negative and significant way

I would like to thank all those who have contributed to my success at the University of Nebraska-Lincoln during my undergraduate and Ph.D studies:

Professor William Walstad introduced me to the rich world of economic education and provided invaluable expertise in the field Professor Walstad was always supportive of my dreams financially, intellectually and morally and stood by my side all along the way Thank you, Professor Walstad, for your guidance and support

The members of my committee: Dr Sam Allgood, for introducing me to the world of economics and providing guidance and support through my years at UNL; Dr Craig MacPhee, for inspiring me to specialize in the field of international economics and providing timely feedback on my dissertation; and Dr Fred Luthans, for always supporting my endeavors

The faculty of the Department of Economics at the University of Lincoln: Dr John Anderson for extending a teaching assistantship to me and for believing in my intellectual and personal abilities; Dr John Austin, for being a great mentor, colleague and friend; Dr Roger Butters and Dr Tammie Fischer, for extending numerous professional opportunities through the Nebraska Council on Economic Education and the UNL Center for Economic Education, and to all of those who believed

Nebraska-in me and have made a difference Nebraska-in my academic and personal life

sources for the data used in this study Sharon Nemeth was instrumental in proofreading and formatting the various documents included in this dissertation

Me gustaria darle las gracias a mis queridos padres, Alberto y Clara, por darme la education, ayuda y cariño necesarios para poder completar este sueno Espero que esteis orgullosos de vuestro hijo

Finally, I would like to thank my wife, who has been supportive of my dream of becoming a doctor since day one Your support, love and care for our family have allowed us to overcome the many obstacles encountered through the past five years This dissertation is dedicated to you and to our beautiful children, Cristian and Kenedi

List of Tables i

Chapter 1: The Nature and Objectives of the Research 1

Chapter 2: Literature Review 7

2.1 The Production Function Model 7

2.1.1 The Education Production Function 8

2.1.2 Educational Production Inputs 10

2.1.3 Educational Production Outputs 11

2.1.4 Conceptual, Methodological and Empirical Issues 13

2.2 Outcome Measures and Assessment 18

2.2.1 The Major Field Test in Business (MFT-B) 19

2.2.2 The Core Curriculum Assessment Program (CCAP) 22

2.2.3 The Collegiate Learning Assessment Instrument (CLA) 24

2.2.4 The Association to Advance Collegiate Schools of Business (AACSB) 26

2.3 Findings on Factors Affecting Educational Outcomes in Economics and Other Business Disciplines 28

2.3.1 Gender 29

2.3.2 Ability 32

2.3.3 Race and Ethnicity 35

2.3.4 Age and Class Standing 39

2.3.5 Transfer Status 41

2.3.6 Business Major 44

2.3.7 Course Grades and Overall Business Performance 47

2.4 Concluding Comments 49

Chapter 3: The University of Nebraska and the College of Business Administration 50

3.1 The University of Nebraska-Lincoln 50

3.2 The College of Business Administration 57

3.2.1 The Business Senior Assessment Course (BSAD098) 61

3.2.2 Common Body of Knowledge Topics and Sequence 65

3.3 Concluding Comments 66

Chapter 4: Variables and Sample 68

4.1 Dependent Variables 68

4.1.1 MFT-B Score (MFTB) 68

4.1.2 CLA Score (VADDCLA) 69

4.2 Independent Variables 69

4.2.1 Student Gender (MALE) 71

4.2.2 Student Age (AGE) 72

4.2.3 Ethnic Background (WHITE, ASIAN, OTHER) 72

4.2.4 Student Citizenship (ORIGIN) 73

4.2.5 Transfer Status and Credits (TRANSFCORE/OTHER,

TRANSFCORECR) 73

4.2.6 SAT Score (SAT) 74

4.2.7 Overall, Core and Other Grade Point Averages (GPA, CORE/OTHERGPA) 74

4.2.8 Student Major 76

4.2.9 Course Grades in the Common Body of Knowledge 76

4.2.10 Common Body of Knowledge Course Transfer (TRANSF+COURSE) 78

4.2.11 Student Major and Transfer Status (MAJOR+TRANSFCORE) 78

4.3 Descriptive Statistics 78

4.3.1 Descriptive Statistics on MFT-B Scores 82

4.3.2 Descriptive Statistics by Major 87

4.3.3 Descriptive Statistics on CLA Scores 92

4.4 Concluding Comments 97

5.1 Main Variables and Descriptive Statistics 100

5.2 Estimation of MFT-B Models 105

5.2.1 Model 1: Choice of Major Effects on MFT-B Performance 105

5.2.2 Model 2: Core Course Achievement Effects on MFT-B Performance 111

5.2.3 Model 3: Transfer of Core Courses Effects on MFT-B Performance 122

5.2.4 Model 4: Choice of Major and Transfer Interaction Effects 131

5.3 Estimation of CLA Models 133

5.3.1 Model 5: Choice of Major Effects on CLA Performance 135

5.3.2 Model 6: Core Course Achievement Effects on CLA Performance 139

5.3.3 Model 7: Transfer of Core Courses Effects on CLA Performance 141

5.4 Concluding Comments 144

Chapter 6: Overview and Conclusions 147

6.1 Literature Review 148

6.2 Results 151

6.3 Implications 157

6.4 Limitations 162

References 165

Appendix 2.1: Major Field Test in Business Sample Questions 176

Appendix 2.2: Major Field Test in Business Content 181

Appendix 2.3: Standards for Business Accreditation 185

Appendix 3.1: CBA Senior Survey 192

Appendix 3.3: Required Core Business Courses, UNL 202

Appendix 4.1: Standard ACT to SAT Table 205

Appendix 5.1: Correlation Coefficients for Models 1-4 206

Appendix 5.2: Transfer Intensity on MFT-B Performance 210

List of Tables

Table 3.1: UNL Undergraduate Enrollment by College and Standing, Fall 2006 53

Table 3.2: UNL Undergraduate Enrollment by College and Ethnicity, Fall 2006 54

Table 3.3: UNL Headcount Enrollment by Class Standing and Gender, Fall 2006 56

Table 3.4: UNL Enrollment by Age, Fall 2006 57

Table 3.5: Major and Total Credit Hours Graduation Requirements 61

Table 4.1: List of Main Variables 70

Table 4.2: Letter Grade and GPA Quality Points 75

Table 4.3: Descriptive Statistics 79

Table 4.4: Descriptive Statistics on MFT-B Scores 83

Table 4.5: Difference in Mean MFT-B Scores by Demographics and Ability 86

Table 4.6: Difference in MFT-B Scores by Major 87

Table 4.7: Descriptive Statistics by Major 88

Table 4.8: Descriptive Statistics on CLA Scores 93

Table 4.9: Difference in Mean CLA Scores by Demographics and Ability 96

Table 4.10: Difference in Mean CLA Scores by Major 97

Table 5.1: List of Main Variables 101

Table 5.2: Descriptive Statistics (n = 689) 104

Table 5.3: Choice of Major Effects on MFT-B Performance 107

Table 5.4: Core Course Achievement Effects on MFT-B Performance 113

Table 5.5: Core Course Achievement Effects on MFT-B Performance by Major 118

Table 5.6: Transfer of Core Courses Effects on MFT-B Performance 124

Table 5.7: Transfer of Core Courses Effects on MFT-B Performance by Major 127

Table 5.8: Choice of Major and Transfer Interaction Effects 132

Table 5.9: Descriptive Statistic (n = 191) 134

Table 5.10: Choice of Major Effects of CLA Performance 138

Table 5.11: Core Course Achievement Effects on CLA Performance 141

Table 5.12: Transfer of Core Courses Effects on CLA Performance 143

Appendix 4.1: Standard ACT to ACT Conversion Table .205

Appendix 5.1: Correlation Coefficients for Models 1-4 206

Model 1 206

Model 2 207

Model 3 208

Model 4 209

Appendix 5.2: Transfer Intensity on MFT-B Performance 210

Chapter 1

The Nature and Objectives of the Research

The production of education is characterized by choices derived from scarcity of resources Due to recent declines in public funding, the burden of education has increasingly fallen on students and parents, and a greater emphasis has been placed on streamlining and improving the efficiency of the educational process by carefully selecting and using the available educational inputs to maximize the creation and retention of knowledge Declining enrollments and shrinking market shares have also created added pressures for legislators and school administrators As a result, universities and colleges are expected to assess and continuously improve the quality of their programs, and accrediting institutions have gain importance in the world of academia (Becker and Andrews, 2004) Generally, the efficiency of educational inputs and the returns to human capital investments are measured with quantitative indicators of institutional, program and student performance (Cohn and Geske, 1990) Nichols and Nichols (2000a) indicate that program assessment should focus on concrete, verifiable results, such as how much students have learned upon graduation The recent use in education of standardized testing instruments has open the doors to new research that could further clarify and identify key inputs in the production and retention of knowledge

The main purpose of this study is to identify the educational inputs that have a statistically significant effect in the production and retention of the core business knowledge and general basic abilities of graduating business students An educational production function was used to perform the production and retention analysis The

production function approach enabled the identification of more efficient inputs and hence more effective production and selection processes Previous published studies using standardized testing instruments as the measure for the evaluation of business programs and student performance have sought to identify key educational factors while using narrow data sets and incomplete production functions (Allen and Bycio, 1997; Bean and Bernardi, 2002; Black and Duhon, 2003) The data set used for this study gathered information from the academic records of 689 graduating seniors from the University of Nebraska-Lincoln (UNL) These records provided vital information on the demographic characteristics, ability levels, transfer status and student achievement in what is known as the “Common Body of Knowledge” sequence of courses Student results from the Major Field Test in Business (MFT-B) and the Collegiate Learning Assessment instrument (CLA) were used as the output measures to draw the conclusion inferred from the cross-sectional analysis presented in this study

The first question that this study seeks to answer focuses on the production of knowledge by graduating business students First, do students who major in a specific business area generate more educational output, in terms of core business knowledge, than similar students who major in other areas within the business curriculum? And if this

is the case, are there other significant factors involved in such production? In other words, are accountants more productive, in terms of their core business knowledge and basic general abilities, than economists after controlling for demographic characteristics, ability levels, and transfer status? There is a possibility that a student’s major may not be

a determinant in explaining student performance on the MFT-B and CLA exams, or that the difference between the core knowledge and basic abilities accumulated by graduating

seniors from different majors not be significant The repercussions of such findings could have a direct impact in labor markets An accounting graduate could become as desirable

as a finance graduate to a prospective general business sector employer if in fact accountants and finance students generate similar amounts of core business knowledge and exhibit similar general ability levels after the completion of their undergraduate programs On the other hand, if certain majors are found to generate statistically larger amounts of core business knowledge before graduation, such information should be made available to students and the majors should be promoted by departments and colleges

The contributions of specific business courses to the production of core business knowledge by undergraduate business students are of special interest in answering the first question asked in this study The identification of significant courses would allow administrators to place more emphasis and increase the requirements in those specific classes so as to improve the student production of core business knowledge and their basic general abilities A secondary incentive for institutions to promote learning in these specific courses includes gaining faster membership or continuous accreditation with their accrediting agencies (i.e The Association to Advance Collegiate Schools of Business (AACSB))

In recent years, universities in the United States have seen an increasing flow of students transferring from community and junior colleges, which could be detrimental to the production of knowledge if such institutions provide a lower undergraduate education than, for example, research institutions (Vincow, 1997; Noll, 1998) At the same time, even those implanted into four-year systems seem to attend different universities during their undergraduate experience One obvious result of this trend is the heterogeneous

nature of the preparation of students for middle and higher level business courses at four- year institutions Previous research in economics and other business disciplines has generally found transfer students at a disadvantage when compared to their native peers (Borg, Mason and Shapiro, 1989; Laband and Piette, 1995; Borde, Byrd and Modani, 1996; Borde, 1998) Researchers have also tried to account for the ability of the students and whether the transferred hours were from a two- or four-year institution in order to shed more light on the effect of transferring courses on undergraduate business performance This study, however, attempts to answer a question that has not been explored in the previous business research: Does the transfer of core courses from outside institutions impair the production of core business knowledge by graduating seniors at four-year institutions? And if this is the case, which courses have a more significant impact on the production of knowledge when transferred? The expectation is that students who complete their core business education in the same four-year institution will produce more core business knowledge than those who chose to enroll and transfer courses from other institutions

The second question that this study seeks to answer is concerned with the effectiveness of graduating seniors in retaining the core business knowledge and basic abilities that they acquired throughout their business education, after controlling for demographic characteristics, ability levels and transfer status Students who enroll in higher education receive basic business training in their first years of attendance Many universities require their students to complete several standardized tests before graduation, including the MFT-B, to assess the amount of knowledge that they have been able to accumulate and retain over their higher education experience The performance in

these standardized exams is, in a way, a measure of how efficient students are at maintaining their basic business and general ability levels because there is a time-lag between the moment they are presented with the materials and the time when they have

to take the assessment instruments In this study, the efficiency of the different majors offered at UNL in retaining basic knowledge will be examined Of special interest is the retention of knowledge for the group of students majoring and minoring in economics The limited availability and recent use of comprehensive business outcome measures, and the reduced number of students graduating with economic degrees has made it impossible for researchers to answer this question Unlike previous research, the dataset used in this study is large and comprehensive, but the number of students majoring in economics is still relatively small Information regarding the minors of graduating seniors, however, is available and will be included in this study Students minoring in economics are required

to enroll in the same general economic courses as economic and business majors, but differ from all other business student because they receive economic training beyond the basic business requirement There are no known published research studies in the area of economic education using production functions where the educational output is the performance on the MFT-B (or any other comprehensive business output measure) and the educational inputs belong or are related to students majoring or minoring in

economics

Finally, this study will measure efficiency by identifying the factors that contribute to higher levels of basic academic ability after controlling for initial levels of general ability, demographic characteristics, other ability measures, transfer status and student majors The Collegiate Learning Assessment instrument (CLA) is an innovative

internet testing instrument designed to simulate complex, ambiguous situations that every successful college graduate may one day face in the form of written communication, critical thinking and analytical skills The CLA is not specifically designed for business students, but places an emphasis on everyday business situations The CLA provides a scaled SAT score for each individual student, and scaled scores take into account entrance SAT scores to assess the educational value-added of a student’s academic experience This approach is innovative for two reasons First, the presence of an overall value-added measure in business education studies could not be found in the literature Previous studies tend to focus on pre- and post-test results in single courses or areas of specialization to assess the value-added of education Most importantly, many in the academic world argue that a multiple-choice test may not be the most appropriate and meaningful assessment measure for students and their programs because such testing instruments fail to assess the skills, attitudes and problem solving capabilities of student and tend to simply focus on the measurement of cognitive knowledge in an specific field

of study The two outcome measures used by institutions to assess the overall performance of business students, namely the Core Curriculum Assessment Program (CCAP) and the MFT-B are multiple-choice instruments The CLA will allow for the comparison of the results arising from two fundamentally different comprehensive testing instruments and shed more light on the factors contributing to the production and retention of the basic general abilities of graduating seniors

Chapter Two

Literature Review

This chapter is organized as follows The first section describes the theoretical model that will be used to assess the effect of different educational inputs on the production of academic business knowledge An overview of the inputs and outputs used

in the literature, as well as a summary of the main conceptual, methodological and empirical issues frequently encountered in production function studies can be found in this section Section two presents several standardized outcome measures available to business schools to assess the overall performance of their students and programs A review of the Association to Advance Collegiate Schools of Business International is included in this second section The chapter concludes by examining the main student characteristics that have been studied in previous education production models The emphasis is placed in the economic education literature but reference is made to other business areas

2.1 The Production Function Model

The 1964 “Coleman Report” was the first and most influential educational production function study ever conducted It included information on over half a million students and more than 300 schools and concluded that traditional school inputs, as reflected by per pupil expenditures, class-size and certain teacher attributes have minimal effects on student achievement Since then, many researchers have attempted to utilize production functions to estimate the relationship between educational inputs and student achievement, both at the pre-college and college and university levels Considerable

confusion remains about how such studies should be conducted and interpreted, as well

as what can be learned from them (Hanushek, 1978; Becker, 2004) More importantly, there seems to be a series of conceptual, methodological and empirical problems that have “shadowed” previous research findings and conclusions in the area of education production functions

This section is organized as follows First, the reader can find a brief description

of the theoretical model that will be used to assess the effect of different educational inputs on the production of academic business knowledge An overview of the inputs and outputs used in the literature, as well as a summary of the main conceptual, methodological and empirical issues frequently encountered in production function studies follows

2.1.1 The Education Production Function

In education, the production function is some mathematical relationship describing how educational resources (inputs) can be transformed into educational outcomes (outputs) (Cohn and Geske, 1990) The production function then represents the maximum amount of output that can be produced for given levels of inputs and its general form can be expressed as

For the purposes of this study, the inputs and outputs are related to undergraduate

students and education in universities Accordingly, in equation 1, Q is a vector of

educational outputs (i.e standardized test score), X is the vector of university related inputs and S is the vector of non-university inputs The internal process of transforming

the educational inputs into output is known as the technology of education This process

is influenced by different variables related to the educational process such as pedagogical techniques or management

Although a considerable body of literature has attempted to identify and estimate the “best” educational production function, most research in the area of economic education has used a linear production function to estimate the effectiveness of educational inputs because of its ease of manipulation Cohn and Geske (1990) note that

a linear specification would be empirically valid to the extent that the curvature of the total output function is only mildly violated by employing a linear approximation (p.167)

In other words, it is important that the approximation be done in a range where diminishing marginal returns are mild; linear approximations are not valid if the range being observed belongs to an area where diminishing marginal returns are considerably strong Cohn and Geske also point out that the conclusions derived from the use of linear analysis should not be applied to input levels beyond the range of the sample observation

The general form of the ith educational production function using a linear

approximation to the production of knowledge is given by:

i j m

j ij h

k

h ih g

In equation 2, a is the intercept, while i b ig s,c ih s and d ij s are the coefficients we wish to estimate, with b ii =0and e being the stochastic term (Cohn and Geske, 1990) i

2.1.2 Educational Production Inputs

There are two general types of inputs that have been identified and used in previous education studies in an effort to estimate educational production functions Most

of the early work with educational production functions focused on the pre-college level

In this case, there are inputs provided by the school (school inputs) and those that are innate or provided to students through their homes or societal interaction (non-school inputs) This pre-college approach to education production is similar to the university approach used for this study

University inputs can be divided into human inputs such as teacher characteristics

or salaries, and physical inputs such as the condition of universities Most of the research using university inputs has been focused on human inputs because a large fraction of universities’ budgets is spent on the teaching staff, making the efficiency of the “labor” component of educational production a main subject of study

The introduction of non-school inputs into the production process is valid because few would argue that the formation of knowledge by students subject to the educational process is influenced by factors other than those provided by the school The largest, most comprehensive and most hotly debated study of educational production functions (Equality of Educational Opportunity, 1966) directed by James S Coleman corroborated this idea by coming to the conclusion that traditional school inputs as reflected by per pupil expenditures, class-size and certain teacher attributes have minimal effects on

student achievement This influential study ignited an intense debate in the areas of education and forced researches to look beyond traditional school inputs when trying to estimate educational production functions Watts (1985) included a “poverty index” variable while testing different specifications of an educational production functions for student belonging to more than two hundred classes in the state of Indiana Family income, number of books at home, the general characteristics of the student body, grade point average for a students’ section, family size, race or sex are some of the non-university inputs that have been frequently used in previous production function estimates (Cohn and Geske, 1990)

It is obvious that although some of the inputs are easily manipulable by university administrators (i.e course content) other are not manipulable (i.e age of students) and can not be controlled and changed to increase educational output The non-manipulative nature of such educational inputs makes the selection process of those entering the educational system, especially at the higher education level, even more relevant as they directly relate to the technical and internal process of generating educational knowledge

2.1.3 Educational Production Outputs

The two types of educational outputs that have been identified and measured in the economics of education literature are consumption and investment outcomes (Cohn and Geske, 1990) Consumption outcomes relate to the present utility that students, their families and society derive from the consumption of education On the other hand, investment outcomes relate to the future productive skills and well-being of society

Consumption outcomes include satisfaction from the direct involvement of students in activities offered by universities or intellectual satisfaction from learning new materials and skills Other consumption outcomes include family relieve of responsibility towards students during school hours, reduced crime rates and lower competition in job markets by restraining the supply of labor

The investment aspect of education focuses on the future benefits of the inculcation of social and moral values as they translate into more civilized and respectful societies Other examples of investment outcomes include the acquisition of basic communication and analytical skills, improvements in health habits and positive changes

in attitudes towards self, family, peers and society

Educational outcomes can be further divided into cognitive and non-cognitive outcomes (Cohn and Geske, 1990) From an economic perspective the classification is of little value because cognitive and non-cognitive outcomes provide both consumption and/or investment benefits Obvious examples of previously studied cognitive and non-cognitive outcomes include basic and vocational skills, creativity and attitudes.1Cognitive outcomes, however, have been extensively used in estimations of educational production functions over non-cognitive outcomes because they are less difficult to measure Cohn and Geske (1990) note that “because attitudes are difficult to quantify

1 Several studies examining higher education have extended the list of educational outputs by including variables pertaining to undergraduate teaching, master and doctoral graduate level of instruction and research productivity as measured by the number of publications or monetary spending in research activities (Verry and Layard, 1975; Verry and Davies, 1976; Psacharopoulos, 1980, 1982; Throsby, 1986; Lloyd et al., 1993; Johnes, 1993; Lewis and Dundar, 1995; Hashimoto and Cohn, 1997; Cruz et al., 2004)

[…] student attitudes have rarely entered a formalized educational input-output model” (p.165).Since the production of education differs from “industrial” production in that the educational industry generates multiple outputs, estimates of educational outcomes should include as many relevant and reliable measures of educational attainment as possible, including the widely used and available battery of standardize test score

2.1.4 Conceptual, Methodological and Empirical Issues

In theory, the production function represents the maximum achievable output for

a given level of inputs and firms make decisions on the optimal amount of inputs to use

in order to maximize their profits The question is whether production functions, as they are used in standard production, are a viable method for modeling the creation of educational output In reality, the application of production functions to education is complicated because the technological process of transforming inputs into output is generally not known and needs to be estimated through observation, the contribution of similar inputs to the creation of knowledge may vary widely and due to the heterogeneous nature of the produced output (individuals with different quality attributes)

Hanushek (1978) and Becker (2004) addressed conceptual and statistical issues related to research on the estimation of educational production functions and teaching methods They observed that student outcomes were generally measured through the use

of standardized tests, but that other plausible outcomes had been studied (i.e attendance rates and continuation or dropout rates) The use of different outcome measures, in conjunction with the variety of inputs introduced in studies of educational production

functions, appears to be one of the reasons why the conclusions of a large number of studies diverge from each other The educational system differs from the standard production system in that education generates heterogeneous outputs and a simple measure of cognitive level may not be appropriate to measure the total effect of receiving

an education Becker (2004) emphasizes this idea and adds that one of the main problems with the input-output approach is that education is a multi-product output that cannot be measured by a multiple-choice test score

Most educational production function studies have examined a single output measure and have used Ordinary Least Squares (OLS) regression analysis to assess the effect of a variety of factors on performance or socialization Unfortunately, OLS is generally an inappropriate statistical method in the presence of multiple outputs produced simultaneously Hanushek (1978) explains that in the case of the educational production

of two final outcomes it is possible to estimate them directly only if a set of alternative inputs and decision-makers’ valuation is available for each of the two outcomes However, the use of similar or identical inputs reflects the production technology and the choice between outputs, and the use of “reduced form” equations might be misleading

The author emphasizes that the importance of this issue depends on the degree of

“jointness” of production, the form of the production function, the variance of choices, the underlying decision rules for determining choices, and the accuracy of measuring inputs Becker (2004) suggests the use of a linear programming technique known as DEA

(data envelope analysis) to evaluate the efficiency of decision-makers when multiple outcomes are present

Using discrete and unordered responses as dependent variables in OLS estimation

of educational production functions is also problematic Becker (2004) suggests that researchers should consider the use of multinomial logit or probit estimation models to

draw any conclusions from their studies

The use of standardize tests to measure educational output raises concerns about their possible lack of external validity, that is, whether or not a test covers material, knowledge, or skills valued by society Such validity has been studied by looking at the effect that education has in labor market performance, socialization, job satisfaction, or a mother’s education effect on the learning of young children It is also possible that in essence, educational achievement or grades serve as a discriminating instrument or

“screen” that is used by employers to differentiate between those who receive a formal education and those who do not Along these lines, schools could produce more qualified individuals by improving their cognitive skills, which could be measured by using standardized test instruments, or simply identify the more able and serve as a screening device (Berg, 1970; Arrow, 1973; Stiglitz, 1975) In the later case, the output produced

by schools is information about the relative abilities of students

Becker (2004) addressed the issues related to the use of value-added measures to assess educational outcomes in input-output models Value-added measures of learning are calculated by taking the difference between pretest and posttest scores in an effort to adjust for lack of randomness in starting positions Becker states that the use of these

“change scores” variables of educational achievement in education production functions present several problems First, researchers must be concerned with “ceilings” when using standardized instruments or course grades (i.e 1600 points on the SAT or 4.00 on

course grades) to calculate “change scores” Ceiling effects are problematic because students with high pretest scores are unable to become measurably better The lack of normally distributed test scores when using change scores is equally important Becker

suggests using a Tobit model to handle ceiling effects, which involves an estimate of each student achieving the ceiling and then simultaneously adjusting the regression in accordance (p 278) Second, a practical problem arises when measuring change in scores

because some students drop out or are forced out between the pretest and the posttest Such data loss may bias the estimators in change-score models (Becker and Walstad, 1990) Finally, the value to students and society of a certain change in score is not clear,

as society tends to reward final accomplishments and not change Becker, however, indicates that score changes provide information regarding learning rates and could be of administrative value to teachers

The presence of multicollinearity in educational production functions, or small values due to high correlations among the explanatory variables, could yield insignificant estimates of the partial effect of the factors contributing to the production of educational output Further, in the presence of positive intercorrelations, the parameters will tend to

t-be negatively correlated so that the coefficients exhibit the opposite signs due to the correlation of variables Although these are important concerns, Hanushek (1978) suggests that the importance of collinearity may be overrated because the previous signs

of multicollinearity (low t-statistics and “wrong signs”) could not be the result of its presence and its importance may depend upon the statistical methods used This issue may however explain the variation in findings across educational production function models since researchers often determine model specifications based on coefficient

significance tests Becker (1983) implies that multicollinearity is more likely to be a sampling problem, rather than a problem of the population from which the regressors are

drawn He adds that unless a researcher can assess the trade-off between the possible introduction of biases and a reduction in mean square error, attempts to correct for multicollinearity maybe difficult to justify in economic education applied research (p 9)

Heij et al (2004) suggest that one method to improve the significance of the explanatory variable is to get more data, if at all possible, or drop some of the explanatory variables to explain the total but not partial effect of the educational factors In reality, Hanushek

concludes by noting that multiple regression analysis is used because there are correlations among “independent” variables (p 375) and the true diagnosis of the

problem is often times difficult

The level of aggregation is also problematic in estimating educational production functions The most relevant aggregation problem is related to the errors of measurement

as researchers find themselves creating models using individual student data and aggregate institutional data In reality, school factors do not affect all students in an

“average” way To imply that all students receive the same “average” allocation of resources, whether is technology use or student attention, is simply pervasive and past research has showed that students might receive different quantities of educational inputs while attending the same educational setting Hanushek (1978) notes, however, that although an important issue, researchers do not fully comprehend the importance of such variation, and suggests shifting the focus of attention to the accuracy of measurement inputs, which in his eyes is a much more important subject Errors in the measurement of the output variable using OLS provides for an unbiased estimator but a reduced overall fit

of the equation However, errors on the measurement of the independent variables biases the estimated coefficients and requires the use of instrumental variables correlated with the independent variables exhibiting the measurement error but uncorrelated with the error terms Becker (2004) suggests that it is important to recognize the need for one or two adjustments to get the correct standard errors, emphasizing that an adjustment for the number of observations making up the aggregate is required if the unit of measurement is aggregated

2.2 Outcome Measures and Assessment

Outcome measures are used in education to quantify and improve the educational output produced by business and other schools in the United States The AACSB requires prospective and current members to demonstrate the achievement of knowledge and skills that students acquire during their undergraduate degree programs (AACSB Standard 16, 2006) For this reason, the demand for reliable outcome measures has increased sharply in recent years However, a limited number of comprehensive business instruments have been used, namely the Major Field Test in Business (MFT-B) and the Core Curriculum Assessment Program (CCAP) instrument The Educational Testing Service’s Major Field Test in Business is the test of choice for a large majority of business institutions in the United States due to logistics, reported scores, and cost reasons

This section examines the two previously mentioned business instruments and describes a newly developed “value-added” test (the Collegiate Assessment Test-CLA) used to assess the written communication, critical thinking and analytical skills of any

two- and four-year institution graduate A description of the Association to Advance Collegiate Schools of Business (AACSB) is also presented in this section and its accreditation standards and steps are briefly overviewed

2.2.1 The Major Field Test in Business (MFT-B)

The development of the Major Field Test (MFT) began in 1989 by the Educational Testing Service (ETS) to serve as a measure of the basic knowledge and understanding achieved by senior undergraduates in their major fields of study The Major Field Tests were constructed according to specifications similar to those found in the GRE Subject Test The developers of the MFT used questions that had been previously used in GRE Subject Tests and added questions written by subject matter experts so that the instrument would be an appropriate measure of knowledge for all graduating seniors and not just those student planning to extend their school years by attending graduate school In the process of creating an unbiased instrument, developers

worked to eliminate language, symbols or content considered to be potentially offensive, inappropriate for any subgroup of the test-taking population or serving to perpetuate any negative attitudes that may be conveyed to these subgroups (ETS, 1994) Also, rigorous

statistical analysis was performed during pilot administrations to determine whether questions yielded expected results Ambiguous or otherwise unsuitable questions were replaced by more reliable questions Tests are currently revised approximately every five years by experienced teaching faculty and questions are rigorously tested for sensitivity and reliability by ETS assessment experts during each revision of the tests

According to the “Test Descriptions” manual (ETS, 2006) of the Major Field Test

in Business provided by ETS in their website,

Colleges and universities use the Major Field Test to measure student academic achievement and growth and to assess the educational outcomes of their major programs In addition, academic departments use the Major Field Test to evaluate their curricula and to measure the progress of their students The test also provides students with an assessment of their own level of achievement within a field of study compared to that of students in their program and to national comparative data

The Major Field in Business (MFT-B) is a two-hour multiple-choice test that contains 120 items.2 The testing instrument can be administered online or in a paper-based setting ETS charges 24 dollars per exam administered to institutions with more than 100 students taking the test The MFT-B attempts to cover both depth and breadth in assessing students’ level of achievement by using questions that represent a wide range of difficulty The main objective of the test is to cover areas included in what is known by business schools and their accrediting agencies as the “Common Body of Knowledge” Such “Body” is assumed to include a common business core formed with knowledge in quantitative analysis, marketing, management, law, finance, economics, accounting and social and international considerations of modern business operations Questions in accounting, economics, management, and quantitative business analysis and information systems each constitute about sixteen percent of the questions on the exam, while finance and marketing each constitutes twelve percent of the questions and legal and social

2 A list of sample questions can be found in Appendix 2.1

environment represents ten percent of the question on the test3 The economics section of

the MFT-B is divided in three main categories that include topics in Macroeconomics, Microeconomics and International Economics The Macroeconomics topics focus on national accounts and income determination, monetary and fiscal policy, the Federal Reserve System and employment, inflation and growth Microeconomics topics covered

on the exam include market structures (i.e monopoly), price theory and supply and demand Finally, the balance of trade and payments, international monetary systems, exchange rates, trade polices (including GATT, NAFTA and the European Union) and comparative advantage are the main topics covered in the international economics area of the Major Field Test in Business

There are two kinds of scores available for the MFT-B Each test produces an individually reliable total scale score for each student (reported on a 120-200 scale) and the mean scale score and standard deviation for the group of students tested Individual scores are reliable and can be used to make decisions about students based on their scores The MFT-B test also allows institutions to collect scores that relate to a subfield within a major field of study (assessment indicator scores or group reliable scores) Assessment indicators are reported for groups with at least five students and the scores are reported as a mean percent correct for the group in each of the assessment indicators

In other words, cohorts of student with a same major are created and the mean percent correct answers are reported for each of the subfields of the test (i.e economics) Assessment indicators are not reported for individual students Also, the Major Field Test only scores correct answers and does not penalize individual students for omitting or

3 A detailed list with the major topics covered in the test can be found in Appendix 2.2

guessing and answer to a question Finally, ETS offers the option to purchase an additional score report (ETS, Item Information Analysis Report, 2006) that provides a tally of the percent correct, percent incorrect and percent omitted for each of the 120 question on the test instrument Each of the questions is identified by subfield of study (i.e accounting) The Item Information Analysis Report is provided to institutions for the whole sample of students and it is not available by cohorts (ETS, 1994)

The reliability of the MFT-B is estimated using the Kuder-Richardson formula 20 Based on data from February 2003 to the present, the Kuder-Richardson coefficient is 89 A scale of reliability ranges from 0.0 to 1.0 The desired level of reliability for the reported total score is 90 or higher (ETS, Description of Test Results, 2006) A value of 89 indicates that the test can be used as a relatively accurate estimate of achievement for the purposes of assessing group performance

2.2.2 The Core Curriculum Assessment Program (CCAP)

A second measure of business student achievement that is available but has been rarely used in recent years is the Core Curriculum Assessment Program (CCAP) instrument Karathanos (November, 1991) examined and compared the CCAP to the MFT-B in terms of content, administration time, types of scores reported, reliability coefficients, national comparative data and costs and concluded that the Major Field Test

in Business has definite advantages, specially in terms of reliability, logistics, costs and scores reported

The CCAP was developed by the American College Testing Program (ACT) in

1987 in response to the need for outcomes assessment in business schools The then

called American Assembly of Collegiate Schools of Business (AACSB) contracted with the ACT to create an instrument capable of measuring the general business core knowledge of business students The test includes questions in each of the following areas: Accounting; Business Environment and Strategy; Finance; Human Resources and Organizational Theory; Marketing; Management Information Systems and Quantitative, Analysis/Operations, Research/Production and Operations Management The test bank includes 70 questions from each of the seven areas of specialization, for a total of 490 items available

The CCAP is a two-hour multiple-choice test that contains 70 items Scores range from 0 to a maximum of 70 points The test is divided into seven different forms, each containing 10 questions from each of the seven business categories Students are required

to take one of the pre-selected forms available The fact that business students who take the instrument at a single institution will end up answering different questions makes the CCAP instrument only useful for group comparisons, and the results reported by the ACT are provided uniquely as average institutional scores For this reason, Karathanos (January, 1991) suggests that the only option for using the CCAP instrument is to obtain value-added scores, and that great care must be exercised in selecting the entrance business group and graduating senior cohort so as to make sure that the representative samples are similar populations

Information regarding the initial administration and reliability of the CCAP (AACSB, 1987) reveals that the instrument was administered to an initial sample of 1,050 entering students and 972 graduating students in 14 institutions The reliability of the instrument was calculated for each of the 14 institutions by using the Kuder-

Richardson formula 20 Based on the data, the Kuder-Richardson coefficient ranged from 79 to 87 depending on the institution As it was pointed out in the previous subpart, the desired level of reliability for the total score is 90 or higher Reliability coefficients in the previous range indicate that the test may not be an accurate estimate of achievement for the purposes of assessing group performance The institutional report also provides information about the means for undergraduate and graduate students, standard deviations, and gain scores Examination of the gain scores for the initial administration sample revealed a large gain variance, with gains ranging from 1.97 to 19.43 point for undergraduate students, and from 5.71 to 12.42 for graduate students

2.2.3 The Collegiate Learning Assessment Instrument (CLA)

Business colleges in the United States are constantly experimenting with new assessment instruments in their effort to continuously review and improve the quality of their programs and maintain their accreditation status The Collegiate Learning Assessment (CLA) is an innovative internet testing instrument designed to simulate complex, ambiguous situations that every successful college graduate, regardless of their major, may one day face in the form of written communication, critical thinking and analytical skills The instrument was created by the Council for Aid to Education, a subsidiary of RAND Corporation, and it is administered nationwide by a variety of two and four year colleges and universities Since its creation in 2002, 134 colleges and universities have participated in the administration of this testing instrument and in the fall of 2005 alone over 18,000 students took the CLA (CLA in Context, 2004-2005)

The CLA is divided into two subgroups of instruments, Performance Tasks and Analytical Writing Tasks Each CLA Performance Task is made of a series of open-ended questions about hypothetical but realistic situations where students are required to use an integrated set of critical thinking, analytical reasoning, problem solving, and written communication skills to prepare an answer within 90 minutes No two CLA Performance Tasks assess the same combination of abilities For example, some tasks require students to identify and compare the strengths and limitation of different hypotheses while others may provide students with a conflict in which they have to suggest and select and argue a course of action Analytical Writing Tasks are divided into two types of essays, namely a “Make-an-argument” section where students need to support or reject a position on some issue, and a “Critique-an-Argument” or “Break-an-Argument” section where student are asked to evaluate the validity of someone else’s argument in a 45-minute time frame Students are expected to address an argument from any perspective by providing relevant reasons and examples to explain and support their views

CLA scores measure the “value added” an institution offers by indicating the degree to which a school’s students earn a higher or lower score than would be expected where the expectation is based on (1) the student’s admission test scores (i.e ACT or SAT scores) and (2) the typical relationship between admission scores and CLA scores across all participating institutions In other words, how well do students at a school do on the CLA tests relative to the scores earned by “similar students” (in terms of entrance examination scores) at other colleges and universities?, and by contrasting the performance of freshmen with seniors Specifically, after holding admission scores

constant, do an institution’s seniors earn significantly higher scores than do its freshmen and most importantly, is the difference larger or smaller than that observed by at other colleges? (CLA Institutional Score Report, 2004-2005) Because each testing instrument

is composed of a series of tasks that rank in difficulty, there is a separate scoring guide

for each of the tasks and scores are converted to “scale” scores In technical terms, this process involved transforming the raw scores in a measure to a score distribution that had the same mean and standard deviation as the ACT/SAT scores of the students that took the measure In non-technical terms, this type of scaling essentially involves assigning the highest raw score that was earned on a task by any freshman the same value as the highest ACT/SAT of any freshmen who took that task (i.e not necessarily the same person) The second highest raw score is then assigned the same value as the second highest ACT/SAT score, and so on A freshman and a senior who earn the same raw score on a task were assigned the same scale score for the task (CLA Institutional

Score Report, 2004-2005)

2.2.4 The Association to Advance Collegiate Schools of Business (AACSB)

The Association to Advance Collegiate Schools of Business (AACSB) was founded in 1916 to promote continuous quality improvements in management education

It was in 1919 that the standards for business administration were created AACSB is a not-for-profit corporation of educational institutions, corporations and other organizations devoted to the promotion and improvement of higher education in business administration and accounting As such, the association is recognized by the Council for Higher Education Accreditation (CHEA) AACSB accreditation for undergraduate and

graduate education programs in business administration and accounting assures quality and promotes excellence and continuous educational improvement The accreditation process follows a continuous chain of fifteen steps designed to flow smoothly and without disruption of the school’s progress The high standards set by the association ensure that only a small percentage of schools offering undergraduate and graduate programs in the United States are fully accredited.4

The AACSB revised its standards in 1994 in hopes of increasing the number of accredited schools The standards were relaxed in the area of research to allow schools that emphasize teaching the possibility of achieving accreditation These new standards allowed schools to be evaluated against their own missions instead of having to meet excellence standards in teaching, research and service A survey of business school deans

by Mayes, Heide and Smith (1993) found that the deans of already accredited schools felt that the degree of difficulty for admission to the Association had not change, while the deans of nonaccredited institutions saw the changes as being positive and helpful for their own accreditation Yunker (1998) also surveyed both groups of deans and reported that all deans, regardless of their accreditation status, believed that the new standards would increase the number of accredited schools Yunker also noted that the new standards were simply a response to the accreditation trend that had been taking place for the last 20 years, namely the fact that newly accredited schools tend to be smaller and have less PhDs on staff In response to the surveys, Jantzen (2000) examined and compared the composition and number of schools accredited by the AACSB since its foundation and the introduction in 1994 of the new “mission-related” standards and found that the

4 A detailed list of the current “Standards for Business Accreditation” can be found in Appendix 2.3

adoption of the new standards did not have an effect on the accreditation growth rate or the character of schools being accredited The author, however, emphasizes that the AACSB has become more inclusive and that this diversity may be detrimental to the distinctiveness of being accredited

According to the Eligibility Procedures and Accreditation Standards for Business

Accreditation manual (AACSB, 2006), Accreditation focuses on the quality of education Standards set demanding but realistic thresholds, challenges educators to pursue continuous improvement, and guide improvement in educational programs The 21

standards emphasize strategic management processes, processes and standards related to students, faculty members, and stake holders, and quality management and learning assurance Individual schools voluntarily apply for AACSB accreditation review and after the initial accreditation process is achieved (which includes a self-evaluation and a peer review), institutions starts a program of periodic review of strategic improvement progress to maintain its accreditation Initial accreditation must be achieved within the first five years following the acceptance of the Accreditation Plan

2.3 Findings on Factors Affecting Educational Outcomes in Economics and

Other Business Disciplines

Individual student characteristics are widely used as control or predictor variables

in education production function models to explain the performance of economic and business students The following section summarizes the findings of some of the previous research in economics and other business disciplines The review focuses mainly on undergraduate education, but relevant findings are also cited from the pre-college or