2.2 Announcement-archival (category 2) studies
2.2.3 Methods of financial performance analysis
There are four methodological ideas that announcement-archival studies bring to bear on research into the lean practice-performance connection which are:
1. Comparison formation – Methods for creating the comparison between practice and non-practice companies and controls for business environmental effects.
2. Measure of central tendency – Selecting a measure of central tendency that serves to minimize problems associated with using non-normally distributed financial data in statistical analysis.
3. Time span of comparison – The number of continuous years used in the comparison and their possible separation in time, ensuring that performance differences are sustained and distinct.
4. Primary analysis method – The appropriate selection of analysis methods when using financial data in order to avoid violating inherent statistical assumptions.
2.2.3.1 Comparison Formation
Studies using archival data have several options for creating a comparison between practice and non-practice companies and range from within-subject longitudinal comparisons (that compare a single firm’s performance pre- and post-practice
implementation) to more aggregate comparisons between groups of practice and non- practice companies. Here, the ability of the comparison formation to control for factors
external to practice usage is important to the study’s validity and reliability since financial performance largely depends upon business environmental conditions. By developing a well-formulated comparison, one can minimize other sources of measurement variation. Several announcement-archival studies use longitudinal comparisons of the same company. Studies that compare pre- and post-adoption
performance with limited environmental controls focus on relatively short time frames in order to minimize the opportunity for dynamic business economic conditions to affect the results. The stock return studies of Hendricks and Singhal (1996) and Howton et al.
(2000), employ event study analysis with a baseline period beginning only 200 days prior to adoption announcement. Biggart & Gargeya (2002) and Billesbach & Hayen (1994b) also use a pre/post comparison, although they compare several years of performance pre- adoption to several years post-adoption.
In contrast to comparisons within the same company over time, there are longitudinal contrast studies that draw comparisons between groups of practice and non-practice companies. All announcement-archival studies included in this literature review that use this technique also compare performance longitudinally between two groups. These studies differ in how well they match practice group to control group. Chang and Lee (1995) match practice companies with control companies based on a 4-digit standard industry classification (SIC), and they compare differences between matched pairs prior to and following implementation. A similar matched pair difference analysis is used by Balakrishnan et al. (1996) who analyze matched pair difference between businesses with matching 3-digit SIC's and comparable net sales. In their follow-up to Balakrishnan et
al.’s 1996 study, Kinney & Wempe (2002) used a more stringent pairing criteria. They matched inventory valuation methods, net sales in the year preceding adoption, 4-digit SIC, and line up the adoption to within +/- 3 years although they do allow reducing the criteria to a two digit SIC code when necessary to find an adequate match.
Easton and Jarrell (1998) created their comparison through a matched portfolio of three companies. Their matching criteria included a detailed review of the company’s products in Value Line, similar time period, similarity in analysts' projections, and, to the “extent possible,” size as measured by total debt as well as market value of equity and preferred stock. The use of a matched portfolio provided a measure of robustness against
company-specific variance. The only other comparison control mechanism employed in the reviewed announcement-archival studies treats industry average performance
measures as control variables in performance regressing over time (Huson & Nanda, 1995).
To create a comparison portfolio, this study uses a 2-digit SIC code match and a size match to within +/- 50 percent of total asset size for a minimum of five companies. The Methods Chapter (4) offers a detailed examination of the development of the sample frame and the process of forming comparison portfolios.
2.2.3.2 Measures of Central Tendency
The appropriate selection of a measure of central tendency is dependent on the distribution of the data to be analyzed. Since financial data suffers from outliers and
asymmetry (Barber & Lyon, 1996), their median values offer a more robust measure of central tendency. Five out of eight announcement-archival studies used the median over the mean when a measure of central tendency was required to estimate company and comparison portfolio performance over time (Balakrishnan et al., 1996; Boyd et al., 2002;
Easton & Jarrell, 1998; Hendricks & Singhal, 1996; Kinney & Wempe, 2002). In financial analysis, the median is usually the measure of choice. Notably, not one of the survey-archival studies employed the median as a measure of central tendency in dealing with financial performance.
2.2.3.3 Comparison Time Span
The time span for comparison runs the gamut in archival data studies. Table 2.2 provides a listing of all the data comparison time frames used by the studies included in this literature review. Billesbach and Hayden (1994b) used the longest time span, covering eleven years, in analyzing inventory performance. Another study of inventory
performance compared performance in years -3 and -2 to years +6 and +7 (Biggart &
Gargeya, 2002). Although this is a wide time frame, the study’s conductors argued that all of their inventory performance ratios are scaled to sales activity and that provided a measure of control for external factors. The earlier study by Billesbach and Hayden (1994b) used a similar approach to compare two years of pre and two years of post JIT implementation performance separated by an interval of eight years. They compared ratios of inventory in conjunction with cost of goods sold (turnover), assets, and sales.
The longer the separation in time frame, the more likely it is that business environmental factors will obscure the affects of practice usage. At the short end of the time spectrum
are stock return studies that used from 200 to 11 days of performance prior to an adoption announcement as a basis for comparison (Hendricks & Singhal, 1996; Howton et al., 2000). To reduce confounding of results, all longitudinal studies allow a separation period between the practice and non-practice observations. This represents a trade off between the length of separation and the amount of data available in longer studies.
It generally is desirable, from a variance-pooling standpoint, to have identical pre and post sample sizes. The Easton and Jarrell (1998) study circumvented the need for a long symmetrical run of data by comparing the median of five years of performance of a TQM company with the median performance for its comparison portfolio. The current study also uses this approach.
2.2.3.4 Methods of Analysis
The final insight gained from this review of archival financial data studies relates to analysis method. Table 2.2 provides a summary of the primary analysis techniques used.
Following the rationale used for employing the median as the primary measure of central tendency would suggest that financial analysis benefits from the use of non-parametric statistics. The Wilcoxon signed ranks test (Siegel & Castellan, 1988) seems to be the most popular method for drawing statistical inferences from comparisons. Four of the ten announcement-archival studies used a Wilcoxon non-parametric test (Billesbach et al., 1994b; Easton & Jarrell, 1998; Hendricks & Singhal, 1996; Kinney & Wempe, 2002). In fact, an extensive study of operations performance measurement distributions with Compustat data showed that the Wilcoxon signed ranks test was uniformly more
powerful then the student t-test (Barber & Lyon, 1996). Easton and Jarrell (1998) go further to point out that, in many instances of financial data analysis, the Wilcoxon rank sum test6 is actually more appropriate for use with accounting data than the signed ranked test. The signed rank test requires an assumption of data symmetry often not present in accounting data. Easton and Jarrell do use the Wilcoxon signed rank test for stock data, because stock data generally meets symmetry requirements and the signed rank test is more powerful than the rank sum test.
The current study embodies a combination of parametric and non-parametric statistical tests as appropriate to evaluate hypotheses. In instances where multivariate normality assumptions do not hold, Wilcoxon rank sum and logistic regression are employed to maintain conformance to statistical assumptions.