Therefore, the scope of the ex-post economic evaluation activities is very broad, ranging from microeconomic evaluations of specific policy interventions decisions or regulations in a we
Evaluation: Definition, objectives and practice
Definition and scope
This opening section outlines the key concepts used throughout the paper, starting with a precise definition of ex-post economic evaluation of competition policy and then detailing the potential scope of ex-post evaluation activities in the realm of competition policy.
This article focuses on the ex-post economic evaluation of competition policy and outlines three essential tasks: explaining the difference between ex-post and ex-ante evaluation; defining economic evaluation itself; and distinguishing the evaluation of the effects of competition policy from the evaluation of competition.
1.1.1 Difference between ex-ante and ex-post evaluation
Evaluation is the systematic identification of the effects of a specific intervention or policy
A key distinction in evaluation is between ex-ante and ex-post analysis (see Table I.1) According to Hüschelrath and Leheyda (2010), ex-post evaluations play a more prominent role in competition policy because they are primarily used to assess the decisions of competition authorities, and by doing so they can improve the quality of these decisions—the main output of competition agencies.
Within the Commission, ex-ante evaluation is called "impact assessment" and uses evidence to forecast the potential impact of a specific EU action—whether legislative or non-legislative, including spending programmes or other measures—and to judge its justification and how it should function to achieve defined objectives Retrospective or ex-post evaluation is a critical, evidence-based assessment of whether an EU action actually met the needs it aimed to satisfy and delivered the expected effects It goes beyond simply noting outcomes by examining causality—whether the action changed behaviours and produced the intended changes or any unintended or unexpected consequences.
Table I.1 Main differences between ex-ante and ex-post evaluations
Ex-ante evaluation or Impact Assessment
Before the action is taken
- Analyse expected effects under different scenarios
- Outline how to organise the monitoring of expected effects in preparation of retrospective evaluations
Retrospective analysis After the action is taken
- Assess whether an intervention has achieved its expected effects
- Analyse mechanisms explaining the effects and lessons learnt
The 2013 Commission Communication on Smart Regulation argues for a continuous loop of evaluation across the policy cycle, where impact assessments and evaluations complement each other Impact assessments draw on evaluations to diagnose why an intervention may not have delivered expected results and to determine how it should be revised, while ex-post evaluations depend on impact assessments to identify the causes of any shortcomings and the appropriate measures to address them.
The 32 intervention was expected to work, so the evaluation cycle begins with an ex-ante (impact) assessment of a new regulatory or policy proposal If later a decision is made to revise this policy or regulatory initiative, the revision proposal should be backed up by a robust ex-post evaluation of the performance of the existing regulation or policy.
Graph I.1 The policy evaluation cycle
Source: European Commission (2013), Public Consultation on Commission Guidelines for Evaluation, http://ec.europa.eu/enterprise/newsroom/cf/_getdocument.cfm?doc_idA6
1.1.2 Direct and indirect economic effects of competition policy
This study provides an ex-post economic evaluation of competition policy enforcement by assessing its effects on prices, mark-ups, productivity, and growth (see Graph I.2) Interventions by competition authorities are shaped by competition rules and institutions, so analyses must control for changes in the regulatory environment We distinguish direct and indirect economic effects: direct effects occur when competition interventions such as merger control and anti-trust decisions improve competitive conditions—for example, by eliminating a cartel or blocking a merger that would raise consumer prices—while indirect effects arise through deterrence Although deterrent effects, such as high cartel fines, are expected to deter other firms from illegal agreements, they are much more difficult to measure.
Competition policy creates direct and indirect economic effects that are transmitted through improvements in allocative, productive, and dynamic efficiency, which in turn influence prices, costs, and the quality and variety of products available to consumers These efficiency changes reshape the decisions and behavior of firms, consumers, households, and public authorities across the economy.
Allocative efficiency occurs when price equals marginal cost, signaling that resources are allocated to the products consumers value most Productive efficiency means producing a given set of products at the lowest possible cost using existing technology Dynamic efficiency describes firms’ drive to compete by improving current products and introducing new technologies For more details, see Part III.
Graph I.2 Economic effects of competition policy enforcement
EU competition policy includes State aid control, and the European Commission’s State Aid Modernisation initiative (2012) aims to concentrate enforcement on aid schemes with the biggest impact on the internal market It also encourages Member States to perform ex-post evaluations of their State aid schemes to assess whether the original objectives have been fulfilled without causing significant distortions to competition and trade The effects of State aid control are qualitatively different from those of other competition policy instruments and belong to a distinct strand of the literature, which is why this review does not cover studies on the impact of State aid control.
1.1.3 Distinction between the effects of competition and those of competition policy
Finally, a key distinction must be made between evaluating the impact of competition policy and measuring the effects of competition itself Competition policy refers to competition legislation—cartel prohibitions, bans on abuse of dominant positions, and merger control—and its enforcement, and it does not include other pro-competitive reforms such as reducing red tape, facilitating entry for new firms, or ex-ante sectoral regulation Assessing the impact of competition policy presents several challenges, including finding appropriate indicators of competition intensity and policy strength, establishing a causal link between policy and actual competition, and disentangling policy effects from other competition-promoting measures like trade liberalisation and better regulation (see part III for a more in-depth discussion).
According to Kovacic (2006), competition agencies can measure performance quality in two main ways: first, by evaluating the effectiveness of the agency’s outputs—such as case decisions and advocacy interventions—and their contribution to the goals of the jurisdiction’s competition law, including the improvement of consumer welfare under EU competition law; and second, by assessing the efficiency of the agency’s operational processes, focusing on how resources are used, the speed of decisions, and the overall cost-effectiveness of the agency’s activities.
Efficient control of state aid ensures that public resources are not wasted while addressing market failures—such as improving access to finance for SMEs and supporting research and development funding—while minimizing distortions to competition and trade within the European Union.
Direct and indirect (deterrence) effects on competition
Costs, mark-ups, prices Business dynamism Management quality Innovation
Competition Policy laws and institutions
This paper examines the quality of a competition agency’s internal operations—the combined managerial methods and organizational choices that shape how resources are allocated Achieving this requires a cost-benefit analysis that compares the benefits of the agency’s actions with their operational costs The study concentrates on this first issue, highlighting how internal governance, process design, and resource allocation decisions influence efficiency, effectiveness, and overall performance within the agency.
The primary outputs of competition authorities (CAs) are the decisions in key policy areas—mergers, cartels, abuse of dominance, and state aid control—and the ongoing development of the competition policy regulatory framework, including legislation, guidelines, and notices In addition, CAs conduct market studies and sector inquiries, engage in advocacy, and collaborate internationally with other competition agencies The scope of work that may be evaluated includes any of these activities, and ex-post economic evaluations of CA outputs can range from microeconomic analyses of specific interventions on a well-defined market to macroeconomic assessments of the broader economic impact of competition policy enforcement.
Overall, one can assess the impact of competition policy decisions and regulations at three levels (from the bottom to the top):
- Market: evaluation of the impact of a specific decision or competition policy rule on the functioning of a well-specified market
- Sector: evaluation of the impact of competition policy interventions on the performances of particular sectors
- Macroeconomic impact: evaluation of the impact of competition policy interventions on welfare, growth or other macroeconomic variables
Objectives
Competition authorities (CAs) began to engage more seriously in ex-post economic evaluations only relatively recently, with very few government assessments of the economic effects of their decisions and policies before 2000 Kovacic (2006) notes that by the end of the 1990s some economists expressed scepticism about the benefits of competition policy, a factor that may have spurred CAs to increase ex-post evaluation of their activities Crandall and Winston (2003) argue that there is little evidence that active competition enforcement yields appreciable consumer benefits, due in part to difficulties identifying problems, the challenge of remedying existing issues, and the role of self-correcting market forces in preventing large, lasting distortions Consequently, one key reason CAs carry out evaluations is to defend their legitimacy; however, the economic literature also identifies other main motivations for such evaluations.
Box I.1: Reasons for performing ex-post evaluations by CAs since 2000: Results of the 2012 OECD survey
An OECD 2012 survey shows that ex-post evaluations are pursued for three main reasons: accountability through annual reports mandated by law or statute that describe CAs' enforcement and advocacy activities; evaluations of specific interventions and their effects on affected markets; and assessments of the broader impact of competition policy Among the 46 competition authorities surveyed, 96% report performing accountability evaluations, 59% assess the impact of particular interventions, and 40% examine the wider effects of competition policy When asked why annual reporting is required, many CAs pointed to the legal obligation itself, while a minority argued that such reporting also ensures accountability for public funds and increases transparency Most CAs conduct ex-post evaluations of their specific decisions to improve internal decision-making processes; they also use these evaluations to boost credibility and set policy priorities Finally, evaluating the broader impact of competition policy helps CAs understand overall effectiveness and raise awareness of their role in the economy.
Four primary reasons are most often cited for leveraging data in competition policy: to improve policy decisions and enforcement practice; to boost the effectiveness of competition law; to set internal priorities; and to safeguard the legitimacy and strengthen the advocacy of competition policy enforcement with hard data Additional rationales include increasing the transparency of policy decisions, reducing the costs for business in dealing with complex procedures, and enabling benchmarking against the performance of other competition agencies These different motivations are described below.
2.1 Improve the effectiveness of competition policy decisions and the enforcement practice of the CAs
Ex-post evaluations improve enforcement effectiveness and the quality of regulatory decisions by enabling learning from past outcomes, including both successes and errors This learning helps authorities refine strategies and enforce rules more effectively As Kovacic (2006) notes, enforcement decisions can be viewed as experiments in which public authorities test hypotheses about business behavior, a process that requires difficult judgment under uncertainty By systematically analyzing post-hoc results, ex-post evaluations reduce uncertainty and strengthen future decision-making, leading to more robust, evidence-based enforcement practice.
Ex-post evaluation projects should target specific decisions with clearly defined objectives aimed at improving enforcement practices For example, critical cases that have sparked in-house or academic and consultancy debate could be revisited after several years to validate the choices made, or the impact of a policy approach used across multiple decisions could be evaluated ex-post to determine whether that approach should be revised In this area, court rulings also provide a form of ex-post evaluation for certain CA decisions (see Section II.1.2.1).
2.2 Improve the quality of competition law
The quality of competition law depends on the overall design of the competition policy regime and its effectiveness in achieving the goals set out in a jurisdiction’s competition framework The application of competition rules relies on notices, guidelines, and regulations that must be periodically revised to remain fit for purpose Any policy reform should begin with an ex-post assessment of existing regulations before considering new options for reform.
Advances in applying an economic approach to competition cases have fostered ex-post economic evaluations In antitrust and merger practice, economic analysis is central to defining the relevant market, assessing the efficiency effects of various forms of business behavior, and designing appropriate remedies or settlements to address competition concerns Empirical economic research, including systematic analyses of past antitrust and merger cases, has driven shifts in doctrine and enforcement policy.
The European Commission's 2005 Merger Remedies Study assessed the impact of 90 different merger remedies on markets and, on that basis, identified several risks and problems in the remedies policy then in place This empirical evidence provided a solid foundation for revising the Merger Remedies Notice, which was subsequently adopted in 2008.
Ex-post evaluation of competition authorities' (CAs) interventions helps set priorities and identify activities to concentrate on By assessing past actions against defined criteria, it reveals which interventions are most needed and/or likely to generate the greatest impact, thereby improving the CAs' productivity through better procedures and resource allocation and serving as an internal quality-control standard Bergman (2008) and Neven and Zenger (2008) argue that ex-post evaluations are useful for both internal and external prioritisation Internally, they enable shifting resources between competition policy instruments—for example, from cartels to mergers Externally, the authority's principal (for a national CA, the government or voters; for DG Competition, the college of Commissioners or the European Parliament) can reallocate resources between competition policy and other policy areas.
2.4 Defend legitimacy and improve advocacy
Demonstrating the benefits of competition policy for the effective functioning of the economy and citizens’ welfare is essential, a task that becomes especially critical in periods of economic crisis when policy must prove its value Niels and Van Dijk (2008) contend that this is further heightened by the increasingly proactive stance of competition authorities (CAs) They also argue that more reflection is needed on the welfare standard used to measure enforcement impact and that the consumer welfare argument often cited by CAs has limitations While competition policy is primarily about enhancing economic efficiency, which typically increases consumer welfare, it is crucial for the credibility and legitimacy of CAs to look beyond simple consumer benefits when assessing outcomes.
Competition authorities typically allocate a substantial share of their resources to advocacy activities, which are more effective when supported by robust ex-post analyses of past competition enforcement They generally measure the benefits to customers from cartel detection and from preventing mergers with anti-competitive effects Empirical evidence also points to the positive impact of competition on growth and its drivers, such as productivity and innovation By contrast, there is relatively less analysis of how competition policy enforcement affects macroeconomic outcomes like aggregate growth and employment, highlighting an area for further research.
38 macroeconomic analysis is done for other policies, such as internal market, labour market and innovation policies This type of analysis would be particularly relevant for advocacy purpose
Additional justifications for ex-post evaluations are sometimes offered Kovacic (2006) argues that ex-post evaluations enhance the transparency of policy decisions, which is essential for informing external observers about the content and the rationale behind specific decisions.
Enforcement of competition law heavily relies on commitments, but non-parties often struggle to assess the basis or significance of these commitments A periodic ex-post evaluation of decisions, and of the effects of substantive interventions such as commitments, would improve transparency in policy decisions and strengthen the credibility of competition enforcement.
Ex-post evaluations also help clarify the costs borne by agencies and by businesses when navigating complex competition procedures and parallel enforcement They are especially valuable when competition authorities share competence with sectoral regulators or when several national or regional competition bodies jointly handle a case, as the evaluation identifies cost burdens and improves coordination across jurisdictions.
Evaluation is a valuable tool for benchmarking and comparing the performance of competition authorities (CAs) with those of other jurisdictions Bergman (2008) and Kovacic (2006) advocate standardized methods and reporting that enable meaningful international comparisons of CAs In this spirit, the OECD (2013b) has recently attempted such cross-jurisdictional comparisons by developing indicators of competition law and policy to assess the strength and scope of competition regimes across 34 OECD and 15 non-OECD jurisdictions.
Practice of ex-post evaluations
Evaluation takes time and resources Therefore, it is essential to organise the planning of evaluation activities
Ex-post evaluation helps gather better data by identifying decisions or regulations as candidates for evaluation at the moment they are issued, enabling useful data to be collected even before the assessment begins It also delivers timely results because competition regulations require regular review by law In addition, a formal action plan, such as the DG Competition multiannual evaluation programme, ensures a predictable and regular evaluation cycle rather than crisis-driven, ad hoc exercises, and expands the pool of ex-post evaluations that can yield more general conclusions Consequently, ex-post assessment should not be reserved for reacting to a serious failure; a better performance measurement approach is to monitor competition policy interventions on an ongoing basis (see Kovacic).
An ex-post evaluation project comprises three main steps: preparation, execution, and exploitation of the results In each step, a set of important questions must be settled to ensure thorough assessment and actionable findings, as highlighted by Hüschelrath and Leheyda (2010) and summarized in Table I.3.
Table I.3 A short checklist of key issues to be tackled in an evaluation project:
1 Define the main objective of the evaluation
2 Select the subject of the evaluation and define the main evaluation questions
3 Select the methods to be used, consider data needs and check the project feasibility
6 Refine the methodology and identify the main indicators to be analysed
8 Check the robustness of the methodologies and the results
9 Draw synthetic conclusions useful for policy makers
10 Consider disclosure and active dissemination of results
3.1 Preparation of the ex-post evaluation project
Effective preparation of an evaluation project begins with clearly defining the objective, which in turn determines the evaluation’s subject and the appropriate methods—qualitative, quantitative, or a mix At this stage, it is also essential to decide who will conduct the evaluations.
3.1.1 Define the main objective and the subject of the evaluation
Effective evaluation preparation starts with clearly defining the main objective and the evaluation to be studied Table I.4 shows how evaluation activities align with the objectives pursued by the CAs; while an activity may contribute to several objectives, it is possible to identify the objective to which it contributes most Moreover, Hüschelrath and Leheyda (2010) warn that trying to achieve every possible objective within a single evaluation project is not advisable.
Choosing the most suitable evaluation subject hinges on balancing the trade-off between how much the results are aggregated and the level of detail and accuracy required This decision should come after clearly defining the main objective of the evaluation If the primary goal is to support high-level decision-making and identify broad trends, prioritize aggregated results that offer a concise overview If the objective is to diagnose specific issues, compare performance across units, or drive precise improvements, focus on granular data and higher accuracy, even at the expense of broader aggregation Align the evaluation subject with the stated objective to ensure the findings are actionable for stakeholders, suitable for reporting, benchmarking, and optimization efforts.
To improve competition enforcement, the CA must operate at a granular level, focusing on individual decisions and systematically analyzing the factors behind success and failure If the goal is to increase the effectiveness of competition law, it should evaluate whether existing regulations deliver the intended outcomes Additionally, the macroeconomic impact of the entire competition policy can offer the strongest support for advocacy and policy reform.
Table I.4 Subject of evaluation according to the main objective pursued by the
Improve effectiveness of competition law √ √
Analysing the broad impact of competition policy or its contribution to the good functioning of a sector can also help to define priorities However, several authors (such as Neven and
Zenger (2008) and Bergman (2008) caution against setting enforcement priorities solely on the global effects of interventions, warning that self-evaluations by competition authorities based on immediate consumer savings from cartel detection and merger prohibitions risk over-enforcement because authorities are incentivized to report large figures Bergman also argues that prioritizing budget allocation based on consumer benefits can be counterproductive: if a competition authority’s budget is tied to measured consumer benefits, it may strive to maximize those figures, potentially neglecting important legal violations in low-value markets where enforcement has strong deterrent effects or lead to blocking too many mergers.
Table I.5 outlines the ex-post evaluations currently underway within DG Competition Historically, these evaluations were mainly qualitative, based on court judgments or public consultations, but DG Competition is increasingly developing quantitative assessments, largely in collaboration with external experts The ongoing quantitative work focuses on individual decisions, including an ex-post evaluation of two telecommunications merger decisions, as well as regulations.
This analysis covers access to the file, policies for rescue and restructuring, and the impact of competition policy at both the sectoral level (energy study) and macroeconomic levels, with the results of these quantitative evaluations expected to be available in 2015.
Table I.5 Illustration: Recent and on-going ex-post evaluations in DG
Competition Subject of evaluations Ex-post evaluations in DG Competition
Individual decisions • Ex-post evaluations by EU Courts
• Ex-post monitoring through Monitoring Trustee (for remedies)
• Targeted occasional evaluations of the effectiveness of enforcement (e.g., merger decisions in telecommunications)
Regulations/policies • Mid-term reviews of existing regulations mainly done via public consultation/questionnaires sent to stakeholders
• Evaluation of the effects of temporary State aid rules adopted in the context of the financial and economic crisis
• Evaluation of access to file rules and practices in antitrust cases
• Evaluation of the impact of R&R decisions on the viability of aided undertakings
• A retrospective study on EU mergers and merger control
Sector and macroeconomic impact • Yearly customer savings resulting from important cartel and merger decisions
• Study on the "Economic Impact of enforcement of competition policy on the functioning of the energy sector"
• Study on "Simulating the effects of competition policy interventions in the EU using a macro-econometric model"
3.1.2 Select the methods to be used
Choosing the right methodologies is essential for answering evaluation questions and assessing data needs and project feasibility Since data can be costly, it is important to consider data requirements and availability early in the process Ex-post evaluations offer a wide range of methods, including qualitative and quantitative approaches and case studies that combine both The suitability of each method depends on data accessibility and the research questions, so using a mix of approaches is recommended to compare results across tools and enhance robustness Ideally, integrating qualitative and quantitative methods provides complementary perspectives and strengthens the overall analysis.
Following Buccirossi et al (2008) and Bergman (2008), who describe the empirical methods used in ex-post evaluations of merger control decisions, and drawing on the survey conducted in this paper, Table I.6 lists the various methods that have been employed in the ex-post evaluation of competition policy.
Table I.6 Methodologies for the ex-post economic evaluation of competition policy
• Court judgements Court judgements in response to appeals by parties concerned challenging the decisions made by competition authorities
• Surveys and peer reviews Interviews with or questionnaires filled out by competitors, suppliers, customers, law firms and competition authorities Quantitative methods
• Estimation and simulation of structural models A fully specified demand side model
• Reduced form estimation Single equation based on a clearly defined theoretical framework
• Quasi-experimental methods Comparison of performance of treatment group of companies with a control group
Stock price reactions among competitors following merger announcements and any related appeals reflect investors’ reassessment of market structure, pricing power, and post‑merger integration risks Meanwhile, the stocks of parties involved in cartel detection or under antitrust investigation respond to the launch and progression or conclusion of probes, with price moves signaling perceived penalties, regulatory constraints, and settlement outcomes Together, these dynamics illustrate how market participants price future profitability, competitive dynamics, and regulatory exposure in response to mergers, cartel findings, and antitrust actions.
• Case studies Combination of the above elements of information concerning a specific case
• Market studies Developments in a specific market following a number of competition policy interventions affecting that market
• Meta-retrospectives Academic papers and publications by competition authorities on selected issues (on antitrust or merger remedies, cartel detection and fines, e.g.)
• Surveys Surveys to assess the effectiveness of competition policy
• Calculation of customer savings Assumptions concerning expected effects on prices, markets concerned and the expected duration of such effects
• Macro-econometric modelling Reduced form estimations or simulations based on macro- econometric models Main sources: Bergman (2008); Hüschelrath and Leheyda (2010) and own review
Qualitative methods used in microeconomic evaluations can be distinguished by their primary source of information for analysis Specifically, performance measurement can be based on court judgments issued in response to appeals lodged by parties concerned with competition policy decisions, while an alternative approach relies on surveys of stakeholders and peer reviews.
Competition authorities rely on a range of quantitative methods, including econometric and statistical modelling techniques, as well as event studies that measure stock-price reactions to a decision The Difference-In-Differences (DiD) approach is notable for not requiring an explicit market model and instead compares post-intervention developments with the counterfactual of what would have happened without the policy In macroeconomic evaluations, methods are predominantly quantitative, ranging from simple estimates of customer savings to econometric modelling Surveys are also employed to assess the effectiveness of competition policy (see Section III.2.2.1).
3.1.3 Choose the evaluation team: using insiders, outsiders or both?
Who should conduct ex-post evaluations of competition policy has been a topic of debate in the literature (Kovacic 2006; Niels and Van Dijk 2008; Hüschelrath and Leheyda 2010) In the European Commission, the vast majority of evaluations are outsourced to safeguard the independence of the evaluator However, independence is not guaranteed, as some external evaluators may hesitate to publish unfavorable results in order to avoid jeopardizing future consulting opportunities.
Experience of other Competition Authorities
This section contrasts DG Competition's approach to its evaluation activities with those of other competition authorities that are particularly active in this area, using the framework described above It analyzes four additional CAs: the US Department of Justice (US DoJ) and the Federal Trade Commission (FTC); the United Kingdom's competition authorities—the Office of Fair Trading (OFT) and the Competition Commission (CC), now merged into the Competition and Markets Authority (CMA); and the Dutch Authority for Consumers and Markets (ACM) See Table I.7 for details.
Across the US, UK, and Netherlands, ex-post evaluations are used primarily as internal management tools to support decision making and justify ongoing work The FTC states its ex-post evaluations aim to inform policy and assess the agency’s performance; the CMA seeks to determine whether it achieves its objectives and does so cost-effectively for taxpayers, citing a 10:1 target whereby direct benefits to consumers should amount to at least ten times the costs to the taxpayer; and the ACM focuses on improving management and legitimising taxpayers’ spending All agencies stress that ex-post evaluations help defend the legitimacy of their work and view evaluations of individual decisions mainly as a means to improve internal decision-making processes.
The scope of the activities is very wide in the four agencies, ranging from the ex-post evaluations of individual decisions (mainly mergers and only a few cases of abuse of dominance in the CMA) to more macroeconomic estimates of consumer benefits All agencies prepare a yearly detailed account of their activities, which includes both a qualitative description of their main investigations and decisions and a set of quantitative indicators that measure their level of activity While all agencies estimate the yearly consumer benefits resulting from their merger and cartel decisions, only the CMA and the ACM have put more emphasis on the broader impact of their interventions These two agencies do this type of analysis mainly to help increase the awareness on the role of the CAs in the economy The former OFT and the CC regularly conducted ex-post evaluations of the impact of its market studies 4 , which are studies analysing the possible causes of malfunctioning in a given market or sector and whether the malfunctioning can be remedied
4 For example, in 2012, the OFT evaluated the impact of the 2004 PFT study into doorstep selling and in
2011, it evaluated the impact of the 2005 OFT study into care homes for older people
Table I.7 Comparison of the ex-post evaluation activities in different CAs
Competition) US (DoJ + FTC) UK (FTC and CC/CMA) NL (ACM) Objectives 1 Improve enforcement
2 Improve effectiveness of competition law
2 Evaluate the agency's performance (efficiency and legitimacy)
3 Improve internal decision making process
1 Internal management (to set priorities)
2 Evaluate the agency's performances (efficiency and legitimacy )
3 Improve internal decision making process
3 Improve internal decision making process
Scope 1 Annual report on enforcement and advocacy activities
3 Mid-reviews of existing regulations
4 Yearly estimates of consumer benefits resulting from cartel and merger decisions (voluntarily)
1 Annual report on enforcement and advocacy activities
3 Yearly estimates of consumer benefits resulting from cartel and merger decisions (required)
1 Annual report on enforcement and advocacy activities
2 Individual competition policy interventions (mainly mergers, but also a few abuse of dominance decisions…)
3 Ex-post evaluation of the impact of the market studies
4 Yearly estimates of consumer benefits resulting from cartel and merger decisions (OFT= required, CC= voluntarily)
5 Impact of their activities on quality and product innovations
1 Annual report on enforcement and advocacy activities
3 Yearly estimates of consumer benefits resulting from cartel and merger decisions (voluntarily)
4 Macroeconomic impact of competition policy enforcement
(public consultation, surveys of competitors and customers) court appeals) Quantitative methods (simulations, event studies, customer savings)
Mainly quantitative methods (merger simulations, reduced form estimation, DiD, customer savings)
Qualitative methods (survey of competitors, interviews)
Quantitative methods merger simulations, reduced form estimations, DiD, customer savings, case studies)
Mainly quantitative methods (micro simulations, customer savings, macro simulations
In-house (-) and outsourced (+) In-house (+) and outsourced (-) In-house (=) and outsourced (=) (OFT) Outsourced (CC)
Disclosure of results Decided on a case-by case basis Widely disclosed Widely disclosed Widely disclosed
Dedicated website In preparation No Yes No
Building on the OFT (2011), which sought to measure how competition interventions influence the drivers of economic growth—natural resources, capital, innovation, and management—the ACM has turned to model simulations to quantify the macroeconomic effects of competition policy on growth and employment.
Earlier work by Sinderen and Kemp (2008) and the efforts of the OFT and the ACM attempted to measure the deterrent multiplier of competition enforcement through surveys (for example OFT 2007 and Baarsma et al 2012) However, these studies did not apply the figures from global assessments of interventions With the creation of the UK Competition and Markets Authority (CMA), evaluation activities are being expanded beyond those of the OFT and the CC In its 2014/15 Annual Plan, the CMA defines its evaluation approach in three strands: (i) estimating the impact of each project as it completes, (ii) evaluating projects to identify costs, benefits, and lessons learned, and (iii) exploring how best to assess the CMA’s broader impact on economic growth—including deterrence and improvements in exit, entry, and innovation in markets (CMA 2014).
Competition authorities routinely combine qualitative and quantitative methods to assess enforcement effects The OECD (2012a) notes that among quantitative techniques used to analyse ex-post impacts of interventions and regulations, simulations and simple difference-in-differences (DiD) are frequently employed In the UK, competition agencies conduct retrospective merger reviews using a mix of questionnaires and interviews with interested parties and related firms, together with simulations and DiD techniques The FTC and the ACM also apply a DiD approach to analyse post-merger price changes and to judge whether mergers were rightly cleared, with the FTC using this method in sectors such as oil refining and hospitals and the ACM in the hospital sector By contrast, DG Competition has carried out relatively few ex-post evaluations of individual decisions, having largely outsourced them; the only ex-post evaluation study cited in the 2012 OECD survey concerns outsourced reviews of merger decisions carried out in 2006 (Buccirossi et al 2006), which combined simulations, DiD, event studies and surveys of main competitors and customers Nevertheless, DG Competition has launched public consultations with a strong ex-post perspective (notably within State Aid Modernisation) and has conducted in-house ex-post evaluations (for example, on Regulation 1/2003 and on the effects of temporary State aid rules adopted during the financial and economic crisis) More recently, DG Competition has begun to develop quantitative methods for ex-post evaluations with the assistance of its Chief Economist Team and external experts (see Table I.5).
Across agencies, evaluation activities share common features: they typically combine in-house and outsourced studies, and agencies with longer experience carry out relatively more in-house work For example, DG Competition has historically outsourced its ex-post evaluation studies, but recent evaluation projects have included more in-house work By contrast, the OFT has long experience with in-house evaluation studies, but the quality of this work is checked.
5 DG Competition is starting to evaluate the impact of two merger decisions in the telecommunication markets
All agencies publicize some of their evaluation activities, but the OFT is the only authority with a dedicated website for evaluation reports In the other institutions, evaluation reports are typically grouped with other studies and economic analyses Among the four competition authorities, the European Commission and the CMA appear to have multi-annual evaluation plans The CMA is obligated to evaluate two cases per year, and the other institutions also have the obligation to perform regular evaluations.
Microeconomic impact of competition policy
Methodologies
This section surveys the ex-post economic evaluations used to measure the impact of competition policy enforcement, distinguishing qualitative approaches from quantitative ones It identifies the main qualitative methods—court judgments, surveys, and peer reviews—and the primary quantitative techniques, including event studies, structural estimation, reduced-form estimation, simple model simulations, and quasi-experimental methods It discusses how combining methods can yield a more balanced assessment of interventions, and it summarizes the pros and cons of each evaluation method Finally, it highlights common challenges faced by evaluators, such as defining the counterfactual, addressing selection bias, and assessing the deterrent effects of competition policy interventions, regardless of the method employed.
1.1 Overview of qualitative and quantitative methods
Most ex-post microeconomic evaluation methods compare actual developments after a competition policy intervention with the counterfactual of what would have happened in its absence This comparison enables assessing the progress toward the competition policy objective achieved by the intervention In the EU, the broad objective of merger control and antitrust enforcement is to improve consumer welfare.
Ex-post microeconomic evaluation of competition policy typically uses a mix of qualitative and quantitative methods, often in combination, with triangulation—applying multiple evaluation approaches—enhancing the robustness of the assessment According to Bergman (2008), quantitative methods yield numerical estimates of progress toward the relevant competition policy objective, while qualitative methods do not produce such numerical estimates In practice, many studies, including case studies and meta-retrospectives, combine qualitative and quantitative techniques to achieve a more comprehensive and robust assessment of the policy intervention being evaluated.
Qualitative methods in competition policy include court judgments responding to appeals from parties affected by decisions, surveys of stakeholders, and peer reviews among competition authorities Evaluations using these qualitative approaches typically assess whether the original expectations underpinning the policy intervention have been realized, offering insights into effectiveness, impact, and areas for improvement.
51 quantitative methods tend to aim higher and establish a causal relationship between the policy intervention and the progress made towards the relevant competition policy objective
Quantitative methods discussed here include event studies, model estimation, simulations, and quasi-experimental approaches Model estimation ranges from fully specified structural oligopoly models to simpler, single‑equation simulation models By contrast, quasi‑experimental methods—especially those that compare firms exposed to a competition policy decision (the treatment group) with similar firms that were not subjected to the decision (the control group)—do not rely on an underlying market model.
Across many jurisdictions, court judgments are the most entrenched mechanism for evaluating competition policy decisions In most countries, parties can appeal competition authority decisions in court, and judges increasingly view it as their obligation to assess the economic arguments underlying those decisions, not only the legal issues Whether a decision is upheld in court may be considered an early indicator of the decision's quality, reflecting how well the competition authority's analysis stands up under scrutiny Conversely, a high rate of upheld decisions in certain domains suggests that the underlying policy guidelines and regulations are functioning effectively.
To perform an ex-ante assessment of a merger proposal or alleged anticompetitive behavior, competition authorities often rely on interviews or questionnaires filled out by competitors, suppliers and customers An ex-post evaluation of whether the merger or antitrust decision taken was appropriate could likewise depend on similar information More generally, competition guidelines and regulations are evaluated by surveying parties obliged to follow them The range of survey respondents assessing the quality of competition policy enforcement can extend beyond the directly affected parties in decisions or rules Lawyers and consultants advising the directly concerned parties or other competition authorities (through peer review) may be better positioned to judge the enforcement measures taken.
Surveys and peer reviews, by their nature, provide subjective indicators of the impact of competition policy enforcement and can be influenced by respondents’ strategic motives Because respondents may answer with strategic considerations in mind, these methods should ideally be supplemented by quantitative assessment measures or the judgments of independent experts, as demonstrated by the Global Competition Review (see Part III.2.2).
Qualitative methods can provide valuable insights, but ideally they should be supplemented by quantitative evidence, for example on the price effects of merger decisions (Ashenfelter et al 2009) The theoretical basis for quantitative assessments of competition policy interventions varies from fully specified models to more flexible approaches, underscoring the importance of integrating qualitative and quantitative analysis in policy evaluation.
52 oligopoly models to simpler reduced-form estimation This sub-section shortly considers the pros and cons of estimating these various types of quantitative methods
Event studies, along with other financial-analysis methods, offer a relatively objective tool for evaluating the effects of mergers, cartels, and antitrust decisions By tracking the stock-price reactions of competitors directly affected by the decision, these studies reveal how market expectations adjust to the outcome When a decision is procompetitive and lowers prices, competitors’ future returns may decline; conversely, an anticompetitive ruling may lift future profits Therefore, analyzing competitors’ stock prices can provide a practical assessment of the competitive effects of such decisions and, in some cases, may serve as a superior measure of competition compared with market prices alone.
When required financial market data from competitors is unavailable—for example, when competitors are not publicly quoted—the stock prices of the parties directly affected by the decision can serve as an alternative data source However, this approach makes it harder to disentangle competitive effects from other consequences of the decision, such as its potentially negative impact on the companies’ image and reputation.
Event studies provide an almost immediate, theory-based assessment of the competitive effects of a decision Empirically, there is a positive, albeit weak, correlation between ex-ante stock market returns and ex-post profitability measures, which supports using stock prices as a measure of competitive strength.
Although the event study approach offers certain benefits, it provides only an imprecise means of assessing the correctness of a given decision At best, it indicates whether the decision was appropriate, rather than delivering a numerical estimate of its effect Because the method does not yield a quantified impact, it should be treated as a qualitative tool for evaluating competition policy enforcement decisions.
1.3.2 Estimation and simulation of structural models including a fully specified demand side
Structural models used for ex-post economic evaluation of competition policy interventions often assume Bertrand competition with differentiated products and therefore specify a complete demand system More sophisticated versions augment this with a supply-side component derived from oligopoly theory, enabling the calculation of changes in both prices and markups However, estimating the relevant parameters for these models is more data-intensive and computationally heavy, a point highlighted by CET (2010).
Econometric estimations of the demand side can be grouped into two broad categories 7 : discrete choice (logit) models and Almost Ideal Demand Systems (AIDS, introduced by
6 Duso et al (2010) use a sample of large horizontal concentrations during the period 1990-2002 involving
459 firms either as merging firms or competitors, and contrast a measure of the mergers’ profitability based on event studies with one based on balance sheet profit data
7 Epstein and Rubinfeld (2001) offer a relatively nontechnical description of the principles of merger simulation using structural models
Deaton and Muellbauer (1980) show that discrete choice models assume the consumer's choice depends on price and product characteristics, and that the nested logit structure accounts for the fact that some products are more similar than others and hence more likely to be substitutes In AIDS models, the demand function is derived from the expenditure minimisation problem of the consumer, who purchases quantities of products to minimise the total cost of achieving a given level of utility Contrary to discrete choice models, no structure is imposed on the specification of the demand function Consequently, AIDS models are more flexible but also more demanding in terms of the number of parameters to be estimated.
Microeconomic impact of competition policy
2.1.1 Microeconomic analysis of merger decisions
Most work on the microeconomic evaluation of competition policy enforcement centers on merger control, employing quantitative and mixed-method approaches to measure the impact of merger decisions This section shows how the methods described in Part II can serve as practical tools for evaluating merger control, illustrating their application and the insights they provide It also highlights the main findings emerging from these analyses and draws attention to the methodological drawbacks that can arise when applying these methods in real-world practice.
Competition authorities approve only a subset of mergers and acquisitions subject to remedies, which can be either behavioural or structural Structural remedies aim to preserve competition by requiring the merged firm to divest certain assets to a new or existing competitor, while behavioural remedies affect the future conduct of the merged firm When structural remedies are involved, it is particularly difficult to distinguish the merger’s effects from those of the remedies, especially if the remedies are implemented at the same time as the merger Although it would be straightforward to jointly assess mergers and the associated remedies, some studies focus on analyzing the effectiveness of merger remedies themselves.
A large share of published studies offering ex-post evaluations of merger decisions focus on the U.S Federal Trade Commission and Department of Justice By contrast, this overview concentrates on the newer evaluations conducted by European competition authorities, while still referencing key papers that analyze U.S decisions.
The European Commission's 2005 Merger Remedies Study used structured interviews with stakeholders from the companies involved to identify serious issues in the design and implementation of remedies, assess the effectiveness of the Commission’s merger remedies policy, and identify areas for improvement The study analyzed 40 decisions covering 96 distinct remedies and, while its primary focus was on remedy design and implementation, it also provided early evidence on the effectiveness of different remedies in preserving effective competition Notably, about 94% of the divested businesses were still in operation three to five years after divestiture.
8 See Duso (2012) for a discussion of recent developments in the literature on ex-post evaluations of merger decisions
9 See e.g European Commission (2005) and Tenn and Yun (2011)
61 exercising some degree of competitive constraint on the merged entity However, the effectiveness of remedies giving access to infrastructure or technology was relatively weak
Lear – Laboratorio di economia, antitrust, regolamentazione (2006) conducted an evaluation for the European Commission of the Pirelli/BICC merger decision, examining Pirelli's 2000 acquisition of six power-cable manufacturing plants The team used a survey of market players to confirm that the merger did not lead to higher prices or worse purchasing conditions Despite the European Commission's support, Lear did not achieve a satisfactory survey response rate, prompting the adoption of an event study methodology as an alternative tool for assessing the merger decision More broadly, employing multiple methods to evaluate a single merger is commendable, as triangulating the results from different assessments yields a more robust conclusion.
From a competition policy perspective, the primary analysis centers on how a merger affects prices and overall customer welfare Beyond price effects, several studies, including the Lear study, also examine how a merger reshapes the profitability of the merged firm's main rivals Standard oligopoly models indicate that, in the absence of substantial efficiency gains, the merged entity tends to cut its output In the resulting equilibrium, rival producers can sell a larger quantity of their products at higher prices, thereby boosting their profitability These insights help explain why merger assessments must consider both consumer outcomes and the competitive position of rivals.
Despite theoretical expectations that a merger between rival power cable producers would raise prices and shrink customer surplus through higher supplier concentration, the stock prices of competing power cable manufacturers did not rise on average after the Pirelli/BICC decision, while customer stocks did This counterintuitive result could stem from selection bias if only about half of the potentially affected firms were publicly traded In any case, drawing broad conclusions from a single merger case remains difficult.
Clougherty and Duso (2009) analyze 165 large horizontal mergers investigated by the European Commission from 1990 to 2002 In contrast to Lear, they find that rival producers on average experience an abnormal stock-price increase at the merger announcement date, suggesting that a majority of mergers have anticompetitive effects They also show that the stock reaction of rival firms is not sensitive to the merger notification's position within the merger wave—pre-crest or post-crest periods Merger waves are identified by subtracting the time trend from the total number of mergers notified over time Consequently, the observed stock-price rise among rivals reflects the merger's impact rather than a higher likelihood that they will become takeover targets.
(iii) Estimation and simulation of structural models
Werden and Froeb (1994) were among the first to apply a structural model to forecast price and welfare effects of mergers among differentiated products, i.e., Bertrand mergers Their analysis combines analytical work with simulations of hypothetical mergers among U.S long-distance carriers The results from these simulations indicate that only mergers involving AT&T, the dominant long-distance carrier at the time, lead to a significant welfare decline.
Since then, a variety of structural models have been used as the basis for simulation analysis On the demand side, a key distinction exists between discrete choice (logit) models and Almost Ideal Demand Systems (AIDS), which adopt different assumptions about consumer decision-making While discrete choice models derive the demand relationship from an explicit consumer utility function, AIDS models represent quantities and prices through a parametric functional form (see Section II.1.3.2 for a detailed discussion) The parameters of these demand models can be calibrated to fit real-world observations—examples include Werden and Froeb, and Epstein and Rubinfeld—or estimated econometrically when data permit, as shown in more recent papers.
Data limitations prevented estimating a fully specified demand model, so researchers often rely on calibration models that impose additional assumptions and require less data Epstein and Rubinfeld (2001) present a simplified AIDS model that can be calibrated using aggregate market shares, the industry price elasticity, and the own-price elasticity for a single brand in the relevant market The model assumes that the market share lost due to a price increase is allocated to competitors in proportion to their market shares, enabling market shares to be used to assess the level of competition in a given market They illustrate their point with the 1995 acquisition of the toilet paper producer Scott by Kimberly-Clark and Heinz's proposed Beech-Nut baby food assets acquisition, which was terminated in 2001, showing that price increases are higher when the merging firms' products are closer substitutes.
Advances in data availability have enabled the estimation of cross-elasticities, allowing researchers to quantify how price changes for one model affect demand for others Ivaldi and Verboven (2005) illustrate this by analyzing the Volvo/Scania merger with a nested logit model that links the base-model list price to total sales across the heavy-truck model range They then simulate the merger's consequences using an oligopoly model with differentiated products and solve the model analytically not only for the merger and non-merger cases but also for an alternative-merger scenario The authors stress that, for merger assessment, the relevant counterfactual is not the status quo but the most plausible alternative merger scenario that would have occurred if the merger had been rejected.
Consistent with the standard approach in merger analysis, Ivaldi and Verboven initially assume that a merger yields no efficiency gains, which implies higher prices and lower consumer surplus In an extension, they relax this assumption by allowing a reduction in marginal costs, noting that such cost savings can be partially passed through to customers As an illustration, they analyze a hypothetical 5% marginal cost reduction to show how efficiency gains from a merger could offset price increases and potentially improve consumer welfare.
The European Commission considered the maximum efficiency claim plausible, with point estimates showing increases in customer surplus in four European countries and declines in twelve European countries In only a minority of countries do the efficiency gains from the proposed merger offset the negative demand-side effects However, in none of the four countries with gains did the merger lead to market dominance according to the European Commission's decision.
Using demand-model simulations of the US airline industry, Peters (2006) forecasts post-merger price trajectories after five airline mergers in the 1980s and compares them with observed post-merger prices He finds that post-merger price changes depend not only on changes in market structure but also on shifts in marginal costs and firm conduct, which are harder to capture with standard demand-side models While conventional merger simulations predict a large portion of the price movement, they cannot explain it entirely By incorporating post-merger information into the model, he assesses the relative importance of supply-side factors that may have contributed to price changes during this period of US airline industry consolidation The paper concludes that supply-side effects, including higher marginal costs from post-merger inefficiencies and potential collusive conduct among remaining market participants, were a non-negligible factor in the observed post-merger price increases.