meta programming and model driven meta program development

Meta-Programming andModel-Driven Meta-Program Development Principles, Processes and Techniques 123... The software content in modern systems within the digital infrastructure is growing

Meta-Program Development

Advanced Information and Knowledge Processing

Meta-Programming and

Model-Driven Meta-Program Development

Principles, Processes and Techniques

123

Prof Vytautas ˇStuikys

Software Engineering Department

Kaunas University of Technology

Kaunas

Lithuania

Robertas DamaˇseviˇciusSoftware Engineering DepartmentKaunas University of TechnologyKaunas

Lithuania

ISSN 1610-3947

ISBN 978-1-4471-4125-9 ISBN 978-1-4471-4126-6 (eBook)

DOI 10.1007/978-1-4471-4126-6

Springer London Heidelberg New York Dordrecht

Library of Congress Control Number: 2012943398

© Springer-Verlag London 2013

This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer Permissions for use may be obtained through RightsLink at the Copyright Clearance Center Violations are liable to prosecution under the respective Copyright Law.

The use of general descriptive names, registered names, trademarks, service marks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made The publisher makes no warranty, express or implied, with respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media ( www.springer.com )

Information communication technologies are further advancing at an uninterruptiblerate The technology advances resulted in creating the computing infrastructure

which, even at the current maturity level, can be called by a general term ‘digital

world’, where humans are working with computer-based systems everywhere: in the

office, at home, during travel, at entertainment spots, etc What is most important

in this case is that they can do that at any time As computational-communicationalcapacities within the digital infrastructure are constantly increasing, we need more

and more software The software content in modern systems within the digital

infrastructure is growing perhaps at the similar rate as technology advances.The aim of the book is to look at the field from different perspectives: (1) histor-ical evolution of the meta-programming concept; (2) overall analysis of the meta-programming principles and techniques as they are treated in software engineeringdisciplines; (3) focus on the insights of the meta-program development processes

in the context of model-driven, feature-based and transformative approaches; (4)taking a background and framework for the further generalization of the paradigm intwo dimensions, that is, multi-linguistic and multi-stage programming; and (5) focus

on the domain-specific view in programming and meta-programming specific languages, a diverse of application domains such as hardware design,embedded software, e-learning, etc.)

(domain-In general, the meta-programming paradigm can be conceived of as a generativetechnology enabling to generalize the program development process and to builddomain program generators Though there are many solutions discussed in theliterature (especially language-based for both kinds of meta-programming, i.e.homogeneous and heterogeneous, e.g within software product families, etc.), buthow the technology is to be bound with the application domain models in asystematic way is yet little known

The distinguishing features of the book are as follows: (1) taxonomy of programming concepts based on the retrieval, analysis and evaluation of thescientific data and knowledge on the subject; (2) the concept of structural hetero-geneous meta-programming based on the original meta-language Open PROMOL;(3) the model-driven concept to the meta-program development process, including

meta-vi Preface

the concept of forward and reverse transformations within the evolution-basedframework to develop and maintain heterogeneous meta-programs and stagedmeta-programs (i.e heterogeneous meta-meta-programs); (4) metrics to evaluatethe complexity of feature-based models and meta-programs; (5) the high-levelspecification of meta-programs using feature-based models that are described byfeature diagrams (including their extensions); and (6) a variety of academic researchcase studies within the different application domains to experimentally verify thesoundness of the investigated approaches

One feature of the book as an important research result should be highlightedseparately Though the dedicated meta-language Open PROMOL has been devel-oped to support and better understand the principles of structural heterogeneousmeta-programming, this programming paradigm is independent upon the use ofthe meta-language type We have shown (using Java and PHP languages as casestudies) that a general-purpose programming language can also be used in the role

of a meta-language, when this meta-programming paradigm is applied In this case,however, the only part of the language that need be used is a subset relevant totextual manipulations

How has the book been written? For more than the last decade, we were involved(and we are till now involved) in the intensive academic research on the topic As

a result, we have published about 70 scientific papers in journals and conferenceproceedings worldwide Thus, a large amount of the book content was extractedfrom our published papers, but this content was revised and extended taking intoaccount a broad context of the large research communities that are considering thesame or related topics To create, outline the context and combine it with our originalideas, we needed to read and analyse more than 390 references It was possiblebecause our teaching courses are closely related with our research activities Thus,the book is a monograph, in which we present the results of our research activities.What is the content of the book? We have divided the content of the book intothree parts, where we consider the following topics: basics of meta-programming,model-driven view to meta-programming and transformations, some generalizationaspects of heterogeneous meta-programming and variability modelling and applica-tions of meta-programming relevant to our research

Part I includes (1) Introduction (Chap.1), (2) Taxonomy of Fundamental cepts of Meta-Programming (Chap.2), (3) A Background of Meta-ProgrammingTechniques (Chap.3), (4) Homogeneous Meta-Programming Techniques with CaseStudy (Chap.4), (5) Structural Heterogeneous Meta-Programming (Chap.5) and(6) Open PROMOL: An Experimental Meta-language for Heterogeneous Meta-Programming (Chap.6)

Con-Part II includes (1) A Framework to Deal with Heterogeneous Programming in Large: Meta-Program Lifecycle (Chap 7), (2) A Model-DrivenView to Meta-Program Development Process (Chap 8), (3) Cognitive Insightsinto Feature Diagram Notation and Beyond (Chap 9), (4) Meta-ProgrammingTask Specification Using Feature Diagram Patterns (Chap.10), (5) Meta-ProgramDevelopment as a Model Transformation Process (Chap.11) and (6) ComplexityEvaluation of Feature-Based Specifications and Meta-Programs (Chap.12)

Meta-Part III includes (1) A Framework: How Can Heterogeneous Meta-Programs

Be Further Generalized? (Chap.13), (2) Meta-Meta-Programming and EquivalentTransformations of Heterogeneous Meta-Programs (with Examples Using PHP

as a Meta-language) (Chap 14), (3) Multi-linguistic Aspects of HeterogeneousMeta-Programming in Web Applications (Chap 15), (4) Applications of Meta-Programming (Chap.16) and (5) What Is on the Horizon?

Who might be interested in reading the book? The book’s content covers (ofcourse, to some extent) the topics that are to be or might be studied as researchissues at the postgraduate level in the Computer Science and Software Engineeringspecialities Thus, master-level and PhD students can benefit from reading the book

in the first place We hope that their supervisors and lecturers can also find pertinentand useful information here As we were seeking for throughout the book to identifythe links of our concepts with the modern software design methodologies, webelieve that software experts will obtain useful ideas to support their solutions too.Though the notion of meta-programming was introduced almost from theemergence of computer science, it still has to live up to its promises: softwarereuse and automated program development A major stumbling block to achievingthe promised benefits is the understanding and learning of the meta-programmingapproach One reason may be that we do not yet thoroughly understand thefundamental concepts that define meta-programming The difficulties with themeta-programming approach are related to the following stages: (1) understand-ing, (2) adoption and (3) application of the meta-programming methods Meta-programming does not allow for its promises to be achieved straight away Rather,

it demands a thorough domain analysis, investment into the creation of programming tools such as meta-language processors, development of genericcomponent libraries and code generators, etc The payoff will come only after sometime, and designers will have to accept that Adoption of meta-programming alsorequires some shift in the attitudes and mind-sets of software developers Currently,

meta-a mmeta-ajority of progrmeta-ammers meta-and designers meta-are meta-accustomed to producing specificsolutions and applications for a particular domain problem They like to reuse theexisting software artefacts, but not much is done and invested into designing forreuse

How should the book be read or studied? This depends on the reader’s previousknowledge and his or her intention For novices, we suggest first to read Part I only.For concreteness, we suggest they also look at the relevant illustrative examples inthe other parts (usually given in another context) and to solve the exercise questionsgiven at the end of each chapter At the next level – for example, for learning – onemight choose or create some environment to test the ideas described in a particularchapter Experts and knowledgeable researchers first could read the introductoryand concluding parts of each chapter, or some selected chapters depending on thereader’s preference If they find the ideas discussed interesting, then they may wish

to explore the more involved and intensive studies within a particular chapter, oreven to explore the referenced sources

Authors would like to thank Dr Gintautas Grigas and anonymous reviewers fortheir valuable suggestions and comments that largely contributed to make the book’scontent improvements Our sincere thanks are also to Helen Desmond, the AssociateEditor, for her excellent management during the book preparation for publishing

We would like also to express our gratitude to the Series Editors ProfessorLakhmi C Jain and Professor Xindong Wu for their contribution and editingsuggestions during the final stages of the book publishing The technical quality

of the book could not have been improved without the endeavours and contributions

of Ben Bishop, Editorial Assistant at Springer London, and V Sabarirajan, SeniorProject Manager at SPi Content Solutions With thanks also to our family membersfor their patience, encouragement and understanding R Damaˇseviˇcius especiallywould like to thank his wife Vaida and sons Paulius and Matas for their patienceand sensibility shown during the writing of this book

ix

Part I

1 Introduction 3

1.1 What Is Meta-Programming? 3

1.2 Origins of Meta-Programming and Its Evolution 5

1.3 Other Definitions of Meta-Programming 7

1.4 Why Is Meta-Programming Needed? 8

1.5 Meta-Programming as a Higher-Level Thinking Paradigm to Develop Systems 11

1.6 The Topics This Book Addresses 11

1.7 Summary and Concluding Remarks 12

1.8 Exercise Questions 13

References 13

2 Taxonomy of Fundamental Concepts of Meta-Programming 17

2.1 What Is Taxonomy? 17

2.2 Fundamental Concepts of Meta-Programming 17

2.2.1 Transformation 18

2.2.2 Generation 19

2.2.3 Meta-Program 19

2.2.4 Levels of Abstraction 20

2.2.5 Generalization 21

2.2.6 Separation of Concerns 21

2.2.7 Reflection 22

2.2.8 Metadata 22

2.3 Taxonomy of Meta-Programming Concepts 22

2.4 Analysis of Meta-Programming Sources 25

2.5 What Can One Learn from Meta-Programming Taxonomy? 25

2.6 Summary 28

2.7 Exercise Questions 28

References 28

xii Contents

3 A Background of Meta-Programming Techniques 31

3.1 Introduction 31

3.2 Preliminary Principles 32

3.3 Fundamental Principles of Meta-Programming 33

3.3.1 Separation of Concepts 33

3.3.2 Taxonomy Axes to Describe Relationships Between Concepts 35

3.3.3 Levels of Abstractions and Meta-Programming 37

3.3.4 Integration of Concepts 38

3.4 Process-Based Relationships of Meta-Programming Concepts 39

3.5 Multidimensional Separation of Concepts and Meta-Programming 41

3.6 Representation of Meta-Programming Concepts Using Feature Diagrams: MDSoC View 43

3.7 Summary and Evaluation 45

3.8 Exercise Questions 46

References 46

4 Homogeneous Meta-Programming Techniques with Case Study 49

4.1 Introduction 49

4.2 Language-Independent Aspects of Homogeneous Meta-Programming 50

4.3 Terminology, Taxonomy and Formal Description of Homogeneous Meta-Programming Domain 52

4.3.1 Functional Homogeneous Meta-Programming 52

4.3.2 Structural Homogeneous Meta-Programming 55

4.3.3 Mixed Homogeneous Meta-Programming 57

4.4 Homogeneous Meta-Programming in Java 58

4.4.1 Basic Built-In Abstractions 58

4.4.2 Taxonomy of Homogeneous Meta-Programming Techniques in Java 59

4.4.3 Functional Homogeneous Meta-Programming Techniques 60

4.4.4 Structural Homogeneous Meta-Programming Techniques 62

4.5 Homogeneous Meta-Programming in VHDL 66

4.5.1 Similarities and Differences Between VHDL and General-Purpose Programming Languages (GPLs) 66

4.5.2 Component Generalization Framework in VHDL 67

4.6 Case Study: Development of Generic GATE Component in VHDL 68

4.6.1 Formulation of Requirements 68

4.6.2 Development of Generic Interface 69

4.6.3 Development of Generic Architecture 71

4.6.4 How Generic Component Is Used to Compose a Higher-Level Component/System 73

4.7 Summary 73

4.8 Exercise Questions 74

References 75

5 Structural Heterogeneous Meta-Programming 79

5.1 Introduction 79

5.2 Analysis of Structural Meta-Programming 79

5.3 Basics of Heterogeneous Meta-Programming 81

5.3.1 Explicit Separation of Concerns: A Two-Dimensional Model 82

5.3.2 Integration of Separated Concerns 84

5.3.3 Component Instance Models 86

5.3.4 Generic Component Models 87

5.4 Pre-Requirements for Meta-Languages 88

5.5 Summary, Evaluation and Conclusions 91

5.6 Exercise Questions 91

References 92

6 Open PROMOL: A Meta-Language for Heterogeneous Meta-Programming 93

6.1 Introduction and Motivation 93

6.2 What Concepts Does the Language Implement? 94

6.3 Basic Features of the Syntax and Semantics 97

6.4 Main Capabilities of the Language 97

6.5 Case Study: VHDL Code Modification via Widening, Narrowing and Isolation 99

6.6 Comparative Studies 100

6.6.1 Open PROMOL vs Java (CCC) as Meta-Languages 100 6.6.2 PHP as Meta-Language for Web-Based Applications 105 6.7 Evaluation of the Approach 107

6.8 Summary, Current State of the Language and Conclusions 108

6.9 Exercise Questions 109

References 110

Part II 7 A Framework to Deal with Heterogeneous Meta-Programming in Large: Meta-Program Lifecycle 115

7.1 Introduction 115

7.2 Preliminary Assumptions and Motivation 116

7.3 General Description of the Framework 117

7.4 Roles of Actors Within the Framework 119

7.5 Meta-Design, Design Spaces and Meta-Programming 121

xiv Contents

7.6 Domain Variability Modelling and Meta-Programming 123

7.7 Design-for-Change, Context Modelling and Meta-Programming 124 7.8 Summary 125

7.9 Exercise Questions 125

References 126

8 A Model-Driven View to Meta-Program Development Process 127

8.1 Introduction 127

8.2 Related Works 128

8.3 Framework for Model-Driven Analysis of Meta-Program Development 129

8.3.1 Basic Assumptions and Terminology 129

8.3.2 Description of the Framework 131

8.3.3 Meta-Model to Specify Problem Domain Abstractions 132 8.3.4 Instances of FD Meta-Model 134

8.3.5 Elements of the Instance of Meta-Program Meta-Model 137

8.4 Interpretation of Transformations 137

8.5 Requirements for Tools to Support (Semi-) Automatic Development of Meta-Programs 139

8.6 Summary and Evaluation 140

8.7 Exercise Questions 141

References 141

9 Cognitive Insights into Feature Diagram Notation and Beyond 143

9.1 Introduction 143

9.2 Overview of Feature Variability Management Research 144

9.3 Introduction into Feature Diagrams 145

9.3.1 Feature Definitions and Original Context of Use 145

9.3.2 Feature Model 145

9.3.3 Feature Types 146

9.3.4 Feature Diagram Definition and Variants of Notation 147 9.3.5 Basic Set of Abstractions in Standard Feature Diagram 150

9.4 Extensions of Feature Diagrams 150

9.4.1 Ontology-Based Extension 150

9.4.2 Extension for Quality-Oriented Modelling 152

9.4.3 Feature Diagram Extension for Variation Sequence Modelling 156

9.4.4 Other Known Extensions of Feature Diagram Notation 163

9.5 Summary 165

9.6 Exercise Questions 165

References 166

10 Meta-Programming Task Specification Using

Feature-Based Patterns and Domain Program Scenarios 171

10.1 Introduction 171

10.2 Problem Statement 171

10.3 Binding the Task with Other Domains 173

10.3.1 Analysis of Feature-Based Modelling Research 173

10.3.2 Analysis of Program Understanding Research 174

10.4 Framework to Consider Meta-Program Specification Tasks 178

10.5 Concept of Feature-Based Patterns 180

10.5.1 Definitions 180

10.5.2 Elements of Feature Diagram Instance as Patterns 182

10.5.3 Some Properties of Patterns 184

10.6 Summary, Evaluation and Further Research 185

10.7 Exercise Questions 185

References 186

11 Meta-Program Development as a Model Transformation Process 189

11.1 Introduction 189

11.2 Meta-Program Development Strategies 189

11.3 Transformation Tasks 190

11.4 Meta-Program Understanding and Concept of Multi-Stage Meta-Programming 191

11.4.1 Definition of Basic Terms 191

11.4.2 Understanding of One-Stage Meta-Programs 192

11.4.3 Understanding of Multi-Stage Meta-Programs 192

11.4.4 Static and Dynamic Analysis and Properties of Models for Understanding 195

11.5 Representation of Source Models 198

11.6 Representation of Target Model and Overall Design Process 200

11.7 Transformation Rules and Methodology 202

11.7.1 Stages of the Methodology 202

11.7.2 Transformation Rules with Example 202

11.8 Summary, Evaluation and Conclusions 206

11.9 Exercise Questions 206

References 207

12 Complexity Evaluation of Feature Models and Meta-Programs 209

12.1 What Is Complexity? 209

12.2 Complexity Management 210

12.3 Complexity Metrics 211

12.4 Complexity Measures of Feature Models as Meta-Programs Specifications 213

12.5 Evaluation of Abstraction Levels 216

12.6 Complexity of Programs and Meta-Programming Techniques 221

xvi Contents

12.7 Complexity Metrics of Heterogeneous Meta-Programs 222

12.7.1 Information Dimension: Relative Kolmogorov Complexity 222

12.7.2 Meta-language Dimension: Meta-language Richness 223

12.7.3 Graph Dimension: Cyclomatic Complexity 223

12.7.4 Algorithmic Complexity: Normalized Difficulty 224

12.7.5 Cognitive Complexity: Cognitive Difficulty 225

12.8 Complexity of Homogeneous Meta-Programming 225

12.9 Theoretical Validation of Complexity Metrics 227

12.10 Examples of Meta-Program Complexity Calculation 229

12.10.1 Complexity of Heterogeneous Meta-Programs 229

12.10.2 Complexity of Homogeneous Meta-Programs 231

12.11 Summary, Evaluation and Future Work 232

12.12 Exercise Questions 234

References 235

Part III 13 A Framework: How Can Heterogeneous Meta-Programs Be Further Generalized? 241

13.1 Introduction 241

13.2 A Framework to Analyse Generalization 242

13.2.1 Generalization Concept and Tasks 242

13.2.2 Why Structural Generalization Is Needed? 243

13.2.3 Why More Than Two Languages Are Needed? 245

13.2.4 More About the Language Aspects 245

13.2.5 Combining the Development and Evolution Stages Through Prediction and Anticipation 248

13.3 Summary 249

13.4 Exercise Questions 249

References 250

14 Meta-Meta-Programming and Equivalent Transformations of Heterogeneous Meta-Programs 253

14.1 Introduction 253

14.2 Related Works 253

14.3 Definitions of Basic Terms 256

14.4 Transformation Tasks 262

14.5 Transformation Method 263

14.6 Transformation Properties 265

14.7 Theoretical Background and Formal Description 267

14.8 Transformation-Based Processes to Develop Multi-Stage Meta-Programs 270

14.9 Summary, Discussion and Evaluation 273

14.10 Conclusions 274

14.11 Exercise Questions 274

References 275

15 Multi-Linguistic Aspects of Heterogeneous Meta-Programming in Web Applications 279

15.1 Introduction 279

15.2 A Survey of Technologies to Support Portal Development 279

15.3 Identification of Typical Web Components 281

15.4 Problem Statement 282

15.5 Properties of Web Component Instances 283

15.6 Web Component Generator Model 284

15.7 Analysis of Web Generators Designed Using Multi-Linguistic Meta-Programming 286

15.8 Summary, Evaluation and Conclusions 286

15.9 Exercise Questions 289

References 289

16 Applications of Meta-Programming Methodology 291

16.1 Introduction 291

16.2 Meta-Programming in HW Design 292

16.3 Meta-Programming for Creating LOs for Teaching HW Design 293 16.4 Context-Related Modelling and Meta-Programming for Embedded SW Domain 297

16.5 Meta-Programming as Component Wrapping Technology 302

16.5.1 Communication-Based Design 303

16.5.2 Reliable System Design 308

16.6 Internet-Based Application of Meta-Programming 309

16.7 Summary and Evaluation 311

16.8 Exercise Questions 314

References 315

What Is on the Horizon? 317

Glossary 321

Index 323

Part I

At the very beginning, our aim is to describe a scheme for the initial understanding

of meta-programming in order to introduce the knowledge required for the thoroughstudies in the field later We start from the statement which gives a very generaldefinition and formulates the primary objectives of the field as follows:

Meta-programming is a higher-level programming paradigm aiming to extend the aries of programming in constructing programs automatically.

bound-The reader should not be confused with the prefix ‘meta’, which means here

‘higher level’ Two important observations follow from the definition:

1 Meta-programming and programming are subjects of the same field; thus,meta-programming cannot be conceived without knowing the fundamentals ofprogramming

2 Meta-programming deals with automatic programming also known as programgeneration

We can look at the programming field from a very simplified perspective

In the simplest case, programming is the activity (process) of writing programsfor a computer system More specifically, this process can be described as atransformation of a given task into the executable specification a computer caninterpret Usually we call this executable specification a computer program, which

is the product of programming How a human being translates a real-world task into

a computer program is a matter of programming – a very wide field of science andengineering

Currently, programming as a field of human activity is deeply integrated withinthe countless appliances of information technology Often, not only domain profes-sionals, software engineers or computer scientists but also other actors having someknowledge in computing are able to write programs and, in this way, are involved

in programming This is happening because programming is not a homogeneous

4 1 Introduction

discipline and consists of many sub-fields (algorithms, programming languages,operating systems, Internet and distributed programming environments, concurrentprogramming, etc.)

It is not our intention to analyse programming in all aspects to which thisparadigm relates, but in this context, it is more important to focus on two aspectsonly:

1 On the program structure

2 On the execution process of a program

What is the structure of a computer program? Wirth has raised that tion already in 1975 when he published the book ‘Algorithms C Data Struc-tures D Programs’ [Wir76] We interpret the title of the book as a concise definition

ques-of the programming field, that is, programs using formula (1.1):

Program D data structure C algorithm (1.1)Though formula (1.1) is very simple, its value was not lost over the decades ofprogramming evolution because this formula expresses and communicates to theessence of programming From the perspective of the execution process, formula(1.1) can be reformulated in the following way: when executed, a program performsdata manipulations on the input data, that is, transforms the input data into the outputdata The transformation process is performed according to the rules specified byoperations of an algorithm With respect to the second aspect, the symbol ‘C’ informula (1.1) denotes manipulations In that sense, programming is a process ofwriting programs and then automatic solving of various computational and othertasks resulting in creating the output data as a solution of the task

Now we can look at the meta-programming field in a similar way that was used

to describe the essence of programming Indeed, meta-programming is an extension

of programming as it was stated at the very beginning The extension includes twoviews to generalization:

1 Generalization of an algorithm

2 Generalization of data

With regard to generalization of data, one can ask: what are data, in what aspectsare they different from programs and in what aspects are they similar to them? Dataand programs are syntactically pre-specified sets of symbols from the same alphabetarranged according to some predefined order of grammar In that sense, programsand data are not very different Of course, in each context we may use differentsyntactic rules enabling to form different structures of data and programs Data andprograms differ in semantics, that is, in how they are interpreted and what result isyielded after the interpretation

As a result of the discussion, we conclude two important findings:

1 Program is some generalization of data

2 In order to manipulate a program as a generalization of data, we need to

generalize an algorithm too and to use a meta-algorithm rather than an algorithm.

Now we can apply the stated findings to the introduced formula (1.1), aiming

to express the essence of meta-programming We can rewrite formula (1.1) in thefollowing way:

Meta-program D program ˚ meta-algorithm (1.2)Here, the term ‘meta-algorithm’, meaning the higher-level algorithm, is used inthe sense of generalization of an algorithm In other words, arguments for meta-operations within a meta-algorithm are programs or their parts, but not data Theterm ‘program’ is used in the sense of the data generalization The symbol ‘˚’ hasthe meaning ‘higher-level program manipulations’

Now looking at formula (1.2), one can better understand the following definition

of meta-programming:

Meta-programming is a higher-level programming paradigm which deals with how ulating programs as data is performed The result of the manipulation is the lower-level program, not data.

manip-After the insight into the meaning and definition of meta-programming, wecontinue with a historical review of the field

1.2 Origins of Meta-Programming and Its Evolution

The very beginning of meta-programming can be traced back to the EDVACcomputer in the 1940s [Tem95] The new concept implemented in the EDVAC’sdesign was to store data and programs in the same memory space This allowedmanipulating programs in the same way as data at run time Later the modification

of program code at run time initiated by an external program (such as a compiler or

a debugger) has become a standard practice

In 1959, the Share 709 system (a suite of system software for the IBM 709computer) introduced the idea of programmer macros as a sequence of computinginstructions available in a single program statement The aim of macros was to savetime and avoid errors in writing a sequence of instructions that repeats often in aprogram [GK59]

In 1966, the IBM/360 system with a high-level programming language PL/1has been introduced This language had the preprocessing capabilities to modify, atsome extent, the source code before translation Preprocessing as the simplest form

of meta-programming has been implemented in many programming environmentsand is still used The IBM/360 and its derivatives also had powerful macro-assembler facilities to generate complete programs or sections of programs.Another idea that influenced the emergence of meta-programming was theconcept of frame languages proposed by Minsky for knowledge representation[Min75] In the frame language, the data is represented in units, called frames.Frames contain not only information about the problem domain but also knowledge

by allowing the tools to handle all the tasks that do not belong to the domain

problem Such an automation of repetitive programming tasks can be seen as

meta-level programming or meta-programming.

Meta-programming was also known and used for a long time in formal logic

programming [Pas04] However, now the scope of the application of the programming techniques is much wider These include the domains such asprogramming language implementation [BDGC95], including parser and compilergeneration [Ter97], application and software generators [Bat98], product lines[Bat98], generic component design [Bec00], program transformations [LH01,ˇStu02], program evaluation and specialization [JGS93], generative reuse [Big98],software maintenance, evolution and configuration [CE00a], middleware appli-cations [CS02], XML-based web applications and web component deployment[LN02]

meta-Applications of meta-programming also include compiler construction, BNF

(Backus-Naur Form) used in compiler generators such as Lex and Yacc [LMB92],macros, higher-order functions in logic meta-programming [Pas04, Tem95], re-cursion, reflection [MJD96] including introspection and intercession [CE00b],meta-classes [KRB91], meta-object protocols [Chi95], template meta-programming[Tah99], anticipatory optimization [Big98], mixin-based programming [Pet96],design patterns [Din03], scripting [Ous98], partial evaluation [JGS93], soft IP(Intellectual Property component) design [ˇSD02], design of generative learningobjects [ˇSD08] and markup languages

Many, if not all, of the presented cases can be summarized as multi-stage

pro-gramming [Dam05,She01,Tah99], that is, the development of programs in several

different stages Other approaches such as parameterized programming [Gog96],

generative programming [CE00a], generic programming [DS98, MS89] and

re-flection-oriented programming [Rid99] are very similar or use the same concepts

as meta-programming Furthermore, meta-programming techniques closely relate

to novel software development technologies such as aspect-oriented programming

[KLMC97]

Thus, meta-programming is widely used in the software development cyclewhere it plays an essential role in program interpreters and compilers Meta-programming as a conceptual approach continues to evolve and its principles

are adapted to the ever-higher levels of abstraction Examples include

meta-modelling [AK02], meta-design [Dam06, FGYC04], model-driven engineering

(MDE) [Sch06] and meta-engineering [Pal02]

As the pursuit for increased productivity in software engineering continues, therole of meta-programming is destined to increase However, meta-programming

is almost never consciously and explicitly integrated in the software developmentprocesses Gaining awareness of its role is required to achieve progress in thisdomain Understanding what concepts characterize meta-programming is important

to both practitioners, who are aiming to adopt novel design methods, as well as toresearchers

1.3 Other Definitions of Meta-Programming

There are many viewpoints meta-programming can be analysed from, such asabstraction, languages, tools or programming techniques Perhaps, the most popular

is the abstraction-based view explained below Therefore, we extend the previouslyintroduced definitions of meta-programming

Software systems typically consist of several levels of abstraction such as chine (or object) code, assembly code, algorithmic or object-oriented language code

ma-or directives fma-or preprocessing Meta-programming, with respect to the abstractionlevel, should be understood as programming at a higher-level of abstraction Forexample, Cordy and Shukla give the following definition [CS92]:

Meta-programming is ‘the technique of specifying generic software source templates from

which classes of software components, or parts thereof, can be automatically instantiated

to produce new software components’.

A meta-language, which is a mechanism for introducing a higher-level of

abstraction, does not appear in this definition It is assumed that ‘source templates’

(such as CCC templates) are higher-level generic abstractions of the source (ordomain) language itself

Another definition introduces a concept of a meta-language explicitly: ‘any

language or symbolic system used to discuss, describe, or analyze another language

or symbolic system is a meta-language’ [Bat98]

A program written in a meta-language is a meta-program According toCzarnecki and Eisenecker [CE00b], a meta-program is ‘a program that generates

the source of the application : : : by composing pre-written code fragments’.

Examples of meta-programs are application generators such as parser generators

Meta-programming then can be defined as ‘creating application programs by

writing programs that produce programs’ [Lev86]

Sheard [She01] emphasizes the role of meta-programming in program generation

explicitly He says that in a meta-programming system, ‘meta-programs manipulate

programs’ A meta-program is a program, which ‘may construct programs, combine object-program fragments into larger object-programs, observe the structure and other properties of object-programs’.

object-Rideau gives a similar definition: ‘Meta-programming, the art of programming

programs that read, transform, or write other programs’ [Rid99]

8 1 Introduction

The Bartlett’s definition is: ‘Meta-programming is writing programs that

themselves write code’ [Bar05]

Veldhuizen expends the view to programming, stating that

meta-programming is ‘the study of generalizations in software and a meta-language

is the one, in which we express those generalizations’ [Vel06]

We accept the latter definition as a roadmap for our vision to the field On thisbasis, we, first, construct the conceptual framework of the topic in Part I of thisbook And, second, we consider also the generalization aspects of heterogeneous

meta-programming per se in Parts II and III of the book.

As a meta-program produces another program, it is often said that the meta-program

is a program generator Thus, meta-programming is about creating programs

automatically The automatic program creation is a value per se Such a statement,

at some extent, motivates the need of meta-programming as a specific kind of gramming, enabling to achieve higher productivity in system development Thoughthere are many driving forces stimulating higher productivity in software design, weemphasize two dominating factors: (a) complexity of systems to be designed growscontinuously and (b) those systems require more and more software content

pro-It has already been stated that the basic driving forces that move programmingtowards higher abstraction levels resulting in the increased degree of automationand, in particular, motivate meta-programming are as follows: (a) complexity ofsystems and (b) the need for more software in system designs

These two factors are interrelated and have a tremendous influence on trends,methods and approaches used in computer-related sciences and engineering There-fore, it is vitally important to look at the roots of the problem First, we look at thecomplexity problem in system design from the methodological perspective.Complexity is the intrinsic attribute of systems and processes through whichsystems are created Complexity measures allow reasoning about system structure,understanding system behaviour, comparing and evaluating systems or foreseeingtheir evolution As there is no precise definition of the term ‘system’, we refer to

a system as an application with software and hardware parts, such as a program or

a component The concept of complexity is discussed in many different contexts,such as system theory, informatics, software engineering and system-on-chip (SoC)design Here we discuss the complexity issues from software engineering and SoCdesign perspective

How serious the problem of complexity in the evolution of computing is, wecan conceive from the observation given by Ozzie, the Chief Software Architect ofMicrosoft in 2006–2010 [Mul06]:

Complexity kills It sucks the life out of developers, it makes products difficult to plan, build and test, it introduces security challenges, and it causes end-user and administrator frustration.

As computing becomes ever more central to business and society, complexity inthe digital world is becoming indeed a great challenge, and as Ozzie observes, itgoes ‘beyond our control’.

What is the main reason of the software complexity growth? The answer is

simple – advancement of technology Therefore, we need to look at the complexityproblem from the technological point of view

Usually technology advancements in computing hardware are characterized by

Moore’s law, which describes trends of the silicon revolution [Bon98] Moore,one of the founders of Intel Inc., observed in an article published in the April 19,

1965, issue of Electronics magazine that innovations in technology would allow a

doubling of the number of transistors in a given space every year, and that the speed

of those transistors would increase Ten years later, in an update article in 1975,Moore adjusted the rate to every 2 years (and yet later to 18 months) to account forthe growing complexity of chips [Hir05] What is less well known is that Moorealso stated that manufacturing costs would dramatically drop as the technologyadvanced

Moore’s prediction, now popularly known as Moore’s law, had some startlingimplications, predicting that computing technology would increase in value atthe same time it would actually decrease in cost This was an unusual idea

at the time since, in a typical industry, building a faster, better widget with twicethe functionality also usually means doubling the widget’s cost However, in thecase of solid-state electronics, the opposite statement is true: each time transistorsize shrinks, integrated circuits (ICs) become cheaper and perform better

Driven by Moore’s law and the continually increasing power of ICs, worldwiderevenue for semiconductors (basic blocks for electronic industry) has grown at over14% per year However, there were periods of the slower growth in electronicsystems that have lead to a slowdown in semiconductor growth to approximately6% [Jon04]

Though the technology steadily advances, the digital world requires more andmore computing capacities such as the ones for super computers [Deb04], forhandset computers, for mobile computers, etc With the advance of technology, thedesign of systems, such as SoC, based on that technology is becoming extremelychallenging With today’s SoC-based designs, Pete Hardee, Director of ProductMarketing (CoWare, Inc.), presents a design team with five facets of complexity

that range from ‘functional complexity, architectural and verification challenges to

design team communication and deep submicron implementation’.

The complexity challenges are also known as ‘design productivity gap’,

indicat-ing that complexity of SoC in terms of logic transistors that can be integrated on

a chip is increasing at the rate of 58 % per year (Moore’s law), while the designproductivity is increasing at the rate of 21 % per year only [SIA01]

As a result of advance in technology, the need for software is steadily creasing too And this is happening even at a much larger extent than advance

in-in technology per se The main-in reason is that SoC design methodologies are

based on a system-level design paradigm, in which the software solutions vail Examples are high-level modelling and validation tools based on hardware

pre-10 1 Introduction

description languages; software-based reconfiguring design systems; and externalrepositories for searching, sharing and reusing of IP components among multipleshareholders These examples are based on meta-level software solutions, UML,metadata, generators and meta-generators [BMP07,MM05]

The second reason is that the bigger part of functionality implemented withinSoC is realized using software and the content of software in such designs is steadilyincreasing On the other hand, SoCs are components of larger systems, where theapplication level is governed by software Finally, today the Internet along withmodern appliances, such as e-commerce, e-government, e-learning, etc., are the

‘largest eaters’ of software content and resources

The complexity growth forces designers and researches to seek for the adequatemeans for managing complexity issues The primary task is the understanding of acomplexity problem and finding the relevant measures for evaluating complexity.These issues may have a direct influence on performance, efficiency and othercharacteristics of systems to be designed

How to manage complexity? Many factors have the influence to better managing

of complexity, which we discuss later in Chap.12 At this point, the reader should

be aware of the commonly recognized principles for managing complexity, which

are as follows: reducing the amount of information, decomposing a system into

modules, abstracting or hiding information, as well as providing different levels

of abstractions.

The above-mentioned principles are general; however, when applied in a concretecontext, they may have a specific interpretation For example, the principles of meta-programming (they will be discussed in detail in Chap.3) also deal with informationhiding, decomposition of programs into parts, provision of different levels ofabstractions, etc., through the introduction of a specific means and mechanisms.Meta-programming also contributes to complexity management The more weuse program generators, the less code we need to write manually; hence, the morecomplex software systems are developed, the better is their quality and the shorter

is the time to market

As a consequence of complexity growth, system design methodologies should

be reconsidered and re-evaluated through the introduction of new concepts and

integration of these concepts into the methodologies The concepts of

‘design-for-change’, ‘product line’ or ‘software families’ anticipate the introduction of variants

in design of multiple applications as early as possible The anticipated variabilityshould be implemented using some technology and techniques Meta-programmingcan be seen as the technology that enables expressing anticipated variability in adomain explicitly even at the analysis stage and in this way leads to the effectiveimplementation of domain tasks When using meta-programming methods, it is

possible to alter the view to the design process per se and to change responsibilities

and roles of designers For example, a meta-designer develops meta-programs toimplement the repetitive and difficult tasks that relate to variability managing Thisenables a designer to concentrate efforts less on programming tasks and more onintegrating tasks, that is, on system design tasks

1.5 Meta-Programming as a Higher-Level Thinking

Paradigm to Develop Systems

So far we have treated meta-programming as a technology to support programgeneration The discussed definitions of meta-programming were focused on thetechnology-based view This view does not consider the entire development cycle of

a meta-program but rather takes an emphasis on the coding phase The development

of any software system or its part, however, starts from the formulation of ments and domain analysis Even more, the state-of-the-art development currentlyfocuses not on the development of a separate system but rather on the development

require-of a system require-of systems (also known as a product line require-of srequire-oftware families, see,e.g., [HHU08]) in order to achieve higher pay-offs through the increased scope ofreuse On the other hand, the complexity issues push design paradigms towardshigher abstraction levels In such a context, the understanding of a domain to beimplemented by software plays an extremely significant role

The domain understanding through its modelling is a well-founded approach that

is widely exploited for more than two decades in software engineering Abstractly,

if one ignores implementation details (e.g., when designing or using some tools),modelling can be conceived of as building of domain models or system models.Actually the development of a program or meta-program starts when we try tocreate models The model-driven approach prevails in modern design paradigmsnow [Sch06] Thus, modelling is a higher-level activity aiming to plan the whole

design at the early stage in order to facilitate the implementation of the designseparate phases later Such an activity requires a great deal of competency of actorswho provide the planning The competency is not only the needed knowledge What

is more important is that the competency means also another way of thinking andreasoning in analysis and modelling

As a result, the researchers have recently introduced new design paradigmsexpressed and described through such terms as meta-model and meta-analysis,meta-meta-model, meta-design or even meta-engineering [Den84] Here, we gener-

alize all these terms using one word – meta-thinking In terms of meta-programming,

by meta-thinking we mean the social aspects of meta-programming in the firstplace [FGYC04] We explain these aspects in more details later By introducingthis term here, we aim to send a message to the reader that meta-programming

is also about meta-analysis, meta-models, meta-program and meta-meta-program

development and other related tasks as it is stated by the term ‘model-driven view’

in the title of the book

1.6 The Topics This Book Addresses

We have divided the content of the book into three parts, where we considerthe following topics: basics of meta-programming, model-driven view to meta-programming and transformations, generalization aspects of heterogeneous

Con-Be Further Generalized? (Chap.13), (2) Meta-Meta-Programming and EquivalentTransformations of Heterogeneous Meta-Programs (with examples using PHP

as a Meta-Language) (Chap 14), (3) Multi-linguistic Aspects of HeterogeneousMeta-Programming in WEB Applications (Chap.15), (4) Applications of Meta-Programming Methodology (Chap.16) and (5) What Is on the Horizon?

We have presented an introduction to understand meta-programming at the tion level as a specific field of programming There is a great deal of definitions

defini-of the term ‘meta-programming’ This phenomenon is important per se providingslightly different aspects and views for better understanding of the topic Thisunderstanding may vary, depending on the context and aims We summarize threedifferent views on meta-programming:

1 Meta-programming is a program manipulation technique where the program istreated as data

2 Meta-programming is a generative technology to build program generators tosupport generative reuse

3 Meta-programming is ‘the study of generalizations in software and a

meta-language is the one, in which we express those generalizations’ [Vel06]

We use all of these definitions throughout the book, depending on aims Forinstance, we accept the latter definition in Part III as a roadmap for our vision

to the field aiming to extend the view by analysing the generalization aspects

of heterogeneous meta-programming per se This is our first contribution to thefield, which was not discussed in a systematic way so far The second result isthe motivation of the benefits and the needs of meta-programming for solving thecomplexity problems of systems to be designed

1.4 Study a short history of the meta-programming field and try to extend it throughthe retrieval of relevant information via Internet search.

1.5 Try to find the track between the system complexity and meta-programmingparadigm

1.6 Explain the concept ‘meta-programming as a higher-level thinking paradigm’and try to outline its importance in general

References

[AK02] Atkinson C, Kuhne T (2002) The role of meta-modeling in MDA In: Bezivin J,

France R (eds) Workshop in software model engineering, University of Nantes, Nantes

[Bar05] Bartlett J (2005) The art of metaprogramming IBM DeveloperWorks, Oct 2005.

lnxw06MetaCoding

http://www-128.ibm.com/developerworks/linux/library/l-metaprog1.html?ca=dgr-[Bat98] Batory J (1998) Product-line architectures Invited presentation, smalltalk and Java

in industry and practical training, Erfurt, Germany, pp 1–12

[BDG+95] Batory D, Dasari S, Geraci B, Singhal V, Sirkin M, Thomas J (1995) Achieving

reuse with software system generators IEEE Softw, Sept, 89–94

[Bec00] Beck K (2000) Extreme programming explained Addison-Wesley, Reading [Big98] Biggerstaff TJ (1998) A perspective of generative reuse Ann Softw Eng 5:169–226 [BMP07] Baily B, Martin G, Piziali A (2007) ESL – a methodology for handling complexity,

slides of presentation at DAC’07, San-Diego, June 2007

[Bon98] Bondyopadhyay PK (1998) Moore’s law governs the silicon revolution Proc IEEE

86(1):78–81

[CE00a] Czarnecki K, Eisenecker UW (2000) Separating the configuration aspect to support

architecture evolution In: Proceedings of 14th European conference on oriented programming (ECOOP’2000), international workshop on aspects and

object-dimensions of concerns, Cannes, France, 11–12 June 2000

[CE00b] Czarnecki K, Eisenecker U (2000) Generative programming: methods, tools and

applications Addison-Wesley, Boston

[Chi95] Chiba S (1995) A metaobject protocol for CCC ACM SIGPLAN Notices

30(10):285–299

[CS02] Cross JK, Schmidt DC (2002) Metaprogramming techniques for distributed

real-time and embedded systems In: Proceedings of 7th IEEE international workshop

on object-oriented real-time dependable systems, San Diego, 7–9 Jan 2002, pp 3–10

[CS92] Cordy JR, Shukla M (1992) Practical metaprogramming In: Proceedings of the

1992 IBM Centre for Advanced Studies conference, Nov 1992, pp 215–224

14 1 Introduction

[Dam05] Damaˇseviˇcius R (2005) Transformational design processes based on higher level

abstractions in hardware and embedded system design Doctoral dissertation, Kaunas University of Technology

[Dam06] Damaˇseviˇcius R (2006) On the application of meta-design techniques in hardware

design domain Int J Comput Sci (IJCS) 1(1):67–77

[Deb04] DeBenedictis EP (2004) Will Moore’s law be sufficient? In: Proceedings of the

ACM/IEEE SC2004 conference on high performance networking and computing,

6–12 Nov 2004, Pittsburgh, p 45

[Den84] Dennet DC (1984) Formulating human purposes: meta-engineering computers for

people In: Mason REA (ed) Information processing 83 Elsevier (North-Holland), Amsterdam, pp 253–258

[Din03] von Dincklage D (2003) Making patterns explicit with metaprogramming In:

Proceedings of 2nd international conference on generative programming and ponent engineering, GPCE 2003, Erfurt, Germany LNCS, vol 2830 Springer, New York, pp 287–306

com-[DS98] Dehnert JC, Stepanov AA (1998) Fundamentals of generic programming, report of

the Dagstuhl seminar on generic programming, Schloss Dagstuhl, Germany LNCS, vol 1766, pp 1–11

[FGY+04] Fischer G, Giaccardi E, Ye Y, Sutcliffe AG, Mehandjiev N (2004) Meta-design: a

manifesto for end-user development CACM 47(9):33–37

[GK59] Greenwald ID, Kane M (1959) The share 709 system: programming and

modifica-tion J ACM 6(2):128–133

[Gog96] Goguen JA (1996) Parameterized programming and software architecture In:

Proceedings of 4th international conference on software reuse, ICSR-4, Orlando,

23–26 April 1996, pp 2–11

[HHU08] Hubaux A, Heymans P, Unphon H (2008) Separating variability concerns in a

product line re-engineering project EA-AOSD’08, Brussels, Belgium, 31 March 2008

[Hir05] Hiremane R (2005) From Moore’s law to Intel innovation—prediction to reality.

Technology@Intel Magazine, pp 1–9

[JGS93] Jones ND, Gomard CK, Sestoft P (1993) Partial evaluation and automatic program

generation Prentice Hall, New York

[Jon04] Jones SW (2004) Exponential trends in the integrated circuit Industry In: IC

knowledge http://paginas.fe.up.pt/ jcf/PCVLSI/2003-04/exponential.pdf[KLM+97] Kiczales G, Lamping J, Mendhekar A, Maeda C, Videira Lopes C, Loingtier

J-M, Irwin J (1997) Aspect-oriented programming In: Proceedings of the European conference on object-oriented programming (ECOOP’1997) LNCS, vol 1241 Springer, New York, pp 220–242

[KRB91] Kiczales G, Rivieres Jdes, Bobrow DG (1991) The art of the metaobject protocol.

MIT Press, Cambridge, MA

[Lev86] Levy LS (1986) A metaprogramming method and its economic justification IEEE

Trans Softw Eng 12(2):272–277

[LH01] Ludwig A, Heuzerouth D (2001) Metaprogramming in the large In: Butler G,

Jarzabek S (eds) Generative and component-based software engineering LNCS, vol

2177 Springer, Berlin, pp 178–187

[LMB92] Levine JR, Mason T, Brown D (1992) Lex and Yacc O’Reilly and Associates, Inc,

Sebastopol

[LN02] L¨owe W, Noga M (2002) Metaprogramming applied to web component deployment.

Electron Notes Theor Comput Sci 65(4):106–116

[Min75] Minsky M (1975) A framework for representing knowledge In: Winston PH (ed)

The psychology of computer vision McGraw-Hill, New York

[MJD96] Malenfant J, Jaques M, Demers F-N (1996) A tutorial on behavioral reflection and

its implementation In: Proceedings of the reflection 96 conference, April 1996, San Francisco, pp 1–20

[MM05] Martin G, M¨uller W (eds) (2005) UML for SOC design Springer, Dordrecht [MS89] Musser DR, Stepanov AA (1989) Generic programming In: Proceedings of sym-

bolic and algebraic computation, international symposium ISSAC’88, Rome, Italy, 4–8 July 1988 LNCS, vol 358 Springer, Berlin, pp 13–25

[Mul06] Multicellular Computing (2006) Online: http://www.runningempty.org/Steve/

Multicellular/CharacterizingComplexity.html

[Ous98] Ousterhout JK (1998) Scripting: higher level programming for the 21st century.

IEEE Comput 31(3):23–30

[Pal02] Palmer KD (2002) Vajra logic and mathematical metamodels for meta-systems

engineering: notes on the foundations of emergent meta-systems theory and practice In: Twelfth annual international symposium of the international council on systems engineering (INCOSE), Las Vegas, 28 July–1 Aug 2002

[Pas04] Pasalic E (2004) The role of type equality in meta-programming Ph.D thesis,

Oregon Health and Sciences University, OGI School of Science and Engineering [Pet96] Pettorosi A (1996) Future directions in program transformation ACM Comput Surv

28(4):171–174

[Rid99] Rideau F (1999) Metaprogramming and free availability of sources In: Proceedings

of Autour du Libre Conference, Bretagne

[Sch06] Schmidt DC (2006) Model-driven engineering IEEE Comput 39(2):25–31 [SD02]ˇ ˇStuikys V, Damaˇseviˇcius R (2002) Taxonomy of the program transformation

processes Inf Technol Control 1(22):39–52

[SD08]ˇ ˇStuikys V, Damaˇseviˇcius R (2008) Development of generative learning objects using

feature diagrams and generative techniques Inform Educ 7(2):277–288

[She01] Sheard T (2001) Accomplishments and research challenges in meta-programming.

In: Proceedings of 2nd international workshop on semantics, application, and implementation of program generation (SAIG’2001), Florence, Italy LNCS, vol

2196 Springer, Berlin, pp 2–44

[SIA01] Semiconductor Industry Association (2001) The international technology roadmap

for semiconductors http://public.itrs.net/

[Stu02]ˇ ˇStuikys V (2002) Metaprogramming techniques for program generation and soft IP

design Habilitation Report, KTU, Kaunas, Lithuania

[Tah99] Taha W (1999) A sound reduction semantics for untyped CBN multi-stage

computation Or, the theory of MetaML is non-trivial ACM SIGPLAN Notices 34(11):34–43

[Tem95] Templ J (1995) Metaprogramming in Oberon Ph.D dissertation, ETH Z¨urich

[Ter97] Terry PD (1997) Compilers and compiler generators: an introduction with CCC.

International Thomson Computer Press, London

[Vel06] Veldhuizen TL (2006) Tradeoffs in metaprogramming In Proceedings of ACM

SIG-PLAN workshop on partial evaluation and semantics-based program manipulation, Charleston, 2006, pp 150–159

[Wir76] Wirth N (1976) Algorithms C data structures D programs Prentice Hall,

Engle-wood Cliffs

of top categories, and descending to more specific types through an arbitrary number of levels.

Taxonomy plays a significant role in cognition of any discipline because ‘there

is nothing more basic than categorization to our thought, perception, action and speech’ [Lak87] Here, we discuss known taxonomies in the field of meta-programming and suggest our own taxonomy based on extensive research presented

in [D ˇS08]

We consider the concepts of meta-programming below that are referred to asfundamental ones We try to arrange them according to their importance andcommonality in the field or other related subjects, though such a view is intuitiveand some other arrangement schemes may take place too We reason about theimportance of a concept by calculating the frequency of its use in the analysedsources [D ˇS08] Furthermore, we seek for to provide a particular concept alongwith its synonymous or terms that have a very close meaning

V ˇStuikys and R Damaˇseviˇcius, Meta-Programming and Model-Driven Meta-Program 17

2.2.1 Transformation

From the nature, a human has his senses to collect information, his mind totransform it, and his hands to apply it in the real world As humans tried to expandtheir capabilities, they invented sensing devices to sharpen the senses, informationsystems to extend the mind, and machines to do the handwork Thus, information

systems inherited the primary task of the mind – transformation of information.

First, we begin with the definition of the concept of program transformation itself[Pet96]:

Program transformation is a meaning preserving mapping defined on a programming language.

Program transformation can be described using a higher-level language such asBNF used in compiler generators [Ter97] or Open PROMOL [ˇSD00, ˇSDZ02], orany other meta-language It is used for the derivation of programs from high-levelspecifications or older program versions in a semantics preserving way

Generally, program transformation is a manipulation with its representationresulting in the change of the form (syntax) of the program Its semantics may

be either changed or not in the process A stepwise manipulation, which (1) isdefined on a programming language domain, (2) uses a formal model to support the

refinement, and (3) simultaneously preserves the semantics, is known as a formal

program transformation [Pet96] The general case of program transformation,however, does not require the definition of a formal model, as well as has norestrictions on the changes of its semantics

Conventional programming is oriented at developing tools for manipulatingwith data, that is, data processing, representation, visualization, communication,etc The inputs to a program are data structures The output is the result (theresulting data) produced by the program in the variety of forms dependent upon

a given application The produced result is used as is by other programs or theend user Meta-programming, on the other hand, is oriented at developing toolsfor manipulating with lower-level programs, that is, automatic analysis (parsing),automatic adaptation (modification) of a program to the context of usage, generation

of instances, etc., which all these can be summarized as transformation

We define program transformation as the process of changing one form of aprogram (source code, specification or model) into another, as well as a formal or ab-stract description of an algorithm that implements this transformation [ˇSD02] Therole of transformation in meta-programming is that the transformation algorithmdescribes generation of a particular instance depending upon values of the genericparameters The transformation algorithm ranges from simple meta-constructs such

as meta-if (conditional generation) and meta-for (repetitive generation) to the

sophisticated application-specific meta-patterns, which are composed of the nestedcombinations of the simpler meta-constructs

2.2 Fundamental Concepts of Meta-Programming 19

of the code generator could be in any language

In a more general sense, code generation is used to produce programs matically, thus reducing the need for human programmers to write code manually.Code generation can be done either at run time, including load time (e.g just-in-time compilers that produce native code from byte-code), or compile time (e.g

auto-a compiler-compilers such auto-as Yauto-acc [Ter97]) A preprocessor is an example of thesimplest code generator, which produces the target code from the source code

by replacing predefined higher-level constructs Thus, software generation is anautomated creation of programs described using high-level specifications

The role of code generation in meta-programming is centred on the development

of program generators, that is, higher-level programs that generate other programsadapted for specific applications The meta-language processor manipulates with theprogram instances or some parts of the instances as well as with data structures Ingeneral, the output is a family of the related program instances, or only one instancefrom the family

2.2.3 Meta-Program

Meta-program is the program, which generates or manipulates program code.Meta-programs are described using generic constructs of high-level languages(such as templates in CCC) or a different language (e.g macro language, meta-language) Other terms synonymous to ‘meta-program’ (such as meta-component,meta-specification) are also used Meta-programs usually are generic and have anumber of parameters; hence, the name ‘generic component’ also is used

A generic component is defined as a software module allowing choosing itsproperties to a certain degree without having to write or change code manually[Bec00] Conceptually, a generic component abstractly and concisely represents

a set of closely related (‘look-alike’) software components with slightly differentproperties Since it is sensible to integrate components that share a considerableamount of common code in a generic component, generic components can be con-sidered as a component family [She01] Such a definition does not provide explicitlyany technology using which the generic component is implemented If we want to

emphasize this property explicitly, we use the term generative component.

Generic components are not specific code fragments or common domainprograms Each generic component contains formal parameters and structures thatallow it to be systematically modified to become any of a set of specific components(instances) [Bas97] Generic parameters together with their respective range ofsupported values are usually identified at the generic component’s interface Hence,instantiating generic component means choosing actual values for the suppliedgeneric parameters to perform the necessary modifications by the appropriategenerator.

Therefore, we can define meta-program as a generic component implementedusing a meta-language that represents a family of similar component instances andcontains different functionality (variations) that can be customized (instantiated)through the parameterization to a specific component instance The role of a meta-program in meta-programming is the same as, for example, of a class in the object-oriented programming, that is, it is a basic unit of abstraction for composing largermeta-programming systems

2.2.4 Levels of Abstraction

Abstraction is the fundamental way of organizing knowledge and grouping ofsimilar facts together [Ste02] Abstraction hides the unimportant details of imple-mentation and emphasizes the information that is important for a developer or enduser at the given context Level of abstraction is a common concept in computerscience There are multiple levels of abstraction in software, where each level isused to represent a different model of the same information and processes, butuses a different semantic system of expression (or grammar) to express the content

of a particular domain Each higher (relatively abstract) level is built on a lower(relatively concrete) level

What is usually common to all cases of meta-programming is that there are two(or more) levels of abstraction Each level of abstraction uses a different semanticsystem The lower level of abstraction is usually domain-oriented and is used todescribe common domain functionality using a domain language The higher level

of abstraction (generic or meta) is used for expressing variability in a domain anddescribing manipulations with the syntactic units of the lower level of abstractionusing a specific meta-language

Summarizing, the levels of abstraction are semantic systems that are groupedtogether to represent different aspects of design in meta-programming systems Therole of levels of abstraction in meta-programming is centred at the construction

of a meta-program Meta-programs embody different (usually orthogonal) aspects

of domain systems Such aspects are implemented and composed by structuringdomain programs in terms of modules or layers, which use different semanticsystems and enable various functionalities to be added

2.2 Fundamental Concepts of Meta-Programming 21

2.2.5 Generalization

Generalization means transition from narrow and specific principles and concepts

to the wider and more general ones A higher, more generalized, level of domainknowledge encapsulates an understanding of the general properties and behaviourpossessed by a subset of its domain entities Introduction of generalization meanstransition to a higher level of abstraction, where domain knowledge is representedand explained more comprehensibly and effectively Thus, generalization allowsintroducing more simplicity into a domain

In computer science, generalization is usually understood as a technique ofwidening of an object (component, system) in order to encompass a larger do-main of objects (systems, applications) of the same or different type [Dam06].Generalization identifies commonalties and variabilities (variations) among a set

of domain entities The commonality may refer to essential features of a domainentity such as attributes or behaviour, or may concern only the similarity in syntacticdescription, while variabilities refer to the specific features pertaining to a specificdomain component or program

Therefore, generalization can be understood as a transformation of a specificdomain component into a generic component (meta-program) that is wider usableand reusable than the original one The role of generalization in meta-programming

is the development of a meta-program using specific domain program (component)

as a basis and involves capturing of domain commonalities, while expressingdomain variations at a higher level of abstraction

2.2.6 Separation of Concerns

Separation of concerns at the conceptual level is generally considered as a primarymeans to manage domain complexity The program parts related to the separatedconcerns are implemented separately, and then they are integrated back to form

a complete design It can be applied in various ways and is actually one ofthe key principles in software engineering [GJM03] This principle states that agiven problem involves different kinds of concerns, which should be identified andseparated to cope with complexity and to achieve the required engineering qualityfactors such as flexibility, dependability, maintainability and reusability

We define separation of concerns as the process of breaking a design probleminto distinct tasks that are orthogonal and can be implemented separately Meta-programming exploits the principle of separation of concerns to separate variableparts of the domain program from the fixed (common) parts (see Sect.3.3, for moredetails) A meta-language also should allow to separate clearly the computational(algorithmic, behavioural) aspects (i.e the ones dealing with domain functionality)and compositional aspects (i.e the ones dealing with component integration,interoperability, etc.), thus achieving a great deal of flexibility and reusability

2.2.7 Reflection

Reflection is the ability of a program to observe and possibly modify its structureand behaviour [MJD96] Usually, reflection refers to run-time or dynamic reflection,though some programming languages support compile-time (static) reflection Forexample, during compilation, when a program source code is compiled, informationabout the structure of the program is usually lost If a system supports reflection,the structure of a program is preserved as metadata embedded with the compiledcode In this context, meta-programming is a reflective activity because it allowsdeveloping programs that can create other programs

Reflection requires parsing and analysis of the domain language source code

An example of reflection was demonstrated in [Dam06], where parser automaticallyparses VHDL code and extracts component interface information that is further used

to generate component wrappers for specific domain applications

Metadata ranges from the finite-state-machine models of the component to theplain documentation In fact, any software engineering artefact can be a metadatumfor a given component, as long as (1) the component developer is involved in itsproduction, (2) it is packaged with the component in a standard way and (3) it isprocessed by automated development tools [OHR00]

Often, metadata are shortly defined as descriptions of data [Sol99] In the context

of meta-programming, metadata is the description of the properties or concerns of

a specific layer of abstraction in a meta-programming system The role of metadata

in meta-programming is to describe and represent additional information aboutthe meta-level of abstraction in meta-programs Examples of metadata includedescriptions of generic parameters in generic components, or the description ofdomain language syntax in compiler generators, or the structure of documents to

be generated

Since now the fundamental concepts have been identified, we can address howthe concepts may fit together in the meta-programming approach In addition

to a lack of consensus on the fundamental concepts, the software engineers

2.3 Taxonomy of Meta-Programming Concepts 23

Table 2.1 Known taxonomies of meta-programming

Sheard’s taxonomy

[ She01 ] Pasalic’s taxonomy [ Pas04 ]

Kind of meta-program Program generator Kind of meta-program Program generator

Separation of

languages

Homogeneous Separation of languages Homogeneous

Separation of static

and dynamic code

Manual Automatic

lack an understanding of how meta-programming concepts can be classified tocharacterize the meta-programming approach There seems to be an absence in theliterature on meta-programming taxonomy Several authors do summarize the basicconcepts of meta-programming; however, this is usually limited to one sentence

For example, ‘meta-programming involves analyzing, generating, and transforming

object programs’ [Wij03]

We found only two taxonomies that are comprehensive (see Table2.1) Thesetaxonomies presented by Sheard [She01] and Pasalic [Pas04] overlap considerably,though there are some differences The most serious objection that can be raisedagainst these taxonomies is that these are not as much the taxonomies of meta-programming concepts as taxonomies of meta-programming systems (generators,analysers) and tools (meta-languages) There are multiple issues addressed in thesetaxonomies, such as:

1 The number of languages used in a meta-programming system (homogenousmeta-programming – meta-language and domain language are the same lan-guages; heterogeneous meta-programming – the ones are actually differentlanguages)

2 The time of usage during software development cycle (static – before tion/execution, run time – during execution)

compila-3 The dependence of a meta-language upon a domain language (closed – dent, open – independent)

depen-4 The separation of static and dynamic parts of a meta-program (manual,automatic)

Classification of concepts in these taxonomies is opaque, many of importantconcepts such as abstraction or reflection are left out, while other issues that are notdirectly related to meta-programming (such as open source) are overemphasized.Our taxonomy of meta-programming concepts (see Table 2.2) introduces ahierarchy of concepts All concepts are categorized either as structural concepts

or process concepts Structural concepts describe the basic abstractions program, metadata) and principles of construction (separation of concerns, levels

(meta-of abstraction) used while developing the meta-programming artefacts Theirproperties are as follows:

Table 2.2 Our taxonomy of meta-programming concepts

Concept

class Concept

Equivalent terms used in

Structure Meta-program Meta-component, template,

generic component,

generative componenta , parameterized component, generator

A generic component implemented using a meta-language that represents a family of similar component instances and contains different functionality (variations) Levels of

abstraction

Layers of abstraction Semantic systems that are grouped

together to represent different aspects of design

Separation of

concerns

Separation of aspects (concepts)

The process of breaking a design problem into distinct tasks that are orthogonal and are implemented separately

Metadata Annotations Description of the properties or

concerns of a specific layer of abstraction in a meta-programming system

Process Transformation Manipulation, modification,

modify its structure and behaviour at run time

Generalization Parameterization Transformation of a specific domain

component into a generic component (meta-program) that is wider usable and reusable than the original one

aGenerative component D generic component(template) C explicitly added generative technology (e.g meta-programming)

1 Static (the structure and capabilities are defined by the designer)

2 Construction time (they are used during construction of the meta-programming

systems and artefacts)

3 Tool-dependent (depend upon specific selection of a meta-language, etc.)

The process concepts, on the other hand, describe the basic operations andprocesses that are performed by the designer and end user of the meta-programmingartefacts They are:

1 Dynamic (describe some method or process rather than a specific tool or

abstraction)

2 Domain-independent (can be implemented using different meta and domain

abstractions and tools)

Three out of four processes (transformation, generation, reflection) are used incompile time or run time, that is, during processing or execution of a meta-program,

2.5 What Can One Learn from Meta-Programming Taxonomy? 25

while the remaining one (generalization) is used during the creation of the programming artefacts

meta-Parts of the known taxonomies can be also found in our taxonomy, for example,types of meta-programs, use time, separation of concerns in meta-programming andusage of metadata However, our taxonomy is more consistent and wider than knowntaxonomies

To support the development of our taxonomy, we have accomplished an experiment

as follows A wide variety of sources (books, journals, conference proceedings),collected from Google Scholar using Publish or Perish program and published from

1972 to 2010 and related to meta-programming (i.e had such keywords as programming’, ‘metaprogramming’ or ‘meta-level programming’), were reviewedfrom different viewpoints (computer science, information systems, software engi-neering)

‘meta-The analysis consisted of reviewing each source document for the identification

of specific concepts as meta-programming concepts The concepts from 345 sourceswere recorded There were 35 concepts mentioned as belonging to the meta-programming approach (such as manipulation, code generation, etc.)

Figure 2.1 presents the numerical frequency of the concepts in the analysedliterature sources (note that our analysis is by no means exhaustive) Since manydifferent yet equivalent terms are used, we had to group the concepts with the similarmeaning into groups or classes Of the 35 concepts, 8 concept classes were identified

by the majority (73 %) of the sources: transformation (including manipulation,modification, adaptation, preprocessing and other synonymous terms), generation(including instantiation and weaving), reflection (including introspection), general-ization (including parameterization), meta-program (including generic component,template, macro, meta-specification, etc.), metadata, level of abstraction (includingvarious aspects of representation) and separation of concerns We have analysedthese concepts considering them as being fundamental to meta-programming as itwas stated at the beginning

Taxonomy?

Meta-programming is a very powerful software engineering method, but it requires

a great deal of discipline to use properly, rather than to resort to its use at everyopportunity Ad hoc application of meta-programming tends to make programsharder to understand, since it alters those programs dynamically in a way that can

be opaque

Fig 2.1 Frequency of main concepts related to meta-programming

Though the notion of meta-programming was introduced almost from theemergence of computer science, it still has to live up to its promises: softwarereuse and automated program development A major stumbling block to achievingthe promised benefits is the understanding and learning the meta-programmingapproach One reason may be that we do not yet thoroughly understand thefundamental concepts that define meta-programming The difficulties with the meta-programming approach are related to the following stages: (1) understanding, (2)adoption and (3) application of the meta-programming methods

Meta-programming does not allow achieving its promises at once Rather, it mands a thorough domain analysis, investment into creation of meta-programmingtools such as meta-language processors, development of generic component li-braries and code generators, etc The pay-off will come only after some time,and the designers will have to accept that Adoption of meta-programming alsorequires some shift of attitudes and mindset of software developers Currently,most of programmers and designers are accustomed to producing specific solutionsand applications for a particular domain problem They like to reuse the existingsoftware artefacts, but not much is done and invested into designing for reuse

de-2.5 What Can One Learn from Meta-Programming Taxonomy? 27

Mass production of software components as it was proposed by McIlroy already

in 1968 [McI68] and software factories have not become a reality partly due

to the lack of coherent reuse methodology and partly due to the reluctance toaccept the fact that mass production requires significant initial investment into tools(code generators, meta-languages, transformation systems) and resources (genericcomponent libraries)

The designers rather should shift their attention to developing generic solutions

to a set of domain problems, which later can be specialized to a specific domainproblem automatically without noticeable loss of performance or usability A wholeset of managerial problems must be solved The software business in order to

overcome the software productivity crisis must come from developing products

extensively (e.g hiring large numbers of less-paid software programmers in veloping countries) to intensive software development as has been already done

de-in manufacturde-ing de-industry and agriculture The software de-industry must focus onthe content (i.e on developing high-quality, highly generic and reusable solutions)rather than on the quantity of software products

One of the biggest problems in meta-programming is that the programmer almostalways has to work with multiple levels of abstraction The programmer not onlyneeds to know the details of how to program using domain-specific languagesand general-purpose languages, but also (1) the details of how they are eachimplemented, (2) how to communicate between them and (3) what sort of impedingmismatches there are between them

Programming requires that programmers would understand fully the syntax,semantics, abilities and limitations of the languages that they program with Asmeta-programming usually means using two (or more) languages – domain lan-guage and meta-language – in one specification or system, the designer must learn atleast as much information as twice Additionally to the difficulties in understanding,

programming in two languages simultaneously, and reading such multi-language

specifications [D ˇS02], the interoperability issues arise The languages usually aredeveloped independently from one another, which add tool-interfacing and syntax-clashing problems to the basket The meta-programmers and meta-designers should

be domain experts that have extensive and thorough knowledge of domain content

as well as meta-programming methods and tools

Additionally to the development problems, maintenance of software systemsdeveloped using meta-programming is a problem For example, the modification

of the generated code usually removes the program from the scope of the programming system The meta-program is no longer of any use, because thechange must be made at a lower level There are some solutions to this problem,but ultimately, it is very difficult if not impossible to solve it completely

meta-Meta-programming as a conceptual approach continues to evolve, and its ples are being adapted to the ever-higher levels of abstraction Meta-programmingplays an important role of organizing software artefacts and automating the pro-duction of domain solutions As the pursuit for increased productivity in softwareengineering continues, the role of meta-programming thanks to its fundamentalconcepts such as code generation and levels of abstraction is only destined toincrease