Each type of document structure is described by means of a three-tiered information structure that describes: — a meta-model, which represents a hierarchy of structural nodes which are r
Introduction
General principles
Linguistic structures vary significantly, encompassing both highly organized formats like terminological databases and loosely structured information such as lightly annotated texts Annotated corpora are represented in a non-hierarchical manner, reflecting multiple perspectives of linguistic information that typically reference primary data, such as part of speech tagging, and occasionally interrelate through annotations like reference based on basic phrase structure.
The following model identifies a class of document structures which could be used to cover a wide range of linguistic annotation formats, and provides a framework which can be applied using XML.
Each type of document structure is described by means of a three-tiered information structure that describes:
— a meta-model, which represents a hierarchy of structural nodes which are relevant for linguistic description;
— specific information units, which can be associated with each structural node of the meta-model;
— relevant annotations, which can be used to qualify some parts of the value associated with a given information unit.
Each structural node is defined by a set of basic or compound information units Basic information units represent properties that can be directly categorized, while compound information units consist of multiple basic units grouped together to convey a coherent piece of information.
For instance, a compound information unit can be used to represent the fact that a transaction can be a combination of a transaction type, a responsibility and the transaction date.
Basic information units, whether they are directly attached to a structural node or within a compound information unit, can take two non-exclusive types of value:
An atomic value can represent a simple type, such as a number, string, or picklist element, as defined in XML schemas, or it may correspond to a mixed content type, particularly in the context of annotated text.
— a reference to a structural node in order to express a relation between it and the current structural node.
Basic and compound information units can be abstractly represented as feature-value structures associated with specific structural nodes in the structural skeleton For instance, a Geneter sub-document identified as
UHB can be modelled as a basic information unit in the following way:
Similarly, the following MSC sub-document
YYY
can be modelled as
The final stage of information representation involves associating semantic information with specific components of information unit values This process often includes annotations that highlight particular references within a terminological definition, such as identifying the genus and differentia For example, consider the definition of a lead pencil.
casing is fixed around a central
graphite medium which is
used for writing or making marks
Such information, also known as mixed content in XML, cannot be directly represented as a feature structure and will be directly expressed in the following GMT representation ( element).
Example
To illustrate how a TDC can be analysed as an abstract structure, let us consider a simple terminological entry expressed as an XML document conforming to MSC specifications:
manufacturing
A value between 0 and 1 used in
alpha smoothing factor
Alfa
The XML document can be aligned with the abstract model by recognizing a structural framework that matches the meta-model and linking the relevant information units to each structural node within this framework, as illustrated in Figure 3.
Here, data categories can be mapped onto the corresponding data categories specified in ISO 12620:
Generic representation of structural levels and information units
Linguistic data can be structured using a versatile architecture that features a graph of fundamental structural nodes, each connected to one or more information units This architecture is illustrated in a UML diagram, which depicts the relationships among the defined classes.
A structural node is a class characterized by a single attribute, LevelName, which serves to identify objects of this type within a specific Linguistic Resource (LR) format, such as TE (Terminological Entry) used for representing TermBanks.
An information unit is a class characterized by three key attributes: IUName, which identifies the object within a specific data category (such as Definition or PartOfSpeech); C_type, which describes the type of content; and C_value, which holds the actual content value.
NOTE The value of C_type can either belong to the set of simple types as defined in XML Schema Part 2: Datatypes or be MIXED as described in 5.2.2.
Data category ISO 12620:1999 number ISO 12620:1999 name id A.10.15 entry identifier subjectField A.4 subject field definition A.5.1 definition lang A.10.7.1 language identifier term A.1 term termType A.2.1 term type fullForm A.2.1.7 full form
Figure 3 — Mapping an XML document to the abstract model
Objects of these two classes can be related in the following ways.
Association refers to the relationship between structural nodes through hierarchical links, allowing for flexibility in the number of connections and the network's structure, which can include trees or directed acyclic graphs.
— hasContent: Relates a structural node to information units (e.g a definition attached to a Terminological Entry) An instance of an information unit is attached to one and only one structural node (1 1).
— refinement: Relates information units that provide additional information to another information unit (IU) (e.g a /note/ refining a /definition/) A refining IU is related to one and only one refined IU (1 1).
The MIXED type represents an organized blend of textual content and information units, encompassing various forms of annotated content In UML, it can be illustrated using the agglomeration operator, as depicted in Figure 5.
Annotations in textual content can be enhanced by incorporating additional information units, such as details about the time and author of the annotation.
The terminological meta-model
The terminological meta-model follows guidelines for terminology management, focusing on the creation of terminological entries as outlined in ISO 704 A key distinction of a terminological entry, in contrast to a lexicographical entry, is its emphasis on concept orientation, addressing a single concept within a specific language, particularly in multilingual contexts.
Figure 4 — UML diagram for structural nodes and information units
The International Standard outlines a meta-model for terminological entries, which may include one or more equivalent concepts in different languages In contrast, a lexicographical entry features a single lemma and multiple definitions across one or more languages While lexicographical entries generally provide information related to words, such as part of speech and gender, some of this information is also commonly found in terminological entries.
A terminological data collection comprises global information about the collection and a number of entries Each entry performs three functions:
— it describes one concept, or two or more totally or partially equivalent concepts, in one or more languages;
— it lists the terms that designate the concept(s);
— it describes the terms themselves.
Each entry may consist of multiple language sections, with each section containing several terminological units Data elements within an entry can be linked to various descriptive and administrative information Additionally, there are external resources, such as bibliographic references, ontology descriptions, and binary data like images, that can be associated with one or more entries.
By instantiating the generic architecture presented in 5.2, the terminological meta-model is described through seven instances from the structural node class.
— TDC (terminological data collection): Top level container for all information contained in a terminology system.
Global information (GI) refers to data that is relevant to all elements within a file, distinguishing it from information that may only apply to specific components Typically, GI includes essential details such as the title of the XML file, the originating institution or individual, address information, copyright details, and update records.
A terminological entry (TE) is a comprehensive piece of information related to a specific concept It typically includes descriptive details relevant to the concept, along with administrative data Depending on the nature of the termbase, a TE may feature one or multiple language sections, accommodating monolingual, bilingual, or multilingual formats.
Complementary information (CI) encompasses various types of data, including textual bibliographical or administrative details, static or dynamic graphic images, videos, audio, and other binary data It may also feature references to terminological resources or contextual links to related text corpora and ontologies CI is considered a shared resource, as it is accessible to all entries within a termbase without the need for repetition.
The LS (language section) encompasses all term sections relevant to a terminological entry in a specific language, along with pertinent information Typically, it includes definitions, contexts, and other details associated with that language or its terms.
The term section (TS) provides essential information about specific terms, including definitions and contextual details related to the concept being defined.
The TCS (term component section) provides insights into the morphemic elements, words, or contiguous strings that form a polymorphemic or multiword term In languages like German and English, distinguishing the individual components of a polynomial term is often unnecessary However, in languages such as French and Spanish, it is crucial to include information like gender for the individual words that comprise a multiword term, as this information is essential for proper usage in texts.
These instances of structural levels implement the “association” relation with constraints on cardinality (see Figure 6), which can also be schematized by the sketch shown in Figure 7.
— A TE can contain any number of LSs (0 *).
— An LS can contain any number of TSs (0 *).
— A TS can contain any number of TCSs (0 *).
A TDC must include precisely one GI (1 1), at most one CI-Level (0 1), and can have any number of TEs (0 *) The hierarchical structure is maintained through the 1 1 constraints defined for the dual cardinalities of each relationship.
Figure 6 — Terminological meta-model — UML diagram
Designing representations of terminological data on the basis of the meta-model
Each Data Category Specification (DCS) must be tailored to meet the specific needs of its user group, incorporating a list of data categories from the Data Category Registry (DCR), such as those defined by ISO 12620 These specifications include constraints on the permissible values for each data category, which can range from "text with markup" for contextual examples to "picklists" for grammatical gender Additionally, specific data types, as outlined in XML Schema Part 2: Datatypes, are utilized to define formats for numbers and dates Furthermore, constraints on descriptive data categories dictate where each category can be positioned within an entry, based on the options available in the core structure of the meta-model, including TE, LS, TS, and TCS.
The meta-model offers a wide variety of data models tailored to specific real-world applications By choosing a structure from the meta-model, it focuses on particular data categories and values relevant to the object classes of TE, LS, TS, and TCS.
All XML formats conforming to this International Standard are based on a) the meta-model, b) subsets of DCSs that are essentially derived from ISO 12620, and c) XML DTDs or XML schemas.
Database applications for terminological data must adhere to this International Standard by utilizing a meta-model, employing Data Category Schemes (DCSs) derived from ISO 12620, and defining data models through entity relationship diagrams.
Interchange, dissemination and interoperability
Interchange refers to the bidirectional transfer of information between two computer systems, utilizing intermediate formats In contrast, dissemination is a unidirectional process intended for either computer systems or human users To ensure interoperability, formats must adhere to the meta-model and the DCSs of ISO 12620, allowing for a seamless conversion of data between formats without any loss of information, often described as a "lossless round trip."
Figure 7 — Terminological meta-model — Schematic view
Blind data interchange occurs when data is so well-defined that the importer does not need to contact the data originators for interpretation This is not a concern when there are only two known interchange partners However, in scenarios involving multiple sources of terminological data being imported by a single routine, especially when adding new sources without altering the import routine is desired, the issue of blindness becomes crucial.
In bidirectional interchange, the goal is to maximize information preservation, while dissemination may involve intentionally omitting certain details from the original data For instance, a representation aimed at human translators might exclude administrative information that is pertinent only to terminologists managing the database.
The design of an XML format is shaped by its intended use, whether for dissemination or interchange, and the necessity for accessibility Once established, the database structure or XML format evolves to accommodate diverse data representations, often beyond the original designer's expectations By adopting an integrated approach, the resulting format is more likely to be flexible and compatible with various database structures and formats.
XML canonical representation of the generic model
Introduction
The meta-model's hierarchical organization and the qualification of each structural level can be effectively implemented in XML by instantiating the abstract structure outlined in section 5.3 and linking relevant information units to this framework.
The meta-model is represented by a generic element , which recursively captures the various representation levels of a terminological data collection Each structural node within the meta-model is identified by a type attribute associated with it.
element The possible values of the type attribute shall be the identifiers of the levels in the meta- model, i.e TDC, GI, CI, TE, LS, TS, TCS.
Basic information units associated with a structural node can be represented using the (for feature) element.
Compound information units are represented using the element, which can include a element followed by various and elements Each unit must have a type attribute that specifies an ISO 12620 data category or a user-defined category Additionally, the element can contain annotations through the element, which is also qualified by a type attribute referencing an ISO 12620 data category or a user-defined equivalent.
Example
This example demonstrates how to encode the information from the MSC terminological entry in GMT format, as outlined in section 5.1.2 The entry includes fundamental information units and maps three levels of the meta-model: TE, LS, and TS.
ID67
manufacturing
A value between 0 and 1 used in
en
alpha smoothing factor
fullForm
hu
Alfa
Description of the GMT format
The element serves as the foundational component of a valid GMT document, encompassing the GI that represents a TDC (the collection) and the CI, which includes essential external resources for detailing the various TEs.
The element serves as a structural node within a structural skeleton, with each node represented by a single element This element is recursive, allowing it to include and/or elements to convey information units relevant to the corresponding level of the meta-model.
The element accepts the following XML attributes:
— type: categorizes the element by identifying a structural node in the meta-model (TDC, GI, TE,
CI, LS, TS or TCS) represented by the element;
— id: allows unique identification of the corresponding information unit in the structural skeleton;
The target serves as a pointer to another element, providing the actual content for the current element This pointer signifies a unification between the current and the pointed-to element.
The element serves to represent information units linked to a structural node within a element or grouped with other units in a element It can encompass various types of tagged data, including strict annotations via the element or modules with external markup tailored for specific applications, with elements from these modules qualified by a designated namespace.
The element accepts the following XML attributes:
The element is categorized by referencing the name of the relevant data category, which should be derived from either ISO 12620 or a specific application context that is not included in ISO 12620.
— target: a pointer to a element in the case the information unit expresses a relation between the current structural node and another structural node in the structural skeleton;
A pointer to a element is utilized when the information unit is described outside the structural node it is intended to attach to This method can effectively describe a database of conceptual links that are external to a TDC.
The following elements constitute valid expressions of information units.
— Basic information unit attached directly to a structural node (level TE):
ID67
— Basic information unit whose value is a reference to a structural node in the structural skeleton and whose id attribute has the value “TE24”:
— Basic information unit anchored at the structural node in the structural skeleton whose id attribute value is “TE24”:
ID67
A compound information unit is linked to the structural node with the ID attribute "TE23," referencing another structural node identified as "TE24" within the structural skeleton.
The element is designed to represent a collection of related information units It includes a list that must contain at least one element, along with any combination of and elements Additionally, the element supports specific attributes.
The source attribute serves as a reference to a element when the associated information units are defined outside the structural node Consequently, this attribute is inherited by all elements within the current element.
The following elements constitute valid expressions of information units.
— Compound information unit comprising two basic features:
xxx
Lenoch
— Compound information unit anchored at a structural node in the structural skeleton where the id attribute value is
xxx
Lenoch
The element is utilized to tag specific portions of the content within a element, contingent upon the permissions set by the content type of the relevant data category This element supports various attributes.
The element is categorized by referencing the name of the relevant data category, which should be derived from either ISO 12620 or a data category defined within the context of a specific application that is not included in ISO 12620.
— target: a pointer to a element in cases where the annotation expresses a relation between the current information unit and another structural node in the structural skeleton.
The following constitutes a valid expression of information units.
pencil whose
casing is fixed around a central
graphite medium which is
used for writing or making marks
Representing languages in a terminological data collection
Data collection adhering to this TMF must clearly differentiate between the working language and the object language, which represent the two categories of language information applicable at any level of the collection.
The working language refers to the language utilized to convey textual content within data collection, represented by the xml:lang attribute as specified by the W3C's XML recommendation This designation applies to the entire sub-document from the point of declaration, unless a different working language is declared for a specific element within that sub-document.
The object language refers to the language of the terminological information described within a terminological data collection, typically at the language section level In the Terminology Management Framework (TMF), it is represented as a data category known as the "language identifier" according to ISO 12620 This language can be formatted in various styles as outlined in the International Standard, with its possible values being those permitted by the reference data category in ISO 12620 or a specific subset defined for a particular Terminology Markup Language (TML).
The following example shows how the two types of language can be used within a LS expressed in GMT:
en
Une valeur entre 0 et 1 utilisée
alpha smoothing factor
fullForm
This same example can be represented in MSC as follows:
Une valeur entre 0 et 1 utilisée
alpha smoothing factor
fullForm
Une valeur entre 0 et 1 utilisée
alpha smoothing factor
General
The specification of a TML shall be considered as a sequence consisting of two phases.
The initial phase involves identifying the necessary data categories for the TML, specifically a DCS This process includes selecting relevant data categories from ISO 12620 and, if required, adding any additional categories essential for the current TML that are not covered by ISO 12620 Ultimately, this phase aims to establish the interoperability conditions required for effective interaction with other TMLs.
The second phase involves creating the TML in XML format by utilizing expansion trees linked to various structural nodes in the meta-model This process includes instantiating the required information styles and vocabularies for the data categories present in these trees This phase ensures the essential information is provided to fully define the XML schemas that govern valid TML instances, as well as the filters necessary for converting a TML instance into a GMT instance and vice versa Further details on these steps can be found in sections 6.2 and 6.3.
Defining interoperability conditions
Interoperability conditions are defined by specifying the valid data categories for a particular TML, which involves detailing a set of properties for each category These properties can be represented using the Resource Description Framework (RDF).
A namespace, in the context of XML namespaces, refers to either the namespace of a Data Category Registry (DCR), such as ISO 12620, from which the data category is derived, or a local namespace linked to application-defined data categories.
— a unique name (DCName property in RDF) within the namespace;
— a type (DCType) which indicates whether the data category describes a possible information unit for the TML (DCType='complex') or is one possible value of an information unit (DCType='simple');
— the list of possible structural nodes (DCLevel) where the data category may occur for the TML;
— the list of the values (Content) that are allowed for the category in the case of a complex data category.
If a data category has been selected from a DCR, the following constraints apply.
The data category content description aligns with ISO 12620 standards Specifically, if a content type is defined, it must be a sub-type of the data type outlined in ISO 12620 Additionally, if the content is represented as a picklist, it should be a subset of the relevant picklist specified in ISO 12620.
— The category can be applied to a list of structural nodes, which is a subset of the list of authorized structural nodes expressed in ISO 12620.
Implementing a TML
Introduction
To achieve interoperability as a TML, it is essential to define the XML structures that will represent the relevant terminological data collection This involves creating an XML outline, which serves as the foundational framework of the instance and provides anchoring mechanisms for the various information units detailed in the DCS.
Implementing the meta-model
The structural component of a TML is established by linking an XML sub-tree, or expansion tree, to each structural node within the meta-model This association applies to every structural node that has a parent in the meta-model.
In the meta-model, an anchor must be established that includes a node from its parent's expansion tree, allowing for the attachment of its own expansion tree.
The XML outline of an instance of a TML comprises all the expansion trees associated with its structural skeleton.
Anchoring data categories on the TML XML outline
The expansion tree linked to a structural node comprises a collection of XML element nodes, each serving as a potential anchor for implementing an information unit permitted at that node By considering the anchoring style of the information unit and the relevant vocabulary in the TML, one can define how the data category will be represented as an XML sub-structure of its anchor Additionally, the properties of anchor, style, and vocabulary will be included as supplementary descriptions with the respective data category in the complete DCS associated with a TML.
In a TML's structural framework, information units can be implemented through five distinct styles: Attribute, Element, TypedElement, ValuedElement, and TypedValuedElement These styles represent the various methods of expressing feature-value pairs in XML.
The Attribute style utilizes XML attributes to represent information units associated with a specific anchor In this context, the vocabulary defines the name of the XML attribute, while the value of the information unit is expressed as the content of that attribute.
The Element style defines an information unit as an XML element that is a child of a specified anchor The vocabulary indicates the name of this XML element, while the value linked to a particular information unit is represented as the content within this XML element.
GMT representation MSC representation (anchor: )
ID67
GMT representation Geneter representation (anchor: )
en
GMT representation MSC representation (anchor: )
alpha smoothing factor
alpha smoothing factor
GMT representation Geneter representation (anchor: )
barbed wire
The TypedElement style defines an information unit as an XML element that is a child of a specified anchor, characterized by an XML attribute type This vocabulary includes the name of the XML element and the corresponding value for the attribute type, with the value of the information unit represented as the content of the XML element.
The ValuedElement style defines an information unit as an XML element that is a child of a specified anchor, characterized by an XML attribute value The vocabulary indicates the name of this XML element, while the value linked to a particular information unit is represented as the content of the XML attribute value.
The TypedValuedElement style defines an information unit as an XML element that is a child of a specified anchor, characterized by an XML attribute type This vocabulary includes the names of both the XML element and the XML attribute, with the value of the information unit represented as the content of the XML attribute type.
6.3.3.3 Constraints on datatypes for information units
Information units using the Element or TypedElement style can include additional markup, such as annotations, while those using the Attribute or ValuedElement style must not contain any markup Therefore, consistency checking is essential when establishing the DCS for a specific TML.
DCSs can incorporate external markup modules within the content model of information units, allowing for the referencing of external objects that align with the CI in the meta-model, such as bibliographical references These modules are linked through a registered namespace.
As an example, the following XML schema declaration can be used to define a content model comprising any element coming from the XHTML The Extensible HyperText Markup Language recommendation.
Implementing annotations
Annotations, represented by the element in GMT, should be integrated similarly to information units linked to the structural framework of a TML However, only the Element and TypedElement styles are permitted for this integration.
GMT representation MSC representation (anchor: )
manufacturing
manufacturing
manufacturing
manufacturing
Implementing brackets
Bracketed information units, denoted in GMT with a element, should be implemented by naming an element that corresponds to the primary information unit of the group, with its content consisting of the realizations of the group's information units.
Namespaces
The DCS includes an XML namespace description that references various XML objects, such as elements or attributes, derived from a specific TML This description is essential when additional markup modules are linked to the TML.
XML schema of the GMT format
This Annex outlines the GMT format specifications using XML Schema Part 2: Datatypes syntax It serves as a reference for verifying the conformity of GMT data that lacks additional markup modules If external markup is utilized, the schema must be adjusted to include the definitions of the associated namespaces.
A description in HTML format of the elements and complexTypes is provided in the following file: Schema gmt.html
A TML, or Translation Markup Language, is established by defining a collection of data categories along with an XML implementation of the meta-model outlined in this International Standard This meta-model, referred to simply as "the meta-model," includes the TML known as MSC (MARTIF with Specified Constraints), which is defined by a specific set of data categories chosen from ISO 12620, as detailed in section B.4.
MSC is designed to represent terminological data for the processes of analysis, dissemination and exchange of information from human-oriented terminological databases (termbases).
The data categories in MSC are designed to facilitate "blind" representation, allowing for interpretation without needing to consult the data provider For instance, the data category of grammatical gender can be clearly understood when defined with values such as masculine, feminine, and neuter, as per their conventional meanings in Western European languages In this case, a value like feminine can be interpreted independently Conversely, if a TML does not specify the possible values for grammatical gender and leaves it to individual data providers, users may struggle to interpret values like 356 without additional context from the provider Thus, the concept of blindness in data representation is relative and varies in degree.
The meta-model implementation in MSC follows ISO 12200, known as MARTIF, and defines expansion trees for each structural node along with XML styles and vocabularies for data categories For instance, the language section is represented by the XML element , while the TypedElement style is utilized for data category definitions, such as This approach facilitates various compliance checking modes, including one that employs a comprehensive XML schema for input into a general-purpose XML parser, generating detailed error messages for expert users Additionally, the use of TypedElement styles allows for a two-stage compliance checking process, where the first stage verifies the correct use of typed elements and structural nodes, and the second stage assesses specific data categories, thus providing clearer error messages for terminologists with limited XML knowledge.
Additional TMLs can be created using the same meta-model as MSC, with varying data categories These sets can either be subsets or supersets of MSC, depending on the specific application A compliance checker tailored for terminologists can be developed to automatically adapt to the selected data categories for each TML.
B.2 An example of an MSC XML document
This example of a simple yet comprehensive MSC document illustrates several key properties: it aligns directly with the meta-model outlined in the International Standard, maintains a well-formed and core-structure-valid format, and complies with the default MSC extensible constraint specification (XCS).
The numbers at the left are line numbers They are not part of the MSC document but serve as identifiers for the comments which follow the MSC document.
2
3
5
from an Oracle corporation termBase
6
MSCdefaultXCS-v-1-0.XML
10 manufacturing
11 A value between 0 and 1 used in
15 alpha smoothing factor
15 fullForm
20
This explanation assumes only a basic understanding of XML, which can be grasped by those familiar with HTML from creating simple web pages It is important to note that XML permits user-defined tag names, unlike HTML, which uses a fixed set of predefined tags The key MSC elements are aligned with the structural components of the meta-model outlined in this International Standard.
Lines 1 and 2