A Visual Retrieval Environment for Hypermedia Information Systems ppt

A Visual Retrieval Environment forHypermedia Information Systems Centro Ricerca di Automatic, ENEL We present agraph-based object model that may be used as a uniform framework for direct

A Visual Retrieval Environment for

Hypermedia Information Systems

Centro Ricerca di Automatic, ENEL

We present agraph-based object model that may be used as a uniform framework for direct manipulation of multimedia information After an introduction motivating tbe need for abstrac- tion and structuring mechanisms in hypermedia systems, we introduce the data model and the notion of perspective, a form of data abstraction that acts as a user interface to the system, providing control over the visibility of the objects and their properties A perspective is defined to include an intension and an extension, The intension is defined in terms of a pattern, a subgraph

of the schema graph, and the extension is the set of pattern-matching instances Perspectives, as well as database schema and instances, are graph structures that can be manipulated in various ways The resulting uniform approach is well suited to a visual interface A visual interface for complex information systems provides high semantic power, thus exploiting the semantic expressibility of the underlying data model, while maintaining ease of interaction with the system In this way, we reach the goal of decreasing cognitive load on the user, with the additional advantage of always maintaining the same interaction style, We present a visual retrieval environment that effectively combines filtering, browsing, and navigation to provide an integrated view of the retrieval problem Design and implementation issues are outlined for MORE (.Multimedia Object Retrieval Environment), a prototype system relying on tbe proposed model, The focus is on the main user interface functionalities, and actual interaction sessions are presented including schema creation, information loading, and information retrieval.

Categories and Subject Descriptors: H.2 1 [Database Management]: Logical Design—datamodels:H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval-query

~ormulation; selection process; H.5 1 [Information Interfaces and Presentation]: MultimediaInformation Systems—hypertext nauigatiorr and maps; H,5.2 [Information Interfaces and Presentation]: User Interfaces-interaction styles

General Terms: Design, Human Factors, Management

Additional Key Words and Phrases: Browsing, complex objects, direct object manipulation, graph-oriented models, hypermedia applications, information filtering, visual interface

This work was supported by the Italian Electrical Energy Company under the research project

0 L.240 Multimedia Systems.

Authors’ addresses D, Lucarella, Centro Ricerca di Automatic, ENEL, Via Volta 1, 1-20093 Cologno Monzese, Milano, Italy and Dipartimento di Scienze dell’Informazione, University degli Studi di Milano, 1-20135 Milano, Italy; email: lucada(Q imicilea.cilea.it; A Zanzi, Centro Ricerca di Automatic, ENEL, Via Volta 1, 1-20093 Cologno Monzese, Milano, Italy; email: zanzifl cra.enel.it.

Permission to make digital/hard copy of part orall of this work for personal orclassroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication, and its date appear, and notice is

given that copying is by permission of ACM, Inc To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee.

@ 1996 ACM 0734-2047/96/0100-0003 $03.50

4. D Lucarella and A Zanzi

1, INTRODUCTION

Hypermedia has been simply defined as a system to manage a collection ofinformation that can be accessed nonsequentially It consists of units ofinformation that are arbitrarily diverse in form and content Such units maycontain texts, graphics, images, sound, video, and animation and are con-nected by logical links to form an information network The variety of nodesand links that can be defined makes hypermedia a very flexible environment

in which information is provided both by what is stored in each node and bythe way the information nodes are linked to each other In addition, currenthypermedia systems provide sophisticated user interface tools that enable thereader to inspect the node content and to navigate through the network byselecting paths to follow on the basis of interests emerging along the way[Nielsen 1990]

There is a growing interest today in such technologies for the tion of massive multimedia information systems, but unfortunately, severalwell-recognized problems continue to be open research issues [Halasz 1988].Among these, central points to be addressed are information modeling andinformation retrieval

implementa-1.1 Information Modeling

The simplicity of the basic hypermedia model does not appropriately sent the structure of the information There is an inadequate separationbetween a node in the hypermedia network and the content associated withthe node Conversely, a strong separation of the structure from the contentwould allow many structures to be superimposed over the same set ofinformation units or a unit to be shared among many nodes within a singlestructure In addition, a node is a storage unit for a collection of data itemswithout any structural information, and each node and link are assumed to

repre-be of the same type As a result, modeling is more or less a bottom-up process

in which we have to analyze how information can be broken down intodifferent elements and then to recognize these individual elements by addinglinks among them The problem here is that such an analysis is only usefulfor that particular instance, and we cannot use this same structure for otherinstances [Tompa 1989]

The key point is that the basic hypermedia data model is too simplistic It

is not suitable for modeling the real world or capturing its semantics asrequired in most applications [Furuta and Stotts 1990; Garzotto et al 1993;Schnase et al 1993a] As a consequence, the user has dif%culty in perceiv-ing the conceptual model of the application, resulting in cognitive overhead[Conklin 1987] In authoring mode, extra mental effort is needed to establishthe links required to connect the created nodes In reading mode, extramental effort is needed for choosing the path to follow through the network,with the risk of becoming lost or disoriented

One of the main ideas proposed by Garg [ 1988] is that information ded into the hypertext network should be described by a set of predefined

embed-domain objects In this way, the actual content of the hypertext would berepresented by a set of information objects, each of which is an instance of adomain object, inheriting by default all of the properties of the domain object.The idea can be compared to the notion of database schema, as opposed to aspecific instance of the database According to this trend, many hypermediasystems have been proposed with the support of underlying databases[Campbell and Goodman 1988; Christodoulakis et al 1986; Lange 1990;Schnase et al 1993b; Schutt and Streitz 1990].

Recently, requirements for representing the structurally complex tionships that arise in hypermedia have generated a renewed interest insemantic data models [Hull and King 1987] Semantic models attempt toprovide more powerful abstraction and structuring mechanisms for specifyingdatabase schemas in order to overcome the limited modeling capabilities oftraditional database systems [Beeri 1990; Lieberherr and Xiao 1993]

interrela-Schnase et al [ 1993a] presented a comparative analysis of semanticmodels, concluding that a structural object-oriented paradigm appears to besuperior for hypermedia modeling Of particular interest are graph-baseddata models since they provide a natural way of handling data that appear inapplications such as hypermedia or multimedia information systems Gyssens

et al [ 1990] proposed a graph-oriented object database model in which thedatabase schema as well as the database instances can be seen as graphswith the data manipulation language expressed in terms of graph transfor-mations Amann and Scholl [1992] presented a graph data model with anassociated algebraic language based on regular expressions over the datatypes and showed how such a language can be exploited for hypertextquerying In the same direction, in this article we propose a graph-basedobject model which provides high semantic expressibility, and we use it as auniform framework both for conceptual modeling and for direct manipulation

of the stored objects

1.2 Information Retrieval

In hypermedia information systems, interaction is mainly devoted to tion retrieval A canonical approach is based on formal querying [Bertino

informa-et al 1992; Straube and Ozsu 1990] Conversely, browsing techniques consist

of exhaustively viewing part of the information base until the desired mation has been found The former approach requires a deep knowledgeabout the query language, the conceptual structure of the application, andthe goals; the latter does not require a preliminary knowledge On the otherhand, a formal query, if correctly formulated, can be directly evaluated andmay yield an immediate answer, whereas a browsing session can take a longtime before converging to the goal or may not converge at all Between thesetwo mentioned interaction techniques, other approaches must be studied withthe aim of balancing expressive power and ease of use

infor-Some approaches to the integration of query-based retrieval strategies in ahypertext network have been proposed recently Logic-based languages have

6 D.Lucarella and A Zanzi

been proposed by Consens and Mendelzon [1989], Lucarella [1990], Afratiand Koutras [1990], and Beeri and Kornatzky [1990]; different attempts toexploit the hypertext links in the retrieval of the relevant nodes have beenreported by Croft and Turtle [1989], Lucarella and Zanzi [1993], and Frei andStieger [1992] A common aspect to such proposals is that no concept ofschema has been introduced, and thus, queries can be specified only over thehypertext network in order to get an optimal starting point for browsing.Conversely, as remarked in the previous section, the approach we aretaking in this work is based on a semantic data model, the primary objectivebeing to provide powerful visual constructs for representing a variety ofabstractions in a structured fashion Unfortunately, as soon as the underlyingdata model becomes more complex, the level of complexity of the associatedquery language and the level of knowledge required by the user also increase.The main goal becomes the design of a language that provides both highsemantic power and ease of interaction with the system

With this objective in mind, we propose a visual query paradigm The userperforms actions symbolically and directly on the screen and is able toexpress operations by grabbing and manipulating visual representations ofobjects The user is not required to know any complex formal language, withthe advantage of maintaining the same interaction style normally usedduring browsing The effect produced by the query is perceived as a form offiltering and navigation space restriction, So it is natural to pass fromquerying to browsing and vice versa, depending on the type of user, the type

of application, and the type of current needs By effectively combining ing and querying under a uniform interface, we provide an integrated view ofthe retrieval problem

brows-Much research has been carried out in the database community on cal query languages that has influenced our approach at different levels.Basic principles and a survey of such efforts can be found in C!atarci [1992]and Batini et al [1992] respectively Most graphical interfaces are based onintensional data models, typically the entity-relationship model [Angelaccio

graphi-et al 1990; Kuntz and Melchert 1989; Wong and Kuo 1982] or the extendedentity-relationship model [Auddino et al 1991; Czejdo et al 1990] ISIS[Goldman et al 1985] and its extension ISIS-V [Davison and Zdonik 1986]provide a visual interface to the semantic data model SDM [Hull and King1987] SNAP [Bryce and Hull 1986] is a system based on the IFO data model[Hull and King 1987] More recently, some projects have dealt with object-ori-ented data models [Epstein 1990], and DBface provides a tool for buildinggraphical interfaces to object-oriented databases [King and Novak 1993].The remainder of this article is organized as follows Section 2 provides adescription of the semantic model on which the MORE system is based Thissection also includes an example subschema that contains multimedia infor-mation about the organization and the activities of our research division Thevisual retrieval environment along with the formal definitions of perspectiveand the operations on perspectives are presented in Section 3 Variousexamples illustrating the expressive power of the language are presented

with reference to the example subschema shown in Section 2 Section 4sketches design issues for the MORE prototype system focusing on the mainfunctionalities and presenting visual interaction screendumps taken from theactual application Section 5 provides a comparison with related work Fi-nally, brief conclusions and future research work are outlined in Section 6.

The basis of the approach is the characterization of the information system interms of objects, attributes, and relationships, namely, a general object-ori-ented conceptual model An object is an entity of the real world, a concept, an

event, a process, or anything else that an application tries to capture andrepresent Objects have their own identity that does not change throughouttheir lifetime and are known by their properties The specific set of propertiesused to describe a given object depends on the point of view and the purpose

of the modeling We recognize properties only through attributes Objectshaving the same structural properties are grouped together to form an objectclass Classes can be related by a superclass-subclass relationship in which

an object in a subclass inherits the structural properties from its classes

super-Object attributes can be divided into two general categories: simple andcomplex The domain of a simple attribute is a system-defined basic type; thedomain of a complex attribute is a class

At the intensional level, the conceptual schema captures this semanticstructure It is defined by a collection of interrelated classes and types, and assuch it can be represented by a directed labeled graph Objects and classesare related by the instantiation relationship At the extensional level, theinformation system can be viewed as a collection of interrelated objects, and

as such it can also be represented by a directed labeled graph Thus, theinformation system can be represented by graphs at both the intensional andthe extensional level A formal definition of such concepts is given next

—T is a finite set of type names (e.g., integer, text, picture) built into thesystem; each t E T denotes a type of primitive object, and V(t) is the set ofassociated values

—A is a finite set of attribute names Attributes are defined on classes.Attributes may be simple or complex The domain of a simple attribute is a

8 D Lucarella and A, Zanzi

basic type t ● T; the domain of a complex attribute is a class c e C Inaddition, we distinguish between single-valued attributes As and multival-ued attributes A., with A = A, U Am.

—9 c C x A x (C u T) is the property relationship If (c,, a, Cj) =9, thenthe class c, has the attribute a, having as a domain the class or type Cj

—% c C x C is the inheritance partial ordering relationship If (cl, c, ) ●%,then the class Ci is a subclass of the class Cj inheriting attributes from CJ

Definition. Given X, the conceptual schema graph is a directed labeledgraph

G(Z) = (iV, E),where:

—N = C U T is the set of nodes For each c = C, we have a shaped node labeled c For each t E T, we have an oval-shaped nodelabeled t

rectangular-—E is the set of edges For each (c,, Cj) = % we have a bold edge connecting

Ci to CJ For each (c,, a, c,) = @ we have an a-labeled edge from Ci to Cj.Particularly, if a = As we have an edge with a single arrow; if a ● Am we

have an edge with a double arrow

Definition The multimedia information system M is defined as the tuple

four-M=(X, O, Y, P),

where:

—2 is the conceptual schema defined above

—O is the set of objects stored into the system

—> c O x C is the instantiation relationship Each object o = O is an stance of a class c = C

in-—% c O X A X (O U V(T)) is the link relationship (o,, a, Oj) =9 denotesthat the attribute a of the object Oi has the value Oj Assuming the o,instance of c, and the Oj instance of c,, we have (o,, a, o,) ~& iff one of thefollowing conditions holde:

Fig 1, Graph-based object model: Intentional and extensional levels

—E is the set of edges For each (o,, a, o~) = Y’, there is an a-labeled edgefrom o, to 0]

Based on this model, Figure 1 gives an example that shows how we can use

a graph-based representation at both the intentional and the extensionallevels Note the effect, at the extensional level, of the inheritance relationshipbetween the class student and the class person.

2.2 A Sample Hypermedia Application

In order to demonstrate the capabilities and the flexibility inherent in theapproach discussed, a hypermedia application has been developed The appli-cation is aimed at storing multimedia information concerning the structure ofthe organization and the activities of our research division It describes thehierarchical structure of the research units, including information aboutmanagement, personnel, financial budget, research projects, and project lead-ers A portion of the schema graph is presented in Figure 2 This schema

is used throughout the article as the knowledge base to which all visualoperations will be posed

With reference to Figure 2, rectangular nodes in the graph representclasses, and oval nodes represent basic types Labeled arrows starting from aclass depict the properties of that class Multivalued properties are shownwith double-headed arrows The bold lines express the inheritance is-u

relationship from a subclass (at the tail of the arrow) to its superclass

In the following, we describe in further detail the meaning of the objectsdepicted Research Unit groups the common attributes (name, direction,

10 D Lucarella and A Zanzi

Research direction

unitt+

I is-a

Division

Fig 2 Conceptualschemagraph

mission, personnel, and expenses) shared by the units at different

hierarchi-cal levels The Research Division represents the administrative and strategiccentral headquarters to which all of the research centers spread throughout

the country report The Research Center is a department, characterized by aspecific research area with its own laboratories The Laboratory is the

operative research unit, with its own equipment, in which the researchprojects are carried out

The Research Project is characterized by title, subject, description ofobjectives, project leader, and a short movie presenting its current state withthe main results achieved Note that some research programs are carried out

as joint projects, and consequently, a cycle is present in the graph The

Experimental Installation represents an installation characterized by itsname and location, where some experiments that cannot be made in thelaboratories are carried out in the field

The Person groups the common attributes (name, resume, and photo)shared by the manager and the project leader The Manager is the head of aresearch unit: the central division, a research center, or a research labora-

tory The Project Leader is a person who is in charge of a research project.Finally, the Employees class gives information about the personnel in aresearch unit, grouped by category and by age, respectively; and the Budget

class represents the financial planning of a unit, both in terms of theestimate of the expenses and of the balance

Note that the conceptual schema of the application is directly entered andmanipulated on the screen by the application designer supported by anappropriate visual tool (see Section 4)

3. VISUAL INFORMATION RETRIEVAL

In this section we deal only with the retrieval and presentation issueswithout considering other functionalities In addition, a clear distinctionbetween the information user and the information supplier is quite common

in these systems, since object loading and updating often require specializedmultimedia editors depending on the type of object manipulated

We have already discussed in the introduction the main reasons for oping a visual interface based on the direct-manipulation paradigm and theexpected advantages for the end users in terms of abstraction power, ease ofinteraction, and flexibility Basic requirements are the visualization of theconceptual schema as well as the database instances, by enabling the user tofilter the amount of information to be displayed Selective information visual-ization can be used to locate relevant information and to restrict the visual-ization to the pertinent parts

devel-A reasonable way to present complex information is to produce multipleviews of the same information, each focusing on different aspects and thusconforming to different needs The cognitive overhead required to face tan-gled information structures can be alleviated if the system presents only therelevant pieces of the stored information while hiding the rest, In analogywith the views in databases, we introduce the notion of perspective, 1 a form

of data abstraction that acts as a user interface, providing control over thevisibility of the system objects A perspective can be tailored to focus selec-tively on the subset of information that is significant to a particular applica-tion Essentially, perspectives are graph structures that are built from theschema graph and are visually manipulated in various ways Related works

on graph-based object manipulation are reported by Andries et al [ 1992] andGuo et al [1991]

In the following, we provide formal definitions for perspectives and a basicset of operations that can be performed on perspectives For each of these inturn, we give the formal definition, the visual expression, and an examplereferring to the previous application

—S is the set of object graphs generated by the perspective graph T throughthe instantiation relationship Given an instance s E S, each node o = N(s)

1The term perspect~[x, has already been introduced by Garzotto et al [ 1993], hut with a different meaning.

2A directed graph is weakly connected iff the graph obtained by removing the arrowheads is connected

ACM Transactions on Information Systems. Vol 14 No 1.January 1996

12 D Lucarella and A Zanzi

Fig 3 A perspective over the schema.

is an instance of the corresponding node c E IV(w) and the edge ( Oi, a, Oj)

= E(s) iff the edge (ci, a,cj) ● E(m)

So, a perspective is defined by a pattern (the intensional representation) and

by the corresponding object graphs (the extensional representation)

In order to define a perspective, the user has to build the pattern into the

“perspective window.” The requested nodes are copied from the “schemawindow” by pointing and clicking The system checks automatically that theresulting graph is connected In this way, incorrect perspectives cannot bespecified, since the patterns conform to the structure of the schema

In Figure 3 we show a perspective focusing on those parts of the tion system in which the user is interested In the example, attention isrestricted to the research centers and their laboratories including, for each ofthese, the research projects and corresponding project leaders

informa-Perspectives can be named, saved, reused, and manipulated in variousways In particular, it is possible to define perspectives on perspectives,thereby producing different levels of abstraction All of the operations onperspectives are closed, thus removing the major drawback of current object-oriented query languages that do not maintain the closure property [Shawand Zdonik 1990] Consequently, in our approach, the result of each operationhas the same structural properties as the original objects; thus, it can befurther processed by the same set of operators

3.2 Object Filtering

In order to restrict attention to a subset of pattern instances in the

perspec-tive, a filter can be defined over it.

ACM Transactions on Information Systems, Vol 14, No 1, January 1996

is-a equipment joint in-char e

M.lumdm Swwm

projects

subject presentation description

Fig,4 Filterspecification

Definition. Given a perspective P(T, S), a filter F is defined in terms of a

set of selection conditions {Cl, Cn) over the pattern Let a, be an attribute

of type tpertaining to a node (class) n, in the pattern m; then C, represents aselection condition over the actual values of the corresponding object in-stances The selection condition is a boolean combination A, v , 1 of simpleexpressions of the form (al e aj ), where al is a type-compatible property or a

constant, that is, some value from the domain t; e is a comparison operatordepending on the type of the operands

In Figure 4 a filter has been defined over the perspective shown in Figure

3, with selection conditions over two (shadowed) nodes In particular, thisexample is the visual expression of the following complex query: “I want toknow if, in the research centers named C.R A or C R 1.S., there are research projects in the field of Multimedia Systems, and if this is the case, I want tosee the laboratories where the activities are carried out and the responsibleproject leaders.” From the user’s point of view, one merely “clicks,” one byone, on the nodes in the pattern over which conditions have to be specified.The clicked node changes its color (shadow in the figure), and a text window

is opened to enable the user to enter the requested conditions

As is well known, in case of recursive properties (e.g., “joint” on “ResearchProject”), a selection condition specified over the class “Research Project” orrelated ones can have different semantics We decided to allow a very limitedform of recursion in order to guarantee immediate comprehension and ease offormulation, which has been our constant guideline in the design of thissystem So the specification of conditions either over the class “ResearchProject” or over related classes yields the retrieval of projects satisfying the

ACM Transactions on Information Systems Vol 14 No 1, January 1996

D Lucarella and A Zanzi

conditions Then the projects joined to each of them are retrieved Namely,the condition is intended as a starting condition for the computation of thetransitive closure The effect of a recursive property at the extensional level isshown in the next example

Definition Given a perspective P(m, S) and a filter F defined over P, a selection operation u returns a subset R c S of pattern instances matchingthe filter:

A pattern instance s matches the filter iff it satisfies all of the conditions

A i ~, CL; a condition C, over the class i is satisfied ifl it is true for thecorresponding object instance values

From the user’s point of view, after having specified the pattern and thefilter, it is enough to “click the “select” button The system notifies the user,resetting the button when (1) the query has been processed and (2) thematching instances have been identified In this way, it is possible to restrictthe attention to a subset of instances according to the conditions specified inthe associated filter From now on, it will be possible to access the singleobjects belonging to the retrieved set R of pattern instances or to iterate theprocess by further modifying the perspective

The impression perceived by the user is that a profile is defined denotingthe user’s interests; the system filters out useless information; and the usersees what is left The effect of this filtering capability is to restrict theattention to a manageable subset of nodes For a discussion on the featuresqualifying the information retrieval and information-filtering processes, seeBelkin and Croft [1992]

In the following example, we show the effect of the selection operation atthe extensional level Assume we have a basic perspective PI focusing on the

“Laboratory,” relative “Budget,” and “Research Project” with “Project Leader.”

In Figure 5(a), the pattern of the perspective is reported together with thecorresponding pattern instances (for simplicity, only rectangular nodes areshown, and link names are not reported) Figure 5(b) presents the instancesretained after the definition of the perspective Pz and the execution of aselection operation on the basis of the specified filter

Definition. Two perspectives P1(T1, S1) and Pz(~z, Sz) are said to be

compatible when they have the same pattern but different instance sets:Tr~= 7r2; sl # S2

This is the case resulting from the application of different filters to thesame original perspective

3.3 Basic Operations on Perspectives

Now we introduce basic binary operations to combine perspectives together

Definition Let Pl(nl, S1 ) and PZ(TZ, Sg ) be two perspectives, with ITl #

Vz and N(TI ) n N(T2 ) # O A composition operation @ over the set of nodesACM Transactions on Information Systems, Vol 14, No 1, January 1996.

Fig, 5 (a) Pattern and instances of perspective PI (b) Result of the selection operation defined

on perspective P2.

N’ = N(TI ) n N(mz ) generates the perspective P(m, S) = PI @ P2 where:

— r is the composition of the two patterns rl and r2, obtained by taking theunion of nodes and edges, respectively, N(n) = N(wl ) UN(mz ) and E(T)

= E(7r1) u E(m2); and

—S is the set of instances of the pattern n, obtained by composing theinstances in SI with those in Sz; two instances can be composed iff, for all

of the nodes (classes) N’ = N(ml ) n N(7Z ), they share the same objectinstance

Figure 6 gives an example showing the effect at the extensional level of thecomposition operation In Figure 6(a), the perspective P~ is shown defining afilter over the “Laboratory,” and then in Figure 6(b) we see the effect ofcomposing P~ with the perspective Pz (Figure 5(b))

Definition Let PI(T ~, S1) and P2(m2, S2 ) be two compatible perspectives

An overlay operation @ generates the perspective P(7, S) = PI @ Pz, where:

—*.=l fiz is the pattern; and

—S is the set of instances included in both of the perspectives, that is,