Tài liệu Character Animation from a Motion Capture Database pptx

Then, the hier-archical fragment method is introduced, which uses signal processing techniques,the skeleton hierarchy information and a simple matching algorithm applied to datafragments

Max-Planck-Institut f ¨ur Informatik Computer Graphics Group

Saarbr ¨ucken, Germany

Character Animation from a Motion Capture Database

Master Thesis in Computer Science

Computer Science DepartmentUniversity of Saarland

Edilson de Aguiar

Supervisors: Dipl Inf Christian Theobalt

Prof Dr Hans-Peter SeidelMax-Planck-Institut f¨ur InformatikComputer Graphics Group

Saarbr¨ucken, Germany

ii

Eidesstattliche Erkl ¨arung

Hiermit erkl¨are ich an Eides statt, dass ich die vorliegende Mastersarbeit selbst¨andig und ohne fremde Hilfe verfasst habe Ich habe dazu keine weiteren als die angef¨uhrten Hilfsmittel benutzt und die aus anderen Quellen entnommenen Stellen als solche gekennzeichnet.

Saarbr¨ucken, den 26 November, 2003

Edilson de Aguiar

iv

This thesis discusses methods that use information contained in a motion capturedatabase to assist in the creation of a realistic character animation Starting with

an animation sketch, where only a small number of keyframes for some degrees offreedom are set, the motion capture data is used to improve the initial motion qual-ity First, the multiresolution filtering technique is presented and it is shown howthis method can be used as a building block for character animation Then, the hier-archical fragment method is introduced, which uses signal processing techniques,the skeleton hierarchy information and a simple matching algorithm applied to datafragments to synthesize missing degrees of freedom in a character animation, from

a motion capture database In a third technique, a principal component model isfitted to the motion capture database and it is demonstrated that using the motionprinciple components a character animation can be edited and enhanced after it hasbeen created After comparing these methods, a hybrid approach combining theindividual technique’s advantages is proposed, which uses a pipeline in order tocreate the character animation in a simple and intuitive way Finally, the methodsand results are reviewed and approaches for future improvements are mentioned

Prof Dr Hans-Peter Seidel for their help and advice during the development

Meyer-Ross, for her help here in Germany For all my colleagues of the ComputerGraphics group at MPI, thank you, specially Volker Blanz, for his help with thePCA theory and Thomas Annen and Grzegorz Krawczyk for helping me with theLatex stuff

I also wish to thank my family, who always supported me, encouraging meand making me never give up, despite the distance Thank you Mom, Dad, Raquel,Rose and Enoc

Edilson de Aguiar

1.1 Motivation 1

1.2 Goals 2

1.3 Thesis Outline 3

2 Fundamentals of Character Animation 5 2.1 Introduction 5

2.2 Keyframing 5

2.3 Physical Simulation 6

2.4 Motion Capture 8

2.5 Project Implementation Aspects 10

2.5.1 Motion Capture Database 11

2.5.2 Skeleton Model 12

3 Multiresolution Filtering Method 15 3.1 Introduction 15

3.2 Related Work 16

3.2.1 Signal Processing Methods 16

3.2.2 Multiresolution Methods 16

3.3 Multiresolution Filtering Method 16

3.4 Multiresolution filtering on motion data 18

3.5 Application to Character Animation 19

3.6 Experiments 20

3.7 Discussion 21

4 Fragment Based Methods 25 4.1 Introduction 25

4.2 Related Work 26

vii

CONTENTS viii

4.3 Overview 27

4.4 Motion Analysis 31

4.4.1 Motion Phases 31

4.4.2 Frequency Analysis 33

4.4.3 Correlation 33

4.5 Motion Synthesis and Texture 34

4.5.1 Fragmentation 34

4.5.2 Matching 35

4.5.3 Joining 36

4.5.4 Smoothing 39

4.6 Experiments 40

4.7 Hierarchical Fragment Method 41

4.7.1 Skeleton Hierarchy and Correlation 43

4.7.2 Method 44

4.7.3 Experiments 47

4.8 Discussion 48

5 Principal Component Analysis 53 5.1 Introduction 53

5.2 Related Work 54

5.3 Principal Component Analysis 55

5.3.1 Overview 55

5.3.2 PCA Theory 55

5.3.3 Data Compression 57

5.4 PCA for motion synthesis 58

5.4.1 Overview 58

5.4.2 Motion Synthesis 58

5.5 Experiments 59

5.6 Discussion 62

6 Hybrid Approach 67 6.1 Introduction 67

6.2 Hybrid Approach 68

6.3 Experiments 69

6.4 Discussion 70

7 Conclusion and Future Work 77

List of Figures

capture the motions used in the database In (a) it is shown thecamera setup and in (b) the subject performing the motion Images

respectively the z-angle values for the pelvis, hip, clavicle, forearm

(shown only for the first 40 frames) The band g0 corresponds tothe original signal The low-pass bands corresponding to the highfrequency are g1 and g2, to the middle are g3 and g4, and to the

(shown only for the first 40 frames) The band-pass bands sponding to the high frequency are l0 and l1, to the middle are l2

ix

LIST OF FIGURES x

and motion capture data are decomposed into frequency bands; (b)animators set the general method parameters Driven and masterjoints are chosen to guide the method, and a particular frequencyband of the master joint is chosen to guide the fragmentation step.Joints in black will be textured and in blue and red will be synthe-

fragmenta-tion (a), matching (b), joining (c) and smoothing At the end, anoriginal keyframed character animation is enhanced by synthesis

human walking cycle; (b) the right hip z-angle values is plotted,

ex-amples in the database The shape shown in red demonstrates a

(b) fragments created at locations where the first derivative changes

fragments are compared with the driven fragments (b) being stretched

or compressed properly (c) At the end, a number of good

are concatenated or blended (a) Three different criteria were testedand compared: (b) best fragment; (c) cost matrix and (d) best ani-

multiplied with a smoothing function (top right) and the result isadded back to the original motion signal In this way, the continu-ous version shown on the bottom left is generated Adapted from

LIST OF FIGURES xi

frag-ments generated in the matching step for the z-angle of the pelvisjoint are shown The final sequence generated by the three ap-

4.10 Example of possible joint correlations: (a) represents a bad relation between joints that do not belong to the same kinematicsub-chain; (b) represents a good correlation between joints that be-

4.11 Example of the root generation stage: (a) driven joints (red) androot joint (green) are shown; (b) the parents of the driven joints aregenerated; (c) next parent joint is generated and (d) root joint is

4.12 Example of the motion generation stage: (a) root joint; (b) rootjoint together with the original driven joints are used to texture orsynthesis of its children; (c) next children are generated; (d) all

4.13 Example of the Hierarchical Fragment method being applied to ahuman character: (a) shows some frames from the initial characteranimation where only lower body joints are keyframed; (b) showsthe respective frames from the resulting character animation whereupper body joints are synthesized and lower body joints are textured 504.14 Another example of the final method being applied to a humancharacter: (a) shows some frames from the initial character an-imation where only upper body joints are keyframed; (b) showsthe respective frames from the resulting character animation wherelower body joints are synthesized and upper body joints are textured 514.15 Another example of the final method being applied to a humancharacter: (a) shows some frames from the initial character anima-tion where all left body side joints are keyframed; (b) shows therespective frames from the resulting character animation where all

LIST OF FIGURES xii

keyframed and motion capture data (a) In (b) one PCA model iscreated for each missing DOF In (c) one PCA model is created foreach frequency band of a missing DOF In (d) one PCA model iscreated for each sequence position of a particular frequency band

frames from the initial animation where the left hip, knee, elbowand upper-arm joints are keyframed; (b) shows the respective frames

of the resulting animation where new motion for the joints are

(a) arm positions are modified by altering the principle componentsfor the right and left clavicle joints; (b) leg positions are also modi-fied by altering the principle components for the right and left knee

animation where left body side joints are keyframed; (b) shows therespective frames after applying the PCA method; (c) and (d) showpossible editing capabilities of the PCA, where the influence of the

initial animation where left body side joints are keyframed; (b)shows the respective frames after applying the first stage (PCA);(c) shows the same frames after applying the Multiresolution Fil-tering method to decrease the low frequency bands, generating a

ini-tial animation where left body side joints are keyframed; (b) showsthe respective frames after applying the first stage; (c) shows theframes after applying the second stage; (d) shows the same framesafter applying the Hierarchical Fragment Method in order to im-

LIST OF FIGURES xiii

the initial animation where left body side joints are keyframed; (b)shows the respective frames from the animation generated auto-matically by the hybrid method; (c) and (d) show the same framesfrom the resulting animations generated when motion principlecomponents are used to change arm and leg positions by adding

LIST OF FIGURES xiv

As the technology for motion capture has improved and the cost decreased, theinterest in using this approach for character animation has also increased The main

challenge that an animator is confronted with is to create sufficient detail in order

to generate an character animation with a realistic appearance Achieving detail in

a keyframed animation is extremely labor intensive However, with motion capture

the details are immediately present In other words, the data contains the motion

signature.

The main problem with motion capture is the lack of flexibility For instance,after collecting the data it is difficult to change it Due to this problem, manyanimators have little interest in using motion capture data Although keyframing islabor intensive, the animator can make a character do exactly what he wants it to

1

CHAPTER 1 INTRODUCTION 2

Since it is often difficult to know exactly what motions are needed before ing a motion capture session, many techniques have been developed to edit motioncapture data after it has been collected (see Sec 2.4) In motion editing, two dif-ferent aspects should be considered: first, it is important to take care not to alterthe motion in such a way that the detail is lost Second, the system should not justprovide editing capabilities but it should capture the essence of the motion, sincethe animator may want a completely different action

enter-Our intent in this project is to consider the case that an animator wants to use

a number of existing motion sequences, for instance stored in a motion capturedatabase, to generate new motions The idea is to use the style and life-like qualities

of the motions in the database to add details and a particular style to an initialkeyframed animation Then, a different approach to create a character animation

is proposed: the animator starts the animation with keyframing, a method that he

is familiar with, to create some degrees of freedom After that, the motion capturedatabase is used interactively to enhance the initial keyframed animation

At the end, using the strengths of keyframing and motion capture, the characterperforms realistic motion while preserving the keyframed style and incorporatingthe details of the motions in the database As a result, the animator does not need

to spend hours defining key poses and the expensive motion capture session can bekept at a minimum

1.2 Goals

As mentioned in the previous section, the main goal of this work is to combine thestrengths of keyframing and motion capture in order to simplify the process of cre-ating character animations In recent years, different approaches have been used inorder to achieve this goal Liu and Popovic [LP02] presented a method for proto-typing realistic character motion using a constraint detection method that automat-ically generates the constraints by analyzing the input motion Tanco and Hilton[TH00] presented a system that synthesizes motion sequences from a database ofmotion capture examples using a statistical model created from the captured data.Pullen and Bregler [PB02] described a method to enhance an initial keyframed an-imation using motion capture data, by decomposing the data into frequency bandsand fragments

In this project, we want to analyze important characteristics of these methods,

by implementing and comparing two different approaches (chapters 4 and 5) and

CHAPTER 1 INTRODUCTION 3

verifying the importance of some aspects of human motion: skeleton hierarchy,joint correlation, motion frequency range and motion phases In addition, we try toanalyze and synthesize motion by fitting a principal component model

Another goal is to understand the details of the characteristics of human tion: variations in repetitive motions and differences in the same movement exe-cuted by different individuals A detailed understanding of these data is important

mo-in fields like biomechanics and medicmo-ine, where many quantitative studies of man motion have been conducted for the purposes of treatment and prevention ofinjuries [AA00] This understanding is also interesting for character animation,since the details of a motion usually reveal mood or personality

hu-Using this understanding, in a long term we intend to develop a system able todescribe and characterize the character motion in a high level way For instance,using simple parameters and descriptions, the system will be able to create charac-ter animations reflecting aspects like gender, mood and personality

1.3 Thesis Outline

Chapter 2 gives a brief review of the three main animation methods Their tages and disadvantages are described in more detail with references to the relatedwork In addition, some implementation aspects of our project are mentioned: theskeleton model and the motion database that was used

advan-Chapter 3 presents the multiresolution filtering technique Possible motionediting capabilities of this technique are investigated and it is shown how thismethod can be used as a building block for character animation

In chapters 4 and 5 methods implemented to enhance an initial keyframed mation are described Chapter 4 introduces the fragment based methods, showingthat they can be successfully applied to character animation After decomposingkeyframed and captured data into frequency bands, motion phases are used to di-vide the motion capture data in small pieces, which are used to improve the originalkeyframed animation

ani-In chapter 5 a principal component model is fitted to the motion capture databaseand it is shown that using motion principle components it is possible to create, editand enhance a character animation

Using the techniques described in the previous chapters as components, ter 6 introduces a pipeline solution to interactively control the creation of a charac-ter animation, which results in a better performance and more expressive results

chap-CHAPTER 1 INTRODUCTION 4Finally, in chapter 7 all techniques and their respective results are briefly re-viewed and approaches for future improvements are mentioned.

of these animation methods will be reviewed in more detail on the basis of theirrelevant related work.

In the last section, some important implementation aspects related to the projectare mentioned: the skeleton model and the motion capture database that was used

in order to create the motion curves that drive the model action

The main problem with keyframing is that this technique is time-consumingand labor intensive A typical articulated kinematic model, such as a humanoidcharacter, usually has at least 50 degrees of freedom In keyframing, an animatormust animate all these DOFs, one at a time To construct a more realistic model, its

5

CHAPTER 2 FUNDAMENTALS OF CHARACTER ANIMATION 6

complexity is increased and the animator must keyframe more degrees of freedom.Usually, constraints, like position of legs and arms at specific times, are always

a problem because all DOFs should be specified at these positions in the character

One possible way to treat this problem is by using inverse kinematics to simplify

keyframe definition In this case, posing arm or leg, it is possible to calculate theposition of parent joints - forearm, hip and knee, in order to permit the character toachieve this point

Another problem with this technique is the interpolation process Usually terpolation is done with smooth splines which fails to model variations in highfrequency that real human motion has The number of keyframes set by the anima-tor is also an important aspect If too few keyframes are set, the motion may lackthe details usually seen in live motion To overcome this problem, trained anima-tors achieve a high level of detail by setting more and more keyframes, but in thiscase, at the expense of more time and effort

in-Although keyframing is extensively used by animators, nowadays it is not atopic of much research Recent works related to keyframing are trying to improveits quality using noise functions to describe variations in the motion signature.Bodenheimer et al [BSH99] described how to introduce natural-looking variabilityinto cyclic human motion animation using a noise function Perlin and Goldberg[PG96] presented Improv, a system for scripting interactive actors in a virtual worldusing Perlin noise functions [Per85] to characterize personality and mood in humanmotion

2.3 Physical Simulation

This technique tries to reduce the animator’s work by using physics to determinethe motion in situations where it can be clearly specified Although these methodshave been successfully applied for animating cloth deformation (DeRose [DKT98]),rigid bodies (Baraff [Bar94] and Moore [MW88]) and to fluids (Foster and Fedkiw[FF01]), the application of physics based methods for the generation of charactermotion remains challenging

Most of the work in physical simulation was done by researchers seeking curate models for use in biomechanical studies, as stated in Pandy [Pan01] Suchmodels were found to be very complex: more than one muscle can control onejoint, muscles exert nonlinear forces on tendons and joints usually have complexkinematics, involving sliding and rotation about multiple axes, as mentioned in

ac-CHAPTER 2 FUNDAMENTALS OF CHARACTER ANIMATION 7

Delp and Loan [DL00]

Clearly, this type of modeling is not practical for computer graphics, where ananimator wants to quickly compose a wide variety of motions In fact, an animatormost certainly will not know the proper configuration of muscles and bones or theamount of energy required in a specific movement

An alternative way to create animations of articulated figures using physics

was done by Witkin and Kass [WK88] in the method called spacetime constraints.

Considering the entire animated motion as a numerical problem, in contrast to mostprevious methods that consider each frame independently, spacetime constraintmethods allow motion editing while preserving the characteristics of the originalmotion and motion animating based on incomplete observations The general ap-proach is to specify some physical parameters of the character, like masses of eachlimb and spring constants of the joints Then using constraints, like leg and armpositions at a specific time, an animator can control the character key positions Atthe end, the motion is determined by solving a constrained optimization problem,where the character energy is minimized while the constraints are satisfied.However, normally it is difficult to model the complex human motion and cor-rectly specify masses and joints spring constants Another problem is the highcomputation cost for solving the constrained optimization problem The most suc-cessful attempts at using physical simulation of human motion come from roboticsresearch For instance, Raibert [Rai02] introduced some form of feedback control

in the robot movement instead of just solving the equations of motion and ing the proper initial conditions

predict-Applying the same principle to character animation, Hodgins et al [HWBO95]developed a method to apply control systems to virtual humans In other work,Raibert et al [RH91] applied these controls to create animations of humans per-forming athletic events like running, biking and a gymnast vaulting Althoughthe characters vaguely performed the activity being simulated, the motions did notlook realistic Another problem is that each activity was treated separately, while itturned out that their method cannot be used in the same way for different activities.Due to lack of control, difficulty of use, instabilities and high computation cost,the use of physics simulation for animation of a complete character is not widelyused yet One possible way to include physics in character animation is by usingspacetime constraints with other techniques, as described in the next section

CHAPTER 2 FUNDAMENTALS OF CHARACTER ANIMATION 8

2.4 Motion Capture

In this technique joint angles of a performing actor are recorded via sensors Thesevalues are then used to create a character motion (Menache [Men99]) In the past,such data was extremely difficult to obtain, as the sensor technology was costly.However, recently the technology has improved, the costs decreased and, therefore,this technique is becoming more available for general use

Currently the most common techniques for obtaining motion capture data aremechanical, optical, magnetic and video-based systems In a mechanical system,tracking is performed by having the subject wear a mechanical device, or exoskele-ton Then, angle measuring equipments are located at exoskeleton joint locations

By measuring the joint angles of the exoskeleton, subject limb orientation is tained The main advantage of this system is that it is accurate, cheap and allowsthe use of haptic devices in order to generate feedback reactions However, it is noteasy to perform fast and expressive motions due to the weight of the exoskeletonand the limited range of the angle measurement devices In addition, problems due

ob-to shift positions of the exoskeleob-ton cause errors in the motion capture process

In an optical system, retro-reflective markers are attached to the body of thesubject Then, a set of cameras surrounds the space where a subject moves andeach camera sends out a beam of infrared light, which is reflected back from themarkers After the marker locations are recorded as 2D frames positions in thecamera image planes, post-processing finds the 3D location of each marker at eachtime instant, and then solves for the joint configurations The main advantages

of optical systems are the very high rates of data collection and the possibility tocreate a great range of motion in a relatively large space However, it requires in-tensive post-processing computations and presents problems related with occludedmarkers and with captured motions from more than one subject, where the markerscan overlap each other

In a magnetic system, a known magnetic field is set up and a subject wearssensors that detect location and orientation of each limb based on the magneticfield This technique allows real-time data collection and there are no problemswith occlusion On the other hand, this method is very sensitive to the area wherethe motion is performed In addition, due to the wires that must be attached tosensors, many motions are awkward for the subject

Systems of the last category, video-based systems, try to get the motion data bymerely using a couple of video cameras Although it is an interesting technique,

CHAPTER 2 FUNDAMENTALS OF CHARACTER ANIMATION 9

it is complicated Standard computer vision techniques do not work for an ulated figure, in which many of the motions cannot be defined by a simple affinetransformation, but involve rotations about all the joints in the kinematic chain.Currently, much research is trying to make video-based motion acquisition moreaccurate and, therefore, useful for motion capturing

artic-The interest in using motion capture for creating character animation is ing The main reason is that this technique can provide motion data for all degrees

increas-of freedom at a high level increas-of detail For instance, using motion capture all the tails of a motion are inherent in the data, hence, coming for free In addition, atransfer of data to a different character with different skeleton dimensions is alsopossible

de-On the other hand, this technique also has disadvantages The main problem isthe lack of flexibility Since it is difficult to modify a captured motion after it hasbeen collected, an animator should know exactly what he wants In general this isnot the case since the process of creating a character animation is normally evolu-tionary Usually, an animator has only a coarse impression of the desired motionbefore he captures it, minor connections might be needed anytime thereafter Inaddition, motion capture sessions are still costly and labor intensive, which makesthe repetition process prohibitive

As a result, a great deal of research in recent years aimed at providing betterways of editing motion capture data after it is collected A general approach is toadapt the motion to different constraints while preserving the style of the originalmotion Wiley and Hahn [WH97] developed a method providing inverse kinemat-ics capability by mixing motions from a database to create a new animation thatmatches a certain specification Witkin and Popovic [WP95] developed a method

in which the motion data is warped between keyframe-like constraints set by theanimator Warping is done by overlapping and blending motion clips Rose et

al [RCB98] developed a method which uses radial basis functions and low orderpolynomials to interpolate between example motions while maintaining inversekinematic constraints

As mentioned in the previous section, spacetime constraints can be used in

or-der to include physics in character animation Gleicher [Gle97] presented a methodthat allows an animator to start with an initial animation and to interactively reposethe character A spacetime constraint solver is then used to minimize the differencebetween the new and old motion, subject to constraints specified by the animator

A similar approach was also used by Gleicher [Gle98] to retarget motions to

char-CHAPTER 2 FUNDAMENTALS OF CHARACTER ANIMATION 10

acters of different dimensions Lee and Shin [LS99] combined a hierarchical curvefitting technique with an inverse kinematics solver to adapt the motion Popovicand Witkin [PW99] developed a method that uses a reduced dimensionality spaceand dynamics to perform the editing process Rose et al [RGBC96] describedthe generation of motion transitions using a combination of spacetime and inversekinematics constraints in order to create seamless and dynamically plausible tran-sitions between motion segments

A more general problem with motion capture is that it is not an intuitive way tostart a character animation Since many factors (such as the environment) influencethe motion, the final motion sequence will not be known with all details right fromthe beginning Animators are usually trained to use keyframes They will oftenbuild an animation by first making an initial motion sketch with a few keyframesand add complexity and detail on top of that later

Therefore, combining the strengths of keyframing and motion capture, the cess of animating a character is expected to be simplified In our approach, ananimator starts a character animation with keyframing, a method that he is familiarwith, to animate some degrees of freedom After that, he uses the motion capturedatabase interactively to enhance the initial animation

pro-2.5 Project Implementation Aspects

The methods described in the next chapters are implemented in our prototype tem using the C++ programming language on the Linux platform In order tofacilitate the skeleton and animation manipulation, the free open-source character

The character animation library, CAL3D, is coded in C++ and uses the STLcontainers to store the data It provides basic data structures for skeleton-basedcharacter animation: sequencing and blending of animations, handling of bones,skeletons, materials and meshes The character motion is composed by two kinds

of transformations: translation and rotation Translations are represented as vectorsand rotations as quaternions In comparison with other rotation representations(e.g explicit matrices or Euler angles), keyframe interpolation using quaternions

is accurate and intuitive

1

http://cal3d.sourceforge.net/

CHAPTER 2 FUNDAMENTALS OF CHARACTER ANIMATION 11

2.5.1 Motion Capture Database

capture company that provides a set of motion sequences to the research nity The company uses a VICON 8 optical motion capture system with 8 to 24cameras for data acquisition Using cameras placed around the capture space totrack the positions of markers attached to the body of performers, triangulation isused to compute the 3D position of a marker at any given sample, from an array of2D information from every camera For data processing the Diva software is used,which applies template matching software algorithms in order to solve occlusionsand marker disappearance problems

commu-Using markers attached to the body of a subject, cameras surrounding the ject send out a beam of infrared light, which is reflected back from the markers.After the marker positions are recorded in 2D from each camera position, post-processing finds the 3D location of each marker at each point in time, and thensolves for the joint configuration At the end, the data is delivered as a set of trans-lations and rotations of the joints of an articulated body which corresponds withthe description of a human character Figure 2.1 shows the motion capture sessionand equipments used to create the motions in the database

sub-To evaluate the methods presented in this work, all the sequences of the databasethat represent variations of one particular action were considered In our project wechose human walking, since it is the most common action that need to be animated

in a realistic way Starting with an initial database of 129 different motions, each of

the length of all animations Since walking can be considered as a cyclic motion,

we created the final animation by repeating the original four times

At the end, each animation in the database has the length of 160 frames Usingthe exporting software provided by the CAL3D library, these animations were ex-ported to be used in our prototype as CAL3D animation files Figure 2.2 shows themotion data plotted over time for different degrees of freedom (i.e joint angles)for a particular animation sequence in the database

2

http://www.e-motek.com

3

http://www.discreet.com/products/3dsmax/

CHAPTER 2 FUNDAMENTALS OF CHARACTER ANIMATION 12

(a) camera setup surrounding the

space where a subject performs a

mo-tion

(b) markers attached to the body of a subject performing the desired mo- tion

Figure 2.1: Example of the motion capture session and equipments used to capturethe motions used in the database In (a) it is shown the camera setup and in (b) the

subject performing the motion Images used from http://www.e-motek.com.

2.5.2 Skeleton Model

In the skeleton-based approach, a skeleton model is a kinematic chain consisting ofbones and connecting joints Figure 2.3 shows the graph structure that specifies anarticulated figure defining a character Each bone in the skeleton, drawn in white, is

a link in the graph and represents a limb Joints, drawn in blue, connect the bonesand represent rigid body transformations between them Translations are repre-sented by vectors and have three degrees of freedom Rotations are represented byunit quaternions and also have three degrees of freedom

As shown in figure 2.4, a skeleton can be treated as a hierarchical model, whereeach joint has a parent and children Joint rotations or translations of a parent af-fects all of its children joints The paths from the skeleton root to the terminatingjoint in the skeleton, shown in the figure, can be treated as kinematic sub-chains,exhibiting different behaviors depending on the motion being performed For in-stance, in a walking motion, the lower kinematic sub-chain, composed by pelvis,hip, knee, foot and toe, is more active, being responsible for the translational move-ment of the body On the other hand, the upper kinematic sub-chain, composed bypelvis, spine, elbow, upper-arm, forearm, hand and finger, helps controlling thebody equilibrium

CHAPTER 2 FUNDAMENTALS OF CHARACTER ANIMATION 13

by three angular degrees of freedom In total, the skeleton model representing thehuman character has 66 degrees of freedom

This skeleton is created in 3D Studio MAX using the biped tool from Character

files in the database

4

http://www.discreet.com/products/cs/

CHAPTER 2 FUNDAMENTALS OF CHARACTER ANIMATION 14

TOE FOOT KNEE

HIP SPINE

ELBOW

UPPERARM FOREARM FINGER HAND NECK

by three degrees of freedom, where each DOF can be treated as a discrete sampledsignal, corresponding to its values at each time instant, also known as frame.

This chapter describes how a technique called multiresolution filtering can be

applied to motion capture data The idea is to use a range of filter-banks to pass themotion signal through a cascade of low-pass filters to produce a set of short-timeband-pass and low-pass signal components After filtering, the motion signal isrepresented as a collection of frequency bands where low frequency bands describethe global pattern of the motion signal, the middle frequency bands provide themotions details, and the high frequency bands usually contain noise like jitter andwiggling

Therefore, decomposing the motion data into frequency bands, existing motiondata can be edited interactively by amplifying or attenuating a particular frequencyband and new motions can be generated by band-wise blending of existing mo-tions In addition, this method can be used to concatenate and to compare motionsequences It will be shown in the subsequent chapters that this functionality can

be used as a building block for character animation

15

CHAPTER 3 MULTIRESOLUTION FILTERING METHOD 16

3.2 Related Work

A number of different approaches have been presented in the literature using signalprocessing analysis for editing motion data Here they are conceptually categorized

in signal processing methods and, more specifically, multiresolution methods

3.2.1 Signal Processing Methods

Unuma et al [UAT95] used Fourier analysis techniques to interpolate and olate motion data in the frequency domain In this work, they could manipulatemotion data and alter the style through interpolation, extrapolation and transitionaltasks Liu et al [LGC94] reported that adaptive refinement with hierarchicalwavelets provides a significant speed-up for spacetime optimization Amaya et

extrap-al [ABC96] used Fourier analysis to generate emotional animation from neutralhuman motion

3.2.2 Multiresolution Methods

The notion of multiresolution analysis was invented by Burt and Adelson [BA83],who introduced a multiresolution image representation, the Gauss-Laplacian pyra-mid, to facilitate such operations as seamless merging of image mosaics and tem-poral dissolving between images Lee and Shin [LS00] developed a multireso-lution analysis method that guarantees coordinate-invariance without singularity,using hierarchical displacement mapping and motion filtering In another work,Lee and Shin [LS01] presented multiresolution motion analysis as a unified frame-work to facilitate a variety of motion editing tasks Bruderlin and Williams [BW95]adopted a digital filter-bank technique to address multiresolution analysis of dis-crete motion data Their hierarchical representation of a motion with frequencybands allows level-by-level editing of motion characteristics

3.3 Multiresolution Filtering Method

Originally the method was developed for image mosaics [BA83] Applying themultiresolution method to a two-dimensional image, the first step is to obtain the

obtained by repeatedly convolving a small weighting function (w(l, m)) with the

image, while it is sub-sampled by a factor of 2 at each iteration (figure 3.1) This

CHAPTER 3 MULTIRESOLUTION FILTERING METHOD 17

Figure 3.1: Generation of the Gaussian pyramid The value of each node in the

this example a sub-array of length five is used Adapted from [BA83]

process is repeated until the image size is reduced to one, which is the average

intensity, or constant value (DC).

The pattern of weights w(l, m) used to generate each pyramid level from its

predecessor is called the generating kernel These weights are chosen subject tofour constraints:

• For computational convenience, the generating kernel is separable:

in sample density

Finally, the low-pass and band-pass bands can be amplified or attenuated arately and the image can be reconstructed by adding up all the band-pass bandsplus the constant value (DC)

sep-CHAPTER 3 MULTIRESOLUTION FILTERING METHOD 18

3.4 Multiresolution filtering on motion data

Following the algorithm described in Bruderlin and Williams [BW95], the ciples of image multiresolution filtering will be applied to a particular degree offreedom of human motion, represented as a one-dimensional signal over time In-stead of constructing a pyramid of low-pass and band-pass sequences by reducing

prin-their lengths, the lengths are kept the same, but the filter kernel (w(m)) is expanded

at each level by inserting zeros between their values For example, using a initialkernel of length five (weights a, b and c), the following kernel sequence will begenerated:

The weights (a, b and c) can be found using the constraints described in section

The length of the signal determines the number of frequency bands, which

 

), the

number of computed bands (nb) is N.

Then, the following steps are performed in order to create the Gaussian (G) andthe Laplacian (L) pyramids:

con-volving the signal with the expanded kernels:

    
  (3.1)This can be calculated efficiently by keeping the kernel constant and skippingsignal data points:

do-main, the signal is reflected on its boundaries

CHAPTER 3 MULTIRESOLUTION FILTERING METHOD 19

• The Laplacian sequences are calculated by subtracting two successive

During the reconstruction of the signal, each frequency band can be summed

increase or decrease the influence of each frequency band in the final reconstructedsignal, generating slightly different signals and maintaining the basic information

3.5 Application to Character Animation

Looking at the motion data for a particular DOF as a one-dimensional signal overtime, the algorithm described in the previous section is applied to the motion cap-ture data motivated by the following intuition: low frequencies contain generaland gross motion patterns, middle frequencies contain the signature of the motionand the high frequencies describe some details and possible noise Then, treatingeach degree of freedom as a one-dimensional signal, it is possible to calculate itsrespective low-pass (G) and band-pass (L) pyramids

Using the Gaussian and Laplacian pyramids, the signal representing the gree of freedom can be reconstructed by summing up all band-pass bands plus theconstant value (DC), as mentioned in section 3.3 By changing the reconstructionweights, it is possible to increase or decrease the effect of each frequency band

de-on the final motide-on signal recde-onstructide-on Different sets of weights will result indifferent motions, since each frequency band generally contains a particular infor-mation or detail of the overall motion In this way, it is possible to perform someanimation editing tasks In addition, using the extracted frequency bands it is pos-sible to create new motions by blending and concatenating frequency bands fromtwo different existing motions

Examples of the Gaussian and Laplacian pyramids generated by the algorithmare shown in figures 3.2 and 3.3 Considering the pelvis z-angle as a sampled

CHAPTER 3 MULTIRESOLUTION FILTERING METHOD 20

signal, figure 3.2 shows the generated Gaussian pyramid The generated Laplacianpyramid is shown in figure 3.3

3.6 Experiments

To demonstrate the frequency-based decomposition and motion creation power ofthe multiresolution method, the following experiments are performed Using ahuman walking sequence with 160 frames as an example, six frequency bandsare generated Then, the multiresolution approach is simultaneously applied to alldegrees of freedom of the skeleton model (see section 2.5.2) The same frequencyband gains are used for all degrees of freedom and the resulting motion is generated

recon-CHAPTER 3 MULTIRESOLUTION FILTERING METHOD 21

sults can also be achieved by changing the low frequencies (bands 5, 6) Increasingthe low frequencies attenuates the motion, and decreasing accelerates it Since thehigh frequency bands generally represent noise, their changes resulted in a addition

of a nervous twitch to the animation

3.7 Discussion

The multiresolution filtering technique provides a rapid interactive loop, and cilitate the reuse and the adaptation of motion data by amplifying or attenuatingimportant frequencies of the motion data This technique can provide a high-levelmotion control and facilitates to reuse and adapt existing motions of an articulatedcharacter In the last instance, it can serve as building block for high-level charactermotion processing

fa-Multiresolution filtering is not usable as a stand-alone technique for characteranimation A drawback of this method is that some constraints such as joint limits

or non-intersection with the floor can be violated in the filtering process However,

it can be solved by employing constraints or optimization after the definition of the

CHAPTER 3 MULTIRESOLUTION FILTERING METHOD 22

general motion Nevertheless, the frequency decomposition can be used to extractcertain features of a motion which can be used during character animation.Combining the multiresolution filtering with the techniques described in chap-ter 4, 5 and 6, a powerful tool has been developed which can be used in the follow-ing tasks:

• Concatenating motions: If two motion sequences should be concatenated,the multiresolution method gives control over the transition zone and thetransition weight coefficient allowing concatenation band-by-band

• Comparing motions: If two motion sequences should be compared, the erated low-pass and band-pass bands can be used in order to perform a com-parison of only a particular frequency band, containing specific information

gen-• Blending motions: The multiresolution method allows blending of an imation band-by-band, where only important bands are added in order to

an-generate a more realistic motion In this case, it is possible to copy a

partic-ular information contained in a frequency band from one signal to another

CHAPTER 3 MULTIRESOLUTION FILTERING METHOD 23

(a) original motion (b) reconstructed motion with

(c) signal comparison (only the first 40 frames)

Figure 3.4: Using multiresolution to increase the gain in the middle frequencies

CHAPTER 3 MULTIRESOLUTION FILTERING METHOD 24

(a) original motion (b) reconstructed

mo-tion with decreased gain

(c) signal comparison (only the first 40 frames)

Figure 3.5: Using multiresolution to decrease the gain in the middle frequencies

tech-an tech-animator tech-and using the details contained in fragments extracted from a motioncapture database.

The main goal is to create a character animation semi-automatically, followingthe same procedure that an animator would do manually Animators are trained touse keyframing, and will often build a character animation by first making a roughanimation with few keyframes to sketch out the motion Complexity and detailsare added on top of that later

Using the same idea, a fragment based method combines the strengths of ing and motion capture Starting from an initial character animation where only fewdegrees of freedom were keyframed, our method will use the information from themotion capture database to add details to the keyframed DOFs and to completelysynthesize the remaining DOFs At the end, an improved character animation will

keyfram-be generated, based on the animator’s keyframed sketch and performing what hewanted

By combining the strengths of keyframing and motion capture, the creationtime of an animation and hence the overall animation costs are reduced As stated

25

CHAPTER 4 FRAGMENT BASED METHODS 26

in chapter 1, an animator does not need to spend hours defining key poses andexpensive motion capture sessions can be kept at a minimum In addition, ourmethod can be used by someone who does not know the details about the motion

he wants to create Using a set of captured motions, for example downloaded from

a database resource on the Internet, he can sketch the desired motion by keyframingsome DOFs and then use this method to develop a enhanced version of it

4.2 Related Work

Considering a general animation, some work have addressed the problem of thesizing motions or altering pre-existing motions to have a particular style Chen-ney and Forsyth [CF00] used a markov chain monte carlo algorithm to samplemultiple animations that satisfy constraints for the case of multi-body collisions ofinanimate objects Schodl et al [SSSE00] used monte carlo techniques and devel-oped the concept of a video texture, where an infinite amount of similar lookingvideo sequences are generated from a starting short video clip

syn-For character animation in particular, probabilistic methods were also used torepresent the data Bregler [Bre97] has used hidden markov models to recognizefull body motions in video sequences Brand and Hertzmann [BH00] have alsoused hidden markov models with an entropy minimization procedure to learn andsynthesize motions with particular styles A different approach was done by Chi at

al [DCB00], where they used the principles of laban movement analysis to create

a method that allows the style enhancement of a pre-existing motion in an intuitivemanner

Recently, some projects try to allow the creation of new character animationsbased on motion capture data Li et al [LWS02] divided the data into motiontextons, where each of which could be modeled by a linear dynamic system In thiscase the motions were synthesized considering the texton likelihood Arikan andForsyth [AF02] developed a method for automatic motion generation at interactiverates using a random search algorithm to find appropriate pieces of motion data.Kovar et al [KGP02] defined the concept of a motion graph to enable the control

of a character locomotion This method allows a user to have high level control over

02] presented a technique for controlling acharacter in real time using several possible interfaces In this case the animationsare created by searching through a motion database using a clustering algorithm.The methods described in this chapter are mainly based on the work of Pullen