3D Graphics with OpenGL ES and M3G- P34 ppt

File Identifier Header Section Scene Section n Compressed: {0, 1} Total Section Length Uncompressed Length Object 0 Objectn Checksum Object Type SkinnedMesh = 16 Length Data .... ‘«’ ‘J’

own exporters, viewers, or optimization tools for M3G content, or simply need to debug

an existing file

The simplicity is achieved by four primary means: First, there is a one-to-one mapping

between object types in the file format and Java classes in the run-time API Second, there

are no forward references, so objects can only refer to objects that reside earlier in the file

Third, compression is based on the widely available zlib, rather than specialized

encod-ings for different data types, or complex algorithms such as arithmetic coding Finally,

the Loader has an all-or-nothing policy: a file is either loaded completely or not at all

No attempt is made to recover from errors

We will now go through the structure of the file format, shown at a high level in

Figure 13.2 To shed light on individual details, we will refer to the example file in

Figure 13.3 Note that what follows is not intended to be an exhaustive description or

a replacement for the specification, but only to give you an idea of how things are laid

out Having read this, you should be able to more quickly dig up whatever details you

need in the spec

File structure

M3G files are easy to recognize in a text editor by their file identifier, <<JSR184>>,

located at the very beginning To be exact, the identifier consists of that string and a few

special characters to quickly catch file transmission errors, for a total of twelve bytes (in

hexadecimal): AB 4A 53 52 31 38 34 BB 0D 0A 1A 0A

File Identifier

Header Section

Scene Section n

Compressed: {0, 1}

Total Section Length Uncompressed Length Object 0

Objectn

Checksum

Object Type (SkinnedMesh = 16) Length Data

Object3D Transformable Node Mesh SkinnedMesh

XREF Section

Scene Section 0

F i g u r e 13.2:M3G files are divided into sections, each containing one or more objects, which are further divided into fields The fields are laid out in the order of class inheritance; for example, a SkinnedMesh object is derived from Object3D,

‘«’ ‘J’ ‘S’ ‘R’ ‘1’ ‘8’ ‘4’ ‘»’ 0x0D 0x0A 0x1A 0x0A

ApproximateSize=16612 null Section Checksum

Section Checksum

UserID=0x1234

UncompressedLength=44

false false

ObjectIndex=2

AnimationTrackCount=0 UserParameterCount=0

Section Checksum

Object3D fields Transformable fields Texture2D fields

ObjectType 0: Header

File Identifier

Header Section

XREF Section

Scene Section

ObjectType 255: XREF

ObjectType 17: Texture2D

Blend mode:

Blend color:

(0, 0, 0)

Filter mode: (NEAREST, NEAREST)

F i g u r e 13.3: The complete contents of our example file, byte by byte There is one root object in the file, a Texture2D, and one non-root object, an Image2D that is loaded from the external file /bg.png All sections are uncompressed Object data are shown in gray boxes, section data in white.

Beyond the file identifier, M3G files are divided into sections that contain one or more objects each Sections may be individually compressed with zlib; this allows

you to selectively compress the sections for which it makes most sense Each section also has an Adler32 checksum (provided by zlib) to allow the Loader to quickly reject a corrupt file without parsing it further Note that the loader will make

no attempt to recover the contents of a damaged file, but will simply throw an IOException

There are three kinds of sections The header section must be present in every file, must

be uncompressed, and must be located right after the file identifier The external

refer-ence section is optional, but must immediately follow the header section if present The

rest of the file is composed of an arbitrary number of scene sections Any legal M3G

file must have at least one section besides the header, and must not have any empty sections

Data types

The file format has data types corresponding to the built-in boolean, byte, short,

int, and float types in Java Booleans are encoded as a single byte, such that 1 indicates

true, 0 indicates false, and other values are disallowed The integer and float types

are stored so that the least significant byte comes first

Besides the basic types, M3G files may also contain null-terminated strings, 3-element

floating-point vectors,4 × 4 matrices, RGB and RGBA colors encoded at 8 bits per color

component, references to other objects, and arbitrary-length arrays of any basic or

com-pound type

Object references

Upon loading, objects are read sequentially from the beginning of the file and assigned a

running number as their index The first object in the file, which is a special header object,

gets the index one, the first actual scene object gets the index two, and so on The index

zero is reserved for null references

Objects can only reference other objects that reside at lower indices, i.e., those that have

already been imported This is to guarantee that the Loader can parse any M3G file

from start to finish in one pass, and also to allow it to type-check the references

immedi-ately Note that the lack of forward references means that a scene graph loaded from a file

can never have cycles, although they are permitted in the runtime scene graph for node

alignment; see Section 15.3.2

Header section

The header section contains exactly one object, the header object, which cannot be present

anywhere else in the file As shown in Figure 13.3, the header object begins with a two-byte

version number, identifying variants of the file format The only legal version number

at this time is 1.0 Note that the file format does not necessarily have the same version

number as the M3G API

Following the version number is a boolean flag telling whether the external reference

sec-tion is present (in our example, it is) The header object also stores the total size of the file,

both with and without the external references The size that includes the external

refer-ences is regarded to be a hint and need not be accurate, so as to allow the referenced files

to be modified independently of the root file In our example, the root file is 112 bytes

(exactly), and the externally referenced PNG is (estimated to be) 16500 bytes

The final item in the header object is the AuthoringField where the authoring tool or

the author herself may store an arbitrary null-terminated string of text, such as a copyright

notice In our example the field is left empty, containing just the terminating null

External reference section

The external reference section stores one or more external references, or XREFs for short External references allow you to build up your scene from a collection of separate M3G and image files Images are typically included by reference rather than embedded into the host file, because dedicated image formats provide better compression than plain zlib A minor disadvantage with external image files is that they have no user ID or user parameters

External references are simply null-terminated strings pointing at named resources, such

as network URLs or files in the JAR package Each external reference yields exactly one root-level Object3D Our example file in Figure 13.3 has just one XREF, pointing at /bg.pngin the JAR package It will be imported as a single Image2D

M3G files may reference an arbitrary number of other M3G files, which in turn may freely reference another set of M3G files, and so on, but the references are not allowed to form

a cycle Also, if you intend an M3G file to be referenced from another, make sure that it only has one root object If there are many root objects, the Loader will pick only the first one and discard the rest

Scene sections

Scene sections store the actual scene graph—or several scene graphs Each scene sec-tion can contain one or more objects, and again each object corresponds to exactly one Object3Din the runtime API The contents of each object are generally the same as that

of the corresponding class in the API

In our example file, there is one scene section, containing a single Texture2D The base class data for Object3D comes first, followed by the other base class Transformable The data for the concrete class is stored last This is the case with all types of objects in the file format

For simplicity, we have no animation tracks or user parameters in our example, and

no texture matrix in Transformable The two false entries in Transformable indicate that theT R S components as well as the M component will assume their default

values, i.e., the identity matrix The fields of the Texture2D object itself are pretty obvious The main thing to note is that the image in /bg.png must have power-of-two dimensions Also note that the Image2D is assigned the object index 2, because the header object always gets the index one, and zero is reserved for null

Special compression

We said in the beginning of this section that there are no special compression formats for different data types in the M3G file format, just zlib for everything, but that is

not strictly true The VertexArray and KeyframeSequence classes do in fact have

special encodings as an option However, the encodings are almost trivial

Vertex arrays—including colors, normals, texture coordinates and vertex positions—can

be compressed with delta encoding This means that each vertex attribute is stored as a

dif-ference vector relative to the previous value The difdif-ferences are unsigned, so for

exam-ple a difference of−2 is encoded as 254 (in case of a byte array) or 65534 (in case of

a short array) Thus, the deltas take up the same number of bits as the raw integers

(8 or 16 bits per component), making the whole encoding seem rather pointless

How-ever, the deltas tend to have fewer significant bits, causing the same bit patterns to repeat

more often across the array This, in turn, allows zlib to compress the array more

effi-ciently Note that delta encoding and zlib are both lossless

Keyframes, which are 32-bit float values in raw format, can be encoded by quantizing

them to 16-bit or 8-bit integers which are then scaled and offset using a bias value The

quantized keyframes consume only a half or a quarter of the file size compared to the raw

format, and that is further reduced by zlib Floating-point bit patterns, on the other

hand, are poorly compressed by zlib

LOW-LEVEL MODELING IN M3G

M3G builds upon the common low-level concepts set forth in Chapter 3 It offers most

of the same functionality that OpenGL ES provides for native applications, but with an object-oriented Java interface Some features are slightly more abstracted in order to reduce API complexity, but the underlying rendering pipeline, be that implemented in software

or hardware, can be shared with OpenGL ES Also, while familiarity with OpenGL ES

is not a prerequisite for understanding M3G, existing knowledge on OpenGL ES will not be wasted on M3G

In this chapter, we walk through the lowest-level parts of M3G By the end of the chapter, you will know how to use M3G to draw polygon meshes in immediate mode, similarly

to OpenGL ES The components discussed here will also serve as building blocks for the higher-level functions which are covered in the following chapters

14.1 BUILDING MESHES

Meshes in M3G are built out of vertex array and buffer objects, triangle buffer objects, and shading parameters specified in various rendering state objects

14.1.1 VertexArray

Low-level modeling begins by defining your vertex data The VertexArray class stores an array of vectors that can then be used for any per-vertex data: positions, normals,

colors, or texture coordinates The class constructor is VertexArray(int numVertices,

int numComponents, int componentSize), where the parameters are the number

of vertices, number of components per vertex, and size of the data type used,

respectively componentSize is 1 for byte, and 2 for short data For a mesh with

100 vertices having vertex positions and colors only, for example, you could create two

arrays:

myPositions = new VertexArray(100, 3, 2); // 16-bit positions

myColors = new VertexArray(100, 4, 1); // 8-bit RGBA colors

Vertex data is loaded into the arrays using the set function, which copies a range of

vertex values from a byte or short array:

void set(intfirstVertex,intnumVertices,byte[]values)

void set(intfirstVertex,intnumVertices,short[]values)

Pitfall: If you plan on reading data back from VertexArray, you may soon find that

the get method for that is not included in M3G 1.0—it was one of the many getters

dropped to minimize the API footprint

The getmethods were added to VertexArray in M3G 1.1, but if you

abso-lutely need equivalent functionality with M3G 1.0, it can be done using the

Transform.transformmethod as described in Section 13.3 Even then, you will

only get the vertex data in floating-point format, not the original 8-bit or 16-bit integers

Now, assume that you have your positions in a short array myPositionData and

your colors in a byte array myColorData The arrays should have at least 300 and

400 elements, respectively We can then load the data values for all 100 vertices into the

previously created vertex arrays:

myPositions.set(0, 100, myPositionData);

myColors.set(0, 100, myColorData);

M3G makes a copy of the data you load into a VertexArray, so myPositionData

and myColorData can be discarded at this point In fact, all data in M3G is stored

internally—client-side arrays are only referenced when copying data from them This

allows M3G to internally organize the data in the most efficient way

14.1.2 VertexBuffer

Once you have the vertex arrays you need, they must be combined into a

VertexBuffer to form the actual vertices The constructor for VertexBuffer

simply creates an empty set of vertices The necessary component arrays are added using

the setPositions, setNormals, setColors, and setTexCoords functions

Note that there are certain restrictions on what kind of vertex data you can use for

T a b l e 14.1: Supported vertex array types in M3G (ticks), relative to OpenGL ES 1.1 (ticks and crosses) The grayed-out boxes indicate combinations that are supported in neither API.

each vertex component—those are summarized in Table 14.1 The setters for colors and normals are trivial, only taking in the array you wish to use for that vertex

com-ponent Normals are automatically normalized For positions, however, additional scale and bias values must be supplied:

void setPositions(VertexArraypositions,floatscale,float[]bias)

Since M3G only supports 8- and 16-bit vertices, scale and bias let you map the quantized

vertices into a wider floating-point domain Before M3G uses any of your vertex data, each quantized vertexviis converted into an equivalent floating-point vertexvi

where s and b are the values of scale and bias, respectively This way, you can author your

model in floating point, quantize the vertices to 16 or 8 bits, and still use the resulting VertexArrayin M3G as you would the original data The precision is sufficient for just about any model, while the memory usage is only a half or one-quarter of full floating-point vertices M3G implementations are also made more efficient, as there is

no need to implement a full floating-point vertex pipeline

Again, using our example arrays, let us set up a VertexBuffer:

myVertices = new VertexBuffer();

myVertices.setColors(myColors);

myVertices.setPositions(myPositions, 100.f / (1<<16), null);

In this case, we set bias to null, signaling a default bias of (0, 0, 0) The scale parameter

scales our 16-bit position data so that the coordinates span 100 units in floating point domain If the full 16-bit range is utilized in myPositions, our model therefore extends from−50 to +50 on each coordinate axis

Performance tip: Always make use of the full range available for your vertex

positions and texture coordinates Quantize your floating-point coordinates so that they fill the 8- or 16-bit numeric range optimally, then map the data back to floating point

by using the scale and bias values There is no additional runtime cost from doing this, but it will let you achieve the maximum precision possible

Texture coordinates take one more additional parameter, the index of the texturing unit:

void setTexCoords(intindex,VertexArraytexcoords,floatscale,float[]bias)

When using multi-texturing, VertexBuffer must contain a set of texture

coordi-nates for each texture unit Of course, you can—and often will—also set the same

VertexArrayfor each texture unit; no data replication is required Arrays can also be

shared between any number of vertex buffers, and nothing prevents you from, for

exam-ple, using the same array for both vertex normals and texture coordinates

Performance tip: Always prefer texturing over pass rendering With

multi-texturing, you get multiple layers of texture by only rendering the geometry once,

whereas multi-pass rendering incurs the transformation and lighting overhead for each

pass Blending and other frame buffer processing will also add to this overhead

Vertex positions are the only piece of data that is required for all rendering If you want

to use lighting, your VertexBuffer will also need normal vectors, and we already

mentioned that texture coordinates are required to apply a texture For vertex colors,

you have a choice of using either a per-vertex color array or a single color for the entire

buffer, set using setDefaultColor To construct a buffer where all the vertices are

red, you can do:

myVertices = new VertexBuffer();

myVertices.setPositions(myPositions, 100.f / (1<<16), null);

myVertices.setDefaultColor(0xFF0000);

Pitfall: M3G versions 1.0 and 1.1 specify slightly different error handling for vertex

components, with M3G 1.0 being more strict It throws exceptions for many cases of

missing data, such as texturing without valid texture coordinates This was viewed as an

extra burden on implementations, and in M3G 1.1 error checking was relaxed so that

vertex normals and texture coordinates default to some undefined value if the respective

arrays do not exist during rendering

As an application developer, you should be aware of the fact that you may not get an

exception if you are missing a required piece of vertex data Instead, your rendering

results will simply be incorrect The reason may be tricky to identify even if you know

what to look for, so if you want to use lighting or texturing, remember to supply those

normals or texture coordinates!

Performance tip: Note that you can use set at any time to modify the contents of a

VertexArray, and you can also use the setters on VertexBuffer to change the

arrays being used Be aware, however, that this may not be cheap, as the implementation

may have to recompute bounding volumes or other cached data that is dependent on

the vertex values As a rule, create your vertex arrays and buffers during the initialization phase of your application and only modify their contents after that if you really need to

14.1.3 IndexBuffer AND RENDERING PRIMITIVES

Vertices alone do not let you render anything You also need to specify the kind of prim-itives you want to draw For this purpose, M3G has the abstract IndexBuffer class that is specialized for each type of primitive With the primitive types, M3G takes rather

an ascetic approach, as the only kind of primitive currently supported is a triangle strip (see Section 3.1.1) Comparing to OpenGL ES in Table 14.2, we see that this is quite

a cut in features Lines and points were dropped because they would have added a lot

of complexity to support quite a few use cases; if necessary, they can be emulated with triangles, which is how most renderers implement them in any case

The reasoning behind supporting triangles as strips only was that they are an efficient primitive both for storing a shape and for rendering it, and most M3G content will come from authoring tools that can easily generate the strips It was perceived that quite a bit

of implementation complexity could be dropped this way Looking back now, this was one of the decisions where features were a little too aggressively cut down in the effort

to minimize complexity—having to use triangle strips instead of triangle lists is quite an annoyance when generating meshes in code, for example

The TriangleStripArray class lets you construct an array of several triangle strips

You have a choice of two flavors of strips: implicit and explicit The former assumes that

the source vertices are ordered sequentially in the ascending order of indices, and you only have to specify the starting vertex and the length of each subsequent strip:

TriangleStripArray(intfirstVertex,int[]stripLengths)

The number of entries in the stripLengths array gives the number of strips For example,

if stripLengths is {3, 4, 3}, the call TriangleStripArray(2, stripLengths) will create three strips with the indices{2, 3, 4},{5, 6, 7, 8}, and{9, 10, 11} This

T a b l e 14.2: Supported rendering primitives in M3G, relative to OpenGL ES 1.1 The grayed-out boxes indicate combinations that are supported in neither API.