3D Graphics with OpenGL ES and M3G- P31 pps

Having to preserve the old frame buffer contents is generally very expensive on hardware-accelerated devices see the next chapter for details, so we advise you to use this hint whenever

reason is the lack of typedef in Java: there is no efficient way to define a fixed data

type to distinguish 16.16 fixed-point values from ordinary int variables Without this

capability, fixed-point code becomes even more unreadable and error-prone than

other-wise Finally, the benefit of doing fixed-point arithmetic may not be as great in Java as in

native code, because you cannot use assembly language, and because the extra bit-shifting

requires more bytecodes than the corresponding float operations See the Java section

in Appendix A for more information

12.2.7 ENUMERATIONS

As the Java programming language has no support for enumerations, they are represented

as constant integer values (i.e., static final int) Enumerations are defined in the

class where they are needed, and do not apply anywhere else For example, the RGB pixel

format token in Image2D has the decimal value 99, and is the only token in the API

having that particular value

Having to prefix all enumerations by their class name is cumbersome, but if you need a

particular enumeration very often, you can copy it into a local variable or instance variable

like this:

int REPEAT = Texture2D.WRAP_REPEAT;

myTexture1.setWrapping(REPEAT, REPEAT);

myTexture2.setWrapping(REPEAT, REPEAT);

12.2.8 ERROR HANDLING

As in any proper Java API, error handling in M3G is based on the built-in exception

mech-anism Thanks to exceptions, no methods in the API need to return error codes or set

internal error flags that the application would have to check separately There are seven

different types of exceptions the API may throw, roughly indicating the type of error The

exceptions are listed in Table 12.1

M3G is generally more stringent about error checking than OpenGL ES For

example, indexing a vertex array out-of-bounds has “undefined” effects in OpenGL ES,

but causes a well-defined exception in M3G The extra error checking may have a minor

impact on performance, but it also makes debugging easier and the implementations

more robust against buggy or malicious code Debugging facilities on mobile devices tend

to be poor or nonexistent, so any help in that area is particularly welcome To minimize

the performance overhead, errors are checked at the earliest possible occasion, typically at

constructors and setters Final validation of the scene graph must be deferred until

ren-dering time, however This is because the validity of an object often depends on other

objects that the application may change at any time

T a b l e 12.1: The exceptions that may be thrown by M3G, and their typical causes Note that the list of causes is not exhaustive.

ArithmeticException Supplying an uninvertible transformation as a parameter

IllegalArgumentException Supplying a wrong enumeration as a parameter

IllegalStateException Attempting to render an incomplete or invalid object, such as a Mesh

with no VertexBuffer attached IndexOutOfBoundsException Attempting to read or write beyond the boundaries of some internal

data structure, such as the list of child nodes in Group, or the list of textures in Appearance

NullPointerException Supplying a null reference as a parameter

SecurityException Attempting to load a remote file without having sufficient network

access permissions IOException Attempting to load an invalid file, or attempting to load a remote file

when the device is out of network coverage

12.3 M3G 1.1

To combat market fragmentation, the upgrade cycle of M3G has been kept relatively slow M3G 1.1 was released in June 2005, a year and a half after the original, but unlike OpenGL ES 1.1, it does not add any new rendering features It is focused on merely improving the performance and interoperability of the existing functionality—in other words, fixing errors and omissions in the original spec Importantly, the file format was not changed at all A complete change log is available on the Overview page of the speci-fication [JCP05]; here we review only the changes that have practical significance

12.3.1 PURE 3D RENDERING

The most important addition to M3G 1.1 is the OVERWRITE hint It lets you tell the implementation that you intend to do full-screen 3D rendering, so any pre-existing con-tents of the designated render target may be discarded Having to preserve the old frame buffer contents is generally very expensive on hardware-accelerated devices (see the next chapter for details), so we advise you to use this hint whenever possible

12.3.2 ROTATION INTERPOLATION

A persistent interoperability issue that was resolved in M3G 1.1 was due to keyframe-animated rotations, or more precisely, interpolation of quaternions (see Section 4.1.2) Some devices and content exporters had taken the liberty to interpolate along the shortest path in 3D space, which is not always the same as the shortest path in 4D quaternion space

If the content producer (i.e., the exporter) assumes that the shortest 3D path is taken, and

the runtime engine takes the shortest 4D path, or vice versa, the resulting animation will

look fascinating, to say the least The typical symptom is some occasional flickering in the

animation sequence; when examined more closely and in slow motion, it turns out that

the object does a full360◦spin around multiple axes when it should only rotate a degree

or two This happens between two successive keyframes, which is often just a blink of an

eye, and is thus perceived as flickering

This issue has been fixed in all the publicly available exporters, as far as we know, but you

may well run into it if you pull some generic quaternion interpolation routine from the

web, and use that in your application or proprietary content processing tools See page 372

in Section 16.2 for more information

12.3.3 PNG AND JPEG LOADING

Another frequent source of problems prior to M3G 1.1 was loading of PNG images

that contain transparency information The PNG file format supports various forms

of transparency, including color-keying, palette entries with alpha information, and

complete alpha channels M3G 1.1 makes it explicit that all these formats must be

supported, regardless of whether the base image is grayscale, indexed color (palletized),

or true color The mapping of these formats to the Image2D internal formats is now

well-specified, too

Support for JPEG images was left optional in both M3G 1.0 and 1.1, for fear of risking

the royalty-free status of M3G and thereby hindering its adoption Thus, depending on

the device, you may or may not be able to load JPEG files using the built-in Loader

On the other hand, including them into m3g files was ruled out completely, as that

would have compromised the portability of art assets

However, these decisions have been later reversed by the new Mobile Service Architecture

(MSA, JSR 248) standard [JCP06]: it requires full JPEG support across the board,

includ-ing in m3g files.8JPEG is clearly superior to PNG for photographic images, and everyone

in the industry has a license for it these days, so it makes sense to use it as widely as

pos-sible The downside is that we now have three kinds of devices in the market with respect

to the availability of JPEG: those with full support, those with no support, and those with

partial support As a further complication, some pre-MSA devices may expand grayscale

JPEGs into RGB, increasing their size by a factor of two to four

If you wish to target your application for all M3G-enabled devices that have ever shipped,

with a minimum effort, we advise you to use png for images that have no transparency,

and m3g for those that include alpha

12.3.4 NEW GETTERS

As we mentioned in the previous section, M3G 1.1 also adds more than thirty getters that were missing from the original release With the complete set of getters available, applications need no longer keep duplicate copies of M3G state attributes on the Java side The getters are also useful for debugging, diagnostics, and content processing purposes The complete list of new getters in each class is shown below

AnimationController:

int getRefWorldTime() Graphics3D:

Object getTarget() boolean isDepthBufferEnabled() int getHints()

int getViewportX() int getViewportY() int getViewportWidth() int getViewportHeight() float getDepthRangeNear() float getDepthRangeFar() Camera getCamera() int getLightCount()

IndexBuffer:

int getIndexCount()

KeyframeSequence:

int getComponentCount() int getKeyframeCount() int getInterpolationType()

int getValidRangeFirst() int getValidRangeLast() Node:

PolygonMode:

boolean isLocalCameraLightingEnabled() boolean isPerspectiveCorrectionEnabled() SkinnedMesh:

Texture2D:

int getLevelFilter()

int getVertexCount()

int getComponentCount()

int getComponentType()

void get(int firstVertex, int numVertices, byte[] values)

void get(int firstVertex, int numVertices, short[] values)

12.3.5 OTHER CHANGES

The other incompatibilities addressed in M3G 1.1 were very minor, and many of them

had been discovered by proofreading the specification, not because they would have posed

problems for developers One thing perhaps worth mentioning is that the Loader now

treats file names as case sensitive; previously this was left ambiguous

Finally, the new version relaxes error checking on situations where the added security

or diagnostic value of throwing an exception was questionable For example, M3G 1.0

used to throw an exception if a polygon mesh had lighting enabled but was lacking

nor-mal vectors Now, M3G 1.1 just leaves the nornor-mals undefined Viewing the erroneously

shaded mesh on the screen probably makes it easier for the developer to figure out what

is wrong than getting an exception that may be caused by half a dozen other reasons

BASIC M3G CONCEPTS

Now is the time to get your hands dirty and begin programming with M3G To get started, you will need a device that supports M3G; almost any mid-category or high-end phone will do For your development PC, you will need a software development kit (SDK) such

as the Java Wireless Toolkit by Sun Microsystems1or Carbide.j by Nokia.2We also recom-mend that you download the official M3G 1.1 specification [JCP05], available as either zipped HTML or PDF Forum Nokia are also hosting an online, browser-friendly copy in their Java ME Developers Library.3More detailed instructions for setting up your devel-opment environment are provided on the companion web site of this book

The first thing that a well-behaved M3G application needs to do is to check the availability

of M3G, as it may not be present on some older devices If M3G is available, its version number should be verified, as many devices only support the 1.0 version The examples in this book are based on M3G 1.1; subtle changes may be needed in some cases to make the code work robustly on a 1.0 implementation The version number can be queried from the system property microedition.m3g.version, as shown below:

String version = System.getProperty("microedition.m3g.version");

else if (version.equals("1.1")) { } // M3G 1.1

Once you have confirmed that M3G is indeed supported on your target device, you can

go ahead and start using the javax.microedition.m3g package The first class that

you are going to need from that package is most probably Graphics3D, and that is also

the logical starting point for learning the API

13.1 Graphics3D

The only class in M3G that you cannot avoid if you want to draw anything at all is the

3D rendering context, Graphics3D This is where all rendering and render target

man-agement takes place, so you can think of it as a combination of OpenGL ES and EGL

It is a lot simpler, though, because most of the OpenGL ES state information is stored

elsewhere, leaving only the viewport, camera, lights, and a few hints to be managed by

Graphics3D Most of EGL is not exposed at all Instead, you just provide a render

tar-get, and all the complexities of managing surfaces and configurations are taken care of

under the hood

13.1.1 RENDER TARGETS

There is only one instance of Graphics3D in the system, and that has been graciously

created for you in advance All you need to do is to get a handle on that single object,

bind a rendering target to it, render your scene, and release the target This is shown

in the example below Since we have not yet discussed rendering, let us just clear the

screen:

void paint(Graphics graphics) {

Graphics3D g3d = Graphics3D.getInstance();

try {

g3d.bindTarget(graphics);

g3d.clear(null);

}

finally {

g3d.releaseTarget();

}

}

This example shows a typical scenario, in which you implement the paint callback

for your Canvas A Canvas represents a displayable surface in MIDP that may or

may not be visible, and may or may not cover the entire display, but for all practical

purposes you can think of it as the screen The rendering target that you bind is not

the Canvas itself, however, but its 2D rendering context, a Graphics object This

is because a Canvas is guaranteed to have access to the frame buffer (or back buffer)

only when its Graphics is available to the application Binding to a Graphics also simplifies things for the developer: you can get a Graphics for off-screen images as well, which means that your code will work unmodified for both on-screen and off-screen targets Rendering to a texture works the same way, except that you bind an Image2D object (see Section 13.2) instead of a Graphics

So what exactly happens when you bind and then later release a target? From the developer’s point of view, nothing much: bindTarget simply flushes all 2D drawing commands so that 3D rendering can proceed, and releaseTarget does the opposite

As a result, the pre-existing contents of the target are nicely overlaid or overwritten by the 3D scene There are only three ground rules: First, do not touch the target while it

is bound, as that may yield unpredictable results In particular, do not try to render any 2D graphics with MIDP Second, do not assume anything about the contents of the depth buffer after bindTarget, because the contents are undefined Third, make sure that your render target gets released no matter what exceptions occur, so that your applica-tion has a chance to recover, or at least make a clean exit The easiest way to do that is a try—finally construct as shown in the example above

If you care about performance, there are two more things to keep in mind First, minimize the number of render targets that you use Binding to a new target may require setting up

a new OpenGL ES rendering context and/or a new back buffer Second, minimize the number of binds and releases that you do per frame Every bind and release bears some amount of overhead, and on hardware-accelerated devices that overhead can be dramatic The reasons boil down to the notoriously poor interworking of 2D and 3D rendering on most Java ME implementations

Synchronizing 2D and 3D

In a typical MIDP implementation, the font engine and all other 2D routines are running

on the CPU, and can only use a back buffer that resides in main memory, while the 3D hardware can only use a back buffer that resides in its local memory The 2D back buffer

is copied from the main memory to the graphics memory at each bindTarget, and a reverse copy takes place at each releaseTarget The extra copying is bad in itself, but the hidden penalties are even worse First of all, reading the 3D frame buffer defeats all parallelism among the CPU and the different stages of the GPU As explained in Section 3.6, this can cut two-thirds of the potential performance Second, the only way to copy the 2D back buffer into the 3D back buffer may be to upload it into an OpenGL ES texture and then render a full-screen quad mapped with that texture Texture uploading is a very costly operation on some architectures

There is no sure-fire way to completely avoid the expensive 2D/3D synchronization points

on all devices; sometimes all you can do is to give some hints to MIDP and M3G and then cross your fingers, hoping for the best A reasonably good advice is to make your application pure, full-screen 3D: keep your Canvas in full-screen mode, and do not allow

anything other than M3G to access it The best and most explicit hint you can provide,

however, is the OVERWRITE flag at bindTarget:

g3d.bindTarget(graphics, , Graphics3D.OVERWRITE);

This tells the implementation not to burn cycles on preserving the pre-existing contents of

the 2D back buffer We have observed frame rates increasing two-, three-, even five-fold on

some devices just because of this The OVERWRITE hint is only available since M3G 1.1,

but some 1.0 devices provide an indirect means to achieve the same effect: just clear the

entire screen before drawing anything The implementation may then conclude that the

2D back buffer does not have to be copied in, as it will be completely cleared anyway

Antialiasing and dithering

There are three other hint bits available in Graphics3D besides OVERWRITE If you

want to use more than one of them at a time, you need to bitwise-OR them together:

int hints = Graphics3D.OVERWRITE | Graphics3D.ANTIALIAS;

g3d.bindTarget(graphics, , hints);

This example shows the overwrite hint combined with ANTIALIAS, requesting the

implementation to turn on antialiasing if possible, even at the expense of reduced

per-formance No specific method of antialiasing is mandated, but some form of full-scene

antialiasing (FSAA) is recommended, and in practice the industry has converged on

multisampling (see Section 3.4.5) There are very few devices on the market that

sup-port any kind of antialiasing, as not even all the hardware-accelerated models can do

it, but those few devices do a pretty good job at it They achieve good quality without

much performance overhead, so we recommend that you at least try the ANTIALIAS

hint To find out if antialiasing is supported on your target platform, use the static

Graphics3D.getPropertiesmethod (see Section 13.1.4)

The remaining two hint bits in Graphics3D are DITHER and TRUE_COLOR The

for-mer turns on dithering to increase the apparent color depth of the display (see

Section 3.5.3) The latter instructs the renderer to use its maximum internal color

pre-cision, even if the display can only reproduce, say, 256 or 4096 colors These hints seemed

useful back in 2002, but the incredibly fast development of color displays soon made

them obsolete—no M3G-enabled device ever shipped with less than 65K colors! Today,

most implementations render in true color regardless of the display color depth or the

TRUE_COLORhint, and any dithering takes place automatically at the display controller,

also regardless of the DITHER hint

Disabling the depth buffer

One final thing to mention about bindTarget is the depth buffer enable flag:

boolean enableDepthBuffer = false;

g3d.bindTarget(graphics, enableDepthBuffer, hints);

This lets you disable depth buffering at the outset if you are drawing some very simple content that is independent of rendering order, or if you are resolving the visibility by yourself, such as when using a height map for simple terrain rendering The implemen-tation can then decide not to allocate a depth buffer at all, saving some 150K bytes

of memory on the typical quarter-VGA device Those savings may not be realized in practice, though Situations where depth buffering is not needed are so few and far between that M3G implementations generally allocate the buffer just in case Besides, depth buffering is typically very efficient, particularly on hardware implementations, and things may only slow down if you come up with clever tricks to avoid it

13.1.2 VIEWPORT

M3G rendering does not necessarily affect the entire rendering target The area that will

be rendered to is determined by the intersection of the viewport defined in Graphics3D and the clipping rectangle defined in Graphics Image2D targets do not have a clipping

rectangle, so the renderable area is defined by the viewport alone If you go with the default settings in Graphics3D, the viewport will cover the entire Canvas, which is usually a good thing If you nonetheless want to restrict your rendering to some rectangular sub-area of the screen, you need to call the setViewport method after bindTarget Note that you will have to do that every frame, as bindTarget resets the viewport back to its default, full-screen state

As described in Section 2.6, the viewport transformation maps vertices from normalized device coordinates (NDC) to screen or window coordinates The mapping is parame-terized by the width, height, and top-left corner of the viewport, all specified in screen pixels To illustrate, let us expand our earlier screen clearing example so that the top half

is cleared with the red color and the bottom half with blue This is done by setting up a Backgroundobject and supplying it as a parameter to clear:

int width = graphics.getClipWidth();

int height = graphics.getClipHeight();

try { g3d.bindTarget(graphics, true, hints);

g3d.clear(myBackground);

g3d.setViewport(0, height/2, width, height); // bottom half

g3d.clear(myBackground);

}

The Background class is pretty self-explanatory It defines whether and how the view-port and the corresponding area in the depth buffer are cleared In this example we used a constant clear color, but with a few more lines of code we could have used a tiled or scaled background image; see Section 14.4.1 for details