Render layers can use blend modes, which combine the results of the render to form a compos- ite. You can preview the composite in the Render View window. Typically you render each layer separately, import the render sequences into compositing software (such as Adobe After Effects or Autodesk Composite), and then apply the blend modes using the controls in the compositing software. Maya gives you the option of creating a very simple composite using render layers, which you can view in the Render View window.
Blend modes use simple algorithms to combine the numeric color values of each pixel to cre- ate a composite. A composite is created by layering two or more images on top of each other. The image on top is blended with the image below. If both images are rendered as Normal, then the top image covers the bottom image completely. If the blend mode is set to Multiply, then the light pixels in the top image are transparent, and the darker pixels of the top image darken the pixels in the bottom image. This technique is often used to add shadowing to a composite. If the blend mode of the top image is set to Screen, then the darker pixels are transparent, and the lighter pixels brighten the pixels of the lower image. You can use this to composite glowing effects.
The blend modes available in Maya are Lighten, Darken, Multiply, Screen, and Overlay:
Lighten This mode compares the layered images and uses the lightest pixel value of the two layers to determine the resulting color. For example, the lower image has a pixel in a particular spot with an RGB value of 0, 125, 255, and the pixel at the same location in the top image has an RGB value of 0, 115, 235. The resulting RGB value for that pixel will be 0, 125, 255.
Darken This is the opposite of Lighten, and the darker value is used. In the example cited previously, the resulting RGB value for the pixel would be 0, 115, 235.
Multiply The pixel values of the top image are multiplied by the pixel values of the bottom image and then divided by 255 to keep the values within the range of 0 to 255. The lighter pixels in the top image are semitransparent, and the darker values of the top image result in a darkening of the lower image.
Screen A slightly more complex algorithm is used for this mode. The formula is 255-[(255- top color RGB pixel value)*(255-bottom color RGB pixel value)/255]= blended RGB pixel value. This has the effect of making darker pixels in the top image semitransparent and lighter, resulting in a lightening of the lower image.
Overlay This combines Multiply and Screen modes so the lighter pixels of the top image brighten the bottom image and the darker pixels of the top image darken the bottom image.
In this exercise, you’ll use blending modes to create soft shadows for the render of the car in the studio lighting scenario.
This scene shows the car in the studio lighting scenario. A single render layer exists already.
Using the technique in this exercise, you’ll eliminate the harsh cast shadows that appear on the ground in the rendered image (shown earlier in the chapter in Figure 12.2) and replace them with soft shadows created using an ambient occlusion shader. First you’ll remove the shadows cast on the ground by the physical lights in the scene (note that physical lights always cast shad- ows; there is no option for turning shadows off when you use these lights).
1. Open the carComposite_v06.ma scene from the chapter12\scenes directory on the DVD.
2. Select the ground object in the Outliner.
3. Open its Attribute Editor, and switch to the groundShape tab.
4. Expand the Render Stats section in the Attribute Editor, and deactivate Receive Shadows (see Figure 12.11).
Note that, for some attributes, changing a setting on a render layer automatically creates a layer override.
5. Select the studioLighting layer, and create a test render in the Render View window using the renderCam camera (see Figure 12.12).
6. In the Outliner, Shift+click the vehicle group and the ground surface.
7. In the Render Layer Editor, choose Layers Create Layer From Selected. Name the new layer AOShadow.
8. Open the Hypershade window. Make sure the AOShadow layer is selected in the Render Layer Editor.
Figure 12.11 Disable Receive Shadows for the ground surface.
9. Create two new surface shaders in the Hypershade (from the Hypershade menu bar, choose Create Materials Surface Shader).
10. Name one of the surface shaders shadowShader and the other whiteMask.
11. In the Outliner, select the vehicle group, and apply the whiteMask shader to this group.
12. Select the ground object, and apply the shadowShader to this surface.
13. Open the Attribute Editor for the whiteMask node, and set Out Color to white.
14. Open the Attribute Editor for the shadowShader.
15. Click the checkered box to the right of Out Color. In the Create Render Nodes window, select Textures under mental ray. Choose the mib_amb_occlusion texture from the node list (Figure 12.13).
16. Open the Attribute Editor for the mib_amb_occlusion1 node, and set Samples to 64.
17. Make sure the AOShadow is selected in the Render Layer Editor; then create a test render using the renderCam camera.
The car appears as flat white, but you can see the soft shadows created on the ground by the ambient occlusion node (Figure 12.14). Later in this chapter you’ll learn more about how ambient occlusion textures create shadows.
18. Now you are ready to preview the composite in the Render View window. In the Render Layer Editor, set the mode of the AOShadow layer to Multiply (see Figure 12.15).
Because the car in this render layer is flat white, when the pixels of the AOShadow layer are multiplied by the pixels of the studioLight layer, only the soft shadows appear in the composite.
Figure 12.12 In this version of the render, the ground does not receive cast shad- ows from the car.
Figure 12.13 Create an ambient occlusion texture, and connect it to the shadowShader’s Out Color channel.
Figure 12.14 The soft shadows created by the ambient occlusion texture appear on the ground while the car is masked in flat white.
Figure 12.15 Set the blend mode of the AOShadow layer to Multiply.
19. In the Render Layer Editor, choose Options Render All Layers Options. In the Options box, set Keep Image Mode to Composite Layers.
There are three choices in the Render All Layers Options dialog box: Composite Layers, Composite and Keep Layers, and Keep Layers.
Choosing “Composite” renders both layers and then composites them in the Render View window.
Choosing “Composite And Keep Layers” creates the composite, but it also keeps the rendered image of each individual layer available in the Render View window.
Choosing Keep Layers” will not composite the layers; instead, it renders all renderable layers and keeps them as individual images in the Render View window.
20. After choosing the Composite Layers option, click Apply and Close.
21. Make sure that Render All Layers is now selected in the Options menu of the Render Layer Editor (Figure 12.16).
22. In the Render Layer Editor, make sure the red X appears on the clapboard icon of master- Layer, indicating that this layer will not render. A green check box should appear next to the studioLighting and AOshadow layers, indicating that they will be rendered.
23. Open the Render View window, and create a test render using the renderCam camera.
You’ll see the studioLighting layer render first, and then the AOShadow layer will render on top of it. Figure 12.17 shows the composited image.
24. Save the scene as carComposite_v07.ma.
Figure 12.16 Select the Render All Layers option in the Options menu.
Figure 12.17 The two images are composited in the Render View window.
To see a finished version of the scene, open the carComposite_v07.ma scene from the chapter12\scenes directory on the DVD.
This is a good way to preview basic composites; however, in practice you will most likely want more control over how the layers are composited. To do this, you should use more advanced compositing software such as Adobe Photoshop (for still images) or Adobe After Effects or Autodesk Composite (for animations).