Cel Shading or Toon Shading

So here's how to cell shade in Maya...

you can do it with maya renderer and toon shader or mental ray and contour shader (whcihc is a bit more complicated but gives better results...

Using the standard Maya Renderer, not Mental Ray

- First light the scene with a standard three point lighting system so you have good lights to work with as opposed to Maya standard lights.  Then duplicate your geometry and rename one 'Cel' and the other 'Outline', then group these together so you can move them together and they won't become misaligned if you move them or rotate them.

- Assign the geometries to a new render layer each, named 'Cel' and 'Outline'.  This is so that they can be rendered separately and composited in After Effects giving you more control over the outcome.

- For the 'Cel' geometry assign a 'Fill Shader'.  Do this in the Rendering menu set ⇒ Toon ⇒ Assign Fill Shader ⇒ then choose Shaded Brightness Tone 2 or 3. In the Attribute Editor adjust the colour in the ramp shader to whatever is it you need the colour to be, and add more if you wish to give a wider tonal range. This is done by clicking in the colour box where you can slide the current colours left and right.

- Once you have the desired colour, you need to add an outline to the 'Outline' geometry.  First assign a Surface Shader and make it white. Then in the Rendering menu set ⇒ Toon ⇒ Assign Outline ⇒ Add New Toon Outline.  You can then edit the outline and crease lines in the Attribute Editor, it's best to adjust settings under the Common, Profile Lines, Crease Lines and Border Lines tabs to achieve the best results.

If you are making an animation, not just a still, all you do is composite the 2 layers in an editing software like After Effects and because of the alpha channels you can overlay the outline on top of the coloured layer.  Then play around with the hue and saturation and what not until you're happy with the results.
To get a nice outline you need to create your model in pieces, this then gives the outline shader more to work with and produce a better result.

I will go through rendering in players/passes and 3 point lighting in another post. 

I also need to research more about Cel Shading with Mental Ray, but when I know how I'll let you know!

I'll have a quick go with the Toon Shader and a model I found on the internet, then blog it up.

A lot of it is explained briefly here by Christophe Desse.