sltokenize -- Tokenizer for the RenderMan Shading Language

This module provides a lexical scanner for the RenderMan Shading Language. You use it in the same way as the Python tokenize module is used.
tokenize(readline, tokeneater)
Reads an input stream and creates tokens.

The first parameter, readline, must be a callable object which provides the same interface as the readline() method of built-in file objects. Each call to the function should return one line of input as a string.

The second parameter, tokeneater, must also be a callable object. It is called with six parameters: the token type, the token string, a tuple (srow, scol) specifying the row and column where the token begins in the source, a tuple (erow, ecol) giving the ending position of the token, the line on which the token was found and the filename of the current file.

The token type can be one of

By default the filename argument is an empty string. It will only be the actual filename if you provide a preprocessed file stream as input (so you should first run cpp on any shader). The tokenizer actually expects preprocessed data as it doesn't handle comments.

Copyright © 2002 Matthias Baas (

The RenderMan (R) Interface Procedures and Protocol are:
Copyright 1988, 1989, 2000, Pixar
All Rights Reserved

RenderMan (R) is a registered trademark of Pixar