Introductory tutorial

Introductory tutorial

SEJITS is a toolkit for writing simple compilers for domain-specific languages embedded in Python. These compilers, called specializers, are typically written in Python and generate C++ just-in-time at runtime. SEJITS then compiles the C++ and executes it immediately.

Templates

SEJITS supplies two main mechanisms for implementing specializers. The first, templates, enable specializer writers to write C++ code containing "placeholders" or "holes" which are filled in at runtime:

for (int i=0; i < ${num_items}; i++) {
    arr[i] *= 2.0;
}

In this example, the loop bound is a constant determined by the Python application at runtime. By filling in the constant before the C++ is compiled, C++ compiler optimizations such as loop unrolling are enabled. In general, templates enable compiler optimizations, adapting to machine parameters, and allow choosing among implementations based on architecture.

Exercise

We're going to modify a template-based SEJITS specializer. Begin by connecting by SSH to:

• Hostname: moonflare.com
• Username: par-lab-all
• Password: 2xyb3pex

Set up your user tutorial directory with "setup yourname". The specializer you're going to modify is in double.py and multiplies each entry of an input vector by 2. Go ahead and run it using "python double.py". The specializer will generate a C++ source file, compile it, and run it. Look under your "cache" subdirectory for the generated module.cpp file, and examine it with your favorite text editor (if you're not sure, use "nano").

Now, we're going to modify the specializer. The double.py specializer uses a template stored in double_template.mako. Open double_template.mako in your text editor. Try changing the template to multiply each entry of the vector by 3 instead of 2. Once you've made your change, run "python double.py" again to try it. Review the generated C++ code to see that it has changed.

Next, we're going to modify the template to allow the Python application to multiply by any scalar it wants. Replace the constant in double_template.mako by the placeholder "${scalar}". Next, edit double.py and follow these steps:

• Add a parameter to double_using_template() for specifying the scalar multiple to multiply by.
• Pass the parameter correctly to mytemplate.render().
• Update test_generated() to pass some constant value in to double_using_template() for the new argument.

Test your changes using "python double.py". This completes the template exercise.

See screencast of this section of the tutorial. It is recommended to complete the tutorial section on your own before viewing the screencast.

Tree transformations

A more flexible, but more complex way of specifying a specializer is to take as input a piece of code written in Python, parse it to obtain an abstract syntax tree data structure, and then perform transformations on this tree in order to convert it into a C++ abstract syntax tree. Once this is done, SEJITS will turn it into C++ source, compile it, and run it. Tree transformations can be used together with templates.

Exercise

We assume you're logged into the server and in your tutorial directory as in the Template exercise. The tree transformation specializer we will be modifying is in calc.py. It takes a Python function which adds integer constants and converts it to C++. Run "python calc.py" to see it working. Examine the C++ output in module.cpp and compare it to the Python input in the kernel() function in calc.py:

class MyKernel(CalcKernel):
    def kernel():
        return 2 + 3

Edit calc.py and modify the kernel() function to use * to multiply the integers instead of + to add them. Run "python calc.py" again. You should receive an error, "No visitor defined for Mult node", indicating that the specializer has not yet implemented translation of the node that represents the "*" operator. Edit calc.py again and fix this error by adding a visit_Mult method to the ConvertAST class. Base it on the existing visit_Add method. Run "python calc.py" again, and you should get the expected result (6).

Next, we're going to implement a (somewhat artificial) new feature in our domain-specific language: whenever the application programmer calls the function "two()", the constant "2" will be substituted for it. The first step is to determine what a call to "two()" looks like in the Python abstract syntax tree. To figure this out, we open up a Python interactive environment by typing "python", then type the following:

from asp.codegen.python_ast import *
dump(parse("two()"))

This will produce the following result:

Module(body=[Expr(value=Call(func=Name(id='two', ctx=Load()), args=[], keywords=[], starargs=None, kwargs=None))])

The part we care about, bolded above, is the Call node, representing a function call. The key-value pairs represent properties of that node, which can be accessed using "." (e.g. if x is of type Call, x.args would give its list of arguments).

To extend the specializer to handle calls to two(), we add a "visit_Call" method to ConvertAST. It should contain an "if" to check the function name is "two", and should produce a C++ node representing the integer 2 (use visit_Num as a model to see how to generate such a C++ node). To test your code, modify kernel() to call two(), and then run "python calc.py".

This completes the introductory tutorial. Visit the Tutorials page to learn more about writing SEJITS specializers.