Simple Script

JJ : The Builder

jj is a make replacement. It works in pretty much the same way, but it fixes some of the things done wrongly in make.

It has proper support for multiple targets being generated by one command. GNU make normally runs a command multiple times if multiple output files generated by the same command is out-of-date.
It supports GNU make's VPATH feature in a more generic way.
It allows user to define new functions which makes it easier to build lots of similar files. Unlike in make, you can have many types of C files each with their own rules.

Unlike make, jj does not include any pre-defined commands to build files. You need to define how to build a file using commands. A command is an object with three important fields: triggers, action and outputs. When you create a command with these fields, jj looks at the files in the triggers field and runs the action, hopefully producing the outputs. Let's see a simple example, C compilation

  {
    {$0.0} {.c} {.o} {$output} rep.sfx
    {$id} cmd.new
    {$id} {
      triggers {$0} 
      sources {$0} 
      outputs {$output}
      flags {$1}
      action {gcc -o $output -c $sources -Wall -g $flags}
      message {Compiling $sources}
    } cmd.set
  } {compile} def
  
  {parser.c} {} compile

As you can see a postfix notation is used for defining and using the functions. In the last line, we call our function compile with two arguments, a list consisting of one file (parser.c) and an empty list. Whenever you call a function, arguments have to be in list notation, enclosed in curly brackets. Otherwise, jj would interpret the argument as a function name and try to call it.

In the first line of the function, we transform the input name to the output name using the rep.sfx function. It replaces the suffix as you can imagine.

rep.sfx: 4 arguments.

The input value.
The old suffix to be replaced. If one of the values in the input list doesn't have this suffix, it will be passed as it is.
The new suffix.
The name of the variable to store the result.

This function transforms the strings in the first argument by removing the old suffix and appending the new suffix. It then stores these strings in the given variable as a list.

The first argument in this call looks rather strange: $0.0. This is a positional parameter and represents the first argument in the call. In our case, it expands to parser.c. Positional parameters are represented as $N where N is a non-negative integer. There is no upper or lower limit on the number of arguments a function can take. So, the proper way to address the first positional parameter would be $0. However, we want only the first item of the list, that is $0.0. The second item of this list would be $0.1 and so on. Please note that addressing items of lists in this way can only be done for positional parameters and not variables.

Anyway, second and third arguments are simple strings. Note that jj is space- insensitive. We could just as well have written

  {$0.0}  {
       .c }{.o        } {$output} rep.sfx

White space is important to separate items of a list from each other. Other than that, it has no value and is discarded after parsing.

The fourth argument is the name of a variable, $output. Lowercase identifiers refer to local variables accessible only in the function they are defined in. Uppercase identifiers refer to global variables. All variable names start with the '$' character. It doesn't matter whether you're accessing it or setting it.

Finally, rep.sfx is the name of the function. Function names can have any number of funny punctuation characters in them, lest they be left or right curly braces. Function names are case sensitive though. Now we create a command using the cmd.new function.

cmd.new: 1 argument

The name of the variable to store the handle for the command.

This function creates a new command and stores the handle for the command in the given variable. The handle is a simple integer. You can later use this handle to modify the contents of the command, as shown below.

cmd.set: 2 arguments

The handle of the command.
A list of assignment pairs.

This function finds the command with the given id and sets its fields as given in the second argument. For the example above,

   {$id} {
      action {gcc -o $output -c $sources -Wall -g $flags}
   } cmd.set

sets the action field of the command to the string in curly braces. The action field is one of the special fields for jj. In this field, we see two variable references: $output and $flags. These are not defined anywhere, where do they get their values? The answer is: the command object. If the message or action fields of a command object contains unresolved variable references, jj tries to resolve the reference by looking at the fields of the command. For the above example, the value of the flags field is inserted in its place when it's time to run the action. This has several advantages. For instance, you can modify the fields of a command after you've defined it. However, a way to access the current value of a command is missing for now. I'll implement it later.

There are two more mandatory fields in a command: triggers and outputs. jj decides whether or not to run a command by looking at the modification times of the files given in these fields. If outputs or triggers are not available for a command, it will always run.

Another field in the command object is the message. If you set this field for your commands and then run jj with the -m option, then jj will print the message instead of displaying the command it's executing. For the above example, the console output of jj would be:

  jj: Compiling parser.c

Anyway, the last two lines of the function definition makes the function available.

   } {compile} def

The def function takes a body and a name as an argument and places the function in the function table.

Predefined Variables

TODO: explain $HERE and $CWD here.

Other Functions

list.trans str.cat {print/dump/dump.cmd} include set eval comment

I need a method of obtaining a program's output. Like the backtick operator in shell. I could then use this to make things like

  {freetype-config --libs} {$FreeTypeLibs} getprgout

This is now done. The prgout function works like this:

  {$LIBS.freetype} {pkg-config} {--libs} {freetype2} prgout

The function interprets the first argument as the variable name to store the result. The rest of the arguments build up the argv vector. The program is then executed directly (i.e. execvp) and the output is put into the given variable. Variables are allowed in the argument specifications.

TODO

rep.sfx should be just like str.cat. It should return a value instead of setting the variable itself. Although it does work for both lists and strings, the str.cat approach is more elegant.

I need to clean up the code a lot. There are some things that could be done inside functions.

I need a way to check conditions. Like

  { code } {variable value} ifeq

or something.

I need to access environment variables from within jj. setenv() would also be a good addition.

I need some standard libraries to compile, generate C code etc. Copying from place to place isn't nice.

I need a way of determining which command is responsible for generating a given output file.

Better argument checking. For instance, I got confused and made this:

  {$id} {
    stuff
  } cmd.new

instead of

  {$id} cmd.new
  {$id} {
      stuff
  } cmd.set

Of course, I ended up with an empty command that does nothing.

I need a way to specify phony targets that don't exist as a file on disk.

I need to make a filter in order to run only the commands that play a role in generating a given target. For instance, if I need only one executable and not the whole set of outputs, I should be able to do:

  jj -t mex

or similar.. Just like

  make mex

This is somewhat optional, but I'd like to stop execution at a certain point. For instance, after all C sources have been generated. I can do this using a function and an extra command field.

   {compilation linking library-generation} forbid

and

   {$id} { class {compilation} } cmd.set

would do it. Whenever you forbid a class of commands, they are removed from the command list prior to dependency checking.

Another nice feature would be a 'make clean' option.. i.e.

   jj -c

removes all files that appear in the output list.

I need a way to import gcc's -MMD output.

A new feature maybe.. Instead of executing commands or writing them to the console, we could maybe make a shell script out of commands. This would have some problems with configuration variables but maybe I can provide protection for them.. For instance:

   { top.dir {$HERE ../}
      protect {$LIBS.fontconfig}
      protect {$LIBS.freetype}
  } script.set

Really tough stuff but worth consideration.

There is no need for protection. Instead we can do this:

  { $LIBS_FREETYPE } env.set.forget

This would set the environment variable to the value of the global variable $LIBS_FREETYPE and then remove that variable from the global function table. This way, when you execute the command which uses $LIBS_FREETYPE, the shell expands it. When you print the command instead, the variable is not expanded and printed as it is: $LIBS_FREETYPE. Thus the resulting string can be put in a script and a shell can later expand it. The variables used in this fashion need to be written in a format compatible with the shell (i.e. no funny punctuation in it).

I need to rename some functions. rep.sfx should really be sfx.rep then maybe I could add sfx.filter etc.. list.trans seems a bit strange. maybe list.tr is better. for traverse and translate.

OK. cool idea. if there are unrecognized command line arguments, they get defined as global variables. i.e.

    jj -debug

would set $DEBUG={yes} and

    jj -K=3

would set $K={3}. Way cool. This will also allow me to implement environments. i.e. think about

    jj -win32

or similar..