Single File Libraries

For various reasons, I like libraries that exist as single files. There are many online, but I occasionally need to package something that doesn't exist in that form.

For this purpose, I created the sinfil utility. It reads definitions and lists of source files, executes #include directives which relate to the source code being packaged, and produces a combined source code.

I have used this on freetype2 and I want to tackle AGG next. Let's review the tool and then get to the results.

The command line to the utility consists of variable definitions and names of specification files. A variable definition on the command line takes the form NAME=VALUE.

Typically, you want the root directory of the software package to be a variable defined in this way. Rest of the arguments name the specification files. These are read in sequence and processed as if you had concatenated them and provided all as a single file.

Specification files are line based text files with the leading word being a command and the rest of the line its argument. # characters start a comment when they are at the beginning of a line. They are processed as normal characters elsewhere.

The first sequence of characters which are not space or control characters on a line form the command name. Following the command, there shall be one or more space characters and after that, we have the arguments.

If there is more than one argument for a command, these are separated by space characters. All arguments are trimmed such that no space or control characters appear at the beginning or the end of the argument. There is no way to include such things in an argument meant to be used as a word. This can be a problem with file names, but space-separated file names are so rare, I don't think this will be a big problem. Let's start with the easy commands.

outfile <filename>: Creates the output file. This should occur early in the spec file because all commands which print to the output need to have this defined.
print <string>: prints the given string to output. Leading and trailing spaces are trimmed.
copyright <filename>: Copies the contents of the given file to the output.
end: Ends the processing of the current specs file. Any contents past this command is irrelevant.
var <name> <value>: Creates a new variable with the given name and value. Variables are referred to using the '$' sign, just like in shells. The only use for a variable is in file paths.

sinfil works by including relevant headers into the final source file. It executes #include directives when it can figure out which file a directive corresponds to. However, each file is included only once. Sometimes this is not the correct behaviour because header files are also used as data files occasionally. For this reason, you may lift the one time inclusion restriction by using the following command on a specific file.

unprotect <filename>: Given file is allowed to be included multiple times.

sinfil isn't very smart. It can get confused if you mention the same file using different names. Therefore, it's best to use absolute paths whenever possible.

So, which headers are incorporated into the final source code? The ones sinfil knows about. One way of informing it is the bpref command. This command will inform sinfil about headers that are under a specific directory.

bpref <directory> <path>: When an include directive of the form #include <directory/*> is encountered, it's searched for under the given path. This command may be given multiple times with the same directory prefix.

For instance, if you have:

 #include <freetype/ftlist.h>

You can inform the position of the whole directory using this command:

bpref freetype  $TOP/include

Now, the above include directive will resolve to

  $TOP/include/freetype/ftlist.h

sinfil by default won't try to execute bracketed #include directives which don't fit into a pattern like the above. Therefore, it will not incorporate things like <stdio.h> However, you may override this behaviour using the braw command.

braw <filename> <path>: Assigns path to filename. When #include <filename> is encountered, the given path is taken to locate the file. The path has to contain the filename.

For instance, I have this in my freetype2 specs:

braw ft2build.h $TOP/include/ft2build.h

This is because many files include this file using:

#include <ft2build.h>

Let's now discuss #include directives that use quotes instead of brackets. sinfil looks for the included file in the directory of the file doing the inclusion first. If it doesn't find it there, the directories given using the following command is used.

qpath <directory>: Given directory is added to list of directories to be used for resolution of #include "file" directives.

If the given file isn't found, the directive doesn't get executed and is passed on to the output.

Sometimes the argument to #include is not a bracketed or quoted file name, but a macro to be expanded. freetype2 uses this a lot for modularization. When that is the case, you may define the relevant macro for the inclusion to proceed.

defmac <macro_name> <path>: Creates a 'macro' definition to be used in #include macro_name directives. The utility isn't very smart, it just looks at the argument of #include and tries to find it within its tables.

When the macro name is found and used, the directive is processed as a quoted include directive.

In the freetype2 example, I have the following in my specs:

defmac FT_CONFIG_CONFIG_H              $CFG/ftconfig.h

Using this, the tool is able to execute directives such as

#include FT_CONFIG_CONFIG_H

The last command affecting file search is replace.

replace <file> <replacement>: Uses 'replacement' instead of given file. When the file is requested to be read, the tool reads replacement instead.

The directory or name of replacement doesn't affect the operation, those of 'file' are still recorded. It's as if you had executed

  cp replacement file

before running the tool. This is handy when you want to patch some file but don't want to modify the distribution tree.

There are commands which do pre or post processing. The 'privident' command declares some identifiers as private to the next source file. All occurences of such identifiers are prefixed with a given string for lines occuring in the source file. The syntax is:

  privident prefix ident1 ident2 ...

If you have a source file which creates a static symbol without a suitable prefix, you can use this command to add that prefix so that the symbol doesn't clash with definitions in other source files. For instance, if you have the following in process.c:

  int N;

  foo () 
  {
    N = 5;
  }

You could do the following in the specs file to make it more sensible:

  privident process_  N
  source process.c

The result would then be:

  int process_N;

  foo ()
  {
    process_N = 5;
  }

The privident is valid only for the next source file mentioned. After that, the list of prividents are reset. Also, only the text within the source file is processed in this way, not any included files.

These identifiers are replaced only if they occur where a symbol would occur. They are not replaced if they are used as references to struct members. This creates some weird potential bug. Let's examine the following example:

process.c:
----------
  struct {
    int foo; 
  } x;
 
  double foo;

  bar()
  {
    foo = 8.0;
    x.foo= 99;
  }
specs.txt:
----------
privident process_ foo
source process.c

Here, the first three mentions of foo would be replaced because they don't follow a '.' or a '->' token. The last one would be passed as-is. This is obviously an incorrect thing to do. If your source file has such problems maybe it's best to make a copy of it and do the necessary modifications manually.

The 'undef' command prints a bunch of '#undef' statements. The syntax is:

 undef ident1 ident2  ..

The last command is the most important one.

source <file>: Incorporates the given file into the output.

Single File Libraries

The freetype2 Library

AGG Library