 |
 |
 |
 |
 |
 |
 |
 |
In the following discussion, I refer to three directories: ‘Y_SITE’ is the directory where the architecture independent parts of Yorick reside; everything Yorick needs at runtime is here. ‘Y_HOME’ is the directory where the libraries and executables you need to build custom versions are stored. ‘Y_LAUNCH’ is the directory containing the executable for your version of Yorick. When you run Yorick, the names of all three directories are available as variables; for example, start yorick and type Y_HOME to print the name of that directory. Also, I will only discuss the UNIX program development environment; the same system works tolerably well under the MinGW or Cygwin environments under Windows, which you can use with the commerical MSVC++ compiler if you wish.
The paradigm for a compiled extension to yorick is called a “compiled package”. The idea is that a compiled package should be indistinguishable from an interpreted package (for example, a .i file in the interpreted library that comes with the distribution). In the case of a purely interpreted package, you write an include file defining your interface. So, too, you must lay out the interpreted interface for a compiled package in an interpreted include file, which we call here ‘mypkg.i’. Including ‘mypkg.i’ makes the compiled package available to the interpreter.
The compiled package include file ‘mypkg.i’ must contain one statement, executed before any code that uses the compiled routines, which distinguishes it from an interpreted package:
plug_in, "mypkg"; |
Here, we assume the package name is “mypkg”, matching the name of the file ‘mypkg.i’, but you can choose any name for the package you like. In fact, you can have any number of include files defining different pieces of the interface to a single compiled package, say ‘mypkg1.i’, ‘mypkg2.i’, and so on, each of which would contain the above plug_in call. In a single compiled package source directory, however, there can be at most one package name. (If you want to build more than one binary library, use more than one directory to hold the source.)
An alternative form is permitted to make it possible to design package include files which are compatible with pre-version-1.6 yorick:
if (!is_void(plug_in)) plug_in, "mypkg"; |
Unlike an interpreted package, before you can use a compiled package, you naturally need to compile it! This is done using the UNIX make utility. Yorick has tools to assist in this build process. The first is the (interpreted) package make.i which generates, or at least provides a starting point for, the ‘Makefile’, which is the input to the make utility. The second is codger, a (compiled) yorick utility which generates the C source code that bridges between your compiled code and yorick’s internal compiled code.
First, you need to put your C (and/or Fortran) source and header files in the directory containing the ‘mypkg.i’ interface definition file(s). Note that you need to design your C interface in an “interpreter friendly” fashion. For example, it is worthless to write a special function that computes a special function at a single point, because the interpreter would need to call it in a loop to compute a vector of values, most likely erasing the entire advantage of writing your special function in compiled code. At the very least, therefore, you want your compiled functions to accept a (potentially long) list of inputs an produce the corresponding list of outputs.
When all the .i, .c, and .h (and Fortran) source files for your package have been collected in this package source directory (and any unrelated source removed), execute the command (in that directory):
yorick -batch make.i |
This creates a ‘Makefile’ for your package. In simple cases, this ‘Makefile’ will be all you need to build your compiled package; in more complicated situations, you will need to edit it by hand to fill in critical information the ‘make.i’ package is unable to determine by its examination of the source files.
The main reason you may need to edit ‘Makefile’ by hand is to add the loader flags (-l and -L) for dependencies, that is, third party libraries, your compiled code calls. You may also need to add compiler flags (-I) to enable the compiler to find the header files which declare the interfaces to such libraries. The corresponding macros in ‘Makefile’ are PKG_DEPLIBS and PKG_CFLAGS, respectively. In general, if your package needs modifications like this and you wish it to be buildable by non-experts, you will need to modify ‘Makefile’ to pick up the results of a configure script, which you will need to design and write. This can be far more difficult than writing the compiled package in the first place. The message is, do not rely on dependent libraries other than libc, libm, and the utilities in yorick’s own ‘play/’ portability layer if you can possibly avoid it.
Once you have the ‘Makefile’, use the make utility to actually build your compiled package. The ‘Makefile’ invokes a more extensive input file, ‘Y_HOME/Makepkg’, which provides a number of useful “targets” you can build. Here are several useful make command lines; read ‘Y_HOME/Makepkg’ to learn about others:
make # build release version make install # install it (may need to be root) make debug # build debuggable version make clean # delete all but source and Makefile |
The release version of your package will be a dynamically loadable yorick plugin, as long as yorick is able to handle plugins on your platform. The debuggable version, and the release version on platforms which do not permit plugins, will be a complete yorick executable, which statically loads your new package at startup. (By default, it will simply be called “yorick”, and installing such a release version it will overwrite the yorick you used to run -batch make.i, so be careful.)
When you want to build your package on a different platform, you always need to run:
yorick -batch make.i |
before you rerun the make utility.
In addition to the plug_in command, your ‘mypkg.i’ package interface file must contain statements that allow the codger utility to construct the bridge code. During the build process, codger examines ‘mypkg.i’ package, looking for interpreted extern statements outside any function body. For each such extern statement, codger generates an interpreted built-in function, or an interpreted pointer to compiled global data. When the interpreter includes the file, these statements are no-ops, except that they may include DOCUMENT comments that are added to the interactive help system. The ‘mypkg.i’ file will usually also contain purely interpreted wrapper code for these compiled objects, in order to create a high-quality interpreted API.
For example, when codger finds this:
extern my_func;
/* DOCUMENT my_func(input)
returns the frobnostication of the array INPUT.
*/
|
in the ‘mypkg.i’ file, it connects the interpreted symbol my_func to a compiled function Y_my_func with a specific prototype, which you are responsible for writing, as described in the following section.
Often you can avoid all these details; codger can generate Y_my_func for you automatically. To do this, put PROTOTYPE comments in your startup include file:
func my_func(input)
/* DOCUMENT my_func(input)
returns the frobnostication of the array INPUT.
*/
{
return my_func_raw(input, numberof(input));
}
extern my_func_raw;
/* PROTOTYPE
double my_func_C_name(double array input, long length)
*/
|
This generates a wrapper for a C function which takes a single array as input and returns a scalar result. If the function had been Fortran, it would have looked like this (Fortran passes all arguments by reference – that is, as if they were arrays):
func my_func(input)
/* DOCUMENT my_func(input)
returns the frobnostication of the array INPUT.
*/
{
return my_func_raw(input, numberof(input));
}
extern my_func_raw;
/* PROTOTYPE FORTRAN
double my_func_Fortran_name(double array input, long array length)
*/
|
Legal data types for the function return result in the PROTOTYPE comment are: void (i.e.- a subroutine), char, short, int, long, float, or double.
Legal data types for the function parameters in the PROTOTYPE comment are: void (only if there are no other parameters), char, short, int, long, float, double, string (char *, guaranteed 0-terminated), or pointer (void *). These may be followed by the word "array", which becomes "*" in the C source code, to indicate an array of that type. The parameter name is optional.
The DOCUMENT comment should start with /* DOCUMENT. They will be returned by the interpreted command help, my_func, and be included in the poor- man’s document produced by Yorick’s "mkdoc" command (see ‘Y_HOME/i/mkdoc.i’).
extern my_global; reshape, my_global, datatype; |
attaches the interpreted variable my_global to a C-compiled global of the same name, which has the data type datatype (this must have been declared in a previous struct or be one of the primitive types). If you want my_global to be attached to a global variable of a different name, use:
extern my_global; /* EXTERNAL my_global_C_name */ reshape, my_global, datatype; |
To attach to a Fortran common block, say
double var1, var2, var3
common /my_common/ var1, var2, var3
save /my_common/
|
(note that this doesn’t make sense unless the common block is saved outside the scope of the functions in which it is used) use:
struct my_common_type { double var1, var2, var3; }
extern my_common;
/* EXTERNAL FORTRAN my_common */
reshape, my_common, my_common_type;
|
If you mix double, integer, and real data in a single common block, you can ensure that you won’t have any alignment difficulties by putting all the doubles first, followed by integers and reals. If you don’t do this, you’re relying on the existence of a Fortran compiler switch which forces proper data alignment – some machine someday won’t have this.
This covers all the things that codger can do for you.
The primary goal of this system is portability. The basic idea is that all the platform specific problems can be solved once in the ‘Y_HOME/Makepkg’ file, so that you can easily move your compiled packages from one platform to another. With the advent of plugins, this system also must cope with packages which are either statically or dynamically loaded, and the same source code must work for either case. The secondary goal is that the final users should do nothing different to use a compiled package than they would to use an interpreted package.
|
|
|
|
|
|
|
|
