## System building

### Introduction

A typical application will consist of multiple lisp files that have to be compiled and which will probably be linked to additional, third party libraries, either written themselves in &CommonLisp; or shipped as foreign C or C++ dynamically or statically linked lirbaries. Not only loading these files into a running &ECL; can be a slow process in some platforms, but shipping code in the form of multiple binaries and a script to load them is far from optimal.

Traditionally, &CommonLisp; implemenations have provided a function to save the dump all data from a running Lisp process into a file. The result was called the Lisp image and could be shipped to other version compatible implementations.Nowadays, having less control of the systems it runs in, a Lisp implementation must work very hard to dump memory images and be able to load and execute them afterwards.

&ECL; has chosen to avoid this process entirely. Instead, we conceive five different portable models for building and shippin your programs. The models, described in table.make-build, enumerate the different kinds of files that &ECL; can portably produce. To get one or more of the products mentioned in the table, you may resort to a low level API described in part.internals. However, we recommend a simpler way based on using System Definition Files to describe the structure of your project and let &ECL; build the desired target for you. This approach is described in the following sections.

### Code distribution models

Model | Description | :TYPE | :MONOLITHIC
Source code | You distribute your programs in source code form. This is the easiest and most portable way, but not the fastest one. | NA | NA
&FASL; or loadable file | Best suited for development. You translate all lisp code to C and link it against possibly other C/C++ libraries to obtain a single binary file with extension `.fas`, like the compiled files you obtain from using `compile-file`. This "unified" &FASL; can be loaded a startup time to add new functionality to the &ECL; environment. | :FASL | T/NIL
Standalone program | Product shipping for final user. You translate all your lisp code to C using the &ECL; compiler. The final object files can be linked against other C/C++ libraries to obtain a standalone executable. | :PROGRAM | T
Statically linked library | For embedding purposes. You translate all your lisp code to C and combine the resulting object files into a single library with `.a` or `.lib` extension. You can distribute this library to other people and the final users can utilize these libraries to build standalone programs. | :LIB | T/NIL
Dynamically linked library | For embedding purposes. Similar to a statically linked library, but it is loaded at run time by the operating system and can be shared by more than one instance of a program. | :LIB | T/NIL

### System definition files

A System Definition File, or just system, is the lisp equivalent of a makefile in the Unix world: it contains a list of source files which are to be loaded or compiled, and dependencies among them ("load source `file1.lsp` before compiling `file2.lsp`", etc).

It is difficult to tell about the Lisp Machines history, but probably the first most popular system definition format was called mk-defsystem or simply defsystem. Written by Mark Kantrowitz [bib.mk-defsystem], this library now lives in the CLOCC repository and is actively maintained. &ECL; ships with a copy of the version 3.x which fortunately has no customizations. You can load this copy by issuing `(require 'defsystem)` from the lisp toplevel.

However, in the last years, Another System Definition Facility known as &ASDF; has become even more popular in the &CommonLisp; world. This new format simplifies writing extensions to handle new kind of source files and integrates very well with the package management utility known as ASDF-install. &ASDF; has a slightly different syntax from mk-defsystem 3.0, but because of reasons of popularity and better integration with &ECL;, in this manual we have focused on this particular library.

A simple &ASDF; definition looks as follows:

    (defsystem test
      :source-pathname "~/src/test/"
      :source-extension "lisp"
      :components ((:module file1
                            :source-pathname "")
                   (:module file2
                            :source-pathname ""
                            :depends-on (file1))))

This example consists of two files, `file1.lisp` and `file2.lisp`, located in `~/src/test/`. When compiling these files, `file1.lisp` will be processed before `file2.lisp`, because the second depends on the former one. There are more complex rules that allow a system to depend on others, and to contain other kind of files, such as C or Java binaries. For further information we recommend reading the online manual.

You can load &ASDF; on a running &ECL; using a single lisp statement `(require 'asdf)`. Once loaded, &ASDF; will extend the function `require` to recognize and load libraries that are placed in standard locations or which have been registered with &ASDF; itself. The following sections describe other features of &ASDF; which are specific to &ECL; and related to the code building and shipping mechanisms introduced before.

### Practical examples

The version of &ASDF; which is shipped with &ECL; has been further customized to allow building all the binary files mentioned in table.make-build. The procedure to do this is documented in a detailed and formal manual page for ref.make-build. However, since practice is the best teacher, we will show a couple of examples of how to use this function before moving into the formal specification.

In `/ecl/examples/asdf` you will find a very simple example that can be built in different forms. The example is built around a system definition file that depends on two sources, `file1.lisp` and `file2.lisp`:

    (defsystem #:example
        :serial t
        :components ((:file "file1")
    		 (:file "file2")))

We can built these files into a single &FASL; file, as shown below. Notice how there is a single file with the name `*.fas`, but there are two object files generated from their respective sources, `file1.o`, `file2.o`.

    > (require 'asdf)
    ;;; Loading #P"/home/jlr/lib/ecl/asdf.fas"
    ("ASDF")
    > (asdf:make-build :example :type :fasl)
    ...
    NIL
    > (directory "*.o")
    (#P"/home/jlr/src/ecls-new/examples/asdf/file2.o"
     #P"/home/jlr/src/ecls-new/examples/asdf/file1.o")
    > (directory "*.fas")
    (#P"/home/jlr/src/ecls-new/examples/asdf/example.fas")
    > (load "example.fas")
    ;;; Loading "/home/jlr/src/ecls-new/examples/asdf/example.fas"
    ======================================================================
    We are now executing FILE1.LSP
    TEST-FUNCTION has been created
    We are now executing FILE2.LSP
    Calling TEST-FUNCTION in FILE2.LSP
    1 + 1 is equal to 2
    Finished
    ======================================================================
    "/home/jlr/src/ecls-new/examples/asdf/example.fas"

The previous sources may be combined into a single program, as shown below. Notice that we choose to execute `ext:quit` right after all compiled files have run. If you do not supply this parameter, `example` will jump to the lisp toplevel right after that.

    > (asdf:make-build :example :type :program
                       :epilogue-code '(ext:quit 0))
    NIL
    > (ext:system "./example")
    ======================================================================
    We are now executing FILE1.LSP
    TEST-FUNCTION has been created
    We are now executing FILE2.LSP
    Calling TEST-FUNCTION in FILE2.LSP
    1 + 1 is equal to 2
    Finished
    ======================================================================

### ASDF Reference

`asdf:make-build`

Block-build an &ASDF; system definition

#### Function

          asdf:make-build
          system-name
          &key;
          type
          monolithic
          ld-flags
          prologue-code
          epilogue-code
          &allow-other-keys;

`system-name`

- A symbol naming the system to be built. Only the symbol name is considered.

`type`

- One of :FASL, :DLL, :LIB or :PROGRAM

`monolithic`

- A boolean value.

`ld-flags`

- A list of strings.

`prologue-code`

- A string.

`epilogue-code`

- A string or a lisp form.

##### Description

This function takes a system definition which is known to &ASDF; and builds one or more binary files, depending on the arguments. The possible output files depend on the value of `type` and are summarized in table.make-build.

Internally the function works similary to the &ASDF; function `asdf:operate` with the `asdf:load-op` operator. It finds out the requested system definition, either by searching in a set of predefined locations or because the system has been already loaded into memory, computes all libraries and components this system depends on, builds them and then produces the desired output.

If the value of `:monolithic` is `NIL` the output binary will contain just the desired system, while in other cases the output will be linked together with all libraries your system depends on. Standalone executables, as specified by `type = :program`, are always monolithic, even without specifing this flag. All other systems might be non-monolithic, but in that case you will have to manually satisfy the required dependencies when using those files (unless you use asdf:load-bundle-op in which case asdf will satisfy required dependencies for you automatically).

This function takes additional values which are related to the low level details of the produced binaries. First of all we find `ld-flags`, a list of strings with arguments for the object linker. You will only need this argument if you have to link your programs with foreign libraries.

The next two arguments represent two pieces of code which are executed before (`prologue-code`) and after (`epilogue-code`) running your lisp code. The prologue code is a string with C code which you will typically use to initialize foreign libraries. It can only be C code because this code may be executed even before &ECL; itself is initialized.

The epilogue code, on the other hand, can be either a string with C statements or a lisp form represented as a list. In the case of executables it conveniently defaults to a call to the toplevel `(SI::TOP-LEVEL)`, while in the case of libraries and &FASL; files it is left empty.

##### Examples

See ext.asdf.make-build.

`asdf:load-bundle-op`

Compile and load one ore more libraries using unified &FASL;

#### &ASDF; operator

          asdf:make-build
          'asdf:load-bundle-op
          system-name
          &key;
          &allow-other-keys;

`system-name`

- A symbol naming the system to be built. Only the symbol name is considered.

##### Description

This function is a replacement for the &ASDF; operator ASDF:LOAD-OP. Given a system name, it will build it and all its dependencies, to load them in the required order. The only difference with respect to ASDF:LOAD-OP is that it builds a single &FASL; file per module, thus being potentially faster and more resource efficient.

##### Examples

Assume you want to load the &CFFI; library, which has been registered with &ASDF;. You will simply type

    > (require 'asdf)
    ;;; Loading #P"/home/jlr/lib/ecl/asdf.fas"
    ("ASDF")
    > (asdf:operate 'asdf:load-bundle-op :cffi)
    ...
    T