buildsystem.txt 9.61 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89
The build system
================

The build system is not based on automake or CMake or any other
makefile generator. They all try to solve the problems of make by
producing very complex and long inputs for make. That, obviously
doesn't work, so you get both problems of make and huge complexity. If
we are to have the problems of make anyway, we can at least have it
simple.

So, the system leverages some nice features of GNU make and uses just
plain makefiles. It includes sub-makefiles instead of running make
recursively, since it allows tracking dependencies across directory
boundaries and uses parallel compilation across directories.

The makefiles
-------------

There are several kinds of makefiles. The first type is the ones under
the `build` directory. They contain the actual rules and definitions
of the build system itself. It's the magic that makes it work and they
require a good knowledge of GNU make syntax to understand and modify.
But it shouldn't be needed to modify these under usual circumstances.

Then there are the `Makefile.dir` ones. There should be one in each
directory with code and they contain the definitions of what should be
compiled. This is the kind of makefile modified when source files or
subdirectories are added.

The third kind is the `Makefile` in the top-level directory of the
project. This one sets default options for compilation on the given
computer. Currently, it's a static one, so if something doesn't work,
proper values need to be either entered manually or passed on the make
command line. But it is expected to be generated by some kind of
configure script in future (and it would be the only generated file).

The final type is the plain `Makefile` in each directory. These are
just thin wrappers to call the whole build system correctly, so one
can simply call make from whatever directory. No rules or definitions
should come in here.

Defining project parts
----------------------

To compile part of the project (either a binary or library), you do
two things:

 * Append the path of the result to given type of target variable.
 * Define variables defining which files it is built from and other
   options.

Imagine you want to build a binary called `hello_world` and the
sources for it are located in `src/hello_world`. You'd use the
following code:

  # Be sure to append by `+=`, not override by `=`. The basename
  # must be unique across the whole project (you can't have another
  # hello_world in other directory).
  BINARIES += src/hello_world/hello_world
  # List sources. Without the .c at the end or path. The input files
  # don't need to be unique, so there can be another main.c in other
  # directory. You don't list the headers.
  hello_world_SOURCES := \
    main \
    hello \
    world
  # Compilation flags, appended to the common ones.
  hello_world_CFLAGS := -fpermisive
  # Linker flags, similar. Don't use to link against other libraries.
  # But this can be used to provide additional paths, for example
  # (though these should come from the top-level `Makefile`).
  hello_world_LDFLAGS := -static
  # Libraries linked from the system. This one links the math library
  # (-lm) and pcap (-llibpcap)
  hello_world_SYSTEM_LIBS := m libpcap
  # The same for libraries from the project. They are separated so
  # the dependencies against the local libraries can be tracked.
  hello_world_LOCAL_LIBS := libworld

Only the `_SOURCES` variable is mandatory. You can create a shared
library the same way, it is only appended to `LIBRARIES` instead to
`BINARIES`.

You can also generate documentation (`html` from the `txt` by
asciidoc). It has no variables to influence it and you append them to
the `DOCS` variable.

Both `LIBRARIES` and `DOCS` are listed without the suffix.

Michal 'vorner' Vaner's avatar
Michal 'vorner' Vaner committed
90 91 92
For compiling lua, use the variable `LUA`. Results are placed under
`lua_lib`. The names are without suffixes too.

93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145
Note that the results are created in `bin/`, `lib` or `docs`
respectively. They are only symlinked to the directory where they come
from. That means `src/hello_world/hello_world` will be symlink to
`bin/hello_world`.

Dependency tracking
-------------------

The system needs to be told about dependencies on libraries (by the
`_LOCAL_LIBS` variable). But dependencies on header files are tracked
automatically. It uses the `gcc`'s feature of storing the dependencies
into a separate file and then using it on successive runs.

This works reasonably well with one exception -- removing a header
file from repository. If you remove all references to the header and
remove the header itself, the dependency on it is still recorded and
make will fail. You need to run `make clean` in that case and compile
from scratch.

The correct way to remove a header is to remove the `#include` lines,
recompile and remove the file afterwards.

Configuring the build
---------------------

There are several variables that influence how the project is built.
They can be specified either in the top-level `Makefile` or on the
`make`'s command line (eg. by calling `make RELEASE=1` to create a
release build). The values on command line take precedence (as usual
with make).

If you provide different options than before, you should start with a
clean build. Otherwise, there could be some confusion if only some
parts would be rebuilt with the new flags and some were left from
previous builds with previous options.

Variables that are expected to be set are:

ASCIIDOC::
  The asciidoc tool used to compile.
CC::
  The C compiler used. It is tested with `gcc`, but others might
  work as well. You can set it to a compiler for other architecture
  for cross-compiling. This overrides whatever is set by the `TARGET`
  variable.
CFLAGS::
  Flags passed to the compiler. This is expected to contain things
  like optimisation flags. There will be other options appended to
  this by the build system.
+
It overrides whatever is set by the `TARGET` variable and `RELEASE`
variable.

146 147 148 149
COMPILE_LUA::
  If set to `yes` (the default), it compiles lua files for syntax
  checking and faster loading. If set to anything else, it leaves the
  lua files be.
150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216
EXTRA_DEFINES::
  Additional defines that will be passed to the compiler.
EXTRA_INCLUDES::
  Additional paths to search for inclusion of header files.
EXTRA_LIBS::
  Additional libraries to link into the result.
LDFLAGS::
  Flags passed to the linker. Other flags will be appended by the
  build system.
+
This overrides whatever is set by the `TARGET` and `RELEASE`
variables.

MAX_LOG_LEVEL::
  Maximum level of log messages that are output. Default is LOG_DEBUG.
NO_DOC::
  Disables compilation of documentation.
O::
  The top-level output compilation directory. See following chapter.
  It is not expected to be changed on command line.
PAGE_SIZE::
  The page size to be passed as a define to the compiled code. It is
  attempted to be auto-detected, but can be redefined to something
  else.
PLUGIN_PATH::
  Path where the plugin libraries should be found in the target
  system. May be absolute or relative. Defaults to ``lib''.
RELEASE::
  By default, it compiles with debug flags (eg. `-O0 -ggdb3`). With
  this set, it uses optimisation flags (`-O2`).
S::
  The top-level source directory. See following chapter. It is not
  expected to be changed on command line.
STATIC::
  If set, link statically instead of dynamically.
TARGET::
  This can be used for cross-compiling for other architectures. It'll
  change the compiler used and flags. Currently, the value of `mips`
  is supported. It also sets the `STATIC` variable, so the generated
  output is easier to just copy to the target machine.
VERBOSE::
  By default, the system prints a short summary (`CC file.o`). If this
  is set to something, it outputs the whole compilation command.

Separate build directory
------------------------

The system allows for building in a separate directory from the
sources. It can be used when the source is on read-only filesystem or
when there should be multiple builds with different options.

The `O` variable specifies the directory where results will be
created. The `s` where the sources reside.

The easiest way to create a new build directory is to copy the
top-level `Makefile` to the place where the build should happen. Then,
the `S` variable there should be adjusted to point to the sources
directory. Optionally, other options can be tweaked there. The `O`
variable can be left intact.

Defines passed to the compiler
------------------------------

We use compiler defines (`-D`) to provide some configuration to the
compiled source code, instead of generating `config.h`. These are the
currently passed ones.

217 218
COMPILE_LUA::
  Defined if the lua files are being compiled.
219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249
DEBUG::
  Defined if the RELEASE is not turned on.
MAX_LOG_LEVEL::
  Maximum level of log messages to be output. It has the value of
  MAX_LOG_LEVEL variable in make.
OUTPUT_DIRECTORY::
  Value of the `O` variable.
PAGE_SIZE::
  The size of the memory page. It has the value set in PAGE_SIZE
  variable in make.
PLUGIN_PATH::
  Where the plugin libraries should be found at runtime.
PREFIX::
  Currently, value of the `O` variable. It is expected to contain
  whatever prefix the project will be installed into, but since
  installation is not yet supported, we opt for this.
SOURCE_DIRECTORY::
  Value of the `S` variable.
STATIC::
  Defined if STATIC is turned on in build system.

Missing bits
------------

* Installation. The current system compiles the project and it must be
  run from the directory where it was built. The idea is to provide
  another target (`install`) which would re-link all the libraries and
  binaries (so correct paths are embedded in them) with output files
  being set to `PREFIX/{bin/lib}`. The `PREFIX` would be new variable.
  But the plug-ins might want to go somewhere else.
* Linking statically.