diff options
| -rw-r--r-- | doc/concepts/overview.org | 193 |
1 files changed, 193 insertions, 0 deletions
diff --git a/doc/concepts/overview.org b/doc/concepts/overview.org new file mode 100644 index 00000000..2799a2eb --- /dev/null +++ b/doc/concepts/overview.org @@ -0,0 +1,193 @@ +* Tool Overview + +** Structuring + +*** Structuring the Build: Targets, Rules, and Actions + +The primary units this build system deals with are targets: the +user requests the system to build (or install) a target, targets +depend on other targets, etc. Targets typically reflect the units a +software developer thinks in: libraries, binaries, etc. The definition +of a target only describes the information directly belonging to +the target, e.g., its source, private and public header files, and +its direct dependencies. Any other information needed to build a +target (like the public header files of an indirect dependency) +are inferred by the build tool. In this way, the build description +can be kept maintainable + +A built target consists of files logically belonging together (like +the actual library file and its public headers) as well as information +on how to use the target (linking arguments, transitive header files, +etc). For a consumer of a target, the definition of this collection +of files as well as the additionally provided information is what +defines the target as a dependency, respectively of where the target +is coming from (i.e., targets coinciding here are indistinguishable +for other targets). + +Of course, to actually build a single target from its dependencies, +many invocations of the compiler or other tools are necessary (so +called "actions"); the build tool translates these high level +description into the individual actions necessary and only re-executes +those where inputs have changed. + +This translation of high-level concepts into individual actions +is not hard coded into the tool. It is provided by the user as +"rules" and forms additional input to the build. To avoid duplicate +work, rules are typically maintained centrally for a project or an +organization. + +*** Structuring the Code: Modules and Repositories + +The code base is usually split into many directories, each containing +source files belonging together. To allow the definition of targets +where their code is, the targets are structured in a similar way. +For each directory, there can be a targets files. Directories for +which such a targets file exists are called "modules". Each file +belongs to the module that is closest when searching upwards in the +directory tree. The targets file of a module defines the targets +formed from the source files belonging to this module. + +Larger projects are often split into "repositories". For this build +tool, a repository is a logical unit. Often those coincide with +the repositories in the sense of version control. This, however, +does not have to be the case. Also, from one directory in the file +system many repositories can be formed that might differ in the +rules used, targets defined, or binding of their dependencies. + +** Staging + +A peculiarity of this build system is the complete separation +between physical and logical paths. Targets have their own view of +the world, i.e., they can place their artifacts at any logical path +they like, and this is how they look to other targets. It is up to +the consuming targets what they do with artifacts of the targets +they depend on; in particular, they are not obliged to leave them +at the logical location their dependency put them. + +When such a collection of artifacts at logical locations (often +referred to as the "stage") is realized on the file system (when +installing a target, or as inputs to actions), the paths are +interpreted as paths relative to the respective root (installation +or action directory). + +This separation is what allows flexible combination of targets from +various sources without leaking repository names or different file +arrangement if a target is in the "main" repository. + +** Repository data + +A repository uses a (logical) directory for several purposes: to +obtain source files, to read definitions of targets, to read rules, +and to read expressions that can be used by rules. While all those +directories can (and often are) be the same, this does not have +to be the case. For each of those purposes, a different logical +directory (also called "root") can be used. In this way, one can, +e.g., add target definitions to a source tree originally written for +a different build tool without modifying the original source tree. + +Those roots are usually defined in a repository configuration. For +the "main" repository, i.e., the repository from which the target +to be built is requested, the roots can also be overwritten at the +command line. Roots can be defined as paths in the file system, +but also as ~git~ tree identifiers (together with the location +of some repository containing that tree). The latter definition +is preferable for rules and dependencies, as it allows high-level +caching of targets. It also motivates the need of adding target +definitions without changing the root itself. + +The same flexibility as for the roots is also present for the names +of the files defining targets, rules, and expressions. While the +default names ~TARGETS~, ~RULES~, and ~EXPRESSIONS~ are often used, +other file names can be specified for those as well, either in +the repository configuration or (for the main repository) on the +command line. + +The final piece of data needed to describe a repository is the +binding of the open repository names that are used to refer to +other repositories. More details can be found in the documentation +on multi-repository builds. + +** Targets + +*** Target naming + +In description files, targets, rules, and expressions are referred +to by name. As the context always fixes if a name for a target, +rule, or expression is expected, they use the same naming scheme. +- A single string refers to the the target with this name in the + same module. +- A pair ~[module, name]~ refers to the target ~name~ in the module + ~module~ of the same repository. There are no module names with + a distinguished meaning. The naming scheme is unambiguous, as + all other names given by lists have length at least 3. +- A list ~["./", relative-module-path, name]~ refers to a target + with the given name in the module that has the specified path + relative to the current moudle (in the current repository). +- A list ~["@", repository, module, name]~ refers to the target + with the specified name in the specified module of the specified + repository. + +Additionally, there are special targets that can also be referred +to in target files. +- An explicit reference of a source-file target in the same module, + specified as ~["FILE", null, name]~. The explicit ~null~ at the + second position (where normally the module would be) is necessary + to ensure the name has length more than 2 to distinguish it from + a reference to the module ~"FILE"~. +- A reference to a tree target in the same module, specified as + ~["TREE", null, name]~. The explicit ~null~ at second position is + required for the same reason as in the explicit file reference. + +*** Data of an analyzed target + +Analyzing a target results in 3 pieces of data. +- The "artifacts" are a staged collection of artifacts. Typically, + these are what is normally considered the main reason to build + a target, e.g., the actual library file in case of a library. +- The "runfiles" are another staged collection of artifacts. Typically, + these are files that directly belong to the target and are somehow + needed to use the target. For example, in case of a library that + would be the public header files of the library itself. +- A "provides" map with additional information the target wants + to provide to its consumers. The data contained in that map can + also contain additional artifacts. Typically, this the remaining + information needed to use the target in a build. + + In case of a library, that typically would include any other + libraries this library transitively depends upon (a stage), + the correct linking order (a list of strings), and the public + headers of the transitive dependencies (another stage). + +A target is completely determined by these 3 pieces of data. A +consumer of the target will have no other information available. +Hence it is crucial, that everything (apart from artifacts and +runfiles) needed to build against that target is contained in the +provides map. + +When the installation of a target is requested on the command line, +artifacts and runfiles are installed; in case of staging conflicts, +artifacts take precedence. + +*** Source targets + +**** Files + +If a target is not found in the targets file, it is implicitly +treated as a source file. Both, explicit and implicit source files +look the same. The artifacts stage has a single entry: the path is +the relative path of the file to the module root and the value the +file artifact located at the specified location. The runfiles are +the same as the artifacts and the provides map is empty. + +**** Trees + +A tree describes a directory. Internally, however, it is a single +opaque artifact. Consuming targets cannot look into the internal +structure of that tree. Only when realized in the file system (when +installation is requested or as part of the input to an action), +the directory structure is visible again. + +An explicit tree target is similar to an explicit file target, except +that at the specified location there has to be a directory rather +than a file and the tree artifact corresponding to that directory +is taken instead of a file artifact. |