doc: Convert orgmode files to markdown

1 files changed, 210 insertions, 0 deletions

diff --git a/doc/concepts/overview.md b/doc/concepts/overview.md
new file mode 100644
index 00000000..a9bcc847
--- /dev/null
+++ b/doc/concepts/overview.md
@@ -0,0 +1,210 @@
+Tool Overview
+=============
+
+Structuring
+-----------
+
+### Structuring the Build: Targets, Rules, and Actions
+
+The primary units this build system deals with are targets: the user
+requests the system to build (or install) a target, targets depend on
+other targets, etc. Targets typically reflect the units a software
+developer thinks in: libraries, binaries, etc. The definition of a
+target only describes the information directly belonging to the target,
+e.g., its source, private and public header files, and its direct
+dependencies. Any other information needed to build a target (like the
+public header files of an indirect dependency) are inferred by the build
+tool. In this way, the build description can be kept maintainable
+
+A built target consists of files logically belonging together (like the
+actual library file and its public headers) as well as information on
+how to use the target (linking arguments, transitive header files, etc).
+For a consumer of a target, the definition of this collection of files
+as well as the additionally provided information is what defines the
+target as a dependency, respectively of where the target is coming from
+(i.e., targets coinciding here are indistinguishable for other targets).
+
+Of course, to actually build a single target from its dependencies, many
+invocations of the compiler or other tools are necessary (so called
+"actions"); the build tool translates these high level description
+into the individual actions necessary and only re-executes those where
+inputs have changed.
+
+This translation of high-level concepts into individual actions is not
+hard coded into the tool. It is provided by the user as "rules" and
+forms additional input to the build. To avoid duplicate work, rules are
+typically maintained centrally for a project or an organization.
+
+### Structuring the Code: Modules and Repositories
+
+The code base is usually split into many directories, each containing
+source files belonging together. To allow the definition of targets
+where their code is, the targets are structured in a similar way. For
+each directory, there can be a targets files. Directories for which such
+a targets file exists are called "modules". Each file belongs to the
+module that is closest when searching upwards in the directory tree. The
+targets file of a module defines the targets formed from the source
+files belonging to this module.
+
+Larger projects are often split into "repositories". For this build
+tool, a repository is a logical unit. Often those coincide with the
+repositories in the sense of version control. This, however, does not
+have to be the case. Also, from one directory in the file system many
+repositories can be formed that might differ in the rules used, targets
+defined, or binding of their dependencies.
+
+Staging
+-------
+
+A peculiarity of this build system is the complete separation between
+physical and logical paths. Targets have their own view of the world,
+i.e., they can place their artifacts at any logical path they like, and
+this is how they look to other targets. It is up to the consuming
+targets what they do with artifacts of the targets they depend on; in
+particular, they are not obliged to leave them at the logical location
+their dependency put them.
+
+When such a collection of artifacts at logical locations (often referred
+to as the "stage") is realized on the file system (when installing a
+target, or as inputs to actions), the paths are interpreted as paths
+relative to the respective root (installation or action directory).
+
+This separation is what allows flexible combination of targets from
+various sources without leaking repository names or different file
+arrangement if a target is in the "main" repository.
+
+Repository data
+---------------
+
+A repository uses a (logical) directory for several purposes: to obtain
+source files, to read definitions of targets, to read rules, and to read
+expressions that can be used by rules. While all those directories can
+(and often are) be the same, this does not have to be the case. For each
+of those purposes, a different logical directory (also called "root")
+can be used. In this way, one can, e.g., add target definitions to a
+source tree originally written for a different build tool without
+modifying the original source tree.
+
+Those roots are usually defined in a repository configuration. For the
+"main" repository, i.e., the repository from which the target to be
+built is requested, the roots can also be overwritten at the command
+line. Roots can be defined as paths in the file system, but also as
+`git` tree identifiers (together with the location of some repository
+containing that tree). The latter definition is preferable for rules and
+dependencies, as it allows high-level caching of targets. It also
+motivates the need of adding target definitions without changing the
+root itself.
+
+The same flexibility as for the roots is also present for the names of
+the files defining targets, rules, and expressions. While the default
+names `TARGETS`, `RULES`, and `EXPRESSIONS` are often used, other file
+names can be specified for those as well, either in the repository
+configuration or (for the main repository) on the command line.
+
+The final piece of data needed to describe a repository is the binding
+of the open repository names that are used to refer to other
+repositories. More details can be found in the documentation on
+multi-repository builds.
+
+Targets
+-------
+
+### Target naming
+
+In description files, targets, rules, and expressions are referred to by
+name. As the context always fixes if a name for a target, rule, or
+expression is expected, they use the same naming scheme.
+
+ - A single string refers to the target with this name in the same
+   module.
+ - A pair `[module, name]` refers to the target `name` in the module
+   `module` of the same repository. There are no module names with a
+   distinguished meaning. The naming scheme is unambiguous, as all
+   other names given by lists have length at least 3.
+ - A list `["./", relative-module-path, name]` refers to a target with
+   the given name in the module that has the specified path relative to
+   the current module (in the current repository).
+ - A list `["@", repository, module, name]` refers to the target with
+   the specified name in the specified module of the specified
+   repository.
+
+Additionally, there are special targets that can also be referred to in
+target files.
+
+ - An explicit reference of a source-file target in the same module,
+   specified as `["FILE", null, name]`. The explicit `null` at the
+   second position (where normally the module would be) is necessary to
+   ensure the name has length more than 2 to distinguish it from a
+   reference to the module `"FILE"`.
+ - A reference to an collection, given by a shell pattern, of explicit
+   source files in the top-level directory of the same module,
+   specified as `["GLOB", null, pattern]`. The explicit `null` at
+   second position is required for the same reason as in the explicit
+   file reference.
+ - A reference to a tree target in the same module, specified as
+   `["TREE", null, name]`. The explicit `null` at second position is
+   required for the same reason as in the explicit file reference.
+
+### Data of an analyzed target
+
+Analyzing a target results in 3 pieces of data.
+
+ - The "artifacts" are a staged collection of artifacts. Typically,
+   these are what is normally considered the main reason to build a
+   target, e.g., the actual library file in case of a library.
+
+ - The "runfiles" are another staged collection of artifacts.
+   Typically, these are files that directly belong to the target and
+   are somehow needed to use the target. For example, in case of a
+   library that would be the public header files of the library itself.
+
+ - A "provides" map with additional information the target wants to
+   provide to its consumers. The data contained in that map can also
+   contain additional artifacts. Typically, this the remaining
+   information needed to use the target in a build.
+
+   In case of a library, that typically would include any other
+   libraries this library transitively depends upon (a stage), the
+   correct linking order (a list of strings), and the public headers of
+   the transitive dependencies (another stage).
+
+A target is completely determined by these 3 pieces of data. A consumer
+of the target will have no other information available. Hence it is
+crucial, that everything (apart from artifacts and runfiles) needed to
+build against that target is contained in the provides map.
+
+When the installation of a target is requested on the command line,
+artifacts and runfiles are installed; in case of staging conflicts,
+artifacts take precedence.
+
+### Source targets
+
+#### Files
+
+If a target is not found in the targets file, it is implicitly
+treated as a source file. Both, explicit and implicit source files
+look the same. The artifacts stage has a single entry: the path is
+the relative path of the file to the module root and the value the
+file artifact located at the specified location. The runfiles are
+the same as the artifacts and the provides map is empty.
+
+#### Collection of files given by a shell pattern
+
+A collection of files given by a shell pattern has, both as
+artifacts and runfiles, the (necessarily disjoint) union of the
+artifact maps of the (zero or more) source targets that match the
+pattern. Only *files* in the *top-level* directory of the given
+modules are considered for matches. The provides map is empty.
+
+#### Trees
+
+A tree describes a directory. Internally, however, it is a single
+opaque artifact. Consuming targets cannot look into the internal
+structure of that tree. Only when realized in the file system (when
+installation is requested or as part of the input to an action), the
+directory structure is visible again.
+
+An explicit tree target is similar to an explicit file target,
+except that at the specified location there has to be a directory
+rather than a file and the tree artifact corresponding to that
+directory is taken instead of a file artifact.