diff options
| -rw-r--r-- | doc/concepts/multi-repo.org | 167 |
1 files changed, 167 insertions, 0 deletions
diff --git a/doc/concepts/multi-repo.org b/doc/concepts/multi-repo.org new file mode 100644 index 00000000..54ef6fb0 --- /dev/null +++ b/doc/concepts/multi-repo.org @@ -0,0 +1,167 @@ +* Multi-repository build + +** Repository configuration + +*** Open repository names + +A repository can have external dependencies. This is realized by +having unbound ("open") repository names being used as references. +The actual definition of those external repositories is not part +of the repository; we think of them as inputs, i.e., we think of +this repository as a function of the referenced external targets. + +*** Binding in a separate repository configuration + +The actual binding of the free repository names is specified in a +separte repository-configuration file, which is specified on the +command line (via the ~-C~ option); this command-line argument +is optional and the default is that the repository worked on has +no external dependencies. Typically (but not necessarily), this +repository-configuration file is located outside the referenced +repositories and versioned separately or generated from such a +file via ~bin/just-mr.py~. It serves as meta-data for a group of +repositories belonging together. + +This file contains one JSON object. For the key ~"repositories"~ the +value is an object; its keys are the global names of the specified +repositories. For each repository, there is an object describing it. +The key ~"workspace_root"~ describes where to find the repository and +should be present for all (direct or indirect) external dependencies +of the repository worked upon. Additional roots file names (for +target, rule, and expression) can be specified. For keys not given, +the same rules for default values apply as for the corresponding +command-line arguments. Additionally, for each repository, the +key "bindings" specifies the map of the open respository names to +the global names that provide these dependencies. Repositories may +depend on each other (or even themselves), but the resulting global +target graph has to be cycle free. + +Whenever a location has to be specified, the value has to be a +list, with the first entry being specifying the naming scheme; the +semantics of the remaning entries depends on the scheme (see "Root +Naming Schemes" below). + +Additionally, the key ~"main"~ (with default ~""~) specifies +the main repository. The target to be built (as specified on the +command line) is taken from this repository. Also, the command-line +arguments ~-w~, ~--target_root~, etc, apply to this repository. If +no option ~-w~ is given and ~"workspace_root"~ is not specified in +the repository-configuration file either, the root is determined +from the working directory as usual. + +The value of ~main~ can be overwritten on the command line (with +the ~--main~ option) In this way, a consistent configuration +of interdependent repositories can be versioned and referred to +regardless of the repository worked on. + +**** Root naming scheme + +***** ~"file"~ + +The ~"file"~ scheme tells that the repository (or respective root) +can be found in a directory in the local file system; the only +argument is the absolute path to that directory. + + +***** ~"git tree"~ + +The ~"git tree"~ scheme tells that the root is defined to be a tree +given by a git tree identifier. It takes two arguments +- the tree identifier, as hex-encoded string, and +- the absolute path to some repository containing that tree + +**** Example + +Consider, for example, the following repository-configuration file. +In the following, we assume it is located at ~/etc/just/repos.json~. + +#+BEGIN_SRC +{ "main": "env" +, "repositories": + { "foobar": + { "workspace_root": ["file", "/opt/foobar/repo"] + , "rule_root": ["file", "/etc/just/rules"] + , "bindings": {"base": "barimpl"} + } + , "barimpl": + { "workspace_root": ["file", "/opt/barimpl"] + , "target_file_name": "TARGETS.bar" + } + , "env": {"bindings": {"foo": "foobar", "bar": "barimpl"}} + } +} +#+END_SRC + +It specifes 3 repositories, with global names ~foobar~, ~barimpl~, +and ~env~. Within ~foobar~, the repository name ~base~ refers to +~barimpl~, the repository that can be found at ~/opt/barimpl~. + +The repository ~env~ is the main repository and there is no workspace +root defined for it, so it only provides bindings for external +repositories ~foo~ and ~bar~, but the actual repository is taken +from the working directory (unless ~-w~ is specified). In this way, +it provides an environment for developping applications based on +~foo~ and ~bar~. + +For example, the invocation ~just build -C /etc/just/repos.conf +baz~ tells our tool to build the target ~baz~ from the module the +working directory is located in. ~foo~ will refer to the repository +found at ~/opt/foobar/repo~ (using rules from ~/etc/just/rules~, +taking ~base~ refer to the the repository at ~/opt/barimpl~) and +~bar~ will refer to the repository at ~/opts/barimpl~. + +** Naming of targets + +*** Reference in target files + +In addition to the normal target references (string for a target in +the name module, moudle-target pair for a target in same repository, +~["./", relpath, target]~ relative addressing, ~["FILE", null, +name]~ explicit file refence in the same moudle), references of the +form ~["@", repo, module, target]~ can be specified, where ~repo~ +is string referring to an open name. That open repository name is +resolved to the global name by the ~"bindings"~ parameter of the +repository the target reference is made in. Within the repository +the resolved name refers to, the target ~[module, target]~ is taken. + +*** Expression language: names as abstract values + +Targets are a global concept as they distinguish targets from different +repositories. Their names, however, depend on the repository they +occur in (as the local names might differ in various repositories). +Moreover, some targets cannot be named in certain repositories as +not every repository has a local name in every other repository. + +To handle this naming problem, we note the following. During the +evaluation of a target names occur at two places: as the result of +evaluating the parameters (for target fields) and in the evaluation +of the defining expression when requesting properties of a target +dependent upon (via ~DEP_ARTIFACTS~ and related functions). In the +later case, however, the only legitimate way to obtain a target +name is by the the ~FIELD~ function. To enforce this behavior, and +to avoid problems with serializing target names, our expression +language considers target names as opaque values. More precisely, +- in a target description, the target fields are evaluated and the + result of the evaluation is parsed, in the context of the module + the ~TARGET~ file belongs to, as a target name, and +- during evaluation of the defining expression of a the target's + rule, when accessing ~FIELD~ the values of target fields will + be reported as abstract name values and when querying values of + dependencies (via ~DEP_ARTIFACTS~ etc) the correct abstract target + name has to be provided. + +While the defining expression has access to target names (via +target fields), it is not useful to provide them in provided data; +a consuming data cannot use names unless it has those fields as +dependency anyway. Our tool will not enforce this policy; however, +only targets not having names in their provided data are eligible +to be used in ~export~ rules. + +** File layout in actions + +As ~just~ does full staging for actions, no special considerations +are needed when combining targets of different repositories. Each +target brings its staging of artifacts as usual. In particular, no +repository names (neither local nor global ones) will ever be visible +in any action. So for the consuming target it makes no difference +if its dependency comes from the same or a different repository. |