summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorKlaus Aehlig <klaus.aehlig@huawei.com>2025-01-23 10:17:08 +0100
committerKlaus Aehlig <klaus.aehlig@huawei.com>2025-01-23 11:07:27 +0100
commitbbfd7d286ee5ed4e3caceb20d9debb0f971adb19 (patch)
tree252d64357d5a52a4e880e36a622937b8ed0099c7
parentfc9f622cba6b6671f5b5f0371de9bf31ae75c7d1 (diff)
downloadjustbuild-bbfd7d286ee5ed4e3caceb20d9debb0f971adb19.tar.gz
Document computed roots as implemented concept
... rather than as future design. While there, also add target-level caching as a service to the list of documentation pages.
-rw-r--r--README.md2
-rw-r--r--doc/concepts/computed-roots.md (renamed from doc/future-designs/computed-roots.md)101
2 files changed, 29 insertions, 74 deletions
diff --git a/README.md b/README.md
index f444864f..5315524b 100644
--- a/README.md
+++ b/README.md
@@ -40,6 +40,8 @@ taken from user-defined rules described by functional expressions.
- [Cache Pragma and Testing](doc/concepts/cache-pragma.md)
- [Anonymous Targets](doc/concepts/anonymous-targets.md)
- [Target-Level Caching](doc/concepts/target-cache.md)
+- [Target-Level Caching as a Service](doc/concepts/service-target-cache.md)
- [Garbage Collection](doc/concepts/garbage.md)
- [Symbolic links](doc/concepts/symlinks.md)
- [Execution properties](doc/concepts/execution-properties.md)
+- [Computed roots](doc/concepts/computed-roots.md)
diff --git a/doc/future-designs/computed-roots.md b/doc/concepts/computed-roots.md
index 1f0a765f..171c7648 100644
--- a/doc/future-designs/computed-roots.md
+++ b/doc/concepts/computed-roots.md
@@ -1,44 +1,8 @@
Computed roots
==============
-Status quo
-----------
-
-As of version `1.0.0`, the `just` build tool requires the repository
-configuration, including all roots, to be specified ahead of time. This
-has a couple of consequences.
-
-### Flexible source views, thanks to staging
-
-For source files, the flexibility of using them in a layout different
-from how they occur in the source tree is gained through staging. If a
-different view of sources is needed, instead of a source target, a
-defined target can be used that rearranges the sources as desired. In
-this way, also programmatic transformations of source files can be
-carried out (while the result is still visible at the original
-location), as is done, e.g., by the `["patch", "file"]` rule of the
-`just` main repository.
-
-### Restricted flexibility in target-definitions via globbing
-
-When defining targets, the general principle is that the definition of
-target and action graph only depends on the description (given by the
-target files, the rules and expressions, and the configuration). There
-is, however, a single exception to that rule: a target file may use the
-`GLOB` built-in construct and in this way depend on the index of the
-respective source directory. This allows, e.g., to define a separate
-action for every source file and, in this way, get good incrementality
-and parallelism, while still having a concise target description.
-
-### Modularity in rules through expressions
-
-Rules might share common tasks. For example, for both `C` binaries and
-`C` libraries, the source files have to be compiled to object files. To
-avoid duplication of descriptions, expressions can be called (also from
-expressions themselves).
-
-Use cases that require more flexibility
----------------------------------------
+Use cases for computed build descriptions
+-----------------------------------------
### Generated target files
@@ -77,28 +41,28 @@ Moreover, using JSON encoding of abstract syntax trees is an
unambiguously readable and easy to automatically process format, but
people argue that it is hard to write by hand. However, it is unlikely
to get agreement on which syntax is best to use. Now, if rule and
-expression files could be generated, this argument would not be
+expression files can be generated, this argument is not
necessary. Moreover, rules are typically versioned and infrequently
changed, so the step of generating the official syntax from the
convenient one would typically be in cache.
-Proposal: Support computed roots
---------------------------------
+Root types depending on computation
+-----------------------------------
-We propose computed roots as a clean principle to add the needed (and a
+There are two additional types of roots that are defined through
+computation. They allow a clean principle to add the needed (and a
lot more) flexibility for the described use cases, while ensuring that
all computations of roots are properly cacheable at high level. In this
way, we do not compromise efficient builds, as the price of the
-additional flexibility, in the typical case, is just a single cache
+additional flexibility; in the typical case, is just a single cache
lookup. Of course, it is up to the user to ensure that this case really
is the typical one, in the same way as it is their responsibility to
describe the targets in a way to have proper incrementality.
-### New root type `"computed"`
+### Root type `"computed"`
-The `just` multi-repository configuration will allow a new type of root
-(besides `"file"` and `"git tree"` and variants thereof), called
-`"computed"`. A `"computed"` root is given by
+The `just` multi-repository configuration allows a type of root,
+called `"computed"`. A `"computed"` root is given by
- the (global) name of a repository
- the name of a target (in `["module", "target"]` format), and
@@ -113,18 +77,18 @@ such that for each computed root, the referenced repository as well as
all repositories reachable from that one via the `"bindings"` map only
contain computed roots earlier in that order.
-### New root type `"tree structure"`
+### Root type `"tree structure"`
In the described use case of generated target files, the tree of
target files only depends on the structure of the workspace root. To
-avoid unnecessary actions, an additional new root type is defined,
+avoid unnecessary actions, an additional root type is defined,
that of a `"tree structure"`. Such a root is given by precisely
one root. It evaluates to that root but with all files replaced
by empty files. Obviously, this computation can be done without
spawning actions and is cachable.
-The serve functionality is extended to also answer queries for the
-tree structure of a given tree.
+The serve functionality also allows to answer queries for the
+tree structure of a given tree known to serve.
### Strict evaluation of roots as artifact tree
@@ -158,23 +122,15 @@ target, the provided serve endpoint (if any) is consulted first.
Only if it is not aware of the root, a local evaluation is carried
out. This strategy is also applied for tree-stucture roots.
-### Computed roots available to the user
-
-As computed roots are defined by export targets, the respective
-artifacts are stored in the local CAS anyway. Additionally, the tree
-that forms the root will be added to CAS as well. Moreover, an option
-will be added to specify a log file that contains, in machine-readable
-way, all the tree identifiers of all computed roots used in this build,
-together with their definition.
-
-### `just-mr` to support computed roots
+### `just-mr` support for computed roots
-To allow simply setting up a `just` configuration using computed roots,
-`just-mr` will allow a repository type `"computed"` with the same
-parameters as a computed root. These repositories can be used as roots,
-like any other `just-mr` repository type. When generating the `just`
-multi-repository configuration, the definition of a `"computed"`
-repository is just forwarded as computed root.
+To allow simply setting up a `just` configuration using computed
+roots, `just-mr` allows a repository type `"computed"` with the same
+parameters as a computed root, as well as a repository type `"tree
+structure"` with another root as parameter. These repositories can
+be used as roots, like any other `just-mr` repository type. When
+generating the `just` multi-repository configuration, the definition
+of a `"computed"` repository is just forwarded as computed root.
### Computed roots and `just serve`
@@ -212,16 +168,13 @@ is `true`. E.g., `["computed", "base", "", "", {}]` is a concrete
computed root and `["computed", "base", "", "", {}, {"absent":
true}]` is the same computed root considered absent.
-### Evaluation of computed roots in connection with absent roots
+#### Evaluation of computed roots in connection with absent roots
-If a computed root is absent then, regardless of whether the base
-repository is absent or not,
+If a computed root is absent then, in native mode, regardless of whether
+the base repository is absent or not,
- serve will be asked for the result, and
- from the result the tree identifier of the root will be computed
- in memory and the root set to that value, as absent; when building
- in compatible mode, the necessary rehashing might have to fetch
- certain artifacts, but this is accepted, as the main intended
- use case is a native build.
+ in memory and the root set to that value, as absent.
If a concrete computed root refers to a base repository with absent
target root,