summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorPaul Cristian Sarbu <paul.cristian.sarbu@huawei.com>2025-06-25 14:40:10 +0200
committerPaul Cristian Sarbu <paul.cristian.sarbu@huawei.com>2025-06-25 16:44:08 +0200
commit6975ae52d7610bdd09faa4a5a3a51f689a67e7cd (patch)
tree48a94e66384835efcc6a5a690090d837f2d77efd /doc
parenta33f4193709da52f0abdd326162f87fdfab3ec8b (diff)
downloadjustbuild-6975ae52d7610bdd09faa4a5a3a51f689a67e7cd.tar.gz
future-designs: Remove entries for debug fission and just-lock
...as these features have already been implemented and appropriately documented.
Diffstat (limited to 'doc')
-rw-r--r--doc/future-designs/debug-fission.md210
-rw-r--r--doc/future-designs/just-lock.md448
2 files changed, 0 insertions, 658 deletions
diff --git a/doc/future-designs/debug-fission.md b/doc/future-designs/debug-fission.md
deleted file mode 100644
index bc86e9b4..00000000
--- a/doc/future-designs/debug-fission.md
+++ /dev/null
@@ -1,210 +0,0 @@
-Implementing debug fission in the C/C++ rules
-=============================================
-
-Motivation
-----------
-
-Building with debug symbols is needed for debugging purposes, however it
-results in artifacts that are many times larger than their release versions.
-In general, debug builds are also slower than release builds due to various
-reasons, the main ones being the disabling of certain code optimizations (in
-order to allow debuggers to properly work) and debug-only checks and
-diagnostics. Furthermore, sections of debug symbols from common dependencies can
-be replicated many times between different artifacts, but also inside single
-artifacts.
-
-In the cases where one does not produce separate release and debug versions,
-it is usual to just generate the debug artifacts, then strip the debug symbols
-from them to obtain a pseudo-release version. Even in this case, being able to
-reduce the time and space requirements for producing the debug artifacts in the
-first place is of value.
-
-Moreover, distributions usually provide debug information in different packages
-and for that purpose apply themselves debug fission or stripping techniques.
-Projects that provide separated debug information have thus an advantage in
-getting accepted by reducing the work involved in packaging them for distros.
-
-[Debug fission](https://www.tweag.io/blog/2023-11-23-debug-fission)
--------------------------------------------------------------------
-
-This approach targets specifically Linux ELF files and `gdb(1)`. This is,
-however, more than enough to cover the most used UNIX-like platforms and most
-general purpose debugging tools, which rely on `gdb(1)`.
-
-### Concept
-
-Fission, aka splitting debug information into separate files, exists in modern
-tools for a [long time](https://gcc.gnu.org/wiki/DebugFission). This method
-can be applied when one already splits builds into their constituting compile
-and link steps, which is the case with most modern build tools, including
-*justbuild*. Debug fission proposes that the compilation step of a debug build
-produce, instead of one object file, two artifacts: a `.dwo` DWARF file
-containing all the debug symbols of the compilation unit, and the (now smaller)
-`.o` object file containing now only references to the debug symbols from the
-`.dwo` file. These object files can be linked as usual to produce the final
-build artifact, and the `.dwo` files can be either retained as-is, or packed
-into a corresponding `.dwp` file.
-
-### Benefits
-
-By splitting the debug symbols of each compilation unit into separate
-artifacts, these can be cached and reused as needed, removing any previous
-debug symbols duplication across build artifacts. This has also a beneficial
-impact on the build times. Moreover, the stripped artifacts are quasi-identical
-(e.g., executables differ only in their internal Build ID).
-
-Proposal
---------
-
-Debug fission requires changes affecting the `OSS` and `rules-cc` rules, as well
-as the OSS toolchain configuration. In order to ensure that new and old
-toolchains and rules, respectively, are still able to work together, the new
-rules will only perform debug fission if the toolchain is configured to use this
-feature.
-
-The following sections describe the needed changes in detail.
-
-### Extend `["CC", "defaults"]` rule with new fields
-
-The `["CC", "defaults"]` should accept 3 new fields:
-
- - `"DWP"`, containing as a singleton list the path to the `dwp` tool to be used
- for packing DWARF files. This field should be handled the same as, e.g., the
- `"CC"` variable. The description of our toolchains should be extended with
- the `"DWP"` field, pointing to the location of the respective tool in the
- staged binaries folder.
-
- - `"ADD_DEBUGFLAGS"`, containing as a list compile flags to be appended to both
- C and C++ flags for debug builds. As with other `"ADD_<X>FLAGS"` variables,
- these extend the existing flags.
-
- - `"DEBUGFLAGS"`, containing as a list the compile flags to be used for debug
- builds, replacing existing.
-
-#### Change `"DEBUG"` configuration variable value type to map
-
-In order for the defaults to properly set the appropriate flags in debug mode,
-the configuration variable `"DEBUG"` is changed to be a mapping. As before, a
-`true` evaluation of its value (now, a not empty map) will signal debug mode.
-The following supported keys are proposed:
-
- - a `"USE_DEBUG_FISSION"` flag, which, if evaluated to `true` enables debug
- fission and otherwise signals regular debug mode, and
-
- - a `"FISSION_CONFIG"` map, which can configure in more detail how debug
- fission behaves. Existence is enforced if debug fission is enabled.
- Otherwise, this field is ignored.
-
-The `"USE_DEBUG_FISSION"` flag only enables the corresponding rules logic, but
-does not change any compile and/or link flags. In particular, instructing the
-compiler to generated any DWARF files to begin with must be done through an
-appropriate `"FISSION_CONFIG"` map.
-
-The `"FISSION_CONFIG"` map should accept the following keys:
-
- - `"USE_SPLIT_DWARF"`: If evaluated to `true`, appends the `-gsplit-dwarf`
- flag to the compile flags. This is currently the only flag that actually
- instructs the compiler to generate .dwo files and as such it should be
- provided in case debug fission is enabled.
-
- - `"DWARF_VERSION"`: Expects a string defining the DWARF format version. If
- provided, appends the `-gdwarf-<value>` flag to the compile flag.
-
- Each toolchain comes with a default in terms of which version of the DWARF
- format is used. Basically all reasonably modern toolchains (GCC >=4.8.1,
- Clang >=7.0.0 at least) and debugging tools (GDB >= 7.0) use DWARFv4 by
- default, with the more recent versions having already switched to using the
- newer, upward compatible [DWARFv5](https://dwarfstd.org/dwarf5std.html)
- format. However, the degree of implementation and default support of the
- various compilers and tools differs, so it is recommended to use version 4.
-
- - `"USE_GDB_INDEX"`: If evaluated to `true`, appends the `-Wl,--gdb-index` flag
- to the compile flags. Defaults to `false`.
-
- This option enables, in linkers that support it, an optimization which
- bundles certain debug symbols sections together into a single `.gdb_index`
- section, reducing the size of the final artifact (quite significantly for
- large artifacts) and drastically improving the debugger start time, but at
- the cost of a slower linking step.
-
- - Known supported linkers: `lld` (LLVM >=7), `gold` (binutils >=2.24*), `mold` (>=2.3)
-
- *`gold` linker additional info:
- As per the [release notes](https://lwn.net/Articles/1007541/), `binutils`
- 2.44 (2025-02-02) does **NOT** come with the `gold` linker anymore, as it
- is considered deprecated and will be removed completely in the near future
- unless new maintainers are found. Note also that Fedora, for concerns of
- bit-rot, moved the `gold` linker from its `binutils` RPM to a separate
- package already since version 31 (2019-10-29).
-
- - Known unsupported linkers: `ld` (GNU)
-
- - `"USE_DEBUG_TYPES_SECTION"`: If evaluated to `true`, appends the
- `-fdebug-types-section` flag to the compile flags. Defaults to `false`.
-
- This option enables, for toolchains supporting at least DWARFv4, an
- optimization that produces separate debug symbols sections for certain large
- data types, thus providing the linker the opportunity to deduplicate more
- debug information, resulting in smaller artifacts.
-
- More performant approaches to reduce the size of the debug information exist,
- but are not as straight-forward to implement as enabling a flag. For example,
- Fedora opted instead to use the `dwz` compression tool, another known
- approach, and make it its default for handling debug RPMs already since
- version 18 (2013-01-15).
-
-The `["CC", "defaults"]` rule interrogates these fields in order to set the
-appropriate debug flags to be provided to the library/binary rules. Note
-that it is the user's responsibility to configure the debug mode accordingly.
-It is always up to each toolchain how unsupported or unexpected combinations of
-flags are being handled.
-
-### Changes to `"library"` and `"binary"` rules
-
-The toolchain flags will be treated as before, with the addition that in debug
-mode, if the final compile flags list is empty, `["-g"]` will be used by
-default. To this resulting flags list any debug fission flags configured via the
-`"FISSION_CONFIG"` map will be added if debug fission is enabled.
-
-The `"USE_DEBUG_FISSION"` flag of `"DEBUG"` will inform these rules on whether
-the debug fission logic should be used or not. In this way, only the combination
-of an appropriate configuration and these updated rules will be able to perform
-debug fission, while all other combinations of toolchains and rules will perform
-as before.
-
-All consumers of the internal `"objects"` expression (i.e., static/dynamic
-libraries and binaries) should provide a new field `"dwarf-pkg"`, defaulting to
-the empty map. If debug fission is enabled, this field will contain the
-corresponding DWARF package file, constructed via a new expression
-`"dwarf package"`, which, based on given `"dwarf objects"`, `"dwarf deps"` and a
-`stage`, uses the given DWARF objects and DWARF package files (provided by the
-`"dwarf deps"`) as inputs to an `ACTION` generating a resulting DWARF package
-file, appropriately staged. Each such consumer will pass the appropriate inputs
-to the new `"dwarf package"` expression considering that `.dwo` files need to be
-gathered the same as `.o` files and `.dwp` files need to be gathered the same as
-libraries/binaries. In particular, the `"dwarf deps"` shall contain only the
-DWARF package files of any link dependencies that are considered for generating
-the regular artifact (library/binary).
-
-As an auxiliary required change, the output of the compile `ACTION` in the
-`"objects"` expression should be extended, if debug fission is enabled, by an
-additional path corresponding to the expected `.dwo` DWARF file, staged next to
-the usual `.o` file. This path needs to be passed to any consumers of
-`"objects"`. For this purpose, the expression should be refactored to return a
-map result instead of a single variable.
-
-### Extend the `"install-with-deps"` rule to stage DWARF files
-
-The `"install-with-deps"` rule should stage also any `"debug-pkg"` entries from
-providers when in debug mode, in the same locations as it does regular
-artifacts. As paths are handled by each library/binary accordingly, the DWARF
-package files should always end up next to their corresponding build artifact,
-i.e., where `gdb(1)` expects them.
-
-### Bootstrappable toolchain to expose the `dwp` binary
-
-The bootstrappable toolchain repository provides several toolchains built from
-source. In the case of the `gcc` and `clang` compilers, a `dwp` tool should be
-part of the produced staged binaries. This can be advertised to consumers of
-these toolchains (compilers, compilers+tools) in their `["CC", "defaults"]` via
-the newly introduced `"DWP"` field.
diff --git a/doc/future-designs/just-lock.md b/doc/future-designs/just-lock.md
deleted file mode 100644
index bb347005..00000000
--- a/doc/future-designs/just-lock.md
+++ /dev/null
@@ -1,448 +0,0 @@
-Just-lock
-=========
-
-Status quo
-----------
-
-The canonical method of performing a multi-repository build using our build tool
-is by running `just-mr` on a configuration file which has a well-defined format
-and contains the description of all the repositories that should be considered
-for the build. This file in many ways acts as a *lock file* of the project,
-maintaining a snapshot in time of the repositories configuration. As with any
-project lock file, this configuration file should be kept under version control
-together with the source code, ensuring all project users can rely on the same
-dependencies.
-
-Dependencies for multi-repository *justbuild* projects require suitable
-repository descriptions. For direct dependencies using *justbuild* one can, and
-we argue should, rely on their committed configuration file. When such a
-dependency is under Git version control, **`just-import-git`**(1) can take care
-of the issue of repository composition, importing a given repository together
-with its transitive dependencies from the target configuration. Multiple
-dependencies are currently treated by multiple import tool calls, with the
-common approach being to pipe the output of each call into the next, thanks to
-a design that expects input at `stdin` and output to `stdout`. Typically,
-deduplication of common transitive dependencies, which can otherwise lead to
-configuration bloat, is handled by a final pipe into
-**`just-deduplicate-repos`**(1).
-
-While Git is a very popular version control system, it is by no means the only
-option, alternatives such as CVS, Subversion, or Mercurial having also large
-audiences. In such cases, a current available solution is to make the desired
-content available locally and use `just add-to-cas` to import the content to
-local CAS as a Git tree, for which then a corresponding repository description
-can be written.
-
-Some languages have their own methods of tracking and managing project
-dependencies, which can and should be leveraged. For example, in the case of
-*Rust*, **`just-import-cargo`**(1) uses *Cargo* to retrieve the dependencies of
-a given Rust crate and generate the appropriate repository descriptions.
-
-Shortcomings
-------------
-
-### Tooling limitations
-
-The existing tools mentioned above are good at tackling their respective
-use-cases, however they do come also with some limitations, the main one being
-that they work only on individual repositories. This means that for projects
-containing several *justbuild* dependencies (especially as our tool gains more
-popularity) one needs to perform multiple imports. Complexity and loss of
-readability increases quite fast with the current approach of chaining many
-calls together and desirable features, like granular error reporting, require
-more layers of scripting. Additionally, the status quo of addressing
-side-effects of one tool by another (such as the already mentioned repository
-deduplication case) can be improved by a unified framework.
-
-### Fragmentation
-
-The generation and maintenance of the configuration file of multi-repository
-*justbuild* projects currently takes place in different manners, ranging from
-direct manual editing to various user-made scripts wrapping calls to existing
-tools. This has the main benefit that users have the highest liberty in deciding
-which tools to use in handling each particular use-case when setting up or
-updating the configuration file. On the other hand, choosing the best tool or
-approach might not always be straight-forward, especially for newer users.
-
-While currently the number of tools available to tackle various specific
-scenarios is small, it is clear that with the continued development of
-*justbuild* more use-cases will arise. Some of these are already known (such as
-imports of non-Git sources or imports from local checkouts), some yet to be
-discovered (for example, those that may come with the introduction of rules
-support for more languages that have their own ecosystem). An approach to assist
-future users can be, of course, to increase the current roster of tools. This
-can however lead to a fragmented ecosystem around *justbuild* that handles to a
-not inconsiderate extent the same task. On the other hand, being reticent in
-introducing tooling options can be detrimental in attracting new users. As in
-many cases, a balanced approach might offer the best of both worlds.
-
-Other use-cases related to repository configuration are also not addressed by
-current tooling. For example, two or more repositories might be logically
-coupled, which, while not recommended, is a situation found in larger projects
-for which a deep refactoring to decouple them can be prohibitively expensive.
-In such a case, local development has to take place in more than one repository.
-Creating local clones aware of the dependency closure of a repository
-(completely defined in the configuration file) can be thus a useful feature.
-
-Proposal: `Just-lock`
----------------------
-
-We propose `just-lock` as a framework for **generating** and **maintaining**
-the multi-repository configuration file of a *justbuild* project. This addresses
-the mentioned shortcomings by providing a common interface for current import
-functionalities, while allowing new tools to be freely used under very lax
-conditions. This is achieved by means of a set of built-in import options,
-describable in a well-defined format.
-
-The framework revolves around the `just-lock` tool, which will implement all the
-required functionality.
-
-### The tool
-
-Usage: generate/update a `just-mr` configuration file
-
-```
-just-lock [-C <repos.in.json>] [-o <repos.json>]
- [--local-build-root <PATH>] [--just <PATH>]
- [--git <PATH>] [-L|--launcher <JSON>]
- [--clone <JSON>]
-
-OPTIONS:
- -C PATH Input file. If missing, searched for in ['repos.in.json', 'etc/repos.in.json']
- with respect to the workspace root.
- -o PATH Output file. If missing, searched for in ['repos.json', 'etc/repos.json'] with respect to
- the workspace root. If none found, placed as 'repos.json' in parent path of input file.
- --local-build-root PATH
- Local build root. Usual `just-mr` rules apply.
- --just PATH Path to a `just` binary. If missing, use the `just` name from PATH.
- --git PATH Git binary to use if needed. If missing, system `git` is used.
- User must pass it also to `just-mr`.
- --launcher JSON Local launcher to use for commands. Given as a JSON list of strings.
- If missing, ["env", "--"] is used. User must pass it also to `just-mr`.
- --clone JSON Mapping from filesystem path to pair of repository name and a list of bindings.
- For each map entry, the target repository, found by following the bindings from a given start
- repository, after all repositories have been imported, will have its workspace root cloned in
- the specified filesystem directory and its description in the output configuration made to
- point to this clone.
- The start repository names must be known, i.e., an initial repository or declared import,
- and both the start and target repositories will be kept during deduplication.
-```
-
-- Notes:
-
- The proposed default naming choice for the input file is chosen to mirror the
- default names of the configuration file of `just-mr`.
-
- The proposed search locations for the output configuration file, if an
- explicit path is not provided, are the same ones used by `just-mr` when run
- with the `--norc` option, but limited to the ones relative to the workspace
- root. This is done to better match the desired lock-file quality of the
- output file and also ensure `just-mr` can pick it up by default.
-
- The specification for finding the target repository for the `--clone` option
- uses the fact that the names of existing repositories and declared imports
- are the only ones known to remain as such in the output configuration, prior
- to deduplication, with all other repositories reachable from one such
- repository via a defined sequence of bindings. The clone locations are
- disjoint (as they are map keys), can be specified both absolute or relative
- to the current directory, and the referred to directory will be created if
- missing.
-
- The `--clone` option will produce an output configuration file meant for
- local development only. Therefore, it is not recommended for such a
- configuration file to be committed.
-
-### Input file format
-
-The input file describes which repositories will be part of the resulting
-configuration file. The file is a `JSON` object. The proposed structure is:
-
-``` jsonc
-{ "main": "<name>"
-, "repositories": {...}
-, "imports": [...]
-, "keep": [...]
-}
-```
-
-The input file is expected to contain at most the mentioned 4 fields: `"main"`,
-`"repositories"`, `"imports"`, and `"keep"`. Any other fields will be ignored.
-
-The `"main"` and `"repositories"` fields maintain their meaning from the
-usual `just-mr` configuration format (**`just-mr-repository-config`**(5)).
-Therefore, neither fields are mandatory and missing fields are treated
-consistently with `just-mr`. In this way, a `just-lock` input file containing
-at most the `"main"` and `"repositories"` fields is a valid `just-mr`
-configuration file. This subset of the input configuration object is referred
-to in the following as the _core_ configuration.
-
-For the input file of `just-lock` the `just-mr` format is simply extended with
-**two** new fields. The value of the `"imports"` field is a list of `JSON`
-objects describing a ***source***. Each _source_ provides information (in the
-form of a well-defined set of fields) about how the _core_ configuration will
-be extended. In most cases, this takes place by importing one or more
-repositories from a specified existing `just-mr` configuration file, each with
-their transitive dependencies, but more general options are available (as will
-be described below). The imports are processed individually and consecutively
-in the well-defined order declared in the input configuration file, meaning
-that each import is extending the configuration obtained after processing all
-preceding imports.
-
-The format imposes a well-defined ordering of imports in order to maintain the
-_naming convention_ already implicitly implemented by the existing tools. This
-states that each import will add to the _core_ configuration only the following
-repository names:
-- the _name_ of the specified repository, which is well-defined by the input
- file format, and
-- names starting with that _name_ followed by `"/"`, corresponding to the
- transitive dependencies of the specified repository.
-
-This naming convention allows open names to be filled later in the import
-sequence by repository names specified in the input file without the fear that
-have been taken up during an earlier import.
-
-If the `"main"` field is provided in the input file, it must match one of the
-repository aliases marked for import or the name of one of the repositories
-given by the `"repositories"` field.
-
-The value of the `"keep"` field is a list of strings stating which repositories,
-besides the one specified by `"main"` and those specified by option `--clone`,
-are to be kept during the final _deduplication_ step, which takes place after
-all imports have been processed. This way, `just-lock` will include all the
-functionality `just-deduplicate-repos` provides. The output configuration file
-of `just-lock` will always have deduplicated entries.
-
-#### Proposed source types
-
-The type of a _source_ is defined by the string value of the mandatory subfield
-`"source"`.
-
-- **git**
-
- This source type encompasses the functionality of `just-import-git`.
-
- We argue that most *justbuild* projects will contain one main configuration
- file, describing one or more repositories. This is why we propose a format
- that allows importing multiple repositories from the same source configuration
- file. Each declared repository is imported independently and consecutively,
- in the well-defined order provided by the user.
-
- If the `"commit"` field is missing, the `HEAD` commit of the specified remote
- branch will be considered. This will have an effect also on the fixed commit
- that will be used in the resulting repository description corresponding to any
- imported `"file"`-type repositories (see `just-import-git`).
-
- If `"as plain": true`, any provided `"special"` key for the `"pragma"` field
- in the source description is unconditionally set in the imported repositories,
- superseding any other config- or import-level treatment of pragmas during the
- import. Note that `"as plain": true` results in only one repository
- (containing the whole source repository tree) being imported.
-
- Proposed format:
- ``` jsonc
- { "source": "git"
- // "source"-specific fields
- // defines which repositories to import from source repository
- , "repos": // mandatory; list of repositories to be imported
- [ { "alias": "<name>" // corresponds to `import_as` var (option --as);
- // mandatory if "repo" value missing, otherwise value of "repo" taken if missing
- , "repo": "<foreign_name>" // optional; corresponds to `foreign_repository_name` var
- , "map": {"from_name": "to_name"} // optional; corresponds to `import_map` var (option --map)
- , "pragma": // optional
- {"absent": true} // corresponds to `absent` var (option --absent)
- }
- , ...
- ]
- // fields related to obtaining source config
- , "url": "https://nonexistent.example.com/repo.git" // mandatory
- , "mirrors": ["https://nonexistent-mirror.example.com/repo.git"] // optional
- , "branch": "master" // mandatory (as we have no sane default value between "master" and "main");
- // corresponds to `branch` var (option -b)
- , "commit": "<HASH>" // optional; if missing, take HEAD commit of branch
- , "inherit env": [...] // optional; corresponds to `inherit_env` var (option --inherit-env)
- , "config": "<foreign_repos.json>" // optional; corresponds to `foreign_repository_config` var (option -R)
- , "as plain": false // optional; corresponds to `plain` var (option --plain)
- , "pragma": {"special": "<value>"} // optional; only considered if `"as plain": true`
- }
- ```
-
-- **file**
-
- This _source_ type behaves similarly to **git**, with the main difference
- being that the referenced source repository is not a Git remote, but a local
- checkout.
-
- The checkout is assumed to be maintained, so that `"file"`-type repositories
- marked to be imported can retain their type. For such transitive dependencies,
- one can also set the `"to_git": true` pragma with a corresponding entry in the
- usual `"pragma"` field.
-
- If `"as plain": true`, any provided `"special"` key for the `"pragma"` field
- in the source description is unconditionally set in the imported repositories,
- superseding any other config- or import-level treatment of pragmas during the
- import. Note that `"as plain": true` results in only one repository
- (containing the whole source repository tree) being imported.
-
- Proposed format:
- ``` jsonc
- { "source": "file"
- // "source"-specific fields
- // defines which repositories to import from source repository
- , "repos": // mandatory; list of repositories to be imported
- [ { "alias": "<name>" // corresponds to `import_as` var (option --as);
- // mandatory if "repo" value missing, otherwise value of "repo" taken if missing
- , "repo": "<foreign_name>" // optional; corresponds to `foreign_repository_name` var
- , "map": {"from_name": "to_name"} // optional; corresponds to `import_map` var (option --map)
- , "pragma": // optional
- { "absent": true // corresponds to `absent` var (option --absent)
- , "to_git": true // any imported "file"-repositories will also be "to_git":true
- }
- }
- , ...
- ]
- // fields related to obtaining source config
- , "path": "<source/repo/path>" // mandatory
- , "config": "<foreign_repos.json>" // optional; corresponds to `foreign_repository_config` var (option -R)
- , "as plain": false // optional; corresponds to `plain` var (option --plain)
- , "pragma": {"special": "<value>"} // optional; only considered if `"as plain": true`
- }
- ```
-
-- **archive**
-
- This _source_ type behaves similarly to **git**, with the main difference
- being that the referenced source repository is not a Git remote, but an
- archive, such as a release tarball.
-
- A field `"subdir"` is provided to account for the fact that source repository
- root often is not the root directory of the unpacked archive.
-
- If `"as plain": true`, any provided `"special"` key for the `"pragma"` field
- in the source description is unconditionally set in the imported repositories,
- superseding any other config- or import-level treatment of pragmas during the
- import. Note that `"as plain": true` results in only one repository
- (containing the whole source repository tree) being imported.
-
- Proposed format:
- ``` jsonc
- { "source": "archive"
- // "source"-specific fields
- // defines which repositories to import from source repository
- , "repos": // mandatory; list of repositories to be imported
- [ { "alias": "<name>" // corresponds to `import_as` var (option --as);
- // mandatory if "repo" value missing, otherwise value of "repo" taken if missing
- , "repo": "<foreign_name>" // optional; corresponds to `foreign_repository_name` var
- , "map": {"<from_name>": "<to_name>"} // optional; corresponds to `import_map` var (option --map)
- , "pragma": // optional
- {"absent": true} // corresponds to `absent` var (option --absent)
- }
- , ...
- ]
- // fields related to obtaining source config
- , "fetch": "<URL>" // mandatory
- , "type": "tar|zip" // optional; type of archive in set ["tar", "zip"]; if missing, default to "tar"
- , "mirrors": ["..."] // optional
- , "subdir": "<REL_PATH>" // optional; relative path defining the actual root of the source repository;
- // if missing, the source root is the root directory of the unpacked archive
- , "content": "<HASH>" // optional; if missing, always fetch; if given, will be checked
- , "sha256": "<HASH>" // optional checksum; if given, will be checked
- , "sha512": "<HASH>" // optional checksum; if given, will be checked
- , "config": "<foreign_repos.json>" // optional; corresponds to `foreign_repository_config` var (option -R)
- , "as plain": false // optional; corresponds to `plain` var (option --plain)
- , "pragma": {"special": "<value>"} // optional; only considered if `"as plain": true`
- }
- ```
-
-- **git tree**
-
- This _source_ type proposed to be the canonical way of importing *justbuild*
- dependencies under version control systems other than Git.
-
- The command that produces the tree is either given explicitly (field `"cmd"`)
- or indirectly by a command-generating command (field `"cmd gen"`). The tool
- will run the so-given command to produce the content in a temporary directory,
- it will import the given subdirectory to Git, and it will generate a
- corresponding `"git tree"`-type repository description to be added to the
- configuration.
-
- The fields `"cmd"`, `"env"`, `"inherit env"` have the same meaning as those
- of the `"git tree"`-type repository (as per `just-mr-repository-config`).
-
- **IMPORTANT:** The user has to be the one to ensure that the environment in
- which `just-lock` is run matches the one intended for running `just-mr` with
- respect to all the provided envariables in the `"inherit env"` list. This is
- because `just-lock` and `just-mr` must produce the same tree when running the
- same command.
-
- NOTE: While the target configuration file has to be part of the specified
- `"subdir"` tree, referenced `"file"`-type repositories marked to be imported
- can point also outside of the `"subdir"`, as long as they are still contained
- in the initial checkout (i.e., the directory generated by the command). All
- such repositories will be translated to appropriate `"git tree"`-type
- repositories in the output configuration.
-
- If `"as plain": true`, any provided `"special"` key for the `"pragma"` field
- in the source description is unconditionally set in the imported repositories,
- superseding any other config- or import-level treatment of pragmas during the
- import. Note that `"as plain": true` results in only one repository
- (containing the whole source repository tree) being imported.
-
- Proposed format:
- ``` jsonc
- { "source": "git tree"
- // "source"-specific fields
- , "repos": // mandatory; list of entries describing repositories to import
- [ { "alias": "<name>" // mandatory; same meaning as `import_as` var
- // mandatory if "repo" value missing, otherwise value of "repo" taken if missing
- , "repo": "<foreign_name>" // optional; same meaning as `foreign_repo_name` var
- , "map": {"<from_name>": "<to_name>"} // optional; corresponds to `import_map` var (option --map)
- , "pragma": // optional
- {"absent": true} // same meaning as `absent` var
- }
- ]
- , "cmd": [...] // one and only one of {"cmd", "cmd_gen"} must be provided;
- // command as list of strings
- , "subdir": "<subdir>" // optional; default is "."; subdir to consider as main entry point
- , "env": {...} // optional; map of envariables needed by "cmd"
- , "inherit env": [...] // optional; list of envariables to inherit
- , "cmd gen": [...] // one and only one of {"cmd", "cmd_gen"} must be provided;
- // command producing the "cmd" value to use, as list of strings
- , "config": "<foreign_repos.json>" // optional; corresponds to `foreign_repository_config` var (option -R)
- // searched for in the "subdir" tree
- , "as plain": false // optional; corresponds to `plain` var (option --plain)
- , "pragma": {"special": "<value>"} // optional; only considered if `"as plain": true`
- }
- ```
-
-- **generic**
-
- This _source_ type is proposed to be the canonical way for users to provide
- their own command which can update a `just-mr` configuration.
-
- The command must accept a `just-mr` configuration as input from `stdin` and
- must output a `just-mr` configuration to `stdout`. The command is run in a
- given subpath of the current directory (by default `"."`) and as such can have
- side-effects on the filesystem.
-
- The input fed to the command is the _current_ configuration, i.e., the
- configuration obtained after processing all preceding imports (according to
- the well-defined order declared in the input configuration file). The output
- configuration is used as input for the succeeding import.
-
- The user must take care to correctly construct the `"imports"` list in order
- to process **generic** entries at the desired time. For example, if a
- **generic** entry needs to be process between the import of two repositories
- from the same **git** source, the user must split that **git** source into two
- corresponding **git** entries and place the **generic** entry between them.
-
- Proposed format:
- ``` jsonc
- { "source": "generic"
- // "source"-specific fields
- , "cwd": "<path>" // optional; relative path to run the script in;
- // if missing, defaults to "."
- , "cmd": [...] // mandatory; command to run, as list of strings
- , "env": {...} // optional; map of envariables needed by script
- , "inherit env": [...] // optional; list of envariables to inherit
- }
- ```