diff options
author | Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com> | 2025-06-25 14:40:10 +0200 |
---|---|---|
committer | Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com> | 2025-06-25 16:44:08 +0200 |
commit | 6975ae52d7610bdd09faa4a5a3a51f689a67e7cd (patch) | |
tree | 48a94e66384835efcc6a5a690090d837f2d77efd | |
parent | a33f4193709da52f0abdd326162f87fdfab3ec8b (diff) | |
download | justbuild-6975ae52d7610bdd09faa4a5a3a51f689a67e7cd.tar.gz |
future-designs: Remove entries for debug fission and just-lock
...as these features have already been implemented and
appropriately documented.
-rw-r--r-- | doc/future-designs/debug-fission.md | 210 | ||||
-rw-r--r-- | doc/future-designs/just-lock.md | 448 |
2 files changed, 0 insertions, 658 deletions
diff --git a/doc/future-designs/debug-fission.md b/doc/future-designs/debug-fission.md deleted file mode 100644 index bc86e9b4..00000000 --- a/doc/future-designs/debug-fission.md +++ /dev/null @@ -1,210 +0,0 @@ -Implementing debug fission in the C/C++ rules -============================================= - -Motivation ----------- - -Building with debug symbols is needed for debugging purposes, however it -results in artifacts that are many times larger than their release versions. -In general, debug builds are also slower than release builds due to various -reasons, the main ones being the disabling of certain code optimizations (in -order to allow debuggers to properly work) and debug-only checks and -diagnostics. Furthermore, sections of debug symbols from common dependencies can -be replicated many times between different artifacts, but also inside single -artifacts. - -In the cases where one does not produce separate release and debug versions, -it is usual to just generate the debug artifacts, then strip the debug symbols -from them to obtain a pseudo-release version. Even in this case, being able to -reduce the time and space requirements for producing the debug artifacts in the -first place is of value. - -Moreover, distributions usually provide debug information in different packages -and for that purpose apply themselves debug fission or stripping techniques. -Projects that provide separated debug information have thus an advantage in -getting accepted by reducing the work involved in packaging them for distros. - -[Debug fission](https://www.tweag.io/blog/2023-11-23-debug-fission) -------------------------------------------------------------------- - -This approach targets specifically Linux ELF files and `gdb(1)`. This is, -however, more than enough to cover the most used UNIX-like platforms and most -general purpose debugging tools, which rely on `gdb(1)`. - -### Concept - -Fission, aka splitting debug information into separate files, exists in modern -tools for a [long time](https://gcc.gnu.org/wiki/DebugFission). This method -can be applied when one already splits builds into their constituting compile -and link steps, which is the case with most modern build tools, including -*justbuild*. Debug fission proposes that the compilation step of a debug build -produce, instead of one object file, two artifacts: a `.dwo` DWARF file -containing all the debug symbols of the compilation unit, and the (now smaller) -`.o` object file containing now only references to the debug symbols from the -`.dwo` file. These object files can be linked as usual to produce the final -build artifact, and the `.dwo` files can be either retained as-is, or packed -into a corresponding `.dwp` file. - -### Benefits - -By splitting the debug symbols of each compilation unit into separate -artifacts, these can be cached and reused as needed, removing any previous -debug symbols duplication across build artifacts. This has also a beneficial -impact on the build times. Moreover, the stripped artifacts are quasi-identical -(e.g., executables differ only in their internal Build ID). - -Proposal --------- - -Debug fission requires changes affecting the `OSS` and `rules-cc` rules, as well -as the OSS toolchain configuration. In order to ensure that new and old -toolchains and rules, respectively, are still able to work together, the new -rules will only perform debug fission if the toolchain is configured to use this -feature. - -The following sections describe the needed changes in detail. - -### Extend `["CC", "defaults"]` rule with new fields - -The `["CC", "defaults"]` should accept 3 new fields: - - - `"DWP"`, containing as a singleton list the path to the `dwp` tool to be used - for packing DWARF files. This field should be handled the same as, e.g., the - `"CC"` variable. The description of our toolchains should be extended with - the `"DWP"` field, pointing to the location of the respective tool in the - staged binaries folder. - - - `"ADD_DEBUGFLAGS"`, containing as a list compile flags to be appended to both - C and C++ flags for debug builds. As with other `"ADD_<X>FLAGS"` variables, - these extend the existing flags. - - - `"DEBUGFLAGS"`, containing as a list the compile flags to be used for debug - builds, replacing existing. - -#### Change `"DEBUG"` configuration variable value type to map - -In order for the defaults to properly set the appropriate flags in debug mode, -the configuration variable `"DEBUG"` is changed to be a mapping. As before, a -`true` evaluation of its value (now, a not empty map) will signal debug mode. -The following supported keys are proposed: - - - a `"USE_DEBUG_FISSION"` flag, which, if evaluated to `true` enables debug - fission and otherwise signals regular debug mode, and - - - a `"FISSION_CONFIG"` map, which can configure in more detail how debug - fission behaves. Existence is enforced if debug fission is enabled. - Otherwise, this field is ignored. - -The `"USE_DEBUG_FISSION"` flag only enables the corresponding rules logic, but -does not change any compile and/or link flags. In particular, instructing the -compiler to generated any DWARF files to begin with must be done through an -appropriate `"FISSION_CONFIG"` map. - -The `"FISSION_CONFIG"` map should accept the following keys: - - - `"USE_SPLIT_DWARF"`: If evaluated to `true`, appends the `-gsplit-dwarf` - flag to the compile flags. This is currently the only flag that actually - instructs the compiler to generate .dwo files and as such it should be - provided in case debug fission is enabled. - - - `"DWARF_VERSION"`: Expects a string defining the DWARF format version. If - provided, appends the `-gdwarf-<value>` flag to the compile flag. - - Each toolchain comes with a default in terms of which version of the DWARF - format is used. Basically all reasonably modern toolchains (GCC >=4.8.1, - Clang >=7.0.0 at least) and debugging tools (GDB >= 7.0) use DWARFv4 by - default, with the more recent versions having already switched to using the - newer, upward compatible [DWARFv5](https://dwarfstd.org/dwarf5std.html) - format. However, the degree of implementation and default support of the - various compilers and tools differs, so it is recommended to use version 4. - - - `"USE_GDB_INDEX"`: If evaluated to `true`, appends the `-Wl,--gdb-index` flag - to the compile flags. Defaults to `false`. - - This option enables, in linkers that support it, an optimization which - bundles certain debug symbols sections together into a single `.gdb_index` - section, reducing the size of the final artifact (quite significantly for - large artifacts) and drastically improving the debugger start time, but at - the cost of a slower linking step. - - - Known supported linkers: `lld` (LLVM >=7), `gold` (binutils >=2.24*), `mold` (>=2.3) - - *`gold` linker additional info: - As per the [release notes](https://lwn.net/Articles/1007541/), `binutils` - 2.44 (2025-02-02) does **NOT** come with the `gold` linker anymore, as it - is considered deprecated and will be removed completely in the near future - unless new maintainers are found. Note also that Fedora, for concerns of - bit-rot, moved the `gold` linker from its `binutils` RPM to a separate - package already since version 31 (2019-10-29). - - - Known unsupported linkers: `ld` (GNU) - - - `"USE_DEBUG_TYPES_SECTION"`: If evaluated to `true`, appends the - `-fdebug-types-section` flag to the compile flags. Defaults to `false`. - - This option enables, for toolchains supporting at least DWARFv4, an - optimization that produces separate debug symbols sections for certain large - data types, thus providing the linker the opportunity to deduplicate more - debug information, resulting in smaller artifacts. - - More performant approaches to reduce the size of the debug information exist, - but are not as straight-forward to implement as enabling a flag. For example, - Fedora opted instead to use the `dwz` compression tool, another known - approach, and make it its default for handling debug RPMs already since - version 18 (2013-01-15). - -The `["CC", "defaults"]` rule interrogates these fields in order to set the -appropriate debug flags to be provided to the library/binary rules. Note -that it is the user's responsibility to configure the debug mode accordingly. -It is always up to each toolchain how unsupported or unexpected combinations of -flags are being handled. - -### Changes to `"library"` and `"binary"` rules - -The toolchain flags will be treated as before, with the addition that in debug -mode, if the final compile flags list is empty, `["-g"]` will be used by -default. To this resulting flags list any debug fission flags configured via the -`"FISSION_CONFIG"` map will be added if debug fission is enabled. - -The `"USE_DEBUG_FISSION"` flag of `"DEBUG"` will inform these rules on whether -the debug fission logic should be used or not. In this way, only the combination -of an appropriate configuration and these updated rules will be able to perform -debug fission, while all other combinations of toolchains and rules will perform -as before. - -All consumers of the internal `"objects"` expression (i.e., static/dynamic -libraries and binaries) should provide a new field `"dwarf-pkg"`, defaulting to -the empty map. If debug fission is enabled, this field will contain the -corresponding DWARF package file, constructed via a new expression -`"dwarf package"`, which, based on given `"dwarf objects"`, `"dwarf deps"` and a -`stage`, uses the given DWARF objects and DWARF package files (provided by the -`"dwarf deps"`) as inputs to an `ACTION` generating a resulting DWARF package -file, appropriately staged. Each such consumer will pass the appropriate inputs -to the new `"dwarf package"` expression considering that `.dwo` files need to be -gathered the same as `.o` files and `.dwp` files need to be gathered the same as -libraries/binaries. In particular, the `"dwarf deps"` shall contain only the -DWARF package files of any link dependencies that are considered for generating -the regular artifact (library/binary). - -As an auxiliary required change, the output of the compile `ACTION` in the -`"objects"` expression should be extended, if debug fission is enabled, by an -additional path corresponding to the expected `.dwo` DWARF file, staged next to -the usual `.o` file. This path needs to be passed to any consumers of -`"objects"`. For this purpose, the expression should be refactored to return a -map result instead of a single variable. - -### Extend the `"install-with-deps"` rule to stage DWARF files - -The `"install-with-deps"` rule should stage also any `"debug-pkg"` entries from -providers when in debug mode, in the same locations as it does regular -artifacts. As paths are handled by each library/binary accordingly, the DWARF -package files should always end up next to their corresponding build artifact, -i.e., where `gdb(1)` expects them. - -### Bootstrappable toolchain to expose the `dwp` binary - -The bootstrappable toolchain repository provides several toolchains built from -source. In the case of the `gcc` and `clang` compilers, a `dwp` tool should be -part of the produced staged binaries. This can be advertised to consumers of -these toolchains (compilers, compilers+tools) in their `["CC", "defaults"]` via -the newly introduced `"DWP"` field. diff --git a/doc/future-designs/just-lock.md b/doc/future-designs/just-lock.md deleted file mode 100644 index bb347005..00000000 --- a/doc/future-designs/just-lock.md +++ /dev/null @@ -1,448 +0,0 @@ -Just-lock -========= - -Status quo ----------- - -The canonical method of performing a multi-repository build using our build tool -is by running `just-mr` on a configuration file which has a well-defined format -and contains the description of all the repositories that should be considered -for the build. This file in many ways acts as a *lock file* of the project, -maintaining a snapshot in time of the repositories configuration. As with any -project lock file, this configuration file should be kept under version control -together with the source code, ensuring all project users can rely on the same -dependencies. - -Dependencies for multi-repository *justbuild* projects require suitable -repository descriptions. For direct dependencies using *justbuild* one can, and -we argue should, rely on their committed configuration file. When such a -dependency is under Git version control, **`just-import-git`**(1) can take care -of the issue of repository composition, importing a given repository together -with its transitive dependencies from the target configuration. Multiple -dependencies are currently treated by multiple import tool calls, with the -common approach being to pipe the output of each call into the next, thanks to -a design that expects input at `stdin` and output to `stdout`. Typically, -deduplication of common transitive dependencies, which can otherwise lead to -configuration bloat, is handled by a final pipe into -**`just-deduplicate-repos`**(1). - -While Git is a very popular version control system, it is by no means the only -option, alternatives such as CVS, Subversion, or Mercurial having also large -audiences. In such cases, a current available solution is to make the desired -content available locally and use `just add-to-cas` to import the content to -local CAS as a Git tree, for which then a corresponding repository description -can be written. - -Some languages have their own methods of tracking and managing project -dependencies, which can and should be leveraged. For example, in the case of -*Rust*, **`just-import-cargo`**(1) uses *Cargo* to retrieve the dependencies of -a given Rust crate and generate the appropriate repository descriptions. - -Shortcomings ------------- - -### Tooling limitations - -The existing tools mentioned above are good at tackling their respective -use-cases, however they do come also with some limitations, the main one being -that they work only on individual repositories. This means that for projects -containing several *justbuild* dependencies (especially as our tool gains more -popularity) one needs to perform multiple imports. Complexity and loss of -readability increases quite fast with the current approach of chaining many -calls together and desirable features, like granular error reporting, require -more layers of scripting. Additionally, the status quo of addressing -side-effects of one tool by another (such as the already mentioned repository -deduplication case) can be improved by a unified framework. - -### Fragmentation - -The generation and maintenance of the configuration file of multi-repository -*justbuild* projects currently takes place in different manners, ranging from -direct manual editing to various user-made scripts wrapping calls to existing -tools. This has the main benefit that users have the highest liberty in deciding -which tools to use in handling each particular use-case when setting up or -updating the configuration file. On the other hand, choosing the best tool or -approach might not always be straight-forward, especially for newer users. - -While currently the number of tools available to tackle various specific -scenarios is small, it is clear that with the continued development of -*justbuild* more use-cases will arise. Some of these are already known (such as -imports of non-Git sources or imports from local checkouts), some yet to be -discovered (for example, those that may come with the introduction of rules -support for more languages that have their own ecosystem). An approach to assist -future users can be, of course, to increase the current roster of tools. This -can however lead to a fragmented ecosystem around *justbuild* that handles to a -not inconsiderate extent the same task. On the other hand, being reticent in -introducing tooling options can be detrimental in attracting new users. As in -many cases, a balanced approach might offer the best of both worlds. - -Other use-cases related to repository configuration are also not addressed by -current tooling. For example, two or more repositories might be logically -coupled, which, while not recommended, is a situation found in larger projects -for which a deep refactoring to decouple them can be prohibitively expensive. -In such a case, local development has to take place in more than one repository. -Creating local clones aware of the dependency closure of a repository -(completely defined in the configuration file) can be thus a useful feature. - -Proposal: `Just-lock` ---------------------- - -We propose `just-lock` as a framework for **generating** and **maintaining** -the multi-repository configuration file of a *justbuild* project. This addresses -the mentioned shortcomings by providing a common interface for current import -functionalities, while allowing new tools to be freely used under very lax -conditions. This is achieved by means of a set of built-in import options, -describable in a well-defined format. - -The framework revolves around the `just-lock` tool, which will implement all the -required functionality. - -### The tool - -Usage: generate/update a `just-mr` configuration file - -``` -just-lock [-C <repos.in.json>] [-o <repos.json>] - [--local-build-root <PATH>] [--just <PATH>] - [--git <PATH>] [-L|--launcher <JSON>] - [--clone <JSON>] - -OPTIONS: - -C PATH Input file. If missing, searched for in ['repos.in.json', 'etc/repos.in.json'] - with respect to the workspace root. - -o PATH Output file. If missing, searched for in ['repos.json', 'etc/repos.json'] with respect to - the workspace root. If none found, placed as 'repos.json' in parent path of input file. - --local-build-root PATH - Local build root. Usual `just-mr` rules apply. - --just PATH Path to a `just` binary. If missing, use the `just` name from PATH. - --git PATH Git binary to use if needed. If missing, system `git` is used. - User must pass it also to `just-mr`. - --launcher JSON Local launcher to use for commands. Given as a JSON list of strings. - If missing, ["env", "--"] is used. User must pass it also to `just-mr`. - --clone JSON Mapping from filesystem path to pair of repository name and a list of bindings. - For each map entry, the target repository, found by following the bindings from a given start - repository, after all repositories have been imported, will have its workspace root cloned in - the specified filesystem directory and its description in the output configuration made to - point to this clone. - The start repository names must be known, i.e., an initial repository or declared import, - and both the start and target repositories will be kept during deduplication. -``` - -- Notes: - - The proposed default naming choice for the input file is chosen to mirror the - default names of the configuration file of `just-mr`. - - The proposed search locations for the output configuration file, if an - explicit path is not provided, are the same ones used by `just-mr` when run - with the `--norc` option, but limited to the ones relative to the workspace - root. This is done to better match the desired lock-file quality of the - output file and also ensure `just-mr` can pick it up by default. - - The specification for finding the target repository for the `--clone` option - uses the fact that the names of existing repositories and declared imports - are the only ones known to remain as such in the output configuration, prior - to deduplication, with all other repositories reachable from one such - repository via a defined sequence of bindings. The clone locations are - disjoint (as they are map keys), can be specified both absolute or relative - to the current directory, and the referred to directory will be created if - missing. - - The `--clone` option will produce an output configuration file meant for - local development only. Therefore, it is not recommended for such a - configuration file to be committed. - -### Input file format - -The input file describes which repositories will be part of the resulting -configuration file. The file is a `JSON` object. The proposed structure is: - -``` jsonc -{ "main": "<name>" -, "repositories": {...} -, "imports": [...] -, "keep": [...] -} -``` - -The input file is expected to contain at most the mentioned 4 fields: `"main"`, -`"repositories"`, `"imports"`, and `"keep"`. Any other fields will be ignored. - -The `"main"` and `"repositories"` fields maintain their meaning from the -usual `just-mr` configuration format (**`just-mr-repository-config`**(5)). -Therefore, neither fields are mandatory and missing fields are treated -consistently with `just-mr`. In this way, a `just-lock` input file containing -at most the `"main"` and `"repositories"` fields is a valid `just-mr` -configuration file. This subset of the input configuration object is referred -to in the following as the _core_ configuration. - -For the input file of `just-lock` the `just-mr` format is simply extended with -**two** new fields. The value of the `"imports"` field is a list of `JSON` -objects describing a ***source***. Each _source_ provides information (in the -form of a well-defined set of fields) about how the _core_ configuration will -be extended. In most cases, this takes place by importing one or more -repositories from a specified existing `just-mr` configuration file, each with -their transitive dependencies, but more general options are available (as will -be described below). The imports are processed individually and consecutively -in the well-defined order declared in the input configuration file, meaning -that each import is extending the configuration obtained after processing all -preceding imports. - -The format imposes a well-defined ordering of imports in order to maintain the -_naming convention_ already implicitly implemented by the existing tools. This -states that each import will add to the _core_ configuration only the following -repository names: -- the _name_ of the specified repository, which is well-defined by the input - file format, and -- names starting with that _name_ followed by `"/"`, corresponding to the - transitive dependencies of the specified repository. - -This naming convention allows open names to be filled later in the import -sequence by repository names specified in the input file without the fear that -have been taken up during an earlier import. - -If the `"main"` field is provided in the input file, it must match one of the -repository aliases marked for import or the name of one of the repositories -given by the `"repositories"` field. - -The value of the `"keep"` field is a list of strings stating which repositories, -besides the one specified by `"main"` and those specified by option `--clone`, -are to be kept during the final _deduplication_ step, which takes place after -all imports have been processed. This way, `just-lock` will include all the -functionality `just-deduplicate-repos` provides. The output configuration file -of `just-lock` will always have deduplicated entries. - -#### Proposed source types - -The type of a _source_ is defined by the string value of the mandatory subfield -`"source"`. - -- **git** - - This source type encompasses the functionality of `just-import-git`. - - We argue that most *justbuild* projects will contain one main configuration - file, describing one or more repositories. This is why we propose a format - that allows importing multiple repositories from the same source configuration - file. Each declared repository is imported independently and consecutively, - in the well-defined order provided by the user. - - If the `"commit"` field is missing, the `HEAD` commit of the specified remote - branch will be considered. This will have an effect also on the fixed commit - that will be used in the resulting repository description corresponding to any - imported `"file"`-type repositories (see `just-import-git`). - - If `"as plain": true`, any provided `"special"` key for the `"pragma"` field - in the source description is unconditionally set in the imported repositories, - superseding any other config- or import-level treatment of pragmas during the - import. Note that `"as plain": true` results in only one repository - (containing the whole source repository tree) being imported. - - Proposed format: - ``` jsonc - { "source": "git" - // "source"-specific fields - // defines which repositories to import from source repository - , "repos": // mandatory; list of repositories to be imported - [ { "alias": "<name>" // corresponds to `import_as` var (option --as); - // mandatory if "repo" value missing, otherwise value of "repo" taken if missing - , "repo": "<foreign_name>" // optional; corresponds to `foreign_repository_name` var - , "map": {"from_name": "to_name"} // optional; corresponds to `import_map` var (option --map) - , "pragma": // optional - {"absent": true} // corresponds to `absent` var (option --absent) - } - , ... - ] - // fields related to obtaining source config - , "url": "https://nonexistent.example.com/repo.git" // mandatory - , "mirrors": ["https://nonexistent-mirror.example.com/repo.git"] // optional - , "branch": "master" // mandatory (as we have no sane default value between "master" and "main"); - // corresponds to `branch` var (option -b) - , "commit": "<HASH>" // optional; if missing, take HEAD commit of branch - , "inherit env": [...] // optional; corresponds to `inherit_env` var (option --inherit-env) - , "config": "<foreign_repos.json>" // optional; corresponds to `foreign_repository_config` var (option -R) - , "as plain": false // optional; corresponds to `plain` var (option --plain) - , "pragma": {"special": "<value>"} // optional; only considered if `"as plain": true` - } - ``` - -- **file** - - This _source_ type behaves similarly to **git**, with the main difference - being that the referenced source repository is not a Git remote, but a local - checkout. - - The checkout is assumed to be maintained, so that `"file"`-type repositories - marked to be imported can retain their type. For such transitive dependencies, - one can also set the `"to_git": true` pragma with a corresponding entry in the - usual `"pragma"` field. - - If `"as plain": true`, any provided `"special"` key for the `"pragma"` field - in the source description is unconditionally set in the imported repositories, - superseding any other config- or import-level treatment of pragmas during the - import. Note that `"as plain": true` results in only one repository - (containing the whole source repository tree) being imported. - - Proposed format: - ``` jsonc - { "source": "file" - // "source"-specific fields - // defines which repositories to import from source repository - , "repos": // mandatory; list of repositories to be imported - [ { "alias": "<name>" // corresponds to `import_as` var (option --as); - // mandatory if "repo" value missing, otherwise value of "repo" taken if missing - , "repo": "<foreign_name>" // optional; corresponds to `foreign_repository_name` var - , "map": {"from_name": "to_name"} // optional; corresponds to `import_map` var (option --map) - , "pragma": // optional - { "absent": true // corresponds to `absent` var (option --absent) - , "to_git": true // any imported "file"-repositories will also be "to_git":true - } - } - , ... - ] - // fields related to obtaining source config - , "path": "<source/repo/path>" // mandatory - , "config": "<foreign_repos.json>" // optional; corresponds to `foreign_repository_config` var (option -R) - , "as plain": false // optional; corresponds to `plain` var (option --plain) - , "pragma": {"special": "<value>"} // optional; only considered if `"as plain": true` - } - ``` - -- **archive** - - This _source_ type behaves similarly to **git**, with the main difference - being that the referenced source repository is not a Git remote, but an - archive, such as a release tarball. - - A field `"subdir"` is provided to account for the fact that source repository - root often is not the root directory of the unpacked archive. - - If `"as plain": true`, any provided `"special"` key for the `"pragma"` field - in the source description is unconditionally set in the imported repositories, - superseding any other config- or import-level treatment of pragmas during the - import. Note that `"as plain": true` results in only one repository - (containing the whole source repository tree) being imported. - - Proposed format: - ``` jsonc - { "source": "archive" - // "source"-specific fields - // defines which repositories to import from source repository - , "repos": // mandatory; list of repositories to be imported - [ { "alias": "<name>" // corresponds to `import_as` var (option --as); - // mandatory if "repo" value missing, otherwise value of "repo" taken if missing - , "repo": "<foreign_name>" // optional; corresponds to `foreign_repository_name` var - , "map": {"<from_name>": "<to_name>"} // optional; corresponds to `import_map` var (option --map) - , "pragma": // optional - {"absent": true} // corresponds to `absent` var (option --absent) - } - , ... - ] - // fields related to obtaining source config - , "fetch": "<URL>" // mandatory - , "type": "tar|zip" // optional; type of archive in set ["tar", "zip"]; if missing, default to "tar" - , "mirrors": ["..."] // optional - , "subdir": "<REL_PATH>" // optional; relative path defining the actual root of the source repository; - // if missing, the source root is the root directory of the unpacked archive - , "content": "<HASH>" // optional; if missing, always fetch; if given, will be checked - , "sha256": "<HASH>" // optional checksum; if given, will be checked - , "sha512": "<HASH>" // optional checksum; if given, will be checked - , "config": "<foreign_repos.json>" // optional; corresponds to `foreign_repository_config` var (option -R) - , "as plain": false // optional; corresponds to `plain` var (option --plain) - , "pragma": {"special": "<value>"} // optional; only considered if `"as plain": true` - } - ``` - -- **git tree** - - This _source_ type proposed to be the canonical way of importing *justbuild* - dependencies under version control systems other than Git. - - The command that produces the tree is either given explicitly (field `"cmd"`) - or indirectly by a command-generating command (field `"cmd gen"`). The tool - will run the so-given command to produce the content in a temporary directory, - it will import the given subdirectory to Git, and it will generate a - corresponding `"git tree"`-type repository description to be added to the - configuration. - - The fields `"cmd"`, `"env"`, `"inherit env"` have the same meaning as those - of the `"git tree"`-type repository (as per `just-mr-repository-config`). - - **IMPORTANT:** The user has to be the one to ensure that the environment in - which `just-lock` is run matches the one intended for running `just-mr` with - respect to all the provided envariables in the `"inherit env"` list. This is - because `just-lock` and `just-mr` must produce the same tree when running the - same command. - - NOTE: While the target configuration file has to be part of the specified - `"subdir"` tree, referenced `"file"`-type repositories marked to be imported - can point also outside of the `"subdir"`, as long as they are still contained - in the initial checkout (i.e., the directory generated by the command). All - such repositories will be translated to appropriate `"git tree"`-type - repositories in the output configuration. - - If `"as plain": true`, any provided `"special"` key for the `"pragma"` field - in the source description is unconditionally set in the imported repositories, - superseding any other config- or import-level treatment of pragmas during the - import. Note that `"as plain": true` results in only one repository - (containing the whole source repository tree) being imported. - - Proposed format: - ``` jsonc - { "source": "git tree" - // "source"-specific fields - , "repos": // mandatory; list of entries describing repositories to import - [ { "alias": "<name>" // mandatory; same meaning as `import_as` var - // mandatory if "repo" value missing, otherwise value of "repo" taken if missing - , "repo": "<foreign_name>" // optional; same meaning as `foreign_repo_name` var - , "map": {"<from_name>": "<to_name>"} // optional; corresponds to `import_map` var (option --map) - , "pragma": // optional - {"absent": true} // same meaning as `absent` var - } - ] - , "cmd": [...] // one and only one of {"cmd", "cmd_gen"} must be provided; - // command as list of strings - , "subdir": "<subdir>" // optional; default is "."; subdir to consider as main entry point - , "env": {...} // optional; map of envariables needed by "cmd" - , "inherit env": [...] // optional; list of envariables to inherit - , "cmd gen": [...] // one and only one of {"cmd", "cmd_gen"} must be provided; - // command producing the "cmd" value to use, as list of strings - , "config": "<foreign_repos.json>" // optional; corresponds to `foreign_repository_config` var (option -R) - // searched for in the "subdir" tree - , "as plain": false // optional; corresponds to `plain` var (option --plain) - , "pragma": {"special": "<value>"} // optional; only considered if `"as plain": true` - } - ``` - -- **generic** - - This _source_ type is proposed to be the canonical way for users to provide - their own command which can update a `just-mr` configuration. - - The command must accept a `just-mr` configuration as input from `stdin` and - must output a `just-mr` configuration to `stdout`. The command is run in a - given subpath of the current directory (by default `"."`) and as such can have - side-effects on the filesystem. - - The input fed to the command is the _current_ configuration, i.e., the - configuration obtained after processing all preceding imports (according to - the well-defined order declared in the input configuration file). The output - configuration is used as input for the succeeding import. - - The user must take care to correctly construct the `"imports"` list in order - to process **generic** entries at the desired time. For example, if a - **generic** entry needs to be process between the import of two repositories - from the same **git** source, the user must split that **git** source into two - corresponding **git** entries and place the **generic** entry between them. - - Proposed format: - ``` jsonc - { "source": "generic" - // "source"-specific fields - , "cwd": "<path>" // optional; relative path to run the script in; - // if missing, defaults to "." - , "cmd": [...] // mandatory; command to run, as list of strings - , "env": {...} // optional; map of envariables needed by script - , "inherit env": [...] // optional; list of envariables to inherit - } - ``` |