| Age | Commit message (Collapse) | Author |
|---|
|
|
|
Often outputs are only referenced as blobs but not downloaded to the
working directory of the test. This can make it hard to understand
errors, as the respective artifacts are not available for inspection.
This is even more important in case of tests with a provided serve
endpoint as then even the error message of a failed serve build is
only referenced as blob. Solve this by keeping the local build root
of the remote-execution service using the fact that all objects are
transferred between the serve endpoint and the client go through
the remote-execution endpoint.
|
|
Before this patch, if the option `--remote-execution-property KEY:VAL`
is repeated multiple times (also with different `KEY`s), only the last
one is taken into account.
This patch fixes the intended behavior.
|
|
|
|
|
|
|
|
This test, among others, verifies the archive functionality by
creating an archive with our library, extractting it with the
system command-line tools, and comparing the result. In order to not
depend on the host system having installed tools for all possible
compression algorithms, it tacitly drops the extraction test if the
respective tool could not be found under /usr/bin. This, however,
assumes that /usr/bin is in path; ensure this, by extending PATH
accordingly.
|
|
... and set quadratic bounding box.
|
|
|
|
|
|
|
|
... which are, in particular, artifacts involved in staging conflicts.
While there, also make disjoint union honor the expression log limit.
|
|
Mentioning in particular the involved artifacts as well as the direct
dependencies that brought them in. Here, we are in a simple situation
as all built-in rules that check conflicts only use artifacts and
runfiles of their dependencies, but not the provided data.
Also, the built-in rules that check staging conflicts do not do
configuration transitions, hence it is enough to show the target
name of the dependencies containing the artifact if for the built-in
target we show the configuration.
|
|
operations
This test creates a "file" repository with pragma "to_git". Move to a
subdirectory to avoid including all the tools in that created root.
|
|
To avoid too many intermediate results, we compute the union of
a list in a divide and conquer fashion. Of course, for a disjoint
union, the recursive calls on the lists of half the length have to
be disjoint as well, i.e., the template parameter kDisjoint has to
be passed on. Fix this.
|
|
grpc is used in the toolchain defaults for proto servive libraries.
Still, it is typically built on its own, with its own toolchain,
flags, etc. Now, grpc, however has a public dependency on a the
rpc-status proto library, that the user may well use on their own,
however building in their own way which can yield conflicts. To solve
this, we hide the dependency on that proto library, as infrastructure
libs should not make assumptions on user-servicable libraries.
- First, we note that the dependency can be made a private one,
which already solves the conflict on header files (which will
essentially be the same, but might be defined in a different way).
- Next, we note that the library at linking basically only acts
as a default implementation; if the user provides their own
version of the rpc-status library, we should prefer that anyway.
As infrastructure is linked last, we have that default character
anyway; the only thing to do is to rename the library that no
staging conflict occurs.
|
|
For historic reasons (as quite some tests date back till before the
public name of the build tools was decided), the end-to-end tests
assume generic names for the tools. This used to be done by simple
staging the artifacts. As soon as we started to support dynamic
linking, we also have to allow the runtime dependnecies, as provided
by our install-with-deps rule. ae2e515ab84ea3ab08764685f84441c0741f8039
attempted to add those dependencies by replacing the staging by
a generic action doing a copy. This, however, made the "lib" dir
containing the dependencies an opaque tree
- defined by different actions, and, more importantly,
- containing only the run-time dependencies of one of the tools.
This causes staging conflicts between those two lib dirs (currently
hidden by a bug in the computation of the disjoint union) and things
only worked because in the canonical configuration used for testing
both "lib" dirs are empty anyway.
The correct way of adding dependencies while renaming the tool is
still staging; fix this.
|
|
|
|
For splicing of large objects from external sources additional checks are performed:
* The digest of the spliced result must be equal to the expected digest;
* The parts of a spliced tree must be in the storage.
Tested:
* Regular splicing of large objects;
* If the result is unexpected, splicing fails;
* If some parts of a tree are missing, splicing fails.
|
|
This is needed for LocalCAS's splice routines.
|
|
* Uplink parts of the large entry before entry itself;
* Uplink large entries in LargeObjectCAS::GetEntryPath to not split things two times;
* Promote spliced tree during uplinking of a large tree entry to properly promote parts of the tree;
* Uplink large entries in LocalUplink{Blob, Tree} to support proper uplinking in Action Cache and Target Cache;
Tested:
* Uplink large blobs and trees;
* Uplink a large object that depends on other large objects.
|
|
Implicitly reconstruct objects during regular uplinking of Blobs/Trees.
|
|
* Add LargeObjectCAS fields for files and trees to LocalCAS;
* Add logic for splitting objects located in the main storage.
Tested:
Splitting of large, small and empty objects.
|
|
|
|
|
|
Every large object is keyed by the hash of the result and contains hashes of the parts from which the result can be reconstructed.
|
|
|
|
|
|
With "remote-execution properties" a primary property, there is no need
anymore to repeat the property as part of the "just args".
|
|
This reverts commit 0db6f248a04f5a23102b4208c1f28b3633c6ef8a.
We have our own build description for lzma, hence we are likely not
affected by CVE-2024-3094. Nevertheless, we should not encourage the
use or distribution of an archive that contains a known backdoor.
Reverting this commit also points us to a mirror that is still
fetchable.
|
|
... and not directly used by our tool.
|
|
|
|
|
|
Trees are first-class objects for justbuild. To allow interoperation
with other tools, it is necessary to provide those objects in a
standard format; for directories, those are archives. Hence procive
a corresponding option.
|
|
|
|
|
|
Main culprits:
- std::size_t, std::nullptr_t, and NULL require <cstddef>
- std::move and std::forward require <utility>
- unordered maps and sets require respective includes
- std::for_each and std::all_of require <algorithm>
|
|
Now the curl URL API always fails to parse the empty string, so
our test was changed to reflect this.
|
|
|
|
|
|
|
|
Also updated the remote fetch repository, as the previous upstream
repository has since been archived.
|
|
Also updates the test-mixed-bootstrap script which must use the
explicit library version.
|
|
|
|
|
|
Numerical values are used at some places in justbuild: as value for
timeout scaling, as well as by the "range" expression that is used,
e.g., to define repreated test runs. Therefore, improve support
for numerical values by adding basic operations.
|
|
For an absent export target, the first step of analysis is to ask
serve for the flexible variables. The answer to this request is,
however, independent of the configuration for this target. So we
can avoid calls by caching the answer in an additional map.
|
|
|
|
For export targets, we know ahead of time the effective configuration;
so, if the current configuration is not the effective anyway, we
can simply analyse the effective configuration and take that result.
In this way, we can avoid calls to serve if a target is analysed
in two configurations that coincide on the flexible variables.
|
|
|
