| Age | Commit message (Collapse) | Author |
|---|
|
... and thus allowing proper quoting of command output.
|
|
...which do not stage also the debug source and header files (while
in debug mode), as this is unnecessary bloat in the tests.
As the tool-under-test and mr-tool-under-test targets should be
used instead of the regular install targets also in the various
extra rules in end-to-end and utils, move their definition in the
outmost test TARGETS file.
|
|
|
|
|
|
|
|
|
|
...instead of the unused ';'
Using spaces allows for simpler looping in shell script
|
|
...that come with installing just. This ensures control on the
subdirectories available to the runner, avoiding any possible
conflicting paths.
|
|
The gsl-lite implementation is slightly more picky in terms of
type conversions and constness resolution in initializers,
therefore small changes were needed.
|
|
The runners used in tests that rely on execution or serve
endpoints to exist can get stuck waiting for these to become
online if for some reason they cannot be set up. This commit fixes
this issue by setting a reasonable timeout, after which we fail
gracefully.
|
|
Also updates the tests and all relevant documentation accordingly.
|
|
|
|
|
|
This test, among others, verifies the archive functionality by
creating an archive with our library, extractting it with the
system command-line tools, and comparing the result. In order to not
depend on the host system having installed tools for all possible
compression algorithms, it tacitly drops the extraction test if the
respective tool could not be found under /usr/bin. This, however,
assumes that /usr/bin is in path; ensure this, by extending PATH
accordingly.
|
|
|
|
Some of the more specific issues addressed:
- missing log_level target/include
- header-only libs wrongly marking deps as private
- missing/misplaced gsl includes
|
|
...with regular instances that have controlled life-times.
This avoids race conditions in tracking and reporting the results
of analysis and build, as the serve endpoint can orchestrate
multiple builds at the same time asynchronously. As a bonus
side-effect this also ensures the correctness of the progress
reporting per orchestrated build.
|
|
|
|
... instead of hard-coding it in the example graphs. In this way,
we can honor a provided PATH.
|
|
...instead of the vector containing the digests of the uploaded
blobs. The returned vector was never inspected by the callers, except
for its size.
The tests have been accordingly amended.
|
|
This way they can be used by 'just serve'.
|
|
The serve service will communicate with this endpoint when needed,
as well as ensure artifacts it provides are synced with the remote
execution CAS, if requested by the client.
If just-mr is given the --fetch-absent option, it Always produce
present roots irrespective of the 'absent' pragma. For Git repositories
marked with the 'absent' pragma, first try to fetch any commit
trees provided by the serve endpoint from the execution endpoint
CAS, before reverting to a network fetch.
Co-authored-by: Klaus Aehlig <klaus.aehlig@huawei.com>
Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
|
|
|
|
For maximum compatibility, we use the uppercase types from the
typing package instead of the built-in types, therefore compliant
with PEP 484 and PEP 526.
|
|
|
|
... by creating a file only once the server is ready and waiting
for that file instead of waiting a fixed period of time.
|
|
|
|
|
|
... as those binaries might need run-libs.
|
|
|
|
This allows better separation and, in particular, repositories
needed only for tests do not have to be provided for building the
tools. This also better documents which dependencies are only needed
for testing.
|
|
...caused by incorrectly setting and resetting the library internal
state and the misuse of pthreads in libgit2.
Normally, git_libgit2_init and git_libgit2_shutdown should span the
life of a worker thread in order to be safely used. However, due to
an incorrect implementation of libgit2's threadstate with pthreads,
on unix systems there is a race condition.
Until the use of pthread_key_t is corrected in libgit2, we need to
apply a workaround by always ensuring that the main thread is the
first thread reaching the GitContext constructor.
|
|
... with two minor code base changes compared to previous
use of gsl-lite:
- dag.hpp: ActionNode::Ptr and ArtifactNode::Ptr are not
wrapped in gsl::not_null<> anymore, due to lack of support
for wrapping std::unique_ptr<>. More specifically, the
move constructor is missing, rendering it impossible to
use std::vector<>::emplace_back().
- utils/cpp/gsl.hpp: New header file added to implement the
macros ExpectsAudit() and EnsureAudit(), asserts running
only in debug builds, which were available in gsl-lite but
are missing in MS GSL.
|
|
tests have been updated accordingly
|
|
|
|
The improved GC implementation uses refactored storage
classes instead of directly accessing "unknown" file paths.
The required storage class refactoring is quite substantial
and outlined in the following paragraphs.
The module `buildtool/file_system` was extended by:
- `ObjectCAS`: a plain CAS implementation for
reading/writing blobs and computing digests for a given
`ObjectType`. Depending on that type, files written to the
file system may have different properties (e.g., the x-bit
set) or the digest may be computed differently (e.g., tree
digests in non-compatible mode).
A new module `buildtool/storage` was introduced containing:
- `LocalCAS`: provides a common interface for the "logical
CAS", which internally combines three `ObjectCAS`s, one
for each `ObjectType` (file, executable, tree).
- `LocalAC`: implements the action cache, which needs the
`LocalCAS` for storing cache values.
- `TargetCache`: implements the high-level target cache,
which also needs the `LocalCAS` for storing cache values.
- `LocalStorage`: combines the storage classes `LocalCAS`,
`LocalAC`, and `TargetCache`. Those are initialized with
settings from `StorageConfig`, such as the build root base
path or number of generations for the garbage collector.
`LocalStorage` is templated with a Boolean parameter
`kDoGlobalUplink`, which indicates that, on every
read/write access, the garbage collector should be used
for uplinking across all generations (global).
- `GarbageCollector`: responsible for garbage collection and
the global uplinking across all generations. To do so, it
employs instances of `LocalStorage` with `kDoGlobalUplink`
set to false, in order to avoid endless recursion. The
actual (local) uplinking within two single generations is
performed by the corresponding storage class (e.g.,
`TargetCache` implements uplinking of target cache entries
between two target cache generations etc.). Thereby, the
actual knowledge how data should be uplinked is
implemented by the instance that is responsible for
creating the data in the first place.
|
|
... instead of using pipes that are prone to get full. Also
increase log level to Trace.
|
|
|
|
... now that they no longer depend on an external remote
execution.
|
|
In order to keep our tests self-contained, do not rely on
an external remote-execution service to be present; instead,
use `just execute` to provide the remote execution service.
|
|
|
|
|
|
... by keeping track of each blob being a file or executable
and storing it to the correct local physical CAS directory.
The new flag is merely a hint and only used by the local
execution API. Leaving it out will still correctly transfer
the blob but may cause unnecessary duplicates in file CAS.
|
|
...in order to not include unwanted dependencies in just proper.
As the whole other_tools folder is meant to be excluded from
bootstrapping, also remove the bootstrap guards.
|
|
...in order to not include unwanted dependencies in just proper.
As the whole other_tools folder is meant to be excluded from
bootstrapping, also remove the bootstrap guards.
|
|
|
|
|
|
|
|
|
|
|
