| Age | Commit message (Collapse) | Author |
|---|
|
- add missing serve_api TARGETS file
- rename service client to align with server naming scheme
- fix inconsistencies in comments between implementation and protocol
|
|
This avoids using the more geenric GitRepoRemote method which
has libcurl as a dependency, something that is not needed for this
Git operation.
|
|
The serve service will communicate with this endpoint when needed,
as well as ensure artifacts it provides are synced with the remote
execution CAS, if requested by the client.
If just-mr is given the --fetch-absent option, it Always produce
present roots irrespective of the 'absent' pragma. For Git repositories
marked with the 'absent' pragma, first try to fetch any commit
trees provided by the serve endpoint from the execution endpoint
CAS, before reverting to a network fetch.
Co-authored-by: Klaus Aehlig <klaus.aehlig@huawei.com>
Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
|
|
|
|
|
|
... causing an error with gcc 13.2.0.
|
|
While a foldl is enough to implement a reverse functionality,
adding it as a built in allows doing so in linear time.
|
|
... to obtain from a list of strings a map with those entries
as keys and true as value. In this way, repeated membership tests
in lists can be implemented more efficiently.
|
|
|
|
|
|
|
|
|
|
Before this patch, when creating an action directory, symlinks were
staged as regular files.
|
|
...that std::filesystem::* calls produce. This is because existence
and type checks use almost exclusively std::filesystem::status,
which follows symbolic links, when being called with path arguments.
Instead, one should instead use these methods with the value
returned by a call of std::filesystem::symlink_status.
This commit also streamlines the FileSystemManager tests, as well
as replace bare calls to std::filesystem with their FileSystemManager
counterparts (where suitable).
|
|
...allowing the skipping of certain subtrees if needed. This is
useful, e.g., in simulating what a 'git add' call would do,
which ignores all '.git' subdirectories.
Also adds a corresponding test for the new method.
|
|
|
|
|
|
|
|
|
|
via a 'SYMLINK' constructor function. This works similarly to the
'FILE' construct, but the name given must point to a non-upwards
symlink and a symlink artifact is being generated from it.
Also updates the relevant tests.
|
|
...and update tests accordingly.
|
|
...as early as possible. This ensures that callers always receive
only the tree entries for the supported object types.
For the symlinks non-upwardness check we pass a lambda capturing
the real backend of the tree entries, such that the symlinks can
be read.
Updates git_tree tests accordingly.
|
|
|
|
|
|
This feature has been introduced with C++20.
|
|
...using the new ignore-special tree variant.
|
|
...by removing unused subfolders and corresponding redundant checks
in the tests using it. Also fixes misues in tests of the GetSubtree
family of methods.
|
|
|
|
This allows better separation and, in particular, repositories
needed only for tests do not have to be provided for building the
tools. This also better documents which dependencies are only needed
for testing.
|
|
... in particular echo(1), which behaves different on dash
and sh.
|
|
... with two minor code base changes compared to previous
use of gsl-lite:
- dag.hpp: ActionNode::Ptr and ArtifactNode::Ptr are not
wrapped in gsl::not_null<> anymore, due to lack of support
for wrapping std::unique_ptr<>. More specifically, the
move constructor is missing, rendering it impossible to
use std::vector<>::emplace_back().
- utils/cpp/gsl.hpp: New header file added to implement the
macros ExpectsAudit() and EnsureAudit(), asserts running
only in debug builds, which were available in gsl-lite but
are missing in MS GSL.
|
|
|
|
Move the creation of a fake repository on top of an existing odb
into the individual threads, to ensure the thread-safety of the
operations on fake repositories is properly tested.
|
|
|
|
... as `git init -b ...` is a rather recent git feature.
|
|
|
|
Using "test" in the name of a test target is unnecessary and
adds gratuitous inconsistencies.
|
|
|
|
tests have been updated accordingly
|
|
...both buildbarn and just execute do not allow a direct upload to the
action cache. Moreover, our tool does not support it, anyway, meaning
it was dead code.
|
|
|
|
The improved GC implementation uses refactored storage
classes instead of directly accessing "unknown" file paths.
The required storage class refactoring is quite substantial
and outlined in the following paragraphs.
The module `buildtool/file_system` was extended by:
- `ObjectCAS`: a plain CAS implementation for
reading/writing blobs and computing digests for a given
`ObjectType`. Depending on that type, files written to the
file system may have different properties (e.g., the x-bit
set) or the digest may be computed differently (e.g., tree
digests in non-compatible mode).
A new module `buildtool/storage` was introduced containing:
- `LocalCAS`: provides a common interface for the "logical
CAS", which internally combines three `ObjectCAS`s, one
for each `ObjectType` (file, executable, tree).
- `LocalAC`: implements the action cache, which needs the
`LocalCAS` for storing cache values.
- `TargetCache`: implements the high-level target cache,
which also needs the `LocalCAS` for storing cache values.
- `LocalStorage`: combines the storage classes `LocalCAS`,
`LocalAC`, and `TargetCache`. Those are initialized with
settings from `StorageConfig`, such as the build root base
path or number of generations for the garbage collector.
`LocalStorage` is templated with a Boolean parameter
`kDoGlobalUplink`, which indicates that, on every
read/write access, the garbage collector should be used
for uplinking across all generations (global).
- `GarbageCollector`: responsible for garbage collection and
the global uplinking across all generations. To do so, it
employs instances of `LocalStorage` with `kDoGlobalUplink`
set to false, in order to avoid endless recursion. The
actual (local) uplinking within two single generations is
performed by the corresponding storage class (e.g.,
`TargetCache` implements uplinking of target cache entries
between two target cache generations etc.). Thereby, the
actual knowledge how data should be uplinked is
implemented by the instance that is responsible for
creating the data in the first place.
|
|
... to bring their own remote-execution endpoint.
|
|
|
|
... by keeping track of each blob being a file or executable
and storing it to the correct local physical CAS directory.
The new flag is merely a hint and only used by the local
execution API. Leaving it out will still correctly transfer
the blob but may cause unnecessary duplicates in file CAS.
|
|
...in order to not include unwanted dependencies in just proper.
The new class extends the GitRepo class used for just's Git tree
operations and gets used in all of just-mr's async maps.
|
|
|
|
... and ensure that cascades of checks are performed with
only a single filesystem stat per method.
|
|
...and use instead the branch name. A valid direct refspec (as those
retrieved by a remote_ls call) will always end in the branch name,
so checking the last path component ('/'-delimited substring) of a
retrieved refspec is enough.
|
|
This also removes the need to call the GET_BRANCH_REFNAME critical
operation.
|
