| Age | Commit message (Collapse) | Author |
|---|
|
The improved GC implementation uses refactored storage
classes instead of directly accessing "unknown" file paths.
The required storage class refactoring is quite substantial
and outlined in the following paragraphs.
The module `buildtool/file_system` was extended by:
- `ObjectCAS`: a plain CAS implementation for
reading/writing blobs and computing digests for a given
`ObjectType`. Depending on that type, files written to the
file system may have different properties (e.g., the x-bit
set) or the digest may be computed differently (e.g., tree
digests in non-compatible mode).
A new module `buildtool/storage` was introduced containing:
- `LocalCAS`: provides a common interface for the "logical
CAS", which internally combines three `ObjectCAS`s, one
for each `ObjectType` (file, executable, tree).
- `LocalAC`: implements the action cache, which needs the
`LocalCAS` for storing cache values.
- `TargetCache`: implements the high-level target cache,
which also needs the `LocalCAS` for storing cache values.
- `LocalStorage`: combines the storage classes `LocalCAS`,
`LocalAC`, and `TargetCache`. Those are initialized with
settings from `StorageConfig`, such as the build root base
path or number of generations for the garbage collector.
`LocalStorage` is templated with a Boolean parameter
`kDoGlobalUplink`, which indicates that, on every
read/write access, the garbage collector should be used
for uplinking across all generations (global).
- `GarbageCollector`: responsible for garbage collection and
the global uplinking across all generations. To do so, it
employs instances of `LocalStorage` with `kDoGlobalUplink`
set to false, in order to avoid endless recursion. The
actual (local) uplinking within two single generations is
performed by the corresponding storage class (e.g.,
`TargetCache` implements uplinking of target cache entries
between two target cache generations etc.). Thereby, the
actual knowledge how data should be uplinked is
implemented by the instance that is responsible for
creating the data in the first place.
|
|
Map for a workspace root given as a known git tree id which is
promised to be obtainable by executing a specified command.
|
|
Reporting and counting should be done as early as possible, once
known that there is nothing cached.
|
|
|
|
This includes also the setup-env command, as well as all ~just~
known subcommands that require a just-mr setup step.
Co-authored-by: Sascha Roloff <sascha.roloff@huawei.com>
|
|
In order to only report actual work, the name of the repository for which
work is actually done needs to be known to certain AsyncMaps used by just-mr.
Also, fetches not originating in a distdir repo checkout need to be counted
additionally, so an additional flag was needed for the content_cas_map.
|
|
|
|
...in order to not include unwanted dependencies in just proper.
The new class extends the GitRepo class used for just's Git tree
operations and gets used in all of just-mr's async maps.
|
|
...in order to not include unwanted dependencies in just proper.
As the whole other_tools folder is meant to be excluded from
bootstrapping, also remove the bootstrap guards.
|
|
This also removes the need to call the GET_BRANCH_REFNAME critical
operation.
|
|
|
|
|
|
|
|
|
|
This provides the fix already performed for the just-mr script in
the internal just-mr as well. In short, this fix makes sure that the
computation of the content key for distdir repositories is
independent of the presence of the respective archives in CAS.
|
|
The subdir is required alongside the commit hash for a valid map
key when importing a Git repository into our Git cache.
|
|
Passing the logger by reference would require the caller to be
kept alive. Also, being a shared_ptr, the logger can be passed
by value at almost no cost.
|
|
Uses libarchive to unarchive the archives from CAS.
|
|
|
|
|
|
|
