| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
The improved GC implementation uses refactored storage
classes instead of directly accessing "unknown" file paths.
The required storage class refactoring is quite substantial
and outlined in the following paragraphs.
The module `buildtool/file_system` was extended by:
- `ObjectCAS`: a plain CAS implementation for
reading/writing blobs and computing digests for a given
`ObjectType`. Depending on that type, files written to the
file system may have different properties (e.g., the x-bit
set) or the digest may be computed differently (e.g., tree
digests in non-compatible mode).
A new module `buildtool/storage` was introduced containing:
- `LocalCAS`: provides a common interface for the "logical
CAS", which internally combines three `ObjectCAS`s, one
for each `ObjectType` (file, executable, tree).
- `LocalAC`: implements the action cache, which needs the
`LocalCAS` for storing cache values.
- `TargetCache`: implements the high-level target cache,
which also needs the `LocalCAS` for storing cache values.
- `LocalStorage`: combines the storage classes `LocalCAS`,
`LocalAC`, and `TargetCache`. Those are initialized with
settings from `StorageConfig`, such as the build root base
path or number of generations for the garbage collector.
`LocalStorage` is templated with a Boolean parameter
`kDoGlobalUplink`, which indicates that, on every
read/write access, the garbage collector should be used
for uplinking across all generations (global).
- `GarbageCollector`: responsible for garbage collection and
the global uplinking across all generations. To do so, it
employs instances of `LocalStorage` with `kDoGlobalUplink`
set to false, in order to avoid endless recursion. The
actual (local) uplinking within two single generations is
performed by the corresponding storage class (e.g.,
`TargetCache` implements uplinking of target cache entries
between two target cache generations etc.). Thereby, the
actual knowledge how data should be uplinked is
implemented by the instance that is responsible for
creating the data in the first place.
|
|
|
|
|
|
|
|
... as internally we do not do the Tree-dance. Instead, we
directly store the root Directory digest for ActionResult's
OutputDirectories.
|
|
|
|
|
|
|
|
... instead of using pipes that are prone to get full. Also
increase log level to Trace.
|
|
|
|
... on reporting the exec command; json encoding is already
quoting enough.
|
|
|
|
The forwarding of the log-limit argument is only useful, if
the value is different from the default.
|
|
A local launcher should be forwarded, if it is different to
the default launcher (which is ["env", "--"]); in particular,
an explictly empy launcher should be forwarded.
|
|
... and, in this way, avoid duplication of these constants that
might lead to inconsistencies between those two tools later.
|
|
|
|
remote endpoint
|
|
... now that they no longer depend on an external remote
execution.
|
|
... to bring their own remote-execution endpoint.
|
|
In order to keep our tests self-contained, do not rely on
an external remote-execution service to be present; instead,
use `just execute` to provide the remote execution service.
|
|
in the tree CAS.
Before this patch, after the checking in the tree cas, also the blob
cas was checked leading to a NOT-FOUND error.
|
|
google::longrunning::Operations::GetOperation
For each action that is executed, an entry is added to a shared thread
safe cache. Once the number of operations stored exceeds twice 2^n,
where n is given by the option --log-operations-threshold, at most 2^n
operations will be removed, in a FIFO scheme.
|
|
|
|
... by reporting unsupported config values explicitly and fixing
inconsistencies.
|
|
Two command line arguments added: one to disable SSL verification,
the other to override the path to the default system CA certificate
bundle.
|
|
|
|
|
|
|
|
... both with respect to just options and the man page specifications.
Option -L of just-mr was reassigned as a short name for
--local-launcher, matching its use in just. Its place is now
correctly held by the full name option --checkout-locations, as
per the section-1 man page.
|
|
|
|
|
|
|
|
|
|
|
|
Map for a workspace root given as a known git tree id which is
promised to be obtainable by executing a specified command.
|
|
|
|
Also update just-mr section-1 man page
|
|
Make clear what fields the defined boolean values refer to.
|
|
|
|
|
|
|
|
... and, in this way, ensure the test also tests properly in
absence of an external remote execution system.
|
|
... similar to ["@", "rules", "shell/test", "script"], but implicitly
starting a remote execution in the background, passing the information
about that endpoint to the test via an environment variable.
|
|
|
|
Reporting and counting should be done as early as possible, once
known that there is nothing cached.
|
|
|
|
|
|
|
