| Age | Commit message (Collapse) | Author |
|---|
|
As the internal distdir data structure now supports the
executable bit, it is also expressive enough to support
foreign-file repositories. Hence we can use this cache,
getting potentially more cache hits.
|
|
We ensure that for each export target to be written to the target
cache all its implied export targets are written to the target
cache first. This ensures that the target cache maintains its
consistency at all times with respect to export target
dependencies.
|
|
... by uplinking them appropriately.
|
|
|
|
|
|
|
|
|
|
TargetCache...
...backed by the same CAS, but the FileStorage uses the given
shard. This is particularly useful for the just-serve server
implementation, since the sharding must be performed according to the
client's request and not following the server configuration.
|
|
This constructor is used by TargetService::ServeTarget
|
|
... which was accidentially a list of (a single) object,
instead of only a single JSON object.
|
|
While we don't want to fail if the 'checkouts' map values are not
strings, we shouldn't just accept non-string values either,
instead we should warn the user and continue without them.
|
|
This is required in order to make them available to 'just serve'
in a minimal just installation.
|
|
The bytestream server implementation (deployed by just execute) now
stores the temporary files under
$local_build_root/protocol-depenedent/generation-0
so that they can be garbage collected if "just exectue" is terminated
before they are cleaned up.
|
|
Extend the configuration data structure by a dispatch list of endpoints
to chose based on the first match of the execution properties.
|
|
|
|
|
|
...as early as possible. This ensures that callers always receive
only the tree entries for the supported object types.
For the symlinks non-upwardness check we pass a lambda capturing
the real backend of the tree entries, such that the symlinks can
be read.
Updates git_tree tests accordingly.
|
|
This feature has been introduced with C++20.
|
|
As local execution is tightly coupled to storage, also specify the
layout in the storage configuration. In this way, we have a central
place specifying the layout of just's cache directory and avoid
accidentally getting into conflicting situations. While there, also
move the execution root under the generation regime, to ensure that
left-over execution directories (e.g., after a forceful termination
of the program) eventually get cleaned up by garbage collection.
|
|
... with two minor code base changes compared to previous
use of gsl-lite:
- dag.hpp: ActionNode::Ptr and ArtifactNode::Ptr are not
wrapped in gsl::not_null<> anymore, due to lack of support
for wrapping std::unique_ptr<>. More specifically, the
move constructor is missing, rendering it impossible to
use std::vector<>::emplace_back().
- utils/cpp/gsl.hpp: New header file added to implement the
macros ExpectsAudit() and EnsureAudit(), asserts running
only in debug builds, which were available in gsl-lite but
are missing in MS GSL.
|
|
In this way, we have the whole layout of the local build root
consolidated in one place. Moreover, in this way, the location
of the git root is also available to the build tool itself and
can, e.g., be used as fallback CAS.
|
|
As those functions indicate success, it is up to the caller to
decide if the error was fatal or not. Reporting an error nevertheless
might result in error messages on successful operaitons, which is
confusing for the user.
|
|
|
|
The improved GC implementation uses refactored storage
classes instead of directly accessing "unknown" file paths.
The required storage class refactoring is quite substantial
and outlined in the following paragraphs.
The module `buildtool/file_system` was extended by:
- `ObjectCAS`: a plain CAS implementation for
reading/writing blobs and computing digests for a given
`ObjectType`. Depending on that type, files written to the
file system may have different properties (e.g., the x-bit
set) or the digest may be computed differently (e.g., tree
digests in non-compatible mode).
A new module `buildtool/storage` was introduced containing:
- `LocalCAS`: provides a common interface for the "logical
CAS", which internally combines three `ObjectCAS`s, one
for each `ObjectType` (file, executable, tree).
- `LocalAC`: implements the action cache, which needs the
`LocalCAS` for storing cache values.
- `TargetCache`: implements the high-level target cache,
which also needs the `LocalCAS` for storing cache values.
- `LocalStorage`: combines the storage classes `LocalCAS`,
`LocalAC`, and `TargetCache`. Those are initialized with
settings from `StorageConfig`, such as the build root base
path or number of generations for the garbage collector.
`LocalStorage` is templated with a Boolean parameter
`kDoGlobalUplink`, which indicates that, on every
read/write access, the garbage collector should be used
for uplinking across all generations (global).
- `GarbageCollector`: responsible for garbage collection and
the global uplinking across all generations. To do so, it
employs instances of `LocalStorage` with `kDoGlobalUplink`
set to false, in order to avoid endless recursion. The
actual (local) uplinking within two single generations is
performed by the corresponding storage class (e.g.,
`TargetCache` implements uplinking of target cache entries
between two target cache generations etc.). Thereby, the
actual knowledge how data should be uplinked is
implemented by the instance that is responsible for
creating the data in the first place.
|
