| Age | Commit message (Collapse) | Author |
|---|
|
...instead of not_null const ptr.
|
|
|
|
|
|
|
|
|
|
...and adjust interfaces.
|
|
...instead of intialization of the singleton.
|
|
...and adjust interfaces.
|
|
...instead of using singleton calls.
|
|
...instead of using singleton calls.
|
|
|
|
...to track changes during refactoring easier.
|
|
...to enable validation of received blobs.
|
|
|
|
Update logic populating containers to use the new method which
is aware of the maximum transfer limit.
|
|
... to allow generating symlinks as part of a rule, as it is already described
in our documentation.
|
|
...to reduce the "price" of copying.
|
|
...instead of BazelBlobContainer to not bring bazel_re::Digest to IExecutionApi.
|
|
|
|
...where the template parameter is the type of a digest.
|
|
...instead of various iterators.
|
|
|
|
|
|
|
|
While this can already be expressed by an "if" statement, having
a dedicated function for logical negation makes some expressions
more readable.
|
|
... using, also for the "then" branch, the empty list as default.
In this way, this statement not only more symmetric, but also
allows shorter representations of some typical expressions.
|
|
Lists are somtimes used in configurations as replacement for tuples.
Providing length gives an easy way to detect usage errors.
|
|
|
|
|
|
During compactification, invalid entries must be deleted.
|
|
During garbage collection split and remove from the storage every entry that is larger than a threshold.
|
|
During garbage collection remove from the storage every entry that has the large entry.
|
|
|
|
and trees.
|
|
executable files during splitting.
|
|
As we use chunking also for reducing storage, we have to consider
the overhead of block devices which is in the order of kB per file.
So our target chunk size should be at least 2 orders of magnitude
above this. This suggests to minimally aim for a chunk size of
128kB, a target size that also has the advantage the that maximal
chunk size associated with this size is 1MB which is still well
below the maximal transmission size of grpc allowing us to avoid
the streaming API.
As we're scaling everything up by a factor of 16, we also have
to increase the number of bits in the involved masks by 4. We use
this to also extend the window size by using the 2 most significant
octets. Following the advice of the paper proposing FastCDC to
spread out the ones roughly equally suggests 0x4444 as a suitable
value for the two most significant octets.
We also change the suggested extension of the remote-execution API
accordingly. As the precise parameters for FastCDC when announced
over the remote-execution APIs are still under discussion upstream,
we simplify the name to not mention the target size.
|
|
|
|
Also adds an appropriate test for this method.
|
|
Also extends the tests accordingly.
|
|
|
|
|
|
|
|
For splicing of large objects from external sources additional checks are performed:
* The digest of the spliced result must be equal to the expected digest;
* The parts of a spliced tree must be in the storage.
Tested:
* Regular splicing of large objects;
* If the result is unexpected, splicing fails;
* If some parts of a tree are missing, splicing fails.
|
|
* Uplink parts of the large entry before entry itself;
* Uplink large entries in LargeObjectCAS::GetEntryPath to not split things two times;
* Promote spliced tree during uplinking of a large tree entry to properly promote parts of the tree;
* Uplink large entries in LocalUplink{Blob, Tree} to support proper uplinking in Action Cache and Target Cache;
Tested:
* Uplink large blobs and trees;
* Uplink a large object that depends on other large objects.
|
|
Implicitly reconstruct objects during regular uplinking of Blobs/Trees.
|
|
* Add LargeObjectCAS fields for files and trees to LocalCAS;
* Add logic for splitting objects located in the main storage.
Tested:
Splitting of large, small and empty objects.
|
|
Numerical values are used at some places in justbuild: as value for
timeout scaling, as well as by the "range" expression that is used,
e.g., to define repreated test runs. Therefore, improve support
for numerical values by adding basic operations.
|
|
For an absent export target, the first step of analysis is to ask
serve for the flexible variables. The answer to this request is,
however, independent of the configuration for this target. So we
can avoid calls by caching the answer in an additional map.
|
|
This allows to be explicit and thus have better control on where
messages get logged.
|
|
Some of the more specific issues addressed:
- missing log_level target/include
- header-only libs wrongly marking deps as private
- missing/misplaced gsl includes
|
