| Age | Commit message (Collapse) | Author |
|---|
|
...instead of std::optional<gsl::not_null<IExecutionApi const*>>
|
|
...and replace verbose constructions.
|
|
...and use it in ApiBundle.
|
|
...instead of not_null const ptr.
|
|
...instead of not_null const ptr.
|
|
|
|
|
|
|
|
...to properly fallback to local api if needed.
|
|
|
|
...to enable validation of received blobs.
|
|
|
|
...bypassing BazelNetwork layer.
|
|
|
|
|
|
IExecutionApi::UploadFile was introduced to handle adding of a temporary file to the storage. The call was redirected to LocalCAS internal methods that do the similar thing, so this virtual method can be removed.
|
|
Update logic populating containers to use the new method which
is aware of the maximum transfer limit.
|
|
...by proactively calling the Upload method for large blobs and
containers as soon as the transfer limit is reached.
|
|
This unifies the signature of all uploader functions consuming a
BlobContainer type.
|
|
As the ContentBlobContainer is used to store actual content needed
to be transferred, it is useful to automatically keep track of the
running (bytes) size of the data being stored.
|
|
...to reduce the "price" of copying.
|
|
...instead of BazelBlobContainer to not bring bazel_re::Digest to IExecutionApi.
|
|
|
|
|
|
|
|
...where the template parameter is the type of a digest.
|
|
...instead of various iterators.
|
|
...fixing potentially dangerous code (evaluation order is unspecified).
|
|
|
|
|
|
...in LocalApi and BazelApi.
|
|
...in LocalApi and BazelApi.
|
|
This reduces the code duplication between the local and bazel APIs
and improves code maintainability.
|
|
|
|
We can avoid doing extra work in converting between bazel digests
and artifact digests by actually using the API interface.
|
|
|
|
|
|
|
|
Once a RepositoryConfig instance gets populated, it must never be
changed again. Therefore, all functions accepting these instances
should only take them as pointers to const.
|
|
The Emit method of the Logger class, when called with a string as
second argument, expects it to be a format string. It should be
considered a programming error to pass a string variable as that
argument without knowing for certain that it does not contain any
format escape character ('{', '}'); instead, one should be
conservative and use the blind format string "{}" as second
argument and pass the unknown string variable as third argument.
|
|
|
|
|
|
As we use chunking also for reducing storage, we have to consider
the overhead of block devices which is in the order of kB per file.
So our target chunk size should be at least 2 orders of magnitude
above this. This suggests to minimally aim for a chunk size of
128kB, a target size that also has the advantage the that maximal
chunk size associated with this size is 1MB which is still well
below the maximal transmission size of grpc allowing us to avoid
the streaming API.
As we're scaling everything up by a factor of 16, we also have
to increase the number of bits in the involved masks by 4. We use
this to also extend the window size by using the 2 most significant
octets. Following the advice of the paper proposing FastCDC to
spread out the ones roughly equally suggests 0x4444 as a suitable
value for the two most significant octets.
We also change the suggested extension of the remote-execution API
accordingly. As the precise parameters for FastCDC when announced
over the remote-execution APIs are still under discussion upstream,
we simplify the name to not mention the target size.
|
|
|
|
|
|
|
|
|
|
Main culprits:
- std::size_t, std::nullptr_t, and NULL require <cstddef>
- std::move and std::forward require <utility>
- unordered maps and sets require respective includes
- std::for_each and std::all_of require <algorithm>
|
|
Some of the more specific issues addressed:
- missing log_level target/include
- header-only libs wrongly marking deps as private
- missing/misplaced gsl includes
|
|
... so that any updates of the local-build-root layout are correctly
taken into account. In particular, this change also moves the temporary
directory under the emphemeral root, allowing more quick clean up.
Co-authored-by: Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com>
|
