| Age | Commit message (Collapse) | Author |
|---|
|
|
|
The context is passed by not_null const pointer in order to avoid
binding to temporaries, and it or parts of it get stored by const
ref where needed.
|
|
|
|
As it is used by just execute only, instantiate it inside the
ExecutionServer, which reads and writes to the cache map, and pass
a const ref to OperationsServer, which only queries.
|
|
|
|
|
|
...to track changes during refactoring easier.
|
|
Use a builder pattern for creation and validation, in a manner that
allows also other authentication methods to be added in the future
besides the current TLS/SSL.
The main Auth instances are built early and then passed by not_null
const pointers, to avoid passing temporaries, replacing the previous
Auth::TLS instances passed by simple nullable const pointers. Where
needed, these passed Auth instances are also stored, by const ref.
Tests also build Auth instances as needed, either with the default
'no certification' or from the test environment arguments.
|
|
|
|
...to more easily keep track of changes during refactoring.
|
|
|
|
|
|
...instead of not_null const ptr.
|
|
|
|
|
|
The Emit method of the Logger class, when called with a string as
second argument, expects it to be a format string. It should be
considered a programming error to pass a string variable as that
argument without knowing for certain that it does not contain any
format escape character ('{', '}'); instead, one should be
conservative and use the blind format string "{}" as second
argument and pass the unknown string variable as third argument.
|
|
As we use chunking also for reducing storage, we have to consider
the overhead of block devices which is in the order of kB per file.
So our target chunk size should be at least 2 orders of magnitude
above this. This suggests to minimally aim for a chunk size of
128kB, a target size that also has the advantage the that maximal
chunk size associated with this size is 1MB which is still well
below the maximal transmission size of grpc allowing us to avoid
the streaming API.
As we're scaling everything up by a factor of 16, we also have
to increase the number of bits in the involved masks by 4. We use
this to also extend the window size by using the 2 most significant
octets. Following the advice of the paper proposing FastCDC to
spread out the ones roughly equally suggests 0x4444 as a suitable
value for the two most significant octets.
We also change the suggested extension of the remote-execution API
accordingly. As the precise parameters for FastCDC when announced
over the remote-execution APIs are still under discussion upstream,
we simplify the name to not mention the target size.
|
|
|
|
|
|
|
|
|
|
Main culprits:
- std::size_t, std::nullptr_t, and NULL require <cstddef>
- std::move and std::forward require <utility>
- unordered maps and sets require respective includes
- std::for_each and std::all_of require <algorithm>
|
|
Some of the more specific issues addressed:
- missing log_level target/include
- header-only libs wrongly marking deps as private
- missing/misplaced gsl includes
|
|
... so that any updates of the local-build-root layout are correctly
taken into account. In particular, this change also moves the temporary
directory under the emphemeral root, allowing more quick clean up.
Co-authored-by: Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com>
|
|
... as the fs_utils have a lot more dependencies making them usable
in less places. Moreover, this function also serves to shape the
layout of the local build root and hence is more appropriately
placed in the config anyway.
|
|
|
|
|
|
|
|
Currently, the implementations of the split and splice operation are both
hidden behind the Bazel API implementation. This was sufficient to implement
splitting at the server and splicing at the client. In order to support the
other direction of splitting at the client and splicing at the server while
reusing their implementations, the code needs to be refactored. First, the
functionality of split and splice are explicitly exposed at the general
execution API interface and implemented in the sub APIs. Second, the
implementations of split and splice are factored into a separate utils class.
|
|
|
|
|
|
|
|
... glibc provides synchronization stubs for single-threaded
environments as weak symobls. When linking pthreads, these
weak symbols must be replaced by the strong symbols provided
by the pthread library. For dynamically linking pthreads,
this is done automatically. However, to support this for
static linking, we must ensure to link the whole archive.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
...of the internal cache used for keeping track of running operations.
|
|
|
|
|
|
There is no need to use the full format API, so avoid increasing
the compile time gratuitously.
|
|
|
|
The bytestream server implementation (deployed by just execute) now
stores the temporary files under
$local_build_root/protocol-depenedent/generation-0
so that they can be garbage collected if "just exectue" is terminated
before they are cleaned up.
|
|
|
|
In preparation for the introduction of our blob splitting protocol as extension
to the remote execution api, we need to update the used remote execution api to
a more recent version than v2.0.0. Since no new tags are available right now,
we update to the preliminary protocol version v2.3 according to the following
discussion: https://github.com/bazelbuild/remote-apis/issues/253
|
|
|
|
... with two minor code base changes compared to previous
use of gsl-lite:
- dag.hpp: ActionNode::Ptr and ArtifactNode::Ptr are not
wrapped in gsl::not_null<> anymore, due to lack of support
for wrapping std::unique_ptr<>. More specifically, the
move constructor is missing, rendering it impossible to
use std::vector<>::emplace_back().
- utils/cpp/gsl.hpp: New header file added to implement the
macros ExpectsAudit() and EnsureAudit(), asserts running
only in debug builds, which were available in gsl-lite but
are missing in MS GSL.
|
|
|
