| Age | Commit message (Collapse) | Author |
|---|
|
The initial assumption before introducing the native remote execution protocol
was that only blobs were transmitted over the wire and put in a blob store.
With the native remote execution protocol, also trees can be transmitted over
the wire and they are put in a different store than blobs. Different hashing
functions are used for blobs and trees and need to be calculated accordingly
for data to be put in the correct store. This distinction of blobs and trees is
now implemented. Additionally, a warning is added that checks equality
between the requested digest and the digest computed by the received
content. This check makes sense to detect, e.g., corrupted messages,
unexpected transmission termination, or remote server failure.
|
|
Signed-off-by: Goetz Brasche <goetz.brasche@huawei.com>
|
|
|
|
When a tree is taken from a git root, it is not necessarily known on
the remote site. So, as any missing artifact it has to be uploaded,
recursively uploading the parts to keep the tree invariant. The
function RetrieveToCas was doing the correct recursiv pattern, however
inspecting trees incorrectly using the function ReadTreeInfos; the
latter function, however, was obtaining all the leafs of the tree
as is needed for a compatible action-input description. Add and
use a function that reads the direct contents of a tree.
|
|
... as for remote execution, the map entries are only used
for the `install` subcommand. For local execution, much less
tree objects are read from CAS when using this map. However,
the performance benefit is barely measurable and therefore
we rather remove this map entirely to reduce complexity.
|
|
... to align with the original idea of caching a flat list
of blob objects, without the need to recursively traverse
any trees. Consequently, we cannot create any map entry in
places where we do not have all sub-tree entries at hand
(e.g., LocalAPI, BazelAPI, BazelResponse).
|
|
... which is not using `Directory` messages nor does it
support the `GetTree()` rpc.
|
|
... so that any infra structure problem on the remote execution
is reported by default.
|
|
... to emphasize that this is the extensional action identifier,
not the intensional one.
|
|
|
|
between different CASes
|
|
|
|
|
|
... otherwise actual blob size might exceed the maximum
transfer size of the CAS client. Therefore, we always have
to use the bytestream client if the size is unknown.
|
|
native mode
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The fact that we happen to use the same protocol as bazel does,
does not mean we are bazel. After all, the remote-build-execution
protocol is meant as a generic protocol. Hence change the names in
logs to avoid confusions.
|
|
... to allow for overwriting
|
|
|
|
|
|
... which was accessing the wrong artifacts in the list if the
artifact list contains a mixture of files and trees.
|
|
... as wrongfully only sub-tree entries were added to
locally cached trees, although they should also store
entries for files and executables.
|
|
This is the initial version of our tool that is able to
build itself. In can be bootstrapped by
./bin/bootstrap.py
Co-authored-by: Oliver Reiche <oliver.reiche@huawei.com>
Co-authored-by: Victor Moreno <victor.moreno1@huawei.com>
|
