| Age | Commit message (Collapse) | Author |
|---|
|
... by avoiding reusing temp dirs for execute. While we are
at it, also refactor LocalFetchViaTmpRepo() to create its
own empty temp dirs, that cannot be reused by the caller.
|
|
|
|
|
|
It is better to avoid empty catches when we have a reasonable way
to log an (even unlikely) exception, even more so if memory
allocations are involved.
|
|
When just serve has to create a distdir from files known to it,
there are two kind of hashes involved.
- The requests, as well as the resulting tree always refer to git
identifiers.
- Additionally, files can be stored in the local CAS of the serve
instance. This CAS might use a different hash, e.g., plain sha256.
Therefore, we cannot simply forward the hash from the request to
a lookup in our local CAS; fortunately, when we add a file to the
local CAS, we are told the value of the applicable hash, represented
as bazel digest, hence we can simply remember essentially this
value (converting it to unprefixed form when running in native
mode, as in the following we need the hash as it is part of an
artifact digest).
|
|
|
|
As serve and just-mr share their caching of description-tree
association, we also have to change this cache. Thanks to json
encoding before hashing, we know that the old and new hash keys
do not overlap, so the change is save. As distdirs also keep the
respective files in the git root, no new downlaoding will happen
either, hence no warning in the CHANGELOG is needed.
|
|
The just serve client-side and API methods, as well as just-mr
utilities should use the noexcept specifier.
|
|
|
|
Firstly, in ServeDistdirTree the tree of a distdir should be
synced from the corresponding witnessing Git repository (as is the
case with all root trees), but was wrongly trying to sync from the
local CAS.
Secondly, a status of SYNC_ERROR, according to the protocol, must
always ensure the tree field is set, but some setter calls were
missing.
Thirdly, ServeContent and ServeTree failed to report on
successfully syncing to remote if the blob or tree, respetively,
were uploaded from the local CAS.
Lastly, ServeDistdirTree contained some legacy code sections and
comments that were superfluous.
This commit fixes these issues.
|
|
This request is needed by just-mr in order to verify that it can
use a provided serve endpoint when setting up the repository roots.
|
|
|
|
|
|
Also cleans up the logging when parsing the serve service
configuration file.
|
|
The requests to retrieve the tree of a commit, archive, or distdir
also set up those trees in a way that the serve endpoint can later
build against them, besides allowing just-mr to set up roots
locally. Therefore, if the witnessing entity (Git commit, content
blob, or distdir, respectively) is known to the serve endpoint,
then failing to set up the root tree there should result in a
failure also of the just-mr setup on the client side.
|
|
For archives and Git repositories we should ensure that not finding
the witnessing entity (archive content blob or Git commit,
respectively) results in a distinct status in the response to a
request that sets up roots on the serve endpoint. This will allow
just-mr to better handle its interaction with the serve endpoint.
|
|
|
|
pragma-related RPCs
|
|
|
|
|
|
The purpose of the requests for the tree of an archive, commit, or
distdir also includes making those trees available for future
builds on the serve endpoint, which currently means being in a
known Git repository.
This commit ensures the distdir tree reqeust also includes the
import of the resulting tree from CAS into the Git cache (if the
tree is not already in a Git repsoitory).
|
|
In order for the serve endpoint to correctly dispatch a build to
the correct remote-execution endpoint, the platform properties and
dispatch list for a build need to be passed explicitly to the
executor (via the graph traverser instance) instead of always being
taken from the RemoteExecutionConfig struct.
This commit implements these changes, including updating existing
tests accordingly.
|
|
|
|
|
|
|
|
|
|
|
|
When serving the tree of an archive, we should check also in the
local CAS for the content blob.
|
|
|
|
... before trying to upload from local storage to the remote CAS.
Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
|
|
The error log level should be reserved for events that inevitably
lead to a failed build. A failure to receive a target from the serve
endpoint, however, is not such a case; for performance reasons,
and also to have the same artifacts as everyone else in the case
of non-reproducible dependencies, just inquires the serve end point
for every export target whenever a serve end point is given. In
this case, the build just continues even if the serve end point
is, e.g., lacking a certain root.
|
|
|
|
|
|
|
|
|
|
This commit simply defines the logic needed for progress reporting.
|
|
request
|
|
|
|
The request should only be restricted to the minimal information
needed by the remote to answer it. In particular, the execution
endpoint address should not be transmitted.
|
|
Only the client needs to make sure that the remote execution
endpoint is set in the case 'just serve' acts also as
'just execute', i.e., when a remote execution endpoint is not
specified, while for setting up the serve server a missing
execution endpoint should remain unset.
|
|
In the scenario when 'just serve' acts as 'just execute', the
remote execution endpoint returned by the serve service should be
allowed to be empty.
In this case, from the server's perspective, there is nothing to be
checked, however a client might still want to ensure that its own
configured serve and execution endpoints match.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
