Age | Commit message (Collapse) | Author |
|
|
|
|
|
As we write the first message about the actual upload of a blob,
we should use future rather than past tense. Also, again at trace
level, add a message of successful completion, if we succeeded.
|
|
|
|
The specification for this status code is as follows.
One or more errors occurred in setting up the action requested,
such as a missing input or command or no worker being available.
The client may be able to fix the errors and retry.
We routinely ensure all inputs are available to the remote execution
before we start an action, so all prerequisites will be there on a
compliant server, however might not actually be on a server where
the CAS only has eventual consistency or is incorrect (due to old
cache entries on CAS purge) in its answer to FindMissingBlobs.
While we have no guarantee that a retry will help, we still retry;
at least in the case of an unavailable worker or CAS entries not
yet available due to eventual consistency, this will help. Also,
we log at debug lvel the full response, including the repeated Any
message. In this way, we can find out what useful information (if
any) is sent by popular remote-execution services and implement
more specific mitigations in the future.
|
|
In BatchUploadBlobs we accept short writes and, in case of no progress,
fall back to single blob upload. Therefore, failure to upload blobs
is not fatal and therefore should not be reported at error level.
Decrease the log level accordingly: a protocol failure to upload is
a performance-related event (as the retry needs additional time),
catching an internal exception is something that shouldn't really
happen, so we warn the user.
|
|
In some situations, we use a retry strategy, but in case of complete
failure have another way of attempting that task. In this case, we
should not emmit an error message right away. Add support for this.
|
|
... instead of relying on those dependencies being pulled in
indirectly.
|
|
...by using the new local api that can handle any remote endpoint,
irrespective of protocol.
Also ensure all tests for the serve service are now being run both
in native and compatible modes.
|
|
...irespective of the used protocol.
This api is useful in enabling just-mr and the SourceTree service
of just serve to interact seamlessly with any remote-execution
endpoint.
|
|
When returning digests in responses, ensure they are of the type
the remote would know. Compatible digests can be found vis the
stored mappings from Git digests to bazel objects.
|
|
...instead of passing just the Git hash, which imposes the remote
to always be native.
The serve service proto file is updated accordingly.
|
|
...to be able to interrogate remotes irrespective of protocol.
When serve endpoint is active, it will provide the correct digest
with which to interrogate the remote endpoint. Otherwise, for a
compatible remote check the file mappings for the correct digest.
The serve service proto file is updated accordingly.
|
|
...besides the simple Git hash, if syncing was done. This way one
can know what digest to ask for from the remote. The serve client
also needs to now know what hash function the remote expects.
The serve service proto file is updated accordingly.
|
|
|
|
In just-mr: to instantiate the new Git api instance, both storage
configs, as well as the compatible storage, need to be passed to
the maps. While there, use more explicit naming schemes for the
storage and CAS instances used.
In serve: also acquire gc locks for the local storages when needed
to instantiate the new Git api, which now has access to the CAS.
In all these instances we also pass, as needed, the local api, which
currently still operates only in native mode. This makes no
difference currently, but will ensure less changes needed when the
future compatible-aware local api will be used instead.
|
|
...irrespective of the used protocol.
This api is useful in enabling just-mr and the SourceTree service
of just serve to interact seamlessly with any remote-execution
endpoint.
|
|
...native and compatible, even if currently only native is active.
While there, be more explicit in which storage instance is being
used.
|
|
...to be later passed to maps that might need them.
This is a preparatory commit. Currently only the native storage is
actively being used.
|
|
As just-mr will always create Git roots, be explicit in the names
of variables to state that the default storage created is native.
|
|
These allow to read and write file associations between known
digests in different CAS instances.
|
|
|
|
Such a file could be used to store mappings of digests from CAS or
Git cache to digests of different hash type that represent same
content.
|
|
|
|
|
|
|
|
|
|
The digest tree check should take into account the protocol.
Also add a TODO to point out the currently needed code duplication.
|
|
The rpc Execution::Execute returns stream
google.longrunning.Operation. When the client reads the stream, the
server can report that the operation is still in progress and the
client has to wait. Before this patch, we were not checking for this
particular condition. As a result, an ongoing action was interpreted
as an execution failure.
|
|
... as described in the documentation. It was never intended to have
a single string being an argument for those.
|
|
|
|
|
|
The origins of actions are useful for understanding the action
graph; if, however, the action graph is only to be used for further
computaiton, this is unnecessary information. Therefore, add an
option to dump the action graph without origins.
|
|
...and private members using lower_case_
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
...since we use recursion for trees a lot, but skip this check manually.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|