summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2024-11-04Release 1.4.0v1.4.0Klaus Aehlig
2024-10-30Release 1.4.0~beta1v1.4.0-beta1Klaus Aehlig
2024-10-29Graph traverser: fix wrong wording in trace-level messageKlaus Aehlig
As we write the first message about the actual upload of a blob, we should use future rather than past tense. Also, again at trace level, add a message of successful completion, if we succeeded.
2024-10-29Clean up unused dependenciesKlaus Aehlig
2024-10-28Retry Execution on FAILED_PRECONDITIONKlaus Aehlig
The specification for this status code is as follows. One or more errors occurred in setting up the action requested, such as a missing input or command or no worker being available. The client may be able to fix the errors and retry. We routinely ensure all inputs are available to the remote execution before we start an action, so all prerequisites will be there on a compliant server, however might not actually be on a server where the CAS only has eventual consistency or is incorrect (due to old cache entries on CAS purge) in its answer to FindMissingBlobs. While we have no guarantee that a retry will help, we still retry; at least in the case of an unavailable worker or CAS entries not yet available due to eventual consistency, this will help. Also, we log at debug lvel the full response, including the repeated Any message. In this way, we can find out what useful information (if any) is sent by popular remote-execution services and implement more specific mitigations in the future.
2024-10-28BatchUploadBlobs: decrease log level for retried tasksKlaus Aehlig
In BatchUploadBlobs we accept short writes and, in case of no progress, fall back to single blob upload. Therefore, failure to upload blobs is not fatal and therefore should not be reported at error level. Decrease the log level accordingly: a protocol failure to upload is a performance-related event (as the retry needs additional time), catching an internal exception is something that shouldn't really happen, so we warn the user.
2024-10-28WithRetry: Support reduced log level for failuresKlaus Aehlig
In some situations, we use a retry strategy, but in case of complete failure have another way of attempting that task. In this case, we should not emmit an error message right away. Add support for this.
2024-10-25Add dependencies explicitly that are included directlyKlaus Aehlig
... instead of relying on those dependencies being pulled in indirectly.
2024-10-25Enable compatible mode for just-mr and SourceTree serve service...Paul Cristian Sarbu
...by using the new local api that can handle any remote endpoint, irrespective of protocol. Also ensure all tests for the serve service are now being run both in native and compatible modes.
2024-10-25Add new local execution api that can interact with any remotePaul Cristian Sarbu
...irespective of the used protocol. This api is useful in enabling just-mr and the SourceTree service of just serve to interact seamlessly with any remote-execution endpoint.
2024-10-25serve service: Set correct protocol-aware digests in responsesPaul Cristian Sarbu
When returning digests in responses, ensure they are of the type the remote would know. Compatible digests can be found vis the stored mappings from Git digests to bazel objects.
2024-10-25serve service: Use digest when requesting serve to set up a treePaul Cristian Sarbu
...instead of passing just the Git hash, which imposes the remote to always be native. The serve service proto file is updated accordingly.
2024-10-25serve service: Respond with digest for blob and tree checksPaul Cristian Sarbu
...to be able to interrogate remotes irrespective of protocol. When serve endpoint is active, it will provide the correct digest with which to interrogate the remote endpoint. Otherwise, for a compatible remote check the file mappings for the correct digest. The serve service proto file is updated accordingly.
2024-10-25serve service: Respond also with digest in serve repository tree RPCsPaul Cristian Sarbu
...besides the simple Git hash, if syncing was done. This way one can know what digest to ask for from the remote. The serve client also needs to now know what hash function the remote expects. The serve service proto file is updated accordingly.
2024-10-25serve proto: Fix inconsistencies in referring to GitPaul Cristian Sarbu
2024-10-25just-mr and SourceTree: Use new Git execution api instancePaul Cristian Sarbu
In just-mr: to instantiate the new Git api instance, both storage configs, as well as the compatible storage, need to be passed to the maps. While there, use more explicit naming schemes for the storage and CAS instances used. In serve: also acquire gc locks for the local storages when needed to instantiate the new Git api, which now has access to the CAS. In all these instances we also pass, as needed, the local api, which currently still operates only in native mode. This makes no difference currently, but will ensure less changes needed when the future compatible-aware local api will be used instead.
2024-10-25Add new Git execution api that can interact with any remotePaul Cristian Sarbu
...irrespective of the used protocol. This api is useful in enabling just-mr and the SourceTree service of just serve to interact seamlessly with any remote-execution endpoint.
2024-10-25serve service: Give SourceTreeService access to both local storagesPaul Cristian Sarbu
...native and compatible, even if currently only native is active. While there, be more explicit in which storage instance is being used.
2024-10-25just-mr: Instantiate both native and compatible storages...Paul Cristian Sarbu
...to be later passed to maps that might need them. This is a preparatory commit. Currently only the native storage is actively being used.
2024-10-25just-mr: Use explicit naming in default instantiated storagePaul Cristian Sarbu
As just-mr will always create Git roots, be explicit in the names of variables to state that the default storage created is native.
2024-10-25Add utility methods for digest mappingsPaul Cristian Sarbu
These allow to read and write file associations between known digests in different CAS instances.
2024-10-25BazelMsgFactory: Add method to create bazel Directory from Git treePaul Cristian Sarbu
2024-10-25StorageUtils: Add generation-aware rehashing ID file mapPaul Cristian Sarbu
Such a file could be used to store mappings of digests from CAS or Git cache to digests of different hash type that represent same content.
2024-10-25ArtifactDigest: Add getter for hash function typePaul Cristian Sarbu
2024-10-25just-mr: Small fixesPaul Cristian Sarbu
2024-10-25distdir_git_map.cpp: fix is_tree valueAlberto Sartori
2024-10-25content_cas_map: Fix missing return after logging fatalPaul Cristian Sarbu
2024-10-25ObjectInfo: Fix digest creation in FromStringPaul Cristian Sarbu
The digest tree check should take into account the protocol. Also add a TODO to point out the currently needed code duplication.
2024-10-25bugfix: bazel_network.cpp: fix handling of ongoing actionsAlberto Sartori
The rpc Execution::Execute returns stream google.longrunning.Operation. When the client reads the stream, the server can report that the operation is still in progress and the client has to wait. Before this patch, we were not checking for this particular condition. As a result, an ongoing action was interpreted as an execution failure.
2024-10-23expressions: enforce strict arguments for "join" and "join_cmd"v1.4.0-alpha+20241023Klaus Aehlig
... as described in the documentation. It was never intended to have a single string being an argument for those.
2024-10-21configured_target: honor --expression-log-limit when shortening representationKlaus Aehlig
2024-10-10Remove from OSS intersecting public-private dependenciesMaksim Denisov
2024-10-08just analyse: support dumping the action graph without originsKlaus Aehlig
The origins of actions are useful for understanding the action graph; if, however, the action graph is only to be used for further computaiton, this is unnecessary information. Therefore, add an option to dump the action graph without origins.
2024-10-08Name local variables using lower_caseMaksim Denisov
...and private members using lower_case_
2024-10-08Name value template parameters using kCamelCase.Maksim Denisov
2024-10-08Name global constants using kCamelCase.Maksim Denisov
2024-10-08Name constexpr variables using kCamelCase.Maksim Denisov
2024-10-08Name static constants using kCamelCase.Maksim Denisov
2024-10-08Name type template parameters using CamelCase.Maksim Denisov
2024-10-08Name classes, structs and enums using CamelCase.Maksim Denisov
2024-10-07Enable cppcoreguidelines-* checks.Maksim Denisov
2024-10-07Disable misc-no-recursion checkMaksim Denisov
...since we use recursion for trees a lot, but skip this check manually.
2024-10-07Enable misc-* checks.Maksim Denisov
2024-10-07Enable readability-* checks.Maksim Denisov
2024-10-07Enable readability-redundant-member-init check.Maksim Denisov
2024-10-07Enable google-* checks.Maksim Denisov
2024-10-07Enable modernize-* checks.Maksim Denisov
2024-10-07Enable bugprone-empty-catch check.Maksim Denisov
2024-10-07Enable bugprone-exception-escape checkMaksim Denisov
2024-10-07Enable bugprone-unhandled-exception-at-new check.Maksim Denisov