summaryrefslogtreecommitdiff
path: root/src/buildtool/execution_api
AgeCommit message (Collapse)Author
2024-02-13local_api: also for retrieve to memory, consult git api as fallbackKlaus Aehlig
2024-01-24just: Wrap IExecutionApi raw pointer argumentsPaul Cristian Sarbu
...in accordance to our coding style.
2024-01-19Add tree invariant check for just execute, when uploading treesSascha Roloff
2024-01-19Add digest validity check to the blob split commandSascha Roloff
2024-01-19Fix digest consistency check to also compare sizesSascha Roloff
2024-01-08bugfix: Add missing return in SetRemoteExecutionDispatchPaul Cristian Sarbu
This was a source of occasional std::bad_variant_access exceptions.
2023-12-21BazelCasClient: remove dead code.Alberto Sartori
The removed code has been superseeded by the CreateBatchRequestsMaxSize member function.
2023-12-21BazelCasClient::DoUploadBlobs: returns the count of the uploaded blobs...Alberto Sartori
...instead of the vector containing the digests of the uploaded blobs. The returned vector was never inspected by the callers, except for its size. The tests have been accordingly amended.
2023-12-21BazelNetwork: simplify logic of DoUploadBlobsAlberto Sartori
2023-12-21BazelCasClient: split DoBatchUploadBlobs into multiple calls...Alberto Sartori
...to honor the message limit imposed by GRPC.
2023-12-21BazelCasClient: split BatchReadBlobs into multiple calls...Alberto Sartori
...to honor the maxBatchTransferSize in grpc calls.
2023-12-21BazelCasClient: Split FindMissingBlobs in multiple batches...Alberto Sartori
...such that each request does not exceeds the message limit imposed by GRPC.
2023-12-21SplitBlob: improve error message in case of a failureAlberto Sartori
2023-12-21BazelCasClient: define new templated CreateBatchRequestsMaxSize member function.Alberto Sartori
This function will ensure that each request does not exceeds the maximum message size, currently set by kMaxBatchTransferSize in the message_limits library.
2023-12-21BazelCasClient::UpdateSingleBlob: Emit a log entry upon failureAlberto Sartori
2023-12-21bazel_network: use message_limits libraryAlberto Sartori
2023-12-21src/buildtool/execution_api/commmon: add message_limits libraryAlberto Sartori
Define the threshold for the grpc messages.
2023-12-13bazel client: use the retry strategy, provided by the retry lib, when doing ↵Alberto Sartori
the rpc calls.
2023-12-13bytestream client: log more info in case of a failureAlberto Sartori
2023-12-13Add debug output for blob splittingSascha Roloff
2023-12-12execution_api: Move dispatch file parser into separate utilityPaul Cristian Sarbu
2023-12-12Filesystem: Fix copy overwrite of symlink with fileOliver Reiche
... and improve log messages in case of failure.
2023-11-30Resolve inconsistencies in third-party headers include formatPaul Cristian Sarbu
2023-11-30local API: Implement upload fileKlaus Aehlig
... overriding the default implementation. In this way, files can be added directly to the local CAS without having to completely reside in memory.
2023-11-30Splice blobs on diskKlaus Aehlig
When using blob splitting, we expect to create a potentially large file. Therefore reconstruct it from its parts on disk rather than keeping the whole file in memory.
2023-11-30remote API: support uploading files from the file systemKlaus Aehlig
Allow implementations to have a specialized way of uploading a file owned by the build process to the applicable cas, both as file, as well as as tree object (in native mode). If no implementation is provided, the default is to read the file into memory and use the Upload method.
2023-11-27Refactoring RepositoryConfigPaul Cristian Sarbu
With the introduction of 'just serve', export targets can now be built also independently from one another based on their corresponding minimal repository configuration, as stored in the target cache key. In this context, this commit changes the RepositoryConfig usage from one global (static) instance to pointers passed as necessary throughout the code.
2023-11-22LocalAction: Fix staging of empty treesOliver Reiche
2023-11-22Implement blob splitting protocol on just client sideSascha Roloff
2023-11-22Implement blob splitting protocol on just server sideSascha Roloff
2023-11-15just-execute: verify the validity of all the hashes received over the wireAlberto Sartori
2023-11-07Bazel API: implement ParallelRetrieveToCasKlaus Aehlig
... using thread-based parallelism for the blobs of each tree.
2023-11-07remote api: support otpional parallel CAS synchronisationKlaus Aehlig
Allow implementations to use a given number of threads to carry out the synchronisation. In this way parallelism can be achieved even in situations where batch reading degrades as objects of unknown size have to be fetched; this situation typically occurs if a tree object has a large number of direct children that are blobs. If no implementation is provided, the default implementation is to fall back to the normal (sequential) CAS synchronisation.
2023-10-25just execute: fix race condition during garbage collection...Alberto Sartori
...of the internal cache used for keeping track of running operations.
2023-10-06CreateExecutionApi: define a new library to avoid code duplicationAlberto Sartori
2023-09-22RemoteServeConfig: Remove problematic inheritancePaul Cristian Sarbu
This was causing the remote serve address to overwrite the one set for remote execution. Also, to keep things clean, some common remote server-related methods and definitions were moved into their own library.
2023-09-15GitApi: implement RetrieveToCASAlberto Sartori
This functionality will be needed to upload git trees to a remote-execution end point by `just serve.
2023-09-13remote: Extract port parsing in own libraryPaul Cristian Sarbu
2023-09-13remote: Extract common client implementation utils in own libraryPaul Cristian Sarbu
2023-09-13targets: Add missing explicit dependencies and fix library namePaul Cristian Sarbu
2023-09-01fmt: Fix includes to only bring in the core APIPaul Cristian Sarbu
There is no need to use the full format API, so avoid increasing the compile time gratuitously.
2023-08-24Execution: only take complete actions from cacheKlaus Aehlig
... which are only actions that, besides giving exit code 0 also created all the outputs they promised to.
2023-08-11Move downstepping to subobjects to a libraryKlaus Aehlig
... in order to make that functionality reusable
2023-08-09Update fmt libraryKlaus Aehlig
2023-08-08just execute: Fix uncollected upwards symlinksPaul Cristian Sarbu
Upwards symlinks should still be collected from actions, even if only the non-upwards symlinks are supported artifact types. The client side is thus the one responsible with enforcing the non-upwardness condition.
2023-07-27Execution API: support fetching a single artifact to memoryKlaus Aehlig
2023-07-26remote api: honor the --raw-tree optionKlaus Aehlig
2023-07-21bugfix: stage symlinks as symlinks when creating an action directoryAlberto Sartori
Before this patch, when creating an action directory, symlinks were staged as regular files.
2023-07-17operation_cache: add missing includeMichael Thies
2023-07-13bytestream_server: store temporary files under the local-buil-rootAlberto Sartori
The bytestream server implementation (deployed by just execute) now stores the temporary files under $local_build_root/protocol-depenedent/generation-0 so that they can be garbage collected if "just exectue" is terminated before they are cleaned up.