summaryrefslogtreecommitdiff
path: root/src/buildtool/execution_api/remote/bazel
AgeCommit message (Collapse)Author
2024-02-26Employ blob splitting and splicing for endpoint dispatchingSascha Roloff
2024-02-26Implement blob splicing protocol at just client sideSascha Roloff
2024-02-26Implement blob chunking algorithm negotiationSascha Roloff
2024-02-26Refactor split and splice implementations.Sascha Roloff
Currently, the implementations of the split and splice operation are both hidden behind the Bazel API implementation. This was sufficient to implement splitting at the server and splicing at the client. In order to support the other direction of splitting at the client and splicing at the server while reusing their implementations, the code needs to be refactored. First, the functionality of split and splice are explicitly exposed at the general execution API interface and implemented in the sub APIs. Second, the implementations of split and splice are factored into a separate utils class.
2024-01-24just: Wrap IExecutionApi raw pointer argumentsPaul Cristian Sarbu
...in accordance to our coding style.
2023-12-21BazelCasClient: remove dead code.Alberto Sartori
The removed code has been superseeded by the CreateBatchRequestsMaxSize member function.
2023-12-21BazelCasClient::DoUploadBlobs: returns the count of the uploaded blobs...Alberto Sartori
...instead of the vector containing the digests of the uploaded blobs. The returned vector was never inspected by the callers, except for its size. The tests have been accordingly amended.
2023-12-21BazelNetwork: simplify logic of DoUploadBlobsAlberto Sartori
2023-12-21BazelCasClient: split DoBatchUploadBlobs into multiple calls...Alberto Sartori
...to honor the message limit imposed by GRPC.
2023-12-21BazelCasClient: split BatchReadBlobs into multiple calls...Alberto Sartori
...to honor the maxBatchTransferSize in grpc calls.
2023-12-21BazelCasClient: Split FindMissingBlobs in multiple batches...Alberto Sartori
...such that each request does not exceeds the message limit imposed by GRPC.
2023-12-21SplitBlob: improve error message in case of a failureAlberto Sartori
2023-12-21BazelCasClient: define new templated CreateBatchRequestsMaxSize member function.Alberto Sartori
This function will ensure that each request does not exceeds the maximum message size, currently set by kMaxBatchTransferSize in the message_limits library.
2023-12-21BazelCasClient::UpdateSingleBlob: Emit a log entry upon failureAlberto Sartori
2023-12-21bazel_network: use message_limits libraryAlberto Sartori
2023-12-13bazel client: use the retry strategy, provided by the retry lib, when doing ↵Alberto Sartori
the rpc calls.
2023-12-13bytestream client: log more info in case of a failureAlberto Sartori
2023-12-13Add debug output for blob splittingSascha Roloff
2023-12-12Filesystem: Fix copy overwrite of symlink with fileOliver Reiche
... and improve log messages in case of failure.
2023-11-30Splice blobs on diskKlaus Aehlig
When using blob splitting, we expect to create a potentially large file. Therefore reconstruct it from its parts on disk rather than keeping the whole file in memory.
2023-11-22Implement blob splitting protocol on just client sideSascha Roloff
2023-11-07Bazel API: implement ParallelRetrieveToCasKlaus Aehlig
... using thread-based parallelism for the blobs of each tree.
2023-09-13remote: Extract port parsing in own libraryPaul Cristian Sarbu
2023-09-13remote: Extract common client implementation utils in own libraryPaul Cristian Sarbu
2023-08-24Execution: only take complete actions from cacheKlaus Aehlig
... which are only actions that, besides giving exit code 0 also created all the outputs they promised to.
2023-07-27Execution API: support fetching a single artifact to memoryKlaus Aehlig
2023-07-26remote api: honor the --raw-tree optionKlaus Aehlig
2023-06-28Update remote execution api to preliminary version v2.3Sascha Roloff
In preparation for the introduction of our blob splitting protocol as extension to the remote execution api, we need to update the used remote execution api to a more recent version than v2.0.0. Since no new tags are available right now, we update to the preliminary protocol version v2.3 according to the following discussion: https://github.com/bazelbuild/remote-apis/issues/253
2023-06-26Allow non-upwards symlinks with Bazel APIPaul Cristian Sarbu
2023-06-26Execution response: Add output symlink pathsPaul Cristian Sarbu
2023-06-26bazel_msg_factory: Allow non-upwards symlinks in uploaded treesPaul Cristian Sarbu
2023-06-26ReadTree: Add check for non-upwards symlinks...Paul Cristian Sarbu
...as early as possible. This ensures that callers always receive only the tree entries for the supported object types. For the symlinks non-upwardness check we pass a lambda capturing the real backend of the tree entries, such that the symlinks can be read. Updates git_tree tests accordingly.
2023-06-06style: Use designated initializersPaul Cristian Sarbu
This feature has been introduced with C++20.
2023-05-03grpc: fix build fail with protobuf in debug modePaul Cristian Sarbu
2023-04-26imports: Switch to Microsoft GSL implementationOliver Reiche
... with two minor code base changes compared to previous use of gsl-lite: - dag.hpp: ActionNode::Ptr and ArtifactNode::Ptr are not wrapped in gsl::not_null<> anymore, due to lack of support for wrapping std::unique_ptr<>. More specifically, the move constructor is missing, rendering it impossible to use std::vector<>::emplace_back(). - utils/cpp/gsl.hpp: New header file added to implement the macros ExpectsAudit() and EnsureAudit(), asserts running only in debug builds, which were available in gsl-lite but are missing in MS GSL.
2023-03-23remote api: honor alternative for retrieving pathsKlaus Aehlig
2023-03-23execution api: support retrieving from more than one APIKlaus Aehlig
2023-03-15bazel api: fix include of proto files...Alberto Sartori
...bazel_types.hpp just requires the messages, not the grpc related services and classes
2023-03-15bazel client: remove BazelAcClient::UpdateActionResult...Alberto Sartori
...both buildbarn and just execute do not allow a direct upload to the action cache. Moreover, our tool does not support it, anyway, meaning it was dead code.
2023-02-22cleanup: Fix typos in code line comments and logging messagesPaul Cristian Sarbu
2023-02-22ExecutionApi: Retrieve to correct CAS directlyOliver Reiche
... by keeping track of each blob being a file or executable and storing it to the correct local physical CAS directory. The new flag is merely a hint and only used by the local execution API. Leaving it out will still correctly transfer the blob but may cause unnecessary duplicates in file CAS.
2023-02-02bytestream_client: use common header with settings shared by both server and ↵Alberto Sartori
client
2023-02-02bytestream_client: report failures on LogLevel::ErrorAlberto Sartori
2023-02-01Update fmtlib to version 9.1.0Klaus Aehlig
Co-authored-by: Michael Thies <mail@mhthies.de>
2023-01-20Add check for same CAS in CAS-to-CAS artifact retrievalSascha Roloff
2023-01-18authentication: add TLS and mTLS supportAlberto Sartori
2022-12-21Git CAS: Move Git tree ops to fake repo wrapper classPaul Cristian Sarbu
2022-11-24Fix root directory upload to honor tree invariant.Sascha Roloff
2022-11-04Remove unwanted native protocol prefixes from user-visible locationsSascha Roloff
2022-10-14Remove default value of ArtifactDigest::Create template parameterSascha Roloff
This enforces the explicit specification, which object type, either file or tree, should be used to create an artifact digest. This also avoids subtile errors at locations as in the previous commit, where files as well as trees are supposed to be handled, but digest creation mistakenly defaults to file object type.