summaryrefslogtreecommitdiff
path: root/src/buildtool/execution_api/remote
AgeCommit message (Collapse)Author
2024-01-24just: Wrap IExecutionApi raw pointer argumentsPaul Cristian Sarbu
...in accordance to our coding style.
2024-01-08bugfix: Add missing return in SetRemoteExecutionDispatchPaul Cristian Sarbu
This was a source of occasional std::bad_variant_access exceptions.
2023-12-21BazelCasClient: remove dead code.Alberto Sartori
The removed code has been superseeded by the CreateBatchRequestsMaxSize member function.
2023-12-21BazelCasClient::DoUploadBlobs: returns the count of the uploaded blobs...Alberto Sartori
...instead of the vector containing the digests of the uploaded blobs. The returned vector was never inspected by the callers, except for its size. The tests have been accordingly amended.
2023-12-21BazelNetwork: simplify logic of DoUploadBlobsAlberto Sartori
2023-12-21BazelCasClient: split DoBatchUploadBlobs into multiple calls...Alberto Sartori
...to honor the message limit imposed by GRPC.
2023-12-21BazelCasClient: split BatchReadBlobs into multiple calls...Alberto Sartori
...to honor the maxBatchTransferSize in grpc calls.
2023-12-21BazelCasClient: Split FindMissingBlobs in multiple batches...Alberto Sartori
...such that each request does not exceeds the message limit imposed by GRPC.
2023-12-21SplitBlob: improve error message in case of a failureAlberto Sartori
2023-12-21BazelCasClient: define new templated CreateBatchRequestsMaxSize member function.Alberto Sartori
This function will ensure that each request does not exceeds the maximum message size, currently set by kMaxBatchTransferSize in the message_limits library.
2023-12-21BazelCasClient::UpdateSingleBlob: Emit a log entry upon failureAlberto Sartori
2023-12-21bazel_network: use message_limits libraryAlberto Sartori
2023-12-13bazel client: use the retry strategy, provided by the retry lib, when doing ↵Alberto Sartori
the rpc calls.
2023-12-13bytestream client: log more info in case of a failureAlberto Sartori
2023-12-13Add debug output for blob splittingSascha Roloff
2023-12-12execution_api: Move dispatch file parser into separate utilityPaul Cristian Sarbu
2023-12-12Filesystem: Fix copy overwrite of symlink with fileOliver Reiche
... and improve log messages in case of failure.
2023-11-30Resolve inconsistencies in third-party headers include formatPaul Cristian Sarbu
2023-11-30Splice blobs on diskKlaus Aehlig
When using blob splitting, we expect to create a potentially large file. Therefore reconstruct it from its parts on disk rather than keeping the whole file in memory.
2023-11-22Implement blob splitting protocol on just client sideSascha Roloff
2023-11-07Bazel API: implement ParallelRetrieveToCasKlaus Aehlig
... using thread-based parallelism for the blobs of each tree.
2023-09-22RemoteServeConfig: Remove problematic inheritancePaul Cristian Sarbu
This was causing the remote serve address to overwrite the one set for remote execution. Also, to keep things clean, some common remote server-related methods and definitions were moved into their own library.
2023-09-13remote: Extract port parsing in own libraryPaul Cristian Sarbu
2023-09-13remote: Extract common client implementation utils in own libraryPaul Cristian Sarbu
2023-08-24Execution: only take complete actions from cacheKlaus Aehlig
... which are only actions that, besides giving exit code 0 also created all the outputs they promised to.
2023-08-09Update fmt libraryKlaus Aehlig
2023-07-27Execution API: support fetching a single artifact to memoryKlaus Aehlig
2023-07-26remote api: honor the --raw-tree optionKlaus Aehlig
2023-07-05RemoteExecutionConfiguration: support dispatch listKlaus Aehlig
Extend the configuration data structure by a dispatch list of endpoints to chose based on the first match of the execution properties.
2023-07-05Add serialisation of ServerAddressKlaus Aehlig
... in a way consistent with what we're using at other places already.
2023-06-28Update remote execution api to preliminary version v2.3Sascha Roloff
In preparation for the introduction of our blob splitting protocol as extension to the remote execution api, we need to update the used remote execution api to a more recent version than v2.0.0. Since no new tags are available right now, we update to the preliminary protocol version v2.3 according to the following discussion: https://github.com/bazelbuild/remote-apis/issues/253
2023-06-26Allow non-upwards symlinks with Bazel APIPaul Cristian Sarbu
2023-06-26Execution response: Add output symlink pathsPaul Cristian Sarbu
2023-06-26bazel_msg_factory: Allow non-upwards symlinks in uploaded treesPaul Cristian Sarbu
2023-06-26ReadTree: Add check for non-upwards symlinks...Paul Cristian Sarbu
...as early as possible. This ensures that callers always receive only the tree entries for the supported object types. For the symlinks non-upwardness check we pass a lambda capturing the real backend of the tree entries, such that the symlinks can be read. Updates git_tree tests accordingly.
2023-06-06style: Use designated initializersPaul Cristian Sarbu
This feature has been introduced with C++20.
2023-05-03grpc: fix build fail with protobuf in debug modePaul Cristian Sarbu
2023-04-26imports: Switch to Microsoft GSL implementationOliver Reiche
... with two minor code base changes compared to previous use of gsl-lite: - dag.hpp: ActionNode::Ptr and ArtifactNode::Ptr are not wrapped in gsl::not_null<> anymore, due to lack of support for wrapping std::unique_ptr<>. More specifically, the move constructor is missing, rendering it impossible to use std::vector<>::emplace_back(). - utils/cpp/gsl.hpp: New header file added to implement the macros ExpectsAudit() and EnsureAudit(), asserts running only in debug builds, which were available in gsl-lite but are missing in MS GSL.
2023-04-26bazel_network: grpc++ is not a direct dependencyOliver Reiche
2023-03-23remote api: honor alternative for retrieving pathsKlaus Aehlig
2023-03-23execution api: support retrieving from more than one APIKlaus Aehlig
2023-03-15bazel api: fix include of proto files...Alberto Sartori
...bazel_types.hpp just requires the messages, not the grpc related services and classes
2023-03-15bazel client: remove BazelAcClient::UpdateActionResult...Alberto Sartori
...both buildbarn and just execute do not allow a direct upload to the action cache. Moreover, our tool does not support it, anyway, meaning it was dead code.
2023-02-22cleanup: Fix typos in code line comments and logging messagesPaul Cristian Sarbu
2023-02-22ExecutionApi: Retrieve to correct CAS directlyOliver Reiche
... by keeping track of each blob being a file or executable and storing it to the correct local physical CAS directory. The new flag is merely a hint and only used by the local execution API. Leaving it out will still correctly transfer the blob but may cause unnecessary duplicates in file CAS.
2023-02-02execution-service: add new subcommand executeAlberto Sartori
This subcommand starts a single node remote execution service honoring the just native remote protocol. If the flag --compatible is provided, the execution service will honor the original remote build execution protocol. New command line args supported by this subcommand: -p,--port INT: Execution service will listen to this port. If unset, the service will listen to the first available one. --info-file TEXT: Write the used port, interface, and pid to this file in JSON format. If the file exists, it will be overwritten. -i,--interface TEXT: Interface to use. If unset, the loopback device is used. --pid-file TEXT Write pid to this file in plain txt. If the file exists, it will be overwritten. --tls-server-cert TEXT: Path to the TLS server certificate. --tls-server-key TEXT: Path to the TLS server key. Co-authored by: Klaus Aehlig <klaus.aehlig@huawei.com>
2023-02-02bytestream_client: use common header with settings shared by both server and ↵Alberto Sartori
client
2023-02-02remote/config.hpp: make ParsePort re-usable for execution-serviceAlberto Sartori
2023-02-02bytestream_client: report failures on LogLevel::ErrorAlberto Sartori
2023-02-01Update fmtlib to version 9.1.0Klaus Aehlig
Co-authored-by: Michael Thies <mail@mhthies.de>