summaryrefslogtreecommitdiff
path: root/src/buildtool/execution_api/remote/bazel
AgeCommit message (Collapse)Author
2024-07-22Use HashFunction functionality via Instance()Maksim Denisov
...to track changes during refactoring easier.
2024-07-19Pass RetryConfig instance to BazelApiPaul Cristian Sarbu
2024-07-19Pass RetryConfig instance to BazelNetwork and bazel clientsPaul Cristian Sarbu
2024-07-19Pass RetryConfig instance to WithRetryPaul Cristian Sarbu
2024-07-16LocalAction, BazelAction: Extend constness in fields and methodsPaul Cristian Sarbu
2024-07-12Capture in TreeReaders by pointer to avoid temporariesMaksim Denisov
2024-07-12Return std::nullopt if creation of an action digest failsMaksim Denisov
...instead of dereferencing nullptr.
2024-07-12Pack arguments of CreateActionDigestFromCommandLine to a structMaksim Denisov
2024-07-10ParallelRetrieveToCas: process prerequisites in parallelKlaus Aehlig
2024-07-10ParallelRetrieveToCas: avoid duplicated requestsKlaus Aehlig
While no additional blob will be transferred, doing a request for missing blobs is still a request and, in particular, an unnecessary round trip. Therefore avoid this, by remembering what we synchronized already.
2024-07-04Replace the Auth and Auth::TLS singletonsPaul Cristian Sarbu
Use a builder pattern for creation and validation, in a manner that allows also other authentication methods to be added in the future besides the current TLS/SSL. The main Auth instances are built early and then passed by not_null const pointers, to avoid passing temporaries, replacing the previous Auth::TLS instances passed by simple nullable const pointers. Where needed, these passed Auth instances are also stored, by const ref. Tests also build Auth instances as needed, either with the default 'no certification' or from the test environment arguments.
2024-07-04Pass Auth::TLS instance to serve and execute clientsPaul Cristian Sarbu
2024-07-04Pass Auth::TLS instance to BazelApi and ServeApiPaul Cristian Sarbu
2024-07-04Mark more constructors 'explicit'Paul Cristian Sarbu
Since c++17 the 'explicit' keyword has use also for constructors with more than one argument and it is recommended to use it by default whereever implicit conversions are not expected bahaviour.
2024-06-27Use a raw pointer for passing optional IExecutionApiMaksim Denisov
...instead of std::optional<gsl::not_null<IExecutionApi const*>>
2024-06-25Introduce a type allias for an optional ptr to IExecutionApiMaksim Denisov
...and replace verbose constructions.
2024-06-25Pass IExecutionApi to IExecutionApi by referenceMaksim Denisov
...instead of not_null const ptr.
2024-06-25Mark IExecutionApi's methods constantMaksim Denisov
2024-06-07Move reading functionality to BazelNetworkReaderMaksim Denisov
...to enable validation of received blobs.
2024-06-07Validate all blobs received over the network.Maksim Denisov
2024-06-07Use BaselCasClient in BazelNetworkReaderMaksim Denisov
...bypassing BazelNetwork layer.
2024-06-07Mark methods constant in BazelCasClient.Maksim Denisov
2024-06-04blob containers: Store and upload taking into account content sizePaul Cristian Sarbu
Update logic populating containers to use the new method which is aware of the maximum transfer limit.
2024-06-04bazel_network: Change UploadBlobs to also accept an rvaluePaul Cristian Sarbu
This unifies the signature of all uploader functions consuming a BlobContainer type.
2024-05-28Use shared_ptr for data in ContentBlobMaksim Denisov
...to reduce the "price" of copying.
2024-05-28Use ArtifactBlobContainer in IExecutionApiMaksim Denisov
...instead of BazelBlobContainer to not bring bazel_re::Digest to IExecutionApi.
2024-05-28Move blob_tree to the common stage since it is used only there.Maksim Denisov
2024-05-28Rename BlobContainer to BazelBlobContainerMaksim Denisov
2024-05-28Convert BlobContainer to a templateMaksim Denisov
...where the template parameter is the type of a digest.
2024-05-28Use TransformedRange in BlobContainerMaksim Denisov
...instead of various iterators.
2024-05-28Move CreateBlobFromPath since it is only used in tests.Maksim Denisov
2024-05-27Fix headers in local_cas and bazel_network.Maksim Denisov
2024-05-27Use common interface for stream dumpingMaksim Denisov
...in LocalApi and BazelApi.
2024-05-27Use common interface for reading tree entries and leafsMaksim Denisov
...in LocalApi and BazelApi.
2024-05-23Execution APIs: Extract common implementation into separate libraryPaul Cristian Sarbu
This reduces the code duplication between the local and bazel APIs and improves code maintainability.
2024-05-23execution_api: Cleanup of includesPaul Cristian Sarbu
2024-05-23bazel_api: Remove extra complexity in RetrieveToCasSplittedPaul Cristian Sarbu
We can avoid doing extra work in converting between bazel digests and artifact digests by actually using the API interface.
2024-05-22Avoid memory usage duplicationMaksim Denisov
2024-05-22Remove unused codeMaksim Denisov
2024-05-21Ignore corrupted blobs during transfer while reading.Maksim Denisov
2024-05-15logging: Do not make assumptions in emit callsPaul Cristian Sarbu
The Emit method of the Logger class, when called with a string as second argument, expects it to be a format string. It should be considered a programming error to pass a string variable as that argument without knowing for certain that it does not contain any format escape character ('{', '}'); instead, one should be conservative and use the blind format string "{}" as second argument and pass the unknown string variable as third argument.
2024-04-29bazel_response: Report failure to read stdout/stderr blobs of an actionPaul Cristian Sarbu
2024-04-29bazel_execution_client: Replace protobuf map use with STLPaul Cristian Sarbu
2024-04-12file chunker: increase chunk sizesKlaus Aehlig
As we use chunking also for reducing storage, we have to consider the overhead of block devices which is in the order of kB per file. So our target chunk size should be at least 2 orders of magnitude above this. This suggests to minimally aim for a chunk size of 128kB, a target size that also has the advantage the that maximal chunk size associated with this size is 1MB which is still well below the maximal transmission size of grpc allowing us to avoid the streaming API. As we're scaling everything up by a factor of 16, we also have to increase the number of bits in the involved masks by 4. We use this to also extend the window size by using the 2 most significant octets. Following the advice of the paper proposing FastCDC to spread out the ones roughly equally suggests 0x4444 as a suitable value for the two most significant octets. We also change the suggested extension of the remote-execution API accordingly. As the precise parameters for FastCDC when announced over the remote-execution APIs are still under discussion upstream, we simplify the name to not mention the target size.
2024-03-26Add missing system includesPaul Cristian Sarbu
Main culprits: - std::size_t, std::nullptr_t, and NULL require <cstddef> - std::move and std::forward require <utility> - unordered maps and sets require respective includes - std::for_each and std::all_of require <algorithm>
2024-03-15Clean up more includes and targetsPaul Cristian Sarbu
Some of the more specific issues addressed: - missing log_level target/include - header-only libs wrongly marking deps as private - missing/misplaced gsl includes
2024-03-07Down-level log message for errors that can be handledKlaus Aehlig
We deliberately have many functions that do not abort the process on failure and instead simply return a corresponding value. It is then up to the caller to decide how to handle this failure; in particular, such a failure can be expected, e.g., if we try to fetch a file from remote execution first, before fetching it from the upstream location. To have a consistent user experience, nothing that can occur in a successfull build should be reported at error level; moreover, messages that routinely occur during successfull builds should not be reported at progress or above, except for the (stage) result messages and the progress reporter.
2024-02-26Employ blob splitting and splicing for endpoint dispatchingSascha Roloff
2024-02-26Implement blob splicing protocol at just client sideSascha Roloff
2024-02-26Implement blob chunking algorithm negotiationSascha Roloff