summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-06-07Remove unused code.Maksim Denisov
IExecutionApi::UploadFile was introduced to handle adding of a temporary file to the storage. The call was redirected to LocalCAS internal methods that do the similar thing, so this virtual method can be removed.
2024-06-07Update ChangelogKlaus Aehlig
2024-06-06Update documentation about tc writing in the presence of serveKlaus Aehlig
2024-06-06Add test verifying cache consistency if some targets are servedKlaus Aehlig
... but not all. Also in this case, if an target-level cache entry is written, the dependencies have to be written as well.
2024-06-06tc writing: allow different log levelKlaus Aehlig
... for failures instead of the binary strict_logging. In this way, we can log at log level Error in the serve instance, and for the client at level Warning or Info, depending on whether failure is expected (i.e., if targets are potentially taken from a serve endpoint) or not.
2024-06-06tc cache: abort writing if a dependent target is not availableKlaus Aehlig
After successful build of an export target target a cache entry is written. As those export targets have a dependency structure, care is taken to write them in correct order. Writing a cache entry for an export target requires knowledge of the analysis result of that target. Originally, all targets used to be analysed locally, so that information was available for every target that was not read from the local target cache already. However, with the introduction of serve endpoints, it can happen that a target was analyzed locally, but it depends on targets obtained from serve. In this case, we have to refrain from writing a target-level cache entry to keep the the consistency invariants of the target-level cache.
2024-06-05Add design on how to extend our handling of symbolic linksKlaus Aehlig
2024-06-04blob containers: Store and upload taking into account content sizePaul Cristian Sarbu
Update logic populating containers to use the new method which is aware of the maximum transfer limit.
2024-06-04common_api: Add method to help limit memory footprint of containers...Paul Cristian Sarbu
...by proactively calling the Upload method for large blobs and containers as soon as the transfer limit is reached.
2024-06-04bazel_network: Change UploadBlobs to also accept an rvaluePaul Cristian Sarbu
This unifies the signature of all uploader functions consuming a BlobContainer type.
2024-06-04ContentBlobContainer: Add content size fieldPaul Cristian Sarbu
As the ContentBlobContainer is used to store actual content needed to be transferred, it is useful to automatically keep track of the running (bytes) size of the data being stored.
2024-06-04rule language: support SYMLINK functionKlaus Aehlig
... to allow generating symlinks as part of a rule, as it is already described in our documentation.
2024-06-03debug rule: Fix staging of headersPaul Cristian Sarbu
For libraries the headers of private dependencies were wrongly skipped from staging. For binaries, no headers were passed at all in the provides map. To fix these issues, an additional field is added in the provides map to ensure we collect, and then properly stage, all needed headers for both libraries and binaries.
2024-06-03test: Use own installed just and just-mr targets...Paul Cristian Sarbu
...which do not stage also the debug source and header files (while in debug mode), as this is unnecessary bloat in the tests. As the tool-under-test and mr-tool-under-test targets should be used instead of the regular install targets also in the various extra rules in end-to-end and utils, move their definition in the outmost test TARGETS file.
2024-06-03test: Fix typos in test runner rulesPaul Cristian Sarbu
2024-06-03debug rule: Add flag field to skip installing extra debug artifactsPaul Cristian Sarbu
This is useful when we want to install targets built in debug mode, but do not want to stage all the additional source and header files if no debugging is being performed, e.g., in tests.
2024-05-28Use shared_ptr for data in ContentBlobMaksim Denisov
...to reduce the "price" of copying.
2024-05-28Use ArtifactBlobContainer in IExecutionApiMaksim Denisov
...instead of BazelBlobContainer to not bring bazel_re::Digest to IExecutionApi.
2024-05-28Use ArtifactBlob in BlobTree instead of BazelBlob.Maksim Denisov
2024-05-28Move blob_tree to the common stage since it is used only there.Maksim Denisov
2024-05-28Rename BlobContainer to BazelBlobContainerMaksim Denisov
2024-05-28Convert BlobContainer to a templateMaksim Denisov
...where the template parameter is the type of a digest.
2024-05-28Use TransformedRange in BlobContainerMaksim Denisov
...instead of various iterators.
2024-05-28Implement "on the fly" transformed range.Maksim Denisov
2024-05-28Copy digest before emplacing in BlobContainerMaksim Denisov
...fixing potentially dangerous code (evaluation order is unspecified).
2024-05-28Move CreateBlobFromPath since it is only used in tests.Maksim Denisov
2024-05-27Fix headers in local_cas and bazel_network.Maksim Denisov
2024-05-27Use common interface for stream dumpingMaksim Denisov
...in LocalApi and BazelApi.
2024-05-27Use common interface for reading tree entries and leafsMaksim Denisov
...in LocalApi and BazelApi.
2024-05-27CC: if no CC/CXX/AR is provided via the toolchain, default to cc/c++/arKlaus Aehlig
... instead of defaulting to "". In this way, an empty default target can be used as toolchain defaults for systems with default names.
2024-05-24Prefer built-in array access over implementing "first" and "last" using foldlKlaus Aehlig
2024-05-23Execution APIs: Extract common implementation into separate libraryPaul Cristian Sarbu
This reduces the code duplication between the local and bazel APIs and improves code maintainability.
2024-05-23execution_api: Cleanup of includesPaul Cristian Sarbu
2024-05-23bazel_api: Remove extra complexity in RetrieveToCasSplittedPaul Cristian Sarbu
We can avoid doing extra work in converting between bazel digests and artifact digests by actually using the API interface.
2024-05-23Add shell toolchain for all pkg-config dependenciesKlaus Aehlig
2024-05-23["CC/pkgconfig", "system_library"]: honor shell toolchainKlaus Aehlig
... instead of hard-coding /bin/sh.
2024-05-22Avoid memory usage duplicationMaksim Denisov
2024-05-22Remove unused codeMaksim Denisov
2024-05-21Ignore corrupted blobs during transfer while reading.Maksim Denisov
2024-05-21["CC/auto", "config"] also honor shell-toolchain-provided PATHKlaus Aehlig
2024-05-21executor.hpp: fix a typoAlberto Sartori
2024-05-21Update CHANGELOGKlaus Aehlig
... to report the changes so far since the last release.
2024-05-17FileRoot: Fix parsing of ignore-special Git tree rootsPaul Cristian Sarbu
When populating the GitTree instance stored in a Git tree-type FileRoot with the ignore-special flag set, the GitTree instance would be created with an empty raw_id_ field, signaling that some of the entries might have been skipped and thus the root tree id is not anymore in a one-to-one correspondence with the stored list of entries. This however caused FileRoot instances with missing tree id information. This commit fixes the issue by always storing the raw_id_ field as the root id of the Git tree, as well as clarifying the relationship between this field and the ignore_special_ flag, including refactoring the tree id getters.
2024-05-17RepositoryConfig: Instance should not be changed once populatedPaul Cristian Sarbu
Once a RepositoryConfig instance gets populated, it must never be changed again. Therefore, all functions accepting these instances should only take them as pointers to const.
2024-05-15test/end-to-end: properly handle multiple key-value pairs in the...Alberto Sartori
...REMOTE_EXECUTION_PROPERTIES env variable.
2024-05-15test/utils/test_env.hpp: split REMOTE_EXECUTION_PROPERTIES on space...Alberto Sartori
...instead of the unused ';' Using spaces allows for simpler looping in shell script
2024-05-15serve target: Improve logic for local build failure triggersPaul Cristian Sarbu
If the serve endpoint reports an internal error, local builds should not continue and the error message should be provided. Similarly, if the serve endpoint promises the target cache value to be in remote CAS, but the client cannot find or process it as needed, then a local build again should not continue and the reason be provided as an error message.
2024-05-15logging: Do not make assumptions in emit callsPaul Cristian Sarbu
The Emit method of the Logger class, when called with a string as second argument, expects it to be a format string. It should be considered a programming error to pass a string variable as that argument without knowing for certain that it does not contain any format escape character ('{', '}'); instead, one should be conservative and use the blind format string "{}" as second argument and pass the unknown string variable as third argument.
2024-05-15Test tuorial: mention creating a defaults definition for the shell toolchainKlaus Aehlig
... so that the tutorial also works with the latest version of our rules.
2024-05-15Add shell defaultsKlaus Aehlig
So far, our rules, where depending on the shell, implicitly use "normal" defaults, hard-coded in the rules. Support configuring those in a default target, in the same way we do so for other tools, like the C compiler. In this, it is also possible to bring your own shell, built as a (compiled) target.