summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-01-08serve distdir tree: Client-side and API implementationsPaul Cristian Sarbu
2024-01-08serve distdir tree: Server-side implementationPaul Cristian Sarbu
2024-01-08GitRepo: Add blob lookup methodPaul Cristian Sarbu
2024-01-08just_serve.proto: Add 'tree of a distdir' requestPaul Cristian Sarbu
2024-01-08serve content or tree: Check for Git object also in local CASPaul Cristian Sarbu
2024-01-08serve source tree: Improve log messages when sync with remote failsPaul Cristian Sarbu
2024-01-08serve archive tree: Add missing check for content in local CASPaul Cristian Sarbu
When serving the tree of an archive, we should check also in the local CAS for the content blob.
2024-01-08serve configuration client: Fix logger namePaul Cristian Sarbu
2024-01-08bugfix: Add missing return in SetRemoteExecutionDispatchPaul Cristian Sarbu
This was a source of occasional std::bad_variant_access exceptions.
2024-01-08git_tree_fetch_map: Fix missing backup-to-remote behaviourPaul Cristian Sarbu
The change added to interrogate also the local CAS for the tree of a 'git tree' repository failed to add support for backing up such tree found into the remote CAS. This commit fixes the issue.
2024-01-08repos.json: ignore special in grpc source treeKlaus Aehlig
The source archive of grp contains upwards symlinks (that do not point outside the source directory). For a normal build this is not an issue; we still can generate a git tree from this archive and the parts we use of this archive never even touch a directory containing an upwards symlink. However, when just-mr desires to get this root from a just serve instance, the tree would have to be synced to the remote-execution CAS and hence the restriction applies that we only consider non-upwards relative symlinks as first-class citizens. Therefore, ignore all symlinks in this archive and this way, without changing the actual build, obtain an easy-to-manage root.
2024-01-08bootstrap: accept a build dir inside the source treeKlaus Aehlig
While it is best practise to build outside the source tree, some package formats require that a build be carried out inside the source tree. As there are no principle obstacles, as long as a non-existing directory is requested as build dir, support it by ignoring the destination in the recursive copy.
2023-12-21serve target handling: store blobs also locallyKlaus Aehlig
... before trying to upload from local storage to the remote CAS. Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
2023-12-21Export ["@", "json", "", "json"]Klaus Aehlig
... to allow target-level caching.
2023-12-21ServeTarget: downgrade log level to performanceKlaus Aehlig
The error log level should be reserved for events that inevitably lead to a failed build. A failure to receive a target from the serve endpoint, however, is not such a case; for performance reasons, and also to have the same artifacts as everyone else in the case of non-reproducible dependencies, just inquires the serve end point for every export target whenever a serve end point is given. In this case, the build just continues even if the serve end point is, e.g., lacking a certain root.
2023-12-21BazelCasClient: remove dead code.Alberto Sartori
The removed code has been superseeded by the CreateBatchRequestsMaxSize member function.
2023-12-21BazelCasClient::DoUploadBlobs: returns the count of the uploaded blobs...Alberto Sartori
...instead of the vector containing the digests of the uploaded blobs. The returned vector was never inspected by the callers, except for its size. The tests have been accordingly amended.
2023-12-21BazelNetwork: simplify logic of DoUploadBlobsAlberto Sartori
2023-12-21BazelCasClient: split DoBatchUploadBlobs into multiple calls...Alberto Sartori
...to honor the message limit imposed by GRPC.
2023-12-21BazelCasClient: split BatchReadBlobs into multiple calls...Alberto Sartori
...to honor the maxBatchTransferSize in grpc calls.
2023-12-21BazelCasClient: Split FindMissingBlobs in multiple batches...Alberto Sartori
...such that each request does not exceeds the message limit imposed by GRPC.
2023-12-21SplitBlob: improve error message in case of a failureAlberto Sartori
2023-12-21BazelCasClient: define new templated CreateBatchRequestsMaxSize member function.Alberto Sartori
This function will ensure that each request does not exceeds the maximum message size, currently set by kMaxBatchTransferSize in the message_limits library.
2023-12-21BazelCasClient::UpdateSingleBlob: Emit a log entry upon failureAlberto Sartori
2023-12-21bazel_network: use message_limits libraryAlberto Sartori
2023-12-21src/buildtool/execution_api/commmon: add message_limits libraryAlberto Sartori
Define the threshold for the grpc messages.
2023-12-21remote/client_common.hpp: add utility functions for logging status of rpc callsAlberto Sartori
2023-12-20test: Serve endpoint provides tc value for cache missesPaul Cristian Sarbu
During the analysis of export targets, if there is a local target cache miss, we should also interrogate the serve endpoint (if given) for the target cache value before proceeding with the analysis. For this test we have extended the "with_serve" runner to expose also the remote-execution build root to the test env.
2023-12-20just: Query serve endpoint during analysis for non-cached export targetsPaul Cristian Sarbu
2023-12-20directory_map: refuse to read contents of absent rootsKlaus Aehlig
While in our setting, a missing directory is generally OK, it is not OK to ask for the content of an absent root. In particular, we should not assume it to be empty, just because the root is absent.
2023-12-20Add end-to-end test serving a tree target from an archive rootKlaus Aehlig
2023-12-19tutorial: explain the difference between action graph and the traversed partKlaus Aehlig
When explaining CC tests, the option --request-action-input is used to get hold of the actual test binary. This is the first time in the tutorial where the action graph is larger than the part that that is traversed during the build. Use this oportunity to explain the difference between those concpets as, reportedly, some users got confused about this.
2023-12-19SystemCommand: Fix handling of child statusOliver Reiche
2023-12-19just-mr git tree: Check for tree also in local CASPaul Cristian Sarbu
Now we look for the tree also in the local CAS, not just in the local Git cache. If found, we import the tree from local CAS to the Git cache and continue as usual.
2023-12-19just-mr fetch content: Check for blob also in Git cachePaul Cristian Sarbu
Now we look for the content blob also in the local Git cache, not just in local CAS. If found, we store the blob read from Git cache into local CAS and continue as usual.
2023-12-19just-mr archive: Check for content blob also in Git cachePaul Cristian Sarbu
Now we look for the content blob also in the local Git cache, not just in local CAS. If found, we store the blob read from Git cache into local CAS and continue as usual.
2023-12-19other_tools: Fix missing includes of direct dependenciesPaul Cristian Sarbu
This fixes the multiple instances of (mainly) missing or wrong includes of standard, third-party, and own headers in the other_tools source folder.
2023-12-19test: Extend general just-mr check to test each repository tooPaul Cristian Sarbu
This way we ensure each repository can be set up on its own and nothing gets cached from other repositories, as opposed to the parallel run where some caching is expected to occur.
2023-12-19just-mr setup: Add missing reporting of failed hostname replacementPaul Cristian Sarbu
2023-12-19just-mr setup: Fix bug in fetching Git reposPaul Cristian Sarbu
This bug was introduced with the alternative mirrors changes.
2023-12-19Fix just-mr handling of --absentKlaus Aehlig
If the option --absent is given in just-mr (either directly or implicitly via the corresponding entry in the rc file), it rewrites the "pragma" entries in the internal representation of the mr repository config. When doing so, however, we have to take into consideration that a workspace root is not necessarily the definition of a repository, but can, alternatively, also be a reference to another root. This was not taken into account; fix this by restricting the rewriting to only json objects.
2023-12-19just-serve-config(5): add client credentialsKlaus Aehlig
The just-serve config already honors specific fields to obtain the information on how to authenticate to the remote-execution endpoint. However, those have not yet been described in the man page yet. Do this now.
2023-12-18future-designs: Add proposal for 'just add-to-cas' subcommandPaul Cristian Sarbu
2023-12-18Add design document for adding garbage collection for build rootsKlaus Aehlig
2023-12-15changelog: Update 'just serve' entryPaul Cristian Sarbu
2023-12-15just serve design doc: Update and move to conceptsPaul Cristian Sarbu
2023-12-14just describe: Query serve endpoint for export targets with absent rootsPaul Cristian Sarbu
2023-12-14just serve target description: Client-side and API implementationPaul Cristian Sarbu
2023-12-14just serve target description: Server-side implementationPaul Cristian Sarbu
2023-12-14just_serve.proto: Add rpc to serve description of an export targetPaul Cristian Sarbu