summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2024-09-09Release 1.3.2v1.3.2Klaus Aehlig
2024-09-09Executor: Also report missing output dirsOliver Reiche
(cherry-picked from 1e5cc37df36204474cedef68486ba8b29492d259)
2024-09-09When creating actions, normalize paths in the action definitionKlaus Aehlig
... and also perform conflict check on the normalized paths. Still, the the output of the "ACTION" funtion be keyed by the representation of the paths as originally described, to allow the author of a rule to use non-normalized paths as well. (cherry-picked from 74a5fbc9644eda7f105d2d2c672a41fa5c34c6a8)
2024-09-09just-mr: Fix wrong exit code for --help callPaul Cristian Sarbu
Due to the specifics of how CLI11 library handles the --help argument and the fact that just-mr uses its own set of exit codes, we were correctly printing the help text, but falsely returning a non-zero exit code as result. This is now fixed. (cherry-picked form faa8fc156ed8314db23a2d1612975ee745a50ade)
2024-09-09Store newly spliced executables as copies.Maksim Denisov
During multithreaded splicing, the main process can be forked (inheriting open file descriptors). In this case, the executable file saved using hardlinking becomes inaccessible. To prevent this, executables must be stored as copies made in a child process. (cherry-picked from a336b835ee0831ba5fe2f4e8f9b1773eff68416b)
2024-09-09tc cache: abort writing if a dependent target is not availableKlaus Aehlig
After successful build of an export target target a cache entry is written. As those export targets have a dependency structure, care is taken to write them in correct order. Writing a cache entry for an export target requires knowledge of the analysis result of that target. Originally, all targets used to be analysed locally, so that information was available for every target that was not read from the local target cache already. However, with the introduction of serve endpoints, it can happen that a target was analyzed locally, but it depends on targets obtained from serve. In this case, we have to refrain from writing a target-level cache entry to keep the the consistency invariants of the target-level cache. (cherry-picked from f63d3df866a2d75b694c8c484b21fbcdccd58acd)
2024-09-09Copy digest before emplacing in BlobContainerMaksim Denisov
...fixing potentially dangerous code (evaluation order is unspecified). (cherry-picked from b91f39d2429704859e799bff2554fc40636438f9)
2024-09-09Start 1.3.2 development cycleKlaus Aehlig
2024-05-22Release 1.3.1v1.3.1Klaus Aehlig
2024-05-21FileRoot: Fix parsing of ignore-special Git tree rootsPaul Cristian Sarbu
When populating the GitTree instance stored in a Git tree-type FileRoot with the ignore-special flag set, the GitTree instance would be created with an empty raw_id_ field, signaling that some of the entries might have been skipped and thus the root tree id is not anymore in a one-to-one correspondence with the stored list of entries. This however caused FileRoot instances with missing tree id information. This commit fixes the issue by always storing the raw_id_ field as the root id of the Git tree, as well as clarifying the relationship between this field and the ignore_special_ flag, including refactoring the tree id getters. (cherry picked from 06336198a38ffaec6c35df34580391579a7a7b22)
2024-05-21serve target: Improve logic for local build failure triggersPaul Cristian Sarbu
If the serve endpoint reports an internal error, local builds should not continue and the error message should be provided. Similarly, if the serve endpoint promises the target cache value to be in remote CAS, but the client cannot find or process it as needed, then a local build again should not continue and the reason be provided as an error message. (cherry picked from ab0a06871391a84c9b904bc9f2cd74ee1e8903ea)
2024-05-21Start 1.3.1 development cycleKlaus Aehlig
2024-05-08Relase 1.3.0v1.3.0Klaus Aehlig
2024-05-03Include environment in action reporting on the command lineKlaus Aehlig
Compared to the command line, the environment usually is quite short. So including it in messages reporting about commands does not introduce a lot of additional noise. However, knowing the environment can help understanding an error message. Therefore, it seems a good trade off to include it. Do so.
2024-05-02Use a named constant length for FileStorage directory namesMaksim Denisov
2024-05-02Release 1.3.0~beta1v1.3.0-beta1Klaus Aehlig
2024-05-02Small changes to allow gsl-lite supportPaul Cristian Sarbu
The gsl-lite implementation is slightly more picky in terms of type conversions and constness resolution in initializers, therefore small changes were needed.
2024-04-30Compactification: Split files concurrently.Maksim Denisov
2024-04-30Compactification: Remove spliced files concurrently.Maksim Denisov
2024-04-30Compactification: Remove invalid files concurrently.Maksim Denisov
2024-04-30Compactification: Implement a generic concurrent logic.Maksim Denisov
2024-04-29bazel_response: Report failure to read stdout/stderr blobs of an actionPaul Cristian Sarbu
2024-04-29bazel_execution_client: Replace protobuf map use with STLPaul Cristian Sarbu
2024-04-29Fix race condition in task queueKlaus Aehlig
To avoid lock contention, our task system queues tasks in several queues. In order to know when there is no more work to be done, a global counter keeps track of the total work. Here, it is important that this counter be incremented before a value is added to the queue, for otherwise some other thread could pick it up and decrement the total work load, hence getting us in a state where the counter underestimates the total amount of work to be done. Fix the order of those operations.
2024-04-26analyse: fix reporting of internal errosKlaus Aehlig
... by taking the correct map to report which targets failed to be analysed and also report about rules that failed to be analysed.
2024-04-25just_serve.proto: Cleanup of specification documentationPaul Cristian Sarbu
Also aligning comments in server-side headers with the proto file documentation.
2024-04-25just serve: Paths in config file should be location objectsPaul Cristian Sarbu
Also updates the tests and all relevant documentation accordingly.
2024-04-25just-mr rc: Extract location object parser in separate libraryPaul Cristian Sarbu
This will make it available also to just.
2024-04-25absent_target_map: fix target_file when requesting flexible variableKlaus Aehlig
... of an absent target. Here, the request is given by repository root and filename; so the filename is to be taken relative to the root, i.e., we have to prefix the targets-file name with the module.
2024-04-24just analyse: add --dump-result command line optionAlberto Sartori
The result of the analysis is a JSON object containing the keys `"artifacts"`, `"runfiles"`, and `"provides"`. This JSON object, by default, is logged. However, it might be useful to process the data contained in it, while, for example, developing new rules. This patch adds a new command line option (`--dump-result`), reserved to the subcommand `analyse`, to dump the analysis result to the given file or stdout (if `-` is given).
2024-04-24expressions: add logical negationKlaus Aehlig
While this can already be expressed by an "if" statement, having a dedicated function for logical negation makes some expressions more readable.
2024-04-24expressions: for "if" expressions, make both branches optionalKlaus Aehlig
... using, also for the "then" branch, the empty list as default. In this way, this statement not only more symmetric, but also allows shorter representations of some typical expressions.
2024-04-24expressions: add "length" functionKlaus Aehlig
Lists are somtimes used in configurations as replacement for tuples. Providing length gives an easy way to detect usage errors.
2024-04-24expressions: add generic assertionsKlaus Aehlig
2024-04-23just-mr: Make outer non-trivial async map consumers noexceptPaul Cristian Sarbu
For setup and update subcommands, the consumer lambdas for their respective async maps are non-trivial (i.e., do more than keeping the map values) and operate on ungaurded JSON objects, thus it is better to guard them against any uncaught exceptions.
2024-04-23just-mr: Fix map key type bug in repository setupPaul Cristian Sarbu
When taking over roots from the input repository description, the JSON value was not properly checked for validity (i.e., if it is of type string), resulting in an unhandled exception being thrown. The issue is fixed with a proper type check.
2024-04-23just describe: support retry argumentsKlaus Aehlig
... as also just describe can communicate to grpc endpoints.
2024-04-22Compactification: Remove invalid entries from the storage.Maksim Denisov
During compactification, invalid entries must be deleted.
2024-04-22Compactification: Obtain hash string length from hasher.Maksim Denisov
2024-04-19Also for absent tragets, track progress by configured targetKlaus Aehlig
Commit f5f9be5bc07b16807aceac86fba9212e3889762a changed from tracking progress by cache key to tracking by configured target; however, the absent-target map was forgotten. Fix this while also switching to the shortend export-target representation introduced in the previous commit.
2024-04-19Export progress: report short configurationKlaus Aehlig
As we always analyse export targets in their canonical configuration (and do count in the progress the mapping from obtained configuraiton to canonical one), the shortend name (i.e., the representation with null values dropped form the configuration) is still a unique representation of the target. Use this in progress reporting to simplify reading the progress sample.
2024-04-18just-mrrc: support retry optionsKlaus Aehlig
2024-04-18just-mr: support retry for temporary grpc failuresKlaus Aehlig
... using the same mechanism that just uses. As just-mr now also does quite some grpc calls, e.g., fetching distribution files from the remote-execution endpoint and negociating roots with the serve endpoint, it may now also be affected by a bad network or a service temporarily be unavailable.
2024-04-18Move retry-related CLI parts into separate librariesKlaus Aehlig
... to simplify reuse.
2024-04-17Compactification: Split large entries.Maksim Denisov
During garbage collection split and remove from the storage every entry that is larger than a threshold.
2024-04-17Compactification: Remove spliced entries.Maksim Denisov
During garbage collection remove from the storage every entry that has the large entry.
2024-04-17Compactification: Obtain generation cache directories for compatibility modes.Maksim Denisov
2024-04-17Compactification: Switch compatibility modes.Maksim Denisov
2024-04-17Compactification: Obtain storage roots from LocalCAS.Maksim Denisov
2024-04-16expression language: add array access by indexKlaus Aehlig