summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2025-06-12Release 1.5.3v1.5.3Klaus Aehlig
2025-06-11Update CHANGELOGKlaus Aehlig
2025-06-11GitRepo: Fix wrong handling of fatal loggingPaul Cristian Sarbu
...in creating Git tree from filesystem directory. (cherry-picked from 6b31a88acbe6a6556e36f838b914a82879949811)
2025-06-11GitRepo: Use std::invoke to call generic loggerPaul Cristian Sarbu
(cherry-picked from cf33c43c8b12d39d0a6b3751db3f58a9895cd41d)
2025-06-11ParallelRetrieveToCasWithCache: abort early on failureKlaus Aehlig
... and do mark artifacts internally as synchronized. First all all, we will abort anyway, to the entry won't even be read and, secondly it is not necessarily true that the artifact is synchronized. (cherry-picked from d1dc5cce32ba20c0dae13aecf63e88fe8ead899f)
2025-06-11WriteTargetCacheEntries: do not use quadratic many jobsKlaus Aehlig
When backing up target-cache entries we use parallelism at two dimensions, the independent cache entries and for each entry we retrieve the artifacts in parallel. If for each dimension we use the full amount of parallelism allowed, that gives a number of threads up to the square of the amount of parallelism specified by the user. Therefore, use in each dimension only the square root of the allowed parallelism keeping the total parallelism (up to rounding) within the specified range. (cherry-picked from f4e9de93b50fe8135d378830577cda687cae28ee)
2025-06-11GitRepo: Methods expected to use a logger should do so...Paul Cristian Sarbu
...in all return paths, including in reporting caught exceptions. In this way give the opportunity for any calling AsyncMap to receive an expected fatal logger call on failure and thus be able to shut down gracefully. This is in line with the AsyncMap design, where the loggers are assumed to be safe to call by a consumer. (cherry-picked from ae33d0f287e83769dbef9287e64006e1c0c463e9)
2025-06-11Start 1.5.3 development cycleKlaus Aehlig
2025-05-19Release 1.5.2v1.5.2Klaus Aehlig
2025-05-19Update CHANGELOGKlaus Aehlig
2025-05-19Update grpc to 1.70.2Klaus Aehlig
... as this contains a bug fix with respect to the currently-used version 1.70.1, see release notes https://github.com/grpc/grpc/releases/tag/v1.70.2 (cherry-picked from e5e56db704df4c025b767563dac59b8b2fbf7a51)
2025-05-19LocalAction: Fix collection of directory symlinkOliver Reiche
(cherry-picked from fe4006fce755432b1ae3a273873f3649512c7f94)
2025-05-19computed_roots: Fix missing return on fatal in evaluation async mapPaul Cristian Sarbu
(cherry-picked from fe11e1cc49ed46b1837336df33fb0805668905a6)
2025-05-19generic rule: fix conflict checkKlaus Aehlig
For a generic rule, it is an error if map union of various inputs (overlayed in correct order) does not form a proper stage. To implement this check properly, we first have to construct the map of all inputs and only then perform the staging check and not do the check with only the runfiles, as 5e104a526cf76fe75312d2fd288a3c88f506fb0a accidentally did. Fix this. (cherry-picked from f249a220913f4421c58c9c6251e4f3dce67e251c)
2025-05-19Start 1.5.2 development cycleKlaus Aehlig
2025-04-01Release 1.5.1v1.5.1Klaus Aehlig
2025-04-01Update CHANGELOGKlaus Aehlig
2025-04-01built-in rule "tree": fix supported fieldsKlaus Aehlig
The built-in rule "tree" does not support the field "data". Hence remove it from the list of white-listed fields so that an appropriate warning is issued if it occurs for that rule. (cherry-picked from 1324f45fa7b1938b39854f7b38d981c238da2fbc)
2025-04-01just-mr progress: consider computed roots as no-workKlaus Aehlig
... as they will be taken care of only during the build. (cherry-picked from ff4ef1723619fb78c6eaf44c35138145f5af39ca)
2025-04-01serve service: receive trees in parallel in local CASKlaus Aehlig
The serve service can be asked to get a tree root from the designated remote-execution endpoint. Speed this process up by going through local CAS; in this way, - identical blobs are fetched only once, and - the fetching happens in parallel. (cherry-picked from 360ddbb0285554c936d5eaa95460d5aad3e13b1e)
2025-04-01just serve: write pid and info files atomicallyKlaus Aehlig
(cherry-picked from 241ccadfc5b9bcebbbd5f41c96eda6b0d208a390)
2025-04-01just execute: write pid and info files atomicallyKlaus Aehlig
(cherry-picked from 5e24dd0fc5e255d094a3d78e8e1b64a202c1e7a4)
2025-04-01Add library for atomically writing filesKlaus Aehlig
... by using the write-rename dance. This allows processes waiting for pid or similar files to rely on the information being available as soon as the file is available. (cherry-picked from 1efdbe20d506990028caccbfbc7fd241b5a01a99)
2025-04-01just-mrrc(5): fix typoKlaus Aehlig
(cherry-pick from 7f3bff62999f927546c6ef7ee6133a89a2bb9cd6)
2025-04-01utils atomic: Update link to libcxx C++20 adoption pagePaul Cristian Sarbu
(cherry-picked from f93e4cdb371aa01ac157ce1a79e392ceb0108dad)
2025-04-01git_tree_fetch_map: Remove stray semicolonPaul Cristian Sarbu
(cherry-picked from 0cef4facae7bd3d2345c8f099b0382168b0c2a63)
2025-04-01Start 1.5.1 development cycleKlaus Aehlig
2025-03-06Release 1.5.0v1.5.0Klaus Aehlig
2025-03-06tutorial: Add chapter on just-lockPaul Cristian Sarbu
Also update the README.
2025-03-06tutorial: Consistently use the fmtlib release tagPaul Cristian Sarbu
...instead of using the master branch.
2025-03-05Move obtain capabilities failures to debug logOliver Reiche
... instead of reporting an error, although not being fatal.
2025-03-05Update CHANGELOGKlaus Aehlig
... documenting fixes since 1.5.0~beta2
2025-03-05end-to-end/just-mr: Add test to check special entries are correctly ignoredPaul Cristian Sarbu
...during setup of git roots, when the appropriate pragma is set. Check both symlinks and other special entries, e.g., submodules.
2025-03-05resolve_symlinks_map: Fix missing ignore special flag in reading treePaul Cristian Sarbu
2025-03-05FileSystemManager: Let caller decide log level for reading directoryPaul Cristian Sarbu
Do not emit errors when it does not lead to build failures. Callers handle the log level at which failures of this method should be logged.
2025-03-05end-to-end tests: Fix small typos and inconsistenciesPaul Cristian Sarbu
2025-03-04Update grpc to v1.70.1Paul Cristian Sarbu
...together with its dependencies: - boringssl dec0d8f681348af8bb675e07bd89989665fca8bc - protobuf v29.0 - abseil 20240722.0 Also update the bootstrap command for ssl to account for the fact that now also the crypto library has fully moved to C++ and the location of its sources has changed.
2025-03-03Add regression test for just-import-gitKlaus Aehlig
... verifying that indiections are correctly taken into account.
2025-03-03prune bootstrap sourcesKlaus Aehlig
Originally, it was a good idea to add the full "bin" directory to the bootstrap sources; all scripts there where used in some form or another during bootstap and by taking the whole directory, we wouldn't have to update the target when adding new scripts. However, times have changed. The set of scripts neede for bootstrapping is stable for quite a while now; on the other hand actively-developped tools (like just-lock and just-import-git) now reside in "bin". So, to avoid unnecessary runs of the bootstrap tests, prune the dependency set. While there, also mark as tainted.
2025-03-03just-lock: follow indirection when checking for computed rootsPaul Cristian Sarbu
When deciding if a repository is a computed root (i.e., of type "computed" or "tree structure"), first follow indirections. Otherwise we would try to read of the "type" entry of a string which, of course, crashes. Based on the similar fix for just-import-git.
2025-03-03just-import-git: follow indirection when checking for computed rootsKlaus T. Aehlig
When deciding if a repository is a computed root (i.e., of type "computed" or "tree structure"), first follow indirections. Otherwise we would try to read of the "type" entry of a string which, of course, crashes.
2025-02-28Release 1.5.0~beta2v1.5.0-beta2Klaus Aehlig
2025-02-28just-lock: Parallelize cloning, subject to GILPaul Cristian Sarbu
Python's Global Interpreter Lock (GIL) hugely reduces the amount of parallelism one can achieve. However, as cloning repositories is I/O bound, the total runtime is improved by running it in parallel.
2025-02-28just-lock imports: Parallelize checkouts, subject to GILPaul Cristian Sarbu
Python's Global Interpreter Lock (GIL) hugely reduces the amount of parallelism one can achieve. However, as the checkout methods are I/O bound, the total runtime is improved by running them in parallel.
2025-02-28just-lock git-tree imports: Separate checkout logic into own methodPaul Cristian Sarbu
This will allow it to be run mostly asynchronously with other checkouts in the future.
2025-02-28just-lock archive imports: Separate checkout logic into own methodPaul Cristian Sarbu
This will allow it to be run mostly asynchronously with other checkouts in the future.
2025-02-28just-lock git imports: Separate checkout logic into own methodPaul Cristian Sarbu
This will allow it to be run mostly asynchronously with other checkouts in the future.
2025-02-28just-lock: Improve thread safety in Git operationsPaul Cristian Sarbu
Similarly to how it is done in just-mr: - lock against concurrent git-tag calls; - extend git-fetch call with arguments that ensure thread-safe operation.
2025-02-28just-lock: Always use os.environ.copy()Paul Cristian Sarbu
2025-02-28just-lock git imports: Improve cache hit ratePaul Cristian Sarbu
...by first retrieving only the desired commit id from the definitive remote in order to allow it to be checked against the local Git cache. This massively reduces the average network traffic from regenerating the just-mr configuration in justbuild projects that always require the latest version of their dependencies.