summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2025-06-04FileRoot: Improve documentationPaul Cristian Sarbu
2025-06-04FileRoot: Ensure all read blobs and trees contain valid entriesPaul Cristian Sarbu
In particular, ensure that Git roots check for, e.g., upwards symlinks, before returning blobs and trees. To ensure that only the bear minimum extra work is performed for this purpose, Git roots now keep also the root's GitTreeEntry as a field, allowing the validity check of root source trees to take place only once and only if required.
2025-06-04FileRoot: Small format fixPaul Cristian Sarbu
2025-06-04FileRoot: Give git-based roots access to storage configPaul Cristian Sarbu
2025-06-04RepositoryConfig: Ensure consistency in reading blobs and treesPaul Cristian Sarbu
...with respect to rejecting invalid entries such as upwards symlinks. Also ensure that valid trees are only checked once by remebering known valid tress though marker files in local storage.
2025-06-04RepositoryConfig: Give access to a persistent storage config...Paul Cristian Sarbu
...whenever it is given access to a Git repository. The referenced storage config needs to outlive the repository config instance.
2025-06-04Add utility methods for caching valid treesPaul Cristian Sarbu
...through marker files kept in storage under generation regime. These can be used to allow valid source trees, i.e., those free of upwards symlinks, to be cached in a persistent manner over multiple builds.
2025-06-04GitTree: Allow tree reading to skip symlinks checkerPaul Cristian Sarbu
This is useful when the caller already knows that the tree to look up is valid, and thus the extra check step can be safely skipped.
2025-06-04GitRepo: Fix wrong logging level in reading treesPaul Cristian Sarbu
This removes a scenario where otherwise successful (exit code 0) calls to just and just-mr would result in an error-level log message.
2025-06-04GitRepo: Add tree reader without symlink checkerPaul Cristian Sarbu
2025-06-04GitTree: Ensure all read entries are validPaul Cristian Sarbu
Match behaviour of reading trees, which always checks for invalid entries, also for reading blobs.
2025-06-04git_cas read object: allow validation of individual blobsPaul Cristian Sarbu
This allows individual blobs read to be checked, e.g., for upwards symlinks, also when not part of a tree, which performs such a validation for its entries during its parsing into a GitTree.
2025-06-04git_cas: Be explicit in hash type (raw or hex) when readingPaul Cristian Sarbu
2025-06-04end-to-end glob: Fix typos in test outputPaul Cristian Sarbu
2025-06-04GitTree: Fix comment typoPaul Cristian Sarbu
2025-06-03Invocation server: also show counts in headingsKlaus Aehlig
Often, the simple number, e.g., for failed actions can already give valuable information. For example, when investigating flakiness, the count of failed actions already gives a hint whether a change increased or decreased flakiness which can be valuable before even investigating if the nature of the failure has changed. As we have that information available and an additional number in a heading does not clutter the page too much, let's just show it.
2025-06-02Use quoting for literalsKlaus Aehlig
... to improve readability of target files.
2025-06-02ParallelRetrieveToCasWithCache: abort early on failureKlaus Aehlig
... and do mark artifacts internally as synchronized. First all all, we will abort anyway, to the entry won't even be read and, secondly it is not necessarily true that the artifact is synchronized.
2025-06-02CHANGELOG: document fix in parallelismKlaus Aehlig
2025-06-02end-to-end just-mr/absent-closure: Fix typoPaul Cristian Sarbu
2025-06-02just-mr: Warn for non-content-fixed repo reached from absent mainPaul Cristian Sarbu
If the main repository is marked absent, warn if during the dependency closure computation any non-content-fixed repositories are reached, i.e., any "file"-type repositories that are neither implicitly nor explicitly marked "to_git". Also warn if the main repository itself is marked absent but is not content fixed. Add small test checking that the new warning is produced.
2025-05-30WriteTargetCacheEntries: do not use quadratic many jobsKlaus Aehlig
When backing up target-cache entries we use parallelism at two dimensions, the independent cache entries and for each entry we retrieve the artifacts in parallel. If for each dimension we use the full amount of parallelism allowed, that gives a number of threads up to the square of the amount of parallelism specified by the user. Therefore, use in each dimension only the square root of the allowed parallelism keeping the total parallelism (up to rounding) within the specified range.
2025-05-30Add test verifying that serve can handle deep treesKlaus Aehlig
Add a test testing the serve instance implicit to the "with serve" rule by asking it to build a target that creates an export target where the output is a deep tree.
2025-05-30Add end-to-end test for serving complex targetsKlaus Aehlig
Add a test asking serve to provide a target where both, the dependency as well as the action graph has both a wide node (a target with a large number of direct dependencies) as well as a deep node (a node where the chain of dependencies is long). That target is requested serveral times in parallel to verify that such targets can also be served under load. The process actually tested is the `just serve` process implicit to the "with serve" rule.
2025-05-30clean up test target filesKlaus Aehlig
Dropping unnecessary "arguments_config" as well as evaluating trivial computations.
2025-05-30Invocation server: add links back to the overview pageKlaus Aehlig
2025-05-30Invocation server: use "build stop time" for build wall-clock timeKlaus Aehlig
As the end of the build can be significantly ahead of the end of the incvocation, a new time stamp was added to the profile. Use this new time stamp in the invocation server when determining the wall-clock time of the build.
2025-05-28profile: include also the end time of the buildKlaus Aehlig
... which might be quite ahead of the end time of the invocation if writing out of the action graph delays the end of the invocation.
2025-05-28Invocation server: also show the build timeKlaus Aehlig
Also show only the build time in the invocations overview, as only for the build phase detailled timing information will be available in the log of a particular information.
2025-05-28GitRepo: Methods expected to use a logger should do so...Paul Cristian Sarbu
...in all return paths, including in reporting caught exceptions. In this way give the opportunity for any calling AsyncMap to receive an expected fatal logger call on failure and thus be able to shut down gracefully. This is in line with the AsyncMap design, where the loggers are assumed to be safe to call by a consumer.
2025-05-28rules documentation: Fix typosPaul Cristian Sarbu
2025-05-26Update cli11 to 2.5.0Klaus Aehlig
2025-05-26profile: include time of build startKlaus Aehlig
When using a serve end point, the analysis phase might take quite long if serve has to actually build a delegated target or, at least, has to synchronize artifacts with the remote end point. Therefore, also record the time the build phase started (if building is requested) as an additional time stamp in the profile.
2025-05-23tutorial: Update example outputs in all chaptersPaul Cristian Sarbu
...to match the ones produces by latest binaries and rules.
2025-05-23tutorial computed: Fix source of non-reproducibility in example scriptPaul Cristian Sarbu
While there, update outputs to what is expected with current released binaries and rules. Co-authored-by: Klaus Aehlig <klaus.aehlig@huawei.com>
2025-05-23tutorial: Add DWP and ADD_DEBUGFLAGSPaul Cristian Sarbu
...to example export targets and defaults.
2025-05-23tutorial debugging: Use ADD_COMPILE_FLAGSPaul Cristian Sarbu
...instead of replacing CFLAGS and CXXFLAGS, thus also fixing a mismatch introduced in c008c07656fff528c80add260397c7c7702aa2a8.
2025-05-23Extend tutorial on cross compiling showing how to use ["test", "matrix"]Klaus Aehlig
2025-05-22invocation server: add presentation of invocation wall-clock timeSascha Roloff
2025-05-22profile: add start/stop timeSascha Roloff
2025-05-22just-mr: Fix wrong setup root being picked upPaul Cristian Sarbu
This fixes a bug in which the setup root was falsely being changed by unconditionally searching early for a default configuration files, despite one being explicitly provided at the command line.
2025-05-21shell-test runner: explicitly set HOMEKlaus Aehlig
... to a value that is a non-existent directory. Too many tools try search for rc-files in the user's home directory. To make things worth, the shell as well as many tools take an unset HOME variable as instruction to look up the the user's home directory in the system configuration. While it is good practise to write tests in such a way that they explcitly do not depend on such machine-specific defaults, still be on the safe side by explictly setting HOME to a directory in the action directory we know our rules will no create. Co-authored-by: Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com> Co-authored-by: Sascha Roloff <sascha.roloff@huawei.com>
2025-05-21tree_strcuture test: provide launcher also for just-mrKlaus Aehlig
While there, remove tray cat whitesapce and redetermine the working directory instead of relying on the environment.
2025-05-21invocation server: fix download linksKlaus Aehlig
... to include only the basename of the requested artifact.
2025-05-20invocation server: add critical-path computationSascha Roloff
2025-05-20invocation server: move show_action macro into separate template fileSascha Roloff
... to allow reuse.
2025-05-20invocation server: move action_data function out of do_get_invocation methodSascha Roloff
... to allow reuse.
2025-05-20invocation server: fix typoSascha Roloff
2025-05-20Add end-to-end test that serve can handle canceled requestsKlaus Aehlig
This test actually tests the serve process brought in by the ["end-to-end", "with serve"] rule. It starts several requests to serve and, while serve is still building those, stops the clients. The serve service should handle those aborted requests gracefully and afterwards still be ready to take on a new requests.
2025-05-20end-to-end tests with serve: also keep serve build rootKlaus Aehlig
... if serve is used and remote-execution is a separate service.