summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-07-21bugfix: stage symlinks as symlinks when creating an action directoryAlberto Sartori
Before this patch, when creating an action directory, symlinks were staged as regular files.
2023-07-19README: fix typo in referenceKlaus Aehlig
2023-07-19Add a tutorial on cross compilingKlaus Aehlig
2023-07-17operation_cache: add missing includeMichael Thies
2023-07-14["shell/test", "script"]: support target architectureKlaus Aehlig
... provided it is specified on how to obtain a remote-execution endpoint for the given architecture; in this case, the test (script and) dependencies will be cross compiled for and run on that architecture. The actual building (as well as summarizing multiple test runs) happens on the default execution end point.
2023-07-13bytestream_server: store temporary files under the local-buil-rootAlberto Sartori
The bytestream server implementation (deployed by just execute) now stores the temporary files under $local_build_root/protocol-depenedent/generation-0 so that they can be garbage collected if "just exectue" is terminated before they are cleaned up.
2023-07-12["CC/test", "test]: support target architectureKlaus Aehlig
... provided it is specified on how to obtain a remote-execution endpoint for the given architecture; in this case, the test binary will be cross compiled for and run on that architecture. The actual building (as well as summarizing multiple test runs) happens on the default execution end point.
2023-07-12Add new transtion "maybe for host"Klaus Aehlig
... that transitions to host, unless the ARCH_DISPATCH map indicates that there is an execution-property transition allowing to run actions on the TARGET_ARCH. Also make the "target properties" available as expression as they will be needed when executing the action on the foreign architecture.
2023-07-12Execution properties are no longer a future designKlaus Aehlig
... but an implemented concept. Update the documentation accordingly.
2023-07-11filesystem: Avoid unwanted indirections...Paul Cristian Sarbu
...that std::filesystem::* calls produce. This is because existence and type checks use almost exclusively std::filesystem::status, which follows symbolic links, when being called with path arguments. Instead, one should instead use these methods with the value returned by a call of std::filesystem::symlink_status. This commit also streamlines the FileSystemManager tests, as well as replace bare calls to std::filesystem with their FileSystemManager counterparts (where suitable).
2023-07-11test: Extend coverage for symlinks and ignore_special rootsPaul Cristian Sarbu
2023-07-11Git: Fix handling of symlinks in tree artifactsPaul Cristian Sarbu
The introduction of non-upwards symlinks as first-class objects should have updated the handling of known git tree artifacts containing symlinks. In particular, one should consider trees in their entirety when uploading (irrespective of the ignore_special flag), and git trees should only be reported as known only if the ignore_special flag is set to false.
2023-07-11bootstrap-test: Increase the timeout scalePaul Cristian Sarbu
When running on a busy machine, bootstrap tests can sometimes surpass the current timeout.
2023-07-10target map: Add missing return statement when rule name is not foundPaul Cristian Sarbu
2023-07-10Update CHANGELOG: Fix mishandling of gitignore filesPaul Cristian Sarbu
2023-07-10test: Extend cas-independent check with .gitignore filePaul Cristian Sarbu
2023-07-10just-mr: Fix handling of .gitignore files in git repositoriesPaul Cristian Sarbu
The command 'git add .' does not include paths found in .gitignore files in the directory tree where the command is issued. This is not the desired behaviour, as we expect for a tree with a given commit id to contain all of the entries, irrespective of their meaning to Git. This commit addresses the issue as described. For the just-mr.py script we modified the staging command to 'git add -f .'. For the compiled just-mr, simply adding the force flag to 'git_index_add_all' did not work as intended for files found in ignored subdirectories. This is a known libgit2 issue which has been fixed in v1.6.3. Until we can upgrade our libgit2 version, a workaround was implemented: we recursively read the directory entries ourselves and add each of them iteratively using 'git_index_add_bypath', making sure to ignore the root '.git' subtree (which cannot be staged). At the moment the handling of Git submodules remains an open issue, as Git does not allow '.git' subtrees to be forcefully added to the index, and thus such directory entries will currently not be considered as part of a git tree. This however is consistent behavior between Git and libgit2.
2023-07-10FileSystemManager: Add recursive directory entries reader...Paul Cristian Sarbu
...allowing the skipping of certain subtrees if needed. This is useful, e.g., in simulating what a 'git add' call would do, which ignores all '.git' subdirectories. Also adds a corresponding test for the new method.
2023-07-05Test dispatch functionality of justKlaus Aehlig
2023-07-05Add CLI argument to dispatch endpointKlaus Aehlig
In this way, the user can specify which execution endpoint to use depending on the execution properties.
2023-07-05Executor: dispatch on action propertiesKlaus Aehlig
2023-07-05RemoteExecutionConfiguration: support dispatch listKlaus Aehlig
Extend the configuration data structure by a dispatch list of endpoints to chose based on the first match of the execution properties.
2023-07-05Add serialisation of ServerAddressKlaus Aehlig
... in a way consistent with what we're using at other places already.
2023-07-05just execute: log blob uploads at trace levelKlaus Aehlig
2023-07-05["@", "test", "end-to-end", "with remote"] rule: fix documentationKlaus Aehlig
Besides stdout and stderr of the test, the output of the remote-execution server are also artifacts of that rule.
2023-07-05main: RemoteExecutionConfig is a direct dependencyKlaus Aehlig
... and hence should be declared as such.
2023-07-05["end-to-end/target-cache", "artifacts-sync"]: add more checks...Alberto Sartori
...that the expected objects are correctly uploaded to the remote cas
2023-07-03Simplify artifacts-sync end-to-end testSascha Roloff
2023-07-03local API: Fix bug in uploading missing artifactsPaul Cristian Sarbu
A recent change made it so that the artifact type was wrongly being taken into account when deciding whether the CAS entry of the artifact would be read or not. This meant that non-file artifacts would be skipped from their content being read, resulting in missing CAS artifacts being reported. This fixes the issue by reverting the offending change.
2023-06-28Fix json to filepath conversion in mainSascha Roloff
2023-06-28just-mr: move tmp-workspace under generation rotationKlaus Aehlig
... so that any left-over directories, in case of abnormal termination are cleaned up eventually.
2023-06-28just-mr: lock garbage collectionKlaus Aehlig
just-mr accesses the CAS at several occations, most importantly, the generated multi-repository configuration is written there. In particular, when used as a launcher, it needs to keep that file in its location as the path is passed to the exec of just.
2023-06-28Update CHANGELOG: Relative non-upwards symlinks support as a new featurePaul Cristian Sarbu
2023-06-28doc: Move non-upwards symlinks to conceptsPaul Cristian Sarbu
Also link it in the README.
2023-06-28doc: Fix description of built-in rule genericOliver Reiche
2023-06-28Update remote execution api to preliminary version v2.3Sascha Roloff
In preparation for the introduction of our blob splitting protocol as extension to the remote execution api, we need to update the used remote execution api to a more recent version than v2.0.0. Since no new tags are available right now, we update to the preliminary protocol version v2.3 according to the following discussion: https://github.com/bazelbuild/remote-apis/issues/253
2023-06-28man pages: fix newline in just-mr manpageSascha Roloff
2023-06-26gc: Uplink action output symlinksPaul Cristian Sarbu
2023-06-26tests: Extend executor with symlink artifactsPaul Cristian Sarbu
2023-06-26tests: Add API checks for uploading non-upwards symlinksPaul Cristian Sarbu
2023-06-26Allow non-upwards symlinks with Bazel APIPaul Cristian Sarbu
2023-06-26Allow non-upwards symlinks with Git APIPaul Cristian Sarbu
2023-06-26Allow non-upwards symlinks with local APIPaul Cristian Sarbu
2023-06-26Execution response: Add output symlink pathsPaul Cristian Sarbu
2023-06-26bazel_msg_factory: Allow non-upwards symlinks in uploaded treesPaul Cristian Sarbu
2023-06-26Add non-upwards symlinks in the rule language...Paul Cristian Sarbu
via a 'SYMLINK' constructor function. This works similarly to the 'FILE' construct, but the name given must point to a non-upwards symlink and a symlink artifact is being generated from it. Also updates the relevant tests.
2023-06-26FileRoot: Add handling of non-upwards symlink...Paul Cristian Sarbu
...and update tests accordingly.
2023-06-26ReadTree: Add check for non-upwards symlinks...Paul Cristian Sarbu
...as early as possible. This ensures that callers always receive only the tree entries for the supported object types. For the symlinks non-upwardness check we pass a lambda capturing the real backend of the tree entries, such that the symlinks can be read. Updates git_tree tests accordingly.
2023-06-26tests: Extend artifact description test with non-upwards symlink examplePaul Cristian Sarbu
2023-06-26ObjectType: Add non-upwards symlinks as a known object type...Paul Cristian Sarbu
...but make sure it is still considered a special type. The only non-special entry types remain file, executable, and tree.