summaryrefslogtreecommitdiff
path: root/src/other_tools
AgeCommit message (Collapse)Author
2024-02-01describe: support -RKlaus Aehlig
... as, for absent repositories, we need to get the description from the serve end point. As a consequence, also support -r and --compatible, as the remote-execution endpoint needs to fit with the one for the serve endpoint.
2024-02-01describe: accept local-build-rootKlaus Aehlig
... as even this command now adds entries to CAS, e.g., the shard for the target-level cache. Also, this command block gc by keeping a lock (also in the local build root).
2024-01-31just-mr: Serve endpoint always implies a remote-execution endpointPaul Cristian Sarbu
As the serve service always has an associated remote-execution endpoint, when just-mr gets passed only a serve endpoint address it will now default to implying a remote-execution endpoint exists too at that address. Additionally, now the implementation of the --backup-to-remote option more clearly shows that it can only be done in native mode.
2024-01-31just-mr: Failure in serve api root tree requests should be fatalPaul Cristian Sarbu
The requests to retrieve the tree of a commit, archive, or distdir also set up those trees in a way that the serve endpoint can later build against them, besides allowing just-mr to set up roots locally. Therefore, if the witnessing entity (Git commit, content blob, or distdir, respectively) is known to the serve endpoint, then failing to set up the root tree there should result in a failure also of the just-mr setup on the client side.
2024-01-31serve source tree: Increase server-side granularity in response statusesPaul Cristian Sarbu
For archives and Git repositories we should ensure that not finding the witnessing entity (archive content blob or Git commit, respectively) results in a distinct status in the response to a request that sets up roots on the serve endpoint. This will allow just-mr to better handle its interaction with the serve endpoint.
2024-01-31just-mrrc: support remote-execution propertiesKlaus Aehlig
While just-mr does not use remote-execution properties, it is still useful to have those as a separate entry in the rc file. With rc-file delegation, this gives committed rc files an easy way to specify the image to be used without having to set all the remaining arguments for the various just subcommands in "just args".
2024-01-30Drop unused -R from install-cas and traverse subcommandsKlaus Aehlig
2024-01-30just-mr: add option to dump effective rcKlaus Aehlig
2024-01-30just-mr rc: honor "rc files" optionKlaus Aehlig
... to pull in rc files from different locations, given by location objects.
2024-01-30Add library for overlaying just-mr rcsKlaus Aehlig
2024-01-30just-mr rc handling: split rc-file search and rc interpretationKlaus Aehlig
... to allow, in a clean way, add computing the effective rc by overlaying delegated rc files.
2024-01-26just-mr 'distdir' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. For a 'distdir' repository we can know the resulting tree identifier directly without actually needing to fetch anything. Therefore, we only set the root as absent if the serve endpoint knows already this tree, if it can set it up itself, or if we can provide this tree to the serve endpoint from one of our CAS locations (local or remote), based on our tree invariant guarantee. A network fetch of the archives never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26just-mr 'archive' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for an 'archive' repository we only set the root as absent if the serve endpoint knows already the root, it can set it up itself, or we can create the root locally without a network fetch and then upload it to the serve endpoint via the remote CAS. A network fetch never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26content_git_map: Simplify method signaturesPaul Cristian Sarbu
...by passing around the AsyncMap key struct instead of individual members. This will also make future code changes more easy to implement and improve code readability.
2024-01-26just-mr 'git' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git' repository we only set the root as absent if the serve endpoint knows already the root, it can set it up itself, or we can create the root locally without a network fetch and then upload it to the serve endpoint via the remote CAS. A network fetch never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26just-mr 'git tree' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git tree' repository we only set the root as absent only if the given serve endpoint has this root, or the tree is known locally and can be provided via the remote CAS. While generating an absent root the fetch command will never be called. Generating an absent root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26fpath_git_map: Sync pragma 'to_git' root trees with serve endpointPaul Cristian Sarbu
Marking a file-type repository as 'to_git' results in a Git-tree type root, which are of course content fixed and can be (and usually are) used by export targets. Therefore, it is beneficial for a serve endpoint, if one is provided, to be aware of such a root and be able to build against it if needed. If the root is marked as absent, this condition becomes mandatory. Generating an absent Git-tree root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26root_maps: Add utility library for handling absent roots syncPaul Cristian Sarbu
2024-01-26just-mr async maps: Wrap passed raw pointersPaul Cristian Sarbu
This is to uphold the coding style guide we employ.
2024-01-26just-mr 'git' repository: Fix wrong witnessing repo when remote CAS provides ↵Paul Cristian Sarbu
tree When the remote CAS provides the root tree, we perform an import-to-git operation, therefore the correct witnessing repository for the tree should always be the Git cache.
2024-01-26root_maps: Small cleanup of includes and targets filePaul Cristian Sarbu
...for more easily readable and maintainable target descriptions.
2024-01-26just-mr: Fix missing ExpressionPtr type checks in parsing the config filePaul Cristian Sarbu
2024-01-24just-mr: in to_git warning, report actual tool namesKlaus Aehlig
... instead of some hard-coded strings, as that can be confusing when the tool is packaged under a different name.
2024-01-22just-mr: Honor archive mirrors when only fetchingOliver Reiche
... which were only honored when doing fetch and setup.
2024-01-09just-mr: don't make the command line pretend a fixed nameKlaus Aehlig
Our fetch and launch tool is parametric in the tool to be launched. Reflect this in the documentation and do not pretend it to be the name "just" hard coded. While there, also fix the hard-coded name "git" in the documentation of the default value.
2024-01-08just-mr distdir: Improved handling of absent rootPaul Cristian Sarbu
If root is marked absent and we're not asked to fetch absent, we can compute the resulting distdir root tree in-memory, as we have all the information. Otherwise, i.e., if we actually need to have the archives locally, we first check if the remote serve can provide them in the remote CAS before continuing as before to fetch the listed archives one at a time.
2024-01-08git_tree_fetch_map: Fix missing backup-to-remote behaviourPaul Cristian Sarbu
The change added to interrogate also the local CAS for the tree of a 'git tree' repository failed to add support for backing up such tree found into the remote CAS. This commit fixes the issue.
2023-12-19just-mr git tree: Check for tree also in local CASPaul Cristian Sarbu
Now we look for the tree also in the local CAS, not just in the local Git cache. If found, we import the tree from local CAS to the Git cache and continue as usual.
2023-12-19just-mr fetch content: Check for blob also in Git cachePaul Cristian Sarbu
Now we look for the content blob also in the local Git cache, not just in local CAS. If found, we store the blob read from Git cache into local CAS and continue as usual.
2023-12-19just-mr archive: Check for content blob also in Git cachePaul Cristian Sarbu
Now we look for the content blob also in the local Git cache, not just in local CAS. If found, we store the blob read from Git cache into local CAS and continue as usual.
2023-12-19other_tools: Fix missing includes of direct dependenciesPaul Cristian Sarbu
This fixes the multiple instances of (mainly) missing or wrong includes of standard, third-party, and own headers in the other_tools source folder.
2023-12-19just-mr setup: Add missing reporting of failed hostname replacementPaul Cristian Sarbu
2023-12-19just-mr setup: Fix bug in fetching Git reposPaul Cristian Sarbu
This bug was introduced with the alternative mirrors changes.
2023-12-19Fix just-mr handling of --absentKlaus Aehlig
If the option --absent is given in just-mr (either directly or implicitly via the corresponding entry in the rc file), it rewrites the "pragma" entries in the internal representation of the mr repository config. When doing so, however, we have to take into consideration that a workspace root is not necessarily the definition of a repository, but can, alternatively, also be a reference to another root. This was not taken into account; fix this by restricting the rewriting to only json objects.
2023-12-12just-mr: Fix wrongly assigned field while parsing RC filePaul Cristian Sarbu
2023-12-11just-mrrc: support file options for the launcher functionalityKlaus Aehlig
When just-mr acts as a launcher, for most subcommand options the "just args" entry in the rc files provides a convenient way to set them. However, some options take a file as argument; for those it can be desirable to set them without assuming a fixed file-system layout and instead refer to logical roots, in particular the work space. for the ones that refer to files, it is often desirable to have a potential reference to the work space. Add this functionality.
2023-12-11just-mr: move rc handling to a separate libraryKlaus Aehlig
2023-12-07just-mr fetch: Extend to include 'git tree' repositoriesPaul Cristian Sarbu
This includes implementing the logic for the --backup-to-remote and --fetch-absent options.
2023-12-07just-mr setup: Allow --fetch-absent also for 'git tree' repositoriesPaul Cristian Sarbu
2023-12-07just-mr git tree: Interrogate also just serve endpoint if providedPaul Cristian Sarbu
2023-12-07just-mr git tree: Extract the fetch logic in its own async mapPaul Cristian Sarbu
2023-12-07just-mr fetch: Clarify naming of current fetch async mapPaul Cristian Sarbu
As preparatory step for extending the types of repositories that are targeted by a just-mr fetch subcommand, names of certain variables and of the currently only fetch operation async map were changed to clarify that they only target archives.
2023-12-07just-mr git tree: Inquire remote execution before calling fetch commandPaul Cristian Sarbu
2023-12-07content_cas_map: Remove unnecessary return typePaul Cristian Sarbu
As this map is supposed to work as an all-or-nothing guarantee of content being in CAS, we can replace the bool return type with the null type instead.
2023-12-04git_repo_remote: Fix wrong header include formatPaul Cristian Sarbu
2023-12-01just-mr archive: Report all tried remotes if fetch failsPaul Cristian Sarbu
2023-12-01just-mr git: Report all tried remotes if fetch failsPaul Cristian Sarbu
2023-12-01commit_git_map: Formatting fixPaul Cristian Sarbu
2023-11-21content_cas_map: Interrogate just serve endpoint also if doing pure fetchPaul Cristian Sarbu
Before trying to fetch an archive content from other non-local sources, check first the just serve endpoint.
2023-11-21archive repository: Query 'just serve' endpoint also when not marked absentPaul Cristian Sarbu
...before going to the network, if 'just serve' endpoint given. ContentCASMap was reworked to provide a cleaner logic for handling pure fetches, which also reduced code duplication in ContentGitMap.