summaryrefslogtreecommitdiff
path: root/src/other_tools/root_maps
AgeCommit message (Collapse)Author
2024-01-26content_git_map: Simplify method signaturesPaul Cristian Sarbu
...by passing around the AsyncMap key struct instead of individual members. This will also make future code changes more easy to implement and improve code readability.
2024-01-26just-mr 'git' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git' repository we only set the root as absent if the serve endpoint knows already the root, it can set it up itself, or we can create the root locally without a network fetch and then upload it to the serve endpoint via the remote CAS. A network fetch never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26just-mr 'git tree' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git tree' repository we only set the root as absent only if the given serve endpoint has this root, or the tree is known locally and can be provided via the remote CAS. While generating an absent root the fetch command will never be called. Generating an absent root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26fpath_git_map: Sync pragma 'to_git' root trees with serve endpointPaul Cristian Sarbu
Marking a file-type repository as 'to_git' results in a Git-tree type root, which are of course content fixed and can be (and usually are) used by export targets. Therefore, it is beneficial for a serve endpoint, if one is provided, to be aware of such a root and be able to build against it if needed. If the root is marked as absent, this condition becomes mandatory. Generating an absent Git-tree root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26root_maps: Add utility library for handling absent roots syncPaul Cristian Sarbu
2024-01-26just-mr async maps: Wrap passed raw pointersPaul Cristian Sarbu
This is to uphold the coding style guide we employ.
2024-01-26just-mr 'git' repository: Fix wrong witnessing repo when remote CAS provides ↵Paul Cristian Sarbu
tree When the remote CAS provides the root tree, we perform an import-to-git operation, therefore the correct witnessing repository for the tree should always be the Git cache.
2024-01-26root_maps: Small cleanup of includes and targets filePaul Cristian Sarbu
...for more easily readable and maintainable target descriptions.
2024-01-24just-mr: in to_git warning, report actual tool namesKlaus Aehlig
... instead of some hard-coded strings, as that can be confusing when the tool is packaged under a different name.
2024-01-08just-mr distdir: Improved handling of absent rootPaul Cristian Sarbu
If root is marked absent and we're not asked to fetch absent, we can compute the resulting distdir root tree in-memory, as we have all the information. Otherwise, i.e., if we actually need to have the archives locally, we first check if the remote serve can provide them in the remote CAS before continuing as before to fetch the listed archives one at a time.
2023-12-19just-mr archive: Check for content blob also in Git cachePaul Cristian Sarbu
Now we look for the content blob also in the local Git cache, not just in local CAS. If found, we store the blob read from Git cache into local CAS and continue as usual.
2023-12-19other_tools: Fix missing includes of direct dependenciesPaul Cristian Sarbu
This fixes the multiple instances of (mainly) missing or wrong includes of standard, third-party, and own headers in the other_tools source folder.
2023-12-19just-mr setup: Add missing reporting of failed hostname replacementPaul Cristian Sarbu
2023-12-19just-mr setup: Fix bug in fetching Git reposPaul Cristian Sarbu
This bug was introduced with the alternative mirrors changes.
2023-12-07just-mr setup: Allow --fetch-absent also for 'git tree' repositoriesPaul Cristian Sarbu
2023-12-07just-mr git tree: Extract the fetch logic in its own async mapPaul Cristian Sarbu
2023-12-07just-mr git tree: Inquire remote execution before calling fetch commandPaul Cristian Sarbu
2023-12-01just-mr git: Report all tried remotes if fetch failsPaul Cristian Sarbu
2023-12-01commit_git_map: Formatting fixPaul Cristian Sarbu
2023-11-21archive repository: Query 'just serve' endpoint also when not marked absentPaul Cristian Sarbu
...before going to the network, if 'just serve' endpoint given. ContentCASMap was reworked to provide a cleaner logic for handling pure fetches, which also reduced code duplication in ContentGitMap.
2023-11-21git repository: Query 'just serve' endpoint also when not marked absentPaul Cristian Sarbu
...before going to the network, if 'just serve' endpoint given.
2023-11-15just-mr: Update to using the static methods of ServeApiPaul Cristian Sarbu
2023-11-15git tree repository: Creating absent roots should not run the commandPaul Cristian Sarbu
Also extended absent-roots test with this scenario.
2023-11-14just-mr archives fetch: Add logic for local mirrors and preferred hostnamesPaul Cristian Sarbu
2023-11-14just-mr git fetch: Add logic for local mirrors and preferred hostnamesPaul Cristian Sarbu
2023-11-14just-mr: Add 'mirrors' field to archive-like repositoriesPaul Cristian Sarbu
Also extends 'distdir' repositories logic accordingly.
2023-11-14just-mr: Add 'mirrors' field to 'git' repositoriesPaul Cristian Sarbu
2023-11-14commit_git_map fix: Fetch from correct location if fetch URL is a pathPaul Cristian Sarbu
2023-11-13"git tree" repositories: honor "inherit env"Klaus Aehlig
In the specification of the action generating a fixed git tree, also honor "inherit env", i.e., inherit the environment variables specified in this field from the environment just-mr is invoked in. As the expected output is fixed ahead of time anyway, this lack of isolation does not affect correctness.
2023-11-02content git map: Add handling of --fetch-absent optionPaul Cristian Sarbu
2023-11-02content git map: Move handling of --fetch-absent flag from checkout to the ↵Paul Cristian Sarbu
map itself
2023-11-02Decoupling symlinks map and CAS utilities from just-mrPaul Cristian Sarbu
This is required in order to make them available to 'just serve' in a minimal just installation.
2023-11-02ArchiveOps: Move libarchive utilities outside other_toolsPaul Cristian Sarbu
This way they can be used by 'just serve'.
2023-09-22commit git map: Fix --fetch-absent storing incorrect tree associationPaul Cristian Sarbu
In order to not pollute builds, one should only store in the id file the association between a commit and its *root* tree. As the need for such a file only happens when asked to fetch absent roots, which should be a quite rare use case, we can request the whole tree of a commit to be served (incl. synced with remote execution) and perform the subdir tree extraction locally instead.
2023-09-22commit git map: Improve warning messagePaul Cristian Sarbu
2023-09-22fpath git map: Remove redundancy in warning messagePaul Cristian Sarbu
2023-09-15just-mr: ensure that after --fetch-absent a local build is possibleKlaus Aehlig
... by also keeping the map of commit to tree locally.
2023-09-15just serve: add remote execution endpoint and --fetch-absent optionPaul Cristian Sarbu
The serve service will communicate with this endpoint when needed, as well as ensure artifacts it provides are synced with the remote execution CAS, if requested by the client. If just-mr is given the --fetch-absent option, it Always produce present roots irrespective of the 'absent' pragma. For Git repositories marked with the 'absent' pragma, first try to fetch any commit trees provided by the serve endpoint from the execution endpoint CAS, before reverting to a network fetch. Co-authored-by: Klaus Aehlig <klaus.aehlig@huawei.com> Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
2023-09-13just-mr: Implement 'absent' rootsPaul Cristian Sarbu
...via an 'absent' pragma in repository descriptions. For 'git'-type repositories, first interrogates a 'just serve' remote, if given, before reverting to fetching from the network.
2023-08-14just-mr archives: Add 7zip support via autodetection optionPaul Cristian Sarbu
Similarly to tarballs, an autodetection option for zip-like archives is added to enable a unified handling of both traditional zip and 7zip formats. Thus, for "zip" archives just-mr uses now this autodetection option.
2023-08-14ArchiveOps: Add proper autodetection option for tarballsPaul Cristian Sarbu
We shouldn't exclude the possibility of receiving uncompressed tarballs as repositories in just-mr. Therefore, we introduce an explicit type that performs the autodetection (default behaviour in just-mr). This is done to also be more in line with our implementation which allows the granular handling of various archive types (currently used only for testing purposes).
2023-08-14ArchiveOps: reduce verbosity in archive type enumPaul Cristian Sarbu
2023-08-07just-mr maps: Add logic for resolving symlinks in archive repositoriesPaul Cristian Sarbu
2023-08-07just-mr maps: Add logic for resolving symlinks in file repositoriesPaul Cristian Sarbu
2023-08-07just-mr: Update async map keys with the 'special' pragma valuePaul Cristian Sarbu
For 'file' and 'archive' checkouts we will handle more than just the 'ignore' value.
2023-08-07just-mr: Set ignore-special roots via a pragma...Paul Cristian Sarbu
...instead of a regular field. Also removes the ignore-special logic from distdir repositories, as those will never contain symlinks, so no entries would be ignored.
2023-08-07just-mr maps: Clean up remnants from progress reporting reworkPaul Cristian Sarbu
2023-06-20just-mr: Auto detect archive compressionOliver Reiche
2023-06-06style: Use designated initializersPaul Cristian Sarbu
This feature has been introduced with C++20.
2023-05-31git tree repos: Add option for ignore-special root...Paul Cristian Sarbu
...and fix missing origin field initialization for progress tracing.
generated by cgit v1.2.3 (git 2.39.1) at 2025-07-20 06:57:20 +0000