summaryrefslogtreecommitdiff
path: root/src/other_tools
AgeCommit message (Collapse)Author
2023-11-02content git map: Move handling of --fetch-absent flag from checkout to the ↵Paul Cristian Sarbu
map itself
2023-11-02Move content archive helper functions in own libraryPaul Cristian Sarbu
2023-11-02Decoupling symlinks map and CAS utilities from just-mrPaul Cristian Sarbu
This is required in order to make them available to 'just serve' in a minimal just installation.
2023-11-02ArchiveOps: Move libarchive utilities outside other_toolsPaul Cristian Sarbu
This way they can be used by 'just serve'.
2023-11-02GitRepo: Add method for async fetch from local repositoryPaul Cristian Sarbu
This avoids using the more geenric GitRepoRemote method which has libcurl as a dependency, something that is not needed for this Git operation.
2023-11-02content CAS map: Check remote CAS only after local distfilesPaul Cristian Sarbu
2023-10-16curl_easy_handle: Ensure we report a fail on HTTP codes >=400Paul Cristian Sarbu
...which signal either cient- or server-side failures. This ensures we exit with a failed network fetch early in cases where it is clear we won't receive useful data.
2023-10-16just-mr fetch: Fix exception on fetched data hash mismatchPaul Cristian Sarbu
After successfully fetching data over the network for an archive, if the optional checksums are not provided, the code will throw if there is a mismatch between the hash of the fetched data stream and the provided content hash. This commit fixes the issue by adding an additional check which properly handles the possible mismatch.
2023-09-22commit git map: Fix --fetch-absent storing incorrect tree associationPaul Cristian Sarbu
In order to not pollute builds, one should only store in the id file the association between a commit and its *root* tree. As the need for such a file only happens when asked to fetch absent roots, which should be a quite rare use case, we can request the whole tree of a commit to be served (incl. synced with remote execution) and perform the subdir tree extraction locally instead.
2023-09-22commit git map: Improve warning messagePaul Cristian Sarbu
2023-09-22fpath git map: Remove redundancy in warning messagePaul Cristian Sarbu
2023-09-20just-mr: add option to override the absent pragmaKlaus Aehlig
Using absent repositories depends on a just-serve endpoint. As such, it is not a datum of the project, but one of the available infrastructure. As the latter can change independently of the project, it is desriable to have the option to specify those pieces of information in separate files.
2023-09-15just-mr: ensure that after --fetch-absent a local build is possibleKlaus Aehlig
... by also keeping the map of commit to tree locally.
2023-09-15just serve: add remote execution endpoint and --fetch-absent optionPaul Cristian Sarbu
The serve service will communicate with this endpoint when needed, as well as ensure artifacts it provides are synced with the remote execution CAS, if requested by the client. If just-mr is given the --fetch-absent option, it Always produce present roots irrespective of the 'absent' pragma. For Git repositories marked with the 'absent' pragma, first try to fetch any commit trees provided by the serve endpoint from the execution endpoint CAS, before reverting to a network fetch. Co-authored-by: Klaus Aehlig <klaus.aehlig@huawei.com> Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
2023-09-13just-mr: Implement 'absent' rootsPaul Cristian Sarbu
...via an 'absent' pragma in repository descriptions. For 'git'-type repositories, first interrogates a 'just serve' remote, if given, before reverting to fetching from the network.
2023-09-13just serve: Add endpoint argumentsPaul Cristian Sarbu
2023-09-13just-mr fetch: Fix minor typo in digest checkPaul Cristian Sarbu
The IsAvailable method with the correct signature is now called.
2023-09-07just-mr: allow subkeys to be skipped in RC filePaul Cristian Sarbu
If a key has a map value, do not force all its subkeys to be present. Fixes changes brought in with supporting remote execution and authentication options in just-mr.
2023-08-25just-mr fetch: Allow to back up fetched archives to a given remote CASPaul Cristian Sarbu
2023-08-25just-mr: Interrogate remote CAS before defaulting to network fetch...Paul Cristian Sarbu
...for archives not already in local CAS.
2023-08-25just-mr utils: Extract setup-related code into separate libraryPaul Cristian Sarbu
This better separates the utility methods used in just-mr commands and avoids any cyclic dependencies that might arise in new utility methods requiring both existing utilities and the command-line arguments.
2023-08-25just-mr: Add remote execution and mTLS argumentsPaul Cristian Sarbu
These have similar meanings as in 'just', are forwarded to 'just' subcommands that support them, and can be set also in the RC file via appropriate fields.
2023-08-23just-mr: Split main codePaul Cristian Sarbu
...by moving subcommands code into separate libraries. This maintains a cleaner code structure and lowers the build time by improving target caching.
2023-08-14just-mr: More consistent error messages in parsing RC filePaul Cristian Sarbu
2023-08-14just-mr archives: Add 7zip support via autodetection optionPaul Cristian Sarbu
Similarly to tarballs, an autodetection option for zip-like archives is added to enable a unified handling of both traditional zip and 7zip formats. Thus, for "zip" archives just-mr uses now this autodetection option.
2023-08-14ArchiveOps: Add handling of 7zip archivesPaul Cristian Sarbu
Also updates the archive usage tests accordingly.
2023-08-14ArchiveOps: Allow lzip and lzma compression filters for tarballsPaul Cristian Sarbu
These are provided by the recently added liblzma external dependency.
2023-08-14ArchiveOps: Add proper autodetection option for tarballsPaul Cristian Sarbu
We shouldn't exclude the possibility of receiving uncompressed tarballs as repositories in just-mr. Therefore, we introduce an explicit type that performs the autodetection (default behaviour in just-mr). This is done to also be more in line with our implementation which allows the granular handling of various archive types (currently used only for testing purposes).
2023-08-14ArchiveOps: reduce verbosity in archive type enumPaul Cristian Sarbu
2023-08-07just-mr maps: Add logic for resolving symlinks in archive repositoriesPaul Cristian Sarbu
2023-08-07just-mr maps: Add logic for resolving symlinks in file repositoriesPaul Cristian Sarbu
2023-08-07just-mr utils: Add resolved tree id file getterPaul Cristian Sarbu
2023-08-07just-mr: Add async map for resolving symlinks in Git treesPaul Cristian Sarbu
2023-08-07just-mr: Update async map keys with the 'special' pragma valuePaul Cristian Sarbu
For 'file' and 'archive' checkouts we will handle more than just the 'ignore' value.
2023-08-07just-mr: Add map for pragma 'special' valuesPaul Cristian Sarbu
Moves away from literal strings in the async maps.
2023-08-07just-mr: Set ignore-special roots via a pragma...Paul Cristian Sarbu
...instead of a regular field. Also removes the ignore-special logic from distdir repositories, as those will never contain symlinks, so no entries would be ignored.
2023-08-07just-mr maps: Clean up remnants from progress reporting reworkPaul Cristian Sarbu
2023-06-28Fix json to filepath conversion in mainSascha Roloff
2023-06-28just-mr: move tmp-workspace under generation rotationKlaus Aehlig
... so that any left-over directories, in case of abnormal termination are cleaned up eventually.
2023-06-28just-mr: lock garbage collectionKlaus Aehlig
just-mr accesses the CAS at several occations, most importantly, the generated multi-repository configuration is written there. In particular, when used as a launcher, it needs to keep that file in its location as the path is passed to the exec of just.
2023-06-20just-mr: Auto detect archive compressionOliver Reiche
2023-06-20ArchiveOps: Support unpacking tar.xz archivesOliver Reiche
2023-06-06style: Use designated initializersPaul Cristian Sarbu
This feature has been introduced with C++20.
2023-05-31Support FINAL_LDFLAGS variable for binariesOliver Reiche
... and set default stack size to 8 MB.
2023-05-31git tree repos: Add option for ignore-special root...Paul Cristian Sarbu
...and fix missing origin field initialization for progress tracing.
2023-05-31distdir repos: Add option for ignore-special rootPaul Cristian Sarbu
2023-05-31archive repos: Add option for ignore-special rootPaul Cristian Sarbu
2023-05-31git repos: Add option for ignore-special rootPaul Cristian Sarbu
2023-05-31file repos: Add option for ignore-special rootPaul Cristian Sarbu
2023-05-31utils: Relocate the std::filesystem::hash_value libc fix...Paul Cristian Sarbu
...and make the use of std::hash consistent. This will make it easier to remove the fix once the libc implementation we use catches up with the C++ standard.