summaryrefslogtreecommitdiff
path: root/src/other_tools/repo_map
AgeCommit message (Collapse)Author
2024-03-05Support checkout of foreign-file reposKlaus Aehlig
2024-03-04serve protocol: support executable distfilesKlaus Aehlig
As serve and just-mr share their caching of description-tree association, we also have to change this cache. Thanks to json encoding before hashing, we know that the old and new hash keys do not overlap, so the change is save. As distdirs also keep the respective files in the git root, no new downlaoding will happen either, hence no warning in the CHANGELOG is needed.
2024-02-28Deduplicate parsing of archive descriptionsKlaus Aehlig
2024-02-27just-mr setup archive: Local roots require the archive blob to be local tooPaul Cristian Sarbu
For archive repositories we need to ensure that a non-absent root is backed by an archive content blob in the local CAS, in order to also keep the proper root tree file associations. This change also simplifies the content_cas_map logic by removing the previous separation of implementation logic between fetching and setting up the workspace root.
2024-02-20git repo fetch: support "inherit env"Klaus Aehlig
When fetching git repositories, just-mr routinely shells out to git. In this case, allow the user to specify via "inherit env", which environment variables from the host environment should be made available in this action. Typical variables to inherit are ones providing credentials, like SSH_AUTH_SOCK. As the repository description specifies the commit that will be taken, and hence the resulting tree, correctness is not affected by the environement leaking in here.
2024-01-26just-mr: Fix missing ExpressionPtr type checks in parsing the config filePaul Cristian Sarbu
2023-12-19other_tools: Fix missing includes of direct dependenciesPaul Cristian Sarbu
This fixes the multiple instances of (mainly) missing or wrong includes of standard, third-party, and own headers in the other_tools source folder.
2023-12-07just-mr git tree: Extract the fetch logic in its own async mapPaul Cristian Sarbu
2023-11-21archive repository: Query 'just serve' endpoint also when not marked absentPaul Cristian Sarbu
...before going to the network, if 'just serve' endpoint given. ContentCASMap was reworked to provide a cleaner logic for handling pure fetches, which also reduced code duplication in ContentGitMap.
2023-11-15just-mr json parsing: Add missing check for items type in listsPaul Cristian Sarbu
2023-11-14just-mr: Add 'mirrors' field to archive-like repositoriesPaul Cristian Sarbu
Also extends 'distdir' repositories logic accordingly.
2023-11-14DistdirCheckout: Improve log messagesPaul Cristian Sarbu
2023-11-14just-mr: Add 'mirrors' field to 'git' repositoriesPaul Cristian Sarbu
2023-11-13"git tree" repositories: honor "inherit env"Klaus Aehlig
In the specification of the action generating a fixed git tree, also honor "inherit env", i.e., inherit the environment variables specified in this field from the environment just-mr is invoked in. As the expected output is fixed ahead of time anyway, this lack of isolation does not affect correctness.
2023-11-02content git map: Move handling of --fetch-absent flag from checkout to the ↵Paul Cristian Sarbu
map itself
2023-11-02Decoupling symlinks map and CAS utilities from just-mrPaul Cristian Sarbu
This is required in order to make them available to 'just serve' in a minimal just installation.
2023-09-15just serve: add remote execution endpoint and --fetch-absent optionPaul Cristian Sarbu
The serve service will communicate with this endpoint when needed, as well as ensure artifacts it provides are synced with the remote execution CAS, if requested by the client. If just-mr is given the --fetch-absent option, it Always produce present roots irrespective of the 'absent' pragma. For Git repositories marked with the 'absent' pragma, first try to fetch any commit trees provided by the serve endpoint from the execution endpoint CAS, before reverting to a network fetch. Co-authored-by: Klaus Aehlig <klaus.aehlig@huawei.com> Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
2023-09-13just-mr: Implement 'absent' rootsPaul Cristian Sarbu
...via an 'absent' pragma in repository descriptions. For 'git'-type repositories, first interrogates a 'just serve' remote, if given, before reverting to fetching from the network.
2023-08-07just-mr: Update async map keys with the 'special' pragma valuePaul Cristian Sarbu
For 'file' and 'archive' checkouts we will handle more than just the 'ignore' value.
2023-08-07just-mr: Add map for pragma 'special' valuesPaul Cristian Sarbu
Moves away from literal strings in the async maps.
2023-08-07just-mr: Set ignore-special roots via a pragma...Paul Cristian Sarbu
...instead of a regular field. Also removes the ignore-special logic from distdir repositories, as those will never contain symlinks, so no entries would be ignored.
2023-06-06style: Use designated initializersPaul Cristian Sarbu
This feature has been introduced with C++20.
2023-05-31git tree repos: Add option for ignore-special root...Paul Cristian Sarbu
...and fix missing origin field initialization for progress tracing.
2023-05-31distdir repos: Add option for ignore-special rootPaul Cristian Sarbu
2023-05-31archive repos: Add option for ignore-special rootPaul Cristian Sarbu
2023-05-31git repos: Add option for ignore-special rootPaul Cristian Sarbu
2023-05-31file repos: Add option for ignore-special rootPaul Cristian Sarbu
2023-04-26just-mr: Fix warning about unused lambda captureOliver Reiche
2023-03-20just-mr: rework progress reporting and statisticsKlaus Aehlig
To avoid unnecessary work, just-mr uses on-disk caches, including for the mapping of a distdir to the corresponding git tree. This, however, implies that the number of repositories that are actually considered varies: in order to fetch a distdir repository, all involved archives have to be fetched, but if we have a cache hit none of them is even looked at. So, in order to have a consistent reporting only count top-level targets (i.e., the reachable repositories) in the statistics, not the archives implicitly contained in a distdir, nor low-level sub tasks. The actual fetch acitvity is shown separately by the task tracker.
2023-03-09Just-MR: Improve logging messagesPaul Cristian Sarbu
... by reporting unsupported config values explicitly and fixing inconsistencies.
2023-03-08Just-MR: Add logic for git tree repository type setupPaul Cristian Sarbu
2023-02-27Just-MR: Add progress reporting for setup commandPaul Cristian Sarbu
This includes also the setup-env command, as well as all ~just~ known subcommands that require a just-mr setup step. Co-authored-by: Sascha Roloff <sascha.roloff@huawei.com>
2023-02-27Just-MR: Extend ops and root maps keys with fields needed in progress reportingPaul Cristian Sarbu
In order to only report actual work, the name of the repository for which work is actually done needs to be known to certain AsyncMaps used by just-mr. Also, fetches not originating in a distdir repo checkout need to be counted additionally, so an additional flag was needed for the content_cas_map.
2023-01-10repos_to_setup_map: Remove unnecessary logger capture in lambda callsPaul Cristian Sarbu
2023-01-10Just-MR: Fix distdir computation in internal just-mrPaul Cristian Sarbu
This provides the fix already performed for the just-mr script in the internal just-mr as well. In short, this fix makes sure that the computation of the content key for distdir repositories is independent of the presence of the respective archives in CAS.
2022-12-23Just-MR: Remove wrong pass-by-reference when wrapping loggersPaul Cristian Sarbu
Passing the logger by reference would require the caller to be kept alive. Also, being a shared_ptr, the logger can be passed by value at almost no cost.
2022-12-21Just-MR: Add repos-to-setup mapPaul Cristian Sarbu
Contains the logic for the checkout of all supported repositories.