summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2024-02-13local_api: also for retrieve to memory, consult git api as fallbackKlaus Aehlig
2024-02-08rule_map: improve error reportingKlaus Aehlig
By showing the full entity name and also adding the usual newline character after every "While ..." clause.
2024-02-08NamedTarget: support ToString()Klaus Aehlig
2024-02-08JsonFileMap: refuse to read absent rootKlaus Aehlig
... instead of erroring on missing file. In this way, whenever a rule or expression from an absent root would have to be read, we get a meaningful error message and not a complaint about a file not being there.
2024-02-08Target map: on error in rule look up, report the rule being looked forKlaus Aehlig
2024-02-01describe: support -RKlaus Aehlig
... as, for absent repositories, we need to get the description from the serve end point. As a consequence, also support -r and --compatible, as the remote-execution endpoint needs to fit with the one for the serve endpoint.
2024-02-01describe: accept local-build-rootKlaus Aehlig
... as even this command now adds entries to CAS, e.g., the shard for the target-level cache. Also, this command block gc by keeping a lock (also in the local build root).
2024-01-31just serve: Cleanup logging to remove tool namePaul Cristian Sarbu
Also cleans up the logging when parsing the serve service configuration file.
2024-01-31just-mr: Serve endpoint always implies a remote-execution endpointPaul Cristian Sarbu
As the serve service always has an associated remote-execution endpoint, when just-mr gets passed only a serve endpoint address it will now default to implying a remote-execution endpoint exists too at that address. Additionally, now the implementation of the --backup-to-remote option more clearly shows that it can only be done in native mode.
2024-01-31just-mr: Failure in serve api root tree requests should be fatalPaul Cristian Sarbu
The requests to retrieve the tree of a commit, archive, or distdir also set up those trees in a way that the serve endpoint can later build against them, besides allowing just-mr to set up roots locally. Therefore, if the witnessing entity (Git commit, content blob, or distdir, respectively) is known to the serve endpoint, then failing to set up the root tree there should result in a failure also of the just-mr setup on the client side.
2024-01-31serve source tree: Increase server-side granularity in response statusesPaul Cristian Sarbu
For archives and Git repositories we should ensure that not finding the witnessing entity (archive content blob or Git commit, respectively) results in a distinct status in the response to a request that sets up roots on the serve endpoint. This will allow just-mr to better handle its interaction with the serve endpoint.
2024-01-31dispatch file parsing: Improve std::variant usagePaul Cristian Sarbu
2024-01-31just-mrrc: support remote-execution propertiesKlaus Aehlig
While just-mr does not use remote-execution properties, it is still useful to have those as a separate entry in the rc file. With rc-file delegation, this gives committed rc files an easy way to specify the image to be used without having to set all the remaining arguments for the various just subcommands in "just args".
2024-01-30Drop unused -R from install-cas and traverse subcommandsKlaus Aehlig
2024-01-30For install-cas with remember use splittingKlaus Aehlig
2024-01-30just-mr: add option to dump effective rcKlaus Aehlig
2024-01-30just-mr rc: honor "rc files" optionKlaus Aehlig
... to pull in rc files from different locations, given by location objects.
2024-01-30Add library for overlaying just-mr rcsKlaus Aehlig
2024-01-30just-mr rc handling: split rc-file search and rc interpretationKlaus Aehlig
... to allow, in a clean way, add computing the effective rc by overlaying delegated rc files.
2024-01-26just-mr 'distdir' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. For a 'distdir' repository we can know the resulting tree identifier directly without actually needing to fetch anything. Therefore, we only set the root as absent if the serve endpoint knows already this tree, if it can set it up itself, or if we can provide this tree to the serve endpoint from one of our CAS locations (local or remote), based on our tree invariant guarantee. A network fetch of the archives never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26just-mr 'archive' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for an 'archive' repository we only set the root as absent if the serve endpoint knows already the root, it can set it up itself, or we can create the root locally without a network fetch and then upload it to the serve endpoint via the remote CAS. A network fetch never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26content_git_map: Simplify method signaturesPaul Cristian Sarbu
...by passing around the AsyncMap key struct instead of individual members. This will also make future code changes more easy to implement and improve code readability.
2024-01-26just-mr 'git' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git' repository we only set the root as absent if the serve endpoint knows already the root, it can set it up itself, or we can create the root locally without a network fetch and then upload it to the serve endpoint via the remote CAS. A network fetch never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26just-mr 'git tree' repository: Absent roots sync with serve endpointPaul Cristian Sarbu
To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git tree' repository we only set the root as absent only if the given serve endpoint has this root, or the tree is known locally and can be provided via the remote CAS. While generating an absent root the fetch command will never be called. Generating an absent root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26fpath_git_map: Sync pragma 'to_git' root trees with serve endpointPaul Cristian Sarbu
Marking a file-type repository as 'to_git' results in a Git-tree type root, which are of course content fixed and can be (and usually are) used by export targets. Therefore, it is beneficial for a serve endpoint, if one is provided, to be aware of such a root and be able to build against it if needed. If the root is marked as absent, this condition becomes mandatory. Generating an absent Git-tree root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26root_maps: Add utility library for handling absent roots syncPaul Cristian Sarbu
2024-01-26serve source tree: Client-side and API implementations of to_git ↵Paul Cristian Sarbu
pragma-related RPCs
2024-01-26serve source tree: Server-side implementation of to_git pragma-related RPCsPaul Cristian Sarbu
2024-01-26just_serve.proto: Extend SourceTree with to_git pragma-related requestsPaul Cristian Sarbu
2024-01-26serve source tree: Distdir tree should end up in Git cachePaul Cristian Sarbu
The purpose of the requests for the tree of an archive, commit, or distdir also includes making those trees available for future builds on the serve endpoint, which currently means being in a known Git repository. This commit ensures the distdir tree reqeust also includes the import of the resulting tree from CAS into the Git cache (if the tree is not already in a Git repsoitory).
2024-01-26just-mr async maps: Wrap passed raw pointersPaul Cristian Sarbu
This is to uphold the coding style guide we employ.
2024-01-26just-mr 'git' repository: Fix wrong witnessing repo when remote CAS provides ↵Paul Cristian Sarbu
tree When the remote CAS provides the root tree, we perform an import-to-git operation, therefore the correct witnessing repository for the tree should always be the Git cache.
2024-01-26root_maps: Small cleanup of includes and targets filePaul Cristian Sarbu
...for more easily readable and maintainable target descriptions.
2024-01-26just-mr: Fix missing ExpressionPtr type checks in parsing the config filePaul Cristian Sarbu
2024-01-24just-mr: in to_git warning, report actual tool namesKlaus Aehlig
... instead of some hard-coded strings, as that can be confusing when the tool is packaged under a different name.
2024-01-24just: Wrap IExecutionApi raw pointer argumentsPaul Cristian Sarbu
...in accordance to our coding style.
2024-01-22just-mr: Honor archive mirrors when only fetchingOliver Reiche
... which were only honored when doing fetch and setup.
2024-01-19Add tree invariant check for just execute, when uploading treesSascha Roloff
2024-01-19Add digest validity check to the blob split commandSascha Roloff
2024-01-19Fix digest consistency check to also compare sizesSascha Roloff
2024-01-16Keep implied target-cache values aliveKlaus Aehlig
... by uplinking them appropriately.
2024-01-16TargetCacheEntry: serialize implied export targetsKlaus Aehlig
2024-01-16Analysed target: keep track of implied export targetsKlaus Aehlig
... that are eligible for caching. In this way, we can accurately keep track of the dependencies between target-level cache entries. Note that it is enough to track the export targets eligible for caching, as no target depending on an ineligible export target can be eligible.
2024-01-16TargetCacheEntry: return implied targets, if serializedKlaus Aehlig
2024-01-09just-mr: don't make the command line pretend a fixed nameKlaus Aehlig
Our fetch and launch tool is parametric in the tool to be launched. Reflect this in the documentation and do not pretend it to be the name "just" hard coded. While there, also fix the hard-coded name "git" in the documentation of the default value.
2024-01-09Refactor GraphTraverser to take in platform properties and dispatch listPaul Cristian Sarbu
In order for the serve endpoint to correctly dispatch a build to the correct remote-execution endpoint, the platform properties and dispatch list for a build need to be passed explicitly to the executor (via the graph traverser instance) instead of always being taken from the RemoteExecutionConfig struct. This commit implements these changes, including updating existing tests accordingly.
2024-01-09just main: Fix log messagePaul Cristian Sarbu
2024-01-08just-mr distdir: Improved handling of absent rootPaul Cristian Sarbu
If root is marked absent and we're not asked to fetch absent, we can compute the resulting distdir root tree in-memory, as we have all the information. Otherwise, i.e., if we actually need to have the archives locally, we first check if the remote serve can provide them in the remote CAS before continuing as before to fetch the listed archives one at a time.
2024-01-08serve distdir tree: Client-side and API implementationsPaul Cristian Sarbu
2024-01-08serve distdir tree: Server-side implementationPaul Cristian Sarbu