summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-03-20Add new subcommand add-to-casKlaus Aehlig
2024-03-20Add functionality to add local files to CASKlaus Aehlig
... and optionally upload them to a remote-execution endpoint.
2024-03-20Add data-structure specific for adding files to CASKlaus Aehlig
2024-03-19serve target: Remove unneeded backend description uploadPaul Cristian Sarbu
2024-03-19serve target: Fix sharding inconsistenciesPaul Cristian Sarbu
When running in single node, serve endpoint should not even consider sharding. Additionally, garbage collection uplinking should also take the shard into account. For this purpose, a TargetCache instance now remembers if it was explicitly sharded and passed that information to the GarbageCollector for uplinking.
2024-03-19garbage_collector: Fix small doc typosPaul Cristian Sarbu
2024-03-19Add test to check handling of failed serve endpoint buildsPaul Cristian Sarbu
2024-03-19serve target: Differentiate between fatal and non-fatal orchestrated buildsPaul Cristian Sarbu
...by increasing granularity in client-side reporting. This allows to correctly continue with builds of local targets if the serve endpoint does not have the requested target, as well as improve the reporting for users on failure.
2024-03-19serve target server: Provide client the log of failed target buildsPaul Cristian Sarbu
When the serve endpoint has all the information to analyse a target, any analysis or build errors should be made available to the client. This way the user can receive a relevant error report.
2024-03-19serve target: Small improvementsPaul Cristian Sarbu
2024-03-19WriteTargetCacheEntries: Be explicit in logging location...Paul Cristian Sarbu
...by allowing a Logger instance to be provided. Also adds a flag in order for failed artifacts from builds orchestrated by the serve endpoint to be able to be reported as errors instead of warnings.
2024-03-19graph traverser: Be explicit in logging locationPaul Cristian Sarbu
2024-03-19result_map: Be explicit in logging location when getting results...Paul Cristian Sarbu
...by allowing a Logger instance to be provided.
2024-03-19just analysis: Be explicit in logging locationPaul Cristian Sarbu
...by allowing a Logger instance to be provided.
2024-03-19logger: Add common methods for global and named loggersPaul Cristian Sarbu
This allows to be explicit and thus have better control on where messages get logged.
2024-03-19just: Report non-zero served targets after analysisPaul Cristian Sarbu
2024-03-18graph_traverser: Remove unnecessary get callPaul Cristian Sarbu
...as it is redundant here and exception prone.
2024-03-18parallelism: Do not spawn more threads than asked toPaul Cristian Sarbu
...in the tool proper, i.e., excepting tests.
2024-03-15Clean up more includes and targetsPaul Cristian Sarbu
Some of the more specific issues addressed: - missing log_level target/include - header-only libs wrongly marking deps as private - missing/misplaced gsl includes
2024-03-14Add end-to-end test with serve building several targetsKlaus Aehlig
... simultaneously, in order to find race conditions during the build.
2024-03-14export of gsl: add missing flexible_config "ENV"Klaus Aehlig
As a library, that target depends implicitly on the toolchain, hence the already present export of "TOOLCHAIN_CONFIG", but also the environment for using that toolchain. So, export "ENV" as well.
2024-03-13Export more third-party top-level entry pointsKlaus Aehlig
... so that they can be served and hence the corresponding dependency can be absent.
2024-03-13BytestreamServer: also use the correct interface function for tmp dirsKlaus Aehlig
... so that any updates of the local-build-root layout are correctly taken into account. In particular, this change also moves the temporary directory under the emphemeral root, allowing more quick clean up. Co-authored-by: Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com>
2024-03-13Fix tmpdir locationKlaus Aehlig
Commit 2ebf355989eb92ac9967eceee0af14d39477afe0 moved the tmpdir creation for various tasks into the task itself. In doing so, TmpDir was called with a relative path; that was, however, is interpreted relative to the working directory, violating the property that our tool never write anything outside the local build root unless explicitly asked to do so (by specifying the output path in an install or install-cas invocation). Fix this, by calling the the tmp-dir function that is storage-layout aware.
2024-03-13Move storage-aware tmpdir creation to configKlaus Aehlig
... as the fs_utils have a lot more dependencies making them usable in less places. Moreover, this function also serves to shape the layout of the local build root and hence is more appropriately placed in the config anyway.
2024-03-12catch2 main: ensure proper build rootKlaus Aehlig
Some of our C-tests under catch2 may rely on the local build root to get a proper tmp directory; ensure this is set properly to avoid falling back to the user's home directory.
2024-03-12storage config: remove unneeded private dependencyKlaus Aehlig
2024-03-12Exports-progress reporter: do not encode sampleKlaus Aehlig
... as it is already encoded to be meaningfully printable.
2024-03-12Export-target progress reporting: report the target nameKlaus Aehlig
... with effective config instead of the actual cache key, which is simply a blob identifier that is probably not so meaningful for the user watching the build.
2024-03-12target evalution: handle error in dependency evaluationKlaus Aehlig
... instead of blindly assuming the evaluation succeeds. Co-authord-by: Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com>
2024-03-12just serve: Improve progress reportingPaul Cristian Sarbu
During analysis it is useful to track and report the progress for all export targets. This is not exclusively linked to a serve endpoint being present, despite most of the time being expected to be spent in export targets being served from the remote endpoint. This commit refactors the current implementation to give proper feedback to the user on the progress of the analysis phase.
2024-03-12serve target: Report the analysed targetPaul Cristian Sarbu
...when orchestrating a build.
2024-03-11target_map: Typo fix in log messagePaul Cristian Sarbu
2024-03-11just: Replace singletons for progress tracking and statistics...Paul Cristian Sarbu
...with regular instances that have controlled life-times. This avoids race conditions in tracking and reporting the results of analysis and build, as the serve endpoint can orchestrate multiple builds at the same time asynchronously. As a bonus side-effect this also ensures the correctness of the progress reporting per orchestrated build.
2024-03-11just analyse: Use explicit target cache instancePaul Cristian Sarbu
The serve endpoint always has to access the correctly sharded target cache, including during analysis. For this purpose, the target cache instance interrogated during analysis has to be explicitly provided.
2024-03-11target cache: Add type aware of bootstrappingPaul Cristian Sarbu
This is needed in order to pass the correctly instantiated TargetCache to AnalyseTarget even when bootstrapping 'just'.
2024-03-11Extend gc end-to-end test to cover --no-rotateKlaus Aehlig
... verifying that - emphemeral directories are removed, - left-over remove-me directories are removed, and - the available content of CAS and cache are not affected.
2024-03-11gc: add option --no-rotationKlaus Aehlig
... to clean up what can be done without losing cache information.
2024-03-11garbage collection: remove ephemeral dataKlaus Aehlig
... and make rotation of generations optional, as with the removal of ephemeral data there is now a useful collection even without rotating generations.
2024-03-11local build root layout: collect all ephemeral directoriesKlaus Aehlig
... under a common root in the youngest generation. This will allow a simple way of cleaning them up during garbage collection.
2024-03-08Remove dead code for host replacementOliver Reiche
2024-03-08doc: Improve description of "preferred hostnames"Oliver Reiche
2024-03-08just-mr: Add test for "preferred hostnames"Oliver Reiche
2024-03-08just-mr: Fix "preferred hostnames"Oliver Reiche
... instead of replacing the host name of each mirror, it should only reorder the list of given mirrors.
2024-03-08mirrors: Add test for SortByHostnameOliver Reiche
2024-03-08mirrors: Implement SortByHostname routineOliver Reiche
2024-03-08CurlURLHandle: Add GetHostname routineOliver Reiche
2024-03-08doc: Update man page on Git file URLsOliver Reiche
2024-03-08just-mr: Normalize paths of Git file URLsOliver Reiche
... and ensure that paths starting with .// remain relative.
2024-03-08just-mr: Report fetch URLs on debug levelOliver Reiche