summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-12-05just main: Move useful results processing methods to own libraryPaul Cristian Sarbu
2023-12-05AnalyseTarget: Simplify method signaturePaul Cristian Sarbu
Similarly to the jobs common argument, we can simply only pass the needed single element from the analysis arguments struct. This will allow for a clearer understanding of what target analysis requires and reduce the overhead when reutilizing this method in just serve.
2023-12-05just main: Move root parser into static FileRoot method...Paul Cristian Sarbu
...to make it available also for setting up 'just serve' builds.
2023-12-05just main: Make capitalization in log messages consistentPaul Cristian Sarbu
2023-12-05executor: Move helper methods inside ExecutorImpl classPaul Cristian Sarbu
...with appropriate scope depending on where they are called from, to avoid unnecessary code duplication.
2023-12-04target.hpp: Fix double include statementPaul Cristian Sarbu
2023-12-04git_repo_remote: Fix wrong header include formatPaul Cristian Sarbu
2023-12-01just-mr archive: Report all tried remotes if fetch failsPaul Cristian Sarbu
2023-12-01just-mr git: Report all tried remotes if fetch failsPaul Cristian Sarbu
2023-12-01commit_git_map: Formatting fixPaul Cristian Sarbu
2023-11-30Resolve inconsistencies in third-party headers include formatPaul Cristian Sarbu
2023-11-30local API: Implement upload fileKlaus Aehlig
... overriding the default implementation. In this way, files can be added directly to the local CAS without having to completely reside in memory.
2023-11-30Splice blobs on diskKlaus Aehlig
When using blob splitting, we expect to create a potentially large file. Therefore reconstruct it from its parts on disk rather than keeping the whole file in memory.
2023-11-30remote API: support uploading files from the file systemKlaus Aehlig
Allow implementations to have a specialized way of uploading a file owned by the build process to the applicable cas, both as file, as well as as tree object (in native mode). If no implementation is provided, the default is to read the file into memory and use the Upload method.
2023-11-30Add files to CAS without fully keeping them in memoryKlaus Aehlig
2023-11-30hash_function: support hashing of filesKlaus Aehlig
Extend our internal hash-function abstraction to support hashing of files. In this way, we can provide a way to compute the hash of a potentially large file without having to fully read it to memory first, while still keeping the details of the used hash function abstracted away.
2023-11-30ObjectType: Add noexcept specifierPaul Cristian Sarbu
2023-11-30FileRoot: Fix content description for ignore-special rootsPaul Cristian Sarbu
Ignore-special git-tree-based roots are still content defined, so one should use the correct marker in the JSON description of the root that will be stored in the repository description of target cache keys. This commit fixes the issue and improves documentation.
2023-11-29Update install section about just-mrOliver Reiche
2023-11-29Do not drop curl for bootstrapOliver Reiche
... as it is part of just-mr, which is an essential part of the build system installation by now.
2023-11-29Add design document on tree-overlay actionsKlaus Aehlig
2023-11-27Refactoring RepositoryConfigPaul Cristian Sarbu
With the introduction of 'just serve', export targets can now be built also independently from one another based on their corresponding minimal repository configuration, as stored in the target cache key. In this context, this commit changes the RepositoryConfig usage from one global (static) instance to pointers passed as necessary throughout the code.
2023-11-24just: Fix wrong access to moved structPaul Cristian Sarbu
The traverser instantiation consumes the arguments.build struct, therefore one must not access that struct later in the code.
2023-11-23Mention blob splitting in changelogSascha Roloff
2023-11-23Move blob-splitting design document to implemented conceptsSascha Roloff
2023-11-22test: Check staging of empty treesOliver Reiche
2023-11-22LocalAction: Fix staging of empty treesOliver Reiche
2023-11-22Add test case for blob splittingSascha Roloff
2023-11-22Implement blob splitting protocol on just client sideSascha Roloff
2023-11-22Implement blob splitting protocol on just server sideSascha Roloff
2023-11-22Patch remote execution protocol with blob splitting APISascha Roloff
2023-11-22Update blob splitting design documentSascha Roloff
2023-11-22Test ["end-to-end/gc", "export"]: fix workspace layoutKlaus Aehlig
This end-to-end test, of course, depends on the tools, provided inside the test's working directory. The test, being end to end, constructs a workspace with multi-repository configuration with the main repo a file repsoitory at the root of the constructed work space; to be able to test exporting this repository has pragma "to_git". Move the constructed work space to sub directory to not include the tools to be tested in the work space that is gitified; one the one hand, the tested tools should not be part of the test workspace and on the other hand, this shaves off a large fraction of the test's run time.
2023-11-21test: Verify fetching from just serve of absent distdir archivesPaul Cristian Sarbu
2023-11-21content_cas_map: Interrogate just serve endpoint also if doing pure fetchPaul Cristian Sarbu
Before trying to fetch an archive content from other non-local sources, check first the just serve endpoint.
2023-11-21just serve: Implement client-side ServeContent and extend serve APIPaul Cristian Sarbu
2023-11-21just serve: Implement server-side ServeContent RPCPaul Cristian Sarbu
2023-11-21just_serve.proto: define ServeContent RPCPaul Cristian Sarbu
2023-11-21Extend --fetch-absent tests to verify regular fetching from just serveKlaus Aehlig
Whenever a just-serve endpoint is given, fetching archives or Git commits from there is preferrable to fetching them from other sources. Verify in the tests that this actually happens. Co-authored-by: Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com>
2023-11-21archive repository: Query 'just serve' endpoint also when not marked absentPaul Cristian Sarbu
...before going to the network, if 'just serve' endpoint given. ContentCASMap was reworked to provide a cleaner logic for handling pure fetches, which also reduced code duplication in ContentGitMap.
2023-11-21git repository: Query 'just serve' endpoint also when not marked absentPaul Cristian Sarbu
...before going to the network, if 'just serve' endpoint given.
2023-11-21SourceTree service: Bump up server-side log levelsPaul Cristian Sarbu
Log levels on the server-side should reflect the meaning of the status codes in the response messages. As such, anything that leads directly to an error-type status code has been bumped up to log level Error, the rest to log level Info.
2023-11-20rules: Fix flag inheritanceOliver Reiche
... which should accumulate values from all base targets unless the flag type (CFLAGS, CXXFLAGS, or LDFLAGS) is explicity overwritten by the inheritor.
2023-11-20Remove O_SYNC from low-level file-writing flags.Sascha Roloff
The man page for open(2) says the following to the O_SYNC flag: 'O_SYNC provides synchronized I/O file integrity completion, meaning write operations will flush data and all associated metadata to the underlying hardware.' This flag results in a high delay when files are stored in casx, e.g., several seconds for medium-sized files such as 23 MB. Since just does not care about persistency, this strong synchronization mechanism is not required and is deactivated.
2023-11-17built-in "install" rule: verify well-formedness of resulting stageKlaus Aehlig
The install target, like any other target, has to have artifacts and runfiles being proper stages, i.e., in such a way that the keys can be interpreted as names in the file system without causing conflicts. This property used to be unchecked, thus allowing users to define mal-formed targets that, when used as inputs to actions, would result in unspecified layout of the action directory. Fix this by adding an appropriate check enforcing well-formedness of the resulting stage.
2023-11-16test: Check that serve service works in compatible modePaul Cristian Sarbu
Added compatibility envariable to the with_serve rule, guarded tests using --fetch-absent such that they run only in native mode, and made all serve-service tests aware of the compatibility flag.
2023-11-16fetch-absent: Only allow option in native modePaul Cristian Sarbu
2023-11-16Add -R as short option for --remote-serve-addressKlaus Aehlig
The `just serve` command defaults to also provide the remote-execution endpoint if none is specified. The `just build` implicitly uses the serve endpoint as remote-execution endpoint if no other endpoint is specified. In this way, the serve endpoint has become the universal endpoint for simple set ups. Simplify that usage further by providing a short command-line option.
2023-11-16remote-execution-endpoint: fall back to remote-serve-endpointAlberto Sartori
If only the `--remote-serve-endpoint` option is specified on the command line, the `--remote-execution-endpoint` is also set to the given value. This makes the spawning and usage of just-execute consistent. When just-serve is started, if no remote execution endpoint is provided, the same process will also act as a just-execute instance. With the current patch, the client can thus only write, on command line, the remote serve address, avoiding the repetition of the same address for two different options.
2023-11-16end-to-end serve-target-cache-hit: test the failure if...Alberto Sartori
...the client and serve use two different remote execution endpoints fixup test