summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-02-09INSTALL.md: document the new NON_LOCAL_DEPS variableKlaus Aehlig
... in PACKAGE=YES builds.
2023-02-09Add test for mixed boot strappingKlaus Aehlig
2023-02-09bootstrap.py: Add env variable NON_LOCAL_DEPSMichael Thies
... to indicate, in case of a package build, which dependencies should not be taken from the local environment. As those need a different target root (and potentially other roots), we keep a copy of file roots modified when transitioning to local builds and rewrite the changed file roots in the NON_LOCAL_DEPS accordingly. Co-authored-by: Klaus Aehlig <aehlig@linta.de>
2023-02-08bash autocompletion: update completion for gc and executeAlberto Sartori
2023-02-02changelog: add just execute subcommandAlberto Sartori
2023-02-02man: add just execute subcommandAlberto Sartori
2023-02-02execution-service: add end-to-end testAlberto Sartori
Co-authored by: Klaus Aehlig <klaus.aehlig@huawei.com>
2023-02-02execution-service: add new subcommand executeAlberto Sartori
This subcommand starts a single node remote execution service honoring the just native remote protocol. If the flag --compatible is provided, the execution service will honor the original remote build execution protocol. New command line args supported by this subcommand: -p,--port INT: Execution service will listen to this port. If unset, the service will listen to the first available one. --info-file TEXT: Write the used port, interface, and pid to this file in JSON format. If the file exists, it will be overwritten. -i,--interface TEXT: Interface to use. If unset, the loopback device is used. --pid-file TEXT Write pid to this file in plain txt. If the file exists, it will be overwritten. --tls-server-cert TEXT: Path to the TLS server certificate. --tls-server-key TEXT: Path to the TLS server key. Co-authored by: Klaus Aehlig <klaus.aehlig@huawei.com>
2023-02-02LocalStorage: add StoreTree(std::filesystem::path const &)Alberto Sartori
2023-02-02bytestream_client: use common header with settings shared by both server and ↵Alberto Sartori
client
2023-02-02remote/config.hpp: make ParsePort re-usable for execution-serviceAlberto Sartori
2023-02-02bytestream_client: report failures on LogLevel::ErrorAlberto Sartori
2023-02-02file_locking: fix comment describing the name of the lock fileKlaus Aehlig
As opposed the earlier ideas, the implemented behaviour is that the file name of the lock is given explicitly. Fix comments accordingly.
2023-02-01Update fmtlib to version 9.1.0Klaus Aehlig
Co-authored-by: Michael Thies <mail@mhthies.de>
2023-01-31Improve error message if git tree walk failedOliver Reiche
2023-01-31Fix segfault if reading git tree failedOliver Reiche
2023-01-31Remove -Werror from externals' cflagsOliver Reiche
... and suppress new warnings thrown by clang-15. Co-authored-by: Michael Thies <mail@mhthies.de>
2023-01-30just: default to the unknown tool chainKlaus Aehlig
In order to be portable, we should not make any assumptions about the preferred toolchains of the users and instead default to the toolchain making the least assumptions about the compiler actually used.
2023-01-26test/bootstrap: fix typo in environemt variableKlaus Aehlig
... and also fix the test if we reached the fixed point; of course, that has to use the same configuration as we do not expect different tool chains to produce bit-by-bit identical output.
2023-01-26test/bootstrap: also test bootstrapping with the gnu compiler familyKlaus Aehlig
2023-01-26Fix boostrapping on less optimizing compilersKlaus T. Aehlig
While compilers are allowed to drop unused functions in anonymous name spaces, and in this way also the open linker symbols referenced there, they are not obliged to do so. Not optimizing away such unused functions when compiled with -DBOOTSTRAP_BUILD_TOOL causes the linking fail in the initial phase of the boostrap process where libgit2 is not yet available (nor really needed). Therefore, ensure that those dead functions are absent in the initial bootstrap phase using appropriate preprocessor directives. Signed-off-by: Klaus T. Aehlig <aehlig@linta.de>
2023-01-24BazelMsgFactory: Improve error reporting for creating treesOliver Reiche
2023-01-24FileSystemManager: Do not follow symlinksOliver Reiche
... and ensure that cascades of checks are performed with only a single filesystem stat per method.
2023-01-24FileSystemManager: Pass ReadDirectory error to callerOliver Reiche
2023-01-24Drop spourious bootstrap dependenciesKlaus Aehlig
A lot of the dependencies in the "just" repository are not for building just itself, but are used the the just-mr tool or for testing. As the bootstrap process, however, only bootstraps the just tool itself, drop those dependencies from the bootstrap process.
2023-01-24bin/boostrap.py: support dropping of repositoriesKlaus Aehlig
Not all repositories mentioned in the repository configuration are needed to boostrap just (e.g., because they are used for testing). Support dropping dependencies for bootstrapping by replacing them by a reference to an empty directory and in this way have less files that need to be fetched.
2023-01-24just-mr.py: Make disdir content reproducible and consistentPaul Cristian Sarbu
In Python the json of a dict does not guarantee an order, while in our internal just-mr implementation the json always has the keys sorted. Also, the JSON dump in Python does not by default use the most compact representation. This change fixes these issues and makes the two just-mr versions produce the same distdir content id.
2023-01-24Just-MR: Remove obsolete GET_BRANCH_REFNAME critical Git operationPaul Cristian Sarbu
As now all remote Git operations in GitRepo require at most just the branch name, there is no more need to inquire the repository about branch refspecs.
2023-01-24GitRepo: Add SSL certificate verification callbacks for remote commit updatePaul Cristian Sarbu
The libgit2 library does not satisfy the http.sslVerify gitconfig field or the GIT_SSL_NO_VERIFY environment variable, so we have to perform these checks ourselves and supply the correct return value from the certificate_check git_fetch_options callback. The callbacks used for fixing the remote fetch SSL certificate verification are reused here.
2023-01-24GitRepo: Add SSL certificate verification callbacks for remote fetchPaul Cristian Sarbu
The libgit2 library does not satisfy the http.sslVerify gitconfig field or the GIT_SSL_NO_VERIFY environment variable, so we have to perform these checks ourselves and supply the correct return value from the certificate_check git_fetch_options callback.
2023-01-24GitRepo: Remove refspec argument in retrieving commit from remote...Paul Cristian Sarbu
...and use instead the branch name. A valid direct refspec (as those retrieved by a remote_ls call) will always end in the branch name, so checking the last path component ('/'-delimited substring) of a retrieved refspec is enough.
2023-01-24GitRepo: Add libgit2 proxy options for remote connection for commit updatePaul Cristian Sarbu
2023-01-24GitRepo: Set libgit2 option for auto-detection of proxy settings in fetching ↵Paul Cristian Sarbu
from remote
2023-01-24GitRepo: Change FetchFromRemote to fetch based on branch namePaul Cristian Sarbu
This also removes the need to call the GET_BRANCH_REFNAME critical operation.
2023-01-24libgit2: Enable https supportPaul Cristian Sarbu
2023-01-24commit_git_map: Clean up passed arguments in EnsureCommitPaul Cristian Sarbu
2023-01-24libgit2: Patch handling of per-remote no proxy settings in gitconfigPaul Cristian Sarbu
2023-01-24libgit2: Update to v1.2.0Paul Cristian Sarbu
2023-01-20["test/end-to-end/just-mr", "fetch"]: fix destination directoryKlaus Aehlig
By default, just-mr fetches to the first existing distribution directory; by default, the first distribution directory to consider is ~/.distfiles. If an explicit destination is desired, it should be specified with the -o option. Fix the invocation of just-mr in the end-to-end test to use the specified destination; this also makes local execution of te test hermetic (as was originally intended), even if the user has a directory ~/.distfiles.
2023-01-20Document the conepts behind garbage collectionKlaus Aehlig
2023-01-20Add basic end-to-end test for just gcKlaus Aehlig
Run build and garbage collection end to end and verify two basic properties. - A target used after the the last gc is still available after another gc, and so are all its parts, even if the output is a single tree. - A target not used between to gc is purged from disk and disk space is not used any more.
2023-01-20Add gc subcommandSascha Roloff
2023-01-20Add local garbage collectionSascha Roloff
2023-01-20Move execution-backend-id calculation from target-level cache to local configSascha Roloff
This code movement is required to break a cyclic dependency coming with the introduction of the garbage collector. target_cache depends on garbage_collector and garbage_collector would depend on target_cache to determine the target-level-cache directory. After moving this calculation to a more general location, the cycle is broken.
2023-01-20Introduce multiple cache generationsSascha Roloff
2023-01-20Configured target uses entity_name_data.hpp instead of entity_name.hppSascha Roloff
This usage of a different header file is required to break a cyclic dependency coming with the introduction of the garbage collector. entity_name implicitly depends on local_cas, which will get a new dependency to garbage_collector. garbage_collector depends on target_cache_entry, which transitively depends on entity_name. entity_name_data instead does not depend on local_cas and breaks the cycle.
2023-01-20Cleanup local CAS dependency for repository configSascha Roloff
2023-01-20Add check for same CAS in CAS-to-CAS artifact retrievalSascha Roloff
2023-01-20Move file_storage.hpp to file_system subdirectorySascha Roloff
2023-01-20Refactor target cache key and entry to separate files and targetsSascha Roloff