summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2023-02-10just execute: bugfix: honor do_not_cache booleanAlberto Sartori
2023-02-02execution-service: add new subcommand executeAlberto Sartori
This subcommand starts a single node remote execution service honoring the just native remote protocol. If the flag --compatible is provided, the execution service will honor the original remote build execution protocol. New command line args supported by this subcommand: -p,--port INT: Execution service will listen to this port. If unset, the service will listen to the first available one. --info-file TEXT: Write the used port, interface, and pid to this file in JSON format. If the file exists, it will be overwritten. -i,--interface TEXT: Interface to use. If unset, the loopback device is used. --pid-file TEXT Write pid to this file in plain txt. If the file exists, it will be overwritten. --tls-server-cert TEXT: Path to the TLS server certificate. --tls-server-key TEXT: Path to the TLS server key. Co-authored by: Klaus Aehlig <klaus.aehlig@huawei.com>
2023-02-02LocalStorage: add StoreTree(std::filesystem::path const &)Alberto Sartori
2023-02-02bytestream_client: use common header with settings shared by both server and ↵Alberto Sartori
client
2023-02-02remote/config.hpp: make ParsePort re-usable for execution-serviceAlberto Sartori
2023-02-02bytestream_client: report failures on LogLevel::ErrorAlberto Sartori
2023-02-02file_locking: fix comment describing the name of the lock fileKlaus Aehlig
As opposed the earlier ideas, the implemented behaviour is that the file name of the lock is given explicitly. Fix comments accordingly.
2023-02-01Update fmtlib to version 9.1.0Klaus Aehlig
Co-authored-by: Michael Thies <mail@mhthies.de>
2023-01-31Improve error message if git tree walk failedOliver Reiche
2023-01-31Fix segfault if reading git tree failedOliver Reiche
2023-01-26Fix boostrapping on less optimizing compilersKlaus T. Aehlig
While compilers are allowed to drop unused functions in anonymous name spaces, and in this way also the open linker symbols referenced there, they are not obliged to do so. Not optimizing away such unused functions when compiled with -DBOOTSTRAP_BUILD_TOOL causes the linking fail in the initial phase of the boostrap process where libgit2 is not yet available (nor really needed). Therefore, ensure that those dead functions are absent in the initial bootstrap phase using appropriate preprocessor directives. Signed-off-by: Klaus T. Aehlig <aehlig@linta.de>
2023-01-24BazelMsgFactory: Improve error reporting for creating treesOliver Reiche
2023-01-24FileSystemManager: Do not follow symlinksOliver Reiche
... and ensure that cascades of checks are performed with only a single filesystem stat per method.
2023-01-24FileSystemManager: Pass ReadDirectory error to callerOliver Reiche
2023-01-24Just-MR: Remove obsolete GET_BRANCH_REFNAME critical Git operationPaul Cristian Sarbu
As now all remote Git operations in GitRepo require at most just the branch name, there is no more need to inquire the repository about branch refspecs.
2023-01-24GitRepo: Add SSL certificate verification callbacks for remote commit updatePaul Cristian Sarbu
The libgit2 library does not satisfy the http.sslVerify gitconfig field or the GIT_SSL_NO_VERIFY environment variable, so we have to perform these checks ourselves and supply the correct return value from the certificate_check git_fetch_options callback. The callbacks used for fixing the remote fetch SSL certificate verification are reused here.
2023-01-24GitRepo: Add SSL certificate verification callbacks for remote fetchPaul Cristian Sarbu
The libgit2 library does not satisfy the http.sslVerify gitconfig field or the GIT_SSL_NO_VERIFY environment variable, so we have to perform these checks ourselves and supply the correct return value from the certificate_check git_fetch_options callback.
2023-01-24GitRepo: Remove refspec argument in retrieving commit from remote...Paul Cristian Sarbu
...and use instead the branch name. A valid direct refspec (as those retrieved by a remote_ls call) will always end in the branch name, so checking the last path component ('/'-delimited substring) of a retrieved refspec is enough.
2023-01-24GitRepo: Add libgit2 proxy options for remote connection for commit updatePaul Cristian Sarbu
2023-01-24GitRepo: Set libgit2 option for auto-detection of proxy settings in fetching ↵Paul Cristian Sarbu
from remote
2023-01-24GitRepo: Change FetchFromRemote to fetch based on branch namePaul Cristian Sarbu
This also removes the need to call the GET_BRANCH_REFNAME critical operation.
2023-01-24commit_git_map: Clean up passed arguments in EnsureCommitPaul Cristian Sarbu
2023-01-20Add gc subcommandSascha Roloff
2023-01-20Add local garbage collectionSascha Roloff
2023-01-20Move execution-backend-id calculation from target-level cache to local configSascha Roloff
This code movement is required to break a cyclic dependency coming with the introduction of the garbage collector. target_cache depends on garbage_collector and garbage_collector would depend on target_cache to determine the target-level-cache directory. After moving this calculation to a more general location, the cycle is broken.
2023-01-20Introduce multiple cache generationsSascha Roloff
2023-01-20Configured target uses entity_name_data.hpp instead of entity_name.hppSascha Roloff
This usage of a different header file is required to break a cyclic dependency coming with the introduction of the garbage collector. entity_name implicitly depends on local_cas, which will get a new dependency to garbage_collector. garbage_collector depends on target_cache_entry, which transitively depends on entity_name. entity_name_data instead does not depend on local_cas and breaks the cycle.
2023-01-20Cleanup local CAS dependency for repository configSascha Roloff
2023-01-20Add check for same CAS in CAS-to-CAS artifact retrievalSascha Roloff
2023-01-20Move file_storage.hpp to file_system subdirectorySascha Roloff
2023-01-20Refactor target cache key and entry to separate files and targetsSascha Roloff
2023-01-20Utils: Add file locking utility classPaul Cristian Sarbu
2023-01-20generic built-in rule: fix typo in error messageKlaus Aehlig
2023-01-19Minor fixes for compiling with clang-14Oliver Reiche
2023-01-18authentication: add TLS and mTLS supportAlberto Sartori
2023-01-16just-mr fetch: fix overwriting distfilesKlaus Aehlig
Both, the python script as well as the compiled variant of just-mr fetch try to overwrite already existing distfiles in the distdir. While it is unclear if overwriting existing distfiles or not copying to distdir if a file with that name already exists is the better behaviour, we at least should not error out, as the compiled just-mr currently does as it sets permission to 444 and hence cannot overwrite later; fix this by addwing owner write permissions before overwriting.
2023-01-16just-mr fetch: fix internal archive descriptionKlaus Aehlig
The absent value for an optional string is std::nullopt, not the empty string.
2023-01-16just-mr fetch: drop redundant directory creationKlaus Aehlig
just-mr fetch is supposed to fetch to the first (in search order) distdir that is an existing directory (and error out if no such directory exists). So, once a directory is found, there is no need to create it---we already verified it is an existing directory.
2023-01-16just-mr: fix quotingKlaus Aehlig
... in error message reporting the lack of present distdirs.
2023-01-16CurlEasyHandle: Fix empty fetches due to unfollowed URLsPaul Cristian Sarbu
For libcurl we need to set the CURL_FOLLOWLOCATION flag (disbaled by default) to enable 3xx redirects. Libcurl has sane defaults for related settings in order to handle redirects when enabled, though for fetches there should be limited risks, as content (and SHA hashes, if provided) is checked to ensure the intended archives are fetched.
2023-01-16fpath_git_map: Fix wrong log messagePaul Cristian Sarbu
2023-01-12source map: quote file names in error messageKlaus Aehlig
... to ensure unique readability even with file names containing spaces or other special symbols; those kind of fancy file names typically occur when a string intended as named target is implicitly taken as a file (e.g., due to a spelling error). While there, also include directory and repository in the error message.
2023-01-10repos_to_setup_map: Remove unnecessary logger capture in lambda callsPaul Cristian Sarbu
2023-01-10Just-MR: Fix distdir computation in internal just-mrPaul Cristian Sarbu
This provides the fix already performed for the just-mr script in the internal just-mr as well. In short, this fix makes sure that the computation of the content key for distdir repositories is independent of the presence of the respective archives in CAS.
2022-12-23Just-MR: Fix wrong commit_git_map keyPaul Cristian Sarbu
The subdir is required alongside the commit hash for a valid map key when importing a Git repository into our Git cache.
2022-12-23Just-MR: Remove wrong pass-by-reference when wrapping loggersPaul Cristian Sarbu
Passing the logger by reference would require the caller to be kept alive. Also, being a shared_ptr, the logger can be passed by value at almost no cost.
2022-12-23Just-MR: Fix bloated reachable reposPaul Cristian Sarbu
Use intermediary unordered sets to store the repos to_setup and to_include to avoid unnecessary duplicates. It is purely a performance benefit.
2022-12-23Just-MR: Fix mrrc config handlingPaul Cristian Sarbu
The parser was exiting early when the user gave no rc file, before having the chance to look for the config file in the default locations.
2022-12-23just-mr: fix exec log messageKlaus Aehlig
... to show the argument vector properly serialized as json. In this way, the vector can be read unambigiously even if arguments contain white space.
2022-12-22Add option --dump-export-targetsKlaus Aehlig
... dumping only the configured targets that are export targets. Those targets are of particular interest as those are the targets, as they are the high-level caching points and also often serve as interface between projects.