summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-01-24Just-MR: Remove obsolete GET_BRANCH_REFNAME critical Git operationPaul Cristian Sarbu
As now all remote Git operations in GitRepo require at most just the branch name, there is no more need to inquire the repository about branch refspecs.
2023-01-24GitRepo: Add SSL certificate verification callbacks for remote commit updatePaul Cristian Sarbu
The libgit2 library does not satisfy the http.sslVerify gitconfig field or the GIT_SSL_NO_VERIFY environment variable, so we have to perform these checks ourselves and supply the correct return value from the certificate_check git_fetch_options callback. The callbacks used for fixing the remote fetch SSL certificate verification are reused here.
2023-01-24GitRepo: Add SSL certificate verification callbacks for remote fetchPaul Cristian Sarbu
The libgit2 library does not satisfy the http.sslVerify gitconfig field or the GIT_SSL_NO_VERIFY environment variable, so we have to perform these checks ourselves and supply the correct return value from the certificate_check git_fetch_options callback.
2023-01-24GitRepo: Remove refspec argument in retrieving commit from remote...Paul Cristian Sarbu
...and use instead the branch name. A valid direct refspec (as those retrieved by a remote_ls call) will always end in the branch name, so checking the last path component ('/'-delimited substring) of a retrieved refspec is enough.
2023-01-24GitRepo: Add libgit2 proxy options for remote connection for commit updatePaul Cristian Sarbu
2023-01-24GitRepo: Set libgit2 option for auto-detection of proxy settings in fetching ↵Paul Cristian Sarbu
from remote
2023-01-24GitRepo: Change FetchFromRemote to fetch based on branch namePaul Cristian Sarbu
This also removes the need to call the GET_BRANCH_REFNAME critical operation.
2023-01-24libgit2: Enable https supportPaul Cristian Sarbu
2023-01-24commit_git_map: Clean up passed arguments in EnsureCommitPaul Cristian Sarbu
2023-01-24libgit2: Patch handling of per-remote no proxy settings in gitconfigPaul Cristian Sarbu
2023-01-24libgit2: Update to v1.2.0Paul Cristian Sarbu
2023-01-20["test/end-to-end/just-mr", "fetch"]: fix destination directoryKlaus Aehlig
By default, just-mr fetches to the first existing distribution directory; by default, the first distribution directory to consider is ~/.distfiles. If an explicit destination is desired, it should be specified with the -o option. Fix the invocation of just-mr in the end-to-end test to use the specified destination; this also makes local execution of te test hermetic (as was originally intended), even if the user has a directory ~/.distfiles.
2023-01-20Document the conepts behind garbage collectionKlaus Aehlig
2023-01-20Add basic end-to-end test for just gcKlaus Aehlig
Run build and garbage collection end to end and verify two basic properties. - A target used after the the last gc is still available after another gc, and so are all its parts, even if the output is a single tree. - A target not used between to gc is purged from disk and disk space is not used any more.
2023-01-20Add gc subcommandSascha Roloff
2023-01-20Add local garbage collectionSascha Roloff
2023-01-20Move execution-backend-id calculation from target-level cache to local configSascha Roloff
This code movement is required to break a cyclic dependency coming with the introduction of the garbage collector. target_cache depends on garbage_collector and garbage_collector would depend on target_cache to determine the target-level-cache directory. After moving this calculation to a more general location, the cycle is broken.
2023-01-20Introduce multiple cache generationsSascha Roloff
2023-01-20Configured target uses entity_name_data.hpp instead of entity_name.hppSascha Roloff
This usage of a different header file is required to break a cyclic dependency coming with the introduction of the garbage collector. entity_name implicitly depends on local_cas, which will get a new dependency to garbage_collector. garbage_collector depends on target_cache_entry, which transitively depends on entity_name. entity_name_data instead does not depend on local_cas and breaks the cycle.
2023-01-20Cleanup local CAS dependency for repository configSascha Roloff
2023-01-20Add check for same CAS in CAS-to-CAS artifact retrievalSascha Roloff
2023-01-20Move file_storage.hpp to file_system subdirectorySascha Roloff
2023-01-20Refactor target cache key and entry to separate files and targetsSascha Roloff
2023-01-20Test: Add tests for file locking utility classPaul Cristian Sarbu
2023-01-20Utils: Add file locking utility classPaul Cristian Sarbu
2023-01-20generic built-in rule: fix typo in error messageKlaus Aehlig
2023-01-20Update Changelog: authentication availableKlaus Aehlig
2023-01-20tests: pass tls certificates and keyAlberto Sartori
2023-01-19Minor fixes for compiling with clang-14Oliver Reiche
2023-01-18authentication: add TLS and mTLS supportAlberto Sartori
2023-01-16just-mr fetch: fix overwriting distfilesKlaus Aehlig
Both, the python script as well as the compiled variant of just-mr fetch try to overwrite already existing distfiles in the distdir. While it is unclear if overwriting existing distfiles or not copying to distdir if a file with that name already exists is the better behaviour, we at least should not error out, as the compiled just-mr currently does as it sets permission to 444 and hence cannot overwrite later; fix this by addwing owner write permissions before overwriting.
2023-01-16Add test to verify just-mr fetch functionalityKlaus Aehlig
2023-01-16just-mr fetch: fix internal archive descriptionKlaus Aehlig
The absent value for an optional string is std::nullopt, not the empty string.
2023-01-16just-mr fetch: drop redundant directory creationKlaus Aehlig
just-mr fetch is supposed to fetch to the first (in search order) distdir that is an existing directory (and error out if no such directory exists). So, once a directory is found, there is no need to create it---we already verified it is an existing directory.
2023-01-16just-mr: fix quotingKlaus Aehlig
... in error message reporting the lack of present distdirs.
2023-01-16test/end-to-end: stage git-import tests to their own directoryKlaus Aehlig
... to avoid potential staging conflicts. Having them staged to just-mr was a copy-and-paste error in the first place anyway.
2023-01-16CurlEasyHandle: Fix empty fetches due to unfollowed URLsPaul Cristian Sarbu
For libcurl we need to set the CURL_FOLLOWLOCATION flag (disbaled by default) to enable 3xx redirects. Libcurl has sane defaults for related settings in order to handle redirects when enabled, though for fetches there should be limited risks, as content (and SHA hashes, if provided) is checked to ensure the intended archives are fetched.
2023-01-16fpath_git_map: Fix wrong log messagePaul Cristian Sarbu
2023-01-12source map: quote file names in error messageKlaus Aehlig
... to ensure unique readability even with file names containing spaces or other special symbols; those kind of fancy file names typically occur when a string intended as named target is implicitly taken as a file (e.g., due to a spelling error). While there, also include directory and repository in the error message.
2023-01-12Installation: be explicit about building with older justKlaus Aehlig
... by specifying that we always ensure the latest stable release of just can build just at head. If the only version available is older than the last stable release, it might be necessary to go through the bootstrap path.
2023-01-10Add basic test for git-importKlaus Aehlig
2023-01-10just-import-git: accept config from stdinKlaus Aehlig
... to allow simple chaining of imports.
2023-01-10just-import-git: carry out the temporary clone in a shallow wayKlaus Aehlig
2023-01-10just-import-git: fix check for main nameKlaus Aehlig
The main repository to be imported should be treated special, whenever given. This has to be verified as the parameter not being None, rather than being true, as the empty string (a traditional name for the main repository) is also a false value in python.
2023-01-10Tests: Add TEST_BOOTSTRAP_JUST_MR config var to decide which just-mr is used ↵Paul Cristian Sarbu
in the end-to-end tests
2023-01-10Targets: Create top-level just-mr install targetPaul Cristian Sarbu
This target uses the just-mr binary and will replace the Python script in all non-bootstrap-related scenarios.
2023-01-10repos_to_setup_map: Remove unnecessary logger capture in lambda callsPaul Cristian Sarbu
2023-01-10Just-MR: Fix distdir computation in internal just-mrPaul Cristian Sarbu
This provides the fix already performed for the just-mr script in the internal just-mr as well. In short, this fix makes sure that the computation of the content key for distdir repositories is independent of the presence of the respective archives in CAS.
2022-12-23Just-MR: Fix wrong commit_git_map keyPaul Cristian Sarbu
The subdir is required alongside the commit hash for a valid map key when importing a Git repository into our Git cache.
2022-12-23Just-MR: Remove wrong pass-by-reference when wrapping loggersPaul Cristian Sarbu
Passing the logger by reference would require the caller to be kept alive. Also, being a shared_ptr, the logger can be passed by value at almost no cost.