summaryrefslogtreecommitdiff
path: root/src/buildtool/file_system/git_repo.cpp
AgeCommit message (Collapse)Author
2024-04-10GitRepo: Add method to keep tree alive by taggingPaul Cristian Sarbu
Also adds an appropriate test for this method.
2024-04-10git_repo: Add blob writer methodPaul Cristian Sarbu
Also extends the tests accordingly.
2024-04-10git_repo: Improve error message for CreateTreePaul Cristian Sarbu
2024-04-08Use properly included standard library types by defaultPaul Cristian Sarbu
2024-03-26Add missing system includesPaul Cristian Sarbu
Main culprits: - std::size_t, std::nullptr_t, and NULL require <cstddef> - std::move and std::forward require <utility> - unordered maps and sets require respective includes - std::for_each and std::all_of require <algorithm>
2024-03-15Clean up more includes and targetsPaul Cristian Sarbu
Some of the more specific issues addressed: - missing log_level target/include - header-only libs wrongly marking deps as private - missing/misplaced gsl includes
2024-03-13Fix tmpdir locationKlaus Aehlig
Commit 2ebf355989eb92ac9967eceee0af14d39477afe0 moved the tmpdir creation for various tasks into the task itself. In doing so, TmpDir was called with a relative path; that was, however, is interpreted relative to the working directory, violating the property that our tool never write anything outside the local build root unless explicitly asked to do so (by specifying the output path in an install or install-cas invocation). Fix this, by calling the the tmp-dir function that is storage-layout aware.
2024-03-08just-mr: Fix shell out executionOliver Reiche
... by avoiding reusing temp dirs for execute. While we are at it, also refactor LocalFetchViaTmpRepo() to create its own empty temp dirs, that cannot be reused by the caller.
2024-01-31serve source tree: Increase server-side granularity in response statusesPaul Cristian Sarbu
For archives and Git repositories we should ensure that not finding the witnessing entity (archive content blob or Git commit, respectively) results in a distinct status in the response to a request that sets up roots on the serve endpoint. This will allow just-mr to better handle its interaction with the serve endpoint.
2024-01-08GitRepo: Add blob lookup methodPaul Cristian Sarbu
2023-11-02GitRepo: Add method for async fetch from local repositoryPaul Cristian Sarbu
This avoids using the more geenric GitRepoRemote method which has libcurl as a dependency, something that is not needed for this Git operation.
2023-11-02GitRepo: Add blob existence checkerPaul Cristian Sarbu
Also fixes a small typo in tree existence checker log messages.
2023-08-07GitRepo: Add method for reading object from tree by its pathPaul Cristian Sarbu
2023-07-11Git: Fix handling of symlinks in tree artifactsPaul Cristian Sarbu
The introduction of non-upwards symlinks as first-class objects should have updated the handling of known git tree artifacts containing symlinks. In particular, one should consider trees in their entirety when uploading (irrespective of the ignore_special flag), and git trees should only be reported as known only if the ignore_special flag is set to false.
2023-07-10just-mr: Fix handling of .gitignore files in git repositoriesPaul Cristian Sarbu
The command 'git add .' does not include paths found in .gitignore files in the directory tree where the command is issued. This is not the desired behaviour, as we expect for a tree with a given commit id to contain all of the entries, irrespective of their meaning to Git. This commit addresses the issue as described. For the just-mr.py script we modified the staging command to 'git add -f .'. For the compiled just-mr, simply adding the force flag to 'git_index_add_all' did not work as intended for files found in ignored subdirectories. This is a known libgit2 issue which has been fixed in v1.6.3. Until we can upgrade our libgit2 version, a workaround was implemented: we recursively read the directory entries ourselves and add each of them iteratively using 'git_index_add_bypath', making sure to ignore the root '.git' subtree (which cannot be staged). At the moment the handling of Git submodules remains an open issue, as Git does not allow '.git' subtrees to be forcefully added to the index, and thus such directory entries will currently not be considered as part of a git tree. This however is consistent behavior between Git and libgit2.
2023-06-26ReadTree: Add check for non-upwards symlinks...Paul Cristian Sarbu
...as early as possible. This ensures that callers always receive only the tree entries for the supported object types. For the symlinks non-upwardness check we pass a lambda capturing the real backend of the tree entries, such that the symlinks can be read. Updates git_tree tests accordingly.
2023-06-26ObjectType: Add non-upwards symlinks as a known object type...Paul Cristian Sarbu
...but make sure it is still considered a special type. The only non-special entry types remain file, executable, and tree.
2023-06-06style: Use designated initializersPaul Cristian Sarbu
This feature has been introduced with C++20.
2023-05-31FileRoot: Add ignore-special roots logicPaul Cristian Sarbu
2023-05-15memcheck: fix race in libgit2...Paul Cristian Sarbu
...caused by incorrectly setting and resetting the library internal state and the misuse of pthreads in libgit2. Normally, git_libgit2_init and git_libgit2_shutdown should span the life of a worker thread in order to be safely used. However, due to an incorrect implementation of libgit2's threadstate with pthreads, on unix systems there is a race condition. Until the use of pthread_key_t is corrected in libgit2, we need to apply a workaround by always ensuring that the main thread is the first thread reaching the GitContext constructor.
2023-04-26imports: Switch to Microsoft GSL implementationOliver Reiche
... with two minor code base changes compared to previous use of gsl-lite: - dag.hpp: ActionNode::Ptr and ArtifactNode::Ptr are not wrapped in gsl::not_null<> anymore, due to lack of support for wrapping std::unique_ptr<>. More specifically, the move constructor is missing, rendering it impossible to use std::vector<>::emplace_back(). - utils/cpp/gsl.hpp: New header file added to implement the macros ExpectsAudit() and EnsureAudit(), asserts running only in debug builds, which were available in gsl-lite but are missing in MS GSL.
2023-03-30GitRepo: Guard fake repository odb wrappingPaul Cristian Sarbu
In the current libgit2 implementation, a fake repository wrapped around an existing odb is being registered as owner the same way as a normal repository object. Therefore, one has to guard both the creation and destruction of the fake repository against all other git operations that might access the internal cache during this transfer of ownership.
2023-03-23GitRepo: Make tag creation operation more robustPaul Cristian Sarbu
Use a similar logic as for repository initialisation: first check if tag has not already been created in another process, and only then try creation; make more tries with more wait in between; only retry if failure was due to internal locking.
2023-03-23GitRepo: Add proper error message for keep tag operationPaul Cristian Sarbu
2023-03-23GitRepo: Make repository initialisation more robustPaul Cristian Sarbu
As the initialisation of Git repositories is something that only takes place once, we should check early and cheaply whether the repository is already there before trying to initialize it. If we do need to initilize a repo, we can afford more attempts and longer wait times between tries to initalize if the failure to initialize happens due to the internal Git locking mechanism.
2023-03-23GitRepo: Add proper error message for repository initialisationPaul Cristian Sarbu
2023-03-23GitRepo: Make repository path usage explicitPaul Cristian Sarbu
Opening a repo should not check parent directories, only try to open at given path.
2023-03-15add missing ldflags -pthread and use -pthread consistentlyAlberto Sartori
2023-03-14GitRepo: Fix memory leak in keep tag operationPaul Cristian Sarbu
2023-03-08GitRepo: Add method to check existence of a Git treePaul Cristian Sarbu
2023-03-07Git: Move 'fake' repository log messages to more appropriate reporting levelPaul Cristian Sarbu
2023-03-06GitRepo: Add missing retval check for git oid libgit2 callsPaul Cristian Sarbu
2023-03-03GitRepoRemote: Correctly honor SSL certification settings in fetch and ↵Paul Cristian Sarbu
commit update Uses the SSL certification utility method to correctly set the certification check options for the remote URL libgit2 calls. Due to the fact that remote operations are done via a temporary repository to allow concurrent work, the correct repository configuration needs to be interrogated. Thankfully, libgit2 provides a thread safe config snapshot object to be used in such scenarios. Also updates the existing GitRepoRemote tests accordingly.
2023-02-17structure cleanup: move remote operations of GitRepo to other_tools...Paul Cristian Sarbu
...in order to not include unwanted dependencies in just proper. The new class extends the GitRepo class used for just's Git tree operations and gets used in all of just-mr's async maps.
2023-01-31Improve error message if git tree walk failedOliver Reiche
2023-01-26Fix boostrapping on less optimizing compilersKlaus T. Aehlig
While compilers are allowed to drop unused functions in anonymous name spaces, and in this way also the open linker symbols referenced there, they are not obliged to do so. Not optimizing away such unused functions when compiled with -DBOOTSTRAP_BUILD_TOOL causes the linking fail in the initial phase of the boostrap process where libgit2 is not yet available (nor really needed). Therefore, ensure that those dead functions are absent in the initial bootstrap phase using appropriate preprocessor directives. Signed-off-by: Klaus T. Aehlig <aehlig@linta.de>
2023-01-24Just-MR: Remove obsolete GET_BRANCH_REFNAME critical Git operationPaul Cristian Sarbu
As now all remote Git operations in GitRepo require at most just the branch name, there is no more need to inquire the repository about branch refspecs.
2023-01-24GitRepo: Add SSL certificate verification callbacks for remote commit updatePaul Cristian Sarbu
The libgit2 library does not satisfy the http.sslVerify gitconfig field or the GIT_SSL_NO_VERIFY environment variable, so we have to perform these checks ourselves and supply the correct return value from the certificate_check git_fetch_options callback. The callbacks used for fixing the remote fetch SSL certificate verification are reused here.
2023-01-24GitRepo: Add SSL certificate verification callbacks for remote fetchPaul Cristian Sarbu
The libgit2 library does not satisfy the http.sslVerify gitconfig field or the GIT_SSL_NO_VERIFY environment variable, so we have to perform these checks ourselves and supply the correct return value from the certificate_check git_fetch_options callback.
2023-01-24GitRepo: Remove refspec argument in retrieving commit from remote...Paul Cristian Sarbu
...and use instead the branch name. A valid direct refspec (as those retrieved by a remote_ls call) will always end in the branch name, so checking the last path component ('/'-delimited substring) of a retrieved refspec is enough.
2023-01-24GitRepo: Add libgit2 proxy options for remote connection for commit updatePaul Cristian Sarbu
2023-01-24GitRepo: Set libgit2 option for auto-detection of proxy settings in fetching ↵Paul Cristian Sarbu
from remote
2023-01-24GitRepo: Change FetchFromRemote to fetch based on branch namePaul Cristian Sarbu
This also removes the need to call the GET_BRANCH_REFNAME critical operation.
2022-12-23Just-MR: Remove wrong pass-by-reference when wrapping loggersPaul Cristian Sarbu
Passing the logger by reference would require the caller to be kept alive. Also, being a shared_ptr, the logger can be passed by value at almost no cost.
2022-12-21Git: Wrap libgit2 raw pointersPaul Cristian Sarbu
2022-12-21Test: Add tests for critical git opsPaul Cristian Sarbu
2022-12-21Just-MR: Add non-critical git ops logic to git repo classPaul Cristian Sarbu
2022-12-21Just-MR: Add critical ops logic to git repo classPaul Cristian Sarbu
2022-12-21Git CAS: Move Git tree ops to fake repo wrapper classPaul Cristian Sarbu
2022-12-20Git CAS: Add fake repository wrapper for git odbPaul Cristian Sarbu