| Age | Commit message (Collapse) | Author |
|---|
|
|
|
Similarly to tarballs, an autodetection option for zip-like
archives is added to enable a unified handling of both traditional
zip and 7zip formats. Thus, for "zip" archives just-mr uses now
this autodetection option.
|
|
Also updates the archive usage tests accordingly.
|
|
These are provided by the recently added liblzma external dependency.
|
|
We shouldn't exclude the possibility of receiving uncompressed
tarballs as repositories in just-mr. Therefore, we introduce an
explicit type that performs the autodetection (default behaviour
in just-mr). This is done to also be more in line with our
implementation which allows the granular handling of various
archive types (currently used only for testing purposes).
|
|
|
|
While a foldl is enough to implement a reverse functionality,
adding it as a built in allows doing so in linear time.
|
|
... to obtain from a list of strings a map with those entries
as keys and true as value. In this way, repeated membership tests
in lists can be implemented more efficiently.
|
|
|
|
|
|
... in order to make that functionality reusable
|
|
The rule generates a non-upwards symbolic link with given target
path.
|
|
|
|
If install-cas finds a file in the target location, unlink it before
writing (in the same way as install does already). In this way, we
avoid changing other file locations in the presence of hard links.
|
|
Upwards symlinks should still be collected from actions, even if
only the non-upwards symlinks are supported artifact types. The
client side is thus the one responsible with enforcing the
non-upwardness condition.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For 'file' and 'archive' checkouts we will handle more than just
the 'ignore' value.
|
|
Moves away from literal strings in the async maps.
|
|
...instead of a regular field.
Also removes the ignore-special logic from distdir repositories,
as those will never contain symlinks, so no entries would be
ignored.
|
|
|
|
|
|
... to allow selecting a sub object of a specified tree.
|
|
|
|
|
|
Before this patch, when creating an action directory, symlinks were
staged as regular files.
|
|
|
|
The bytestream server implementation (deployed by just execute) now
stores the temporary files under
$local_build_root/protocol-depenedent/generation-0
so that they can be garbage collected if "just exectue" is terminated
before they are cleaned up.
|
|
...that std::filesystem::* calls produce. This is because existence
and type checks use almost exclusively std::filesystem::status,
which follows symbolic links, when being called with path arguments.
Instead, one should instead use these methods with the value
returned by a call of std::filesystem::symlink_status.
This commit also streamlines the FileSystemManager tests, as well
as replace bare calls to std::filesystem with their FileSystemManager
counterparts (where suitable).
|
|
The introduction of non-upwards symlinks as first-class objects
should have updated the handling of known git tree artifacts
containing symlinks. In particular, one should consider trees in
their entirety when uploading (irrespective of the ignore_special
flag), and git trees should only be reported as known only if
the ignore_special flag is set to false.
|
|
|
|
The command 'git add .' does not include paths found in .gitignore
files in the directory tree where the command is issued. This is
not the desired behaviour, as we expect for a tree with a given
commit id to contain all of the entries, irrespective of their
meaning to Git.
This commit addresses the issue as described.
For the just-mr.py script we modified the staging command to
'git add -f .'.
For the compiled just-mr, simply adding the force flag to
'git_index_add_all' did not work as intended for files found in
ignored subdirectories. This is a known libgit2 issue which has
been fixed in v1.6.3. Until we can upgrade our libgit2 version,
a workaround was implemented: we recursively read the directory
entries ourselves and add each of them iteratively using
'git_index_add_bypath', making sure to ignore the root '.git'
subtree (which cannot be staged).
At the moment the handling of Git submodules remains an open issue,
as Git does not allow '.git' subtrees to be forcefully added to the
index, and thus such directory entries will currently not be
considered as part of a git tree. This however is consistent
behavior between Git and libgit2.
|
|
...allowing the skipping of certain subtrees if needed. This is
useful, e.g., in simulating what a 'git add' call would do,
which ignores all '.git' subdirectories.
Also adds a corresponding test for the new method.
|
|
In this way, the user can specify which execution endpoint
to use depending on the execution properties.
|
|
|
|
Extend the configuration data structure by a dispatch list of endpoints
to chose based on the first match of the execution properties.
|
|
... in a way consistent with what we're using at other places
already.
|
|
|
|
... and hence should be declared as such.
|
|
A recent change made it so that the artifact type was wrongly being
taken into account when deciding whether the CAS entry of the
artifact would be read or not. This meant that non-file artifacts
would be skipped from their content being read, resulting in
missing CAS artifacts being reported.
This fixes the issue by reverting the offending change.
|
|
|
|
... so that any left-over directories, in case of abnormal termination
are cleaned up eventually.
|
|
just-mr accesses the CAS at several occations, most importantly,
the generated multi-repository configuration is written there. In
particular, when used as a launcher, it needs to keep that file in
its location as the path is passed to the exec of just.
|
|
In preparation for the introduction of our blob splitting protocol as extension
to the remote execution api, we need to update the used remote execution api to
a more recent version than v2.0.0. Since no new tags are available right now,
we update to the preliminary protocol version v2.3 according to the following
discussion: https://github.com/bazelbuild/remote-apis/issues/253
|
|
|
|
|
