Age | Commit message (Collapse) | Author |
|
... as, for absent repositories, we need to get the description from
the serve end point. As a consequence, also support -r and --compatible,
as the remote-execution endpoint needs to fit with the one for the
serve endpoint.
|
|
... as even this command now adds entries to CAS, e.g., the shard
for the target-level cache. Also, this command block gc by keeping
a lock (also in the local build root).
|
|
As the serve service always has an associated remote-execution
endpoint, when just-mr gets passed only a serve endpoint address it
will now default to implying a remote-execution endpoint exists too
at that address. Additionally, now the implementation of the
--backup-to-remote option more clearly shows that it can only be
done in native mode.
|
|
The requests to retrieve the tree of a commit, archive, or distdir
also set up those trees in a way that the serve endpoint can later
build against them, besides allowing just-mr to set up roots
locally. Therefore, if the witnessing entity (Git commit, content
blob, or distdir, respectively) is known to the serve endpoint,
then failing to set up the root tree there should result in a
failure also of the just-mr setup on the client side.
|
|
For archives and Git repositories we should ensure that not finding
the witnessing entity (archive content blob or Git commit,
respectively) results in a distinct status in the response to a
request that sets up roots on the serve endpoint. This will allow
just-mr to better handle its interaction with the serve endpoint.
|
|
While just-mr does not use remote-execution properties, it is
still useful to have those as a separate entry in the rc file. With
rc-file delegation, this gives committed rc files an easy way to
specify the image to be used without having to set all the remaining
arguments for the various just subcommands in "just args".
|
|
|
|
|
|
... to pull in rc files from different locations, given by
location objects.
|
|
|
|
... to allow, in a clean way, add computing the effective rc
by overlaying delegated rc files.
|
|
To take advantage of absent roots, we need to ensure that a given
serve endpoint can build against the tree of this generated root.
For a 'distdir' repository we can know the resulting tree
identifier directly without actually needing to fetch anything.
Therefore, we only set the root as absent if the serve endpoint
knows already this tree, if it can set it up itself, or if
we can provide this tree to the serve endpoint from one of our
CAS locations (local or remote), based on our tree invariant
guarantee. A network fetch of the archives never gets performed
for an absent root.
If a serve endpoint is not provided, an absent root can still be
generated, but only if no network fetches are required. In this
case a warning is emitted.
|
|
To take advantage of absent roots, we need to ensure that a given
serve endpoint can build against the tree of this generated root.
To this end, for an 'archive' repository we only set the root as
absent if the serve endpoint knows already the root, it can set it
up itself, or we can create the root locally without a network
fetch and then upload it to the serve endpoint via the remote CAS.
A network fetch never gets performed for an absent root.
If a serve endpoint is not provided, an absent root can still be
generated, but only if no network fetches are required. In this
case a warning is emitted.
|
|
...by passing around the AsyncMap key struct instead of individual
members. This will also make future code changes more easy to
implement and improve code readability.
|
|
To take advantage of absent roots, we need to ensure that a given
serve endpoint can build against the tree of this generated root.
To this end, for a 'git' repository we only set the root as absent
if the serve endpoint knows already the root, it can set it up
itself, or we can create the root locally without a network fetch
and then upload it to the serve endpoint via the remote CAS.
A network fetch never gets performed for an absent root.
If a serve endpoint is not provided, an absent root can still be
generated, but only if no network fetches are required. In this
case a warning is emitted.
|
|
To take advantage of absent roots, we need to ensure that a given
serve endpoint can build against the tree of this generated root.
To this end, for a 'git tree' repository we only set the root as
absent only if the given serve endpoint has this root, or the tree
is known locally and can be provided via the remote CAS. While
generating an absent root the fetch command will never be called.
Generating an absent root without being provided a serve endpoint
is still allowed, but results in a warning.
|
|
Marking a file-type repository as 'to_git' results in a Git-tree
type root, which are of course content fixed and can be (and
usually are) used by export targets. Therefore, it is beneficial
for a serve endpoint, if one is provided, to be aware of such a
root and be able to build against it if needed. If the root is
marked as absent, this condition becomes mandatory.
Generating an absent Git-tree root without being provided a serve
endpoint is still allowed, but results in a warning.
|
|
|
|
This is to uphold the coding style guide we employ.
|
|
tree
When the remote CAS provides the root tree, we perform an
import-to-git operation, therefore the correct witnessing
repository for the tree should always be the Git cache.
|
|
...for more easily readable and maintainable target descriptions.
|
|
|
|
... instead of some hard-coded strings, as that can be confusing
when the tool is packaged under a different name.
|
|
... which were only honored when doing fetch and setup.
|
|
Our fetch and launch tool is parametric in the tool to be launched.
Reflect this in the documentation and do not pretend it to be the
name "just" hard coded. While there, also fix the hard-coded name
"git" in the documentation of the default value.
|
|
If root is marked absent and we're not asked to fetch absent, we
can compute the resulting distdir root tree in-memory, as we have
all the information. Otherwise, i.e., if we actually need to have
the archives locally, we first check if the remote serve can
provide them in the remote CAS before continuing as before to fetch
the listed archives one at a time.
|
|
The change added to interrogate also the local CAS for the tree of
a 'git tree' repository failed to add support for backing up such
tree found into the remote CAS. This commit fixes the issue.
|
|
Now we look for the tree also in the local CAS, not just in the
local Git cache. If found, we import the tree from local CAS to the
Git cache and continue as usual.
|
|
Now we look for the content blob also in the local Git cache, not
just in local CAS. If found, we store the blob read from Git cache
into local CAS and continue as usual.
|
|
Now we look for the content blob also in the local Git cache, not
just in local CAS. If found, we store the blob read from Git cache
into local CAS and continue as usual.
|
|
This fixes the multiple instances of (mainly) missing or wrong
includes of standard, third-party, and own headers in the
other_tools source folder.
|
|
|
|
This bug was introduced with the alternative mirrors changes.
|
|
If the option --absent is given in just-mr (either directly or
implicitly via the corresponding entry in the rc file), it rewrites
the "pragma" entries in the internal representation of the mr
repository config. When doing so, however, we have to take into
consideration that a workspace root is not necessarily the definition
of a repository, but can, alternatively, also be a reference to
another root. This was not taken into account; fix this by restricting
the rewriting to only json objects.
|
|
|
|
When just-mr acts as a launcher, for most subcommand options the
"just args" entry in the rc files provides a convenient way to set
them. However, some options take a file as argument; for those it can
be desirable to set them without assuming a fixed file-system layout
and instead refer to logical roots, in particular the work space.
for the ones that refer to files, it is often desirable to have a potential
reference to the work space. Add this functionality.
|
|
|
|
This includes implementing the logic for the --backup-to-remote
and --fetch-absent options.
|
|
|
|
|
|
|
|
As preparatory step for extending the types of repositories that
are targeted by a just-mr fetch subcommand, names of certain
variables and of the currently only fetch operation async map were
changed to clarify that they only target archives.
|
|
|
|
As this map is supposed to work as an all-or-nothing guarantee of
content being in CAS, we can replace the bool return type with the
null type instead.
|
|
|
|
|
|
|
|
|
|
Before trying to fetch an archive content from other non-local
sources, check first the just serve endpoint.
|
|
...before going to the network, if 'just serve' endpoint given.
ContentCASMap was reworked to provide a cleaner logic for handling
pure fetches, which also reduced code duplication in ContentGitMap.
|