Age | Commit message (Collapse) | Author |
|
|
|
... to allow, in a clean way, add computing the effective rc
by overlaying delegated rc files.
|
|
To take advantage of absent roots, we need to ensure that a given
serve endpoint can build against the tree of this generated root.
For a 'distdir' repository we can know the resulting tree
identifier directly without actually needing to fetch anything.
Therefore, we only set the root as absent if the serve endpoint
knows already this tree, if it can set it up itself, or if
we can provide this tree to the serve endpoint from one of our
CAS locations (local or remote), based on our tree invariant
guarantee. A network fetch of the archives never gets performed
for an absent root.
If a serve endpoint is not provided, an absent root can still be
generated, but only if no network fetches are required. In this
case a warning is emitted.
|
|
To take advantage of absent roots, we need to ensure that a given
serve endpoint can build against the tree of this generated root.
To this end, for an 'archive' repository we only set the root as
absent if the serve endpoint knows already the root, it can set it
up itself, or we can create the root locally without a network
fetch and then upload it to the serve endpoint via the remote CAS.
A network fetch never gets performed for an absent root.
If a serve endpoint is not provided, an absent root can still be
generated, but only if no network fetches are required. In this
case a warning is emitted.
|
|
...by passing around the AsyncMap key struct instead of individual
members. This will also make future code changes more easy to
implement and improve code readability.
|
|
To take advantage of absent roots, we need to ensure that a given
serve endpoint can build against the tree of this generated root.
To this end, for a 'git' repository we only set the root as absent
if the serve endpoint knows already the root, it can set it up
itself, or we can create the root locally without a network fetch
and then upload it to the serve endpoint via the remote CAS.
A network fetch never gets performed for an absent root.
If a serve endpoint is not provided, an absent root can still be
generated, but only if no network fetches are required. In this
case a warning is emitted.
|
|
To take advantage of absent roots, we need to ensure that a given
serve endpoint can build against the tree of this generated root.
To this end, for a 'git tree' repository we only set the root as
absent only if the given serve endpoint has this root, or the tree
is known locally and can be provided via the remote CAS. While
generating an absent root the fetch command will never be called.
Generating an absent root without being provided a serve endpoint
is still allowed, but results in a warning.
|
|
Marking a file-type repository as 'to_git' results in a Git-tree
type root, which are of course content fixed and can be (and
usually are) used by export targets. Therefore, it is beneficial
for a serve endpoint, if one is provided, to be aware of such a
root and be able to build against it if needed. If the root is
marked as absent, this condition becomes mandatory.
Generating an absent Git-tree root without being provided a serve
endpoint is still allowed, but results in a warning.
|
|
|
|
pragma-related RPCs
|
|
|
|
|
|
The purpose of the requests for the tree of an archive, commit, or
distdir also includes making those trees available for future
builds on the serve endpoint, which currently means being in a
known Git repository.
This commit ensures the distdir tree reqeust also includes the
import of the resulting tree from CAS into the Git cache (if the
tree is not already in a Git repsoitory).
|
|
This is to uphold the coding style guide we employ.
|
|
tree
When the remote CAS provides the root tree, we perform an
import-to-git operation, therefore the correct witnessing
repository for the tree should always be the Git cache.
|
|
...for more easily readable and maintainable target descriptions.
|
|
|
|
... instead of some hard-coded strings, as that can be confusing
when the tool is packaged under a different name.
|
|
...in accordance to our coding style.
|
|
... which were only honored when doing fetch and setup.
|
|
|
|
|
|
|
|
... by uplinking them appropriately.
|
|
|
|
... that are eligible for caching. In this way, we can accurately keep
track of the dependencies between target-level cache entries. Note
that it is enough to track the export targets eligible for caching,
as no target depending on an ineligible export target can be eligible.
|
|
|
|
Our fetch and launch tool is parametric in the tool to be launched.
Reflect this in the documentation and do not pretend it to be the
name "just" hard coded. While there, also fix the hard-coded name
"git" in the documentation of the default value.
|
|
In order for the serve endpoint to correctly dispatch a build to
the correct remote-execution endpoint, the platform properties and
dispatch list for a build need to be passed explicitly to the
executor (via the graph traverser instance) instead of always being
taken from the RemoteExecutionConfig struct.
This commit implements these changes, including updating existing
tests accordingly.
|
|
|
|
If root is marked absent and we're not asked to fetch absent, we
can compute the resulting distdir root tree in-memory, as we have
all the information. Otherwise, i.e., if we actually need to have
the archives locally, we first check if the remote serve can
provide them in the remote CAS before continuing as before to fetch
the listed archives one at a time.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
When serving the tree of an archive, we should check also in the
local CAS for the content blob.
|
|
|
|
This was a source of occasional std::bad_variant_access exceptions.
|
|
The change added to interrogate also the local CAS for the tree of
a 'git tree' repository failed to add support for backing up such
tree found into the remote CAS. This commit fixes the issue.
|
|
... before trying to upload from local storage to the remote CAS.
Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
|
|
The error log level should be reserved for events that inevitably
lead to a failed build. A failure to receive a target from the serve
endpoint, however, is not such a case; for performance reasons,
and also to have the same artifacts as everyone else in the case
of non-reproducible dependencies, just inquires the serve end point
for every export target whenever a serve end point is given. In
this case, the build just continues even if the serve end point
is, e.g., lacking a certain root.
|
|
The removed code has been superseeded by the
CreateBatchRequestsMaxSize member function.
|
|
...instead of the vector containing the digests of the uploaded
blobs. The returned vector was never inspected by the callers, except
for its size.
The tests have been accordingly amended.
|
|
|
|
...to honor the message limit imposed by GRPC.
|
|
...to honor the maxBatchTransferSize in grpc calls.
|
|
...such that each request does not exceeds the message limit imposed
by GRPC.
|
|
|