| Age | Commit message (Collapse) | Author |
|---|
|
|
|
The implicit extensional projection of target-level caching and
garbage collection interact in a subtile way. Add a design document
describing how we keep the invariant required by target-level
caching in the presence of garbage collection. While, techincally,
this just describes how to fix a bug, a careful design is still
needed, as some fundamental changes are made.
|
|
Certain end-to-end tests require custom services. While normally
they come up in quickly (less than 2 seconds), on heavily overloaded
machines it might take longer. So increase the amount of time
these tests are willing to wait for the binary to start up to avoid
flakyness in our CI runs.
|
|
Our fetch and launch tool is parametric in the tool to be launched.
Reflect this in the documentation and do not pretend it to be the
name "just" hard coded. While there, also fix the hard-coded name
"git" in the documentation of the default value.
|
|
|
|
|
|
... as it is used by various libaries that are exported independently.
|
|
|
|
In order for the serve endpoint to correctly dispatch a build to
the correct remote-execution endpoint, the platform properties and
dispatch list for a build need to be passed explicitly to the
executor (via the graph traverser instance) instead of always being
taken from the RemoteExecutionConfig struct.
This commit implements these changes, including updating existing
tests accordingly.
|
|
|
|
|
|
|
|
If root is marked absent and we're not asked to fetch absent, we
can compute the resulting distdir root tree in-memory, as we have
all the information. Otherwise, i.e., if we actually need to have
the archives locally, we first check if the remote serve can
provide them in the remote CAS before continuing as before to fetch
the listed archives one at a time.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
When serving the tree of an archive, we should check also in the
local CAS for the content blob.
|
|
|
|
This was a source of occasional std::bad_variant_access exceptions.
|
|
The change added to interrogate also the local CAS for the tree of
a 'git tree' repository failed to add support for backing up such
tree found into the remote CAS. This commit fixes the issue.
|
|
The source archive of grp contains upwards symlinks (that do not
point outside the source directory). For a normal build this is
not an issue; we still can generate a git tree from this archive
and the parts we use of this archive never even touch a directory
containing an upwards symlink.
However, when just-mr desires to get this root from a just serve
instance, the tree would have to be synced to the remote-execution CAS
and hence the restriction applies that we only consider non-upwards
relative symlinks as first-class citizens.
Therefore, ignore all symlinks in this archive and this way, without
changing the actual build, obtain an easy-to-manage root.
|
|
While it is best practise to build outside the source tree, some
package formats require that a build be carried out inside the
source tree. As there are no principle obstacles, as long as a
non-existing directory is requested as build dir, support it by
ignoring the destination in the recursive copy.
|
|
... before trying to upload from local storage to the remote CAS.
Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
|
|
... to allow target-level caching.
|
|
The error log level should be reserved for events that inevitably
lead to a failed build. A failure to receive a target from the serve
endpoint, however, is not such a case; for performance reasons,
and also to have the same artifacts as everyone else in the case
of non-reproducible dependencies, just inquires the serve end point
for every export target whenever a serve end point is given. In
this case, the build just continues even if the serve end point
is, e.g., lacking a certain root.
|
|
The removed code has been superseeded by the
CreateBatchRequestsMaxSize member function.
|
|
...instead of the vector containing the digests of the uploaded
blobs. The returned vector was never inspected by the callers, except
for its size.
The tests have been accordingly amended.
|
|
|
|
...to honor the message limit imposed by GRPC.
|
|
...to honor the maxBatchTransferSize in grpc calls.
|
|
...such that each request does not exceeds the message limit imposed
by GRPC.
|
|
|
|
This function will ensure that each request does not exceeds the
maximum message size, currently set by kMaxBatchTransferSize in the
message_limits library.
|
|
|
|
|
|
Define the threshold for the grpc messages.
|
|
|
|
During the analysis of export targets, if there is a local target
cache miss, we should also interrogate the serve endpoint
(if given) for the target cache value before proceeding with the
analysis.
For this test we have extended the "with_serve" runner to expose
also the remote-execution build root to the test env.
|
|
|
|
While in our setting, a missing directory is generally OK, it is
not OK to ask for the content of an absent root. In particular, we
should not assume it to be empty, just because the root is absent.
|
|
|
|
When explaining CC tests, the option --request-action-input is used
to get hold of the actual test binary. This is the first time in
the tutorial where the action graph is larger than the part that
that is traversed during the build. Use this oportunity to explain
the difference between those concpets as, reportedly, some users
got confused about this.
|
|
|
|
Now we look for the tree also in the local CAS, not just in the
local Git cache. If found, we import the tree from local CAS to the
Git cache and continue as usual.
|
|
Now we look for the content blob also in the local Git cache, not
just in local CAS. If found, we store the blob read from Git cache
into local CAS and continue as usual.
|
|
Now we look for the content blob also in the local Git cache, not
just in local CAS. If found, we store the blob read from Git cache
into local CAS and continue as usual.
|
|
This fixes the multiple instances of (mainly) missing or wrong
includes of standard, third-party, and own headers in the
other_tools source folder.
|
