| Age | Commit message (Collapse) | Author |
|---|
|
This change is introduced to be prepared for future changes such as garbage
collection. It is an incompatible change compared to earlier just versions
since it modifies the local path to the just cache directory, where among
others the CASes, action cache, target-level cache are located.
|
|
Before this change, TargetResults that appear inside of
other TargetResults (typically via value nodes) were
serialised via the top-level serialisation function for
TargetResults. While technically correct, it is rather
inefficient as identical expressions from outer and inner
TargetResults are not properly deduplicated and a deeply
nested data structure is maintained.
With this change, expressions of inner TargetResults are
serialised in the context of outer TargetResults, resulting
in a flat list of all transitively contained expressions
with proper deduplication applied.
As this serialisation of TargetResult is used in
target-level cache entries, the new format is a breaking
change to existing entries. Therefore, after switching to
the new serialisation format introduced by this commit,
users are required to clean their target-level cache.
This also reverts commit d22adef666d704680ee74b35a46d530f6b6d5f15,
"Recursively scan provided results for known artifacts".
|
|
This recursive scan of the provided results map of a target-cache entry for
known artifacts is required since the serialization format of target results
currently does not produce a single flat artifact list, but keeps the result
entry structure with a nested list of artifacts. Once the serialization format
of target results is changed in a way that a flat list of artifacts is
produced, this recursive scan is not required anymore and can be reverted.
|
|
This enforces the explicit specification, which object type, either file or
tree, should be used to create an artifact digest. This also avoids subtile
errors at locations as in the previous commit, where files as well as trees are
supposed to be handled, but digest creation mistakenly defaults to file object
type.
|
|
The initial assumption before introducing the native remote execution protocol
was that only blobs were transmitted over the wire and put in a blob store.
With the native remote execution protocol, also trees can be transmitted over
the wire and they are put in a different store than blobs. Different hashing
functions are used for blobs and trees and need to be calculated accordingly
for data to be put in the correct store. This distinction of blobs and trees is
now implemented. Additionally, a warning is added that checks equality
between the requested digest and the digest computed by the received
content. This check makes sense to detect, e.g., corrupted messages,
unexpected transmission termination, or remote server failure.
|
|
Only if the environment variable DEBUG is set, fall back to sequentially
executing the graph generated by the bootstrap version of just.
|
|
|
|
Signed-off-by: Goetz Brasche <goetz.brasche@huawei.com>
|
|
|
|
|
|
|
|
... if the object does not even exist. In that case, a trace
message is produced and the caller is responsible for
reporting errors.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
When a tree is taken from a git root, it is not necessarily known on
the remote site. So, as any missing artifact it has to be uploaded,
recursively uploading the parts to keep the tree invariant. The
function RetrieveToCas was doing the correct recursiv pattern, however
inspecting trees incorrectly using the function ReadTreeInfos; the
latter function, however, was obtaining all the leafs of the tree
as is needed for a compatible action-input description. Add and
use a function that reads the direct contents of a tree.
|
|
... as for remote execution, the map entries are only used
for the `install` subcommand. For local execution, much less
tree objects are read from CAS when using this map. However,
the performance benefit is barely measurable and therefore
we rather remove this map entirely to reduce complexity.
|
|
... to align with the original idea of caching a flat list
of blob objects, without the need to recursively traverse
any trees. Consequently, we cannot create any map entry in
places where we do not have all sub-tree entries at hand
(e.g., LocalAPI, BazelAPI, BazelResponse).
|
|
... which is not using `Directory` messages nor does it
support the `GetTree()` rpc.
|
|
|
|
... so that any infra structure problem on the remote execution
is reported by default.
|
|
... to emphasize that this is the extensional action identifier,
not the intensional one.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
... while technically not required, it makes it harder to
run into nasty errors.
|
|
A checkout does not necessarily fetch an archive: if we already
have the git tree for that archive, this is enough to create a
build root. For the fetch command, however, we need to have the
actual archive as we have to copy it to the distdir. Of course, we
only create the git tree once we got hold of the archive. However,
with the introduction of CAS purging, the invariant that we have
the archive whenever we can create a build root will no longer be
true. This is acutally a feature as the git trees can be stored
more compactly if we accumulate different release archives of the
same upstream project. However, it also means that we explicitly
have to fetch the archive in the fetch subcommand. Do this.
|
|
The version subcommand now has no expected command line options, as expected.
Also the version subcommand was missing from the man page synopsis.
|
|
|
|
... at INFO level, in the same way as all other dumping of analysis
results happen.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
