| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
|
|
... if none is specified on command line or in the config.
|
|
|
|
This commit also introduces an incompatible change, since it modifies the way
how files are stored in the just cache directory. This modification reduces the
number of files per directory and only introduces a maximum number new
directories to avoid possible performance bottlenecks.
|
|
This change is introduced to be prepared for future changes such as garbage
collection. It is an incompatible change compared to earlier just versions
since it modifies the local path to the just cache directory, where among
others the CASes, action cache, target-level cache are located.
|
|
Before this change, TargetResults that appear inside of
other TargetResults (typically via value nodes) were
serialised via the top-level serialisation function for
TargetResults. While technically correct, it is rather
inefficient as identical expressions from outer and inner
TargetResults are not properly deduplicated and a deeply
nested data structure is maintained.
With this change, expressions of inner TargetResults are
serialised in the context of outer TargetResults, resulting
in a flat list of all transitively contained expressions
with proper deduplication applied.
As this serialisation of TargetResult is used in
target-level cache entries, the new format is a breaking
change to existing entries. Therefore, after switching to
the new serialisation format introduced by this commit,
users are required to clean their target-level cache.
This also reverts commit d22adef666d704680ee74b35a46d530f6b6d5f15,
"Recursively scan provided results for known artifacts".
|
|
This recursive scan of the provided results map of a target-cache entry for
known artifacts is required since the serialization format of target results
currently does not produce a single flat artifact list, but keeps the result
entry structure with a nested list of artifacts. Once the serialization format
of target results is changed in a way that a flat list of artifacts is
produced, this recursive scan is not required anymore and can be reverted.
|
|
This enforces the explicit specification, which object type, either file or
tree, should be used to create an artifact digest. This also avoids subtile
errors at locations as in the previous commit, where files as well as trees are
supposed to be handled, but digest creation mistakenly defaults to file object
type.
|
|
The initial assumption before introducing the native remote execution protocol
was that only blobs were transmitted over the wire and put in a blob store.
With the native remote execution protocol, also trees can be transmitted over
the wire and they are put in a different store than blobs. Different hashing
functions are used for blobs and trees and need to be calculated accordingly
for data to be put in the correct store. This distinction of blobs and trees is
now implemented. Additionally, a warning is added that checks equality
between the requested digest and the digest computed by the received
content. This check makes sense to detect, e.g., corrupted messages,
unexpected transmission termination, or remote server failure.
|
|
Signed-off-by: Goetz Brasche <goetz.brasche@huawei.com>
|
|
|
|
|
|
... if the object does not even exist. In that case, a trace
message is produced and the caller is responsible for
reporting errors.
|
|
|
|
|
|
When a tree is taken from a git root, it is not necessarily known on
the remote site. So, as any missing artifact it has to be uploaded,
recursively uploading the parts to keep the tree invariant. The
function RetrieveToCas was doing the correct recursiv pattern, however
inspecting trees incorrectly using the function ReadTreeInfos; the
latter function, however, was obtaining all the leafs of the tree
as is needed for a compatible action-input description. Add and
use a function that reads the direct contents of a tree.
|
|
... as for remote execution, the map entries are only used
for the `install` subcommand. For local execution, much less
tree objects are read from CAS when using this map. However,
the performance benefit is barely measurable and therefore
we rather remove this map entirely to reduce complexity.
|
|
... to align with the original idea of caching a flat list
of blob objects, without the need to recursively traverse
any trees. Consequently, we cannot create any map entry in
places where we do not have all sub-tree entries at hand
(e.g., LocalAPI, BazelAPI, BazelResponse).
|
|
... which is not using `Directory` messages nor does it
support the `GetTree()` rpc.
|
|
... so that any infra structure problem on the remote execution
is reported by default.
|
|
... to emphasize that this is the extensional action identifier,
not the intensional one.
|
|
|
|
|
|
|
|
... while technically not required, it makes it harder to
run into nasty errors.
|
|
The version subcommand now has no expected command line options, as expected.
Also the version subcommand was missing from the man page synopsis.
|
|
... at INFO level, in the same way as all other dumping of analysis
results happen.
|
|
|
|
|
|
... by dropping curl-brace-initializers for nlohmann::json,
which calls the intializer-list constructor converting any
JSON type to array.
|
|
|
|
|
|
case they got pruned
|
|
|
|
between different CASes
|
|
|
|
|
|
in daca274041e31636f08927b0cebc56fd33c7bbdf it has been added a local
tree-cas. This cas is populated only when just runs in native
mode. However, even in compatible mode, a tree must be looked for in
the tree-cas. This patch set the path of the tree-cas to be the same
of file-cas while running in compatible mode.
|
|
|
|
... to be able to report the respective graph for later analysis
by other tools.
|
|
... as this is relevant for performance of analysis. We log the total
numer of trees at performance level and the individual directories
at debug level, if requested.
|
|
When reporting the percentage already completed, only report actions
that were actually run. Cache hits are considered by reducing the
overall work to be considered, i.e., by making the completed actions
count more. In practice, however, most cache hits are discovered
ayway till the first progress status is reported.
|
|
|
|
|
|
|
|
Like file or tree references, globs are restricted to the current
module; in fact, by the way we evaluate them, even to the top-level
directory of that module: a glob is a target having as artifacts and
runfiles those entries of the top-level directory of the specified
module that match the given pattern.
|
|
When constructing module names, we normalize the directory path
implicit in it. However, we take the empty string as valid canonical
name for the top-level directory (as it is the empty relative path).
When accessing git roots, however, only the file-system canonical
path "." is special cased as self reference. Hence explicitly
convert the empty string.
|
