| Age | Commit message (Collapse) | Author |
|---|
|
When backing up target-cache entries we use parallelism at two
dimensions, the independent cache entries and for each entry we
retrieve the artifacts in parallel. If for each dimension we use
the full amount of parallelism allowed, that gives a number of
threads up to the square of the amount of parallelism specified by
the user. Therefore, use in each dimension only the square root
of the allowed parallelism keeping the total parallelism (up to
rounding) within the specified range.
|
|
... which might be quite ahead of the end time of the invocation if
writing out of the action graph delays the end of the invocation.
|
|
...in all return paths, including in reporting caught exceptions.
In this way give the opportunity for any calling AsyncMap to
receive an expected fatal logger call on failure and thus be able
to shut down gracefully. This is in line with the AsyncMap design,
where the loggers are assumed to be safe to call by a consumer.
|
|
When using a serve end point, the analysis phase might take quite
long if serve has to actually build a delegated target or, at
least, has to synchronize artifacts with the remote end point.
Therefore, also record the time the build phase started (if building
is requested) as an additional time stamp in the profile.
|
|
|
|
This fixes a bug in which the setup root was falsely being changed
by unconditionally searching early for a default configuration
files, despite one being explicitly provided at the command line.
|
|
... so that at this level, the full activity of the serve service can
be monitored.
|
|
|
|
It is already supported to ask just-mr (via the rc file) to log for
each invocation the artifacts that were built. Add a similar option
for the artifacts that were to be built, i.e., for dumping the
intensional description of the output artifacts. That information
can be used, e.g., to compute the critical path.
|
|
|
|
|
|
... which requires all entries to be sorted in
lexicographical order.
|
|
... by ignoring it rather than trying to dereference a nullptr.
|
|
The directory name is designed to allow an easy rough sorting by
time. This can also be used to identify the latest build for a
specific user. However, users often run several builds in a single
minute; therefore, increase time-stamp precission to include
seconds as well. While still lexicographic and chronological order
will disagree, at least for a single user it will be correct most
of the times.
|
|
It allows to specify a list of environment variables, which are captured at
invocation time and stored as key-value pairs in the metadata file. This allows
to get some information about the invocation context such as username,
merge-request ID or source branch (on a CI runner), or others.
|
|
|
|
|
|
Include in the profile also the effective remote-execution endpoint,
properties, and dispatch list. Software projects are often tested
in a variety of environments or hardware configurations; as,
obviously, the performance might differ significantly (especially
depending on the used hardware) a proper analysis therefore requires
the possibility to distinguish the various backends. Adding the
effective configuration adds this posibility.
|
|
|
|
|
|
|
|
... by the requested subcommand. In particular, do not set it for
pure analyse requests.
|
|
For a generic rule, it is an error if map union of various
inputs (overlayed in correct order) does not form a proper stage. To
implement this check properly, we first have to construct the map of
all inputs and only then perform the staging check and not do the check
with only the runfiles, as 5e104a526cf76fe75312d2fd288a3c88f506fb0a
accidentally did. Fix this.
|
|
|
|
instead of ArtifactDigest
|
|
functions of the class
|
|
|
|
|
|
|
|
|
|
|
|
Allow to specify a custom string that is extended by the basename
of the logging directory, in case invocation logging is activated.
This can be used, e.g., to point to the user to service doing
something useful with the logged data (or simply presenting it in
a nicer form).
|
|
During execution, paths are relative to the working directory of the
action; however, in our representation, all paths are always relative
to the action root. Commit d65d711f844224dcf9215c52be8f69fd2885adfc
tried to change the reporing to our usual standard, however got
the direction wrong; fix this.
|
|
...when calling std::filesystem::weakly_canonical, since the latter converts the argument path to an absolute path internally.
|
|
|
|
|
|
|
|
In all presentations of actions to the user, we use output paths
relative to the root of the action directory. Therefore, we should
do the same in the profile. However, when noting the completion of
an action, we get paths as in the wire protocol, i.e., relative to
the working directory of the action. Therefore, rebase appropriately.
|
|
|
|
and while there, replace `auto` with explicit signatures.
|
|
|
|
|
|
We use a pointer to the actual Profile object to handle profiling,
if requested. For this to work, actual object needs to stay in
scope. However, we handle most of the operations, including parsing
of arguments, in a global try-catch block. In order to be able to
also correctly write a profile file outside this block, move the
scope of the Profile object to top-level in main.
While there, also improve the signature of the Profile class. That
class is only meaningful, if a profile should eventually be writting
to disk. So reflect this in the constructur. Also, once we know the
file name to write the profile to (if any), we have already parsed
the command line; so the making available of the command line to the
profile can be enforced by adding this to the constructor as well.
Co-authored-by: Denisov Maksim <denisov.maksim@huawei.com>
|
|
As parsing the the command-line is non-trivial, we include all the
relevant information about the command line in the profile. This
should also include the subcommand. For sake of completeness, we
also include the non-option arguments of the subcommand.
|
|
Extend the profile by including non-zero exit codes of individual
actions. When looking at an individual build invocation, the actions
with non-zero exit code are often the interesting ones, like root
cause of a build failure, or failing tests. Therefore, it is useful
information to include this information; by leaving out the exit
code if it is zero, we do not significantly increase the profile.
|
|
|
|
The BazelNetworkReader contains an optimization for reading directories
in case the remote execution (in compatible mode) supports the GetTree
request. This is, however not the case for many remote exeuciton
services, including our own single-node execution service. So the
code is basically untested and rarely used, if at all. Moreover,
justbuild is usually used in native mode and using compatibility
mode is expected to handle tree operations less efficient.
Therefore, remove this basically dead code and decrease complexity
this way.
|
|
|
|
... instead of at debug. We expect actions to be not in cache, so the fact
that we experience cache misses is not surprising. Given the information
available at this point, a useful logging indicating (in terms meaningful
to the user) is not possible. Therefore, keep the debug-level log clean.
|
|
At various places in the build tool, we try to read files from various
CASes and caches. The absence of a file there is normal; therefore,
reduce log level in order to not overload the debug-level log.
|
