| Age | Commit message (Collapse) | Author |
|---|
|
Now that we generate tree-overlay action identifiers that are disjoint
from identifiers of regular actions, having a joint origin map
does not cause any confusion. Concerning the numbering of subtasks,
we always see tree-overlay actions following the regular actions,
even if defined in a different order. In this way, the identifiers
for the regular actions don't change. Including tree-overlay
actions in the origin map also has the advantage that the origin
is properly reported in case of failure (e.g., non-disjointness of
the obtained trees).
|
|
|
|
In order to stay backwards compatible, the "tree_overlays" entry
in action-graph descriptions is optional.
|
|
... to avoid duplicate log entries.
|
|
If --dump-graph or --dump-plain-graph is given several times, the
action graph wil also be written several times. In this way, regular
use of those options will not be affected by adding them implicitly
through invocation-logging options in the rc file.
|
|
While reporting an origin target with action number within that
target describes an action uniquely in a way meaningful to the
user, it might not always be eay to unserstand which precise action
is currently running. For example, for a library with many source
files, we have a target generating a large number of actions and
the association of source file to action number requires detailled
knowledge of the build description. The name of the primary ouput
of that action, on the other hand immediately identifies the file
that is compiled. Therefore, report this as well.
|
|
... and use it when normalizing the origins of actions. For this
task, any well-defined linear order is sufficient. Using a native
comparision (rather than comparing the canonical serialisation)
significantly speeds up that normalisation process, as the assumption
that it would be rare that an action has more than one origin turned
out to be false. In fact, we have seeen cases where this sorting
used to take several seconds before this change, so that this change
reduced analysis time by more than a factor of 5.
|
|
Requires the use of a pragma to avoid wrong removal suggestion for
path_hash.hpp.
Co-authored-by: Maksim Denisov <denisov.maksim@huawei.com>
|
|
...since we use recursion for trees a lot, but skip this check manually.
|
|
|
|
|
|
! => not; && => and, || => or
|
|
|
|
Main culprits:
- std::size_t, std::nullptr_t, and NULL require <cstddef>
- std::move and std::forward require <utility>
- unordered maps and sets require respective includes
- std::for_each and std::all_of require <algorithm>
|
|
...by allowing a Logger instance to be provided.
|
|
...with regular instances that have controlled life-times.
This avoids race conditions in tracking and reporting the results
of analysis and build, as the serve endpoint can orchestrate
multiple builds at the same time asynchronously. As a bonus
side-effect this also ensures the correctness of the progress
reporting per orchestrated build.
|
|
This feature has been introduced with C++20.
|
|
... with two minor code base changes compared to previous
use of gsl-lite:
- dag.hpp: ActionNode::Ptr and ArtifactNode::Ptr are not
wrapped in gsl::not_null<> anymore, due to lack of support
for wrapping std::unique_ptr<>. More specifically, the
move constructor is missing, rendering it impossible to
use std::vector<>::emplace_back().
- utils/cpp/gsl.hpp: New header file added to implement the
macros ExpectsAudit() and EnsureAudit(), asserts running
only in debug builds, which were available in gsl-lite but
are missing in MS GSL.
|
|
The improved GC implementation uses refactored storage
classes instead of directly accessing "unknown" file paths.
The required storage class refactoring is quite substantial
and outlined in the following paragraphs.
The module `buildtool/file_system` was extended by:
- `ObjectCAS`: a plain CAS implementation for
reading/writing blobs and computing digests for a given
`ObjectType`. Depending on that type, files written to the
file system may have different properties (e.g., the x-bit
set) or the digest may be computed differently (e.g., tree
digests in non-compatible mode).
A new module `buildtool/storage` was introduced containing:
- `LocalCAS`: provides a common interface for the "logical
CAS", which internally combines three `ObjectCAS`s, one
for each `ObjectType` (file, executable, tree).
- `LocalAC`: implements the action cache, which needs the
`LocalCAS` for storing cache values.
- `TargetCache`: implements the high-level target cache,
which also needs the `LocalCAS` for storing cache values.
- `LocalStorage`: combines the storage classes `LocalCAS`,
`LocalAC`, and `TargetCache`. Those are initialized with
settings from `StorageConfig`, such as the build root base
path or number of generations for the garbage collector.
`LocalStorage` is templated with a Boolean parameter
`kDoGlobalUplink`, which indicates that, on every
read/write access, the garbage collector should be used
for uplinking across all generations (global).
- `GarbageCollector`: responsible for garbage collection and
the global uplinking across all generations. To do so, it
employs instances of `LocalStorage` with `kDoGlobalUplink`
set to false, in order to avoid endless recursion. The
actual (local) uplinking within two single generations is
performed by the corresponding storage class (e.g.,
`TargetCache` implements uplinking of target cache entries
between two target cache generations etc.). Thereby, the
actual knowledge how data should be uplinked is
implemented by the instance that is responsible for
creating the data in the first place.
|
|
|
|
Allow in the addition of a target to the result map to indicate
that it was an export target; in this way, the information is
available as a result of the analysis.
|
|
Signed-off-by: Goetz Brasche <goetz.brasche@huawei.com>
|
|
... at INFO level, in the same way as all other dumping of analysis
results happen.
|
|
|
|
... to be able to report the respective graph for later analysis
by other tools.
|
|
... as this is relevant for performance of analysis. We log the total
numer of trees at performance level and the individual directories
at debug level, if requested.
|
|
|
|
|
|
For a user, an important information is to know which actions
are currently running and, more importantly, the target that
caused them. To do so, we need a bit of infrastructure.
- We have to keep track of begin and end of running actions,
as well as the order in which they were started. That has
to happen efficiently and in a thread-safe way.
- We have to compute and keep the origin map for actions,
even if we don't serialize the action graph.
|
|
|
|
On the one hand, this message is after an important step in the
build process, to giving the user a better insight into what is
going on. On the other hand, the size of the discovered graph is
useful information, e.g., when comparing with the number of actions
actually traversed when building the requested artifacts.
|
|
... to avoid unnecessary copying and moving of larger objects.
|
|
|
|
This is the initial version of our tool that is able to
build itself. In can be bootstrapped by
./bin/bootstrap.py
Co-authored-by: Oliver Reiche <oliver.reiche@huawei.com>
Co-authored-by: Victor Moreno <victor.moreno1@huawei.com>
|
