| Age | Commit message (Collapse) | Author |
|---|
|
... instead of using pipes that are prone to get full. Also
increase log level to Trace.
|
|
remote endpoint
|
|
... and, in this way, ensure the test also tests properly in
absence of an external remote execution system.
|
|
... similar to ["@", "rules", "shell/test", "script"], but implicitly
starting a remote execution in the background, passing the information
about that endpoint to the test via an environment variable.
|
|
All desired outputs of just-mr (e.g., the config file after a setup call)
are printed to stdout, while all logging messages are printed to stderr.
Therefore, one should avoid explicitly piping stderr to stdout, as this
can break the intended use of just-mr.
|
|
... instead of only using the "install-cas" subcommand,
which considers the file and executable CAS as one single
logical instance. Rather also check the CAS directories
directly to verify that files and executables are actually
downloaded to the correct CAS. Furthermore, added "main" as
an export target to produce an executable file for running
the test.
|
|
Co-authored by: Klaus Aehlig <klaus.aehlig@huawei.com>
|
|
By default, just-mr fetches to the first existing distribution
directory; by default, the first distribution directory to consider
is ~/.distfiles. If an explicit destination is desired, it should
be specified with the -o option. Fix the invocation of just-mr in
the end-to-end test to use the specified destination; this also makes
local execution of te test hermetic (as was originally intended), even
if the user has a directory ~/.distfiles.
|
|
Run build and garbage collection end to end and verify two basic
properties.
- A target used after the the last gc is still available after
another gc, and so are all its parts, even if the output is a
single tree.
- A target not used between to gc is purged from disk and disk
space is not used any more.
|
|
|
|
|
|
Both, the python script as well as the compiled variant of just-mr
fetch try to overwrite already existing distfiles in the distdir.
While it is unclear if overwriting existing distfiles or not
copying to distdir if a file with that name already exists is the
better behaviour, we at least should not error out, as the compiled
just-mr currently does as it sets permission to 444 and hence cannot
overwrite later; fix this by addwing owner write permissions before
overwriting.
|
|
|
|
... to avoid potential staging conflicts. Having them staged to
just-mr was a copy-and-paste error in the first place anyway.
|
|
|
|
in the end-to-end tests
|
|
... and, instead, stores all needed information in git and
CAS-independent index files.
|
|
... so that we can switch in one go and have all tests use a new
just-mr tool, e.g., once we consider the C++ implementation fully
ready. This single point of switch will be relevant as the next
commit will add the first end-to-end test for just-mr itself.
|
|
Compared to the previous commit, the action graphs for just
and its unit tests are unchanged.
- Git hash of action graph for ["",""]:
c6e75f17abd7ffaab6ff9bb725ad67ec0bf6c973
- Git hash of action graph for ["test/buildtool","TESTS"]:
8063dfb3dd7daa9ae01d95c177e14946f785c57e
Refactor:
- "local cflags" to "private-cflags"
- "local defines" to "private-defines"
- "link externals" to "private-ldflags"
- "deps" to "private-deps" for (test) binaries
- "proto" to "private-proto" for binaries
Improvements:
- consistent variable declaration order:
OS, ARCH, HOST_ARCH, TARGET_ARCH,
CC, CXX, CFLAGS, CXXFLAGS, ADD_CFLAGS, ADD_CXXFLAGS,
AR, ENV, PATH
- use fields close to their definition (in RULES)
- use common expression for binaries and test binaries
- split expression "flags" and "compiler"
... to separate ones for CC and CXX.
- rename "transition" to "deps-transition"
... to avoid conflicts with other transitions.
- support "defaults-transition" for CC expressions
Implement:
- "cflags" for libraries
- "private-cflags" for (test) binaries
- "private-defines" for test binaries
- "private-ldflags" for test binaries
- (public) "defines" for libraries
|
|
... in favor of the "tree" built-in rule.
|
|
... so that for every construct supported in rules there is
an analogous one as a built-in rule to allow ad-hoc constructions
without having to write a rule (even though writing a rule is
recommended for everything occuring more than once):
- the "generic" rule allows an ad-hoc ACTION,
- the "file_gen" rule allows an ad-hoc BLOB,
- the "tree" rule allows an an ad-hoc TREE, and
- the "configure" rule allows an ad-hoc configuration transition.
|
|
The dependencies requested in order to read their "outs" or
"runfiles" also contribute to the effective configuration.
|
|
While the built-in "configure" rule is forwarding the result of
the configured target, its own effective configuration is not that
of the configured target. The effective configuration is defined
to be that part of the incoming configuration that can potentially
influence the result of that target; in particular, this includes
variables read in defining the configuration transition.
|
|
... and thus allowing the "business logic" in the configuration
target (e.g., setting defaults and derived options) to be shared
by many targets.
|
|
This commit extends the test case for target-level cached artifacts
synchronization to include all locations of a target-cache entry where known
artifacts can be mentioned. These locations are the 'artifacts', 'runfiles',
and 'provides' map and all are considered now as part of the test.
|
|
This change is introduced to be prepared for future changes such as garbage
collection. It is an incompatible change compared to earlier just versions
since it modifies the local path to the just cache directory, where among
others the CASes, action cache, target-level cache are located.
|
|
Signed-off-by: Goetz Brasche <goetz.brasche@huawei.com>
|
|
|
|
artifacts between remote and local CAS
|
|
|
|
|
|
... also when determining default module or target.
|
|
|
|
|
|
|
|
Before this patch, the built-in "generic" type allowed for just output
files, listed in the field "outs". Now, the type also supports output
directories, listed in the "out_dirs" field. The output directories
are created before the command is executed.
|
|
|
|
Not only trees, but also regular files can disallow paths reaching
into them. If we have a file at a/b then another file at a/b/c
is a staging conflict as well. Make our tool recognize this.
|
|
|
|
Our maps serve two purposes: on the one hand, they can be a generic
key-value association with arbitrary strings as keys. On the other
hand, we use them to describe arrangements of files (inputs to actions,
artifacts or runfiles generated). In this function, certain keys refer
to the same path and hence have to be identifed. Therefore, at places
where the keys clearly have to be paths in the file system, implicitly
normalize them and check for conflicts.
|
|
|
|
... allowing to provide additional information in case of conflict
during flat staging.
|
|
|
|
|
|
Add a test that compiles many C binaries and runs them. The main
purpose of the test is to detect any race conditions in this
scenario.
|
|
|
|
Use the log functionality instead of relying on stderr being
precisely the log. Also check for the number of processed
actions instead of any number of actions that might be
reported in the log. While there, redirect stderr to stdout
to have a unified cronological log.
|
|
Repositories are indentified by free names that are bound,
in a per-repository way, in a global configuration. Add
a test verifying this: the name "other" refers to differnt
repositories in the repositories "A" and "B" and chains
of depending on "other" can be followed.
|
|
... in particular verifying that relative references cannot go
outside a repository and explict file ferences cannot go upwards.
|
|
This test also demonstrates the notion of equality used in our
action graph: actions are considered equal, if they are defined in
the same way (regardless of where they are defined); when looking up
actions in cache, however, the inputs are considered extensionally.
The test also verifies that if one dumps the action graph, the
origins of an action (as the same action can be defined in many
places) are reported correctly.
|
