| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
Compared to the command line, the environment usually is quite
short. So including it in messages reporting about commands does
not introduce a lot of additional noise. However, knowing the
environment can help understanding an error message. Therefore, it
seems a good trade off to include it. Do so.
|
|
|
|
|
|
The gsl-lite implementation is slightly more picky in terms of
type conversions and constness resolution in initializers,
therefore small changes were needed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
To avoid lock contention, our task system queues tasks in several
queues. In order to know when there is no more work to be done, a
global counter keeps track of the total work. Here, it is important
that this counter be incremented before a value is added to the
queue, for otherwise some other thread could pick it up and decrement
the total work load, hence getting us in a state where the counter
underestimates the total amount of work to be done. Fix the order
of those operations.
|
|
... by taking the correct map to report which targets failed
to be analysed and also report about rules that failed to be
analysed.
|
|
Also aligning comments in server-side headers with the proto file
documentation.
|
|
Also updates the tests and all relevant documentation accordingly.
|
|
This will make it available also to just.
|
|
... of an absent target. Here, the request is given by repository
root and filename; so the filename is to be taken relative to the
root, i.e., we have to prefix the targets-file name with the module.
|
|
The result of the analysis is a JSON object containing the keys
`"artifacts"`, `"runfiles"`, and `"provides"`. This JSON object, by
default, is logged. However, it might be useful to process the data
contained in it, while, for example, developing new rules.
This patch adds a new command line option (`--dump-result`), reserved
to the subcommand `analyse`, to dump the analysis result to the given
file or stdout (if `-` is given).
|
|
While this can already be expressed by an "if" statement, having
a dedicated function for logical negation makes some expressions
more readable.
|
|
... using, also for the "then" branch, the empty list as default.
In this way, this statement not only more symmetric, but also
allows shorter representations of some typical expressions.
|
|
Lists are somtimes used in configurations as replacement for tuples.
Providing length gives an easy way to detect usage errors.
|
|
|
|
For setup and update subcommands, the consumer lambdas for their
respective async maps are non-trivial (i.e., do more than keeping
the map values) and operate on ungaurded JSON objects, thus it is
better to guard them against any uncaught exceptions.
|
|
When taking over roots from the input repository description, the
JSON value was not properly checked for validity (i.e., if it is of
type string), resulting in an unhandled exception being thrown. The
issue is fixed with a proper type check.
|
|
... as also just describe can communicate to grpc endpoints.
|
|
During compactification, invalid entries must be deleted.
|
|
|
|
Commit f5f9be5bc07b16807aceac86fba9212e3889762a changed from
tracking progress by cache key to tracking by configured target;
however, the absent-target map was forgotten. Fix this while also
switching to the shortend export-target representation introduced
in the previous commit.
|
|
As we always analyse export targets in their canonical configuration (and
do count in the progress the mapping from obtained configuraiton
to canonical one), the shortend name (i.e., the representation
with null values dropped form the configuration) is still a unique
representation of the target. Use this in progress reporting to
simplify reading the progress sample.
|
|
|
|
... using the same mechanism that just uses. As just-mr now also
does quite some grpc calls, e.g., fetching distribution files from
the remote-execution endpoint and negociating roots with the serve
endpoint, it may now also be affected by a bad network or a service
temporarily be unavailable.
|
|
... to simplify reuse.
|
|
During garbage collection split and remove from the storage every entry that is larger than a threshold.
|
|
During garbage collection remove from the storage every entry that has the large entry.
|
|
|
|
|
|
|
|
|
|
... so that they are available in machine-readable form. In this
way, all logs can automatically be collected without the need of
parsing human-targeted error messages.
|
|
... in a structured way to eventually support machine-readable access to
the identifiers of the log files.
|
|
... in logger extension.
|
|
This means that on the serve endpoint, where a logger for the
GraphTraverser is explicitly set, the log messages during action
execution is also made available to the client via the CAS-stored
analysis and build log blob.
|
|
and trees.
|
|
executable files during splitting.
|
|
Configured targets, by design, cannot distinguish between a value
not occuring in the configuration and occuring there with value
null. Therefore, to understand the conflict, we can as well drop
all the null values of the target configuration when reporting it.
|
|
As we use chunking also for reducing storage, we have to consider
the overhead of block devices which is in the order of kB per file.
So our target chunk size should be at least 2 orders of magnitude
above this. This suggests to minimally aim for a chunk size of
128kB, a target size that also has the advantage the that maximal
chunk size associated with this size is 1MB which is still well
below the maximal transmission size of grpc allowing us to avoid
the streaming API.
As we're scaling everything up by a factor of 16, we also have
to increase the number of bits in the involved masks by 4. We use
this to also extend the window size by using the 2 most significant
octets. Following the advice of the paper proposing FastCDC to
spread out the ones roughly equally suggests 0x4444 as a suitable
value for the two most significant octets.
We also change the suggested extension of the remote-execution API
accordingly. As the precise parameters for FastCDC when announced
over the remote-execution APIs are still under discussion upstream,
we simplify the name to not mention the target size.
|
|
... as the typical chunk size is mainly determined by the masks
used internally. So, as long as we hard code them, we should be
honest to ourselves and accept that the chunking parameters are
hard-coded as well.
|
|
... as this is the only thing the user cares about when trying
to investigate why that action failed.
|
|
... with only the non-null entries of the configuration. This
information is enough for the user to build this target, e.g.,
when searching for the cause of a build failure.
|
