| Age | Commit message (Collapse) | Author |
|---|
|
This service allows to query the configuration of the just-serve
instance. In particular, it is used to double-check that the
associated remote end point is the same used by the interrogating
client.
|
|
It defines two RPC:
- ServeTarget: Given a target-level caching key, returns the computed
value.
- ServeTargetVariables: Given the target-level root tree and the name
of an export target, returns the list of flexible variables from
that target's description.
Co-authored-by: Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com>
|
|
|
|
|
|
Absent roots are characterised only by a Git tree hash, so a new
variant of the underlying stored information was added in the form
of a plain string.
In order to avoid unwanted implicit conversions when instantiating
via literal strings, we force callers of the constructors to
explicitly differentiate between plain strings and filesystem
paths. Existing tests were updated to reflect this.
Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
|
|
|
|
TargetCache...
...backed by the same CAS, but the FileStorage uses the given
shard. This is particularly useful for the just-serve server
implementation, since the sharding must be performed according to the
client's request and not following the server configuration.
|
|
This constructor is used by TargetService::ServeTarget
|
|
|
|
The function ToJson used `file_type` key to express the type of the
artifact, on the other hand, FromJson was expecting `type`.
This patch makes the two functions consistent, prefering `file_type`
for historical reasons.
|
|
|
|
... which was accidentially a list of (a single) object,
instead of only a single JSON object.
|
|
While we don't want to fail if the 'checkouts' map values are not
strings, we shouldn't just accept non-string values either,
instead we should warn the user and continue without them.
|
|
Make sure that all CopyFile, WriteFile, and CreateSymlink
functions properly unlink the target file (if it exists and
overwrite requested) to avoid interferences of the install
command. With this change, the clean up step for install-cas
and the within GraphTraverser can new be omitted.
|
|
... using thread-based parallelism for the blobs of each tree.
|
|
Use parallelism if provided by the build API when synchronizing
artifacts of export targets and when synchronizing artifacts due
to the --remember option. Do so at build parallelism as this the
parallelism suitable for the build API.
|
|
Allow implementations to use a given number of threads to carry out
the synchronisation. In this way parallelism can be achieved even
in situations where batch reading degrades as objects of unknown
size have to be fetched; this situation typically occurs if a tree
object has a large number of direct children that are blobs.
If no implementation is provided, the default implementation is to
fall back to the normal (sequential) CAS synchronisation.
|
|
Typically, the number of export targets is small compared to the
size of the respective export targets. Moreover, export targets are
often chained, resulting in overlapping artifacts that we want to
fetch only once. Therefore, fetch export targets sequentially, giving
room to introduce parallelism in the individual fetch steps later.
While there, also log the beginning of the artifact synchronisation
for the export targets.
|
|
|
|
Also adds missing TARGETS file in serve_api folder and ensures
code comments are consistent with the proto file.
|
|
- add missing serve_api TARGETS file
- rename service client to align with server naming scheme
- fix inconsistencies in comments between implementation and protocol
|
|
This is required in order to make them available to 'just serve'
in a minimal just installation.
|
|
This avoids using the more geenric GitRepoRemote method which
has libcurl as a dependency, something that is not needed for this
Git operation.
|
|
Also fixes a small typo in tree existence checker log messages.
|
|
Co-authored-by: Paul Cristian Sarbu <paul.cristian.sarbu@huawei.com>
|
|
The cache key for an export target should contain as target name
that of the export target (and its effective configuration) rather
than the exported target. As we computed the repository part of
the cache key for the target included in the key, this was still a
correct cache key except in the case an explicit file reference was
exported (as here, the information that the file was to be taken
rather than the target of the same name got lost). We still fix
this issue by making the implementation match our design (rather
than by including the file-reference bit in the cache key), as the
original design gives the cleaner protocol for target-level caching
as a service.
|
|
...of the internal cache used for keeping track of running operations.
|
|
|
|
This was causing the remote serve address to overwrite the one set
for remote execution.
Also, to keep things clean, some common remote server-related
methods and definitions were moved into their own library.
|
|
Just like the remote-execution protocol has several services (Execution,
ActionCache, ContentAddressableStorage, etc), so will the serve
protocol: the actual target-level caching, as well auxilliary
services, like the service to obtain the tree for a given root. Already
follow that scheme, before the protocol gets part of any release.
Also, move the status enum into the respective answer messages. In this
way, we can have different enums for different requests without causing
conflicts on the named enum constants.
|
|
server is started
|
|
The serve service will communicate with this endpoint when needed,
as well as ensure artifacts it provides are synced with the remote
execution CAS, if requested by the client.
If just-mr is given the --fetch-absent option, it Always produce
present roots irrespective of the 'absent' pragma. For Git repositories
marked with the 'absent' pragma, first try to fetch any commit
trees provided by the serve endpoint from the execution endpoint
CAS, before reverting to a network fetch.
Co-authored-by: Klaus Aehlig <klaus.aehlig@huawei.com>
Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
|
|
This functionality will be needed to upload git trees
to a remote-execution end point by `just serve.
|
|
|
|
|
|
...able to request the tree of a commit known to the remote.
|
|
Initial version, to be extended later with other RPCs.
|
|
|
|
|
|
|
|
|
|
There is no need to use the full format API, so avoid increasing
the compile time gratuitously.
|
|
|
|
... in the human-readable description. More precisely, for
any implicit target field, show
- the field documentation given (if any), and
- the resolved targets this field contains.
However, do not show the actual name of the field, as this name
is internal to the rule and not part of the interface.
|
|
|
|
|
|
... to silence false-positive 'possible dangling reference'
warning produced by gcc 13.2.0.
|
|
... which are only actions that, besides giving exit code 0 also
created all the outputs they promised to.
|
|
|
|
... by moving the diagnose and cli related code into to separate
libraries. In this way, we hopefully improve maintainability of
the code base.
|
