summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-03-08mirrors: Add test for SortByHostnameOliver Reiche
2024-03-08mirrors: Implement SortByHostname routineOliver Reiche
2024-03-08CurlURLHandle: Add GetHostname routineOliver Reiche
2024-03-08doc: Update man page on Git file URLsOliver Reiche
2024-03-08just-mr: Normalize paths of Git file URLsOliver Reiche
... and ensure that paths starting with .// remain relative.
2024-03-08just-mr: Report fetch URLs on debug levelOliver Reiche
2024-03-08SystemCommand: Transfer outfile responsibility to callerOliver Reiche
2024-03-08just-mr: Fix shell out executionOliver Reiche
... by avoiding reusing temp dirs for execute. While we are at it, also refactor LocalFetchViaTmpRepo() to create its own empty temp dirs, that cannot be reused by the caller.
2024-03-07just-mr: Fix unguarded optional accessOliver Reiche
2024-03-07TmpDir: Getter should return const refOliver Reiche
... and prohibit moves and move assignments.
2024-03-07Improve documentation for the (internal) test rules with infrastructureKlaus Aehlig
... explaining, in particular, which environment variables are provided by the test-runner framework.
2024-03-07Add test verifying that just-mr only uses the network if necessaryKlaus Aehlig
2024-03-07distdir setup: do not ask remote-excution endpoint for treeKlaus Aehlig
Asking the remote-exeuction endpoint for the tree means, that, upon success, we fetch every archive from the remote execution endpoint, even those that we could get from a provided distdir. As our preference is to only use the network if necessary, simply use the standard way of creating a distdir: create it locally and for each file, try to get it from local resources first, then from the remote-execution endpoint, and finally from other network resources.
2024-03-07Add test verifying reporting levelsKlaus Aehlig
... in particular, verify that no error is reported on successfull command completion.
2024-03-07Down-level log message for errors that can be handledKlaus Aehlig
We deliberately have many functions that do not abort the process on failure and instead simply return a corresponding value. It is then up to the caller to decide how to handle this failure; in particular, such a failure can be expected, e.g., if we try to fetch a file from remote execution first, before fetching it from the upstream location. To have a consistent user experience, nothing that can occur in a successfull build should be reported at error level; moreover, messages that routinely occur during successfull builds should not be reported at progress or above, except for the (stage) result messages and the progress reporter.
2024-03-07serve target: Improved log messagesPaul Cristian Sarbu
2024-03-07serve target: Use CopyFrom to populate Digest message fieldsPaul Cristian Sarbu
2024-03-07exceptions handling: small improvementsPaul Cristian Sarbu
It is better to avoid empty catches when we have a reasonable way to log an (even unlikely) exception, even more so if memory allocations are involved.
2024-03-07just-mr fetch: Correctly report if no fetch requiredPaul Cristian Sarbu
2024-03-07absent_target_map: Improve log messagesPaul Cristian Sarbu
2024-03-07just-mr: Correctly pass client-side auth args to 'just'...Paul Cristian Sarbu
...also for 'analyse' and 'describe'.
2024-03-07just: 'analyse' and 'describe' require client-side authenticationPaul Cristian Sarbu
As these subcommands can access the serve endpoint, it is not enough anymore to just know the remote-execution endpoint, but one must also try to connect to it. Thus, the client-side authentication arguments need to be supported and read. The just.1 man page is also updated accordingly.
2024-03-05end-to-end test: also test foreign-file setup in compatible modeKlaus Aehlig
2024-03-05test distdir serving: verify the generic caseKlaus Aehlig
... where the files are known to serve, but not the resulting directory. We achieve this, by adding an unrealted file to the repository exposed to the serve server.
2024-03-05distdir serving: do not confuse hashesKlaus Aehlig
When just serve has to create a distdir from files known to it, there are two kind of hashes involved. - The requests, as well as the resulting tree always refer to git identifiers. - Additionally, files can be stored in the local CAS of the serve instance. This CAS might use a different hash, e.g., plain sha256. Therefore, we cannot simply forward the hash from the request to a lookup in our local CAS; fortunately, when we add a file to the local CAS, we are told the value of the applicable hash, represented as bazel digest, hence we can simply remember essentially this value (converting it to unprefixed form when running in native mode, as in the following we need the hash as it is part of an artifact digest).
2024-03-05foreign_file_git_map: add missing returnKlaus Aehlig
... after handling absent roots. Otherwise, we violate the condition of calling the setter precisely once.
2024-03-05test summarizer: in case of FLAKY tests, also report failure rateKlaus Aehlig
2024-03-05Update documentation, now that "foreign file" roots are implementedKlaus Aehlig
2024-03-05Add a test verifying that foreign files correctly use serveKlaus Aehlig
... by using the distdir request of the serve API.
2024-03-05Add a test for foreign-file rootsKlaus Aehlig
2024-03-05Support fetching of foreign filesKlaus Aehlig
2024-03-05Support checkout of foreign-file reposKlaus Aehlig
2024-03-05Add a root_map for foreign-file reposKlaus Aehlig
2024-03-05serve client: add custom function to ask for a foreign-file rootKlaus Aehlig
2024-03-05fs_utils: specify path for foreign-file root cacheKlaus Aehlig
As the internal distdir data structure now supports the executable bit, it is also expressive enough to support foreign-file repositories. Hence we can use this cache, getting potentially more cache hits.
2024-03-05Add data structure for foreign-file rootsKlaus Aehlig
2024-03-04serve protocol: support executable distfilesKlaus Aehlig
As serve and just-mr share their caching of description-tree association, we also have to change this cache. Thanks to json encoding before hashing, we know that the old and new hash keys do not overlap, so the change is save. As distdirs also keep the respective files in the git root, no new downlaoding will happen either, hence no warning in the CHANGELOG is needed.
2024-02-29expressions: add kFalseKlaus Aehlig
2024-02-29archive_fetch_map: only have the actual archive content in the interfaceKlaus Aehlig
... as this is the only part this map depends on, and not the information on what to do with the archive, like style of unpacking, etc. In this way, this map will also be reusable for fetching foreign files.
2024-02-29just-mr and serve: Extend use of noexcept specifierPaul Cristian Sarbu
The just serve client-side and API methods, as well as just-mr utilities should use the noexcept specifier.
2024-02-29end-to-end: Print envariables in all just serve testsPaul Cristian Sarbu
...to help with debugging.
2024-02-29verify_hash utils: Clarify return value meaningPaul Cristian Sarbu
2024-02-29serve source tree: Do not sync trees with the remote if in compatible modePaul Cristian Sarbu
2024-02-29serve source tree: Fix bugs and inconsistencies in remote syncPaul Cristian Sarbu
Firstly, in ServeDistdirTree the tree of a distdir should be synced from the corresponding witnessing Git repository (as is the case with all root trees), but was wrongly trying to sync from the local CAS. Secondly, a status of SYNC_ERROR, according to the protocol, must always ensure the tree field is set, but some setter calls were missing. Thirdly, ServeContent and ServeTree failed to report on successfully syncing to remote if the blob or tree, respetively, were uploaded from the local CAS. Lastly, ServeDistdirTree contained some legacy code sections and comments that were superfluous. This commit fixes these issues.
2024-02-28Deduplicate parsing of archive descriptionsKlaus Aehlig
2024-02-27["end-to-end/git-import", "annotations"]: propagate PATHKlaus Aehlig
... to generic actions, in order to not rely on "standard paths" being pulled in.
2024-02-27["end-to-end/remote-execution", "split-splice"] set default TIMEOUT_SCALEKlaus Aehlig
As this test takes significantly more time than the other end-to-end tests, set a suitable default value for the configuration variable TIMEOUT_SCALE.
2024-02-27with_{remote,serve}: honor TIMEOUT_SCALEKlaus Aehlig
Especially the end-to-end tests that use "remote" end points can grow large; therefore, honor the configuration variable TIMEOUT_SCALE, to allow the author to specify that this test takes more time.
2024-02-27test just-mr and serve: Check distdir repository root creationPaul Cristian Sarbu
Extends test coverage for the interaction between 'just-mr setup' and the serve endpoint for distdir repositories.
2024-02-27test just-mr and serve: Check git-tree repository root creationPaul Cristian Sarbu
Extends test coverage for the interaction between 'just-mr setup' and the serve endpoint for git-tree repositories.