~aehlig/just-buildsystem/justbuild.git - Main repository of just-buildsystem, containing the buildtool (just) in bootstrappable form, the multi-repository setup and launch tool (just-mr) and some auxilliary tools.

Age	Commit message (Collapse)	Author
2024-02-20	git repo fetch: support "inherit env"	Klaus Aehlig
	When fetching git repositories, just-mr routinely shells out to git. In this case, allow the user to specify via "inherit env", which environment variables from the host environment should be made available in this action. Typical variables to inherit are ones providing credentials, like SSH_AUTH_SOCK. As the repository description specifies the commit that will be taken, and hence the resulting tree, correctness is not affected by the environement leaking in here.
2024-02-20	Git URL scheme handling: whitelist path instead of non-path protocols	Klaus Aehlig
	At some point we have to decide if a given git repository URL is a path. So far, we assumed that anything not starting with ssh://, http://, or https:// is a path. This ignores the facts that - the file:// scheme, while referring to a file, does not denote a relative path starting file://, - the [user@]host:path scheme is not a path on the local machine, - there exist the URL schemes git://, ftp://, and ftps://, and - future extension might add additional schemes. To also correctly handle new schemes that git might add (which we indeed can handle, as we simply shell out to the git binary), we reverse the approach: we give the user the means to unambigiously specify that they refer to a path on the local machine, by either - using the file:// scheme, - providing an absolute path starting with /, or - providing a relative path starting with ./ All other schemes will not be modified. The file scheme, as well as the git://, http://, and https:// scheme, are handled interally using libgit2; all others are passed on to git in an unmodified form.
2024-02-16	async maps: Create utility library to handle cycle detection	Paul Cristian Sarbu

2024-01-31	just-mr: Serve endpoint always implies a remote-execution endpoint	Paul Cristian Sarbu
	As the serve service always has an associated remote-execution endpoint, when just-mr gets passed only a serve endpoint address it will now default to implying a remote-execution endpoint exists too at that address. Additionally, now the implementation of the --backup-to-remote option more clearly shows that it can only be done in native mode.
2024-01-31	just-mr: Failure in serve api root tree requests should be fatal	Paul Cristian Sarbu
	The requests to retrieve the tree of a commit, archive, or distdir also set up those trees in a way that the serve endpoint can later build against them, besides allowing just-mr to set up roots locally. Therefore, if the witnessing entity (Git commit, content blob, or distdir, respectively) is known to the serve endpoint, then failing to set up the root tree there should result in a failure also of the just-mr setup on the client side.
2024-01-31	serve source tree: Increase server-side granularity in response statuses	Paul Cristian Sarbu
	For archives and Git repositories we should ensure that not finding the witnessing entity (archive content blob or Git commit, respectively) results in a distinct status in the response to a request that sets up roots on the serve endpoint. This will allow just-mr to better handle its interaction with the serve endpoint.
2024-01-26	just-mr 'distdir' repository: Absent roots sync with serve endpoint	Paul Cristian Sarbu
	To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. For a 'distdir' repository we can know the resulting tree identifier directly without actually needing to fetch anything. Therefore, we only set the root as absent if the serve endpoint knows already this tree, if it can set it up itself, or if we can provide this tree to the serve endpoint from one of our CAS locations (local or remote), based on our tree invariant guarantee. A network fetch of the archives never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26	just-mr 'archive' repository: Absent roots sync with serve endpoint	Paul Cristian Sarbu
	To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for an 'archive' repository we only set the root as absent if the serve endpoint knows already the root, it can set it up itself, or we can create the root locally without a network fetch and then upload it to the serve endpoint via the remote CAS. A network fetch never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26	content_git_map: Simplify method signatures	Paul Cristian Sarbu
	...by passing around the AsyncMap key struct instead of individual members. This will also make future code changes more easy to implement and improve code readability.
2024-01-26	just-mr 'git' repository: Absent roots sync with serve endpoint	Paul Cristian Sarbu
	To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git' repository we only set the root as absent if the serve endpoint knows already the root, it can set it up itself, or we can create the root locally without a network fetch and then upload it to the serve endpoint via the remote CAS. A network fetch never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26	just-mr 'git tree' repository: Absent roots sync with serve endpoint	Paul Cristian Sarbu
	To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git tree' repository we only set the root as absent only if the given serve endpoint has this root, or the tree is known locally and can be provided via the remote CAS. While generating an absent root the fetch command will never be called. Generating an absent root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26	fpath_git_map: Sync pragma 'to_git' root trees with serve endpoint	Paul Cristian Sarbu
	Marking a file-type repository as 'to_git' results in a Git-tree type root, which are of course content fixed and can be (and usually are) used by export targets. Therefore, it is beneficial for a serve endpoint, if one is provided, to be aware of such a root and be able to build against it if needed. If the root is marked as absent, this condition becomes mandatory. Generating an absent Git-tree root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26	root_maps: Add utility library for handling absent roots sync	Paul Cristian Sarbu

2024-01-26	just-mr async maps: Wrap passed raw pointers	Paul Cristian Sarbu
	This is to uphold the coding style guide we employ.
2024-01-26	just-mr 'git' repository: Fix wrong witnessing repo when remote CAS provides ↵	Paul Cristian Sarbu
	tree When the remote CAS provides the root tree, we perform an import-to-git operation, therefore the correct witnessing repository for the tree should always be the Git cache.
2024-01-26	root_maps: Small cleanup of includes and targets file	Paul Cristian Sarbu
	...for more easily readable and maintainable target descriptions.
2024-01-24	just-mr: in to_git warning, report actual tool names	Klaus Aehlig
	... instead of some hard-coded strings, as that can be confusing when the tool is packaged under a different name.
2024-01-08	just-mr distdir: Improved handling of absent root	Paul Cristian Sarbu
	If root is marked absent and we're not asked to fetch absent, we can compute the resulting distdir root tree in-memory, as we have all the information. Otherwise, i.e., if we actually need to have the archives locally, we first check if the remote serve can provide them in the remote CAS before continuing as before to fetch the listed archives one at a time.
2023-12-19	just-mr archive: Check for content blob also in Git cache	Paul Cristian Sarbu
	Now we look for the content blob also in the local Git cache, not just in local CAS. If found, we store the blob read from Git cache into local CAS and continue as usual.
2023-12-19	other_tools: Fix missing includes of direct dependencies	Paul Cristian Sarbu
	This fixes the multiple instances of (mainly) missing or wrong includes of standard, third-party, and own headers in the other_tools source folder.
2023-12-19	just-mr setup: Add missing reporting of failed hostname replacement	Paul Cristian Sarbu

2023-12-19	just-mr setup: Fix bug in fetching Git repos	Paul Cristian Sarbu
	This bug was introduced with the alternative mirrors changes.
2023-12-07	just-mr setup: Allow --fetch-absent also for 'git tree' repositories	Paul Cristian Sarbu

2023-12-07	just-mr git tree: Extract the fetch logic in its own async map	Paul Cristian Sarbu

2023-12-07	just-mr git tree: Inquire remote execution before calling fetch command	Paul Cristian Sarbu

2023-12-01	just-mr git: Report all tried remotes if fetch fails	Paul Cristian Sarbu

2023-12-01	commit_git_map: Formatting fix	Paul Cristian Sarbu

2023-11-21	archive repository: Query 'just serve' endpoint also when not marked absent	Paul Cristian Sarbu
	...before going to the network, if 'just serve' endpoint given. ContentCASMap was reworked to provide a cleaner logic for handling pure fetches, which also reduced code duplication in ContentGitMap.
2023-11-21	git repository: Query 'just serve' endpoint also when not marked absent	Paul Cristian Sarbu
	...before going to the network, if 'just serve' endpoint given.
2023-11-15	just-mr: Update to using the static methods of ServeApi	Paul Cristian Sarbu

2023-11-15	git tree repository: Creating absent roots should not run the command	Paul Cristian Sarbu
	Also extended absent-roots test with this scenario.
2023-11-14	just-mr archives fetch: Add logic for local mirrors and preferred hostnames	Paul Cristian Sarbu

2023-11-14	just-mr git fetch: Add logic for local mirrors and preferred hostnames	Paul Cristian Sarbu

2023-11-14	just-mr: Add 'mirrors' field to archive-like repositories	Paul Cristian Sarbu
	Also extends 'distdir' repositories logic accordingly.
2023-11-14	just-mr: Add 'mirrors' field to 'git' repositories	Paul Cristian Sarbu

2023-11-14	commit_git_map fix: Fetch from correct location if fetch URL is a path	Paul Cristian Sarbu

2023-11-13	"git tree" repositories: honor "inherit env"	Klaus Aehlig
	In the specification of the action generating a fixed git tree, also honor "inherit env", i.e., inherit the environment variables specified in this field from the environment just-mr is invoked in. As the expected output is fixed ahead of time anyway, this lack of isolation does not affect correctness.
2023-11-02	content git map: Add handling of --fetch-absent option	Paul Cristian Sarbu

2023-11-02	content git map: Move handling of --fetch-absent flag from checkout to the ↵	Paul Cristian Sarbu
	map itself
2023-11-02	Decoupling symlinks map and CAS utilities from just-mr	Paul Cristian Sarbu
	This is required in order to make them available to 'just serve' in a minimal just installation.
2023-11-02	ArchiveOps: Move libarchive utilities outside other_tools	Paul Cristian Sarbu
	This way they can be used by 'just serve'.
2023-09-22	commit git map: Fix --fetch-absent storing incorrect tree association	Paul Cristian Sarbu
	In order to not pollute builds, one should only store in the id file the association between a commit and its root tree. As the need for such a file only happens when asked to fetch absent roots, which should be a quite rare use case, we can request the whole tree of a commit to be served (incl. synced with remote execution) and perform the subdir tree extraction locally instead.
2023-09-22	commit git map: Improve warning message	Paul Cristian Sarbu

2023-09-22	fpath git map: Remove redundancy in warning message	Paul Cristian Sarbu

2023-09-15	just-mr: ensure that after --fetch-absent a local build is possible	Klaus Aehlig
	... by also keeping the map of commit to tree locally.
2023-09-15	just serve: add remote execution endpoint and --fetch-absent option	Paul Cristian Sarbu
	The serve service will communicate with this endpoint when needed, as well as ensure artifacts it provides are synced with the remote execution CAS, if requested by the client. If just-mr is given the --fetch-absent option, it Always produce present roots irrespective of the 'absent' pragma. For Git repositories marked with the 'absent' pragma, first try to fetch any commit trees provided by the serve endpoint from the execution endpoint CAS, before reverting to a network fetch. Co-authored-by: Klaus Aehlig <klaus.aehlig@huawei.com> Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
2023-09-13	just-mr: Implement 'absent' roots	Paul Cristian Sarbu
	...via an 'absent' pragma in repository descriptions. For 'git'-type repositories, first interrogates a 'just serve' remote, if given, before reverting to fetching from the network.
2023-08-14	just-mr archives: Add 7zip support via autodetection option	Paul Cristian Sarbu
	Similarly to tarballs, an autodetection option for zip-like archives is added to enable a unified handling of both traditional zip and 7zip formats. Thus, for "zip" archives just-mr uses now this autodetection option.
2023-08-14	ArchiveOps: Add proper autodetection option for tarballs	Paul Cristian Sarbu
	We shouldn't exclude the possibility of receiving uncompressed tarballs as repositories in just-mr. Therefore, we introduce an explicit type that performs the autodetection (default behaviour in just-mr). This is done to also be more in line with our implementation which allows the granular handling of various archive types (currently used only for testing purposes).
2023-08-14	ArchiveOps: reduce verbosity in archive type enum	Paul Cristian Sarbu