~aehlig/just-buildsystem/justbuild.git - Main repository of just-buildsystem, containing the buildtool (just) in bootstrappable form, the multi-repository setup and launch tool (just-mr) and some auxilliary tools.

Age	Commit message (Collapse)	Author
2024-03-08	just-mr: Normalize paths of Git file URLs	Oliver Reiche
	... and ensure that paths starting with .// remain relative.
2024-03-08	just-mr: Fix shell out execution	Oliver Reiche
	... by avoiding reusing temp dirs for execute. While we are at it, also refactor LocalFetchViaTmpRepo() to create its own empty temp dirs, that cannot be reused by the caller.
2024-03-07	distdir setup: do not ask remote-excution endpoint for tree	Klaus Aehlig
	Asking the remote-exeuction endpoint for the tree means, that, upon success, we fetch every archive from the remote execution endpoint, even those that we could get from a provided distdir. As our preference is to only use the network if necessary, simply use the standard way of creating a distdir: create it locally and for each file, try to get it from local resources first, then from the remote-execution endpoint, and finally from other network resources.
2024-03-05	foreign_file_git_map: add missing return	Klaus Aehlig
	... after handling absent roots. Otherwise, we violate the condition of calling the setter precisely once.
2024-03-05	Add a root_map for foreign-file repos	Klaus Aehlig

2024-02-27	just-mr: Correct handling of remotes with respect to compatibility mode	Paul Cristian Sarbu
	In order to set up roots, just-mr is able to interrogate, if given, serve and/or remote-execution endpoints. However, just-mr operates only with Git hashes, i.e., with a native mode CAS. This commit ensures the correct interactions occur between just-mr and the provided endpoints not only in native mode, but also in comaptible mode, where a serve endpoint might be present even if one cannot make use of its associated remote-exection endpoint. The user always gets informed if any incompatibilities are detected.
2024-02-27	commit_git_map: Proper handling of file association	Paul Cristian Sarbu
	This commit fixes the invariant that a file association between a Git commit and the root tree should only be set if that tree is found in our own Git cache. This ensures consistency between present and absent roots and in the interaction with the serve endpoint.
2024-02-27	just-mr setup archive: Local roots require the archive blob to be local too	Paul Cristian Sarbu
	For archive repositories we need to ensure that a non-absent root is backed by an archive content blob in the local CAS, in order to also keep the proper root tree file associations. This change also simplifies the content_cas_map logic by removing the previous separation of implementation logic between fetching and setting up the workspace root.
2024-02-20	git repo fetch: support "inherit env"	Klaus Aehlig
	When fetching git repositories, just-mr routinely shells out to git. In this case, allow the user to specify via "inherit env", which environment variables from the host environment should be made available in this action. Typical variables to inherit are ones providing credentials, like SSH_AUTH_SOCK. As the repository description specifies the commit that will be taken, and hence the resulting tree, correctness is not affected by the environement leaking in here.
2024-02-20	Git URL scheme handling: whitelist path instead of non-path protocols	Klaus Aehlig
	At some point we have to decide if a given git repository URL is a path. So far, we assumed that anything not starting with ssh://, http://, or https:// is a path. This ignores the facts that - the file:// scheme, while referring to a file, does not denote a relative path starting file://, - the [user@]host:path scheme is not a path on the local machine, - there exist the URL schemes git://, ftp://, and ftps://, and - future extension might add additional schemes. To also correctly handle new schemes that git might add (which we indeed can handle, as we simply shell out to the git binary), we reverse the approach: we give the user the means to unambigiously specify that they refer to a path on the local machine, by either - using the file:// scheme, - providing an absolute path starting with /, or - providing a relative path starting with ./ All other schemes will not be modified. The file scheme, as well as the git://, http://, and https:// scheme, are handled interally using libgit2; all others are passed on to git in an unmodified form.
2024-02-16	async maps: Create utility library to handle cycle detection	Paul Cristian Sarbu

2024-01-31	just-mr: Serve endpoint always implies a remote-execution endpoint	Paul Cristian Sarbu
	As the serve service always has an associated remote-execution endpoint, when just-mr gets passed only a serve endpoint address it will now default to implying a remote-execution endpoint exists too at that address. Additionally, now the implementation of the --backup-to-remote option more clearly shows that it can only be done in native mode.
2024-01-31	just-mr: Failure in serve api root tree requests should be fatal	Paul Cristian Sarbu
	The requests to retrieve the tree of a commit, archive, or distdir also set up those trees in a way that the serve endpoint can later build against them, besides allowing just-mr to set up roots locally. Therefore, if the witnessing entity (Git commit, content blob, or distdir, respectively) is known to the serve endpoint, then failing to set up the root tree there should result in a failure also of the just-mr setup on the client side.
2024-01-31	serve source tree: Increase server-side granularity in response statuses	Paul Cristian Sarbu
	For archives and Git repositories we should ensure that not finding the witnessing entity (archive content blob or Git commit, respectively) results in a distinct status in the response to a request that sets up roots on the serve endpoint. This will allow just-mr to better handle its interaction with the serve endpoint.
2024-01-26	just-mr 'distdir' repository: Absent roots sync with serve endpoint	Paul Cristian Sarbu
	To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. For a 'distdir' repository we can know the resulting tree identifier directly without actually needing to fetch anything. Therefore, we only set the root as absent if the serve endpoint knows already this tree, if it can set it up itself, or if we can provide this tree to the serve endpoint from one of our CAS locations (local or remote), based on our tree invariant guarantee. A network fetch of the archives never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26	just-mr 'archive' repository: Absent roots sync with serve endpoint	Paul Cristian Sarbu
	To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for an 'archive' repository we only set the root as absent if the serve endpoint knows already the root, it can set it up itself, or we can create the root locally without a network fetch and then upload it to the serve endpoint via the remote CAS. A network fetch never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26	content_git_map: Simplify method signatures	Paul Cristian Sarbu
	...by passing around the AsyncMap key struct instead of individual members. This will also make future code changes more easy to implement and improve code readability.
2024-01-26	just-mr 'git' repository: Absent roots sync with serve endpoint	Paul Cristian Sarbu
	To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git' repository we only set the root as absent if the serve endpoint knows already the root, it can set it up itself, or we can create the root locally without a network fetch and then upload it to the serve endpoint via the remote CAS. A network fetch never gets performed for an absent root. If a serve endpoint is not provided, an absent root can still be generated, but only if no network fetches are required. In this case a warning is emitted.
2024-01-26	just-mr 'git tree' repository: Absent roots sync with serve endpoint	Paul Cristian Sarbu
	To take advantage of absent roots, we need to ensure that a given serve endpoint can build against the tree of this generated root. To this end, for a 'git tree' repository we only set the root as absent only if the given serve endpoint has this root, or the tree is known locally and can be provided via the remote CAS. While generating an absent root the fetch command will never be called. Generating an absent root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26	fpath_git_map: Sync pragma 'to_git' root trees with serve endpoint	Paul Cristian Sarbu
	Marking a file-type repository as 'to_git' results in a Git-tree type root, which are of course content fixed and can be (and usually are) used by export targets. Therefore, it is beneficial for a serve endpoint, if one is provided, to be aware of such a root and be able to build against it if needed. If the root is marked as absent, this condition becomes mandatory. Generating an absent Git-tree root without being provided a serve endpoint is still allowed, but results in a warning.
2024-01-26	root_maps: Add utility library for handling absent roots sync	Paul Cristian Sarbu

2024-01-26	just-mr async maps: Wrap passed raw pointers	Paul Cristian Sarbu
	This is to uphold the coding style guide we employ.
2024-01-26	just-mr 'git' repository: Fix wrong witnessing repo when remote CAS provides ↵	Paul Cristian Sarbu
	tree When the remote CAS provides the root tree, we perform an import-to-git operation, therefore the correct witnessing repository for the tree should always be the Git cache.
2024-01-26	root_maps: Small cleanup of includes and targets file	Paul Cristian Sarbu
	...for more easily readable and maintainable target descriptions.
2024-01-24	just-mr: in to_git warning, report actual tool names	Klaus Aehlig
	... instead of some hard-coded strings, as that can be confusing when the tool is packaged under a different name.
2024-01-08	just-mr distdir: Improved handling of absent root	Paul Cristian Sarbu
	If root is marked absent and we're not asked to fetch absent, we can compute the resulting distdir root tree in-memory, as we have all the information. Otherwise, i.e., if we actually need to have the archives locally, we first check if the remote serve can provide them in the remote CAS before continuing as before to fetch the listed archives one at a time.
2023-12-19	just-mr archive: Check for content blob also in Git cache	Paul Cristian Sarbu
	Now we look for the content blob also in the local Git cache, not just in local CAS. If found, we store the blob read from Git cache into local CAS and continue as usual.
2023-12-19	other_tools: Fix missing includes of direct dependencies	Paul Cristian Sarbu
	This fixes the multiple instances of (mainly) missing or wrong includes of standard, third-party, and own headers in the other_tools source folder.
2023-12-19	just-mr setup: Add missing reporting of failed hostname replacement	Paul Cristian Sarbu

2023-12-19	just-mr setup: Fix bug in fetching Git repos	Paul Cristian Sarbu
	This bug was introduced with the alternative mirrors changes.
2023-12-07	just-mr setup: Allow --fetch-absent also for 'git tree' repositories	Paul Cristian Sarbu

2023-12-07	just-mr git tree: Extract the fetch logic in its own async map	Paul Cristian Sarbu

2023-12-07	just-mr git tree: Inquire remote execution before calling fetch command	Paul Cristian Sarbu

2023-12-01	just-mr git: Report all tried remotes if fetch fails	Paul Cristian Sarbu

2023-12-01	commit_git_map: Formatting fix	Paul Cristian Sarbu

2023-11-21	archive repository: Query 'just serve' endpoint also when not marked absent	Paul Cristian Sarbu
	...before going to the network, if 'just serve' endpoint given. ContentCASMap was reworked to provide a cleaner logic for handling pure fetches, which also reduced code duplication in ContentGitMap.
2023-11-21	git repository: Query 'just serve' endpoint also when not marked absent	Paul Cristian Sarbu
	...before going to the network, if 'just serve' endpoint given.
2023-11-15	just-mr: Update to using the static methods of ServeApi	Paul Cristian Sarbu

2023-11-15	git tree repository: Creating absent roots should not run the command	Paul Cristian Sarbu
	Also extended absent-roots test with this scenario.
2023-11-14	just-mr archives fetch: Add logic for local mirrors and preferred hostnames	Paul Cristian Sarbu

2023-11-14	just-mr git fetch: Add logic for local mirrors and preferred hostnames	Paul Cristian Sarbu

2023-11-14	just-mr: Add 'mirrors' field to archive-like repositories	Paul Cristian Sarbu
	Also extends 'distdir' repositories logic accordingly.
2023-11-14	just-mr: Add 'mirrors' field to 'git' repositories	Paul Cristian Sarbu

2023-11-14	commit_git_map fix: Fetch from correct location if fetch URL is a path	Paul Cristian Sarbu

2023-11-13	"git tree" repositories: honor "inherit env"	Klaus Aehlig
	In the specification of the action generating a fixed git tree, also honor "inherit env", i.e., inherit the environment variables specified in this field from the environment just-mr is invoked in. As the expected output is fixed ahead of time anyway, this lack of isolation does not affect correctness.
2023-11-02	content git map: Add handling of --fetch-absent option	Paul Cristian Sarbu

2023-11-02	content git map: Move handling of --fetch-absent flag from checkout to the ↵	Paul Cristian Sarbu
	map itself
2023-11-02	Decoupling symlinks map and CAS utilities from just-mr	Paul Cristian Sarbu
	This is required in order to make them available to 'just serve' in a minimal just installation.
2023-11-02	ArchiveOps: Move libarchive utilities outside other_tools	Paul Cristian Sarbu
	This way they can be used by 'just serve'.
2023-09-22	commit git map: Fix --fetch-absent storing incorrect tree association	Paul Cristian Sarbu
	In order to not pollute builds, one should only store in the id file the association between a commit and its root tree. As the need for such a file only happens when asked to fetch absent roots, which should be a quite rare use case, we can request the whole tree of a commit to be served (incl. synced with remote execution) and perform the subdir tree extraction locally instead.