tutorial: Add chapter on just-lock

Also update the README.

1 files changed, 236 insertions, 0 deletions

diff --git a/doc/tutorial/just-lock.md b/doc/tutorial/just-lock.md
new file mode 100644
index 00000000..03a65063
--- /dev/null
+++ b/doc/tutorial/just-lock.md
@@ -0,0 +1,236 @@
+Multi-repository configuration management: `just-lock`
+======================================================
+
+The multi-repository build configuration used as input by `just-mr` acts for
+all intents and purposes as a _lockfile_ for *justbuild* projects, containing
+the pinned versions of all content-fixed repositories. This is expected to be
+stored and shipped together with the source code, allowing a consistent
+development environment for all users.
+
+With dependencies of projects coming in many shapes and forms, `just-lock` is
+a tool for generating and maintaining the multi-repositories configuration 
+(lock)file of *justbuild* projects. This tool performs several functionalities,
+such as importing dependencies with repository composition (extending the
+functionality available in the `just-import-git` tool), automatic deduplication
+of identical transitive dependencies (a functionality available also standalone
+in the `just-deduplicate-repos` tool), or setup of repository clones for local
+development. For the purposes of this tutorial, we will focus only on the
+dependency import aspects.
+
+Basic use of `just-lock`
+------------------------
+
+In order to produce the multi-repository build configuration file, `just-lock`
+expects an input configuration file. The format of this file is an extension of
+the one of `just-mr` and it is read as a `JSON` object. The file defines four
+elements:
+
+ - the description of local repositories, as the value for mandatory field
+ `"repositories"`. This usually defines local checkouts, patches, overlays.
+ - the description of remote dependencies, as the optional value for field
+ `"imports"`. For *justbuild* dependencies, their multi-repository configuration
+ (lock)file is taken as the ground truth for importing repository descriptions
+ into the current project. Non-*justbuild* dependencies can also be described
+ here, but can only be imported as-is, with overlays defining how such source
+ code should be integrated (as it will be shown below).
+ - the repository to consider as the main entry point for the build, as the
+ value of the optional field `"main"`.
+ - a list of repository names, as the value of the field `"keep"`. These names
+ will be kept in the final configuration during the automatic process of
+ deduplicating any repeated repositories brought in as transitive dependencies
+ from different imports. For the purposes of this tutorial, the use of this
+ field is not exemplified.
+
+### The input file
+
+For the following, we return to the *hello_world* example from the section on
+[*Building Third-party dependencies*](./third-party-software.md), which depends
+on the open-source project [fmtlib](https://github.com/fmtlib/fmt), in the
+setup that enables high-level target caching.
+
+We define the following `repos.in.json` input configuration file for
+`just-lock`:
+
+``` {.jsonc srcname="repos.in.json"}
+{ "main": "tutorial"
+, "repositories":
+  { "rules-cc":
+    { "repository": "rules-cc-rules-sources"
+    , "target_root": "tutorial-defaults"
+    , "rule_root": "rules-cc"
+    }
+  , "tutorial":
+    { "repository": {"type": "file", "path": "."}
+    , "bindings": {"rules": "rules-cc", "format": "fmtlib"}
+    }
+  , "tutorial-defaults":
+    { "repository":
+      { "type": "file"
+      , "path": "./tutorial-defaults"
+      , "pragma": {"to_git": true}
+      }
+    }
+  , "fmt-targets-layer":
+    { "repository":
+      { "type": "file"
+      , "path": "./fmt-layer"
+      , "pragma": {"to_git": true}
+      }
+    }
+  , "fmtlib":
+    { "repository": "fmtlib-sources"
+    , "target_root": "fmt-targets-layer"
+    , "bindings": {"rules": "rules-cc"}
+    }
+  }
+, "imports":
+  [ { "source": "git"
+    , "branch": "master"
+    , "commit": "3a5f0f0f50c59495ffc3b198df59e6edb8416450"
+    , "url": "https://github.com/just-buildsystem/rules-cc.git"
+    , "repos": [{"alias": "rules-cc-rules-sources", "repo": "rules"}]
+    }
+  , { "source": "git"
+    , "branch": "8.1.1"
+    , "url": "https://github.com/fmtlib/fmt.git"
+    , "repos": [{"alias": "fmtlib-sources"}]
+    , "as plain": true
+    }
+  ]
+}
+```
+
+As already mentioned, the `"imports"` field provides a description of the
+external (usually third-party) dependencies of a project. In our case, we
+use the `rules-cc` and `fmtlib` libraries, described as `git` sources.
+
+The first thing to note is the use of overlays. In our simple project, as we
+do not want to import anything other than the source trees of the needed
+dependencies, we create the `"rules-cc-rules-sources"` and `"fmtlib-sources"`
+names, which will bind the workspace roots of our `"rules-cc"` and `"fmtlib"`
+repositories, respectively, to the descriptions of the remote repositories
+providing the necessary source trees.
+
+The first import description object is for `rules-cc`, which is a *justbuild*
+project hosted as a Git repository, from which we would like to import its
+`"rules"` subdirectory. Thankfully, that project offers a useful shorthand by
+defining in its own locked repositories configuration file the `"rules"` overlay
+repository pointing to the respective subdirectory. `just-lock` will read that
+configuration file in order to produce the resulting configuration, caching any
+fetched source trees. It is thus highly recommended that the same build root as
+the one subsequently used by `just-mr` is provided to `just-lock` (via the
+`--local-build-root` option, with same default behaviour as in `just-mr`).
+
+In the case of `fmtib`, which is also hosted as a Git repository but does not
+provide a *justbuild* configuration file, we can only import it as-is, as a
+complete repository, signaled by setting the `"as plain"` flag. Do note that in
+this case we can limit ourselves to also just providing the `"branch"` field,
+and not also a specific commit. This is because `just-lock` automatically
+interrogates the remote in order to retrieve the top commit associated to a
+certain reference (in this case, a release tag) and pin it to a hard reference
+(in this case, the commit identifier) into the output configuration. This is a
+useful feature, as it is often the case that information about dependencies
+comes in the form of "loose" references, such as release tags or even simply the
+remote location of a distfile, but which `just-lock` then will pin down to a
+content-defined reference, such as a commit, blob, or tree identifier.
+
+### Generating the configuration
+
+We generate an output configuration file `repos.out.json` by running `just-lock`
+with the appropriate arguments, then we build the `helloworld` target with this
+configuration:
+
+``` sh
+$ just-lock -C repos.in.json -o repos.out.json
+[...]
+$
+$ just-mr -C repos.out.json build helloworld
+INFO: Performing repositories setup
+INFO: Found 5 repositories to set up
+INFO: Setup finished, exec ["just","build","-C","...","helloworld"]
+INFO: Requested target is [["@","tutorial","","helloworld"],{}]
+INFO: Analysed target [["@","tutorial","","helloworld"],{}]
+INFO: Export targets found: 1 cached, 0 uncached, 0 not eligible for caching
+INFO: Discovered 4 actions, 2 trees, 0 blobs
+INFO: Building [["@","tutorial","","helloworld"],{}].
+INFO: Processed 4 actions, 4 cache hits.
+INFO: Artifacts built, logical paths are:
+        helloworld [18d25e828a0176cef6fb029bfd83e1862712ec87:132736:x]
+$
+```
+
+As it can be seen, everything comes from cache and the new configuration behaves
+identical to the manually written one. If available, one can inspect the
+difference in content between the two files using the `jq` command-line tool:
+
+``` sh
+$ diff -y <(jq --sort-keys . repos.out.json) <(jq --sort-keys . repos.json)
+{                                                               {
+  "main": "tutorial",                                             "main": "tutorial",
+  "repositories": {                                               "repositories": {
+    "fmt-targets-layer": {                                          "fmt-targets-layer": {
+      "repository": {                                                 "repository": {
+        "path": "./fmt-layer",                                          "path": "./fmt-layer",
+        "pragma": {                                                     "pragma": {
+          "to_git": true                                                  "to_git": true
+        },                                                              },
+        "type": "file"                                                  "type": "file"
+      }                                                               }
+    },                                                              },
+    "fmtlib": {                                                     "fmtlib": {
+      "bindings": {                                                   "bindings": {
+        "rules": "rules-cc"                                             "rules": "rules-cc"
+      },                                                              },
+      "repository": "fmtlib-sources",                         <
+      "target_root": "fmt-targets-layer"                      <
+    },                                                        <
+    "fmtlib-sources": {                                       <
+      "repository": {                                                 "repository": {
+        "branch": "8.1.1",                                              "branch": "8.1.1",
+        "commit": "b6f4ceaed0a0a24ccf575fab6c56dd50ccf6f1a9",           "commit": "b6f4ceaed0a0a24ccf575fab6c56dd50ccf6f1a9",
+        "repository": "https://github.com/fmtlib/fmt.git",              "repository": "https://github.com/fmtlib/fmt.git",
+        "type": "git"                                                   "type": "git"
+      }                                                       |       },
+                                                              >       "target_root": "fmt-targets-layer"
+    },                                                              },
+    "rules-cc": {                                                   "rules-cc": {
+      "repository": "rules-cc-rules-sources",                 <
+      "rule_root": "rules-cc",                                <
+      "target_root": "tutorial-defaults"                      <
+    },                                                        <
+    "rules-cc-rules-sources": {                               <
+      "repository": {                                                 "repository": {
+        "branch": "master",                                             "branch": "master",
+        "commit": "3a5f0f0f50c59495ffc3b198df59e6edb8416450",           "commit": "3a5f0f0f50c59495ffc3b198df59e6edb8416450",
+        "repository": "https://github.com/just-buildsystem/ru           "repository": "https://github.com/just-buildsystem/ru
+        "subdir": "rules",                                              "subdir": "rules",
+        "type": "git"                                                   "type": "git"
+      }                                                       |       },
+                                                              >       "rule_root": "rules-cc",
+                                                              >       "target_root": "tutorial-defaults"
+    },                                                              },
+    "tutorial": {                                                   "tutorial": {
+      "bindings": {                                                   "bindings": {
+        "format": "fmtlib",                                             "format": "fmtlib",
+        "rules": "rules-cc"                                             "rules": "rules-cc"
+      },                                                              },
+      "repository": {                                                 "repository": {
+        "path": ".",                                                    "path": ".",
+        "type": "file"                                                  "type": "file"
+      }                                                               }
+    },                                                              },
+    "tutorial-defaults": {                                          "tutorial-defaults": {
+      "repository": {                                                 "repository": {
+        "path": "./tutorial-defaults",                                  "path": "./tutorial-defaults",
+        "pragma": {                                                     "pragma": {
+          "to_git": true                                                  "to_git": true
+        },                                                              },
+        "type": "file"                                                  "type": "file"
+      }                                                               }
+    }                                                               }
+  }                                                               }
+}                                                               }
+```
+
+Except for the two overlays, kept from the `just-lock` input file configuration,
+the two configuration files have the same content.