path: root/share/man/just.1.org

#+TITLE: JUST
#+MAN_CLASS_OPTIONS: section-id=1

* NAME

just - a generic build tool.

* SYNOPSIS

just version\\
just {analyse|build} [OPTIONS] [[module] target]\\
just install -o <OUTPUT_DIR> [OPTIONS] [[module] target]\\
just install-cas [OPTIONS] <OBJECT_ID>\\
just describe [OPTIONS] [[module] target]\\
just rebuild [OPTIONS] [[module] target]\\
just traverse -g <GRAPH_FILE> -o <OUT_DIR> [OPTIONS]

* DESCRIPTION

Just is a generic multi-repository build system; language-specific
knowledge is described in separate rule files. For every build action,
the relative location of the inputs is independent of their physical
location. This staging allows taking sources from different locations
(logical repositories), including bare git repositories. Targets are
defined using JSON format, in proper files (by default, named
~TARGETS~). Targets are uniquely identified by their name, the
repository, and the module they belong to. A module is the relative
path from the repository target root directory to a subdirectory
containing a target file.

The module's name of the targets defined at the target root level is
the empty string. Specifying the correct repository, target root,
module, and target name allows to process that target independently of
the current working directory.

If the module is not specified on the command line, ~just~ sets the
module corresponding to the current working directory.

If a target is not specified, the lexicographically-first target,
according to native byte order, is used. So, a target named with an
empty string will always be the default target for that module.

If a target depends on other targets defined in other modules or
repositories, ~just~ will recursively visit all and only the required
modules.

The main repository is the repository containing the target specified
on the command line. The main repository can either be read from the
multi-repository configuration file if it contains the key ~main~ or
through the option ~--main~. The command-line option ~--main~
overrides what is eventually read from the multi-repository
configuration file. If neither the multi-repository configuration file
contains the ~main~ key nor the ~--main~ option is provided, the lexicographical
first repository from the multi-repository configuration file is used as main.

The ~workspace_root~ of the main repository is then defined as
follows. If the option ~--workspace-root~ is provided, then
~workspace_root~ is set accordingly. If the option is not provided,
~just~ checks if it is specified within the multi-repository
configuration file. If it is, then it is set accordingly. If not, just
starts looking for a marker in the current directory first, then in
all the parent directories until it finds one. The supported markers
are
   - ROOT file (can be empty, content is ignored)
   - WORKSPACE (can be empty, content is ignored)
   - .git (can be either a file - empty or not, content is ignored -
     or the famous directory)
If it fails, ~just~ errors out.

For non-main repositories, the ~workspace_root~ entry must be declared
in the multi-repository configuration file.

Afterwards, ~target_root~, ~rule_root~, and ~expression_root~
directories for the main repository are set using the following
strategy. If the corresponding command-line option is specified, then
it is honored. Otherwise, it is read from the multi-repo configuration
file. If it is not specified there, the default value is used. The
default value of ~target_root~ is ~workspace_root~, of ~rule_root~ is
~target_root~, and of ~expression_root~ is ~rule_root~.

Finally, the file names where ~targets~, ~rules~, and ~expressions~
are defined for the main repository. If the corresponding key is
present in the multi-repository configuration file, it is set
accordingly. If the user gives the corresponding command-line option,
it overwrites what is eventually read from the configuration file. If
they are not explicitly stated neither within the multi-repository
configuration file nor on the command line, the default values are
used, which are ~TARGETS~, ~RULES~, and ~EXPRESSIONS~, respectively.

For non-main repositories (i.e., repositories defining targets
required by the target given on the command line), ~target_root~,
~rule_root~, ~expression_root~, and file names of targets, rules and
expressions, if different from the default values, must be declared in
the multi-repository configuration file.

* SUBCOMMANDS

** version

Print on stdout a JSON object providing version information about
the version of the tool used. This JSON object will contain at
least the following keys.
- ~"version"~ The version, as a list of numbers of length at least
  3, following the usual convention that version numbers are compared
  lexicographically.
- ~"suffix"~ The version suffix as a string. Generally, suffixes
  starting with a ~+~ symbol are positive offsets to the version,
  while suffixes starting with a ~~~ symbol are negative offsets.
- ~"SOURCE_DATE_EPOCH"~ Either a number or ~null~. If it is a
  number, it is the time, in seconds since the epoch, of the last
  commit that went into this binary. It is ~null~ if that time is
  not known (e.g., in development builds).

** analyse|build|install

The subcommands ~analyse~, ~build~, and ~install~ are strictly
related. In fact, from left to right, one is a subset of the
other. ~build~ performs work on top of ~analyse~, and ~install~ on top
of ~build~. When a user issues ~build~, the ~analyse~ is called
underneath. In particular, there is no need to run these three
subcommands sequentially.

*** analyse

~analyse~ reads the target graph from TARGETS files for the given
target, computes the action graph (required by e.g., ~build~,
~install~, ~traverse~), and reports the artifacts, provides, and
runfiles of the analysed target.

In short, the ~analyse~ subcommand identifies all the steps required to
build a given target without actually performing those steps.

This subcommand, issued with proper flags, can dump in JSON format
artifacts, action graph, nodes, actions, (transitive) targets (both
named and anonymous), and trees.

*** build

This subcommand performs the actions contained in the action graph
computed through the ~analyse~ phase.

If building locally, the building process is performed in temporary
separate directories to allow for staging according to the logical
path described in the TARGETS file. Since artifacts are only stored in
the CAS, the user has to use either the ~install~ or ~install-cas~
subcommand to get them.

~just~ allows for both local (i.e., on the same machine where ~just~
is used) and remote compilation (i.e., by sending requests over a TCP
connection, e.g., to a different machine, cluster or cloud
infrastructure). In case of a remote compilation, artifacts are
compiled remotely and stored in the remote CAS. ~install~ and
~install-cas~ subcommands can be used to locally fetch and stage the
desired artifacts.

*** install

The ~install~ subcommand determines which (if any) actions need to be
(re)done and issues the command to (re)run them. Then, it installs the
artifacts (stored in the local or remote CAS) of the processed target
under the given output-dir (set by option ~-o~) honoring the logical
path (aka, staging). If the output path does not exist, it will create
all the necessary folders and subfolders. If files are already present,
they will be overwritten.

** rebuild

This subcommand inspects if builds are fully reproducible or not
(e.g., time stamps are used). It simply rebuilds and compares
artifacts to the cached build reporting actions with different
output. To do so in a meaningful way, it requires that previous build
is already in the cache (local or remote).

** describe

The ~describe~ subcommand allows for describing the rule generating a
target. The rule is resolved in precisely the same way as during the
analysis. The doc-strings (if any) from the rule definition (if
user-defined) are reported, together with a summary of the declared
fields and their types. The multi-repository configuration is honored
in the same way as during ~analyse~ and ~build~; in particular, the
rule definition can also reside in a git-tree root.

** install-cas

~install-cas~ fetches artifacts from CAS (Content Addressable
Storage) by means of their object identifier. The canonical format
of an object identifier is ~[<hash>:<size>:<type>]~; however, when
parsing an object identifier, ~install-cas~ uses the following
default rules, to make usage simpler.
- The square brackets are optional.
- If the size is missing (e.g., because the argument contains no
  colon), or cannot be parsed as a number, this is not an error,
  and the value ~0~ is assumed. While this is almost never the
  correct size, many CAS implementations, including the local CAS
  of just itself, ignore the size for lookups.
- From the type, only the first letter (~f~ for non-executable
  file, ~x~ for executable file, and ~t~ for tree) is significant;
  the rest is ignored. If the type is missing (e.g., because the
  argument contains less than two colons), or its first letter is
  not one of the valid ones, ~f~ is assumed.

Depending on whether the output path is set
or not, the behavior is different.

*** Output path is omitted

If the output path is omitted, it prints the artifact content to
stdout and if the artifact is a tree, it will print a human readable
description.

*** Output path is set

**** Output path does not exist

The artifact will be staged to that path. If artifact is a file, the
installed one will have the name of the output path. If the artifact
is a tree, it will create a directory named like the output path, and
will stage all the entries (subtrees included) under that directory.

**** Output path exists and it is a directory

If the artifact is a tree, a directory named with the hash of tree
itself is created under the output path, and all the entries and
subtrees are installed inside the hash-named directory.

If the artifact is a file, it is installed under the output path and
named according to the hash of the artifact itself.

**** Output path exists and it is a file

If the artifact is a file, it will replace the existing file. If
the artifact is a tree, it will cause an error.

** traverse

It allows for the building and staging of requested artifacts from a
well-defined graph file. See *just-graph-file(5)* for more details.

* OPTIONS

** Generic program information

   *-h*, *--help*
     Output a usage message and exit.\\
     Supported by: all subcommands.

** Compatibility options

   *--compatible* \\
   At increased computational effort, be compatible with the original
   remote build execution protocol. As the change affects identifiers,
   the flag must be used consistently for all related invocations.\\
   Supported by: analyse|build|install-cas|install|rebuild|traverse.

** Build configuration options

   *--action-timeout* NUM\\
   Action timeout in seconds. (Default: 300). The timeout is honored
   only for the remote build.\\
   Supported by: build|install|rebuild|traverse.

   *-c*, *--config* PATH\\
   Path to configuration file.\\
   Supported by: analyse|build|describe|install|rebuild.

   *-C*, *--repository-config* PATH\\
   Path to configuration file for multi-repository builds. See
   *just-repository-config(5)* for more details.\\
   Supported by: analyse|build|describe|install|rebuild|traverse.

   *-D*, *--defines* JSON\\
   Defines, via an in-line JSON object a configuration to overlay
   (in the sense of ~map_union~) to the configuration
   obtained by the *--config* option. If *-D* is given several times,
   only the latest *-D* option is taken.\\
   Supported by: analyse|build|describe|install|rebuild.

   *--request-action-input* ACTION\\
   Modify the request to be, instead of the analysis result of the requested
   target, the input stage of the specified action as artifacts, with empty
   runfiles and a provides map providing the remaining information about the
   action, in particular as ~"cmd"~ the arguments vector and ~"env"~ the
   environment.

   An action can be specified in the following ways
   - an action identifier prefixed by the ~%~ character
   - a number prefixed by the ~#~ character (note that it requires quoting
     on most shells). This specifies the action with that index of the actions
     associated directly with that target; the indices start from ~0~ onwards,
     and negative indices count from the end of the array of actions.
   - an action identifier or number without prefix, provided the action
     identifier does not start with either ~%~ or ~#~ and the number does not
     happen to be a valid action identifier.
   Supported by: analyse|build|describe|install|rebuild.

   *--expression-file-name* TEXT\\
   Name of the expressions file.\\
   Supported by: analyse|build|describe|install|rebuild.

   *--expression-root* PATH\\
   Path of the expression files' root directory. Default: Same as --rule-root.\\
   Supported by: analyse|build|describe|install|rebuild.

   *-L*, *--local-launcher* <JSON_ARRAY>\\
   JSON array with the list of strings representing the launcher to
   prepend actions' commands before being executed locally. Default
   value: ~["env", "--"]~ \\
   Supported by: build|install|rebuild|traverse.

   *--local-build-root* PATH\\
   Root for local CAS, cache, and build directories. The path will be
   created if it does not exist already.\\
   Supported by: build|install-cas|install|rebuild|traverse.

   *--main* NAME\\
   The repository to take the target from.\\
   Supported by: analyse|build|describe|install|rebuild|traverse.

   *--rule-file-name* TEXT\\
   Name of the rules file.\\
   Supported by: analyse|build|describe|install|rebuild.

   *--rule-root* PATH\\
   Path of the rule files' root directory. Default: Same as --target-root\\
   Supported by: analyse|build|describe|install|rebuild.

   *--target-file-name* TEXT\\
   Name of the targets file.\\
   Supported by: analyse|build|describe|install|rebuild.

   *--target-root* PATH\\
   Path of the target files' root directory. Default: Same as --workspace-root\\
   Supported by: analyse|build|describe|install|rebuild.

   *-w*, *--workspace-root* PATH\\
   Path of the workspace's root directory.\\
   Supported by: analyse|build|describe|install|rebuild|traverse.

** General output options

   *--dump-artifacts-to-build* PATH\\
   File path for writing the artifacts to build to. Output format is
   JSON map with staging path as key, and intentional artifact
   description as value.\\
   Supported by: analyse|build|install|rebuild.

   *--dump-artifacts* PATH\\
   Dump artifacts generated by the given target. Using ~-~ as PATH, it
   is interpreted as stdout. Note that, passing ~./-~ will instead
   create a file named ~-~ in the current directory. Output format is
   JSON map with staging path as key, and object id description (hash,
   type, size) as value. Each artifact is guaranteed to be KNOWN in
   CAS. Therefore, this option cannot be used with ~analyse~.\\
   Supported by: build|install|rebuild|traverse.

   *--dump-graph* PATH\\
   File path for writing the action graph description to. See
   *just-graph-file(5)* for more details.\\
   Supported by: analyse|build|install|rebuild.

   *-f,--log-file* PATH\\
   Path to local log file. ~just~ will store the information printed
   on stderr in the log file along with the thread id and timestamp
   when the output has been generated.\\
   Supported by: analyse|build|describe|install|install-cas|rebuild|traverse.

   *--log-limit* NUM\\
   Log limit (higher is more verbose) in interval [0,6] (Default: 3).\\
   Supported by: analyse|build|describe|install|install-cas|rebuild|traverse.

   *--expression-log-limit* NUM\\
   In error messages, truncate the entries in the enumeration of the active
   environment, as well as the expression to be evaluated, to the specified
   number of characters (default: 320).\\
   Supported by: analyse|build|install

   *-P*, *--print-to-stdout* LOGICAL_PATH\\
   After building, print the specified artifact to stdout.\\

   *-s*, *--show-runfiles* \\
   Do not omit runfiles in build report.\\
   Supported by: build|install|rebuild|traverse.

** Output dir and path

   *-o*, *--output-dir* PATH\\
   Path of the directory where outputs will be copied. If the output
   path does not exist, it will create all the necessary folders and
   subfolders. If the artifacts have been already staged, they will be
   overwritten.\\
   Required by: install|traverse.

   *-o*, *--output-path* PATH\\
   Install path for the artifact. Refer to *install-cas* section for
   more details.\\
   Supported by: install-cas

** Parallelism options

   *-J*, *--build-jobs* NUM\\
   Number of jobs to run during build phase. Default: same as --jobs.\\
   Supported by: build|install|rebuild|traverse.

   *-j*, *--jobs* NUM\\
   Number of jobs to run. Default: Number of cores.\\
   Supported by: analyse|build|describe|install|rebuild|traverse.


** Remote execution options
   As remote execution properties shard the target-level cache, they are also
   available for analysis. In this way, the same action identifiers can be
   achieved despite the extensional projection inherent to target level caching,
   e.g., in conjunction with *--request-action-input*.

   *--remote-execution-property* KEY:VAL\\
   Property for remote execution as key-value pair. Specifying this
   option multiple times will accumulate pairs. If multiple pairs with
   the same key are given, the latest wins.\\
   Supported by: analysis|build|install|rebuild|traverse.

   *-r*, *--remote-execution-address* NAME:PORT\\
   Address of the remote execution service.\\
   Supported by: analysis|build|install-cas|install|rebuild|traverse.


** *analyse* specific options

   *--dump-actions* PATH\\
   Dump actions to file. ~-~ is treated as stdout. Output is a list of
   action descriptions, in JSON format, for the given target.

   *--dump-anonymous* PATH\\
   Dump anonymous targets to file. ~-~ is treated as stdout. Output is
   a JSON map, for all transitive targets, with two entries: ~nodes~
   and ~rule_maps~. The former contains maps between node id and the
   node description. ~rule_maps~ states the maps between the
   ~mode_type~ and the rule to use in order to make a target out of
   the node.

   *--dump-blobs* PATH\\
   Dump blobs to file. ~-~ is treated as stdout. The term ~blob~
   identifies a collection of strings that the execution back end
   should be aware of before traversing the action graph. A blob, will
   be referred to as a KNOWN artifact in the action graph.

   *--dump-nodes* PATH\\
   Dump nodes of only the given target to file. ~-~ is treated as
   stdout. Output is a JSON map between node id and its description.

   *--dump-vars* PATH\\
   Dump configuration variables to file. ~-~ is treated as
   stdout. The output is a JSON list of those variable names (in
   lexicographic order) at which the configuration influenced the
   analysis of this target. This might contain variables unset
   in the configuration if the fact that they were unset (and
   hence treated as the default ~null~) was relevant for the
   analysis of that target.

   *--dump-targets* PATH\\
   Dump all transitive targets to file for the given target. ~-~ is
   treated as stdout. Output is a JSON map of all targets encoded as
   tree by their entity name:

   #+BEGIN_SRC shell
    { "#": // anonymous targets
      { "<rule_map_id>":
        { "<node_id>": ["<serialized config1>", ...] } // all configs this target is configured with
      }
    , "@": // "normal" targets
      { "<repo>":
        { "<module>":
          { "<target>": ["<serialized config1>", ...] } // all configs this target is configured with
        }
      }
    }
   #+END_SRC

   *--dump-targets-graph* PATH\\
   Dump the graph of configured targets to a file (even if it
   is called ~-~). In this graph, only non-source targets are
   reported. The graph is represented as a JSON object. The keys
   are the nodes of the graph, and for each node, the value is a
   JSON object containing the different kind of dependencies (each
   represented as a list of nodes).
   - "declared" are the dependencies coming from the target fields
     in the definition of the target
   - "implicit" are the dependencies implicit from the rule definition
   - "anonymous" are the dependencies on anonymous targets implicitly
     referenced during the evaluation of that rule
   While the node names are strings (so that they can be keys in
   a JSON object), they can themselves be decoded as JSON and in
   this way precisely name the configured target. More precisely,
   the JSON decoding of a node name is a list of length two,
   with the first entry being the target name (as ~["@", repo,
   module, target]~ or ~["#", rule_map_id, node_id]~) and the
   second entry the effective configuration.

   *--dump-trees* PATH\\
   Dump trees and all subtrees of the given target to file. ~-~ is
   treated as stdout. Output is a JSON map between tree ids and the
   corresponding artifact map, which maps the path to the artifact
   description.

** *install-cas* specific options

** *rebuild* specific options

   *--vs* NAME:PORT|"local"\\
   Cache endpoint to compare against (use "local" for local cache).

   *--dump-flaky* PATH\\
   Dump flaky actions to file.

** *traverse* specific options

   *-a*, *--artifacts* TEXT\\
   JSON maps between relative path where to copy the artifact and its
   description (as JSON object as well).

   *-g*, *--graph-file* TEXT ~[[REQUIRED]]~ \\
   Path of the file containing the description of the actions. See
   *just-graph-file(5)* for more details.

   *--git-cas* TEXT\\
   Path to a Git repository, containing blobs of potentially missing
   KNOWN artifacts.

** *describe* specific options

   *--json*\\
   Omit pretty-printing and describe rule in JSON format.

   *--rule*\\
   Module and target arguments refer to a rule instead of a target.

* EXIT STATUS

The exit status of ~just~ is one of the following values:
   - 0: the command completed successfully
   - 1: the command could not complete due to some errors (e.g.,
     compilation errors, missing arguments, syntax errors, etc.)
   - 2: the command successfully parsed all the needed files (e.g.,
     TARGETS), successfully compiled the eventually required objects,
     but the generation of some artifacts failed (e.g., a test
     failed).