Document the way we use our documentation strings

1 files changed, 117 insertions, 0 deletions

diff --git a/doc/concepts/doc-strings.org b/doc/concepts/doc-strings.org
new file mode 100644
index 00000000..387749df
--- /dev/null
+++ b/doc/concepts/doc-strings.org
@@ -0,0 +1,117 @@
+* Documentation of build rules, expressions, etc
+
+Build rules can obtain a non-trivial complexity. This is especially
+true if several rules have to exist for slightly different use
+cases, or if the rule supports many different fields. Therefore,
+documentation of the rules (and also expressions for the benefit
+of rule authors) is desirable.
+
+Experience shows that documentation that is not versioned together with
+the code it refers to quickly gets out of date, or lost. Therefore,
+we add documentation directly into the respective definitions.
+
+** Multi-line strings in JSON
+
+In JSON, the newline character is encoded specially and not taken
+literally; also, there is not implict joining of string literals.
+So, in order to also have documentation readable in the JSON
+representation itself, instead of single strings, we take arrays
+of strings, with the understanding that they describe the strings
+obtained by joining the entries with newline characters.
+
+** Documentation is optional
+
+While documentation is highly recommended, it still remains optional.
+Therefore, when in the following we state that a key is for a list
+or a map, it is always implied that it may be absent; in this case,
+the empy array or the empty map is taken as default, respectively.
+
+** Rules
+
+Each rule is described as a JSON object with a fixed set of keys.
+So having fixed keys for documentation does not cause conflicts.
+More precisely, the keys ~doc~, ~field doc~, and ~config_doc~
+are reserved for documentation. Here, ~doc~ has to be a list of
+strings describing the rule in general. ~field doc~ has to be a map
+from (some of) the field names to an array of strings, containing
+additional information on that particular field. ~config_doc~ has
+to be a map from (some of) the config variables to an array of
+strings describing the respective variable.
+
+*** Example
+
+#+BEGIN_SRC
+{ "library":
+  { "doc":
+    [ "A C library"
+    , ""
+    , "Define a library that can be used to be statically linked to a"
+    , "binary. To do so, the target can simply be specified in the deps"
+    , "field of a binary; it can also be a dependecy of another library"
+    , "and the information is then propagated to the corresponding binary."
+    ]
+  , "string_fields": ["name"]
+  , "target_fields": ["srcs", "hdrs", "private-hdrs", "deps"]
+  , "field_doc":
+    { "name":
+      ["The base name of the libary (i.e., the name without the leading lib)."]
+    , "srcs": ["The source files (i.e., *.c files) of the library."]
+    , "hdrs":
+      [ "The public header files of this library. Targets depending on"
+      , "this library will have access to those header files"
+      ]
+    , "private-hdrs":
+      [ "Additional internal header files that are used when compiling"
+      , "the source files. Targets depending on this library have no access"
+      , "to those header files."
+      ]
+    , "deps":
+      [ "Any other libraries that this library uses. The dependency is"
+      , "also propagated (via the link-deps provider) to any consumers of"
+      , "this target. So only direct dependencies should be declared."
+      ]
+    }
+  , "config_vars": ["CC"]
+  , "config_doc":
+    { "CC":
+      [ "single string. defaulting to \"cc\", specifying the compiler"
+      , "to be used. The compiler is also used to launch the preprocessor."
+      ]
+    }
+  , "expression": { ... }
+  }
+}
+#+END_SRC
+
+** Expressions
+
+Expressions are also described by a JSON object with a fixed set of
+keys. Here we use the keys ~doc~ and ~vars_doc~ for documentation,
+where ~doc~ is an array of strings describing the expression as a
+whole and ~vars_doc~ is a map from (some of) the ~vars~ to an array
+of strings describing this variable.
+
+** Export targets
+
+As export targets play the role of interfaces between repositories,
+it is important that they be documented as well. Again, export targets
+are described as a JSON object with fixed set of keys amd we use
+the keys ~doc~ and ~config_doc~ for documentation. Here ~doc~ is an
+array of strings describing the targted in general and ~config_doc~
+is a map from (some of) the variables of the ~flexible_config~ to
+an array of strings describing this parameter.
+
+** Presentation of the documentation
+
+As all documentation are just values (that need not be evaluated)
+in JSON objects, it is easy to write tool rendering documentation
+pages for rules, etc, and we expect those tools to be written
+independently. Nevertheless, for the benefit of developers using
+rules from a git-tree roots that might not be checked out, there is
+a subcommand ~describe~ which takes a target specification like the
+~analyze~ command, looks up the corresponding rule and describes
+it fully, i.e., prints in human-readable form
+- the documentation for the rule
+- all the fields available for that rule together with
+  - their type (~string_field~, ~target_field~, etc), and
+  - their documentation.