1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
|
# Tree Overlays
The underlying idea of a tree object is that it is an opaque object,
that can be passed around as a single artifact. Trees can be obtained
as a directory output of an action or as explicit reference to a
source directory. Using a tree rather than a collection of individual
files can be useful for reserving a whole directory for headers
for a particular library in order to avoid future conflicts, or if
the individual outputs of an action are not known in advance. The
latter happens often when calling foreign build systems to build
a third-party library or even toolchain; still, usually those
outputs can be used as opaque trees using appropriate staging.
There are, however, a few examples where opaque trees have to be
combined into a single one, e.g., if there is an external requirement
that certain files be staged flatly in a single directory.
To also support those rare use cases, `just` supports in-memory
actions to compute the overlay of two trees, optionally rejecting
conflicts instead of resolving them in a latest-wins way. Those
actions can be requested in user-defined rules, as well as by the
built-in rules `tree_overlay` and `disjoint_tree_overlay` (where
the latter causes a build error if the trees cannot be overlayed
in a conflict-free way).
Here we demonstrate the way these rules work on an artificial
example which we start from scratch.
``` sh
$ touch ROOT
```
We simply work locally, so our `repos.json` is trivial.
``` {.jsonc srcname="repos.json"}
{"repositories": {"": {"repository": {"type": "file", "path": "."}}}}
```
As a replacement for a foreign-build-system call, consider a target
having a single tree artifact as output, with contents heavily
depending on the configuration.
``` {.jsonc srcname="TARGETS"}
{ "foo":
{ "type": "generic"
, "arguments_config": ["FOO_BINS"]
, "out_dirs": ["bin"]
, "cmds":
[ "mkdir -p bin"
, { "type": "join"
, "$1":
[ "for tool in "
, {"type": "join_cmd", "$1": {"type": "var", "name": "FOO_BINS"}}
, " ; do echo \"foo binary ${tool}\" > bin/\"${tool}\""
, " ; chmod 755 bin/\"${tool}\""
, " ; done"
]
}
]
}
}
```
So, depending on the configuration the output tree has different entries.
``` sh
$ just-mr build -D '{"FOO_BINS": ["version", "upload", "download"]}' -p foo
INFO: Performing repositories setup
INFO: Found 1 repositories involved
INFO: Setup finished, exec ["just","build","-C","...","-D","{\"FOO_BINS\": [\"version\", \"upload\", \"download\"]}","-p"]
INFO: Requested target is [["@","","","foo"],{"FOO_BINS":["version","upload","download"]}]
INFO: Analysed target [["@","","","foo"],{"FOO_BINS":["version","upload","download"]}]
INFO: Discovered 1 actions, 0 tree overlays, 0 trees, 0 blobs
INFO: Building [["@","","","foo"],{"FOO_BINS":["version","upload","download"]}].
INFO: Processed 1 actions, 0 cache hits.
INFO: Artifacts built, logical paths are:
bin [8cd3ecc03f0ba26d9e104e52b40f88a0bc5a84b9:105:t]
{
"download": "[f270834a3411ba7d9e6fab59a8d93e1fbd6e55a3::x]",
"upload": "[af38d5c40e8828c67d0031fce42da86b68f56182::x]",
"version": "[a263fff2a3b94429878303875861fd93bcdbe248::x]"
}
$ just-mr build -D '{"FOO_BINS": ["version", "ci", "co", "rlog"]}' -p foo
...
INFO: Processed 1 actions, 0 cache hits.
INFO: Artifacts built, logical paths are:
bin [eaee8451fe929904016b73c73f466f534e248c3d:127:t]
{
"ci": "[eb8e3ec5baca0f16da7fe8b200181bde123c11bf::x]",
"co": "[1cb77df2638381460b338882e2942b3eb0a09975::x]",
"rlog": "[157a0fca577e8717d68920f76a998ab1b398594d::x]",
"version": "[a263fff2a3b94429878303875861fd93bcdbe248::x]"
}
```
Now, assume we have another such target.
``` {.jsonc srcname="TARGETS"}
...
, "bar":
{ "type": "generic"
, "arguments_config": ["BAR_BINS"]
, "out_dirs": ["bin"]
, "cmds":
[ "mkdir -p bin"
, { "type": "join"
, "$1":
[ "for tool in "
, {"type": "join_cmd", "$1": {"type": "var", "name": "BAR_BINS"}}
, " ; do echo \"bar binary ${tool}\" > bin/\"${tool}\""
, " ; chmod 755 bin/\"${tool}\""
, " ; done"
]
}
]
}
...
```
Both targets produce a tree `bin` instead of a collection of files
within the directory `bin`. Therefore, an overlay at analysis time
could only result in one or the other directory. However, we overlay
the results at build time.
``` {.jsonc srcname="TARGETS"}
...
, "both": {"type": "tree_overlay", "deps": ["foo", "bar"]}
, "both-noconflict": {"type": "disjoint_tree_overlay", "deps": ["foo", "bar"]}
...
```
If the entries do not conflict, in both cases, we get the union of the files.
``` sh
$ just-mr build -D '{"FOO_BINS": ["ci", "co"], "BAR_BINS": ["up", "down"]}' -P bin both
...
INFO: Processed 2 actions, 0 cache hits.
INFO: Artifacts built, logical paths are:
[98e5f0eed05bf887a6c9df7616b7d83323acf677:30:t]
INFO: 'bin' not a direct logical path of the specified target; will take subobject 'bin' of ''
{
"ci": "[eb8e3ec5baca0f16da7fe8b200181bde123c11bf::x]",
"co": "[1cb77df2638381460b338882e2942b3eb0a09975::x]",
"down": "[53aa524d39aff972c976bfd729a3fd26c5d364fd::x]",
"up": "[3bfadc230e82da61f056e1d9acd854298f0b19c3::x]"
}
$ just-mr build -D '{"FOO_BINS": ["ci", "co"], "BAR_BINS": ["up", "down"]}' -P bin both-noconflict
...
INFO: Processed 2 actions, 2 cache hits.
INFO: Artifacts built, logical paths are:
[98e5f0eed05bf887a6c9df7616b7d83323acf677:30:t]
INFO: 'bin' not a direct logical path of the specified target; will take subobject 'bin' of ''
{
"ci": "[eb8e3ec5baca0f16da7fe8b200181bde123c11bf::x]",
"co": "[1cb77df2638381460b338882e2942b3eb0a09975::x]",
"down": "[53aa524d39aff972c976bfd729a3fd26c5d364fd::x]",
"up": "[3bfadc230e82da61f056e1d9acd854298f0b19c3::x]"
}
```
In case of a conflict, however, the targets differ. The first one, `both`, will
overlay the files in a latest-wins fashion, whereas the second will fail when
handling the overlay action.
``` sh
$ just-mr build -D '{"FOO_BINS": ["version", "ci", "co"], "BAR_BINS": ["version", "up", "down"]}' -P bin/version both
...
INFO: Processed 2 actions, 0 cache hits.
INFO: Artifacts built, logical paths are:
[2a26b3dc1774df55eb1f1d9a865611a413204ab2:30:t]
INFO: 'bin/version' not a direct logical path of the specified target; will take subobject 'bin/version' of ''
bar binary version
$ just-mr build -D '{"FOO_BINS": ["version", "ci", "co"], "BAR_BINS": ["version", "up", "down"]}' both-noconflict || :
INFO: Performing repositories setup
INFO: Found 1 repositories involved
INFO: Setup finished, exec ["just","build","-C","...","-D","{\"FOO_BINS\": [\"version\", \"ci\", \"co\"], \"BAR_BINS\": [\"version\", \"up\", \"down\"]}","both-noconflict"]
INFO: Requested target is [["@","","","both-noconflict"],{"BAR_BINS":["version","up","down"],"FOO_BINS":["version","ci","co"]}]
INFO: Analysed target [["@","","","both-noconflict"],{"BAR_BINS":["version","up","down"],"FOO_BINS":["version","ci","co"]}]
INFO: Discovered 2 actions, 1 tree overlays, 2 trees, 0 blobs
INFO: Building [["@","","","both-noconflict"],{"BAR_BINS":["version","up","down"],"FOO_BINS":["version","ci","co"]}].
ERROR (action:aec6a6e881d3c3884420bee1330fbe9702ad3a7af11ee7f21fb59f88d6ba8f38):
Tree-overlay computation failed:
While merging the trees:
- [394c2bd3bf4a07f8e22f6e73c2adcb03f28349df:30:t]
- [e83b879ac94530f49855004138ea96df758b14d1:30:t]
While merging the trees:
- [7a5ed39406620e7b844788589f92fa4c17361c19:0:t]
- [28891f0d46f5053861ac36bc7da18285e65ed267:0:t]
Naming conflict detected at path "version":
- [a263fff2a3b94429878303875861fd93bcdbe248:0:x]
- [e9ebc93e1f91db5fab1fde7d22d8c6f59afab9f1:0:x]
ERROR (action:aec6a6e881d3c3884420bee1330fbe9702ad3a7af11ee7f21fb59f88d6ba8f38):
Failed to execute tree-overlay action.
requested by
- [["@","","","both-noconflict"],{"BAR_BINS":["version","up","down"],"FOO_BINS":["version","ci","co"]}]#0
inputs were
- 394c2bd3bf4a07f8e22f6e73c2adcb03f28349df:30:t
- e83b879ac94530f49855004138ea96df758b14d1:30:t
ERROR: Build failed.
```
|
