~helmut/mdbp.git
3 weeks agopbuilder: fail when apt-get update fails master
Helmut Grohne [Sat, 10 Jul 2021 16:53:45 +0000 (18:53 +0200)]
pbuilder: fail when apt-get update fails

3 weeks agostop using from __future__ import annotations
Helmut Grohne [Sat, 10 Jul 2021 06:09:42 +0000 (08:09 +0200)]
stop using from __future__ import annotations

Its future in Python upstream sees uncertain.

3 weeks agopbuilder: uninline a function for locating the base.tgz
Helmut Grohne [Fri, 9 Jul 2021 18:57:52 +0000 (20:57 +0200)]
pbuilder: uninline a function for locating the base.tgz

3 weeks agouninline helper function temporary_static_file
Helmut Grohne [Thu, 8 Jul 2021 07:11:10 +0000 (09:11 +0200)]
uninline helper function temporary_static_file

3 weeks agopbuilder: rewrite cmd.extend sequence as list
Helmut Grohne [Tue, 6 Jul 2021 12:11:27 +0000 (14:11 +0200)]
pbuilder: rewrite cmd.extend sequence as list

3 weeks agopbuilder: implement support for build_path
Helmut Grohne [Tue, 6 Jul 2021 12:05:25 +0000 (14:05 +0200)]
pbuilder: implement support for build_path

3 weeks agopbuilder: use an ExitStack
Helmut Grohne [Tue, 6 Jul 2021 12:02:29 +0000 (14:02 +0200)]
pbuilder: use an ExitStack

Doing so bears three advantages:
 * Adding more context managers - especially conditional ones - becomes
   easier.
 * Context managers can be delayed until the point where they're
   actually needed without adding to indentation.
 * We get rid of two local variables. Less state in your head.

3 weeks agommdebstrap and sbuild shall agree on what build_path means
Helmut Grohne [Tue, 6 Jul 2021 11:51:41 +0000 (13:51 +0200)]
mmdebstrap and sbuild shall agree on what build_path means

While mmdebstrap would extract the package precisely to the build_path,
sbuild would create a subdirectory <source>-<upstream_version> inside
the build_path. Changing sbuild to do the mmdebstrap behaviour looks
next to impossible, so this commit changes mmdebstrap to behave like
sbuild.

pbuilder also allows changing the build_path in principle. The
documented BUILDDIR variable works quite similar to the sbuild option
--build-path. pbuilder also allows changing the sub directory using the
undocumented BUILDSUBDIR variable. That's another reason to do it like
sbuild.

4 weeks agosbuild: fix quoting of config strings
Helmut Grohne [Thu, 1 Jul 2021 13:11:04 +0000 (15:11 +0200)]
sbuild: fix quoting of config strings

Not escaping a $ would make perl interpolate it at config parsing time.

Fixes: 8ea45852d692 ("sbuild: make environment passing work")

4 weeks agodrop unnecessary execution bit from mdbp/debspawn.py
Helmut Grohne [Thu, 1 Jul 2021 13:08:09 +0000 (15:08 +0200)]
drop unnecessary execution bit from mdbp/debspawn.py

Fixes: c03690f53941 ("implement basic debspawn backend")

5 weeks agorename a number of fields to become similar to debusine
Helmut Grohne [Thu, 24 Jun 2021 11:53:31 +0000 (13:53 +0200)]
rename a number of fields to become similar to debusine

debusine describes a PackageBuildTask in docs/devel/ontology.rst.
Changes performed herein:

 * dscpath -> source_package_path
 * dscuri -> source_package_url
 * buildarch -> build_architecture
 * hostarch -> host_architecture
 * options -> build_options
 * profiles -> build_profiles
 * buildpath -> build_path

6 weeks agoimplement basic debspawn backend
Helmut Grohne [Tue, 15 Jun 2021 12:38:59 +0000 (14:38 +0200)]
implement basic debspawn backend

6 weeks agoallow passing custom options to pbuilder and sbuild
Helmut Grohne [Tue, 15 Jun 2021 11:36:34 +0000 (13:36 +0200)]
allow passing custom options to pbuilder and sbuild

2 months agomention mdbp-ssh in the README.md
Helmut Grohne [Mon, 31 May 2021 06:08:21 +0000 (08:08 +0200)]
mention mdbp-ssh in the README.md

2 months agocorrectly handle integer values for .parallel
Helmut Grohne [Thu, 27 May 2021 17:49:01 +0000 (19:49 +0200)]
correctly handle integer values for .parallel

Fixes: 29859ef384a5 ("support a new field "parallel"")

2 months agoprefer subprocess.call over subprocess.run
Helmut Grohne [Thu, 27 May 2021 17:20:58 +0000 (19:20 +0200)]
prefer subprocess.call over subprocess.run

Avoid pylint complaining and we don't need the CompletedProcess object.

2 months agofix setup.py
Helmut Grohne [Thu, 27 May 2021 16:50:39 +0000 (18:50 +0200)]
fix setup.py

2 months agopbuilder: allow missing .lintian in json
Helmut Grohne [Thu, 27 May 2021 09:59:12 +0000 (11:59 +0200)]
pbuilder: allow missing .lintian in json

Fixes: 65431c5ef882 ("pbuilder: support running lintian")

2 months agoadd new field .output.artifacts to schema
Helmut Grohne [Thu, 27 May 2021 09:54:04 +0000 (11:54 +0200)]
add new field .output.artifacts to schema

Using the field you can specify an ORed set of positive glob-style
patterns for artifacts to retain in the .output.directory. It defaults
to including all artifacts.

2 months agocommon: simplify code using contextlib.suppress
Helmut Grohne [Thu, 20 May 2021 04:41:03 +0000 (06:41 +0200)]
common: simplify code using contextlib.suppress

2 months agoschema: fix indentation
Helmut Grohne [Thu, 20 May 2021 04:39:51 +0000 (06:39 +0200)]
schema: fix indentation

Fixes: 29859ef384a5 ("support a new field "parallel"")

2 months agossh: improve handling of failed builds
Helmut Grohne [Wed, 19 May 2021 12:20:43 +0000 (14:20 +0200)]
ssh: improve handling of failed builds

When the wrapped backend fails, streamapi does not produce an artifact
tar on stdout. When this goes missing, we'll see a traceback. Hide that
traceback and adapt the exit code if necessary.

2 months agosbuild: don't just apt upgrade, dist-upgrade instead
Helmut Grohne [Wed, 19 May 2021 11:28:02 +0000 (13:28 +0200)]
sbuild: don't just apt upgrade, dist-upgrade instead

2 months agopbuilder: support running lintian
Helmut Grohne [Wed, 19 May 2021 04:34:29 +0000 (06:34 +0200)]
pbuilder: support running lintian

2 months agopbuilder: always update and upgrade the chroot
Helmut Grohne [Wed, 19 May 2021 04:33:41 +0000 (06:33 +0200)]
pbuilder: always update and upgrade the chroot

2 months agosbuild: always update and upgrade the chroot
Helmut Grohne [Wed, 19 May 2021 04:31:04 +0000 (06:31 +0200)]
sbuild: always update and upgrade the chroot

2 months agommdebstrap: tell debconf not to generate its database
Helmut Grohne [Tue, 18 May 2021 16:32:19 +0000 (18:32 +0200)]
mmdebstrap: tell debconf not to generate its database

Bluntly copied from pbuilder as it does not seem to cause any problems
there.

2 months agoenable building a source package by name and version
Helmut Grohne [Tue, 18 May 2021 16:25:18 +0000 (18:25 +0200)]
enable building a source package by name and version

If a package is specified by name, it is downloaded inside the chroot
using the embedded deb-src uris. sbuild implements this natively,
pbuilder does not implement this yet, but there is #988419 and
mmdebstrap can make it work.

2 months agommdebstrap: reimplement build hook recursively
Helmut Grohne [Tue, 18 May 2021 13:43:14 +0000 (15:43 +0200)]
mmdebstrap: reimplement build hook recursively

Instead of writing a complex shell script in a limited sub-language of
shell, call out to ourselves and implement the build functionality in
Python.

2 months agommdebstrap: don't keep a lintian.deb either
Helmut Grohne [Mon, 17 May 2021 19:31:56 +0000 (21:31 +0200)]
mmdebstrap: don't keep a lintian.deb either

2 months agommdebstrap: fix logic error in parent commit
Helmut Grohne [Mon, 17 May 2021 19:30:05 +0000 (21:30 +0200)]
mmdebstrap: fix logic error in parent commit

Fixes: a5d4cfd4b996 ("mmdebstrap: don't keep downloaded .deb files")

2 months agommdebstrap: don't keep downloaded .deb files
Helmut Grohne [Sun, 16 May 2021 12:10:44 +0000 (14:10 +0200)]
mmdebstrap: don't keep downloaded .deb files

Since we delete the chroot anyway, there is no need to keep the .debs
for the build.

2 months agommdebstrap: reorder source unpack vs build-dep
Helmut Grohne [Sun, 16 May 2021 12:05:40 +0000 (14:05 +0200)]
mmdebstrap: reorder source unpack vs build-dep

By doing the unpack before the satisfying build dependencies, we can
point apt at the extracted source package and stop talking about the
.dsc file. It also means that we can clean those artifacts earlier.

2 months agommdebstrap: allow a PurePath for chdir in priv_drop
Helmut Grohne [Sun, 16 May 2021 11:35:36 +0000 (13:35 +0200)]
mmdebstrap: allow a PurePath for chdir in priv_drop

2 months agosbuild: make environment passing work
Helmut Grohne [Wed, 12 May 2021 19:26:41 +0000 (21:26 +0200)]
sbuild: make environment passing work

sbuild filters the environment and removes all but a set of whitelisted
variables. Unfortunately, the filter is quite restrictive and filters
e.g. "TARGET", which can be used to configure a binutils build. There is
no good way to get around this environment filter from the command line.
The one way that does work, is passing a SBUILD_CONFIG that contains the
desired variables. Once we do have such a file, we can encode a number
of other command line switches as well.

2 months agoadd ssh remote backend and streamapi wrapper
Helmut Grohne [Tue, 11 May 2021 12:52:29 +0000 (14:52 +0200)]
add ssh remote backend and streamapi wrapper

The ssh backend calls into another host and passes all information on
stdin/stdout/stderr. To do so, it uses a streamapi wrapper on the remote
side that itself calls into a regular backend.

2 months agommdebstrap: install fakeroot
Helmut Grohne [Sun, 9 May 2021 20:56:26 +0000 (22:56 +0200)]
mmdebstrap: install fakeroot

When the package does not say "Rules-Requires-Root: no", we need a root
command and dpkg-buildpackage defaults to fakeroot. Unfortunately, we
cannot tell whether the packages requires fakeroot, because the stanza
is not copied to the .dsc. Unconditionally install fakeroot then.

2 months agoschema: allow more characters in DEB_BUILD_OPTIONS
Helmut Grohne [Sun, 9 May 2021 20:55:27 +0000 (22:55 +0200)]
schema: allow more characters in DEB_BUILD_OPTIONS

In particular, allow DEB_BUILD_OPTIONS=nolang=objc,obj-c++ as this is
used by gcc-N.

2 months agosupport a new field "parallel"
Helmut Grohne [Sun, 9 May 2021 20:30:29 +0000 (22:30 +0200)]
support a new field "parallel"

It is basically the parallel field from DEB_BUILD_OPTION which a special
value "auto" that is converted to the CPU count.

2 months agoschema: fix type of additionalProperties
Helmut Grohne [Sun, 9 May 2021 09:10:14 +0000 (11:10 +0200)]
schema: fix type of additionalProperties

Fixes: bf30d5dd0350 ("schema: generally set additionalProperties to false")

2 months agothe stdout fd contains the build log
Helmut Grohne [Sun, 9 May 2021 05:50:18 +0000 (07:50 +0200)]
the stdout fd contains the build log

Using a pipe enables streaming of logs, which would not be possible with
a regular file output.

3 months agoschema: generally set additionalProperties to false
Helmut Grohne [Tue, 27 Apr 2021 10:01:30 +0000 (12:01 +0200)]
schema: generally set additionalProperties to false

That way the backends abort upon seeing unrecognized properties.

3 months agommdebstrap: move the default mirror into argparse
Helmut Grohne [Mon, 26 Apr 2021 20:36:31 +0000 (22:36 +0200)]
mmdebstrap: move the default mirror into argparse

3 months agosbuild: pass --nolog
Helmut Grohne [Fri, 23 Apr 2021 18:00:14 +0000 (20:00 +0200)]
sbuild: pass --nolog

Neither pbuilder nor mmdebstrap produce a separate log file (beyond the
output on the standard streams). Disabling the extra file improves
consistency.

3 months agopbuilder: pass --no-source-only-changes
Helmut Grohne [Fri, 23 Apr 2021 17:59:19 +0000 (19:59 +0200)]
pbuilder: pass --no-source-only-changes

Neither sbuild nor mmdebstrap produce a source changes file by default.
Skipping it for pbuilder improves consistency.

3 months agoremove redundant values from schema
Helmut Grohne [Fri, 23 Apr 2021 10:35:43 +0000 (12:35 +0200)]
remove redundant values from schema

The bd-uninstallable-explainer value null is equivalent to skipping the
option and thus dropped. The network value "undefined" is equivalent to
skipping the option and thus dropped.

Reported-by: David Kalnischkies <donkult@debian.org>
3 months agofix build/host confusion in mmdebstrap backend
Helmut Grohne [Thu, 22 Apr 2021 18:21:49 +0000 (20:21 +0200)]
fix build/host confusion in mmdebstrap backend

3 months agoforward exit code
Helmut Grohne [Thu, 22 Apr 2021 18:19:46 +0000 (20:19 +0200)]
forward exit code

3 months agofix relative path resolution
Helmut Grohne [Thu, 22 Apr 2021 15:25:01 +0000 (17:25 +0200)]
fix relative path resolution

For .input.dscpath, a relative path is supposed to be relative to the
json file, but it was relative to cwd. For .output.directory, the matter
was unspecified, but should actually work in the same way. Adapt
implementation to documentation and fix the missing specification.

3 months agommdebstrap: add support for lintian
Helmut Grohne [Sun, 18 Apr 2021 18:36:45 +0000 (20:36 +0200)]
mmdebstrap: add support for lintian

3 months agosbuild: add support for lintian.options
Helmut Grohne [Sun, 18 Apr 2021 18:36:28 +0000 (20:36 +0200)]
sbuild: add support for lintian.options

3 months agoinitial checkin of mdbp
Helmut Grohne [Sun, 18 Apr 2021 12:42:27 +0000 (14:42 +0200)]
initial checkin of mdbp

Proof-of-concept status. Some things work.