pmbootstrap: kill process if silent for 5 minutes (rewrite logging)

Created by: ollieparanoid

Motivation

Sometimes the build process stops without printing any output when compiling packages for a foreign architecture. This commit enables pmbootstrap to detect such hanging processes and raise an exception at that point. The repository build script can then automatically try the build again instead of waiting forever for completion.

Overview

In pmbootstrap, we had the function pmb.helpers.run.core(), which got called internally when executing these high level functions:

• pmb.helpers.run.user()
• pmb.helpers.run.root()
• pmb.chroot.user()
• pmb.chroot.root()

These redirected most of their parameters to core(), let's look at pmb.helpers.run.user() for example:

def user(args, cmd, log=True, working_dir=None, return_stdout=False,
     check=True, background=False, env={})

In order to implement the new timeout feature in a nice way, the parameters have been changed to:

def user(args, cmd, working_dir=None, output="log",
         output_return=False, check=None, env={}, kill_as_root=False)

1. The output parameter is the big change, we use that instead of log and background now, and it accepts the following values:

• "log" (redirect output to the log file, use timeout)
• "stdout" (redirect to pmbootstrap stdout and log, use timeout)
• "interactive" (redirect to the log, no timeout)
• "tui" (for text-based user interfaces like ncurses; redirect to stdout only, no timeout)
• "background" (redirect to the log, run in background, no timeout)

2. output_return is the same as the old return_stdout. It was renamed, because we look at both the output of stdout and stderr of the program and the location where we redirect it to doesn't have to be pmbootstrap's stdout. So giving it the generic output name makes more sense.

3. kill_as_root should be set to True in case the program gets started with sudo. Then we need root rights to kill it in case it hits the timeout. That only gets used internally, the high level functions set it properly.

Details

• Move pmb.helpers.run.core() to pmb.helpers.run_core.core()
• Rewrite core() to read the output of subprocesses with pipes (in all output modes except for "background" and "tui")
• Create pmb.helpers.run_core.foreground_pipe() to read the output of the subprocess in non-blocking mode using the select syscall (indirectly through Python's selectors.DefaultSelector()), so it really pauses until there is more data in the pipe or until the timeout is hit.
• Adjust the functions pmb.chroot.user(), pmb.chroot.root(), pmb.helpers.run.user(), pmb.helpers.run.root() to use the new parameters
• Adjust usage of these functions to the new parameters everywhere
• Simplify a few of these calls, that had suffix=suffix where simply suffix is enough
• Fix one invalid specification of suffix in a pmb.helpers.run.root call (these only make sense with pmb.chroot.* functions!)
• Remove redundant check=True statements, because we always check the return code by default
• Moved the "Run 'pmbootstrap log' for details message below the other error messages that appear when showing an error, because it is the only one without a timestamp.
• pmb.chroot.zap.zap_pkgs_online_mismatch(): rename "chroot" variable to "suffix" for consistency
• Add test cases for the new functions
• Add pmbootstrap chroot --output to force a specific output mode for easy testing
• Add pmbootstrap --timeout to specify a different timeout value
• Set the default shell to interactive shell for pmbootstrap chroot

Nice side effect

It is now possible to return the output of a command and to write its output to the log and/or pmbootstrap's stdout in real time!

How to test

• Do a full installation
• Test QEMU
• Test the new pmbootstrap parameters
• Test menuconfig

Close #1487 (closed).

---

[x] Merge on GitHub (see https://postmarketos.org/merge)