Kill the child processes spawned by a run command

Review changes
Download
Patches
Plain diff

Merged Imported Kill the child processes spawned by a run command

Overview 8
Commits 1
Pipelines 0
Changes 3

Merged Imported Administrator requested to merge fix-timeout-kill-process into master 6 years ago

Overview 8
Commits 1
Pipelines 0
Changes 3

The timeout feature should kill all of the processes that are created by the command. The run command can (and often does) spawn multiple processes which can remain running after the process.pid is killed.

Reproducing the problem:

Use device qemu-amd64
Zap x86_64 apk cache (or delete the cached linux kernel apks then run pmbootstrap index)
Run pmbootstrap install with a short timeout. A timeout shorter than it will take to download the linux kernel package on your connection.
e.g. pmbootstrap.py -t 15 install --no-fde

Observe the processes while the packages are downloading/installing. Notice that the chroot command has spawned 3 processes.

$ps xao pid,ppid,pgid,cmd
31738 23247 31738 python3 ./pmbootstrap.py -t 15 install --no-fde
31746 31738 31738 sudo env -i /bin/sh -c CHARSET=UTF-8 HISTFILE='~/.ash_history' PATH=/usr/lib/ccache/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin SHELL=/bin/ash TERM=xterm /usr/sbin/chroot /ubuntu_home/home/r/.local/var/pmbootstrap/chroot_rootfs_qemu-amd64 /bin/sh -c 'cd /;apk --no-progress add -u --virtual .pmbootstrap postmarketos-base ttf-droid device-qemu-amd64 device-qemu-amd64-kernel-stable postmarketos-ui-weston mesa-dri-virtio dbus chrony cryptsetup eudev haveged htop iw nano openssh postmarketos-mkinitfs sudo tmux util-linux wpa_supplicant shadow networkmanager fontconfig linux-postmarketos-stable weston-clients weston weston-shell-desktop weston-backend-fbdev weston-backend-drm weston-backend-x11 weston-terminal weston-xwayland xorg-server-xwayland postmarketos-demos postmarketos-artwork-wallpapers llvm5-libs libdrm libelf expat libgcc mesa-glapi libstdc++ dbus-libs libcap cryptsetup-libs popt libuuid udev-init-scripts eudev-libs libblkid kmod ncurses-libs libnl3 libmagic openssh-client openssh-sftp-server openssh-server busybox-extras lddtree multipath-tools postmarketos-splash device-mapper parted e2fsprogs e2fsprogs-extra osk-sdl charging-sdl triggerhappy ncurses-terminfo libevent findmnt libfdisk libmount libsmartcols pcsc-lite-libs linux-pam dhcpcd iptables libcurl dbus-glib glib libintl jansson libmm-glib libndp newt nspr nss polkit readline libteam freetype mesa-egl mesa-gles cairo libjpeg-turbo pango pixman libpng wayland-libs-client wayland-libs-cursor wayland-libs-egl libwebp libxkbcommon libinput-libs libunwind wayland-libs-server libweston libweston-desktop mesa-gbm libva mesa-dri-swrast libx11 libxcb libxcursor font-misc-misc font-cursor-misc xkeyboard-config xkbcomp xinit mesa-gl libxau libxdmcp libxfont2 libepoxy libxshmfence gtk+3.0 libffi libpciaccess argon2-libs device-mapper-libs json-c xz-libs ncurses-terminfo-base openssh-keygen openssh-server-common libaio userspace-rcu ttf-dejavu py3-pillow lvm2-libs libcom_err e2fsprogs-libs sdl2 sdl2_ttf directfb libmnl libnftnl-libs ca-certificates nghttp2-libs libssh2 pcre slang sqlite-libs libdaemon libnl3-cli libbz2 libxext libxrender libxft fribidi harfbuzz libevdev mtdev libxfixes encodings font-alias mkfontscale mkfontdir util-macros libxkbfile xauth mcookie xmodmap xrdb libxdamage libxxf86vm libbsd libfontenc shared-mime-info gtk-update-icon-cache libxcomposite libxi libxinerama libxrandr atk at-spi2-atk cairo-gobject cups-libs gdk-pixbuf py3-olefile lcms2 openjpeg python3 tiff device-mapper-event-libs tslib graphite2 libxmu mcpp libxml2 hicolor-icon-theme at-spi2-core avahi-libs gnutls gdbm libxt mcpp-libs libxtst gmp nettle p11-kit libtasn1 libunistring libice libsm'
31747 31746 31738 /bin/sh -c CHARSET=UTF-8 HISTFILE='~/.ash_history' PATH=/usr/lib/ccache/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin SHELL=/bin/ash TERM=xterm /usr/sbin/chroot /ubuntu_home/home/r/.local/var/pmbootstrap/chroot_rootfs_qemu-amd64 /bin/sh -c 'cd /;apk --no-progress add -u --virtual .pmbootstrap postmarketos-base ttf-droid device-qemu-amd64 device-qemu-amd64-kernel-stable postmarketos-ui-weston mesa-dri-virtio dbus chrony cryptsetup eudev haveged htop iw nano openssh postmarketos-mkinitfs sudo tmux util-linux wpa_supplicant shadow networkmanager fontconfig linux-postmarketos-stable weston-clients weston weston-shell-desktop weston-backend-fbdev weston-backend-drm weston-backend-x11 weston-terminal weston-xwayland xorg-server-xwayland postmarketos-demos postmarketos-artwork-wallpapers llvm5-libs libdrm libelf expat libgcc mesa-glapi libstdc++ dbus-libs libcap cryptsetup-libs popt libuuid udev-init-scripts eudev-libs libblkid kmod ncurses-libs libnl3 libmagic openssh-client openssh-sftp-server openssh-server busybox-extras lddtree multipath-tools postmarketos-splash device-mapper parted e2fsprogs e2fsprogs-extra osk-sdl charging-sdl triggerhappy ncurses-terminfo libevent findmnt libfdisk libmount libsmartcols pcsc-lite-libs linux-pam dhcpcd iptables libcurl dbus-glib glib libintl jansson libmm-glib libndp newt nspr nss polkit readline libteam freetype mesa-egl mesa-gles cairo libjpeg-turbo pango pixman libpng wayland-libs-client wayland-libs-cursor wayland-libs-egl libwebp libxkbcommon libinput-libs libunwind wayland-libs-server libweston libweston-desktop mesa-gbm libva mesa-dri-swrast libx11 libxcb libxcursor font-misc-misc font-cursor-misc xkeyboard-config xkbcomp xinit mesa-gl libxau libxdmcp libxfont2 libepoxy libxshmfence gtk+3.0 libffi libpciaccess argon2-libs device-mapper-libs json-c xz-libs ncurses-terminfo-base openssh-keygen openssh-server-common libaio userspace-rcu ttf-dejavu py3-pillow lvm2-libs libcom_err e2fsprogs-libs sdl2 sdl2_ttf directfb libmnl libnftnl-libs ca-certificates nghttp2-libs libssh2 pcre slang sqlite-libs libdaemon libnl3-cli libbz2 libxext libxrender libxft fribidi harfbuzz libevdev mtdev libxfixes encodings font-alias mkfontscale mkfontdir util-macros libxkbfile xauth mcookie xmodmap xrdb libxdamage libxxf86vm libbsd libfontenc shared-mime-info gtk-update-icon-cache libxcomposite libxi libxinerama libxrandr atk at-spi2-atk cairo-gobject cups-libs gdk-pixbuf py3-olefile lcms2 openjpeg python3 tiff device-mapper-event-libs tslib graphite2 libxmu mcpp libxml2 hicolor-icon-theme at-spi2-core avahi-libs gnutls gdbm libxt mcpp-libs libxtst gmp nettle p11-kit libtasn1 libunistring libice libsm'
31748 31747 31738 apk --no-progress add -u --virtual .pmbootstrap postmarketos-base ttf-droid device-qemu-amd64 device-qemu-amd64-kernel-stable postmarketos-ui-weston mesa-dri-virtio dbus chrony cryptsetup eudev haveged htop iw nano openssh postmarketos-mkinitfs sudo tmux util-linux wpa_supplicant shadow networkmanager fontconfig linux-postmarketos-stable weston-clients weston weston-shell-desktop weston-backend-fbdev weston-backend-drm weston-backend-x11 weston-terminal weston-xwayland xorg-server-xwayland postmarketos-demos postmarketos-artwork-wallpapers llvm5-libs libdrm libelf expat libgcc mesa-glapi libstdc++ dbus-libs libcap cryptsetup-libs popt libuuid udev-init-scripts eudev-libs libblkid kmod ncurses-libs libnl3 libmagic openssh-client openssh-sftp-server openssh-server busybox-extras lddtree multipath-tools postmarketos-splash device-mapper parted e2fsprogs e2fsprogs-extra osk-sdl charging-sdl triggerhappy ncurses-terminfo libevent findmnt libfdisk libmount libsmartcols pcsc-lite-libs linux-pam dhcpcd iptables libcurl dbus-glib glib libintl jansson libmm-glib libndp newt nspr nss polkit readline libteam freetype mesa-egl mesa-gles cairo libjpeg-turbo pango pixman libpng wayland-libs-client wayland-libs-cursor wayland-libs-egl libwebp libxkbcommon libinput-libs libunwind wayland-libs-server libweston libweston-desktop mesa-gbm libva mesa-dri-swrast libx11 libxcb libxcursor font-misc-misc font-cursor-misc xkeyboard-config xkbcomp xinit mesa-gl libxau libxdmcp libxfont2 libepoxy libxshmfence gtk+3.0 libffi libpciaccess argon2-libs device-mapper-libs json-c xz-libs ncurses-terminfo-base openssh-keygen openssh-server-common libaio userspace-rcu ttf-dejavu py3-pillow lvm2-libs libcom_err e2fsprogs-libs sdl2 sdl2_ttf directfb libmnl libnftnl-libs ca-certificates nghttp2-libs libssh2 pcre slang sqlite-libs libdaemon libnl3-cli libbz2 libxext libxrender libxft fribidi harfbuzz libevdev mtdev libxfixes encodings font-alias mkfontscale mkfontdir util-macros libxkbfile xauth mcookie xmodmap xrdb libxdamage libxxf86vm libbsd libfontenc shared-mime-info gtk-update-icon-cache libxcomposite libxi libxinerama libxrandr atk at-spi2-atk cairo-gobject cups-libs gdk-pixbuf py3-olefile lcms2 openjpeg python3 tiff device-mapper-event-libs tslib graphite2 libxmu mcpp libxml2 hicolor-icon-theme at-spi2-core avahi-libs gnutls gdbm libxt mcpp-libs libxtst gmp nettle p11-kit libtasn1 libunistring libice libsm

Wait for the timeout as it is downloading a large package such as the linux kernel. Notice that it only kills 1/3 processes.

(031738) [17:16:32] Process did not write any output for 15.0 seconds. Killing it.
(031738) [17:16:32] NOTE: The timeout can be increased with 'pmbootstrap -t'.
(031738) [17:16:32] % sudo kill -9 31746

$ps xao pid,ppid,pgid,cmd
31747 31746 31738 /bin/sh -c CHARSET=UTF-8 HISTFILE='~/.ash_history' PATH=/usr/lib/ccache/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin SHELL=/bin/ash TERM=xterm /usr/sbin/chroot /ubuntu_home/home/r/.local/var/pmbootstrap/chroot_rootfs_qemu-amd64 /bin/sh -c 'cd /;apk --no-progress add -u --virtual .pmbootstrap postmarketos-base ttf-droid device-qemu-amd64 device-qemu-amd64-kernel-stable postmarketos-ui-weston mesa-dri-virtio dbus chrony cryptsetup eudev haveged htop iw nano openssh postmarketos-mkinitfs sudo tmux util-linux wpa_supplicant shadow networkmanager fontconfig linux-postmarketos-stable weston-clients weston weston-shell-desktop weston-backend-fbdev weston-backend-drm weston-backend-x11 weston-terminal weston-xwayland xorg-server-xwayland postmarketos-demos postmarketos-artwork-wallpapers llvm5-libs libdrm libelf expat libgcc mesa-glapi libstdc++ dbus-libs libcap cryptsetup-libs popt libuuid udev-init-scripts eudev-libs libblkid kmod ncurses-libs libnl3 libmagic openssh-client openssh-sftp-server openssh-server busybox-extras lddtree multipath-tools postmarketos-splash device-mapper parted e2fsprogs e2fsprogs-extra osk-sdl charging-sdl triggerhappy ncurses-terminfo libevent findmnt libfdisk libmount libsmartcols pcsc-lite-libs linux-pam dhcpcd iptables libcurl dbus-glib glib libintl jansson libmm-glib libndp newt nspr nss polkit readline libteam freetype mesa-egl mesa-gles cairo libjpeg-turbo pango pixman libpng wayland-libs-client wayland-libs-cursor wayland-libs-egl libwebp libxkbcommon libinput-libs libunwind wayland-libs-server libweston libweston-desktop mesa-gbm libva mesa-dri-swrast libx11 libxcb libxcursor font-misc-misc font-cursor-misc xkeyboard-config xkbcomp xinit mesa-gl libxau libxdmcp libxfont2 libepoxy libxshmfence gtk+3.0 libffi libpciaccess argon2-libs device-mapper-libs json-c xz-libs ncurses-terminfo-base openssh-keygen openssh-server-common libaio userspace-rcu ttf-dejavu py3-pillow lvm2-libs libcom_err e2fsprogs-libs sdl2 sdl2_ttf directfb libmnl libnftnl-libs ca-certificates nghttp2-libs libssh2 pcre slang sqlite-libs libdaemon libnl3-cli libbz2 libxext libxrender libxft fribidi harfbuzz libevdev mtdev libxfixes encodings font-alias mkfontscale mkfontdir util-macros libxkbfile xauth mcookie xmodmap xrdb libxdamage libxxf86vm libbsd libfontenc shared-mime-info gtk-update-icon-cache libxcomposite libxi libxinerama libxrandr atk at-spi2-atk cairo-gobject cups-libs gdk-pixbuf py3-olefile lcms2 openjpeg python3 tiff device-mapper-event-libs tslib graphite2 libxmu mcpp libxml2 hicolor-icon-theme at-spi2-core avahi-libs gnutls gdbm libxt mcpp-libs libxtst gmp nettle p11-kit libtasn1 libunistring libice libsm'
31748 31747 31738 apk --no-progress add -u --virtual .pmbootstrap postmarketos-base ttf-droid device-qemu-amd64 device-qemu-amd64-kernel-stable postmarketos-ui-weston mesa-dri-virtio dbus chrony cryptsetup eudev haveged htop iw nano openssh postmarketos-mkinitfs sudo tmux util-linux wpa_supplicant shadow networkmanager fontconfig linux-postmarketos-stable weston-clients weston weston-shell-desktop weston-backend-fbdev weston-backend-drm weston-backend-x11 weston-terminal weston-xwayland xorg-server-xwayland postmarketos-demos postmarketos-artwork-wallpapers llvm5-libs libdrm libelf expat libgcc mesa-glapi libstdc++ dbus-libs libcap cryptsetup-libs popt libuuid udev-init-scripts eudev-libs libblkid kmod ncurses-libs libnl3 libmagic openssh-client openssh-sftp-server openssh-server busybox-extras lddtree multipath-tools postmarketos-splash device-mapper parted e2fsprogs e2fsprogs-extra osk-sdl charging-sdl triggerhappy ncurses-terminfo libevent findmnt libfdisk libmount libsmartcols pcsc-lite-libs linux-pam dhcpcd iptables libcurl dbus-glib glib libintl jansson libmm-glib libndp newt nspr nss polkit readline libteam freetype mesa-egl mesa-gles cairo libjpeg-turbo pango pixman libpng wayland-libs-client wayland-libs-cursor wayland-libs-egl libwebp libxkbcommon libinput-libs libunwind wayland-libs-server libweston libweston-desktop mesa-gbm libva mesa-dri-swrast libx11 libxcb libxcursor font-misc-misc font-cursor-misc xkeyboard-config xkbcomp xinit mesa-gl libxau libxdmcp libxfont2 libepoxy libxshmfence gtk+3.0 libffi libpciaccess argon2-libs device-mapper-libs json-c xz-libs ncurses-terminfo-base openssh-keygen openssh-server-common libaio userspace-rcu ttf-dejavu py3-pillow lvm2-libs libcom_err e2fsprogs-libs sdl2 sdl2_ttf directfb libmnl libnftnl-libs ca-certificates nghttp2-libs libssh2 pcre slang sqlite-libs libdaemon libnl3-cli libbz2 libxext libxrender libxft fribidi harfbuzz libevdev mtdev libxfixes encodings font-alias mkfontscale mkfontdir util-macros libxkbfile xauth mcookie xmodmap xrdb libxdamage libxxf86vm libbsd libfontenc shared-mime-info gtk-update-icon-cache libxcomposite libxi libxinerama libxrandr atk at-spi2-atk cairo-gobject cups-libs gdk-pixbuf py3-olefile lcms2 openjpeg python3 tiff device-mapper-event-libs tslib graphite2 libxmu mcpp libxml2 hicolor-icon-theme at-spi2-core avahi-libs gnutls gdbm libxt mcpp-libs libxtst gmp nettle p11-kit libtasn1 libunistring libice libsm

This patch recursively kills the PID of the command and its child processes.

There is a simpler method to kill processes by group ID but that doesn't seem to be usable for this particular case. The subprocess.Popen has a start_new_session[1] argument which will run the command under a new session by calling setsid[2]. The command will have its own group ID which can be used to kill all of the child processes at once. But by using setsid the process has no controlling terminal. It looks like this is something we don't want because the command should be a subprocess of pmbootstrap.py otherwise the terminal interaction will break.

[1] https://docs.python.org/3.4/library/subprocess.html?highlight=subprocess#popen-constructor
[2] https://linux.die.net/man/2/setsid

Edited 6 years ago by Administrator