Planet Debian

Subscribe to Planet Debian feed
Planet Debian - https://planet.debian.org/
Updated: 1 hour 58 min ago

Rhonda D'Vine: Oxa

4 hours 26 min ago

It's been a while. And to be honest, I'm overdue with a few things that I want to get out. One of those things is … Brazil doesn't let me go. I'm watching this country since over a year now, hopefully understandable with the political changes last year and this year's debconf being there, and I promise to go into more details with that in the future because there is more and more to it …

Because one of those things that showed me that Brazil doesn't want to let me go was stumbling upon this artist. They were shared by some friends, and I instantly fell for them. This is about Oxa, but see for yourself:

  • Toy: Their first performance at the show »The Voice of Germany«, where they also stated that they are non-binary. And the song is lovely.
  • Born This Way: With this one, the spoken word interlude gave me goosebumps and I'm astonished that this was possible to get into the show. Big respect!
  • I'm Still Standing: The lyrics in this song are also just as powerful as the other chosen ones. Extremely fine selection!

I'm absolute in love with the person on so many levels–and yes, they are from Brazil originally. Multo brigado, Brazil!

/music | permanent link | Comments: 0 | Flattr this

Enrico Zini: Read only rootfs

4 hours 27 min ago

This is part of a series of post on the design and technical steps of creating Himblick, a digital signage box based on the Raspberry Pi 4.

Another nice to have in a system like Himblick is the root filesystem mounted readonly, with a volatile tempfs overlay on top. This would kind of always guarantee a clean boot without leftovers from a previous run, especially in a system where the most likely mode of shutdown is going to be pulling the plug.

This won't be a guarantee about SD issues developing over time in such a scenario, but it should at least cover the software side of things.

In theory, systemd supports this out of the box with the systemd.volatile=overlay kernel command line option, including integrating this with journald and the way other things get mounted.

In practice:

While things catch up, dracut has a rootovl option that implements something similar.

The procedure becomes, roughly:

# apt install --no-install-recommends dracut
# echo filesystems+=overlay > /etc/dracut.conf.d/overlay.conf
# dracut /boot/initrd.img
# sed -ri -e '/$/ rootovl/' /boot/cmdline.txt" rootovl" in cmdline
# echo "initramfs initrd.img" >> /boot/config.txt

Here's how it ended up in python:

    def setup_readonly_root(self):
        """
        Setup a readonly root with a tempfs overlay
        """
        # Set up a readonly root using dracut's 'rootovl' feature.
        # Eventually do this with systemd's systemd.volatile=overlay option.
        # See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945596
        self.apt_install("dracut")
        self.write_file("/etc/dracut.conf.d/overlay.conf", "filesystems+=overlay\n")
        self.run(["dracut", "--force", "/boot/initrd.img", "4.19.75-v7l+"])
        with self.edit_kernel_commandline("/boot/cmdline.txt") as parts:
            # Add 'rootovl' to /etc/cmdline
            if "rootovl" not in parts:
                parts.append("rootovl")

        # Add initramfs initrd.img to config.txt
        with self.edit_text_file("/boot/config.txt") as lines:
            if "initramfs initrd.img" not in lines:
                lines.append("initramfs initrd.img")

Vincent Bernat: Replacing Orange Livebox router by a Linux box

6 December, 2019 - 18:15

A few months ago, I moved back to France and I settled for Orange as an ISP with a bundle combining Internet and mobile subscription. In Switzerland, I was using my own router instead of the box provided by Swisscom. While there is an abundant documentation to replace the box provided by Orange, the instructions around a plain Linux box are kludgy. I am exposing here my own variation. I am only interested in getting IPv4/IPv6 access: no VoIP, no TV.

Hardware

Orange is using GPON for its FTTH deployment. Therefore, an ONT is needed to encapsulate and decapsulate Ethernet frames into GPON frames. Two form-factors are available. It can be small Huawei HG8010H box also acting as a media converter to Ethernet 1000BASE-T:

The rebranded Huawei HG8010H is acting as an ONT and media converter

With a recent Livebox, Orange usually provides an SFP to be plugged inside the Livebox. For some reason I got the external ONT instead of the SFP version. As I have a Netgear GS110TP with two SFP ports, I have bought an SFP GPON FGS202 on eBay. It is the same model than Orange is providing with its Livebox 4. However, I didn’t get the motivation to test it.1

The Sercomm FGS202 GPON SFP ONT IPv4 configuration

Internet is provided over VLAN 832 and configured with DHCPv4. The first step is to setup the DHCP client to send some additional information, notably the RFC 3118 authentication string. It includes the alphanumeric connection identifier prefixed by fti/ and provided by snail mail. /etc/dhcp/dhclient.conf looks like this:

option rfc3118-authentication code 90 = string;
interface "internet" {
  timeout 60;
  retry 1;
  select-timeout 0;
  send vendor-class-identifier "sagem";
  send user-class "+FSVDSL_livebox.Internet.softathome.Livebox4";
  # fti/xxxxxx identifier can be converted to hexadecimal with:
  #  echo -n 123456 | od -A n -t x1
  send rfc3118-authentication 00:00:00:00:00:00:00:00:00:00:00:1a:09:00:00:05:58:01:03:41:01:0d:66:74:69:2f:xx:xx:xx:xx:xx:xx:xx;
  request subnet-mask, routers,
          domain-name-servers, domain-name,
          broadcast-address,
          dhcp-lease-time, dhcp-renewal-time, dhcp-rebinding-time,
          rfc3118-authentication;
}

Orange expects some control packets, notably DHCP, to be tagged with 802.1p PCP 6. This is a 3-bit field within the Ethernet frame when using VLANs. By default, Linux leaves this field blank. With ip link, we can translate Linux’s skb->priority to a PCP. On Debian, here is how to declare the VLAN interface:2

auto internet
iface internet inet dhcp
  pre-up    ip link add link eno1 name internet type vlan id 832 egress-qos-map 0:0 6:6
  pre-up    /etc/firewall/run
  post-down ip link del internet

The last step is to add the appropriate code in /etc/firewall/run to ensure DHCP, ARP, IGMP and ICMP packets have an internal priority of 6. Netfilter’s CLASSIFY target would be the easiest solution. However, ISC DHCP client is using raw sockets and the packets it sent won’t pass throught Netfilter. A clean solution is to use tc to modify packets just before handing them to the network card. The skbedit action allows to change the priority associated to a packet:

# We need a qdisc to set filters
tc qdisc replace dev internet root handle 1: prio
tc filter del dev internet

# DHCP (raw sockets, do not specify "protocol ip")
tc filter add dev internet parent 1: prio 1 u32 \
     match ip protocol 17 ff \
     match ip dport 67 ffff \
     action skbedit priority 0:6
# ARP
tc filter add dev internet parent 1: prio 2 protocol 0x806 u32 \
     match u32 0 0 \
     action skbedit priority 0:6
# IGMP
tc filter add dev internet parent 1: prio 3 protocol ip u32 \
     match ip protocol 2 ff \
     action skbedit priority 0:6
# ICMP
tc filter add dev internet parent 1: prio 4 protocol ip u32 \
     match ip protocol 1 ff \
     action skbedit priority 0:6

With this configuration in place, ifup internet should get you connected through IPv4.

IPv6 configuration

Native IPv6 is also available over the same VLAN. SLAAC autoconfiguration should be used to get a default route, but not the IP address. Instead, Orange is providing a /60 prefix through DHCPv6 “prefix delegation.”

The DHCP configuration is completed to send the DHCPv6 equivalents for vendor class, user class and authentication string:

# […]
option dhcp6.auth code 11 = string;
option dhcp6.userclass code 15 = string;
option dhcp6.vendorclass code 16 = string;
interface "internet" {
  timeout 60;
  retry 1;
  select-timeout 0;
  # […]
  send dhcp6.vendorclass 00:00:04:0e:00:05:73:61:67:65:6d;
  send dhcp6.userclass 00:2b:46:53:56:44:53:4c:5f:6c:69:76:65:62:6f:78:2e:49:6e:74:65:72:6e:65:74:2e:73:6f:66:74:61:74:68:6f:6d:65:2e:6c:69:76:65:62:6f:78:34;
  send dhcp6.auth 00:00:00:00:00:00:00:00:00:00:00:1a:09:00:00:05:58:01:03:41:01:0d:66:74:69:2f:xx:xx:xx:xx:xx:xx:xx;
  also request dhcp6.auth, dhcp6.vendorclass, dhcp6.userclass;
}

The firewall script is amended to classify DHCPv6 and ICMPv6 packets with priority 6:

# DHCPv6
tc filter add dev internet parent 1: prio 5 protocol ipv6 u32 \
     match ip6 protocol 17 ff \
     match ip6 dport 547 ffff \
     action skbedit priority 0:6
# ICMPv6
tc filter add dev internet parent 1: prio 6 protocol ipv6 u32 \
     match ip6 protocol 58 ff \
     action skbedit priority 0:6

The definition of the internet interface is updated to invoke the DHCPv6 client:

auto internet
iface internet inet dhcp
  pre-up    ip link add link eno1 name internet type vlan id 832 egress-qos-map 0:0 6:6
  pre-up    /etc/firewall/run
  post-down ip link del internet
  post-up   /lib/ifupdown/wait-for-ll6.sh && \
            dhclient -6 -P -pf /run/dhclient6.$IFACE.pid \
                           -lf /var/lib/dhcp/dhclient6.$IFACE.leases \
                           -df /var/lib/dhcp/dhclient.$IFACE.leases \
                           $IFACE
  post-down dhclient -6 -r -pf /run/dhclient6.$IFACE.pid dhclient \
                           -lf /var/lib/dhcp/dhclient6.$IFACE.leases \
                           -df /var/lib/dhcp/dhclient.$IFACE.leases \
                           $IFACE || true

The /lib/ifupdown/wait-for-ll6.sh script waits for the interface to get a link-local address before continuing. The -P option for the DHCPv6 client enables prefix delegation and disables the normal address query.

It is not over: the DHCPv6 client will receive a /60 prefix but there is nothing configured to make use of it. You need to drop a script in /etc/dhcp/dhclient-exit-hooks.d to actually distribute this prefix to your internal network. Here is a simplified non-tested version of this script:

#!/bin/sh
IA_PD_IFACES="lan-trusted lan-guest lan-games"

case $reason in
  BOUND6|EXPIRE6|REBIND6|REBOOT6|RENEW6)
    offset=0
    for iface in $IA_PD_IFACES; do
      # Remove old /64 prefix if there is a change
      [ -n "$old_ip6_prefix" ] && \
        [ "$old_ip6_prefix" != "$new_ip6_prefix" ] && \
        ip -6 addr flush dev $iface scope global
      # Compute and add new /64 prefix
      [ -n "$new_ip6_prefix" ] && {
        offset=$((offset + 1))
        address=$(sipcalc --v6split=64 --split-verbose "$new_ip6_prefix" \
                   | grep '^Compressed' \
                   | awk "(NR == $offset)"' { print $NF }')1/64
        ! ip -6 addr show dev $iface | grep -qwF $address || \
          ip -6 addr add $address dev $iface
    done
esac

At the top of the script, the IA_PD_IFACES variable contains the list of internal interfaces. From the /60 provided in $new_ip6_prefix, the script will assign a /64 to each of them—along with the first address. For example, when being assigned 2001:db8:f:b00::/60, we get:

$ ip -brief -6 a show scope global
lan-trusted@eno1  UP  2001:db8:f:b00::1/64
lan-guest@eno1    UP  2001:db8:f:b01::1/64
lan-games@eno1    UP  2001:db8:f:b02::1/64

I am using dnsmasq to offer IPv6 router advertisements to hosts in each network. This is done through the dhcp-range directive:

dhcp-range=::,constructor:lan-trusted,ra-names
dhcp-range=::,constructor:lan-guest,ra-names
dhcp-range=::,constructor:lan-games,ra-names

The script also handles the default route by switching accept_ra to 2 for the internet interface to accept IPv6 router advertisements even when forwarding is enabled and sending an IPv6 router discovery packet using rdisc6:

case $old_ip6_prefix,$new_ip6_prefix in
  *,)
    # No IPv6 prefix delegation, remove old route
    sysctl -qw net/ipv6/conf/$interface/accept_ra=0
    ip -6 route del default proto ra || true
    ;;
  *)
    # Otherwise, get a default route
    sysctl -qw net/ipv6/conf/$interface/accept_ra=2
    rdisc6 $interface
    ;;
esac

Be sure to use the complete script instead of the shortened code above! If after ifdown internet && ifup internet, you don’t get a /60 prefix, you may have to reboot the ONT to clear an old DHCP lease.

  1. As Orange is using the serial number to authorize the ONT, my plan is to call Orange customer service, pretend I have got a replacement and provide the new serial number. ↩︎

  2. There is no need to have the VLAN number in the interface name. I usually leaves them out as it doesn’t help to describe the interface. The VLAN number can still be recovered with ip -d link show. ↩︎

Gunnar Wolf: My greedy hands are full of books! (Made With Creative Commons) @ccmx @creativecommons @xattack @scannopolis

6 December, 2019 - 07:44

FINALLY!

Made with Creative Commons is translated to Spanish, printed, and available!

Over two years after starting the project, 976 commits, getting involved in long processes (besides the scope we originally envisioned, such as waiting for the translation to be refereed or going over two quite long rounds of copyediting), after leading a team of five translators to Spanish and working closely with a similar team that's quite close to publishing the equivalent translation in Norwegian... Behold!

I won't get again in details on the contents of this book, as I have repeatedly talked about it in the blog. The photo above is of the pages where the CC licensing schemes are presented. And the following is a page I like including in all of my (so far, three) published books:

I have made a point of requiring my university's editorial department to use the legal page to be very explicit regarding the expected usage of this book, by inviting every person that comes across it to copy this book.

So... Where can you get your paws on one of them? Well, of course, you are welcome to come to our institute's bookstore and buy one. For people in general, MX$280 (≈US$15), for UNAM-community, MX$140 (≈US$7.50).

Of course, that's quite inconvenient for people living over 15Km from me, right? What about those living in other countries?

The book can also be downloaded from the Institute's repository. And I will soon upload it to an online, on-demand printing site (probably lulu.com or something like that. Can you suggest one?).

AttachmentSize Finally: a box full of books80.66 KB How to use CC licenses?78.48 KB Copy this book!70.5 KB

Enrico Zini: Temporarily suspending automount on a device

6 December, 2019 - 06:00

This is part of a series of post on the design and technical steps of creating Himblick, a digital signage box based on the Raspberry Pi 4.

After seeing lots of automatic mount/umount notifications during provisioning, we wondered if it would be possibile to temporarily disable them while we're working on the SD card.

It turns out that it's possible, and here's a convenient python context manager to do it cleanly, based on /usr/lib/udisks2/udisks2-inhibit, but adding the possibility of inhibiting automounting only on one specific device:

    @contextmanager
    def pause_automounting(self, dev: Dict[str, Any]):
        """
        Pause automounting on the device for the duration of this context
        manager
        """
        # See /usr/lib/udisks2/udisks2-inhibit
        devpath = dev["path"]
        rules_dir = "/run/udev/rules.d"
        os.makedirs(rules_dir, exist_ok=True)
        rule_file = os.path.join(rules_dir, "90-udisks-inhibit-" + devpath.replace("/", "_") + ".rules")
        with open(rule_file, "wt") as fd:
            print('SUBSYSTEM=="block", ENV{DEVNAME}=="' + devpath + '*", ENV{UDISKS_IGNORE}="1"', file=fd)
            fd.flush()
            os.fsync(fd.fileno())
        run(["udevadm", "control", "--reload"])
        run(["udevadm", "trigger", "--settle", "--subsystem-match=block"])
        try:
            yield
        finally:
            os.unlink(rule_file)
            run(["udevadm", "control", "--reload"])
            run(["udevadm", "trigger", "--settle", "--subsystem-match=block"])

Molly de Blanc: Free software activities (November 2019)

5 December, 2019 - 22:14

November brings two things very demanding of my time: Thanksgiving and the start of fundraising season.

Free software activities (personal)
  • The Open Source Initiative had it’s twice-a-year face to face board meeting! Good times all around.
  • Debian is having a GR. I’ve been following the development of proposals and conversation, which is basically a part time job in and of itself.
  • Participated in Debian Community Team meetings.
  • I started drafting Bits from the Debian Community Team.
  • Wrote some blog posts! I liked them this month.
  • Wearing multiple hats I attended SustainNYC, talking about sustainability in free and open source software.
  • I submitted to some CFPs — SCaLE, FOSSASIA, and OSCON.
  • I am serving on the papers committee for CopyLeftConf, and for this I reviewed proposals.
Free software activities (professional)
  • We launched a fundraiser! (About a patent infringement case)
  • Funding a legal case is an expensive proposition, so I am also meeting with companies and potential large donors interested in helping out with the case.
  • We launched another fundraiser! (About general Foundation activities)
  • I participated in the hiring process to fill two roles at the GNOME Foundation.

Jonas Meurer: switch to sway

5 December, 2019 - 21:10
Switching from Gnome to a tiling window manager

After having thought about it since "forever", I finally decided to switch to a tiling window manager. I went with sway since it runs on wayland and since it seems to be the recommended "wayland version of i3", a tiling window manager that many of my tech friends use ;)

After a few days of using sway, I'm pretty sure that I won't switch back anytime soon. It feels super convenient to have all windows tiled on the screen and being able to rearrange and resize them easily with a few keyboard shortcuts.

There's still some things that didn't work instantly, so I'll try to document them here in hope that it's useful to others. Feedback welcome!

This blog post covers the following topics:

Install sway on Debian Buster

I run Debian Buster on my work machine. The sway components aren't available in Buster or buster-backports yet, so I went with installing the packages from Unstable or experimental manually. I'll probably help with backporting them to buster-backports once I settled on using sway.

Lucky enough, sway packages only bring one dependency that's not satisfied in Buster, which is libjson-c4. So for now, to install the sway Debian packages on Buster, you have to do the following:

mkdir ~/devel/sway && cd ~/devel/sway

wget http://ftp.de.debian.org/debian/pool/main/w/wlroots/libwlroots3_0.7.0-2_amd64.deb
wget http://ftp.de.debian.org/debian/pool/main/s/scdoc/scdoc_1.10.0-1_amd64.deb
wget http://ftp.de.debian.org/debian/pool/main/s/swaybg/swaybg_1.0-2_amd64.deb
wget http://ftp.de.debian.org/debian/pool/main/s/swaylock/swaylock_1.4-1_amd64.deb
wget http://ftp.de.debian.org/debian/pool/main/s/swayidle/swayidle_1.5-1_amd64.deb
wget http://ftp.de.debian.org/debian/pool/main/s/sway/sway-backgrounds_1.2-1_all.deb
wget http://ftp.de.debian.org/debian/pool/main/j/json-c/libjson-c4_0.13.1+dfsg-6_amd64.deb
wget http://ftp.de.debian.org/debian/pool/main/s/sway/sway_1.2-1_amd64.deb

apt install ./libwlroots3_0.7.0-2_amd64.deb ./scdoc_1.10.0-1_amd64.deb ./swaybg_1.0-2_amd64.deb ./swaylock_1.4-1_amd64.deb ./swayidle_1.5-1_amd64.deb ./sway-backgrounds_1.2-1_all.deb ./libjson-c4_0.13.1 ./sway_1.2-1_amd64.deb

apt install dunst i3status suckless-tools
Basic sway configuration

Sway brings a good basic configuration at /etc/sway/config. In order to customize it, copy the file over to ~/.config/sway/config. First things I changed were the following:

# Disable windows title bars
default_borter pixel

# Use tilix wrapper as terminal emulator (more on that later)
set $term ~/.config/sway/scripts/tilix-wrapper.sh

# My internal laptop screen
set $laptop_screen eDP-1

# Command to lock screen
set $lock 'swaylock -F -f -e -K -l -c 000000'

# Default wallpaper
output * bg ~/Pictures/favourite_background.jpg fill

# Idle configuration
exec swayidle -w \
         timeout 300 $lock \
         timeout 600 'swaymsg "output * dpms off"' \
         resume 'swaymsg "output * dpms on"' \
         before-sleep $lock

# Internal Thinkpad Keyboard
input "1:1:AT_Translated_Set_2_keyboard" {
    xkb_layout de,us
    # Change keyboard layouts on <Super>+<Space>
    xkb_options grp:win_space_toggle
}

# Cherry Keyboard
input "1130:275:Cherry_GmbH_CHERRY_Wired_Keyboard" {
    xkb_layout de,us
    # Change keyboard layouts on <Super>+<Space>
    xkb_options grp:win_space_toggle
}

# Internal Thinkpad Touchscreen
input "2:7:SynPS/2_Synaptics_TouchPad" natural_scroll "enabled"

# Status Bar
bar {
    position top
    # Use i3status as status bar
    status_command i3status
}

# Custom key bindings

# Lock screen
bindsym $mod+Escape exec $lock

# Audio and brightness key bindings
bindsym XF86AudioRaiseVolume exec pactl set-sink-volume @DEFAULT_SINK@ +5%
bindsym XF86AudioLowerVolume exec pactl set-sink-volume @DEFAULT_SINK@ -5%
bindsym XF86AudioMute exec pactl set-sink-mute @DEFAULT_SINK@ toggle
bindsym XF86AudioMicMute exec pactl set-source-mute @DEFAULT_SOURCE@ toggle
bindsym XF86MonBrightnessDown exec brightnessctl set 5%-
bindsym XF86MonBrightnessUp exec brightnessctl set +5%
bindsym XF86AudioPlay exec playerctl play-pause
bindsym XF86AudioNext exec playerctl next
bindsym XF86AudioPrev exec playerctl previous

# Bindings for Firefox and Thunderbird
bindsym $mod+Shift+b exec "env MOZ_ENABLE_WAYLAND=1 firefox"
bindsym $mod+Shift+m exec "thunderbird"

# Autostart

# Start dunst, a notification daemon
exec dunst

# Start some programs in fixed worspaces
assign [app_id="firefox"] → 1
exec "env MOZ_ENABLE_WAYLAND=1 firefox"
assign [class="thunderbird"] → 2
exec "thunderbird"
Picking an application launcher

The default application launcher to be used is dmenu (from suckless-tools). While it works okayish, I don't particularly like it. In my eyes, it looks rather old-fashioned, and even worse, it doesn't seem to have support for freedesktop.org desktop entries.

I looked around a bit and wofi sounded pretty promising. It's not in Debian yet but was easy to compile. A big downer though is that it depends on a newer libglib2.0 version (2.60) than in Debian Buster. I still compiled it in a Bullseye schroot and got a first impression. I like it's look and feel (after a bit CSS customization) and probably I'll go with packaging it for Debian.

For the moment, I'm stuck with dmenu on my working system, though.

Configure the status bar

I decided to go with the i3status status bar and it serves my purposes pretty well. Here's my config (/.config/i3status/config):

# i3status configuration file.
# see "man i3status" for documentation.

# It is important that this file is edited as UTF-8.
# The following line should contain a sharp s:
# ß
# If the above line is not correctly displayed, fix your editor first!

general {
        #colors = true
        colors = false
        interval = 5
}

order += "load"
order += "wireless _first_"
order += "ethernet _first_"
order += "path_exists VPN"
order += "battery all"
order += "tztime local"

# Customized wireless status
wireless _first_ {
        format_up = "W: (%quality at %essid) %ip"
        format_down = "W: down"
}

# Only show ethernet status when connected
ethernet _first_ {
        # if you use %speed, i3status requires root privileges
        format_up = "E: %ip"
        format_down = ""
}

# Display VPN status
path_exists VPN {
        # path exists when a VPN tunnel launched by nmcli/nm-applet is active
        path = "/proc/sys/net/ipv4/conf/tun0"
}

# Customized battery status
battery all {
        format = "%status %percentage"
        status_chr = "⚡"
        status_bat = "🔋"
        status_full = "☻"
}

# Localized time format
tztime local {
        #format = "%Y-%m-%d %H:%M:%S"
        format = "%a %d. %b %Y %H:%M"
}

load {
        format = "L: %1min"
}
Configure a notification daemon

I'm really used to getting notifications by my chat programs (XMPP, IRC, Signal), and I don't want to dismiss this. So I installed dunst and configured sway to auto-start it (see above). That's it, it worked instantly. Well, that was easy :)

Preserve working directory in new terminal instances

One thing that really annoyed me after switching to sway was, that the working directory wasn't preserved when spawning new terminal instances. I often open five or more terminal instances in parallel when working on a complex project, and I'm very used to just open a new terminal and continue working in the same directory there immediately.

So I was really eager to find a solution here. Turned out that it's not that easy and needs a bit of dirty scripting, but I found a solution (with help from some nice folks in #sway on Freenode).

First some words about the problem: spawning a new terminal in sway doesn't use whatever sophisticated means to spawn new instances of the same terminal process. Instead, it just spawns a fresh process of your favourite terminal emulator. While I really like tilix and used it as a tiling terminal emulator, I no longer want to use it's tiling features when I now have a tiling window manager. I'll stick for tilix for now as I like its look and feel, though.

So if the new terminal emulator process doesn't know about the working directory of your former terminal, what to do about it?

The solution: Luckily, it's possible to identify the PID of your focused window in sway using swaymsg -t get_tree. In case that the focused window is a terminal emulator, it's parent ID should be your shell. And the shells PWD can easily be determined by reading the symlink /proc/$PID/cwd.

So let's put this in a wrapper script under ~/.config/sway/scripts/tilix-wrapper.sh:

#!/bin/sh

# Small script that tries to determine the PWD of the focused terminal
# (in sway tiling window manager) and pass it to the newly spawned one.

TERMINAL_CMD="tilix --new-process"

FOCUSED_PID=""
if [ ! type jq 2>/dev/null ]; then
    echo "ERROR: jq not installed" >&2
else
    FOCUSED_PID="$(swaymsg -t get_tree | jq '.. | select(.type?) |
        select(.type=="con") | select(.focused==true).pid')"
fi

FOCUSED_PWD=""
# Check if $FOCUSED_PID is an integer
if [ "$FOCUSED_PID" -eq "$FOCUSED_PID" 2>/dev/null ]; then
    FOCUSED_PPID="$(ps -o pid= --ppid "$FOCUSED_PID" | awk '{print $1}')"
    if [ "$FOCUSED_PPID" -eq "$FOCUSED_PPID" 2>/dev/null ]; then
        FOCUSED_PWD="$(readlink "/proc/$FOCUSED_PPID/cwd")"
    fi
fi

# Spawn terminal in background
if [ -d "$FOCUSED_PWD" ]; then
    $TERMINAL_CMD --working-directory="$FOCUSED_PWD" $@ &
else
    $TERMINAL_CMD $@ &
fi

Finally, we have to set the script as $term in sways config (see above). Yay, now I've a solution to preserve my working directory when spawning new terminals!

Use gnome-keyring as SSH agent with sway

Another super annoying thing was that my SSH agent no longer worked with sway, mostly because I used gnome-keyring before and it wasn't spawned automatically when starting sway. So let's change that. I found it a bit complicated to get this working as docs on the internet said a lot of different things, but in the end, the following worked.

Since I still use gdm3 as desktop manager, gnome-keyring-daemon is started automatically during login. So the only thing that's missing is to initalize the gnome-keyring-daemon when starting a terminal. To do so, add the following to ~/.profile (in order to only do it on a login shell):

# Connect to and initalize gnome-keyring-daemon when in sway session
if [ "$DESKTOP_SESSION" = "sway" ]; then
    export $(gnome-keyring-daemon --start)
fi

What's missing
  • I want to start profanity (XMPP client) and irssi (IRC client) automatically in workspace 3, but so far I failed to find a working filter for sways assign feature to identify tilix instances with profanity/irssi (in order to automatically assign those terminals to workspace 3).
  • I miss the redshift feature of gnome 3. redshift itself doesn't support wayland yet. There's a fork with wayland support, but I didn't find time to look into it yet.
  • I'll probably switch from i3status to py3status soon as it's list of modules looks really promising.

Enrico Zini: himblick media player

5 December, 2019 - 06:00

This is part of a series of post on the design and technical steps of creating Himblick, a digital signage box based on the Raspberry Pi 4.

Finally, we have enough pieces to start working on the media player. It's been way more work that expected getting to this point, and I hope that this series of posts could help others getting a faster start.

To begin with, we'd like to be able to show:

  • PDF files automatically looping through the pages
  • Image galleries automatically looping
  • Looping videos
  • ODP presentations
Configuring the screen

The first thing to do on startup is to configure the screen based on the himblick.conf settings:

    def configure_screen(self):
        """
        Configure the screen based on himblick.conf
        """
        # Set screen orientation
        orientation = self.settings.general("screen orientation")
        if orientation:
            run(["xrandr", "--orientation", orientation])

        mode = self.settings.general("screen mode")
        if mode:
            res = run(["xrandr", "--query"], capture_output=True, text=True)
            re_output = re.compile(r"^(\S+) connected ")
            for line in res.stdout.splitlines():
                mo = re_output.match(line)
                if mo:
                    output_name = mo.group(1)
                    break
            else:
                output_name = None
            run(["xrandr", "--output", output_name, "--mode", mode])

This had the extra complication of needing to parse xrandr --query output to figure out the name of the HDMI output in use, since the RaspberryPi 4 has two of them. It would be nice if xrandr could join the ranks of tools with a machine parsable output, like for example lsblk is doing.

Scanning the media directory

The next step is finding what to play. We scan the media directory looking at file mimetypes, to avoid having to hardcode all the possible file extension that image and video files can have. Then we group media by type, and pick the group with the most recent files:

    def find_presentation(self, path):
        """
        Find the presentation to play from a given media directory
        """
        if not os.path.isdir(path):
            return None
        pdf = PDFPresentation()
        videos = VideoPresentation()
        images = ImagePresentation()
        odp = ODPPresentation()
        all_players = [pdf, videos, images, odp]

        for fn in os.listdir(path):
            abspath = os.path.abspath(os.path.join(path, fn))
            base, ext = os.path.splitext(fn)
            mimetype = mimetypes.types_map.get(ext)
            if mimetype is None:
                log.info("%s: mime type unknown", fn)
                continue
            else:
                log.info("%s: mime type %s", fn, mimetype)
            if mimetype == "application/pdf":
                pdf.add(abspath)
            elif mimetype.startswith("image/"):
                images.add(abspath)
            elif mimetype.startswith("video/"):
                videos.add(abspath)
            elif mimetype == "application/vnd.oasis.opendocument.presentation":
                odp.add(abspath)

        player = max(all_players, key=lambda x: x.mtime)
        if not player:
            return None
        return player
Caffeinate

We don't want power management to kick in and turn our signage screens black, so we are running all the media players under caffeine.

We automated with with a simple subprocess.run wrapper. Note the -- to prevent caffeinate from choking on the options passed to the actual media players:

def run(cmd: List[str], check: bool = True, **kw) -> subprocess.CompletedProcess:
    """
    Logging wrapper to subprocess.run.

    Also, default check to True.
    """
    log.info("Run %s", " ".join(shlex.quote(x) for x in cmd))
    return subprocess.run(cmd, check=check, **kw)


class Presentation:
    """
    Base class for all presentation types
    """
    def run_player(self, cmd, **kw):
        """
        Run a media player command line, performing other common actions if
        needed
        """
        # Run things under caffeinate
        # See also: https://stackoverflow.com/questions/10885337/inhibit-screensaver-with-python
        cmd = ["caffeinate", "--"] + cmd
        run(cmd, **kw)
Showing PDFs

okular seems to be the only PDF reader in Debian that can be convinced to do looping non interactive full screen presentations, with only a bit of tampering with its configuration files:

class PDFPresentation(SingleFileMixin, Presentation):
    def run(self):
        log.info("%s: PDF presentation", self.fname)

        confdir = os.path.expanduser("~/.config")
        os.makedirs(confdir, exist_ok=True)

        # TODO: configure slide advance time

        # Configure okular
        with open(os.path.expanduser(os.path.join(confdir, "okularpartrc")), "wt") as fd:
            print("[Core Presentation]", file=fd)
            print("SlidesAdvance=true", file=fd)
            print("SlidesAdvanceTime=2", file=fd)
            print("SlidesLoop=true", file=fd)
            print("[Dlg Presentation]", file=fd)
            print("SlidesShowProgress=false", file=fd)
            # print("SlidesTransition=GlitterRight", file=fd)

    # Silence a too-helpful first-time-run informational message
        with open(os.path.expanduser(os.path.join(confdir, "okular.kmessagebox")), "wt") as fd:
            print("[General]", file=fd)
            print("presentationInfo=4", file=fd)

        # Remove state of previous okular runs, so presentations begin at the
        # beginning
        docdata = os.path.expanduser("~/.local/share/okular/docdata/")
        if os.path.isdir(docdata):
            shutil.rmtree(docdata)

        self.run_player(["okular", "--presentation", "--", self.fname])

I was surprised at how looping a PDF presentation doesn't seem to be a well supported use case in PDF viewers. If it's somewhat painful to do it in okular, it's downright impossible to do it with evince: try evince --fullscreen --presentation: slides won't advance, and it still shows a toolbar!

Showing images
class ImagePresentation(FileGroupMixin, Presentation):
    def run(self):
        self.files.sort()
        log.info("Image presentation of %d images", len(self.files))
        with tempfile.NamedTemporaryFile("wt") as tf:
            for fname in self.files:
                print(fname, file=tf)
            tf.flush()

            # TODO: adjust slide advance time
        self.run_player(["feh", "--filelist", tf.name, "--fullscreen",
                         "--hide-pointer", "--slideshow-delay", "1.5"])

feh does everything needed and more. It seems to support our use case explictly, with useful knobs exposed on the command line, clean, straightforward, beautiful!

Showing videos

Most internet posts about playing media on Raspberry Pi, suggest omxplayer. After trying it it looked quite worrysome, as it seemed to fail with any media format not supported in hardware, there did not seem to be a way to ask it whether a file would be in a playable format or not, and one of the failures left the screen in the wrong resolution.

We would like the media player to be able to play the widest possible range of media, hardware accelerated if possible, software if not.

Luckily, it turned out that vlc can use the Raspberry Pi 4 hardware acceleration, and playing a 1920x1080 video full screen on it would consume only 4% of CPU, which is the same that omxplayer was using.

That was very relieving, as vlc can also play a wide range of media, has excellent support for gapless looping, can be invoked without a UI, and can even do playlists of multiple media.

Here is the corresponding player code:

class VideoPresentation(FileGroupMixin, Presentation):
    def run(self):
        self.files.sort()
        log.info("Video presentation of %d videos", len(self.files))
        with tempfile.NamedTemporaryFile("wt", suffix=".vlc") as tf:
            for fname in self.files:
                print(fname, file=tf)
            tf.flush()

            self.run_player(
                    ["cvlc", "--no-audio", "--loop", "--fullscreen",
                        "--video-on-top", "--no-video-title-show", tf.name])
Showing presentations

The code here is quite straightforward, but it took a while to put together that command line:

class ODPPresentation(SingleFileMixin, Presentation):
    def run(self):
        log.info("%s: ODP presentation", self.fname)
        self.run_player(["loimpress", "--nodefault", "--norestore", "--nologo", "--nolockcheck", "--show", self.fname])

I was surprised that I could not find a way to tell Impress to just play a presentation without other things getting in the way. Even like that, there is a moment in which the UI can be seen to come up on the screen before being covered by the full screen presentation.

There is also no way to force a presentation to loop or to advance slides after a given timeout: both features needs to be set in the presentation itself.

People will have to do a test run of their presentations with a checklist before putting them on the player. It would have been nice to have an easy way to guarantee that a presentation wouldn't get stuck on the player.

That is not a requirement for now anyway. If it ever becomes one, I guess we can always write code to check and tweak the .odp presentation file: it's thankfully a well known and open format.

Auditing recommends

So far we installed everything with --no-install-recommends, but it's risky to do so when dealing with packages with many dependencies like vlc, okular, and impress.

Aptitude offers the possibility to audit recommends: Views / Audit Recommendations will show a list of recommended but not installed packages.

That turned out some font packages that it's maybe nice to have, and libreoffice-avmedia-backend-vlc that may come in handy if people decide to play presentations with embedded videos.

Existing presentation software

Our needs for media playing so far have been simple. Should they become more complex, here are some pointers to existing, more featureful projects:

Vincent Bernat: Self-hosted videos with HLS: subtitles

4 December, 2019 - 14:45

In a previous article, I have described a solution to self-host videos while offering a delivery adapted to each user’s bandwith, thanks to HLS and hls.js. Subtitles1 were not part of the game. While they can be declared inside the HLS manifest or embedded into the video, it is easier to include them directly in the <video> element, using the WebVTT format:

<video poster="poster.jpg"
       controls preload="none">
  <source src="index.m3u8"
          type="application/vnd.apple.mpegurl">
  <source src="progressive.mp4"
          type='video/mp4; codecs="avc1.4d401f, mp4a.40.2"'>
  <track src="de.vtt"
         kind="subtitles" srclang="de" label="Deutsch">
  <track src="en.vtt"
         kind="subtitles" srclang="en" label="English">
</video>

Watch the following demonstration, featuring Agent 327: Operation Barbershop, a video created by Blender Animation Studio and currently released under the Creative Commons Attribution No Derivatives 2.0 license:

You may want to jump to 0:12 for the first subtitle. Most browsers should display a widget to toggle subtitles. This works just fine with Chromium but Firefox will not show the menu until the video starts playing, unless you enable preloading. Another annoyance: there is no simple way to specify safe margins for subtitles and they get stuck at the bottom. These two issues seem minor enough to not warrant pulling hundred of kilobytes of JavaScript for a custom player.

  1. Some people may be picky over the difference between closed captions and subtitles. Closed captions are usually targeted at people with hearing impairment and they include non-speech information like sound effects. Subtitles assume the viewer can hear but may not understand the language. ↩︎

Dima Kogan: tee is broken?

4 December, 2019 - 13:31

Just found a highly surprising behavior in a core tool I've used for decades, so clearly I'm making a note here. None of these are surprising:

$ seq 1000 | wc -l

1000


$ seq 1000 | tee /dev/null | wc -l

1000


$ seq 1000 | tee >( true ) | wc -l

1000


$ seq 1000 > >( true ) | wc -l

1000

I.e. I can write 1000 lines into tee, do stuff in one of the children, and the other child get my 1000 lines still. The last one uses multios in zsh for the tee. But check out what happens when I bump up the data size:

$ seq 100000 | wc -l

100000


$ seq 100000 | tee /dev/null | wc -l

100000


$ seq 100000 | tee >( true ) | wc -l

14139


$ seq 100000 > >( true ) | wc -l

1039

Whoa. What the hell? When I stumbled on this I had another, unrelated problem breaking things in this area, which made for a long debugging session. Here're some runs that give a hint of what's going on:

$ seq 100000 | tee >( true ) | wc -c

73728


$ seq 100000 > >( true ) | wc -c

4092


$ seq 100000 | tee >( cat > /dev/null ) | wc -l

100000

Figure it out?

Answer time! After a tee, a single writer parent feeds two reader children. If a child exits before reading all the data, then when the parent tries to feed that dead child, the parent will get a SIGPIPE. And apparently the default behavior of tee in GNU coreutils (and in the zsh multios redirection) is to give up and to stop feeding all the children at that point. So the second child (wc -l in the examples) ends up with incomplete input. No errors are thrown anywhere, and there's no indication at all that any data was truncated. Lots of the data is just silently missing.

The GNU coreutils implementation of tee has an innocuous-looking option:

-p     diagnose errors writing to non pipes

I read the manpage several times, and it's still not obvious to me that -p does anything more than change something about diagnostic printing. But it does: tee -p feeds all the children as much as it can until they're all dead (i.e. what everybody was assuming it was doing the whole time):

$ seq 100000 | tee -p >( true ) | wc -l

100000

There's also pee, specific tee-to-process utility in the Debian moreutils package. This utility can be used here, and it does the reasonable thing by default:

$ seq 100000 | pee true 'wc -l'

100000

So yeah. I'm not the first person to discover this, but I'm certain this was quite surprising to each of us.

Enrico Zini: X autologin

4 December, 2019 - 06:00

This is part of a series of post on the design and technical steps of creating Himblick, a digital signage box based on the Raspberry Pi 4.

When powered on, the Pi units should go straight into X, and start the media player.

X autologin has long been a gripe of mine, and surprisingly hard to do right and reliably, integrating well with PAM, with no greeters or switches to text mode flashing quickly on the screen, and so on.

I once wrote nodm. Then lightdm-autologin-greeter, so I didn't have to maintain a whole display manager for this. Lightdm's autologin used to be insufficient, because when the X session ended, lightdm assumed one wanted to log out and showed the greeter again.

Now I'm very pleased to see that, in 2019, almost 2020, just setting autologin-user in lightdm does the right thing, and I'm very happy to be able to remove some of the kludges from the geological strata of kludges that accreted over the years to work around these kinds of things.

First thing first, do not wait for network to be online during boot. We don't need it in a media player that should be able to work just as well offline:

       # Do not wait for being online to finish boot
       chroot.systemctl_disable("systemd-networkd-wait-online.service", mask=True)

Setting up autologin now really is quite beautifully straightforward:

 - name: "Install X dependencies"
   apt:
      pkg:
      # basic X server
       - xserver-xorg
       - lightdm
      state: present
      update_cache: no
      install_recommends: no

 - name: "Enable lightdm autologin"
   lineinfile:
      path: /etc/lightdm/lightdm.conf
      regexp: ".*autologin-user\\s*=\\s*($|pi)"
      line: "autologin-user = pi"

A nice trick for a media display: X without a mouse cursor accidentally hovering like a fly over things:

 - name: "Disable mouse cursor in X"
   lineinfile:
      path: /etc/lightdm/lightdm.conf
      regexp: "(^|#)xserver-command\\s*=\\s*"
      line: "xserver-command = X -nocursor"

Finally, we need to start the player in the X session.

There are quite a lot of ways to autostart things in X sessions, and the Xsession page in the Debian Wiki has excellent documentation on it.

Since the machine will only be a media player, as a first attempt we decided to try and be the X session, starting the player directly with no desktop environment, no window manager, nothing, ohne alles:

 - name: "Use himblick player as X session"
   copy:
      dest: /home/pi/.xsession
      owner: pi
      group: pi
      mode: '0644'
      content: |
          exec /usr/bin/himblick player

This way, the player can stay there and do its thing without anything popping up in its way, and if it dies, X ends, and lightdm restarts it all.

Norbert Preining: Debian breaking Unison (again)

4 December, 2019 - 04:59

Congratulations – Debian/sid now contains a unison binary that is incompatible with Debian/buster, the stable release. That means, everyone who relies on unison for file synchronization across servers (running buster) and development machines (running sid) is now busted. Trying to use the new binary from sid on buster also doesn’t work, due to GLIBC incompatibility.

For now the only solution I see is using the versions from Debian/buster and hold them, never to be upgraded for the next 2 years. At least that worked for me.

Ben Hutchings: Debian LTS work, November 2019

4 December, 2019 - 01:16

I was assigned 24.5 hours of work by Freexian's Debian LTS initiative and carried over 0.5 hours from October. I worked 21.25 hours this month, so will carry over 3.75 hours to December.

I released Linux 3.16.76, rebased the Debian package onto that, and sent out a request for testing.

I backported the mitigation for TSX Asynchronous Abort (CVE-2019-11135) and reporting of iTLB multihit (CVE-2018-12207) to 3.16 (this work started in October). I applied these and a GPU security fix, uploaded the Debian package and issued DLA-1989-1.

I backported the latest security update for Linux 4.9 from stretch to jessie and issued DLA-1990-1 for that.

I prepared and, after, review, released Linux 3.16.77 and 3.16.78. I rebased the Debian package onto 3.16.78 and sent out a request for testing.

Enrico Zini: ssh setup

3 December, 2019 - 06:00

This is part of a series of post on the design and technical steps of creating Himblick, a digital signage box based on the Raspberry Pi 4.

Time to setup ssh. We want to have admin access to the pi user, and we'd like to have a broader access to a different, locked down user, to use to manage media on the boxes via sftp.

The first step is to mount the exFAT media partition into /srv/media:

---
 - name: "Install exfat drivers"
   apt:
      name: exfat-fuse,exfat-utils
      state: present
      update_cache: no

 - name: "Create /srv directory"
   file:
      path: "/srv"
      state: directory
      owner: root
      group: root
      mode: 0755

 - name: "Create /media mount point"
   file:
      path: "/srv/media"
      state: directory
      owner: pi
      group: pi
      mode: 0755

 - name: "Configure mounting media directory"
   copy:
      src: srv-media.mount
      dest: /etc/systemd/system/srv-media.mount
      owner: root
      group: root
      mode: 0644

Mounting exFAT before Linux kernel 5.4 requires FUSE. Using a mount unit allows us to bring up the mount after FUSE is up, and get it mounted at boot reliably.

We add a round of filesystem checking, too: if people plug the SD into a computer to load media into it, we can't be sure that they unmount it cleanly.

This is srv-media.mount; note that .mount unit names need to match the path of the mount point:

[Unit]
Description=Media Directory (/srv/media)
Before=local-fs.target
After=sys-fs-fuse-connections.mount

[Mount]
What=/dev/disk/by-label/media
Where=/srv/media
Type=exfat
Options=uid=0,gid=1001,fmask=117,dmask=007,rw,noatime,nosuid,nodev
ExecStartPre=-/sbin/fsck.exfat -a /dev/disk/by-label/media

[Install]
WantedBy=local-fs.target

gid 1001 is the media group id, shared by the pi user that runs the media player, and by the media user that does sftp. We make everything the media mount group-writable by the media user so both users can access it.

Next, we prepare a chroot jail for the media user. The root of the jail needs to be writable only by root, so we bind mount the media directory inside it:

 - name: "Create the chroot jail for media: /srv"
   file:
      path: "/srv"
      state: directory
      owner: root
      group: root
      mode: 0755

 - name: "Create the chroot jail for media: /srv/jail"
   file:
      path: "/srv/jail"
      state: directory
      owner: root
      group: root
      mode: 0755

 - name: "Create the chroot jail for media: /srv/jail/media"
   file:
      path: "/srv/jail/media"
      state: directory
      owner: root
      group: media
      mode: 0755

 - name: "Bind mount /srv/media under /srv/jail/media"
   copy:
      src: srv-jail-media.mount
      dest: /etc/systemd/system/srv-jail-media.mount
      owner: root
      group: root
      mode: 0644

This is the srv-jail-media.mount mount unit, neatly ordered to start after /srv/media is mounted:

[Unit]
Description=Media Directory in sftp jail (/srv/jail/media)
Before=local-fs.target
After=srv-media.target

[Mount]
What=/srv/media
Where=/srv/jail/media
Type=none
Options=bind

[Install]
WantedBy=local-fs.target

Finally, the ssh configuration:

---
 - name: "Disable ssh password authentication"
   lineinfile:
      path: /etc/ssh/sshd_config
      regexp: '\bPasswordAuthentication\b'
      line: 'PasswordAuthentication no'

 - name: "Install ssh admin access key"
   authorized_key:
      user: pi
      state: present
      key: "{{SSH_AUTHORIZED_KEY}}"
   when: SSH_AUTHORIZED_KEY is defined

 - name: "Install ssh media access key"
   authorized_key:
      user: media
      state: present
      key: "{{SSH_MEDIA_PUBLIC_KEY}}"
   when: SSH_MEDIA_PUBLIC_KEY is defined

 - name: "Install media access key for the pi user"
   copy:
      dest: "/home/pi/.ssh/id_media"
      content: "{{SSH_MEDIA_PRIVATE_KEY}}"
      owner: pi
      group: pi
      mode: 0600
   when: SSH_MEDIA_PRIVATE_KEY is defined

 - name: "Configure internal sftp, so ssh does not need binaries inside the jail"
   lineinfile:
      path: /etc/ssh/sshd_config
      regexp: ".*Subsystem\\s+sftp"
      line: "Subsystem sftp internal-sftp"

 - name: "Configure sftp chroot jail for user media"
   blockinfile:
      path: /etc/ssh/sshd_config
      block: |
         Match User media
              ChrootDirectory /srv/jail
              AllowTcpForwarding no
              X11Forwarding no
              ForceCommand internal-sftp

Don't forget to enable the media units:

       # Enable the /srv/media mount point, which ansible, as we run it
       # now, is unable to do
       chroot.systemctl_enable("srv-media.mount")
       chroot.systemctl_enable("srv-jail-media.mount")

Jelmer Vernooij: The Debian Janitor

3 December, 2019 - 04:45

There are a lot of small changes that can be made to the Debian archive to increase the overall quality. Many of these changes are small and have just minor benefits if they are applied to just a single package. Lintian encourages maintainers to fix these problems by pointing out the common ones.

Most of these issues are often trivially fixable; they are in general an inefficient use of human time, and it takes a lot of effort to keep up with. This is something that can clearly be automated.

Several tools (e.g. onovy's mass tool, and the lintian-brush tool that I've been working on) go a step further and (for a subset of the issues reported by lintian) fix the problems for you, where they can. Lintian-brush can currently fix most instances of close to 100 lintian tags.

Thanks to the Vcs-* fields set by many packages and the APIs provided by hosting platforms like Salsa, it is now possible to proactively attempt to fix these issues.

The Debian Janitor is a tool that will run lintian-brush across the entire archive, and propose fixes to lintian issues via pull request.

Objectives

The aim of Debian Janitor is to take some drudge work away from Debian maintainers where possible, so they can spend their time on more important packaging work. Its purpose is to make automated changes quick and easy to apply, with minimal overhead for package maintainers. It is essentially a bit of infrastructure to run lintian-brush across all of the archive.

The actions of the bot are restricted to a limited set of problems for which obviously correct actions can be taken. It is not meant to automate all packaging, or even to cover automating all instances of the issues it knows about.

The bot is designed to be conservative and delight with consistently correct fixes instead of proposing possibly incorrect fixes and hoping for the best. Considerable effort has been made to avoid the janitor creating pull requests with incorrect changes, as these take valuable time away from maintainers, the package doesn't actually improve (since the merge request is rejected) and it makes it likelier that future pull requests from the Debian Janitor bot are ignored or rejected.

In short: The janitor is meant to propose correct changes if it can, and back off otherwise.

Design

The Janitor finds package sources in version control systems from the Vcs*- control field in Debian source packages. If the packaging branch is hosted on a hosting platform that the Janitor has a presence on, it will attempt to run lintian-brush on the packaging branch and (if there are any changes made) build the package and propose a merge. It is based on silver-platter and currently has support for:

The Janitor is driven from the lintian and vcswatch tables in UDD. It queries for packages that are affected by any of the lintian tags that lintian-brush has a fixer script for. This way it can limit the number of repositories it has to process.

Ensuring quality

There are a couple of things I am doing to make sure that the Debian Janitor delights rather than annoys.

High quality changes

Lintian-brush has end-to-end tests for its fixers.

In order to make sure that merge requests are useful and high-value, the bot will only propose changes from lintian-brush that:

  • successfully build in a chroot and pass autopkgtest and piuparts;
  • are not completely trivial - e.g. only stripping whitespace

Changes for a package will also be reviewed by a human before they make it into a pull request.

One open pull request per package

If the bot created a pull request previously, it will attempt to update the current request by adding new commits (and updating the pull request description). It will remove and fix the branch when the pull request conflicts because of new upstream changes.

In other words, it will only create a single pull request per package and will attempt to keep that pull request up to date.

Gradual rollout

I'm slowly adding interested maintainers to receiving pull requests, before opening it up to the entire archive. This should help catch any widespread issues early.

Providing control

The bot will be upfront about its pull requests and try to avoid overwhelming maintainers with pull requests by:

  • Clearly identifying any merge requests it creates as being made by a bot. This should allow maintainers to prioritize contributions from humans.
  • Limiting the number of open proposals per maintainer. It starts by opening a single merge request and won't open additional merge requests until the first proposal has a response
  • Providing a way to opt out of future merge requests; just a reply on the merge request is sufficient.

Any comments on merge requests will also still be reviewed by a human.

Current state

Debian janitor is running, generating changes and already creating merge requests (albeit under close review). Some examples of merge requests it has created:

Using the janitor

The janitor can process any package that’s maintained in Git and has its Vcs-Git header set correctly (you can use vcswatch to check this).

If you're interested in receiving pull requests early, leave a comment below. Eventually, the janitor should get to all packages, though it may take a while with the current number of source packages in the archive.

By default, salsa does not send notifications when a new merge request for one of the repositories you're a maintainer for is created. Make sure you have notifications enabled in your Salsa profile, by ticking "New Merge Requests" for the packages you care about.

You can also see the number of open merge requests for a package repository on tracker - it's the ! followed by a number in the pull request column.

It is also possible to download the diff for a particular package (if it's been generated) ahead of the janitor publishing it:

$ curl https://janitor.debian.net/api/lintian-fixes/pkg/PACKAGE/diff

E.g. for i3-wm, look at https://janitor.debian.net/api/lintian-fixes/pkg/i3-wm/diff.

Future Plans

The current set of supported hosting platforms covers the bulk of packages in Debian that is maintained in a VCS. The only other 100+ package platform that's unsupported is dgit. If you have suggestions on how best to submit git changes to dgit repositories (BTS bugs with patches? or would that be too much overhead?), let me know.

The next platform that is currently missing is bitbucket, but there are only about 15 packages in unstable hosted there.

At the moment, lintian-brush can fix close to 100 lintian tags. It would be great to add fixers for more common issues.

The janitor should probably be more tightly integrated with other pieces of Debian infrastructure, e.g. Jenkins for running jobs or linked to from the tracker or lintian.debian.org.

More information

See the FAQ on the homepage.

If you have any concerns about these roll-out plans, have other ideas or questions, please let me know in the comments.

Julien Danjou: Properly managing your .gitignore file

2 December, 2019 - 20:09

There's not a single month where I don't have to explain this. I thought it'd be a good opportunity to write about this .gitignore file so everyone is up to date on this magic file.

The purpose of .gitignore

The .gitignore file is meant to be a list of files that Git should not track. It resides at the root directory of your repository. It can be a list of file path relative to the repository, or a list of wildcard. The file format and location is fully documented in Git documentation.

For example, this is a valid content for a .gitignore file:

foo
bar/*

When you're using Git commands such as git add, all the files matching what's listed in .gitignore are ignored. That makes sure you don't commit a file that should not be there by mistake. In the example above, any file in the bar directory or any file named foo will be completely ignored by all Git commands.

Awesome!

What's the problem with it?

Soon, developers realize that their directory is cluttered with temporary files. It might be from their build system, their editors or some test files they wrote.

So what do they do? They add those files to .gitignore for their project. You end up with a .gitignore file that contains entries like:

*~
.vscode
*.DS_Store
.idea

With that, you're sure to ignore backup files from vim, folders from MacOS and temporary files from Visual Studio code, etc.

Don't do this. Not everybody uses your editor or favorite pet tool, and nobody cares. The repository you're working in is shared with a lot of others developers. Sending pull requests to just add this kind of entry to ignore files generated by your pet editor is wrong and annoying.

Wait, how do I ignore my editor files then?

If you read through Git documentation, the answer lies there: Git has a global ignore file that works for EVERY repository on your system. No need to hack each repository. By default, it's in ~/.config/git/ignore. Here's mine:

.#*
*.swp
.DS_Store
.dir-locals.el
.dir-locals-2.el

That's enough to ignore my editors and OS files in all my repositories so I don't git add wrong files by mistake. You can tweak this global file location by changing by tweaking core.excludesFile in your Git configuration.

So what should I put in .gitignore?

You should put in .gitignore all files and patterns that are generated by the build system of your project, or any file that it might output while running.

For example, for a Python project, it's common to have this:

*.pyc
__pycache__

With this, it makes sure that nobody is committing compiled Python files.

Thanks for reading through this. I hope you'll write better .gitignore files in the future. 🤞

Enrico Zini: Multicast DNS

2 December, 2019 - 06:00

This is part of a series of post on the design and technical steps of creating Himblick, a digital signage box based on the Raspberry Pi 4.

We now have little box that can connect to the wifi, and we can easily give it a name like "front_entrance" or "room3_ceiling".

It would be nice if people who need to manage media on the display, could be able to connect via sftp or a web interface to a box given its name. Thankfully it's 2019 (almost 2020) and this doesn't require tinkering with interfacing the DHCP server with the local DNS anymore.

Enter mDNS:

 - name: "Make machine ssh-discoverable"
   copy:
      remote_src: yes
      src: /usr/share/doc/avahi-daemon/examples/ssh.service
      dest: /etc/avahi/services/

That's it: copy /usr/share/doc/avahi-daemon/examples/ssh.service to /etc/avahi/services/, and from any other machine in the local network you can ssh into the box simply by name.

This is taken care by Avahi.

There are a couple of packages of interest:

  • libnss-mdns that allows any unix program to transparently resolve names via mDNS
  • avahi-utils, that provides command line tools useful for debugging.

I particularly found it useful to use avahi-resolve to test direct and reverse resolution to see if everything was working, and everything generally was, provided, of course, that the Pi's monitor was set to the correct resolution.

More can be done with this. We're quite interesting in reopening this chapter in the future, in order to make the devices able to discover each other. One interesting idea would be to make it so that displays could be grouped, and that updating media in any one display could automatically propagate to all the other displays in the group.

Shirish Agarwal: No country for women – rehashed

2 December, 2019 - 04:20

This is not the first time I am writing about it, sadly I had written it twice before in the preceding months. But the scenario instead of changing for the better is changing for the worse. Sharing a screenshot shared by somebody today on social-media.

Indian Express page inside page screenshot Page 9, Decamber 1, 2019 Reportage of Rape only.

The most recent gruesome addition to the list was that of a 26 year-old veterinarian. From the various news reports it was a pre-planned rape and murder of the woman. What was startling to me when I read the report that it seems people still either don’t know or police officers who should know and inform people do not inform people of their rights such as zero FIR . Maybe some RTI activist can find out how many people have used zero FIR or not and make efforts to popularize the concept and its utility. If people are courageous enough, they can even live-shoot video when asking a police officer or constable to file a zero FIR . The other part is I do hope the police do investigate and do the DNA matching so they have conclusive proof other than the statements which the rapists may later claim as being under duress. The views of police is almost Victorian in nature irrespective of whether it’s an officer or a lower hawaldar/constable on the beat.

While there have been many attempts to try and figure out solutions, it somehow always falls short. There was the ‘Occupy streets by women’ movement in 2017 and so many small and major attempts but almost all have fallen short. Simply put, neither the society, nor the police or the judiciary seems to have any answers. I probably had shared about India’s Daughter , a documentary which shared an insight into the rapist’s mindset. Except for the juvenile, all the other rapists who had confessed to the crime shared their opinion on why the rape happened and most of it was victim-blaming or victim-shaming. The producer Leslee Udwin at the time she made and released the documentary shared that she herself had been raped. The statements made by various BJP leaders as well as some from the opposition were so in favor of the men who had committed the rape were such that they had to finally ban the documentary as lot of hypocricy was being unveiled.

From 2012, till date there probably would be a dozen or more apps. which are designed to help somehow never work. There have been few women cab services, driven by women for women but that too has not scaled. The couple I know of are kaola kabs based out of Bangalore and women cabs in Delhi . The problem with Kaola is that they don’t have night services when most of the rapes occur and when they are needed, about womencabs maybe somebody who may have used the services could perhaps share their their experience. There have been many experiments, right from 2012 onwards but more often than not, most have been more or less a failure. The problems stem from many systemic and societal problems. The problem is not just of women but men too. Places which are unsafe for women are unsafe for all vulnerable groups, whether it is children, the elderly and anybody else who is not a bully. This simple understanding somehow fails to connect or be understood by people. We had this happen everywhere, whether in Bollywood or whether they are women cops or the judiciary. The situation for women is the same everywhere. And before somebody accuses me of India-bashing even though I live here. We know the same happens in all countries, surprisingly more in European countries where I expected human rights to be a savior but it seems they have their own fights to win. I guess if you are a predator you can find your prety no matter what your race or orientation is as shared by a victim recently.

Sex Education

While we can’t take the excuse of Sex education, the short video does illustrate the views of most people. If they don’t want to be responsible about children’s understanding and influence about sex especially in the nuclear families most are, then who will ?

https://www.invidio.us/watch?v=feQocm2TSBc

I had to look back at how I had helped Dr. Shome in her book where we had gone back and forth on her book on sex education titled ‘Let’s talk about guys, girls and sex’ which is there on Amazon, Kindle and elsewhere. My discussions as well as discussions with others, she had shared when last we met that she had material for at least another 2 books if she decided to go that way. The problem even with that such books do not make for casual reading and nor would either most parents or schools will attempt to have discussions around a book like that.

Mobile application for women safety

There are attempts to make a mobile application but I don’t see how any mobile application can help when it’s society’s issues at large. I am guessing funding is also going to be a big issue as well as transportation of people from place X to Y itself carries a cost other than costs and expenses incurred at various places.

Ancedote

I can share an anecdotal tale I heard from a gentleman couple of years ago, and like many stories or incidents this one also didn’t make the newspaper. A few years ago, I had gone to a friend’s place who was staying at Baner, at a somewhat secluded place. We used to meet for talking about FOSS, sharing foss, eating, drinking and sleeping at his place at night and next morning having a chai before using public transport to come back. We used to do this once a month or once two months till he shifted to Bangalore. Anyways, one night after partying I didn’t sleep well and went outside to the bus station quite early, around 0600 hrs or something like that. There used to be quite a few cycle rickshaw shops ( similar to food carts that you see in the west but somewhat more rundown.) The thing about Baner and the area I was in, this was factory area so most factories have 3-4 shifts so most of these food carts (called thelas in the vernacular) make brisk business. Unlike other times, instead of 3-4 food carts, I saw only one and also a bit of eerie silence. After a bit of talking, cajoling the owner, came to know that a naked woman had come and created a scene and other thela walas had run away. Whether she was mentally unstable or had she been raped was not known. Even the police jeep for some reason didn’t want to take her as they suspected they may blame the policeman. Even he had run away but as he was old he couldn’t tow the food cart so he simply ran away because in his twilight he didn’t want to get into any such complication.

The reason I shared the above is, in such a scenario how are men supposed to act and react, I don’t know. Whether it’s just fight or flight or some other behavioral patterns that should be incultated by men in such a scenario ?

Anthillhacks

FWIW, there is anthillhacks happening in Bangalore. For people interested in hacking or doing something like above, this may be a good chance to connect with like-minded people and have some sort of rough prototype out or at least hash out much more face-to-face rather than using the web. I had been a part of this just as have been part of similar things in the past and my experience has been swell. Unfortunately travel budget is an issue this year otherwise for sure would have been part of it. Hopefully people enjoy and also blog about their experiences while staying there.

Thorsten Alteholz: Debian-Med Bug Squashing

2 December, 2019 - 02:08

As it is again this time of the year, I would also like to draw some attention to the Debian Med Advent Calendar. Like the past years, the Debian Med team starts a bug squashing event from the December 1st to 24th. Every bug that is closed will be registered in the calendar. So instead of taking something from the calendar, this special one will be filled and at Christmas hopefully every Debian Med related bug is closed. Don’t hestitate, start to squash :-).

The announcement on the mailing list can be found here.

Sam Hartman: Voting Guide for Debian Init Systems GR

2 December, 2019 - 00:22

There are a lot of options on the ballot for the Init Systems GR.
There have been hundreds of messages on debian-vote, and more scattered
across debian-devel, debian-project, and bugs. Reading all that is no
easy task. so I would like to summarize my understanding of the options
and what they would mean. I've tried to remove as much bias as I can,
but this is still Sam's personal opinion.



I'm focused entirely on the effects of the proposals. Several options
(D, F, and G) spend significant time outlining principles. That is
something I'm ignoring here in this post, although I acknowledge it is
really important to some.



Areas of Agreement

One of the big surprises for me in this discussion is things that are
true of all the ballot options so far. We are agreed that programs
designed to use systemd features that only work with systemd are welcome
in Debian. That's true even if there is no way for these programs to
work without systemd. Under Proposal D, this is a bug, but not a
release-critical bug.



Init Diversity Options

Several options focus on encouraging or requiring packages to support
init systems others than systemd when that is possible. These include
Proposal
E
, Proposal F,
and Proposal
A
. Under proposal E, it is a release critical bug if a program does
not work when something other than systemd is pid 1 unless that program
is designed explicitly to work with systemd and no support for running
without systemd is available. Lack of an init script alone is not
sufficient to count as designed to work exclusively with systemd.



So, under this proposal, a maintainer must integrate support for running
without systemd if it is available. They are responsible for going out
and finding this support. If the support is as simple as writing an
init script, the maintainer has an RC bug until they write the init
script. If the support is more complex, the maintainer is not
responsible for writing it. Proposal A is the same as Proposal E,
except that the bug is not release-critical. I'll go into Proposal A in
more detail after discussing Proposal D.



Proposal D is similar to Proposal E. My interpretation is that Proposal D places
somewhat less of a burden on maintainers to go out and find existing
non-systemd support. My interpretation is that the bug becomes RC when
someone contributes that support. (Or if the support is present in the
upstream but turned off in the package). Proposal D requires that
non-systemd support not have a substantial effect on systemd
installations. So where as Proposal E uses the designed exclusively for
systemd criteria, Proposal D uses the no substantial effect on systemd
systems criteria to determine whether working only with systemd is
acceptable. The discussions seemed to imply that if Gnome uses systemd
features in excess of what technologies like elogind can handle, it is
likely to meet both criteria.



Proposal D goes into a lot more detail than Proposal E. Proposal E
would likely be a long-term block on using systemd facilities like
sysusers. Proposal D specifically proposes a mechanism whereby such
facilities can be documented in policy. This mechanism is only
available if it is likely that developers of non-systemd (including
non-Linux) systems will implement the facility. After a six-to-twelve
month transition time, the facility can be used even on non-systemd
systems. So, sufficiently low effort in the non-systemd community that
it is unreasonable to expect a facility could be implemented could still
permanently block adoption of such facilities. Proposal D is definitely
about a long-term commitment to non-systemd systems even if the effort
in the non-systemd community is not as high as we'd like to adopt new
features elsewhere.



Proposal D also includes a number of guidelines for proper behavior
around these emotionally charged issues.



The only difference between Proposal E and Proposal A is the severity of
the bug when non-systemd support is not in a package. In Proposal A,
this bug is important: potentially sufficient for a stable update, but
not release critical. As a practical matter, Proposal A allows the
non-systemd community to contribute (and NMU) patches for non-systemd
support. However, it does not place an obligation on maintainers to
write this support themselves. Proposal A would permit systemd
facilities like sysusers to be used, although doing so might be a bug.
In the specific case of sysusers, someone could justify NMUing a patch
to use adduser in a maintainer script. Unlike Proposal D, Proposal A
places the burden of keeping up with systemd facilities fully on the
non-systemd community. Proposal A does not have Proposal D's
requirement that it be reasonable to expect that the non-systemd
community can implement the facility.



Systemd Options

There are two systemd options: Proposal F
and Proposal
B
. Proposal F replaces a previous Proposal C. As far as I can tell
Proposal F and C do the same thing, but Proposal F has some text
describing principles. As I said in the introduction, I'm not
discussing that here.



Under Proposal F, systemd is the only officially supported option.
Other init systems may be explored at wishlist priority. Systemd
facilities such as sysusers are encouraged, and we will use our usual
mechanisms to plan transitions from Debian facilities where appropriate.



Proposal B does not explicitly say that alternate init system work can
only be wishlist. Under Proposal B, I think it would be reasonable to
file some bugs at normal severity, but it would also be reasonable for a
maintainer to downgrade them. I don't consider that a significant
difference.



The big difference is that Proposal B commits us as a project to
reviewing integrations of technologies that are alternatives to
systemd facilities. The current example is elogind. But things like
a non-systemd implementation of sysusers, tmpfiles.d, etc, would also
qualify. The rationale is that sometimes alternatives like that touch
on core infrastructure, and even if other maintainers are doing the
work, gatekeeper review is needed. Under Proposal B, the alternate technologies would be available, but whether to use them in a specific package would be up to the maintainer. I've discussed in another, more opinionated blog post why this might be a good idea.



Proposal G

As best I can tell Proposal G is just a set of principles. so, in the context of the analysis I have set out to perform here, I think there is nothing to say.

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้