Planet Debian

Subscribe to Planet Debian feed
Planet Debian - http://planet.debian.org/
Updated: 1 hour 57 min ago

Thorsten Alteholz: My Debian Activities in May 2017

2 June, 2017 - 04:01

FTP assistant

This month I only marked 39 packages for accept and rejected 5 packages.

Debian LTS

This was my thirty-fifth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.

This month my all in all workload has been 27.25h. During that time I did LTS uploads or prepared one for Jessie/Sid:

  • [DLA 934-1] radicale security update for one CVE
  • [DLA 942-1] jbig2dec security update for three CVEs
  • [DLA 947-1] icu security update for two CVEs
  • [DLA 950-1] libtasn1-3 security update for one CVE
  • [DSA 3861-1] libtasn1-6 security update for one CVE
  • [DLA 956-1] libsndfile security update for four CVE
  • [DLA 957-1] bind9 security update for three CVEs
  • [DLA 962-1] tnef security update for one CVE
  • [DSA 3869-1] tnef security update for one CVE

For [DLA 948-1] dropbear and [DLA 958-1] libonig I only did the LTS bookkeeping and sent the DLA.

The icu upload would not have been possible without the help of Roberto.

I also tried to work on jasper, libxml2, libytnef and swftools but unfortunately all upstreams did not finish their respective patches this month, so maybe there will be an upload in June.

Other stuff

Again this has been a busy LTS month, so I only uploaded a new version of smstools, which closed most of its bugs and adopted adopted ptpd as DOPOM.

As a prerequisite of wview I uploaded radlib. Unfortunately I could not do anything for wview, so work on this has to be postponed. Another new package is te923con, which I hope is able to read data from my weather station.

Last but no least I fixed an RC bug in alljoyn-services-1504.

Steve Kemp: So I accidentally wrote a linux security module

2 June, 2017 - 04:00

Tonight I read this weeks LWN quotes-page a little later than usual because I was busy at work for most of the day. Anyway as always LWNs content was awesome, and this particular list lead to an interesting discussion about a new Linux-Security-Module (LSM).

One of the later replies in the thread was particularly interesting as it said:

Suggestion:

Create an security module that looks for the attribute

    security.WHITELISTED

on things being executed/mmapped and denys it if the attribute
isn't present. Create a program (whitelistd) that reads
/etc/whitelist.conf and scans the system to ensure that only
things on the list have the attribute.

So I figured that was a simple idea, and it didn't seem too hard even for myself as a non-kernel non-developer. There are several linux security modules included in the kernel-releases, beneath the top-level security/ directory, so I assumed I could copy & paste code around them to get something working.

Brief attr primer

If you're not familiar with the attr tool it's pretty simple. You can assign values to arbitrary labels on files. The only annoying thing is you have to use extra-flags to commands like rsync, tar, cp, etc, to preserve the damn things.

Set three attributes on the file named moi:

$ touch moi
$ attr -s forename -V "Steve"      moi
$ attr -s surname  -V "Kemp"       moi
$ attr -s name     -V "Steve Kemp" moi

Now list the attributes present:

$ attr -l moi
Attribute "name" has a 10 byte value for moi
Attribute "forename" has a 5 byte value for moi
Attribute "surname" has a 4 byte value for moi

And retrieve one?

$ attr -q -g name moi
Steve Kemp
LSM Skeleton

My initial starting point was to create "steve_lsm.c", with the following contents:

 #include <linux/lsm_hooks.h>

 /*
  * Log things for the moment.
  */
 static int steve_bprm_check_security(struct linux_binprm *bprm)
 {
     printk(KERN_INFO "STEVE LSM check of %s\n", bprm->filename);
     return 0;
 }

 /*
  * Only check exec().
  */
 static struct security_hook_list steve_hooks[] = {
     LSM_HOOK_INIT(bprm_check_security, steve_bprm_check_security),
 };

 /*
  * Somebody set us up the bomb.
  */
 static void __init steve_init(void)
 {
     security_add_hooks(steve_hooks, ARRAY_SIZE(steve_hooks), "steve");
     printk(KERN_INFO "STEVE LSM initialized\n");
 }

With that in place I had to modify the various KBuild files beneath security/ to make sure this could be selected as an LSM, and add in a Makefile.

With the boiler-plate done though, and the host machine rebooted it was simple to test things out.

Obviously the first step, post-boot, is to make sure that the module is active, which can be done in two ways, looking at the output of dmesg, and explicitly listing the modules available:

 ~# dmesg | grep STEVE | head -n2
 STEVE LSM initialized
 STEVE LSM check of /init

 $ echo $(cat /sys/kernel/security/lsm )
 capability,steve
Making the LSM functional

The next step was to make the module do more than mere logging. In short this is what we want:

  • If a binary is invoked by root - allow it.
  • If a binary is invoked by anybody else look for an extended attribute called security.WHITELISTED.
    • If this is present we allow the execution.
    • If this is missing we deny the execution.

NOTE we don't care what the content of the extended attribute is, we just care whether it exists or not.

Reading the extended attribute is thankfully pretty simple, using the __vfs_getxattr function. All in all our module becomes this: this:

  #include <linux/xattr.h>
  #include <linux/binfmts.h>
  #include <linux/lsm_hooks.h>
  #include <linux/sysctl.h>
  #include <linux/ptrace.h>
  #include <linux/prctl.h>
  #include <linux/ratelimit.h>
  #include <linux/workqueue.h>
  #include <linux/string_helpers.h>
  #include <linux/task_work.h>
  #include <linux/sched.h>
  #include <linux/spinlock.h>
  #include <linux/lsm_hooks.h>


  /*
   * Perform a check of a program execution/map.
   *
   * Return 0 if it should be allowed, -EPERM on block.
   */
  static int steve_bprm_check_security(struct linux_binprm *bprm)
  {
         // The current task & the UID it is running as.
         const struct task_struct *task = current;
         kuid_t uid = task->cred->uid;

         // The target we're checking
         struct dentry *dentry = bprm->file->f_path.dentry;
         struct inode *inode = d_backing_inode(dentry);

         // The size of the label-value (if any).
         int size = 0;

         // Root can access everything.
         if ( uid.val == 0 )
            return 0;

         size = __vfs_getxattr(dentry, inode, "user.whitelisted", NULL, 0);
         if ( size >= 0 )
         {
             printk(KERN_INFO "STEVE LSM check of %s resulted in %d bytes from 'user.whitelisted' - permitting access for UID %d\n", bprm->filename, size, uid.val );
             return 0;
         }

         printk(KERN_INFO "STEVE LSM check of %s denying access for UID %d [ERRO:%d] \n", bprm->filename, uid.val, size );
         return -EPERM;
  }

  /*
   * The hooks we wish to be installed.
   */
  static struct security_hook_list steve_hooks[] = {
       LSM_HOOK_INIT(bprm_check_security, steve_bprm_check_security),
  };

  /*
   * Initialize our module.
   */
  void __init steve_add_hooks(void)
  {
       /* register ourselves with the security framework */
       security_add_hooks(steve_hooks, ARRAY_SIZE(steve_hooks), "steve");

       printk(KERN_INFO "STEVE LSM initialized\n");
  }

Once again we reboot with this new kernel, and we test that the module is active. After the basic testing, as before, we can now test real functionality. By default no binaries will have the attribute we look for present - so we'd expect ALL commands to fail, unless executed by root. Let us test that:

~# su - nobody -s /bin/sh
No directory, logging in with HOME=/
Cannot execute /bin/sh: Operation not permitted

That looks like it worked. Let us allow users to run /bin/sh:

 ~# attr -s whitelisted -V 1 /bin/sh

Unfortunately that fails, because symlinks are weird, but repeating the test with /bin/dash works as expected:

 ~# su - nobody -s /bin/dash
 No directory, logging in with HOME=/
 Cannot execute /bin/dash: Operation not permitted

 ~# attr -s whitelisted -V 1 /bin/dash
 ~# attr -s whitelisted -V 1 /usr/bin/id

 ~# su - nobody -s /bin/dash
 No directory, logging in with HOME=/
 $ id
 uid=65534(nobody) gid=65534(nogroup) groups=65534(nogroup)
 $ uptime
 -su: 2: uptime: Operation not permitted

And our logging shows the useful results as we'd expect:

  STEVE LSM check of /usr/bin/id resulted in 1 bytes from 'user.WHITELISTED' - permitting access for UID 65534
  STEVE LSM check of /usr/bin/uptime denying access for UID 65534 [ERRO:-95]
Surprises

If you were paying careful attention you'll see that we changed what we did part-way through this guide.

  • The initial suggestion said to look for security.WHITELISTED.
  • But in the kernel module I look for user.whitelisted.
    • And when setting the attribute I only set whitelisted.

Not sure what is going on there, but it was very confusing. It appears to be the case that when you set an attribute a secret user. prefix is added to the name.

Could be worth some research by somebody with more time on their hands than I have.

Anyway I don't expect this is a terribly useful module, but it was my first, and I think it should be pretty stable. Feedback on my code certainly welcome!

Ben Hutchings: Debian LTS work, May 2017

1 June, 2017 - 22:24

I was assigned 15 hours of work by Freexian's Debian LTS initiative and carried over 3 hours. I worked 13 hours and will carry over 5 hours.

I prepared a security update for sudo and issued DLA-970-1. I backported several security fixes for the Linux kernel, but have not yet uploaded a new version. I also continued catching up with the backlog of fixes for the Linux 3.2 longterm stable branch.

Rapha&#235;l Hertzog: My Free Software Activities in May 2017

1 June, 2017 - 21:59

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donors (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it’s one of the best ways to find volunteers to work with me on projects that matter to me.

Debian LTS

I was allocated 12 hours to work on security updates for Debian 7 Wheezy. During this time I did the following:

  • Reviewed CVE against ntp (and mark them as no-dsa)
  • Prepared and released DLA-944-1 for openvpn 2.2.1-8+deb7u4 fixing CVE-2017-7479.
  • Prepared and released DLA-946-1 for nss 3.26-1+debu7u3 fixing two CVE.
  • Worked on bin/lts-cve-triage.py to no longer hide CVE on unsupported packages so that we actually add the proper status marker on each CVE.
  • Handled CVE triage for a whole week.
Misc Debian work

Debian Handbook. I started to work on the update of the Debian Administrator’s Handbook for Debian 9 Stretch. As part of this, I noticed a regression in dblatex and filed this issue both in the upstream tracker and in Debian and got that issue fixed in sid and stretch (sponsored the actual upload, filed the unblock request). I also stumbled on a regression in dia which was due to an incorrect Debian-specific patch that I reverted with a QA upload since the package is currently orphaned.

Django. On request of Scott Kitterman, I uploaded a new security release of Django 1.8 to jessie-backports but that upload got rejected because stretch no longer has Django 1.8 and I’m not allowed to maintain that branch in that repository. Ensued a long and heated discussion that has no clear resolution yet. It seems likely that some solution will be found for Django (the 1.8.18 that was rejected was accepted as a one-time update already, and our plans for the future make it clear that we would have like to have an LTS version in stretch in the first place) but the backports maintainers are not willing to change the policy to accomodate for other similar needs in the future.

The discussion has been complicated by the intervention of Neil Williams who brought up an upgrade problem of lava-server (#847277). Instead of fixing the root-problem in Django (#863267), or adding a work-around in lava-server’s code, he asserted that upgrading first to Django 1.8 from jessie-backports was the only upgrade path for lava-server.

Thanks

See you next month for a new summary of my activities.

No comment | Liked this article? Click here. | My blog is Flattr-enabled.

Markus Koschany: My Free Software Activities in May 2017

1 June, 2017 - 18:59

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you’re interested in  Java, Games and LTS topics, this might be interesting for you.

Debian Games Bug fixes
  • ufoai (RC #861979) : Robert Hackbauer discovered that ufoai crashed as soon as one player joined a game. I had never seen this crash before and the bug probably surfaced due to the recompilation last month but fortunately I could get a meaningful backtrace and upstream was able to provide a patch within 24 hours.
  • pixbros (RC #861612): The RC bug in pixbros was a rather sad story as it was claimed that the level design (the design, not the artwork) was non-free. The bug submitter argued that there was a high degree of resemblance with one of the original games (pixbros is an amalgamation of several games) thus making pixbros unsuitable for Debian and non-free. This was the kind of bug report which you will probably only see in the games section. We have many games in the archive that try to be a clone and free software alternative of a more popular commercial and non-free game. Not only are they sometimes developed in a completely different programming language, their new artwork, even the gameplay can differ heavily. In this case the level design was just two-dimensional horizontal and vertical bars on which the protagonists perform their actions and in my opinion this is not what we call non-free in Debian. The sad part was, because it happens rather frequently, that random people think they are copyright and trademark experts although they are neither lawyers nor the original copyright holder and, to underline the layman status, often end their sentences with the ominous IANAL. I would like to see that people focus more on improving the games section by packaging new games and maintaining existing ones instead of playing hobby lawyer and creating issues where issues don’t exist.
  • doomsday (RC #847651, #863536): Doomsday failed to start but Bernhard Übelacker provided a patch to fix #847651. If nobody beats me to it, I will also upload the fix for #863536 very soon.
New upstream release
  • I mentioned torcs in my last report which I adopted earlier. It turned out that some car models were non-free (not like pixbros but this time for real) because the license didn’t allow modification. I repacked the tarball and released version 1.3.3+dfsg2-1 for Stretch (#861959) and pushed the latest upstream release to experimental. I also discovered that torcs would FTBFS due to a bug in debhelper and reported it. (#861852)
  • I packaged new upstream versions of freeorion, springlobby, freeciv and bzflag.
Debian Java
  • Elana Hashman is working on the clojure eco-system in Debian. I reviewed and sponsored libbultitude-clojure for her.
  • I fixed a follow-up bug in pdfsam (#855324) and documented in a NEWS file that the config file in $HOME must be updated by hand when a user upgraded from Jessie to Stretch.
  • I uploaded a new upstream release of activemq to experimental and fixed a minor changelog typo bug.
Debian LTS

This was my fifteenth month as a paid contributor and I have been paid to work 27,25 hours on Debian LTS, a project started by Raphaël Hertzog. In that time I did the following:

  • From 1. May until 7. May I was in charge of our LTS frontdesk. I triaged security issues in rxvt, imagemagick, libtirpc, rpcbind, binutils, wordpress, eglibc and tiff3.
  • I prepared a security update for wordpress fixing 6 CVE. I contacted the maintainer, Craig Small, for feedback and intend to release the update soon.
  • I have been working on smb4k which is currently affected by a root privilege vulnerability. Backporting the fix is non-trivial and requires more testing.
  • DLA-933-1. Issued a security update for roundcube fixing 1 CVE.
  • DLA-936-1. Issued a security update for libtirpc fixing 1 CVE.
  • DLA-937-1. Issued a security update for rpcbind fixing 1 CVE.
  • DLA-938-1. Issued a security update for git fixing 1 CVE.
  • DLA-924-1. Issued a regression update for tomcat7 and fixed bug #861872.
  • DLA-941-1. Issued a security update for squirrelmail fixing 1 CVE.
  • DLA-945-1. Issued a security update for mysql-connector-java fixing 3 CVE.
  • DLA-953-1. Issued a security update for graphicsmagick fixing 1 CVE.
  • DLA-968-1. Issued a security update for libpodofo fixing 10 CVE.
  • DLA-969-1. Issued a security update for tiff fixing 2 CVE.
Misc
  • Nikolaus Rath discovered that adding files to a tar archive with xarchiver would actually delete the existing archive (#862593). The issue occured when the archive name contained shell meta characters which were improperly escaped. While I was trying to find the root cause for this issue Chris Lamb provided an alternative solution to fix this problem.

Thanks for reading and see you next time.

Russ Allbery: Review: Migration

1 June, 2017 - 10:03

Review: Migration, by Julie E. Czerneda

Series: Species Imperative #2 Publisher: DAW Copyright: 2005 ISBN: 0-7564-0260-3 Format: Hardcover Pages: 453

Migration is the second book of the Species Imperative, and this is the old-fashioned type of trilogy that you very much want to read in order. Start with Survival. There is a (slightly awkward) recap of the previous book at the start, though, if it's been a bit since you read it.

In my review of Survival, I praised Czerneda's ability to capture the feel of academic research and the sense of real scientists doing science. I thought I went out on a bit of a limb, not being a scientist myself (just someone who worked at a university for decades), but Czerneda was still holding back. I'm now completely convinced: whatever else this series is, and it contains a lot of politics and world-building and fascinating (if very human-like) aliens, it's some of the best science fiction about practicing scientists I've ever read.

I cannot express how much I adore the fact that the center of this book is not space combat, not daring adventure across alien landscapes, but getting a bunch of really smart experts in their field together in a room with good equipment and good computers to chase an intellectual problem from their own individual perspectives. And if Mac is perhaps a bit *too* good at quickly overcoming interpersonal conflict and suspicion, I'll forgive that for the deft sense of politics. Mac's success may be a bit unrealistic, but the direction and thrust of her tactics are spot-on. This is how interactions between smart and curious people often work, at least if they're sufficiently motivated to put aside pettier political infighting. This is also how the dynamics of emergency war rooms work: if you can give people a focus and divide up the work, the results can be amazing.

The second best part of the book is Oversight. The first book opened with the latest round of Mac's ongoing war with Charles Mudge III, the oversight board of the neighboring wilderness trust. He shows up again at the start of this book, acting completely consistent to his stubborn idealism shown in Survival, and then develops into one of the best characters in the book. Unexpected allies is one of the tropes I love most in fiction in general, but this one resonates so deeply with the way grudging respect and familiar patterns, even patterns of argument, work on people. Czerneda had me grinning. It's just perfectly in line with Mac's character, her single-minded focus on work that tended to miss a few points of human connection, and the sort of deepening respect that builds up even between adversaries when they know deep inside that they are following different interpretations of the same principles.

I'm going to be rather sketchy on the plot, since Migration follows closely on from Survival and is concerned almost entirely with the aftermath of the climactic events at the end of that book. But as you can tell, this is more of Mac, and she's not managed to separate herself from Dhryn problems or from the Ministry of Extra-Solar Affairs. She does, however, get rather far away from Norcoast for a while, an interlude in the wild northern Canadian wilderness that once again proves Czerneda to be the type of writer who can make the quotidian as engrossing as alien dramatics. She's also suffering from nightmares, anxiety, and a lot of circular thinking, making this one of the series that shows the realistic toll of dramatic events on human psychology.

There was a bit of a nascent love story in Survival; there's a lot more of that here. It's the one bit of the book that I have mixed feelings about, since it feels a touch unnecessary to me, and therefore a bit intrusive. It also involves a fair bit of love at, well, not first sight but surprisingly fast, which is something I know intellectually that other people think happens, but which always undermines my suspension of disbelief. That said, Czerneda gives Mac a clear tendency in how she forms emotional attachments and sticks with it throughout this series to date, which I do like, and she keeps the romance consistent with that. It thankfully does not get too much in the way of the plot, although I could have done with just a few fewer determined proclamations that the characters won't let love get in the way of doing what they need to do.

That quibble aside, this is fantastic stuff that avoids most of the cliches of this sort of story of alien politics and possible war. The focus is firmly on analysis and understanding rather than guns and action, the portrayal of scientists, analysis, and problem-solving is spot on, the aliens are delightfully different (and different from each other within the same alien species, which is important depth), and Mac is a fantastic protagonist. She's vulnerable, wounded, and out of her depth, but she knows how to map new situations to her areas of competence and how to admit when she doesn't know something, and her effectiveness is well-grounded and believable. Oh, and there are some amazing descriptions of the Canadian wilderness that almost make me want to find a secluded cabin without Internet access. (At least if it had all of the convenient technology that Mac's future Earth has.)

It's a rare middle book of a trilogy that's better than the first, but this one is. Much better. And I already liked the first book. Highly recommended; I think this is one of Czerneda's best.

Followed by Regeneration.

Rating: 9 out of 10

Paul Wise: FLOSS Activities May 2017

1 June, 2017 - 07:44
Changes Issues Review Administration
  • Debian: discuss mail bounces with a hoster, check perms of LE results, add 1 user to a group, re-sent some TLS cert expiry mail, clean up mail bounce flood, approve some debian.net TLS certs, do the samhain dance thrice, end 1 samhain mail flood, diagnose/fix LDAP update issue, relay DebConf cert expiry mails, reboot 2 non-responsive VM, merged patches for debian.org-sources.debian.org meta-package,
  • Debian mentors: lintian/security updates & reboot
  • Debian wiki: delete stray tmp file, whitelist 14 email addresses, disable 1 accounts with bouncing email, ping 3 persons with bouncing email
  • Debian website: update/push index/CD/distrib
  • Debian QA: deploy my changes, disable some removed suites in qadb
  • Debian PTS: strip whitespace from existing pages, invalidate sigs so pages get a rebuild
  • Debian derivatives census: deploy changes
  • Openmoko: security updates & reboots.
Communication
  • Invite Purism (on IRC), XBian (also on IRC), DuZeru to the Debian derivatives census
  • Respond to the shutdown of Parsix
  • Report BlankOn fileserver and Huayra webserver issues
  • Organise a transition of Ubuntu/Endless Debian derivatives census maintainers
  • Advocate against Debian having a monopoly on hardware certification
  • Advocate working with existing merchandise vendors
  • Start a discussion about Debian membership in other organisations
  • Advocate for HPE to join the LVFS & support fwupd
Sponsors

All work was done on a volunteer basis.

Enrico Zini: Today I Learnt

1 June, 2017 - 02:00
Build a system that can install GRUB2 on UEFI and on legacy systems

grub-efi-amd64 and grub-pc are not coinstallable. It turns out however that they do not contain GRUB, but the machinery to keep GRUB configuration up to date on the current system. If I want to be able to install GRUB on other systems, I can use the -bin packages:

apt install grub-common grub2-common grub-efi-amd64-bin grub-pc-bin

That gave me a grub-install command that worked on both kinds of systems.

GRUB configuration on a UEFI system

An old GRUB configuration on a UEFI system gave me this:

error: no suitable mode found
Booting blind

which boots on a blank screen until the kernel reinitialises the video hardware.

The Arch Linux Wiki has excellent documentation for this case, and here's the resulting UEFI GRUB snippet:

insmod efi_gop
insmod efi_uga
insmod font
if loadfont ${prefix}/fonts/unicode.pf2
then
    insmod gfxterm
    set gfxmode=auto
    set gfxpayload=keep
    terminal_output gfxterm
fi

# Follow with the usual GRUB menu entries…
Use an unsigned local APT repository for testing/development purposes

I found out today that one can have options in square brackets in sources.list:

# In /etc/apt/sources.list.d/local-devel.list
deb [trusted=yes] http://localhost:1234/debian jessie main
Booting Jessie Debian Live with a kernel from jessie-backports

This requires working around #844749 and 844749.

In hooks/9000-fix-bugs.chroot I ended up having this:

# Workaround per https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844749
if ! grep -q ^nls_ascii /etc/initramfs-tools/modules
then
        echo "nls_ascii" >> /etc/initramfs-tools/modules
fi

# Workaround per https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844749
if ! grep -q ^overlay /etc/initramfs-tools/modules
then
        echo "overlay" >> /etc/initramfs-tools/modules
fi
Using a custom kernel in Jessie Debian Live

How do I have live-build pick a custom kernel package instead of the default one?

  1. lb config --linux-packages linux-image-$SOMETHING
  2. Use equivs to build a linux-image-$SOMETHING-$ARCH package that depends on the kernel that you built.

Enrico Zini: Debian Jessie Live on UEFI part 2

1 June, 2017 - 01:29

A refinement on my previous attempt.

This is how to configure a Jessie live-build environment to boot on UEFI systems, and get a USB key image that works:

# Build a FAT image instead of an ISO image...
lb config -b hdd

# ...and work around #773833
echo "/usr/lib/syslinux/mbr/*.bin /usr/lib/syslinux/" > hooks/9000-fix-773833.chroot

# Get EFI Shell from https://svn.code.sf.net/p/edk2/code/trunk/edk2/ShellBinPkg/UefiShell/X64/
curl -o binary/efi/boot/Bootx64.efi https://svn.code.sf.net/p/edk2/code/trunk/edk2/ShellBinPkg/UefiShell/X64/Shell.efi

# Configure the EFI shell to boot the live setup
echo 'live\vmlinuz initrd=live\initrd.img append boot=live components' > binary/startup.nsh

Rationale: UEFI understants FAT filesystems, and would run EFI binaries placed under efi/boot.

For a hard drive, it only considers a FAT filesystem on a GPT partition marked with a special UUID, so that it doesn't get confused with other FAT filesystems that are on disk.

For a USB key, it seems that most hardware will happily look for efi/boot even if the partition table is the old MBR kind.

live-build can build a FAT image for USB keys, losing the ability to boot on CDROMs and DVDs. Since I don't need that ability, I can use -b hdd to get the live system packaged inside a container that UEFI hardware can understand (FAT).

At that point, enabling UEFI boot on a Live Debian Jessie is just a matter of adding an efi/boot/Bootx64.efi binary that is able to load the kernel and initrd in memory, and blow life into them.

Chris Lamb: Free software activities in May 2017

1 June, 2017 - 01:25

Here is my monthly update covering what I have been doing in the free software world (previous month):

  • Wrote and released installation-birthday. Installing this package will celebrate the anniversary of installing your system by sending you an email via cron(8).
  • Fixed an issue in the Django web development framework where you couldn't run the testsuite against a read-only copy of the source code. This was found by the Debian Continuous Integration service. (#26755)
  • Provided a pull request for the "wammu" mobile phone manager to ensure the build is reproducible. (#49)
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

(I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.)

This month I:

I also made the following changes to our tooling:

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • Don't fail when run under perversely-recursive input files. (#780761).

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Move from verbose_print to nonquiet_print so we print when normalising a file. This is so we can start to target the removal of strip-nondeterminism itself.
  • Only print log messages by default if the file was actually modified. (#863033)
  • Update package long descriptions to clarify that the tool itself is a temporary workaround. (#862029)


Debian

My activities as the current Debian Project Leader are covered in my "Bits from the DPL" email to the debian-devel-announce list.

However, I:

  • Represented Debian at the OSCAL 2017 in Tirana, Albania.
  • Attended the Reproducible Builds hackathon in Hamburg, Germany. (Report)
  • Finally, I attended Debian SunCamp 2017 in Lloret de Mar in Catalonia, Spain.

Patches contributed
  • xarchiver: Adding files to .tar.xz deletes existing content. (#862593)
  • screen-message: Please invert the default colours. (#862056)
  • fontconfig: fc-cache returns with exit code 0 on 256 errors. (#863427)
  • quadrapassel: Segfaults when unpausing a paused finished game. (#863106)
  • camping: Broken symlink. (#861040)
  • dns-root-data: Does not build if /bin/sh is Bash. (#862252)
  • dh-python: bit.ly link doesn't work anymore. (#863074)
Debian LTS

This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, adding links to upstream patches, etc.
  • Issued DLA 930-1 fixing a remote application crash vulnerability in libxstream-java, a Java library to serialize objects to XML and back again
  • Issued DLA 935-1 correcting a local denial of service vulnerability in lxterminal, the terminal emulator for the LXDE desktop environment.
  • Issued DLA 940-1 to remedy an issue in sane-backends which allowed remote attackers to obtain sensitive memory information via a crafted SANE_NET_CONTROL_OPTION packet.
  • Issued DLA 943-1 for the deluge bittorrent client to fix a directory traversal attack vulnerability in the web user interface.
  • Issued DLA 949-1 fixing an integer signedness error in the miniupnpc UPnP client that could allow remote attackers to cause a denial of service attack.
  • Issued DLA 959-1 for the libical calendaring library. A use-after-free vulnerability could allow remote attackers could cause a denial of service and possibly read heap memory via a specially crafted .ICS file.
Uploads
  • redis (3:3.2.9-1) — New upstream release.
  • python-django:
    • 1:1.11.1-1 — New upstream minor release.
    • 1:1.11.1-2 & 1:1.11.1-3 — Add missing Build-Depends on libgdal-dev due to new GIS tests.
  • docbook-to-man:
    • 1:2.0.0-36 — Adopt package. Apply a patch to prevent undefined behaviour caused by a memcpy(3) parameter overlap. (#842635, #858389)
    • 1:2.0.0-37 — Install manpages using debian/docbook-to-man.manpages over manual calls.
  • installation-birthday — Initial upload and misc. subsequent fixes.
  • bfs:
    • 1.0-3 — Fix FTBFS on hurd-i386. (#861569)
    • 1.0.1-1 — New upstream release & correct debian/watch file.

I also made the following non-maintainer uploads (NMUs):

  • ca-certificates (20161130+nmu1) — Remove StartCom and WoSign certificates as they are now untrusted by the major browser vendors. (#858539)
  • sane-backends (1.0.25-4.1) — Correct missing error handler in (generated) prerm script. (#862334)
  • seqan2 (2.3.1+dfsg-3.1) — Fix broken /usr/bin/splazers symlink on 32-bit architectures. (#863669)
  • jackeq (0.5.9-2.1) — Fix a segmentation fault caused by passing a truncated pointer instead of a GtkType. (#863416)
  • kluppe (0.6.20-1.1) — Fix segmentation fault at startup. (#863421)
  • coyim (0.3.7-2.1) — Skip tests that require internet access to avoid FTBFS. (#863414)
  • pavuk (0.9.35-6.1) — Fix segmentation fault when opening "Limitations" window. (#863492)
  • porg (2:0.10-1.1) — Fix broken LD_PRELOAD path. (#863495)
  • timemachine (0.3.3-2.1) — Fix two segmentation faults caused by truncated pointers. (#863420)
Debian bugs filed
  • acct: Docs incorrectly installed to "accounting.html" directory. (#862180)
  • git-hub: Does not work with 2FA-enabled accounts. (#863265)
  • libwibble: Homepage and Vcs-Darcs fields are outdated. (#861673)


I additionally filed 2 bugs for packages that access the internet during build against flower and r-bioc-gviz.



I also filed 6 FTBFS bugs against cronutils, isoquery, libgnupg-interface-perl, maven-plugin-tools, node-dateformat, password-store & simple-tpm-pk11.

FTP Team

As a Debian FTP assistant I ACCEPTed 105 packages: boinc-app-eah-brp, debug-me, e-mem, etcd, fdroidcl, firejail, gcc-6-cross-ports, gcc-7-cross-ports, gcc-defaults, gl2ps, gnome-software, gnupg2, golang-github-dlclark-regexp2, golang-github-dop251-goja, golang-github-nebulouslabs-fastrand, golang-github-pkg-profile, haskell-call-stack, haskell-foundation, haskell-nanospec, haskell-parallel-tree-search, haskell-posix-pty, haskell-protobuf, htmlmin, iannix, libarchive-cpio-perl, libexternalsortinginjava-java, libgetdata, libpll, libtgvoip, mariadb-10.3, maven-resolver, mysql-transitional, network-manager, node-async-each, node-aws-sign2, node-bcrypt-pbkdf, node-browserify-rsa, node-builtin-status-codes, node-caseless, node-chokidar, node-concat-with-sourcemaps, node-console-control-strings, node-create-ecdh, node-create-hash, node-create-hmac, node-cryptiles, node-dot, node-ecc-jsbn, node-elliptic, node-evp-bytestokey, node-extsprintf, node-getpass, node-gulp-coffee, node-har-schema, node-har-validator, node-hawk, node-jsprim, node-memory-fs, node-pbkdf2, node-performance-now, node-set-immediate-shim, node-sinon-chai, node-source-list-map, node-stream-array, node-string-decoder, node-stringstream, node-verror, node-vinyl-sourcemaps-apply, node-vm-browserify, node-webpack-sources, node-wide-align, odil, onionshare, opensvc, otb, perl, petsc4py, pglogical, postgresql-10, psortb, purl, pymodbus, pymssql, python-decouple, python-django-rules, python-glob2, python-ncclient, python-parse-type, python-prctl, python-sparse, quoin-clojure, quorum, r-bioc-genomeinfodbdata, radlib, reprounzip, rustc, sbt-test-interface, slepc4py, slick-greeter, sparse, te923con, trabucco, traildb, typescript-types & writegood-mode.

I additionally filed 6 RC bugs against packages that had incomplete debian/copyright files against: libgetdata, odil, opensvc, python-ncclient, radlib and reprounzip.

Craig Small: The sudo tty bug and procps

31 May, 2017 - 19:30

There have been recent reports of a security bug in sudo (CVE-2017-1000367) where you can fool sudo into thinking what controlling terminal it is running on to bypass its security checks.  One of the first things I thought of was, is procps vulnerable to the same bug? Sure, it wouldn’t be a security bypass, but it would be a normal sort of bug. A lot of programs  in procps have a concept of a controlling terminal, or the TTY field for either viewing or filtering, could they be fooled into thinking the process had a different controlling terminal?

Was I going to be in the same pickle as the sudo maintainers? The meat between the stat parsing sandwich? Can I find any more puns related somehow to the XKCD comic?

TLDR: No.

How to find the tty

Most ways of finding what the controlling terminal for a process is on are very similar. The file /proc//stat is a one-liner pseudo file that the kernel creates on access that has information about the particular process.  A typical file would look like:

20209 (bash) S 14762 20209 20209 34822 20209 4194304 32181 4846307 625 1602 66 3
0 16265 4547 20 0 1 0 139245105 25202688 1349 18446744073709551615 4194304 52421
32 140737059557984 0 0 0 0 3670020 1266777851 1 0 0 17 1 0 0 280 0 0 7341384 738
8228 39092224 140737059564618 140737059564628 140737059564628 140737059569646 0

The first field is the PID, the second the process name (which may be different than the command line, but that’s another story), then skip along to field #7 which in this case is 34822. Also notice the process name is in brackets; that is important.

So 34822, how do we figure out what device this is? The number is the major and minor device numbers of the controlling terminal. 38422 in hex is 8806, the device has a major number of 88h or 136 and a minor number of 06. Most programs just scan the usual device directories until they find a match (which is basically how procps does it).

Device 136,6 is /dev/pts/6

crw--w---- 1 user tty 136, 6 May 29 16:20 /dev/pts/6
$ ps -o tty,cmd 20209
TT CMD
pts/6 /bin/bash
The Bug

The process of taking the raw stat file and having a bunch of useful fields is called parsing. The bug in sudo was due to how they parsed the file.  The stat file is a space-delimited file. The program scanned the file, character by character, until it came across the 6th space. The problem is, you can put spaces in your command and fool sudo.

Once you know that, you can make sudo think the program is running on any (or at least a different) controlling terminal. The bug reporters then used some clever symlinking and race techniques to then get root.

What about procps?

The parsing of the current (as of writing) procps on the stat file is found in proc/readproc.c within the function stat2proc().  However, it is not just a simple sscanf or something that runs along the line looking for spaces. To find the command, the program does the following:

 S = strchr(S, '(') + 1;
 tmp = strrchr(S, ')');
 num = tmp - S;
 if(unlikely(num >= sizeof P->cmd)) num = sizeof P->cmd - 1;
 memcpy(P->cmd, S, num);
 P->cmd[num] = '\0';
 S = tmp + 2; // skip ") "

The sscanf then comes after we have found the command, using the variable S, to fill in the other fields including the controlling terminal device numbers. procps library looks for the command within brackets. So if your program has spaces in it, it is still found. By using a strrchr (effectively, find the last) you cannot fool it with a bracket in the command either.

So procps is not vulnerable to this sort of trickery.

Incidently, the fix for the sudo bug now uses strrchr for a close bracket, so it solves the problem the same way.  The check for the close bracket appeared in procps 3.1.4 back in 2002, though the stat2proc function was warning about odd named processes before then. As it says in the 2002 change:

Reads /proc/*/stat files, being careful not to trip over processes with names like “:-) 1 2 3 4 5 6”.

That’s something we can all agree on!

Neil McGovern: GNOME ED Update – Week 22

31 May, 2017 - 00:19
Delayed update

Firstly, an apology – I’ve been rather lax about doing these updates. I’ll try and highlight more happenings in the project more frequently in future.

GUADEC Birthday party

In August it will have been 20 years since the GNOME project was founded. To celebrate this occasion, a special party is being organised. All current and former GNOME Foundation members are especially welcome to attend. We’d love it if you could join us! The party is taking place as part of GUADEC 2017. For more details and announcements, follow @guadec.

Date Saturday 29th July 2017 Time 19:00 Location Manchester, United Kingdom Venue Museum of Science and Industry Engagement team

Did you know about the GNOME Engagement Team? This is the team that helps promote GNOME and push for adoption of GNOME. It’s a fantastic way to get involved in the non-technical part of the project, and they’re always looking for more help

PIA affiliate programme

Private Internet Access is a long time supporter of the project, and we’ve recently worked out an affiliate deal with them – If you sign up for their VPN services via https://www.privateinternetaccess.com/pages/buy-vpn/GNOME, then the foundation receives a contribution to help further the project. I’ve personally signed up for PIA’s VPN services, as working remotely I sometimes find myself on public wifi hotspots, and the risks of these are fairly well documented.

Board elections

One of the characteristics of the Foundation is that the board is elected by foundation members. It’s the time of year again where this happens and we have a great list of candidates. Voting is open until 9th June, so I’d encourage any foundation member to read up, and vote accordingly!

Reproducible builds folks: Reproducible Builds: week 109 in Stretch cycle

31 May, 2017 - 00:05

Here's what happened in the Reproducible Builds effort between Sunday May 21 and Saturday May 27 2017:

Past and upcoming events

Bernhard M. Wiedemann gave a short talk on reproducible builds in openSUSE at the openSUSE Conference 2017. Slides and video recordings are available on that page.

Chris Lamb will present at the Hong Kong Open Source Conference 2017 on reproducible builds on June 9th.

Our next IRC meeting has been scheduled for Thursday June 1 at 16:00 UTC with this agenda.

Academia

Justin Cappos continued his work on the reproducible builds paper, with text and suggestions from Ximin Luo integrated.

Toolchain developments

#863470: "ftp.debian.org: security sync must not exclude .buildinfo" - while this bug isn't fixed, you need to make sure not to build jessie updates with stretch's dpkg, or else the upload will be rejected.

Ximin Luo built GCC twice and ran diffoscope on them. Unfortunately the results were 1.7 GB in size and it can't be displayed in a web browser. 99/171 of the .debs are reproducible, though. He's now working on diffoscope (see below) to make it generate output more intelligently for such large size diffs. Here is a summary diff where the recursion depth cut-off was set low, so the size is reasonable and one can still see the outlines of where to look next.

debuerreotype was newly added to Debian unstable. It is a reproducible, snapshot-based Debian rootfs builder.

Patches and bugs filed Reviews of unreproducible packages

29 package reviews have been added, 49 have been updated and 23 have been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Adrian Bunk (10)
  • Chris Lamb (2)
  • James Clarke (1)
diffoscope development

Development continued in git, with commits from:

  • Ximin Luo:
    • Refactor container-related logic to make the code clearer.
    • Various improvements to the progress bar, making it behave more accurately and make it compatible with --debug logging output.
    • Fix --exclude control.tar.gz.
    • When enforcing max-container-depth, show which internal files differ, without showing their details.
    • Add --max-container-depth CLI option.
strip-nondeterminism development

Version 0.034-1 was uploaded to unstable by Chris Lamb. It included previous weeks' contributions from:

  • Chris Lamb
    • Only print log messages by default if the file was actually modified. (Closes: #863033)
  • Bernhard M. Wiedemann
    • zip: make sure we have permissions on extracted file
    • Add function prototypes.
tests.reproducible-builds.org:
  • Alexander Couzens
    • Use Alexander's LEDE git repo to test his mksquashfs patches.
  • Daniel Shahaf
    • Refactored reproducible_remote_scheduler.py to add support for multiple suites in one invocation.
  • Holger Levsen
    • Prevent the two fdroid jobs from running together by using the Build Blocker Plugin.
    • A niceness variation was also added (see #863440) to the Debian tests, but this change was reverted for now, as it was breaking stuff and needs to be readded properly.
    • Some adjustments to the Debian scheduler, still due to the improve performance through the new build services.
  • Mattia Rizzolo
    • Update dsa-check-running-kernel from dsa-nagios (to support kernel 4.x as present in stretch) on all jenkins nodes.
Misc.

This week's edition was written by Ximin Luo, Bernhard M. Wiedemann, Chris Lamb and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Lars Wirzenius: Using a Yubikey 4 for ensafening one's encryption

30 May, 2017 - 20:53
Introduction

I've written before about using a U2F key with PAM. This post continues the theme and explains how to use a smartcard with GnuPG for storing OpenPGP private keys. Specifically, a Yubikey 4 card, because that's what I have, but any good GnuPG compatible card should work. The Yubikey is both a GnuPG compatible smart card, and a U2F card. The Yubikey 4 can handle keys up to 4096 bits. Older Yubikeys can only handle keys up to 2095 bits.

The reason to do this is to make it harder for an attacker to steal your encryption keys.

I will assume you don't already have an OpenPGP key, or are willing to generate a new one. I will also assume you run Debian stretch; some of the desktop environment setup details may differ between Debian versions or between Linux distributions. You will need:

  • A Yubikey 4 (or other GnuPG compatible smartcard).
  • Two USB memory sticks to store master copies of the key you create.
  • Either a lot of patience, or a ChaosKey or something else to generate a lot of entropy for the kernel. Entropy is used by GnuPG to create encryption keys.
Terminology

Some terminology:

  • OpenPGP is a standard of encryption keys.

  • GnuPG (also known as GPG or gpg) is an encryption program that supports OpenPGP.

  • Encryption key or key pair is a secret and a public key for encryption. Key and key pair are often used as synonyms.

  • The secret keys is yours, and only you will have it or be able to you use. The public key is public, anyone can have a copy. The two are linked and you can't create a secret key linked to a public key. (If you can, you will become a famous cryptographer. Like Isaac Newton famous among apple growers.)

  • Master key is the important thing to keep track of. In the encryption world, the master key is what identifies you.

  • A subkey is a secondary key or key pair for encryption, derived from the master key. Subkeys can be created and revoked almost at will, and there can be any number of them. Subkeys are always associated with a single master key.

  • Subkeys can be dedicated for encryption, signing, and authentication. You can have one of each.

  • Encryption is the process of taking a file and making it illegible to everyone except the owner of a secret key. When the public key is used to encrypt, only the secret key can decrypt.

  • Signing is using a secret key to make a separate file that others can verify using the corresponding public key. This means the signer can "prove" they have the file. For example, you might sign emails to prove they came from you.

  • Authentication is the process of proving you're you. Typically, for example, a server will know your public key, and will give a large random number encrypted with your public key. Only you have the corresponding secret key, so only you can send the random number back to the server, and thereby the server knows you're you.

Outline

The process outline is:

  1. Create a new, signing-only master key with GnuPG.

  2. Create three "subkeys", one each for encryption, signing, and authentication. These subkeys are what everyone else uses.

  3. Export copies of the master key pair and the subkey pairs and put them in a safe place.

  4. Put the subkeys on the Yubikey.

  5. GnuPG will automatically use the keys from the card. You have to have the card plugged into a USB port for things to work. If someone steals your laptop, they won't get the private subkeys. Even if they steal your Yubikey, they won't get them (the smartcard is physically designed to prevent that), and can't even use them (because there's PIN codes or passphrases and getting them wrong several times locks up the smartcard).

  6. Use gpg-agent as your SSH agent, and the authentication-only subkey on the Yubikey is used as your ssh key.

Configure GnuPG

The process in more detail:

  • Configure GnuPG with regards to checksum and encryption algorithms. You can use the defaults, but depending on the version of GnuPG you have, they may be weaker than is recommended. These values are from Riseup.net OpenPGP guide, see link at the end.

    Add the following lines to ~/.gnupg/gpg.conf:

      personal-digest-preferences SHA512
      cert-digest-algo SHA512
      default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed
      personal-cipher-preferences TWOFISH CAMELLIA256 AES 3DES
      keyserver pool.sks-keyservers.net
    
Create new keys
  • Create new sign-only master key.
$ gpg --full-generate-key
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
 
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
       = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:43:54 PM EEST
Is this correct? (y/N) y

GnuPG needs to construct a user ID to identify your key.

Real name: Lars Wirzenius
Email address: liw@liw.fi
Comment: test key
You selected this USER-ID:
    "Lars Wirzenius (test key) <liw@liw.fi>>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: key 25FB738D6EE435F7 marked as ultimately trusted
gpg: directory '/home/liw/.gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/home/liw/.gnupg/openpgp-revocs.d/A734C10BF2DF39D19DC0F6C025FB738D6EE435F7.rev'
public and secret key created and signed.

Note that this key cannot be used for encryption. You may want to use
the command "--edit-key" to generate a subkey for this purpose.
pub rsa4096 2017-05-29 [SC] [expires: 2018-05-29]
      A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
      A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
uid Lars Wirzenius (test key) <liw@liw.fi>
  • Note that I set a 1-year expiration for they key. The expiration can be extended at any time (if you have the master secret key), but unless you do, the key won't accidentally live longer than the chosen time.

  • Review the key:

$ gpg --list-secret-keys
/home/liw/.gnupg/pubring.kbx
----------------------------
sec rsa4096 2017-05-29 [SC] [expires: 2018-05-29]
      A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
uid [ultimate] Lars Wirzenius (test key) <liw@liw.fi>
  • You now have the signing-only master key. You should now create three subkeys (keyid is the key identifier shown in the key listing, A734C10BF2DF39D19DC0F6C025FB738D6EE435F7 above). Use the --expert option to be able to add an authentication-only subkey.
$ gpg --edit-key --expert A734C10BF2DF39D19DC0F6C025FB738D6EE435F7z
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

sec rsa4096/25FB738D6EE435F7
     created: 2017-05-29 expires: 2018-05-29 usage: SC
     trust: ultimate validity: ultimate
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> addkey
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
  (10) ECC (sign only)
  (11) ECC (set your own capabilities)
  (12) ECC (encrypt only)
  (13) Existing key
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
       = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:44:52 PM EEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

sec rsa4096/25FB738D6EE435F7
     created: 2017-05-29 expires: 2018-05-29 usage: SC
     trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
      created: 2017-05-29 expires: 2018-05-29 usage: S
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> addkey
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
  (10) ECC (sign only)
  (11) ECC (set your own capabilities)
  (12) ECC (encrypt only)
  (13) Existing key
Your selection? 6
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:45:22 PM EEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

sec rsa4096/25FB738D6EE435F7
     created: 2017-05-29 expires: 2018-05-29 usage: SC
     trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
      created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
      created: 2017-05-29 expires: 2018-05-29 usage: E
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> addkey
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
  (10) ECC (sign only)
  (11) ECC (set your own capabilities)
  (12) ECC (encrypt only)
  (13) Existing key
Your selection? 8

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Sign Encrypt

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? a

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Sign Encrypt Authenticate

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? s

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Encrypt Authenticate

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? e

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Authenticate

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? q
RSA keys may be btween 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
       = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:45:56 PM EEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

sec rsa4096/25FB738D6EE435F7
     created: 2017-05-29 expires: 2018-05-29 usage: SC
     trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
     created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
     created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
     created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> save Export secret keys to files, make a backup
  • You now have a master key and three subkeys. They are hidden in the ~/.gnupg directory. It is time to "export" the secret keys out from there.
$ gpg --export-secret-key --armor keyid > master.key
$ gpg --export-secret-subkeys --armor keyid > subkeys.key
  • You should keep these files safe. You don't want to lose them, and you don't want anyone else to get access to them. I recommend you format two USB memory sticks, format them using full-disk encryption, and copy the exported files to both of them. Then keep them somewhere safe.

    There's ways of making this part more sophisticated, but that's for another time.

  • The next step involves some hoop-jumping. What we want is to have the master secret key NOT on you machine, so we tell GnuPG to remove it. We exported it above, so we won't lose it. However, deleting the master secret key also removes the secret subkeys. But we can import those without importing the master secret key.

$ gpg --delete-secret-key keyid
$ gpg --import subkeys.key
  • Now verify that you have the secret subkeys, but not the master key. There should be one line starting with sec# (note the hash mark, which indicates the key isn't available), and three lines starting with ssb (no hash mark).
$ gpg -K
/home/liw/.gnupg/pubring.kbx
----------------------------
sec rsa4096 2017-05-29 [SC] [expires: 2018-05-29]
      A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
uid [ultimate] Lars Wirzenius (test key) <liw@liw.fi>
ssb rsa4096 2017-05-29 [S] [expires: 2018-05-29]
ssb rsa4096 2017-05-29 [E] [expires: 2018-05-29]
ssb rsa4096 2017-05-29 [A] [expires: 2018-05-29]
Install subkeys on a Yubikey
  • Now insert the Yubikey in a USB slot. We can start transferring the secret subkeys to the Yubikey. If you want, you can set your name and other information, and change PIN codes. There's several types of PIN codes: normal use, unblocking a locked card, and a third PIN code for admin operations. Changing the PIN codes is a good idea, otherwise everyone will just try the default of 123456 (admin 12345678). However, I'm skipping that in the interest of brevity.
$ gpg -card-edit
...
  • Actually move the subkeys to the card. Note that this does a move, not a copy, and the subkeys will be removed from your ~/.gnupg (check with gpg -K).
$ gpg --edit-key liw
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 1

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb* rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> keytocard
Please select where to store the key:
(1) Signature key
(3) Authentication key
Your selection? 1

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb* rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 1

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 2

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb* rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> keytocard
Please select where to store the key:
(2) Encryption key
Your selection? 2

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb* rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 2

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 3

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb* rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> keytocard
Please select where to store the key:
(3) Authentication key
Your selection? 3

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb* rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> save
  • If you want to use several Yubikeys, or have a spare one just in case, repeat the previous four steps (starting from importing subkeys back into ~/.gnupg).

  • You're now done, as far GnuPG use is concerned. Any time you need to sign, encrypt, or decrypt something, GnuPG will look for your subkeys on the Yubikey, and will tell you to insert it in a USB port if it can't find the key.

Use subkey on Yubikey as your SSH key
  • To actually use the authentication-only subkey on the Yubikey for ssh, you need to configure your system to use gpg-agent as the SSH agent. Add the following line to .gnupg/gpg-agent.conf:

     enable-ssh-support
    
  • On a Debian stretch system with GNOME, edit /etc/xdg/autostart/gnome-keyring-ssh.desktop to have the following line, to prevent the GNOME ssh agent from starting up:

     Hidden=true
    
  • Edit /etc/X11/Xsession.options and remove or comment out the line that says use-ssh-agent. This stops a system-started ssh-agent from being started when the desktop start.

  • Create the file ~/.config/autostart/gpg-agent.desktop with the following content:

     [Desktop Entry]
     Type=Application
     Name=gpg-agent
     Comment=gpg-agent
     Exec=/usr/bin/gpg-agent --daemon
     OnlyShowIn=GNOME;Unity;MATE;
     X-GNOME-Autostart-Phase=PreDisplayServer
     X-GNOME-AutoRestart=false
     X-GNOME-Autostart-Notify=true
     X-GNOME-Bugzilla-Bugzilla=GNOME
     X-GNOME-Bugzilla-Product=gnome-keyring
     X-GNOME-Bugzilla-Component=general
     X-GNOME-Bugzilla-Version=3.20.0
    
  • To test, log out, and back in again, run the following in a terminal:

$ ssh-add -l

The output should contain a line that looks like this:

    4096 SHA256:PDCzyQPpd9tiWsELM8LwaLBsMDMm42J8/eEfezNgnVc cardno:000604626953 (RSA)
  • You need to export the authentication-only subkey in the SSH key format. You need this for adding to .ssh/authorized_keys, if nothing else.
$ gpg --export-ssh-key keyid > ssh.pub
  • Happy hacking.
See also

See also the following links. I've used them to learn enough to write the above.

Sean Whitton: Corbyn and May

30 May, 2017 - 15:51

Since arriving back in the UK I’ve found myself appreciating Sheffield, and indeed British life more generally, far more than I expected, and far more than I have on any previous return, during the time I’ve been working and now studying abroad.

On Sunday, John Prescott came to give a speech to those of us campaigning for Labour, before we set to work. A heckler came over and shouted at Prescott: how could he vote for Labour with Corbyn in charge? Prescott did not break his stride, shouting something in response to the man and then returning to his speech, and someone went to the man and said, “he came here to speak to us, please don’t interrupt, come over here and let’s talk about Corbyn.” And the man did. Real democracy on a street corner, where people are able to fully express themselves without watching their words, or being told they’re being uncivil, and without any hint of police or security (note, for those outside the UK reading this post, that John Prescott was the Deputy Prime Minister for 8 years – he arrived in a squat people carrier).

I think that living in the US had made me believe that this kind of engagement with politics was over. Since I value these battles for ideas so highly, it makes me want to leave Arizona sooner rather than later.

In last night’s “Corbyn vs. May”, in which each of the two answered audience questions and were then interviewed by the aggressive Jeremy Paxman – May has refused to engage in a head-to-head debate – we saw Corbyn at his best. I don’t think that there was a clear loser, but there was an opportunity to see that Corbyn is quite capable of oratory. For me, there were two highlights. A small businessman asked Corbyn how he could vote for someone who was raising both corporation tax and the minimum wage. Without showing a grain of disrespect, Corbyn challenged him to reconsider his position on the grounds that we are all better off if everyone is better off. The second highlight was Corbyn’s firm response to Paxman going on and on about why abolishing the monarchy was not in the manifesto, while Corbyn is a known republican: “we’re not going to abolish the monarchy because I’m fighting this election for social justice” (paraphrased). This is the slightly old-fashioned sense of ‘social justice’: truly universal entitlement to health and education, because that is the mark of a civilised nation. What a privilege it is to be able to both campaign and vote for such a man.

I’ve been thinking about the responses we should make to neo-liberals who say that pouring money into health and education for those who can already afford it results in inefficiency and waste, rendering everyone worse off. There are many such people in the Arizona philosophy department.

I do not believe that this economic argument has yet been won by the neo-liberals. A different response, though, is to think about the opportunities for the development of virtue that are lost when we introduce markets. I think that fear is one of the greatest barriers to the development of the virtues. It closes us down. Fundamentally, social justice is about the removal of fear, so that people are able to flourish. The neo-liberals would rather encourage and exploit fear, in all stratas of society (they want themselves to be afraid of being a bit less rich, and respond accordingly).

Keith Packard: DRM-lease-3

30 May, 2017 - 15:41
DRM leasing part three (Vulkan)

With the kernel APIs off for review, and the X RandR bits looking like they're in reasonable shape, I finally found some time to sit down and figure out how I wanted to integrate this into Vulkan.

Avoiding two DRM file descriptors

Given that a DRM lease is represented by a DRM master file descriptor, we want to use that for all of the operations in the driver, including rendering and mode setting. Using the vulkan driver render node and the lease master node together would require passing buffer objects between the kernel contexts using even more file descriptors.

The Mesa Vulkan drivers open the device nodes while enumerating devices, not when they are created. This seems a bit early to me, but it makes sure that the devices being enumerated are actually available for use, and not just present in the system. To replace the render node fd with the lease master fd means hooking into the system early enough that the enumeration code can see the lease fd. And that means creating an instance extension as the instance gets created before devices are enumerated.

The VK_KEITHP_kms_display instance extension

This simple instance extension provides the necessary hooks to get the lease information from the application down into the driver before the DRM node is opened. In the first implementation, I added a function that could be called before the devices were enumerated to save the information in the Vulkan loader. That worked, but required quite a bit of study of the Vulkan loader and its XML description of the full Vulkan API.

Mark Young suggested that a simpler plan would be to chain the information into the VkInstanceCreateInfo pNext field; with no new APIs added to Vulkan, there shouldn't be any need to change the Vulkan loader -- the device driver would advertise the new instance extension and the application could find it.

That would have worked great, except the Vulkan loader 'helpfully' elides all instance extensions it doesn't know about before returning the list to the application. I'd say this was a bug and should be fixed, but for now, I've gone ahead and added the few necessary definitions to the loader to make it work.

In the application, it's a simple matter of searching for this extension, constructing the VkKmsDisplayInfoKEITHP structure, chaining that into the VkInstanceCreateInfo pNext list and passing that in to the vkCreateInstance call.

typedef struct VkKmsDisplayInfoKEITHP {
    VkStructureType         sType;  /* VK_STRUCTURE_TYPE_KMS_DISPLAY_INFO_KEITHP */
    const void*             pNext;
    int                     fd;
    uint32_t                crtc_id;
    uint32_t                *connector_ids;
    int                     connector_count;
    drmModeModeInfoPtr      mode;
} VkKmsDisplayInfoKEITHP;

As you can see, this includes the master file descriptor along with all of the information necessary to set the desired video mode using the specified resources.

The driver just walks the pNext list from the VkInstanceCreateInfo structure looking for any provided VkKmsDisplayInfoKEITHP structure and pulls the data out.

To avoid questions about file descriptor lifetimes, the driver dup's the provided fd. The application is expected to close their copy at a suitable time.

The VK_KHR_display extension

Vulkan already has an API for directly accessing the raw device, including code for exposing video modes and everything. As tempting as it may be to just go do something simpler, there's a lot to be said for using existing APIs.

This extension doesn't provide any direct APIs for acquiring display resources, relying on the VK_EXT_acquire_xlib_display extension for that part. And that takes a VkPhysicalDisplay parameter, which is only available after the device is opened, which is why I created the VK_KEITHP_kms_display extension instead of just using the VK_EXT_acquire_xlib_display extension -- we can't increase the capabilities of the render node opened by the driver, and we don't want to keep two file descriptors around.

With the information provided by the VK_KEITHP_kms_display extension, we can implement all of the VK_KHR_display extension APIs, including enumerating planes and modes and creating the necessary display surface. Of course, there's only one plane and one mode, so some of the implementation is pretty simplistic.

The big piece of work was to create the swap chain structure and associated frame buffers.

A working example

I've taken the 'cube' example from the Vulkan loader and hacked it up to use XCB to construct a DRM lease, the VK_KEITHP_kms_display extension to pass that lease into the Vulkan driver. The existing support for the VK_KHR_display extension "just worked", which was pretty satisfying.

It's a bit of a mess

I'm not satisfied with the mesa code at this point; there's a bunch of code in the radeon driver which should be in the vulkan WSI bits, and the vulkan WSI bits should probably not have the KMS interfaces wired in. I'll ask around and see what other Mesa developers think I should do before restructuring it further; I'll probably have to rewrite it all at least one more time before it's ready to upstream.

Seeing the code

I'll be cleaning the code up a bit before sending it out for review, but it's already visible in my own repositories:

Steinar H. Gunderson: Nageru 1.6.0 released

30 May, 2017 - 15:00

I've released version 1.6.0 of Nageru, my live video mixer, together with dependent libraries Movit 1.5.1 and bmusb 0.7.0.

The primray new feature this time is integration with CasparCG, the dominating open-source broadcast graphics system, which opens up a whole new world of possibilities for intelligent overlay graphics. (Actually, the feature is a bit more generic than that; any FFmpeg file or stream will do as input. Audio isn't supported yet, though.) You can see a simple HTML5 CasparCG setup in the ultimate tournament stream test we did in April, in preparation of a larger event in September; CasparCG generates a stream with alpha, which is then fed into Nageru and used on top of the three camera sources.

Apart from that, there's a new frame analyzer that helps with calibrating your signal chain; there are lots of devices that will happily mess with your signal, and measuring is the first step in counteracting that. (There's also a few input interpretation tweaks that will help most common issues.)

1.6.0 is on its way to Debian experimental, along with its dependencies (stretch will release with Nageru 1.4.2); there are likely to be backports when stretch releases and the backport queue opens up.

Norbert Preining: Debian/TeX Live 2017 is ready

30 May, 2017 - 06:11

TeX Live 2017 is expected to be released next week, and the Debian packages for it are already uploaded to the Debian servers. Time to prepare for release parties and stack up prosecco!

I have uploaded all the packages matching with the planned release of TeX Live 2017 to Debian/experimental, and most of them should be already available there. texlive-extra has to go through NEW processing, though.

There are too many changes and updates to mention since the last release, but a few things might be worth mentioning:

  • several packages have been merged, some are dropped (eg. texlive-htmlxml) and one new package (texlive-plain-generic) has been added
  • luatex got updated to 1.0.4, and is now considered stable
  • updmap and fmtutil now require either -sys or -user
  • tlmgr got a shell mode (interactive/scripting interface) and a new feature to add arbitrary TEXMF trees (conf auxtrees)

Other changes can be found in the svn repository.

Until the processing of the NEW queue is finished, the package are available at (binaries only for amd64, for other archs please use experimental):

deb http://www.preining.info/debian/ tl2017 main
deb-src http://www.preining.info/debian/ tl2017 main

The packages are signed with my usual Debian GPG key.

Enjoy, and let me know if there are any problems!

Steve Kemp: Security is hard ..

30 May, 2017 - 04:00
3D-Printing

I continued to be impressed with local vendors, found on 3dhubs. I've had several more things printed out, including an "internet button", and some card-holders for Settlers of Catan.

The most recent print I had made was a collection of display cases, for holding an OLED display, as well as an ESP8266 device.

Unfortunately at the same time as I was falling in love with the service I discovered a glaring XSS attack against the site itself.

Anybody who viewed my profile page could have arbitrary javascript executed, which in some cases would actually disclose their private details - such as:

  • Their forename & surname.
  • Their email-address.
  • Their telephone number.
  • Their GeoIP details.

Discovering this took minutes, writing it up took an hour, and a month later it is still unfixed.

I've deleted my account.

Enrico Zini: Egg-walking with qemu-nbd and kpartx

30 May, 2017 - 01:36

I wanted to retrieve a file from a VirtualBox VDI image for this blog post.

I followed these instructions and ended up here:

Once having used nbd0, only rebooting the system makes it possible to mount another image ... a little bit unpractical.

What happened was this:

# modprobe nbd
# qemu-nbd -c /dev/nbd0 file.vdi
# kpartx -d /dev/nbd0
# mount /dev/nbd0… EHI! Where's /dev/nbdpp1 ??
# qemu-nbd -d /dev/nbd0
# rmmod nbd
rmmod: ERROR: Module nbd is in use
# kpartx -d /dev/nbd0
read error, sector 0
llseek error
llseek error
llseek error
# rmmod nbd
rmmod: ERROR: Module nbd is in use
# WHAT THE…

It turns out it's really modprobe nbd max_part=16, otherwise max_part defaults to, uhm, zero? really? and kpartx cannot create device mappings because there are not enough (as in, not even a single one) partition devices available.

At this point, however, kpartx did create some mappings connected to, uhm, probably Ancient Beings from beyond spacetime, and because of those the device is in use and cannot be removed, and unmapping doesn't work either because the Ancient Beings from beyond spacetime are keeping the device busy by feeding on it.

I energized the pentacle and tried a desperate ritual of banishment:

# # Reconnect nbd0 to the vdi file to Restore the Balance
# qemu-nbd --verbose -c /dev/nbd0 file.vdi
# # This works now
# kpartx -vd /dev/nbd0
del devmap : nbd0p5
del devmap : nbd0p2
del devmap : nbd0p1
# # This too, the Ancient Beings lie asleep yet again
# modprobe nbd -r

At this point I managed to get my file, almost:

# modprobe nbd max_part=16
# qemu-nbd --verbose -c /dev/nbd0 file.vdi
NBD device /dev/nbd0 is now connected to file.vdi
# kpartx -va /dev/nbd0
add map nbd0p1 (254:12): 0 60260352 linear 43:0 2048
add map nbd0p2 (254:13): 0 2 linear 43:0 60264446
add map nbd0p5 (254:14): 0 2648064 linear 43:0 60264448
# mount /dev/nbd0p1 /mnt
mount: /dev/nbd0p1 is already mounted or /mnt busy
# # WHAT NOW?!
# lsblk
NAME                                       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
…
nbd0                                        43:0    0    30G  0 disk
├─nbd0p1                                    43:1    0  28.8G  0 part
├─nbd0p2                                    43:2    0     1K  0 part
├─nbd0p5                                    43:5    0   1.3G  0 part
├─nbd0p1                                   254:12   0  28.8G  0 part
├─nbd0p2                                   254:13   0     1K  0 part
└─nbd0p5                                   254:14   0   1.3G  0 part
# # WHAAAT?!!
# kpartx -vd /dev/nbd0
del devmap : nbd0p5
del devmap : nbd0p2
del devmap : nbd0p1
# lsblk
NAME                                       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
…
nbd0                                        43:0    0    30G  0 disk
├─nbd0p1                                    43:1    0  28.8G  0 part
├─nbd0p2                                    43:2    0     1K  0 part
└─nbd0p5                                    43:5    0   1.3G  0 part
# mount /dev/nbd0p1 /mnt
# # I got my file, my preciouss file!
# umount /mnt
# kpartx -vd /dev/nbd0
# qemu-nbd -d /dev/nbd0
# rmmod nbd
# # sit in a corner hugging my precious file and sobbing quietly

As can be seen from the multiple exclamation marks, those Ancient Beings from beyond spacetime did manage to have a bite on my sanity after all.

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้