Planet Debian

Subscribe to Planet Debian feed
Planet Debian -
Updated: 1 hour 9 min ago

Sean Whitton: Using Propellor to provision your Debian development laptop

24 November, 2017 - 05:38

sbuild is a tool used by those maintaining packages in Debian, and derived distributions such as Ubuntu. When used correctly, it can catch a lot of categories of bugs before packages are uploaded. It does this by building the package in a clean environment, and then running the package through the Lintian, piuparts, adequate and autopkgtest tools. However, configuring sbuild so that it makes use of all of these tools is cumbersome.

In response to this complexity, I wrote a module for the Propellor configuration management system to prepare a system such that a user can just go ahead and run the sbuild(1) command.

This module is useful on one’s development laptop – if you need to reinstall your OS, you don’t have to look up the instructions for setting up sbuild again. But it’s also useful on throwaway build boxes. I can instruct propellor to provision a new virtual machine to build packages with sbuild, and all the different tools mentioned above will be connected together for me.

I just uploaded Propellor version 5.1.0 to Debian unstable. The version overhauls the API and internals of the Sbuild module to take better advantage of Propellor’s design. I won’t get into those details in this post. What I’d like to do is demonstrate how you can set up sbuild on your own machines, using Propellor.

Getting started with Propellor

apt-get install propellor, and then propellor --init.

As mentioned, at the time of writing you’ll need to install from Debian unstable. For this tutorial you need version 5.1.0 or greater.

You’ll be offered two setups, options A and B. I suggest starting with option B.

If you never use Propellor for anything other than provisioning sbuild, you can stick with option B. If this tutorial makes you want to check out more features of Propellor, you might consider switching to option A and importing your old configuration.

Open ~/.propellor/config.hs. You will see something like this:

-- The hosts propellor knows about.
hosts :: [Host]
hosts =
        [ mybox

-- An example host.
mybox :: Host
mybox = host "" $ props
        & osDebian Unstable X86_64
        & Apt.stdSourcesList
        & Apt.unattendedUpgrades
        & Apt.installed ["etckeeper"]
        & Apt.installed ["ssh"]
        & User.hasSomePassword (User "root")
        & File.dirExists "/var/www"
        & Cron.runPropellor (Cron.Times "30 * * * *")

You’ll want to customise this so that it reflects your computer. My laptop is called iris, so I might replace the above with this:

-- The hosts propellor knows about.
hosts :: [Host]
hosts =
        [ iris

-- My laptop.
iris :: Host
iris = host "" $ props
        & osDebian Testing X86_64

The list of lines beginning with & are the properties of the host iris. Here, I’ve removed all properties except the osDebian property, which informs propellor that iris runs Debian testing and has the amd64 architecture.

The effect of this is that Propellor will not try to change anything about iris. In this tutorial, we are not going to let Propellor configure anything about iris other than setting up sbuild.

(The osDebian property is a pure info property, which means that it tells Propellor information about the host to which other properties might refer, but it doesn’t itself change anything about iris.)

Telling Propellor to configure sbuild

First, add to the import lines at the top of config.hs the lines:

import qualified Propellor.Property.Sbuild as Sbuild
import qualified Propellor.Property.Schroot as Schroot

to enable use of the Sbuild module. Here is the full config for iris, which I’ll go through line-by-line:

-- The hosts propellor knows about.
hosts :: [Host]
hosts =
        [ iris

-- My laptop.
iris :: Host
iris = host "" $ props
        & osDebian Testing X86_64
        & Apt.useLocalCacher
        & sidSchrootBuilt
        & Sbuild.usableBy (User "spwhitton")
        & Schroot.overlaysInTmpfs
        & Cron.runPropellor (Cron.Times "30 * * * *")
        sidSchrootBuilt = Sbuild.built Sbuild.UseCcache $ props
                & osDebian Unstable X86_64
                & Sbuild.update `period` Daily
                & Sbuild.useHostProxy iris
  • Apt.useLocalCacher set up apt-cacher-ng and points apt on iris at the cacher. This is the most efficient way to share a cache of packages between apt on iris, and apt within the sbuild chroot.
  • sidSchrootBuilt builds the sbuild schroot. Unfortunately, we have to use a where clause because of Haskell’s rules about operator precedence. But it’s just a simple substitution: imagine that & Sbuild.built ... replaces & sidSchrootBuilt.
    • osDebian specifies that this is a Debian unstable chroot. You can easily change this, or add another chroot, for building stable backports, etc.
    • Sbuild.UseCcache enables ccache for builds in this chroot. You can replace this with Sbuild.NoCcache when building a package which is broken by ccache, which happens from time-to-time.
    • Sbuild.update updates the chroot once per day.
    • Sbuild.useHostProxy iris causes Propellor to propagate iris’s apt proxy into the chroot, so that apt in the chroot will also use iris’s apt cacher.
  • Sbuild.usableBy adds spwhitton to the right group, so that he is allowed to invoke sbuild(1).
  • Schroot.overlaysInTmpfs configures sbuild to install build dependencies and build packages in tmpfs. You can omit this property on machines with low amounts of memory.
  • Cron.runPropellor sets up a cron job to re-run propellor once per hour. This is needed to ensure that things like Sbuild.update actually happen. It will also normalise sbuild configuration files, replace chroots that you accidently deleted, etc.
Running Propellor to configure your laptop
$ propellor

In this configuration, you don’t need to worry about whether the hostname actually resolves to your laptop. However, it must be possible to ssh root@localhost.

This should be enough that spwhitton can:

$ sbuild -A --run-lintian --run-autopkgtest --run-piuparts foo.dsc
Further configuration

It is easy to add new schroots; for example, for building backports:

        & stretchSchrootBuilt
        stretchSchrootBuilt = Sbuild.built Sbuild.UseCcache $ props
                & osDebian (Stable "stretch") X86_64
                & Sbuild.update `period` Daily
                & Sbuild.useHostProxy iris

You can even use architectures other than X86_64. Propellor knows how to invoke qemu when it needs to do this to build the chroot, though sbuild does not know how to actually use chroots built in this way.

You can also add additional properties to configure your chroot. Perhaps on your LAN you need sbuild to install packages via https, and you already have an apt cacher available. You can replace the apt-cacher-ng configuration like this:

        sidSchrootBuilt = Sbuild.built Sbuild.UseCcache $ props
                & osDebian Unstable X86_64
                & Sbuild.update `period` Daily
                & Apt.mirror "https://foo.mirror/debian/"
                & Apt.installed ["apt-transport-https"]

Thanks to Propellor’s author, Joey Hess, for help navigating Propellor’s type system while performing the overhaul included in version 5.1.0. Also for a conversation at DebConf17 which enabled this work by clearing some misconceptions of mine.

Russ Allbery: Holiday haul

24 November, 2017 - 03:24

Catching up on accumulated book purchases. I'm going to get another burst of reading time over the holidays (and am really looking forward to it).

Alfred Bester — The Stars My Destination (sff)
James Blish — A Case of Conscience (sff)
Leigh Brackett — The Long Tomorrow (sff)
Algis Budrys — Who? (sff)
Frances Hardinge — Fly By Night (sff)
Robert A. Heinlein — Double Star (sff)
N.K. Jemisin — The Obelisk Gate (sff)
N.K. Jemisin — The Stone Sky (sff)
T. Kingfisher — Clockwork Boys (sff)
Ursula K. Le Guin — City of Illusions (sff)
Ursula K. Le Guin — The Complete Orsinia (historical)
Ursula K. Le Guin — The Dispossessed (sff)
Ursula K. Le Guin — Five Ways to Forgiveness (sff)
Ursula K. Le Guin — The Left Hand of Darkness (sff)
Ursula K. Le Guin — Planet of Exile (sff)
Ursula K. Le Guin — Rocannon's World (sff)
Ursula K. Le Guin — The Telling (sff)
Ursula K. Le Guin — The World for Word Is Forest (sff)
Fritz Leiber — The Big Time (sff)
Melina Marchetta — Saving Francesca (mainstream)
Richard Matheson — The Shrinking Man (sff)
Foz Meadows — An Accident of Stars (sff)
Dexter Palmer — Version Control (sff)
Frederick Pohl & C.M. Kornbluth — The Space Merchants (sff)
Adam Rex — True Meaning of Smekday (sff)
John Scalzi — The Dispatcher (sff)
Julia Spencer-Fleming — In the Bleak Midwinter (mystery)
R.E. Stearns — Barbary Station (sff)
Theodore Sturgeon — More Than Human (sff)

I'm listing the individual components except for the Orsinia collection, but the Le Guin are from the Library of America Hainish Novels & Stories two-volume set. I had several of these already, but I have a hard time resisting a high-quality Library of America collection for an author I really like. Now I can donate a bunch of old paperbacks.

Similarly, a whole bunch of the older SF novels are from the Library of America American Science Fiction two-volume set, which I finally bought since I was ordering Library of America sets anyway.

The rest is a pretty random collection of stuff, although several of them are recommendations from Light. I was reading through her old reviews and getting inspired to read (and review) more.

Jonathan Dowland: Concreate and Red Hat JBoss OpenShift image sources

23 November, 2017 - 17:48

Last year I wrote about some tools for working with Docker images. Since then, we've deprecated the dogen tool for our own images and have built a successor called Concreate.

Concreate takes a container image definition described in a YAML document and generates an output image. To do so, it generates an intermediate Dockerfile, along with the scripts and artefacts you references in the YAML file, and by default invokes docker build on the result. However, it can use other builders, such as the OpenShift Build Service, which is what we use for our production images.

Concreate can also manage running tests against the image. As with the Container Testing Framework that I mentioned last time, these tests are defined using the Behave system.

In related news is that we have published the sources for all of our images. You can now go and read the image.yaml file for EAP 7 on OpenShift to give you an example of what a real image using Concreate looks like.

Louis-Philippe Véronneau: DebConf Videoteam sprint report - day 4

23 November, 2017 - 12:00

Day 4 of the videoteam sprint!

Pictures of Our Lives

"Countless" people wrote to me asking for more pictures of our marvelous sprint, especially of our feline friend. How could I resist?

Now that we've got this covered, here's what we did today.

Report tumbleweed

Stefano had to do some office work today and had very little time to hack on the videoteam stuff.

He did get access to the pentabarf XML of DebConf 7 to 13. This should help a great lot generating the video metadata for our archive.

He also played around with YouTube a little. It seems we already got our first copyright strike! Gotta love remixes of The Lion King hidden in DebConf videos.


Ivo left us to work with the Debian Release Team today. Sad!


Andy and Kyle got together today and worked on making a list of the audio hardware we should buy to replace our old kit.

He also finished the video loop we are going to use at the mini-conf.


Kyle has some time today and gave Andy a call to work on our audio gear wishlist. He also worked on designing a flight case for it.

We will eventually submit a budget request to buy said kit.


Nicolas mainly worked on refactoring the ansible module that generates the TLS certificates for our streaming network.


Our ansible roles are now all documented! I'm happy this is all done. Next, I'll try to remove some of the ugly hacks in our ansible repository.

We finished the day by going to the Polish Club. Here's a picture of the team!

Daniel Pocock: VR Hackathon at FIXME, Lausanne (1-3 December 2017)

23 November, 2017 - 02:25

The FIXME hackerspace in Lausanne, Switzerland is preparing a VR Hackathon on the weekend of 1-3 December.

Competitors and visitors are welcome, please register here.

Some of the free software technologies in use include Blender and Mozilla VR.

Louis-Philippe Véronneau: DebConf Videoteam sprint report - day 3

22 November, 2017 - 12:00

Erf, I'm tired and it is late so this report will be short and won't include dank memes or funny cat pictures. Come back tomorrow for that.


Stefano worked all day long on the metadata project and on YouTube uploads. I think the DebConf7 videos have just finished being uploaded, check them out!


Apart from the wonderful lasagna he baked for us, Andy continued working on the scraping scheme, helping tumbleweed.


Nattie has been with us for a few days now, but today she did some great QA work on our metadata scraping of the video archive.


More tests, more bugs! Ivo worked quite a bit on the Opsis board today and it seems everything is ready for the mini-conf. \0/


Nicolas built the streaming network today and wrote some Ansible roles to manage TLS cert creation through Let's Encrypt. He also talked with DSA some more about our long term requirements.


I forgot to mention it yesterday because he could not come to Cambridge, but Wouter has been sprinting remotely, working on the reviewing system. Everything with regards to reviewing should be in place for the mini-conf.

He also generated the intro and outro slides for the videos for us.

KiBi and Julien

KiBi and Julien arrived late in the evening, but were nonetheless of great assistance.

Neither are technically part of the videoteam, but their respective experience with Debian-Installer and general DSA systems helped us a great deal.


I'm about 3/4 done documenting our ansible roles. Once I'm done, I'll try to polish some obvious hacks I've seen while documenting.

Norbert Preining: Kobo firmware 4.6.10075 mega update (KSM, nickel patch, ssh, fonts)

22 November, 2017 - 08:18

A new firmware for the Kobo ebook reader came out and I adjusted the mega update pack to use it. According to the comments in the firmware thread it is working faster than previous releases. The most incredible change though is the update from wpa_supplicant 0.7.1 (around 2010) to 2.7-devel (current). Wow.

For details and warning please consult the previous post.

Download Mark6 – Kobo GloHD

firmware: Kobo 4.6.9995 for GloHD

Mega update: Kobo-4.6.10075-combined/Mark6/KoboRoot.tgz

Mark5 – Aura

firmware: Kobo 4.6.9995 for Aura

Mega update: Kobo-4.6.10075-combined/Mark5/KoboRoot.tgz

Mark4 – Kobo Glo, Aura HD

firmware: Kobo 4.6.9995 for Glo and AuraHD

Mega update: Kobo-4.6.10075-combined/Mark4/KoboRoot.tgz


Louis-Philippe Véronneau: DebConf Videoteam sprint report - day 2

21 November, 2017 - 12:00

Another day, another videoteam report! It feels like we did a lot of work today, so let's jump right in:


Stefano worked most of the day on the DebConf video archive metadata project. A bunch of videos already have been uploaded to YouTube.

Here's some gold you might want to watch.

By the end of our sprint, we should have generated metadata for most of our archive and uploaded a bunch of videos to YouTube. Don't worry though, YouTube is only a mirror and we'll keep our current archive as a video master.


Andy joined us today! He hacked away with Stefano for most of the day, working on the metadata format for our videos and making schemes for our scraping tools.


Ivo built and tested a good part of our video setup today, fixing bugs left and right in Ansible. We are prepared for the Cambridge Mini-DebConf!


Nicolas finished his scripts to automatically spool up and down our streaming mirrors via the DigitalOcean API today and ran our Ansible config against those machines to test our setup.


For my part, I completed a huge chunk of my sprint goals: we now have a website documenting our setup! It is currently hosted on Alioth pages, but olasd plans to make a request to DSA to have it hosted on the machine. The final URL will most likely be something like:

The documentation is still missing the streaming section (our streaming setup is not final yet, so not point in documenting that) and a section hosting guides for the volunteers. With some luck I might write those later this week.

I've now moved on documentation our various Ansible roles.

Oh, and we also ate some cheese fondue:

Jonathan Carter: New powerline goodies in Debian

21 November, 2017 - 02:22
About powerline

Powerline does some font substitutions that allow additional theming for terminal applications such as tmux, vim, zsh, bash and more. The powerline font has been packaged in Debian for a while now, and I’ve packaged two powerline themes for vim and zsh. They’re currently only in testing, but once my current todo list on packages look better, I’ll upload them to stretch-backports.

For vim, vim-airline

vim-airline is different from previous vim powerline plugins in that it doesn’t depend om perl or python, it’s purely implemented in vim config files.


Here’s a gif from the upstream site, they also demo various themes on there that you can get in Debian by installing the vim-airlines-themes package.

How to enable

Install the vim-airline package, and add the following to your .vimrc file:

" Vim Airline theme
let g:airline_theme='powerlineish'
let g:airline_powerline_fonts = 1
let laststatus=2

The vim-airline-themes package contains additional themes that can be defined in the snippet above.

For zsh, powerlevel9k


Here’s a gif from upstream that walks through some of its features. You can configure it to display all kinds of system metrics and also information about VCS status in your current directory.

Powerlevel9k has lots of options and features. If you’re interested in it, you should probably take a look at their readme file on GitHub for all the details.

How to enable

Install the zsh-theme-powerlevel9k package and add the following to your to your .zshrc file.

source /usr/share/powerlevel9k/powerlevel9k.zsh-theme

Reproducible builds folks: Reproducible Builds: Weekly report #133

20 November, 2017 - 22:52

Here's what happened in the Reproducible Builds effort between Sunday November 5 and Saturday November 11 2017:

Upcoming events

On November 17th Chris Lamb will present at Open Compliance Summit, Yokohama, Japan on how reproducible builds ensures the long-term sustainability of technology infrastructure.

We plan to hold an assembly at 34C3 - hope to see you there!

LEDE CI tests

Thanks to the work of lynxis, Mattia and h01ger, we're now testing all LEDE packages in our setup. This is our first result for the ar71xx target: "502 (100.0%) out of 502 built images and 4932 (94.8%) out of 5200 built packages were reproducible in our test setup." - see below for details how this was achieved.

Bootstrapping and Diverse Double Compilation

As a follow-up of a discussion on bootstrapping compilers we had on the Berlin summit, Bernhard and Ximin worked on a Proof of Concept for Diverse Double Compilation of tinycc (aka tcc).

Ximin Luo did a successful diverse-double compilation of tinycc git HEAD using gcc-7.2.0, clang-4.0.1, icc-18.0.0 and pgcc-17.10-0 (pgcc needs to triple-compile it). More variations are planned for the future, with the eventual aim to reproduce the same binaries cross-distro, and extend it to test GCC itself.

Packages reviewed and fixed, and bugs filed

Patches filed upstream:

  • Bernhard M. Wiedemann:
    • clang - ASLR affects objective-C binaries.
  • Chris Lamb:
    • nbsphinx (merged) - Random UUIDs used as element selectors.
    • stardicter (merged) - SOURCE_DATE_EPOCH support.
    • stetl - Build path in documentation.

Patches filed in Debian:

Patches filed in OpenSUSE:

  • Bernhard M. Wiedemann:
    • i4l-base (merged) - Uninitialized memory written to output.
Reviews of unreproducible packages

73 package reviews have been added, 88 have been updated and 40 have been removed in this week, adding to our knowledge about identified issues.

4 issue types have been updated:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Adrian Bunk (69)
  • Andreas Beckmann (3)
  • Dmitry Shachnev (1)
  • Graham Inggs (1)
diffoscope development

Mattia Rizzolo uploaded version 88~bpo9+1 to stretch-backports.

reprotest development
  • Ximin Luo:
    • build: add comment that util-linux confirmed bug in nsenter, awaiting fix.
    • Make --print-sudoers work for --env-build as well.
reproducible-website development
  • Holger Levsen:
    • rws3: add OTF as sponsor
    • rws3: add F-Droid,
  • Chris Lamb:
    • Move the "contribute" page from the Debian wiki to /contribute/ on our main website.
  • Eitan Adler:
    • Fix typo in FreeBSD mailing list.
theunreproduciblepackage development in detail
  • Mattia Rizzolo:

    • reproducible archlinux: enable debugging mode
    • reproducible archlinux: don't use hidden files for the package lists
    • reproducible fedora: don't use hidden files for the package lists
    • udd-query: move from to
    • udd-query: remove the temporary file with a trap in case this script is called with the wrong argument, and in case of failures, etc, the temporary file would be left around otherwise
    • reproducible debian: schroot-create: drop the reproducible gpg keyring into /etc/apt/trusted.gpg.d/ instead of using apt-key add
    • reproducible debian: setup_pbuilder: drop the reproducible gpg keyring into /etc/apt/trusted.gpg.d/ instead of using apt-key add
    • reprodocible debian: setup_pbuilder: stop installing gnupg2 in our chroot, not needed anymore now
    • Mattia also merged and deployed some commits from others this week.
  • Alexander 'lynxis' Couzens

    • reproducible_lede: use correct place/variable to save results: Results on remote nodes are expected to be under $TMPDIR, which defined by openwrt_build. RESULTSDIR is undefined on the remote node
    • reproducible_lede: enable building all packages again, after it was disabled to improve the debug speed.
    • reproducible_lede: correct given path for node_cleanup_tmpdirs & node_save_logs- reproducible_lede: enable CONFIG_BUILDBOT to reduce inodes while building.
  • kpcyrd:

    • reproducible-archlinux: try porting abs to asp
    • reproducible-archlinux: explicitly sync packages
    • reproducible-archlinux: use sudo for pacman
  • Hans-Christoph Steiner:

    • reproducible fdroid: point jenkins to canonical URL
    • reproducible_fdroid: separate testsuite into its own job
    • reproducible fdroid: sync upstream script names with, make things self-documenting by reusing the same names everywhere.
    • reproducible_fdroid_test: make script executable
  • Chris Lamb:

    • Move some IRC announcements to #debian-reproducible-changes.
  • Holger Levsen:

    • reproducible LEDE: try to deal gracefully with problems and report
    • as usual, Holger merged many of the above commits and deployed them.

This week's edition was written by Ximin Luo, Bernhard M. Wiedemann, Chris Lamb and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

NOKUBI Takatsugu: Debian seminar in Yokohama, 2017/11/18

20 November, 2017 - 16:01

I had attended to Tokyo area debian seminar #157. The day’s special guest is Chris Lamb, the Debian Project Leader in 2017. He had attended to Open Compliance Summit, so we invited him as our guest.

The following pdf file is the day’s presentation:

And Hideki Yamane(henrich) talked about a new idea of Debian distribution ‘fresh’, pull-based rolling release. The details would be published by him in a few days.

There were some discussion, and we need to introduce more information aboud Japanese Debian/FLOSS scene, so now I am writing this article.

Anything else, I ccould get good time with debian developers and community. Our community, especially in Japan, requires more new commers, young people.

Louis-Philippe Véronneau: DebConf Videoteam sprint report - day 1

20 November, 2017 - 12:00

Another videoteam report! We've now been hacking for a full day and we are slowly starting to be productive. It's always hard to get back in a project when you haven't touched it in a while...

Anyway, let's start this report with some important announcement: we finally have been able to snap a good picture of the airbnb's cat!

No more nagging me about the placeholder image from Wikipedia I used in yesterday's report!

Set up

Here's what the team did today:


Stefano started the day by hacking away on our video archive. We eventually want to upload all our videos to YouTube to give them exposure, but sadly our archive metadata is in a pretty poor shape.

With the script tumbleweed wrote, we can scrape the archive for matches against the old DebConf's pentabarf XML we have.

tumbleweed also helped Ivo with the ansible PXE setup he's working on. Some recent contributions from a collaborator implemented new features (like a nice menu to choose from) but also came with a few annoying bugs.


Ivo continued working on the PXE setup today. He also tried to break our ansible setup by using fresh installs with different user cases (locales, interfaces, etc.), with some success.

The reason he and Stefano are working so hard on the PXE boot is that we had a discussion about the future of our USB install method. The general consensus on this was although we would not remove it, we would not actively maintain it anymore.

PXE is less trouble for multiple machines. For single machines or if you don't control the DHCP server, using ansible manually on a fresh Debian install will be the recommended way.


After a very long drive, olasd arrived late in the evening with all our gear. Hurray! We were thus able to set up some test boxes and start wiring the airbnb properly. Tomorrow will certainly be more productive with all this stuff at our disposition.


Today I mainly worked on setting up our documentation website. After some debate, we decided that sphinx was the right tool for the job.

I am a few pages in and if I work well I think we'll have something to show for at the end of the sprint!

I also was thrown back into ansible after witnessing a bug in the locale management. I'm still rusty, but it's slowly coming back to me.

Let's end this blog post with a picture of the neon pineapple that sits on the wall of the solarium.

Dirk Eddelbuettel: RcppEigen

20 November, 2017 - 07:23

A maintenance release of RcppEigen is now on CRAN (and will get to Debian soon). It brings Eigen 3.3.* to R.

The impetus was a request from CRAN to change the call to Rcpp::Rcpp.plugin.maker() to only use :: as the function has in fact been exported and accessible for a pretty long time. So now the usage pattern catches up. Otherwise, Haiku-OS is now supported and a minor Travis tweak was made.

The complete NEWS file entry follows.

Changes in RcppEigen version (2017-11-19)
  • Compilation under Haiku-OS is now supported (Yu Gong in #45).

  • The Rcpp.plugin.maker helper function is called via :: as it is in fact exported (yet we had old code using :::).

  • A spurious argument was removed from an example call.

  • Travis CI now uses https to fetch the test runner script.

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: RcppClassic 0.9.9

20 November, 2017 - 07:22

A maintenance release RcppClassic 0.9.9 is now at CRAN. This package provides a maintained version of the otherwise deprecated first Rcpp API; no new projects should use it.

Per a request from CRAN, we changed the call to Rcpp::Rcpp.plugin.maker() to only use :: as the function has in fact been exported and accessible for a pretty long time. So now the usage pattern catches up.

Courtesy of CRANberries, there are changes relative to the previous release.

Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Colin Watson: Kitten Block equivalent for Firefox 57

20 November, 2017 - 07:00

I’ve been using Kitten Block for years, since I don’t really need the blood pressure spike caused by accidentally following links to certain UK newspapers. Unfortunately it hasn’t been ported to Firefox 57. I tried emailing the author a couple of months ago, but my email bounced.

However, if your primary goal is just to block the websites in question rather than seeing kitten pictures as such (let’s face it, the internet is not short of alternative sources of kitten pictures), then it’s easy to do with uBlock Origin. After installing the extension if necessary, go to Tools → Add-ons → Extensions → uBlock Origin → Preferences → My filters, and add and, each on its own line. (Of course you can easily add more if you like.) Voilà: instant tranquility.

Incidentally, this also works fine on Android. The fact that it was easy to install a good ad blocker without having to mess about with a rooted device or strange proxy settings was the main reason I switched to Firefox on my phone.

Joey Hess: custom ARM disk image generation with propellor

20 November, 2017 - 02:51

Following up on propelling disk images, Propellor can now build custom ARM disk images for a variety of different ARM boards. The disk image build can run on a powerful laptop or server, so it's super fast and easy compared with manually installing Debian on an ARM board.

Here's a simple propellor config for a Olimex LIME board, with ssh access and a root password:

lime :: Host
lime = host "" $ props
    & osDebian Unstable ARMHF
    & Machine.olimex_A10_OLinuXino_LIME
    & hasPartition (partition EXT4 `mountedAt` "/" `setSize` MegaBytes 8192)
        & hasPassword (User "root")
        & Ssh.installed
    & Ssh.permitRootLogin (RootLogin True)

To make a disk image for that board, I only have to add this property to my laptop:

& imageBuiltFor lime
    (RawDiskImage "/srv/lime.img")
    (Debootstrapped mempty)

Propellor knows what kernel to install and how to make the image bootable for a bunch of ARM boards, including the Olimex LIME, the SheevaPlug, Banana Pi, and CubieTruck.

To build the disk image targeting ARM, propellor uses qemu. So it's helpful that, after the first build, propellor incrementally updates disk images, quite quickly and efficiently.

Once the board has the image installed, you can run propellor on it to further maintain it, and if there's a hardware problem, you can quickly replace it with an updated image.

computer tower that I will be maintaining with propellor

It's fairly simple to teach propellor about other ARM boards, so it should be quite easy to keep propellor knowing about all ARM boards supported by Debian (and other distros). Here's how I taught it about the Olimex LIME:

olimex_A10_OLinuXino_LIME :: Property (HasInfo + DebianLike)
olimex_A10_OLinuXino_LIME = FlashKernel.installed "Olimex A10-OLinuXino-LIME"
    `requires` sunixi "A10-OLinuXino-Lime"
    `requires` armmp

My home server is a CubieTruck which serves as a wireless access point, solar panel data collector, and git-annex autobuilder. It's deployed from a disk image built by propellor, using this config. I've been involved with building disk image for ARM boards for a long time -- it was part of my job for five years -- and this is the first time I've been entirely happy with the process.

Louis-Philippe Véronneau: DebConf Videoteam sprint report - day 0

19 November, 2017 - 12:00

First day of the videoteam autumn sprint! Well, I say first day, but in reality it's more day 0. Even though most of us have arrived in Cambridge already, we are still missing a few people.

Last year we decided to sprint in Paris because most of our video gear is stocked there. This year, we instead chose to sprint a few days before the Cambridge Mini-Debconf to help record the conference afterwards.

Since some of us arrived very late and the ones who did arrive early are still mostly jet lagged (that includes me), I'll use this post to introduce the space we'll be working from this week and our general plan for the sprint.

House Party

After some deliberations, we decided to rent a house for a week in Cambridge: finding a work space to accommodate us and all our gear proved difficult and we decided mixing accommodation and work would be a good idea.

I've only been here for a few hours, but I have to say I'm pretty impressed by the airbnb we got. Last time I checked (it seems every time I do, some new room magically appears), I counted 5 bedrooms, 6 beds, 5 toilets and 3 shower rooms. Heck, there's even a solarium and a training room with weights and a punching bag on the first floor.

Having a whole house to ourselves also means we have access to a functional kitchen. I'd really like to cook at least a few meals during the week.

There's also a cat!

It's not the house's cat per say, but it's been hanging out around the house for most of the day and makes cute faces trying to convince us to let it come inside. Nice try cat. Nice try.

Here are some glamour professional photos of what the place looks like on a perfect summer day, just for the kick of it:

Of course, reality has trouble matching all the post-processing filters.

Plan for the week

Now on a more serious note; apart from enjoying the beautiful city of Cambridge, here's what the team plans to do this week:


Stefano wants to continue refactoring our ansible setup. A lot of things have been added in the last year, but some of it are hacks we should remove and implement correctly.


Jonathan won't be able to come to Cambridge, but plans to work remotely, mainly on our desktop/xfce session implementation. Another pile of hacks waiting to be cleaned!


Ivo has been working a lot of the pre-ansible part of our installation and plans to continue working on that. At the moment, creating an installation USB key is pretty complicated and he wants to make that simpler.


Nicolas completely reimplemented our streaming setup for DC17 and wants to continue working on that.

More specifically, he wants to write scripts to automatically setup and teardown - via API calls - the distributed streaming network we now use.

Finding a way to push TLS certificates to those mirrors, adding a live stream viewer on and adding a viewer to our archive are also things he wants to look at.


For my part, I plan to catch up with all the commits in our ansible repository I missed since last year's sprint and work on documentation.

It would be very nice if we could have a static website describing our work so that others (at mini-debconfs for examples) could replicate it easily.

If I have time, I'll also try to document all the ansible roles we have written.

Stay tuned for more daily reports!

Matthieu Caneill: MiniDebconf in Toulouse

19 November, 2017 - 06:00

I attended the MiniDebconf in Toulouse, which was hosted in the larger Capitole du Libre, a free software event with talks, presentation of associations, and a keysigning party. I didn't expect the event to be that big, and I was very impressed by its organization. Cheers to all the volunteers, it has been an amazing week-end!

Here's a sum-up of the talks I attended.

Du logiciel libre à la monnaie libre

Speaker: Éloïs

The first talk I attended was, translated to English, "from free software to free money".

Éloïs compared the 4 freedoms of free software with money, and what properties money needs to exhibit in order to be considered free. He then introduced Ğ1, a project of free (as in free speech!) money, started in the region around Toulouse. Contrary to some distributed ledgers such as Bitcoin, Ğ1 isn't based on an hash-based proof-of-work, but rather around a web of trust of people certifying each other, hence limiting the energy consumption required by the network to function.


Speaker: Jimmy Monin

I then attended a presentation of YunoHost. Being an happy user myself, I was happy to discover the future expected features, and also meet two of the developers. YunoHost is a Debian-based project, aimed at providing all the tools necessary to self-host applications, including email, website, calendar, development tools, and dozens of other packages.

Premiers pas dans l'univers de Debian

Speaker: Nicolas Dandrimont

For the first talk of the MiniDebConf, Nicolas Dandrimont introduced Debian, its philosophy, and how it works with regards to upstreams and downstreams. He gave many details on the teams, the infrastructure, and the internals of Debian.

Trusting your computer and system

Speaker: Jonas Smedegaard

Jonas introduced some security concepts, and how they are abused and often meaningless (to quote his own words, "secure is bullshit"). He described a few projects which lean towards a more secure and open hardware, for both phones and laptops.

Automatiser la gestion de configuration de Debian avec Ansible

Speaker: Jérémy Lecour

Jérémy, from Evolix, introduced Ansible, and how they use it to manage hundreds of Debian servers. Ansible is a very powerful tool, and a huge ecosystem, in many ways similar to Puppet or Chef, except it is agent-less, using only ssh connections to communicate with remote machines. Very nice to compare their use of Ansible with mine, since that's the software I use at work for deploying experiments.

Making Debian for everybody

Speaker: Samuel Thibault

Samuel gave a talk about accessibility, and the general availability of the tools in today's operating systems, including Debian. The lesson to take home is that we often don't do enough in this domain, particularly when considering some issues people might have that we don't always think about. Accessibility on computers (and elsewhere) should be the default, and never require complex setups.

Retour d'expérience : mise à jour de milliers de terminaux Debian

Speaker: Cyril Brulebois

Cyril described a problem he was hired for, an update of thousands of Debian servers from wheezy to jessie, which he discovered afterwards was worse than initially thought, since the machines were running the out-of-date squeeze. Since they were not always administered with the best sysadmin practices, they were all exhibiting different configurations and different packages lists, which raised many issues and gave him interesting challenges. They were solved using Ansible, which also had the effect of standardizing their system administration practices.

Retour d'expérience : utilisation de Debian chez Evolix

Speaker: Grégory Colpart

Grégory described Evolix, a company which manages servers for their clients, and how they were inspired by Debian, for both their internal tools and their practices. It is very interesting to see that some of the Debian values can be easily exported for a more open and collaborative business.

Lightning talks

To close the conference, two lightning talks were presented, describing the switch from Windows XP to Debian in an ecologic association near Toulouse; and how can be used with bisections to find the source of some regressions.


A big thank you to all the organizers and the associations who contributed to make this event a success. Cheers!

Russ Allbery: Free software log (October 2017)

19 November, 2017 - 05:08

I've not been taking the time to write these promptly in part because I've not found the time to do much free software work. Thankfully, November will at least contain some work-sponsored work (on a package that isn't widely used yet, but maybe we can make it appealing enough).

Anyway, that's for next month. For October, the only thing I have to report is refreshing the signing key for my personal Debian repository (generating a new key for the new release) and finally updating the distributions to move stretch to stable, jessie to oldstable, and create the new testing distribution (buster). If for some strange reason you're using my personal repositories (there probably isn't much reason just at the moment), be sure to upgrade eyrie-keyring, since I'm going to switch signing over to the new key shortly.

Petter Reinholdtsen: Legal to share more than 3000 movies listed on IMDB?

19 November, 2017 - 03:20

A month ago, I blogged about my work to automatically check the copyright status of IMDB entries, and try to count the number of movies listed in IMDB that is legal to distribute on the Internet. I have continued to look for good data sources, and identified a few more. The code used to extract information from various data sources is available in a git repository, currently available from github.

So far I have identified 3186 unique IMDB title IDs. To gain better understanding of the structure of the data set, I created a histogram of the year associated with each movie (typically release year). It is interesting to notice where the peaks and dips in the graph are located. I wonder why they are placed there. I suspect World War II caused the dip around 1940, but what caused the peak around 2010?

I've so far identified ten sources for IMDB title IDs for movies in the public domain or with a free license. This is the statistics reported when running 'make stats' in the git repository:

  249 entries (    6 unique) with and   288 without IMDB title ID in free-movies-archive-org-butter.json
 2301 entries (  540 unique) with and     0 without IMDB title ID in free-movies-archive-org-wikidata.json
  830 entries (   29 unique) with and     0 without IMDB title ID in free-movies-icheckmovies-archive-mochard.json
 2109 entries (  377 unique) with and     0 without IMDB title ID in free-movies-imdb-pd.json
  291 entries (  122 unique) with and     0 without IMDB title ID in free-movies-letterboxd-pd.json
  144 entries (  135 unique) with and     0 without IMDB title ID in free-movies-manual.json
  350 entries (    1 unique) with and   801 without IMDB title ID in free-movies-publicdomainmovies.json
    4 entries (    0 unique) with and   124 without IMDB title ID in free-movies-publicdomainreview.json
  698 entries (  119 unique) with and   118 without IMDB title ID in free-movies-publicdomaintorrents.json
    8 entries (    8 unique) with and   196 without IMDB title ID in free-movies-vodo.json
 3186 unique IMDB title IDs in total

The entries without IMDB title ID are candidates to increase the data set, but might equally well be duplicates of entries already listed with IMDB title ID in one of the other sources, or represent movies that lack a IMDB title ID. I've seen examples of all these situations when peeking at the entries without IMDB title ID. Based on these data sources, the lower bound for movies listed in IMDB that are legal to distribute on the Internet is between 3186 and 4713.

It would be great for improving the accuracy of this measurement, if the various sources added IMDB title ID to their metadata. I have tried to reach the people behind the various sources to ask if they are interested in doing this, without any replies so far. Perhaps you can help me get in touch with the people behind VODO, Public Domain Torrents, Public Domain Movies and Public Domain Review to try to convince them to add more metadata to their movie entries?

Another way you could help is by adding pages to Wikipedia about movies that are legal to distribute on the Internet. If such page exist and include a link to both IMDB and The Internet Archive, the script used to generate free-movies-archive-org-wikidata.json should pick up the mapping as soon as wikidata is updates.


Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้