There was a Debian meeting at Sevilla last week.
The meeting was meant to be informal, just to know each other, hang out and drink some beers. There are a several people involved or interested in Debian and FLOSS here, but we barely know in person.
The event was announced with more than 1 month of anticipation in several mailing list (debian lists, local lists and university lists as well) and there was about 14 people interested in the event who said it explicitly.
You can see the Wiki page I set: https://wiki.debian.org/DebianEvents/es/2015/SevillaMeetingOctober
From the 14 people, 3 contacted before the event to announce they won't attend. The lasting 11 people were supposed to attend :-) In the end, we were 4 people and 3 of us know each other beforehand.
- vejeta (Juan, perhaps the main supporter of the meeting, Debian user)
- Pablo Neira (Netfilter upstream project and Linux kernel maintainer)
- Javier Barroso (IT architect in the local goverment and Debian user)
- me (the only directly involved in the Debian project, I'm DM)
This is a picture of that moment:
The truth is that we had a very good afternoon, but I personally missed some more people.
I remember Ana Guerrero telling me that a meeting of just 4-5 is indeed a victory :-)
We will have to try again :-)
For my home office network I have been using Dnsmasq for some time. Dnsmasq provides me with DNS, DHCP, DHCPv6, and IPv6 Router Advertisement. I run dnsmasq on a Debian Jessie server, but it works similar with OpenWRT if you want to use a smaller device. My entire /etc/dnsmasq.d/local configuration used to look like this:
dhcp-authoritative interface=eth1 read-ethers dhcp-range=192.168.1.100,192.168.1.150,12h dhcp-range=2001:9b0:104:42::100,2001:9b0:104:42::1500 dhcp-option=option6:dns-server,[::] enable-ra
Here dhcp-authoritative enable DHCP. interface=eth1 says to listen on eth1 only, which is my internal (IPv4 NAT) network. I try to keep track of the MAC address of all my devices in a /etc/ethers file, so I use read-ethers to have dnsmasq give stable IP addresses for them. The dhcp-range is used to enable DHCP and DHCPv6 on my internal network. The dhcp-option=option6:dns-server,[::] statement is needed to inform the DHCP clients of the DNS resolver’s IPv6 address, otherwise they would only get the IPv4 DNS server address. The enable-ra parameter enables IPv6 router advertisement on the internal network, thereby removing the need to run radvd too — useful since I prefer to use copyleft software.
Recently I had a desire to use DNSSEC, and enabled it in Dnsmasq using the following statements:
dnssec trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5 dnssec-check-unsigned
The dnssec keyword enable DNSSEC validation in dnsmasq, using the indicated trust-anchor (get the root-anchors from IANA). The dnssec-check-unsigned deserves some more discussion. The dnsmasq manpage describes it as follows:
As a default, dnsmasq does not check that unsigned DNS replies are legitimate: they are assumed to be valid and passed on (without the “authentic data” bit set, of course). This does not protect against an attacker forging unsigned replies for signed DNS zones, but it is fast. If this flag is set, dnsmasq will check the zones of unsigned replies, to ensure that unsigned replies are allowed in those zones. The cost of this is more upstream queries and slower performance.
For example, this means that dnsmasq’s DNSSEC functionality is not secure against active man-in-the-middle attacks between dnsmasq and the DNS server it is using. Even if example.org used DNSSEC properly, an attacker could fake that it was unsigned to dnsmasq, and I would get potentially incorrect values in return. We all know that the Internet is not a secure place, and your threat model should include active attackers. I believe this mode should be the default in dnsmasq, and users should have to configure dnsmasq to not be in that mode if they really want to (with the obvious security warning).
Running with this enabled for a couple of days resulted in frustration about not being able to reach a couple of domains. The behaviour was that my clients would hang indefinitely or get a SERVFAIL, both resulting in lack of service. You can enable query logging in dnsmasq with log-queries and enabling this I noticed three distinct form of error patterns:
jow13gw dnsmasq 460 - - forwarded www.fritidsresor.se to 22.214.171.124 jow13gw dnsmasq 460 - - validation result is BOGUS
jow13gw dnsmasq 547 - - reply cloudflare-dnssec.net is BOGUS DNSKEY jow13gw dnsmasq 547 - - validation result is BOGUS
jow13gw dnsmasq 547 - - reply linux.conf.au is BOGUS DS jow13gw dnsmasq 547 - - validation result is BOGUS
The first only happened intermittently, the second did not cause any noticeable problem, and the final one was reproducible. To be fair, I only found the last example after starting to search for problem reports (see post confirming bug).
At this point, I had a confirmed bug in dnsmasq that affect my use-case. I want to use official packages from Debian on this machine, so installing newer versions manually is not an option. So I started to look into alternatives for DNS resolving, and quickly found Unbound. Installing it was easy:
apt-get install unbound unbound-control-setup
I created a local configuration file in /etc/unbound/unbound.conf.d/local.conf as follows:
server: interface: 127.0.0.1 interface: ::1 interface: 192.168.1.2 interface: 2001:9b0:104:42::2 access-control: 127.0.0.1 allow access-control: ::1 allow access-control: 192.168.1.2/24 allow access-control: 2001:9b0:104:42::2/64 allow outgoing-interface: 126.96.36.199 outgoing-interface: 2001:9b0:1:1a04::2 # log-queries: yes # verbosity: 2
The interface keyword determine which IP addresses to listen on, here I used the loopback interface and the local address of the physical network interface for my internal network. The access-control allows recursive DNS resolving from those networks. And outgoing-interface specify my external Internet-connected interface. log-queries and/or verbosity are useful for debugging.
To make things work, dnsmasq has to stop providing DNS services. This can be achieved with the port=0 keyword, however that will also disable informing DHCP clients about the DNS server to use. So this has to be added in manually. I ended up adding the two following lines to /etc/dnsmasq.d/local:
Restarting unbound and dnsmasq now leads to working (and secure) internal DNSSEC-aware name resolution over both IPv4 and IPv6. I can verify that resolution works, and that Unbound verify signatures and reject bad domains properly with dig as below, or use online DNSSEC resolver test page although I’m not sure how confident you can be in the result from that page.
$ host linux.conf.au linux.conf.au has address 188.8.131.52 linux.conf.au mail is handled by 1 linux.org.au. $ host sigfail.verteiltesysteme.net ;; connection timed out; no servers could be reached $
I use Munin to monitor my services, and I was happy to find a nice Unbound Munin plugin. I installed the file in /usr/share/munin/plugins/ and created a Munin plugin configuration file /etc/munin/plugin-conf.d/unbound as follows:
[unbound*] user root env.statefile /var/lib/munin-node/plugin-state/root/unbound.state env.unbound_conf /etc/unbound/unbound.conf env.unbound_control /usr/sbin/unbound-control env.spoof_warn 1000 env.spoof_crit 100000
I run munin-node-configure --shell|sh to enable it. To work unbound has to be configured as well, so I create a /etc/unbound/unbound.conf.d/munin.conf as follows.
server: extended-statistics: yes statistics-cumulative: no statistics-interval: 0 remote-control: control-enable: yes
The graphs may be viewed at my munin instance.
Review: Hawk, by Steven BrustSeries: Vlad Taltos #14 Publisher: Tor Copyright: October 2014 ISBN: 0-7653-2444-X Format: Hardcover Pages: 320
This is the fourteenth book in the Vlad Taltos series (not counting the various associated books and other series), builds directly on the long-term plot arc of the series (finally!), and is deeply entangled with Vlad's friends and former life as a Jhereg boss. As you might imagine from that introduction, this is absolutely not the place to start with this series.
For the past few books, Brust has been following a pattern of advancing the series plot in one book and then taking the next book to fill in past history or tell some side story. That means, following Tiassa, we were due some series advancement, and that's exactly what we get. We also, finally, get some more details about Lady Teldra. Nothing all that revelatory, but certainly intriguing, and more than just additional questions (at last). When Brust finally takes this gun off the wall and fires it, the resulting bits of world-building might be even better than Issola.
At its heart, though, Hawk is a caper novel. If you're like me, you're thinking "it's about time." I think this is the sort of story Brust excels at, particularly with Vlad as his protagonist. Even better, unlike some of the other multi-part novels, this is a book-length caper focused on a very important goal, and with the potential to get rid of some annoyances in Vlad's life that have lingered for rather too long. We see many of Vlad's Dragaeran friends, but (apart from Daymar) mostly in glimpses. This is Vlad's book, with heavy helpings of Loiosh.
The caper is also a nicely twisty one, involving everything from different types of magic to the inner workings of the Jhereg organization. As is typical for Vlad's schemes, there are several false fronts and fake goals, numerous unexpected twists, and a rather fun guest appearance. Oh, and lots and lots of snark, of course. I think my favorite part of the book was the interaction between Vlad and Kragar, which added a lot of emotional depth both to this story and to some of the previous stories of Vlad's life as a Jhereg. And I'm hoping that where Brust leaves things at the end of this book implies a Vlad who is more free to act, to see his friends, and to get entangled in Imperial politics, since I think that leads to the best stories.
Of course, if Brust holds to pattern, the next book will be backfill or side stories and we'll have to wait longer for a continuation of the main story. As much as I like those side stories, I'm hoping Brust will break pattern. I'm increasingly eager to see where this story will go. The all-too-brief interaction with Sethra in this book promises so much for the future.
If you like the Vlad Taltos books overall, you'll probably like this one. It's a return to the old scheming Vlad, but tempered by more experience and different stakes. There's a bit of lore, a bit of world-building, and a lot of Vlad being tricky. This series is still going strong fourteen books in.
Rating: 8 out of 10
I looked at two interesting issues today around the ps program in the procps project. One had a solution and the other I’m puzzled about.ps User-defined Format
Issue #9 was quite the puzzle. The output of ps changed depending if a different option had a hyphen before it or not.
First, the expected output
$ ps p $$ -o pid=pid,comm=comm pid comm 31612 bash
Next, the unusual output.
$ ps -p $$ -o pid=pid,comm=comm pid,comm=comm 31612
The difference being the second we have -p not p. The second unexpected thing about this is, it was designed that way. Unix98 standard apparently permits this sort of craziness. To me it is a useless feature that will more likely than not confuse people. Within ps, depending on what sort of flags you start with, you use a sysv parser or a bsd parser. One of them triggered the Unix98 compatibility option while the other did not, hence the change in behavior. The next version of procps will ship with a ps that has the user-defined output format of the first example. I had a look at the latest standard, IEEE 1003.1-2013, doesn’t seem to have anything like that in it.
Short Month Length
This one has got me stuck. A user has reported in issue #5 that when they use their locale columns such as start time get mis-aligned because their short month is longer than 3 characters. They also mention some other columns for memory etc are not long enough but that’s a generic problem that is impossible to fix sensibly.
OK, for the month problem the fix would be to know what the month length is and set the column width for those columns to that plus a bit more for the other fixed components, simple really. Except; how do you know for a specific locale what their short month length is? I always assumed it was three! I haven’t found anything that has this information. Note, I’m not looking for strlen() but some function that has the maximum length for short month names (e.g. Jan, Feb, Mar etc). This also got me thinking how safe some of those date to string functions are if you have a static buffer as the destination. It’s not safe to assume they will be “DD MMM YYYY” because there might be more Ms.
So if you know how to work out the short month name length, let me know!
TLDR: With the help of Helmut Grohne I finally figured out most of the bits necessary to unshare everything without becoming root (though one might say that this is still cheated because the suid root tools newuidmap and newgidmap are used). I wrote a Perl script which documents how this is done in practice. This script is nearly equivalent to using the existing commands lxc-usernsexec [opts] -- unshare [opts] -- COMMAND except that these two together cannot be used to mount a new proc. Apart from this problem, this Perl script might also be useful by itself because it is architecture independent and easily inspectable for the curious mind without resorting to sources.debian.net (it is heavily documented at nearly 2 lines of comments per line of code on average). It can be retrieved here at https://gitlab.mister-muffin.de/josch/user-unshare/blob/master/user-unshare
Long story: Nearly two years after my last last rant about everything needing superuser privileges in Linux, I'm still interested in techniques that let me do more things without becoming root. Helmut Grohne had told me for a while about unshare(), or user namespaces as the right way to have things like chroot without root. There are also reports of LXC containers working without root privileges but they are hard to come by. A couple of days ago I had some time again, so Helmut helped me to get through the major blockers that were so far stopping me from using unshare in a meaningful way without executing everything with sudo.
My main motivation at that point was to let dpkg-buildpackage when executed by sbuild be run with an unshared network namespace and thus without network access (except for the loopback interface) because like pbuilder I wanted sbuild to enforce the rule not to access any remote resources during the build. After several evenings of investigating and doctoring at the Perl script I mentioned initially, I came to the conclusion that the only place that can unshare the network namespace without disrupting anything is schroot itself. This is because unsharing inside the chroot will fail because dpkg-buildpackage is run with non-root privileges and thus the user namespace has to be unshared. But this then will destroy all ownership information. But even if that wasn't the case, the chroot itself is unlikely to have (and also should not) tools like ip or newuidmap and newgidmap installed. Unsharing the schroot call itself also will not work. Again we first need to unshare the user namespace and then schroot will complain about wrong ownership of its configuration file /etc/schroot/schroot.conf. Luckily, when contacting Roger Leigh about this wishlist feature in bug#802849 I was told that this was already implemented in its git master \o/. So this particular problem seems to be taken care of and once the next schroot release happens, sbuild will make use of it and have unshare --net capabilities just like pbuilder already had since last year.
With the sbuild case taken care of, the rest of this post will introduce the Perl script I wrote. The name user-unshare is really arbitrary. I just needed some identifier for the git repository and a filename.
The most important discovery I made was, that Debian disables unprivileged user namespaces by default with the patch add-sysctl-to-disallow-unprivileged-CLONE_NEWUSER-by-default.patch to the Linux kernel. To enable it, one has to first either do
echo 1 | sudo tee /proc/sys/kernel/unprivileged_userns_clone > /dev/null
sudo sysctl -w kernel.unprivileged_userns_clone=1
The tool tries to be like unshare(1) but with the power of lxc-usernsexec(1) to map more than one id into the new user namespace by using the programs newgidmap and newuidmap. Or in other words: This tool tries to be like lxc-usernsexec(1) but with the power of unshare(1) to unshare more than just the user and mount namespaces. It is nearly equal to calling:
lxc-usernsexec [opts] -- unshare [opts] -- COMMAND
Its main reason of existence are:
- as a project for me to learn how unprivileged namespaces work
- written in Perl which means:
- architecture independent (same executable on any architecture)
- easily inspectable by other curious minds
- tons of code comments to let others understand how things work
- no need to install the lxc package in a minimal environment (perl itself might not be called minimal either but is present in every Debian installation)
- not suffering from being unable to mount proc
I hoped that systemd-nspawn could do what I wanted but it seems that its requirement for being run as root will not change any time soon
Another tool in Debian that offers to do chroot without superuser privileges is linux-user-chroot but that one cheats by being suid root.
Had I found lxc-usernsexec earlier I would've probably not written this. But after I found it I happily used it to get an even better understanding of the matter and further improve the comments in my code. I started writing my own tool in Perl because that's the language sbuild was written in and as mentioned initially, I intended to use this script with sbuild. Now that the sbuild problem is taken care of, this is not so important anymore but I like if I can read the code of simple programs I run directly from /usr/bin without having to retrieve the source code first or use sources.debian.net.
The only thing I wasn't able to figure out is how to properly mount proc into my new mount namespace. I found a workaround that works by first mounting a new proc to /proc and then bind-mounting /proc to whatever new location for proc is requested. I didn't figure out how to do this without mounting to /proc first partly also because this doesn't work at all when using lxc-usernsexec and unshare together. In this respect, this perl script is a bit more powerful than those two tools together. I suppose that the reason is that unshare wasn't written with having being called without superuser privileges in mind. If you have an idea what could be wrong, the code has a big FIXME about this issue.
Finally, here a demonstration of what my script can do. Because of the /proc bug, lxc-usernsexec and unshare together are not able to do this but it might also be that I'm just not using these tools in the right way. The following will give you an interactive shell in an environment created from one of my sbuild chroot tarballs:
$ mkdir -p /tmp/buildroot/proc $ ./user-unshare --mount-proc=/tmp/buildroot/proc --ipc --pid --net \ --uts --mount --fork -- sh -c 'ip link set lo up && ip addr && \ hostname hoothoot-chroot && \ tar -C /tmp/buildroot -xf /srv/chroot/unstable-amd64.tar.gz; \ /usr/sbin/chroot /tmp/buildroot /sbin/runuser -s /bin/bash - josch && \ umount /tmp/buildroot/proc && rm -rf /tmp/buildroot' (unstable-amd64-sbuild)josch@hoothoot-chroot:/$ whoami josch (unstable-amd64-sbuild)josch@hoothoot-chroot:/$ hostname hoothoot-chroot (unstable-amd64-sbuild)josch@hoothoot-chroot:/$ ls -lha /proc | head total 0 dr-xr-xr-x 218 nobody nogroup 0 Oct 25 19:06 . drwxr-xr-x 22 root root 440 Oct 1 08:42 .. dr-xr-xr-x 9 root root 0 Oct 25 19:06 1 dr-xr-xr-x 9 josch josch 0 Oct 25 19:06 15 dr-xr-xr-x 9 josch josch 0 Oct 25 19:06 16 dr-xr-xr-x 9 root root 0 Oct 25 19:06 7 dr-xr-xr-x 9 josch josch 0 Oct 25 19:06 8 dr-xr-xr-x 4 nobody nogroup 0 Oct 25 19:06 acpi dr-xr-xr-x 6 nobody nogroup 0 Oct 25 19:06 asound
Of course instead of running this long command we can also instead write a small shell script and execute that instead. The following does the same things as the long command above but adds some comments for further explanation:
# I'm using /tmp because I have it mounted as a tmpfs
# bring the loopback interface up
ip link set lo up
# show that the loopback interface is really up
# make use of the UTS namespace being unshared
# extract the chroot tarball. This must be done inside the user namespace for
# the file permissions to be correct.
# tar will fail to call mknod and to change the permissions of /proc but we are
# ignoring that
tar -C "$rootdir" -xf /srv/chroot/unstable-amd64.tar.gz || true
# run chroot and inside, immediately drop permissions to the user "josch" and
# start an interactive shell
/usr/sbin/chroot "$rootdir" /sbin/runuser -s /bin/bash - josch
# unmount /proc and remove the temporary directory
rm -rf "$rootdir"
$ mkdir -p /tmp/buildroot/proc $ ./user-unshare --mount-proc=/tmp/buildroot/proc --ipc --pid --net --uts --mount --fork -- ./chroot.sh
As mentioned in the beginning, the tool is nearly equivalent to calling lxc-usernsexec [opts] -- unshare [opts] -- COMMAND but because of the problem with mounting proc (mentioned earlier), lxc-usernsexec and unshare cannot be used with above example. If one tries anyways one will only get:
$ lxc-usernsexec -m b:0:1000:1 -m b:1:558752:1 -- unshare --mount-proc=/tmp/buildroot/proc --ipc --pid --net --uts --mount --fork -- ./chroot.sh unshare: mount /tmp/buildroot/proc failed: Invalid argument
I'd be interested in finding out why that is and how to fix it.
The Linux Kernel Oops website collects kernel errors from all over the World helping kernel developers finding issues occurring in the wild but they cannot help if no one sends reports to them.
The Kerneloops client used to be part of Debian releases but it has been removed from the archive due to not working with the new collector site.
When I started observing oopses on my machine I first thought of submitting a bug against the linux package in BTS, but looking at the numerous bugs opened already I looked for a more automated solution which would also help others. Reviving the kerneloops package involved switching it to the new submission URL, fixing a few memory allocation bugs in C (this is the first package I found using Valgrind by default for running ) and ensuring that upstream was still active. The last step took the most of the time but finally Anton Arapov kindly accepted my patches and everything was set for the new upload.
The package is now available from unstable and if you feel so (especially if you experience oopses) please give it a try and report any problems you find. I’m also happy to receive success stories about oopses fixed after discovering and collecting them with the client.
Review: The Thrilling Adventures of Lovelace and Babbage, by Sydney PaduaPublisher: Pantheon Copyright: 2015 ISBN: 0-307-90827-5 Format: Hardcover Pages: 317
I seem to be writing a lot of reviews lately where any reader can easily do their own research and see if this is something they'd like. This is another of those, but perhaps a bit more obscure.
As Padua explains in the preface, the first Lovelace and Babbage strip was intended to be a one-off joke. A friend suggested that she draw a comic telling the (short and rather tragic) life of Ada Lovelace, widely considered the first computer programmer even though the computer for which she wrote programs was never built: Charles Babbage's Analytic Engine. Padua found the story of Babbage and Lovelace's collaboration fascinating but too grim, and so wrote an alternate ending involving a pocket universe in which Babbage and Lovelace lived on to fight crime! Or at least fought against street organs and poetry.
Padua meant this as a joke. The Internet took it as a teaser. Quite to her surprise, she found herself writing occasional additional episodes and getting lost in fascinating research about Lovelace and Babbage. The result is all at 2dgoggles.com. This book is a curated collection of those comics, including much of their (extensive) footnotes and research notes, in a very attractive and well-constructed book form. The best review, therefore, would be to go read some of the comics yourself, easily accessible via the Comics tab on the web site, and see if this is the sort of thing you'd enjoy. There is some material in the book that isn't on the web site, but most of it is there (including one full story that didn't make it into the book).
There are a few reasons to buy a collection like this of material that previously appeared on the Internet. One, of course, is out of gratitude to support the author, which is the main reason I bought it. If you want to do that, though, you probably already know you do. Another reason is for additional unpublished material, but that's not really the case here. But a third is that, despite all the technology of the web, books can sometimes provide a more elegant and beautiful presentation. And that's very much the case for The Thrilling Adventures of Lovelace and Babbage.
The hardcover is a beautiful work of art. It has what you would expect from a hardcover graphic novel, of course: sturdy paper, lovely story headings in the form of Victorian-style posters, and very nice artwork beneath the dust jacket. (I've been so happy to see that in the last few hardcovers I've reviewed.) But, best of all, it presents the primary research notes on each page underneath comic panels.
Those who have been reading Lovelace and Babbage as it appears on the web know Padua's copious research notes. But they're normally presented at the end of each complete part, and I found myself often not reading them in detail. This might not be the case for everyone, but at least for me this presentation works so much better. Historical notes are at the bottom of (nearly) every page, in context. This might sound like it would distract from and break up the flow of the story, but at least for me it does the opposite. The comics feel so much richer when intermixed with the actual history (which in some cases is surprisingly similar to what seemed to be wild, invented scenarios).
The historical notes also have end notes, and quite substantial ones. I found that organization less successful (readers of my reviews will know I have a long-standing antipathy towards end notes for anything other than cross-references), but I have to admit I have no idea how Padua would have fit those on the same page as footnotes. They're also written well enough, and with sufficient detail, that I could usually read a chapter and then read all the end notes and mostly remember what they referred to.
Padua also takes advantage of the format to play a few neat games with frame breaks: characters commenting on the notes, stabbing things into them, or otherwise affecting them. I have to admit to some mild frustration where this makes the notes unreadable, since they're nearly as fascinating as the comics, but the overall effect is still worth it. (And, of course, the full notes are available on the web if one wishes to look them up.)
Rounding out the book version are a few of the best original source documents that Padua found, thankfully excerpted and well-edited to not outlive their welcome for those of us who don't like pouring over Victorian letters. And, finally, some beautiful illustrations of the mechanisms of the Analytic Engine with supporting descriptions of how they were supposed to work. I'm not particularly mechanically inclined, nor that fascinated by steampunk, and I still found these diagrams impressive and fun to look at. Someone more interested in such things will be in for a treat.
I've been thoroughly enjoying the (sadly infrequent) installments of Lovelace and Babbage for years now, and am utterly delighted by their hardcover appearance. If you already knew about Padua's work and have any interest in nice hardcovers of such things, I don't think you'll regret the purchase. If you haven't heard of her, or this series, before, some quick reading at the web site should quickly reveal whether this is something you'd enjoy.
Rating: 9 out of 10
A couple more simple fixes in the continuing failure of this script to get rewritten in Python and integrated properly with the git-buildpackage distribution.... (But hey, I'm caught up on writing book reviews!)
When used with qemubuilder, it was checking for the existence of entirely the wrong file. Thanks to James Clarke for the patch.
Running git-pbuilder with a command requires sudo and an appropriate sudo configuration. There's not much we can do to detect the latter, but Guido Günther suggested at least detecting the former. git-pbuilder now does so and prints out a hopefully informative error. This is also better-documented in the man page.
You can get the latest release from my scripts page.
Release 0.1.8 brings over a handfull new access points relative to the previous release 0.1.6: bday(), family_(), shutup(), zayn(), dalton(), dosomething(), retard(), and thumbs(). One of these may make a fine addition for beloved post-test comments in testthat:
R> library(rfoaas) R> dosomething("Fix", "function", "testthat")  "Fix the fucking function! - testthat" R>
We did not make a release following version 0.1.7 of FOAAS: my testing revealed somewhat less consistent results for the language / i18n filter added in 0.1.6. So I filed issue ticket #95, but as it still lingers unanswered, I consider this feature to be unsupported for now; the tests for the language filter were disabled for now as well.
Vincent Sanders: It takes courage to sit on a jury. How many of us want to decide the fate of another person's life or freedom?
The actual process of becoming a juror on a case is something I had not been aware of previously. You simply receive a letter telling you to be at the court on a specific date and that you are required to be available for at least ten days, possibly more. The only qualification to receive the letter is to be on the electoral roll and it is an invitation with few options to refuse without serious repercussions.
When you arrive at the court you are directed to the Jury lounge (practical hint: take a book) where you notice there are over forty people, which would seem odd until you realise there are three courtrooms and they each need a jury, even then there are an excess of people which is because of the selection process.
The process of jury selection is fairly simple, an usher for a court comes and calls fourteen names which forms the jury in waiting. The group is taken up to the court waiting room (this room gets terribly familiar over the forthcoming weeks) and then twelve names are called.
As each person is called they enter the jury box in order which persists for the entire trial (practical hint:remember your juror number). Before each person is sworn or affirmed there is the possibility they will be found unsuitable and will be replaced by one of the previously unselected jurors. Any unselected jurors are then sent back to the jury lounge and become available for forming another jury in waiting.
Anyone unselected at the end of the process has to remain available to return to the court to form a jury in waiting when a previous trial ends until they have exhausted their duty. There were a few of these unfortunate people who were kept in a state of limbo for several days and I am relieved this did not happen to me.
Being a juror, from a purely practical perspective, felt like working an office job for ten days. Duties consisting of attending a series of meetings with strange rules and the typically understated British approach to mentioning the result of breaking them.
I participated in two cases, both of which were (almost by definition) unpleasant happenings, these were a case of Grievous bodily harm and an offence under section 7 of the Sexual Offences Act 2003.
Both cases were challenging in their own ways the first because of the way the case was presented and the second because of its subject matter. One of the most important rules is "Do not discuss anything with anyone as it might be perjury" so going along with that I will not be discussing any details. Because I cannot be specific this post has become a little impersonal, you will have to forgive me as I found I had to remove a great deal of more content which was not appropriate.
An important thing to note is that the trials bore no resemblance to TV courtroom drama. The trials proceed in a professional manner with very little theatrics. The prosecution barrister commences outlining the case against the accused, calling witnesses and reading into evidence uncontested material. The defence then gets to present their case, again calling witnesses and placing documents into evidence.
One of the striking things about this process is that if the barristers do not call a witness or present evidence that would seem to be pertinent, the jury must not draw inference from that omission, which is especially bizarre when a central witness referred to by almost everyone involved with the case is not called.
Once the case is presented the jury is sequestered in a room and must come to a unanimous decision on each of the charges. This was, for me, the most challenging part of the whole process. Twelve people with unique views on the information presented have to attempt to discuss the evidence and not simply go with their first impressions based on their preconceptions.
The jury is allowed to ask for some evidence to be repeated and if deliberations take some time the jury may be instructed that a majority of 10 to 2 may be accepted. I imagine at some point the jury will run out of time to make a decision and something else will happen but I did not experience this.
Overall the experience was enlightening if not enjoyable, I understand the process a lot more and am happy to have discharged my duty and am equally glad the responsibility will not come around again for at least a few years.
Review: Oathbreakers, by Mercedes LackeySeries: Vows and Honor #2 Publisher: DAW Copyright: January 1989 ISBN: 0-88677-454-3 Format: Mass market Pages: 318
The Tarma and Kethry stories tend to be stand-alone and are readable out of order, and this isn't an exception. But if you want their background, consider reading Oathblood or (less recommended) The Oathbound before reading this book. (Reading Oathblood first may require a bit of finesse, since some of the stories in that book come after this novel. Unfortunately, there is no good ordering or collection of these stories that maintains internal chronological order.)
This is more like it. This is the Tarma and Kethry story that I remembered when calling them my favorite characters in the Valdemar universe.
Following the short stories merged into The Oathbound fixup novel, Tarma and Kethry are still trying to gather the resources required to start a school and to rebuild Tarma's clan. That's led to them signing with a highly-respected mercenary company: Idra's Sunhawks. Idra renounced her claims to the Rethwellen line of royal succession to lead the Sunhawks, creating a mercenary band that's legendary for their quality and battlefield capabilities. The story opens with a campaign in Jkatha, on one side of a civil war, which is mostly an opportunity to get to know the Sunhawks and to see Tarma and Kethry show their competence. The real story starts later, when Idra is called back to Rethwellen for family business and something goes very wrong.
I think Lackey is best at two types of stories: misunderstood young people who grow into themselves and their place in the world, and competent people displaying their competence. The Tarma and Kethry stories, and particularly Oathbreakers, are of the latter type. This is clearly wish fulfillment: Lackey's stories often lack nuance, there's rarely any doubt as to who the good and bad guys are, and, although very bad things can happen, you're probably going to get some sort of happy ending. But if you're in the mood for that sort of story, it's so satisfying.
The Tarma and Kethry we see here are a mature, experienced fighter and mage team (plus Warrl, who provides vitally important magical and combat assistance, as well as some pointed advice). They know what they're doing, they care deeply for each other, and both their relationship patterns and their capabilities are well-understood. Both do a bit of growing over the course of this novel, but that's not really the point. The point is seeing them take on unfamiliar challenges and tricky investigations while being very good at what they do. In other words, this isn't bildungsroman or high fantasy; it's sword and sorcery, and an excellent example of the genre.
Reading these books as part of the overall Valdemar series provides some enjoyable moments with the first explicit contact between Valdemar and its Heralds and Tarma and Kethry's world. The maps here firmly establish their home regions as well to the south of Valdemar and multiple kingdoms away, but Rethwellen (as previously established in earlier Herald-focused trilogies) is on Valdemar's southern border. Seeing Lackey's very separate magic and divinity systems cross and meet, with a bit of initial mutual suspicion, is a rather fun moment (if, at least, you're in the mood for a story in which the world has a vested interest in making sure all the good people like each other). Although I'm wondering why Kethry didn't get extremely uncomfortable when she crossed the border into Valdemar due to the trick that Vanyel pulled in his trilogy. (I seem to recall this is explained away at some point.)
Be warned that this novel does contain other elements typical of early Lackey. There is, for example, the inevitable rape, although thankfully off-camera and not quite as central to the plot. (Although in a way that makes it worse since it felt gratuitous. I'm unconvinced that the rape was at all necessary to the story that Lackey was telling.) Revenge and eye-for-an-eye justice are hotly defended by the protagonists. This isn't a series to look to for subtle and complex solutions to political problems; instead, everything gets better if you just kill all the evil people. There isn't anything quite as egregious as the actions of the supposed good guys in The Oathbound, but you still have to read past a certain bloodthirstiness in the stated good side of a very black-and-white morality.
That means this isn't a novel for all people or all moods. But within those genre conventions, which aren't that unusual for sword and sorcery, Oathbreakers is a lot of fun. It's one of the few Valdemar novels I've read during this re-read that lived up to my memory of it. Recommended if you like this sort of thing.
Rating: 8 out of 10
The 12 member nations of the Trans-Pacific Partnership pact have agreed to prohibit demands that companies reveal software source code*, a step that appears aimed at curbing efforts by China to gain access to this sensitive information, The Yomiuri Shimbun has learned.
Source code is the confidential information for software and is a “blueprint” embedded in many commonly used products, such as vehicles, mobile phones and home appliances. Source code is usually tightly guarded because it conatins commands that make using the software easier.
China requires foreign companies operating there to hand over source code, a move that has sparked sharp criticism from many countries. Observers believe the decision by TPP participants to ban demands to reveal this code is intended to restrain China's move.
The TPP's electronic commerce chapter in principle prohibits the 12 member nations from demanding access to source code for mass-produced software. According to the Economy, Trade and Industry Ministry, the Japan-Mongolia economic partnership agreement signed in February contains a stipulation banning demands for such information, but there are very few other examples around the world.
Japan, the United States and other nations that are home to many information technology companies want to make the stipulation effectively a global standard, and they are considering whether to incorporate such a condition in economic partnership agreements that will be inked in the future.
Source code is an important corporate secret for development firms.
*Source code—A software program written in a language a computer understands. As well as containing expert details about the unique functions of the product, the software is essential for fixing glitches and making improvements. Hackers have attempted to access source code because gaining this information would make it easier to create viruses that could exploit any software defects. In the software business, it is rarely made public, as it is considered a corporate secret.
Petter Reinholdtsen: "Free Culture" by @lessig - The background story for Creative Commons - new edition available
In 2004, as the Creative Commons movement gained momentum, its creator Lawrence Lessig wrote the book Free Culture to explain the problems with increasing copyright regulation and suggest some solutions. I read the book back then and was very moved by it. Reading the book inspired me and changed the way I looked on copyright law, and I would love it if more people would read it too.
Because of this, I decided in the summer of 2012 to translate it to Norwegian Bokmål and publish it for those of my friends and family that prefer to read books in Norwegian. I translated the book using docbook and a gettext PO file, and a byproduct of this process is a new edition of the English original. I've been in touch with the author during by work, and he said it was fine with him if I also published an English version. So I decided to do so. Today, I made this edition available for sale on Lulu.com, for those interested in a paper book. This is the cover:
The Norwegian Bokmål version will be available for purchase in a few days. I also plan to publish a French version in a few weeks or months, depending on the amount of people with knowledge of French to join the translation project. So far there is only one active person, but the French book is almost completely translated but need some proof reading.
The book is also available in PDF, ePub and MOBI formats from my github project page. Note the ePub and MOBI versions have some formatting problems I believe is due to bugs in the docbook tool dbtoepub (Debian BTS issues #795842 and #796871), but I have not taken the time to investigate. I recommend the PDF and ePub version for now, as they seem to show up fine in the viewers I have available.
After the translation to Norwegian Bokmål was complete, I was able to secure some sponsoring from the NUUG Foundation to print the book. This is the reason their logo is located on the back cover. I am very grateful for their contribution, and will use it to give a copy of the Norwegian edition to members of the Norwegian Parliament and other decision makers here in Norway.
I have had some success signing an Italian fattura elettronica with OpenSSL.
I am amazed to realise that the software they gave me to do the job is of such bad quality that I felt like spending a few hours trying to do the same thing with OpenSSL instead.
Why don't they donate to Core Infrastructure of their business? It's okay for its license, just Ethical thing. Not much for strategical benefit. But... you, leaders? I think you're enough to pay some money for it.
Review: Watchtower, by Elizabeth A. LynnSeries: Chronicles of Tornor #1 Publisher: Berkley Copyright: February 1979 Printing: November 1982 ISBN: 0-425-06195-7 Format: Mass market Pages: 226
Ryke is a soldier in service to Athor, lord of Tornor Keep. Or, at least, he was. Shortly before the opening lines of this book, Tornor Keep was taken by a southerner named Col Istor, and Athor was killed. Ryke is about to change sides under duress: his service to Col as a watch commander is the price of keeping Errel, Athor's son, alive. Alive does not mean treated with dignity, however; Errel's new place is to become court jester for Col's small domain.
At the start of this book, I had no idea who Col Istor was or why he'd want to take over one of the border keeps that guard the country of Arun against its northern neighbor Anhard. At the end of this book, I still had little idea. Watchtower's relationship with world-building is not what you'd normally expect from a fantasy novel.
There are only a few locations in this novel, and they're described in close detail but without much historical or strategic context. The narrative, despite being third person, follows Ryke very closely, and Ryke is not interested in thinking much about the history of his world that he knows and the reader doesn't. Or he may not care. Ryke does not come across as a curious or thoughtful person. He's a soldier, he lives to serve his lord, and that service transfers from Athor to Errel and simply stays there, with little discussion or argument.
The opening section of this novel shows Ryke trying to take care of Errel while serving Col and listening to his plans to take over the rest of the northern keeps. I found it slow, claustrophobic, and not very interesting. It reads a bit like a day in the life in some grim medieval military, with little world-building or context. Watchtower is technically fantasy, but mostly because the geography and politics are imaginary. Magic is limited to a very tiny bit of (essentially) Tarot card reading and has little impact on the plot.
The story does get somewhat more interesting when two green clan messengers enter the story, and Ryke and Errel escape. Lynn introduces another community and another way of life into the story than the one Ryke is used to, one that seemed modeled after monastic orders (but with the religion toned down to nearly nonexistent). I think the intended conflict of the story is between those ways of life, their draws and implications, but even that conflict is never clearly stated. Ryke seems to move from one inevitability to another, and while he does question some of his previous views and seems to open his mind a bit, actual growth is limited in this story. I reached the end of the book feeling the world was largely unchanged from its start.
This story won the World Fantasy award for best novel in 1980. I have to admit I'm baffled, although partly that's because this is not really my thing. I prefer my fantasy to have more epic sweep and a lot more world-building. But perhaps the award was for the writing, which is evocative and fills the moments of the story with closely-observed detail. It has a distinctive, choppy feel, full of short declarative sentences:
The wind blew. Like some night creature caught in a trap, Col Istor's banner flapped on its pole. Ryke wondered where his wolfhound was. Sheltering, he hoped, in some warm and windless corner. He thrust his hands beneath his armpits to warm them. He entered the kitchen, nodding at the guard who stood there, and went toward the scullery. The kitchenboys huddled together in sleep like dogs in front of the oven. He stepped over their legs. The iron pots on the walls vibrated softly.
That's a paragraph chosen at random from the start of the book and should give you a good feel for the style.
I think I would have liked this novel better if it had followed anyone other than Ryke. Not only did I not care for him and his simplistic loyalties, he's the least interesting character in this story. Everyone else — Col, Errel, Sorren, Van, and others — is more complex and more thoughtful than he is, and I think many of them would have made a better protagonist. He spends much of the center part of the book being suspicious and closed and incurious, which means the reader can only guess at what's going on between the other characters.
I can't really recommend this book, but the writing is solid and it has some interesting properties for late 1970s fantasy, including both a frank look at the role of women in this society and same-sex relationships. If your taste in fantasy leans towards the gritty and realistic, you might like it more than I did. I may try the later books of the series in the hope that they have more interesting protagonists.
Followed by Dancers of Arun.
Rating: 6 out of 10
Following up on Then and Now ...
In quiet moments at ICFP last August, I finished teaching Propellor to generate disk images. With an emphasis on doing a whole lot with very little new code and extreme amount of code reuse.
For example, let's make a disk image with nethack on it. First, we need to define a chroot. Disk image creation reuses propellor's chroot support, described back in propelling containers. Any propellor properties can be assigned to the chroot, so it's easy to describe the system we want.
nethackChroot :: FilePath -> Chroot nethackChroot d = Chroot.debootstrapped (System (Debian Stable) "amd64") mempty d & Apt.installed ["linux-image-amd64"] & Apt.installed ["nethack-console"] & accountFor gamer & gamer `hasInsecurePassword` "hello" & gamer `hasLoginShell` "/usr/games/nethack" where gamer = User "gamer"
Now to make an image from that chroot, we just have to tell propellor where to put the image file, some partitioning information, and to make it boot using grub.
nethackImage :: RevertableProperty nethackImage = imageBuilt "/srv/images/nethack.img" nethackChroot MSDOS (grubBooted PC) [ partition EXT2 `mountedAt` "/boot" `setFlag` BootFlag , partition EXT4 `mountedAt` "/" `addFreeSpace` MegaBytes 100 , swapPartition (MegaBytes 256) ]
The disk image partitions default to being sized to fit exactly the files from the chroot that go into each partition, so, the disk image is as small as possible by default. There's a little DSL to configure the partitions. To give control over the partition size, it has some functions, like addFreeSpace and setSize. Other functions like setFlag and extended can further adjust the partitions. I think that worked out rather well; the partition specification is compact and avoids unecessary hardcoded sizes, while providing plenty of control.
By the end of ICFP, I had Propellor building complete disk images, but no boot loader installed on them.
Fast forward to today. After stuggling with some strange grub behavior, I found a working method to install grub onto a disk image.
The whole disk image feature weighs in at:
203 lines to interface with parted
88 lines to format and mount partitions
90 lines for the partition table specification DSL and partition sizing
196 lines to generate disk images
75 lines to install grub on a disk image
652 lines of code total
Which is about half the size of vmdebootstrap 1/4th the size of partman-base (probably 1/100th the size of total partman), and 1/13th the size of live-build. All of which do similar things, in ways that seem to me to be much less flexible than Propellor.
One thing I'm considering doing is extending this so Propellor can use qemu-user-static to create disk images for eg, arm. Add some u-boot setup, and this could create bootable images for arm boards. A library of configs for various arm boards could then be included in Propellor. This would be a lot easier than running the Debian Installer on an arm board.
Oh! I only just now realized that if you have a propellor host configured, like this example for my dialup gateway, leech --
leech = host "leech.kitenet.net" & os (System (Debian (Stable "jessie")) "armel") & Apt.installed ["linux-image-kirkwood", "ppp", "screen", "iftop"] & privContent "/etc/ppp/peers/provider" & privContent "/etc/ppp/pap-secrets" & Ppp.onBoot & hasPassword (User "root") & Ssh.installed
-- The host's properties can be extracted from it, using eg hostProperties leech and reused to create a disk image with the same properties as the host!
So, when my dialup gateway gets struck by lightning again, I could use this to build a disk image for its replacement:
import qualified Propellor.Property.Hardware.SheevaPlug as SheevaPlug laptop = host "darkstar.kitenet.net" & SheevaPlug.diskImage "/srv/images/leech.img" (MegaBytes 2000) (& propertyList "has all of leech's properties" (hostProperties leech))
This also means you can start with a manually built system, write down the properties it has, and iteratively run Propellor against it until you think you have a full specification of it, and then use that to generate a new, clean disk image. Nice way to transition from sysadmin days of yore to a clean declaratively specified system.
Yesterday we have released CafeOBJ 1.5.4 with a long list of changes, and many more internal changes. Documentation pages have been updated with the latest reference manual (PDF, Html) as well as some new docs on CITP (in Japanese for now) and tutorials.
To quote from our README:
CafeOBJ is a new generation algebraic specification and programming language. As a direct successor of OBJ, it inherits all its features (flexible mix-fix syntax, powerful typing system with sub-types, and sophisticated module composition system featuring various kinds of imports, parameterised modules, views for instantiating the parameters, module expressions, etc.) but it also implements new paradigms such as rewriting logic and hidden algebra, as well as their combination.Changes
Changes since the last release are as follows
- CITP changes
- new commands :ctf- and :csp-
- new command :def(ine) to turn :ctf(-) and :csp(-) into proper tactics for :apply
- new tactics nf, ct
- improvements in the online help system
- bug fixes in AC rewriting
- small changes in the set of abbreviations of the Emacs mode
- module system: require/provide can use Perl style :: separators to load modules from the libpath and its sub-directories
- term inspection (binspect) to analyze xor terms
Binary packages for Windows, MacOS, and Linux have been built, and can be found at the CafeOBJ download page. The source code can also be found on the download page, or directly from here: cafeobj-1.5.4.tar.gz.
The CafeOBJ Debian package has been updated and is already included in Debian/sid.
Macports file has also been updated, please see the above download/install page for details how to add our sources to your macport.Bug reports
If you find a bug, have suggestions, or complains, please open an issue at the issue tracker for CafeOBJ.
For other inquiries, please use email@example.com
If anybody would like to have their blog added or suggest other blogs that deserve recognition, please email me. If a blog is not entirely dedicated to SIP, it is better to use a tag or category with a dedicated RSS feed URL for SIP articles.
Articles related to Asterisk, FreeSWITCH, SIP proxies and softphones are all welcome.
Please also remember to create links to the planet sites from your own blogs and web sites, it helps solutions based on free software to become more prominent online.Streamlining planet-site management
I'm also looking for feedback about how to streamline management of the planet sites, maybe providing a self-service portal or letting people add themselves with Github pull requests. Has anybody else seen solutions for this?
A few weeks ago, I was offered the opportunity to give a guest talk in the Romanian Association for Better Software.
RABS is a group of people interested in improving the trade, and regularly hold events where invited speakers give presentations on a wide array of topics. The speakers are usually pretty high-profile, so this is quite a responsibility! To make things more interesting, much of the target audience works on enterprise software, under Windows platforms. Definitely outside my comfort zone!
Considering all this, we decided the topic was going to be about Site Reliability Engineering (SRE), concentrating on some aspects of it which I believe could be useful independently of the kind of company you are working for.
I finally gave the talk last Monday, and the audience seemed to enjoy it, so I am going to post here my notes, hopefully some other people will like it too.Why should I care?
I prepared this thinking of an audience of software engineers, so why would anyone want to hear about this idea that only seems to be about making the life of the operations people better?
The thing is, having your work as a development team supported by an SRE team will also benefit you. This is not about empowering Ops to hit you harder when things blow apart, but to have a team that is your partner. A partner that will help you grow, handle the complexities of a production environment so you can concentrate on cool features, and that will get out of the way when things are running fine.
A development team may seem to only care about adding features that will drive more and more users to your service. But an unreliable service is a service that loses users, so you should care about reliability. And what better to have a team has Reliability on their name?What is SRE?
SRE means Site Reliability Engineering, Reliability Engineering applied to "sites". Wikipedia defines Reliability Engineering as:
[..] engineering that emphasizes dependability in the life-cycle management of a product.
This is, historically, a branch of engineering that made possible to build devices that will work as expected even when their components were inherently unreliable. It focused on improving component reliability, establishing minimum requirements and expectations, and a heavy usage of statistics to predict failures and understand underlying problems.
SRE started as a concept at Google about 12 years ago, when Ben Treynor joined the company and created the SRE team from a group of 7 production engineers. There is no good definition of what Site Reliability Engineering means; while the term and some of its ideas are clearly inspired in the more traditional RE, he defines SRE with these words1:
Fundamentally, it's what happens when you ask a software engineer to design an operations function.Only hire coders
After reading that quote it is not surprising that the first item in the SRE checklist2, is to only hire people who can code properly for SRE roles. Writing software is a key part of being SRE. But this does not mean that there is no separation between development and operations, nor that SRE is a fancy(er) name for DevOps3.
It means treating operations as a software engineering problem, using software to solve problems that used to be solved by hand, implementing rigorous testing and code reviewing, and taking decisions based on data, not just hunches.
It also implies that SREs can understand the product they are supporting, and that there is a common ground and respect between SREs and software engineers (SWEs).
There are many things that make SRE what it is, some of these only make sense within a special kind of company like Google: many different development and operations teams, service growth that can't be matched by hiring, and more importantly, firm commitment from top management to implement these drastic rules.
Therefore, my focus here is not to preach on how everybody should adopt SRE, but to extract some of the most useful ideas that can be applied in a wider array of situations. Nevertheless, I will first try to give an overview of how SRE works at Google.
That's it for today. In the next post I will talk about how to end the war between developers and SysAdmins. Stay tuned!
SRE checklist extracted from Treynor's talk at SREcon14: https://www.usenix.org/conference/srecon14/technical-sessions/presentation/keys-sre ↩
By the way, I am still not sure what DevOps mean, it seems that everyone has a different definition for it. ↩