I have released whatmaps 0.0.9 a tool to check which processes map shared objects of a certain package. It can integrate into apt to automatically restart services after a security upgrade.
This release fixes the integration with recent systemd (as in Debian Jessie), makes logging more consistent and eases integration into downstream distributions. It's available in Debian Sid and Jessie and will show up in Wheezy-backports soon.
This blog is flattr enabled.
There are many things to like about the content of the lectures, beginning with some pearls and wisdom about the craft of writing software (even though this is not really a "software enginneering" book), the clarity with which the concepts are described, the Freedom-friendly aspects of the authors regarding the material that they produced and much, the breadth of the subjects covered and much more.The videos, their length, and splitting them
The course consists of 20 video files and they are all uploaded on Youtube already.
There is one thing, though: while the lectures are naturally divided into segments (the instructors took a break in after every 30 minutes or so worth of lectures), the videos corresponding to each lecture have all the segments concatenated.
To better watch them, accounting for the easier possibility to put a few of the lectures in a mobile device or to avoid fast forwarding long videos from my NAS when I am watching them on my TV (and some other factors), I decided to sit down, take notes for each video of where the breaks where, and write a simple Python script to help split the videos in segments, and, then, reencode the segments.
I decided not to take the videos from Youtube to perform my splitting activities, but, instead, to operate on one of the "sources" that the authors once had in their homepage (videos encoded in DivX and audio in MP3). The videos are still available as a torrent file (with a magnet link for the hash 650704e4439d7857a33fe4e32bcfdc2cb1db34db), with some very good souls still seeding it (I can seed it too, if desired). Alas, I have not found a source for the higher quality MPEG1 videos, but I think that the videos are legible enough to avoid bothering with a larger download.
I soon found out that there are some beneficial side-effects of splitting the videos, like not having to edit/equalize the entire audio of the videos when only a segment was bad (which is understandable, as these lectures were recorded almost 30 years ago and technology was not as advanced as things are today).
So, since I already have the split videos lying around here, I figured out that, perhaps, other people may want to download them, as they may be more convenient to watch (say, during commutes or whatever/whenever/wherever it best suits them).
Of course, uploading all the videos is going to take a while and I would only do it if people would really benefit from them. If you think so, let me know here (or if you know someone who would like the split version of the videos, spread the word).
Neil has abandoned his reputation as an RM machine, and instead concentrated on making the delayed queue as long as he can. I’m reliably informed that it’s now at a 3-year high. Steve is delighted that his reigning-in work is finally having an effect.Alcester BSP, day two is a post from: jwiltshire.org.uk | Flattr
I signed up to the CPAN Pull Request Challenge - apparently I'm entrant 170 of a few hundred.
Then I extended the Travis file to generate coverage reports, and separately realised the docs weren't quite fully complete, so fixed this and added a test.
Two of these have already been merged by the author, who was very responsive.
Part of me worries that Github is a centralized, proprietary platform that we now trust most of our software source code to. But activities such as this are surely a good thing - how much harder would it be to co-ordinate 300 volunteers to submit patches in a distributed fashion? I suppose you could do something similar with the list of Debian source packages and metadata about the upstream VCS, say...
I have gotten my first patch to the Pidgin AppArmor profile accepted upstream. One of my mentors thus suggested that I’d patch the updated profile in the Debian package myself. This is fairly easy and requires simply that one knows how to use Git.
If you want to get write access to the apparmor-profiles-extra package in Debian, you first need to request access to the Collaborative Maintenance Alioth project, collab-maint in short. This also requires setting up an account on Alioth.
Once all is set up, one can export the apparmor-profiles-extra Git repository.
If you simply want to submit a patch, it’s sufficient to clone this repository anonymously.
Otherwise, one should use the “–auth” parameter with “debcheckout”. The “debcheckout” command is part of the “devscripts” package:
debcheckout --auth apparmor-profiles-extra
Go into the apparmor-profiles-extra folder and create a new working branch:
git branch workingtitle git checkout workingtitle
Get the latest version of profiles from upstream. In “profiles”, one can edit the profiles.
The debian/README.Debian file should be edited: add what relevant changes one just imported from upstream.
Then, one could either push the branch to collab-maint:
git commit -a git push origin workingtitle
or simply submit a patch to the Debian Bug Tracking System against the apparmor-profiles-extra package.
The Debian AppArmor packaging team mailing list will receive a notification of this commit. This way, commits can be peer reviewed and merged by the team.
To keep up with GNOMEs schedule I've released krb5-auth-dialog 3.15.4. The changes of 3.15.1 and 3.15.4 include among updated translations, the replacement of deprecated GTK+ widgets, minor UI cleanups and bug fixes a header bar fix that makes us only use header bar buttons iff the desktop environment has them enabled:
This makes krb5-auth-dialog better ingtegrated into other desktops again thanks to mclasen's awesome work.
This blog is flattr enabled.
What’s that? The third edition of Link Pack of course!
Playing with Power (7 minutes, Vimeo)
A super awesome story about a stop motion animator that turned a Nintendo Power Glove into the perfect animation tool. It’s a fun, inspiring video :-). I love the Power Glove, it’s so bad.
The Power Glove – Angry Video Game Nerd – Episode 14 (12 minutes, YouTube)
On the topic of the Power Glove, here’s the now classic Angry Video Game Nerd video about it. James Rolfe is funny.
Ship Your Enemies Glitter
A rising star in the internet business landscape. You pay them $9.99 and they send an envelope full of glitter to your worst enemy. They promise it will jump into everything, as usual. Damn you glitter.
A Guide to Practical Contentment
Be happy with what you have, but understand why:
(…) if you start in this place of fixing what’s wrong with you, you keep looking for what else is wrong with you, what else you need to improve. So maybe now feel like you don’t have enough muscles, or six pack abs, or you think your calves don’t look good, or if it’s not about your body, you’ll find something else.
So it’s this never-ending cycle for your entire life. You never reach it. If you start with a place of wanting to improve yourself and feeling stuck, even if you’re constantly successful and improving, you’re always looking for happiness from external sources. You don’t find the happiness from within, so you look to other things.
The Comments Section For Every Video Where Someone Does A Pushup
Comments. From YouTube. Enough said.
“These are dips. Not pushups. In the entire history of the world, no one has ever successfully performed a pushup. They’re all just dips.”
“STOP DRIVING WITH YOUR HIPS. IF YOU’RE DOING A PUSHUP CORRECTLY, YOUR HIPS SHOULD CEASE TO EXIST.”
“You could do 100 pushups like this and it wouldn’t improve your strength at all. You’re just bending your arms.”
Self-Taught Chinese Street Photographer Tao Liu Has an Eye for Peculiar Moments
This Chinese photog uses his lunch break to snap interesting street photography. Funny selection by PetaPixel, his Flickr page has even more stuff. Even more in his photoblog.
Enrique Castro-Mendivil’s Agua Dulce photo set
Another interesting photo link. This time it’s the most popular beach in Lima, with most people coming from low income neighborhoods, it shows how fragmented the city is.
Perhaps I should say evening one, since we didn’t get going until nine or so. I have mostly been processing unblocks – 13 in all. We have a delayed upload and a downgrade in the pipeline, plus a tested diff for Django. Predictably, Neil had the one and only removal request so far.Alcester BSP, day one is a post from: jwiltshire.org.uk | Flattr
The UDD bugs interface currently knows about the following release critical bugs:
- In Total:
178 bugs affecting
- Affecting Jessie:
172 (key packages:
104) That's the number we need to get down to zero
before the release. They can be split in two big categories:
- Affecting Jessie and unstable:
128 (key packages:
80) Those need someone to find a fix, or to finish the
work to upload a fix to unstable:
- 19 bugs are tagged 'patch'. (key packages: 10) Please help by reviewing the patches, and (if you are a DD) by uploading them.
- 8 bugs are marked as done, but still affect unstable. (key packages: 5) This can happen due to missing builds on some architectures, for example. Help investigate!
- 101 bugs are neither tagged patch, nor marked done. (key packages: 65) Help make a first step towards resolution!
- Affecting Jessie only: 44 (key packages: 24) Those are already fixed in unstable, but the fix still needs to migrate to Jessie. You can help by submitting unblock requests for fixed packages, by investigating why packages do not migrate, or by reviewing submitted unblock requests.
- Affecting Jessie and unstable: 128 (key packages: 80) Those need someone to find a fix, or to finish the work to upload a fix to unstable:
- Affecting Jessie: 172 (key packages: 104) That's the number we need to get down to zero before the release. They can be split in two big categories:
How do we compare to the Squeeze release cycle?Week Squeeze Wheezy Jessie 43 284 (213+71) 468 (332+136) 319 (240+79) 44 261 (201+60) 408 (265+143) 274 (224+50) 45 261 (205+56) 425 (291+134) 295 (229+66) 46 271 (200+71) 401 (258+143) 427 (313+114) 47 283 (209+74) 366 (221+145) 342 (260+82) 48 256 (177+79) 378 (230+148) 274 (189+85) 49 256 (180+76) 360 (216+155) 226 (147+79) 50 204 (148+56) 339 (195+144) ??? 51 178 (124+54) 323 (190+133) 189 (134+55) 52 115 (78+37) 289 (190+99) 147 (112+35) 1 93 (60+33) 287 (171+116) 140 (104+36) 2 82 (46+36) 271 (162+109) 157 (124+33) 3 25 (15+10) 249 (165+84) 172 (128+44) 4 14 (8+6) 244 (176+68) 5 2 (0+2) 224 (132+92) 6 release! 212 (129+83) 7 release+1 194 (128+66) 8 release+2 206 (144+62) 9 release+3 174 (105+69) 10 release+4 120 (72+48) 11 release+5 115 (74+41) 12 release+6 93 (47+46) 13 release+7 50 (24+26) 14 release+8 51 (32+19) 15 release+9 39 (32+7) 16 release+10 20 (12+8) 17 release+11 24 (19+5) 18 release+12 2 (2+0)
As I said, I did not certain events that begun with “lea” and end with “ing” prevent me from organising a Debian/m68k hack weekend. Well, that weekend is now.
I’m too unorganised, and I spent too much time in the last few evenings to organise things so I built up a sleep deficit already ☹ and the feedback was slow. (But so are the computers.) And someone I’d have loved to come was hurt and can’t come.
On the plus side, several people I’ve long wanted to meet IRL are coming, either already today or tomorrow. I hope we all will have a lot of fun.
Legal disclaimer: “Debian/m68k” is a port of Debian™ to m68k. It used to be official, but now isn’t. It belongs to debian-ports.org, which may run on DSA hardware, but is not acknowledged by Debian at large, unfortunately. Debian is a registered trademark owned by Software in the Public Interest, Inc.
I stumbled upon this site thanks to Helga: Parable of the Polygons. On the site you can interactively find out how harmless choices can make a harmful world. I found it quite eye opening. And what most catched me but isn't part of the site is that only unhappy polygons are willing to move. Those who are just ok with their neighbourhood but not really happy about it aren't willing to move. Which made me try it out in my own way: Trying to create the most diverse possible environment by temporarily making as many polygons unhappy to find out if it's possible to make as many polygons happy in the long run as possible.
... which is actually part of the way I see my own life. I always sort-of tried to confront people to think. I mean, it's not that common that you see a by-the-looks male person wearing a skirt. And ... since I moved out in July into a small intermediate flat and thus a new neighbourhood, I found the confidence (in parts also to be attributed to the confidence built up at these fine feministic conferences) to walk my hometown in a skirt. Only on some few occations, when meeting up with friends, mostly at evening/night, but it was always a nice experience. And I only felt once uncomfortable to be honest, when there was a probably right-winged skinhead at the subway station. Too many other people around, so I tried to avoid eye contact, but it didn't feel good.
Diversity is something that society needs. In all aspects. Also within the Debian project. I believe strongly in that there can't be much of innovation and moving forward if all people do think the same direction. That only means that potential alternative paths won't even get considered, and potentially get lost. That's one of the core parts of what makes the Free Software community livid and useful. People try different approaches, and in the end there will be adopters of what they believe is the better project. Projects pop up every now and then, others starve because of loss of interest, users not picking it up, developers spending their time on other stuff, and that's absolutely fine too. There is always something to be learned even from those situations.
Speaking of diversity, there is this protest going on later today because the boss of a cafe here in Vienna considered it a good idea to kick out a lesbian couple because they kissed each other for greeting and told them that they don't have a place for their "otherness" in her traditional viennese cafe and they rather should take it to a brothel. She excused yesterday for her tone that she used, she said she should have been more relaxed—as the CEO of that cafe. Which literally means that she only exused for the tone she used in her role, but not at all for the message she transported. So meh, hope there will be many people at the protest. Yes, there is some anti discrimination law around, but that only covers the workplace, and not service areas. Welcome to Austria.
On the upside, court striked down ban on same-sex couple adoption just the other day. Hopefully there is still hope for this country. :)
cassarossa:~> time locate asdkfjhasekjrxhw locate asdkfjhasekjrxhw 19,49s user 0,46s system 82% cpu 24,071 total
It's 2015. locate still works by a linear scan through a flat file.
In December 46 work hours have been equally split among 4 paid contributors (note that Thorsten and Raphaël have actually spent more hours because they took over some hours that Holger did not do over the former months). Their reports are available:
Compared to last month, the number of paid work hours has almost not increased (we are at 48 hours per month). We still have a couple of new sponsors in the pipe but with the new year they did not complete the process yet. Hopefully next month will see a noticeable increase.
As usual, we are looking for more sponsors to reach our our minimal goal of funding the equivalent of a half-time position. Those of you who are struggling to spend money in the last quarter due to budget overrun, now is a good time to see if you want to include Debian LTS support in your 2015 budget!
In terms of security updates waiting to be handled, the situation looks similar to last month: the dla-needed.txt file lists 30 packages awaiting an update (3 more than last month), the list of open vulnerabilities in Squeeze shows about 56 affected packages in total. We do not manage to clear the backlog but it’s not getting significantly worse either.Thanks to our sponsors
- Gold sponsors:
- Silver sponsors:
- AD&D – David Ayers – IntarS Austria
- Domeneshop AS
- Trollweb Solutions
- Université Lille 3
- Bronze sponsors:
A persistent problem that I encounter with hard disks is the capacity limit. If only hard disks could expand like the Tardis.
My current setup at home involves a HP Microserver. It has four drive bays carrying two SSDs (for home directories) and two Western Digital RE4 2TB drives for bulk data storage (photos, source tarballs and other things that don't change often). Each pair of drives is mirrored. I chose the RE4 because I use RAID1 and they offer good performance and error recovery control which is useful in any RAID scenario.
When I put in the 2TB drives, I created a 1TB partition on each for Linux md RAID1 and another 1TB partition on each for BtrFs.
Later I added the SSDs and I chose BtrFs again as it had been working well for me.Where to from here?
Since getting a 36 megapixel DSLR that produces 100MB raw images and 20MB JPEGs I've been filling up that 2TB faster than I could have ever imagined.
I've also noticed that vendors are offering much bigger NAS and archive disks so I'm tempted to upgrade.
First I looked at the Seagate Archive 8TB drives. 2TB bigger than the nearest competition. Discussion on Reddit suggests they don't have Error Recovery Control / TLER however and that leaves me feeling they are not the right solution for me.
Then I had a look at WD Red. Slightly less performant than the RE4 drives I run now, but with the possibility of 6TB per drive and a little cheaper. Apparently they have TLER though, just like the RE4 and other enterprise drives.Will 6 or 8TB create new problems?
This all leaves me scratching my head and wondering about a couple of things though:
- Will I run into trouble with the firmware in my HP Microserver if I try to use such a big disk?
- Should I run the whole thing with BtrFs and how well will it work at this scale?
- Should I avoid the WD Red and stick with RE4 or similar drives from Seagate or elsehwere?
If anybody can share any feedback it would be really welcome.
After releasing Weblate 2.0 with Bootstrap based UI, there was still lot of things to improve. Weblate 2.1 brought more consistency in using buttons with colors and icons. Weblate 2.2 will bring some improvements in other graphics elements.
One of thing which was for quite long in our issue tracker is to provide own renderer for SVG status badge. So far Weblate has offered either PNG badge or external SVG rendered by shields.io. Relying on external service was not good in a long term and also caused requests to third party server on many pages, what could be considered bad privacy wise.
Since this week, Weblate can render SVG badge on it's own and they are also matching current style used by other services (eg. Travis CI):
If you're running Spamassassin on Debian or Ubuntu, have you enabled automatic rule updates? If not, why not? If possible, you should enable this feature. It should be as simple as setting "CRON=1" in /etc/default/spamassassin. If you choose not to enable this feature, I'd really like to hear why. In particular, I'm thinking about changing the default behavior of the Spamassassin packages such that automatic rule updates are enabled, and I'd like to know if (and why) anybody opposes this.
Spamassassin hasn't been providing rules as part of the upstream package for some time. In Debian, we include a snapshot of the ruleset from an essentially arbitrary point in time in our packages. We do this so Spamassassin will work "out of the box" on Debian systems. People who install spamassassin from source must download rules using spamassassin's updates channel. The typical way to use this service is to use cron or something similar to periodically check for rule changes via this service. This allows the anti-spam community to quickly adapt to changes in spammer tactics, and for you to actually benefit from their work by taking advantage of their newer, presumably more accurate, rules. It also allows for quick reaction to issues such as the one described in bug 738872 and 774768.
If we do change the default, there are a couple of possible approaches we could take. The simplest would be to simply change the default value of the CRON variable in /etc/default/spamassassin. Perhaps a cleaner approach would be to provide a "spamassassin-autoupdates" package that would simply provide the cron job and a simple wrapper program to perform the updates. The Spamassassin package would then specify a Recommends relationship with this package, thus providing the default enabled behavior while still providing a clear and simple mechanism to disable it.
Unfortunately I could not go on stage at the 31st Chaos Communication Congress to present reproducible builds in Debian alongside Mike Perry from the Tor Project and Seth Schoen from the Electronic Frontier Foundation. I've tried to make it up for it, though… and we have made amazing progress.Wiki reorganization
What was a massive and frightening wiki page now looks really more welcoming:
Depending on what one is looking for, it should be much easier to find. There's now a high-level status overview given on the landing page, maintainers can learn how to make their packages reproducible, enthusiasts can more easily find what can help the project, and we have even started writing some history..buildinfo for all packages
New year's eve saw me hacking Perl to write dpkg-genbuildinfo. Similar to dpkg-genchanges, it's run by dpkg-buildpackage to produce .buildinfo control files. This is where the build environment, and hash of source and binary packages are recorded. This script, integrated with dpkg, replace the previous debhelper interim solution written by Niko Tyni.
We used to fix mtimes in control.tar and data.tar using a specific addition to debhelper named dh_fixmtimes. To better support the ALWAYS_EXCLUDE environment variable and for pragramtic reasons, we moved the process in dh_builddeb.
Both changes were quickly pushed to our continuous integration platform. Before, only packages using dh would create a .buildinfo and thus eventually be considered reproducible. With these modifications, many more packages had their chance… and this shows:
Yes, with our experimental toolchain we are now at more than eighty percent! That's more than 17200 source packages!srebuild
Given a .buildinfo file, it first finds a timestamp of Debian Sid from snapshot.debian.org which contains the requested packages in their exact versions. It then runs sbuild with the right architecture as given by the .buildinfo file and the right base system to upgrade from, as given by the version of the base-files package version in the .buildinfo file. Using two hooks it will install the right package versions and verify that the installed packages are in the right version before the build starts.Understanding problems
Over 1700 packages have now been reviewed to understand why build results could not be reproduced on our experimental platform. The variations between the two builds are currently limited to time and file ordering, but this still has uncovered many problems. There are still toolchain fixes to be made (more than 180 packages for the PHP registry) which can make many packages reproducible at once, but others like C pre-processor macros will require many individual changes.
debbindiff, the main tool used to understand differences, has gained support for .udeb, TrueType and OpenType fonts, PNG and PDF files. It's less likely to crash on problems with encoding or external tool. But most importantly for large package, it has been made a lot faster, thanks to Reiner Herrmann and Helmut Grohne. Helmut has also been able to spot cross-compilation issues by using debbindiff!Targeting our efforts
It gives warm fuzzy feelings to hit the 80% mark, but it would be a bit irrelevant if this would not concern packages that matter. Thankfully, Holger worked on producing statistics for more specific package sets. Mattia Rizzolo has also done great work to improve the scripts generating the various pages visible on reproducible.debian.net.
All essential and build-esential packages, except gcc and bash, are considered reproducible or have patches ready. After some lengthy builds, I also managed to come up with a patch to make linux build reproducibly.Miscellaneous
After my initial attempt to modify r-base to remove a timestamp in R packages, Dirk Eddelbuettel discussed the issue with upstream and came up with a better patch. The latter has already been merged upstream!
Identifiers generated by xsltproc have also been an issue. After reviewing my initial patch, Andrew Awyer came up with a much nicer solution. Its potential performance implications need to be evaluated before submission, though.
Chris West has been working on packages built with Maven amongst other things.
PDF generated by GhostScript, another painful source of troubles, is being worked on by Peter De Wachter.
Holger got X.509 certificates signed by the CA cartel for jenkins.debian.net and reproducible.debian.net. No more scary security messages now. Let's hope next year we will be able to get certificates through Let's Encrypt!Let's make a difference together
As you can imagine with all that happened in the past weeks, the #debian-reproducible IRC channel has been a cool place to hang out. It's very energizing to get together and share contributions, exchange tips and discuss hardest points. Mandatory quote:
* h01ger is very happy to see again and again how this is a nice learning circle...! i've learned a whole lot here too... in just 3 months... and its going on...!
Reproducible builds are not going to change anything for most of our users. They simply don't care how they get software on their computer. But they care to get the right software without having to worry about it. That's our responsibility, as developpers. Enabling users to trust their software is important and a major contribution, we as Debian, can make to the wider free software movement. Once Jessie is released, we should make a collective effort to make reproducible builds an highlight of our next release.
Last week, I visited London for the January Docker meetup, which was the first time I'd attended this group.
It was a talk-oriented format, with around 200 attendees packed into Shoreditch Village Hall; free pizza and beer was provided thanks to the sponsors, which was awesome (and makes logistics easier when you're travelling there from work).
There were three talks.
First, Andrew Martin from British Gas spoke about how they use Docker for testing and continuous deployment of their Node.js microservices - buzzword bingo! But it's helpful to see how companies approach these things.
Second, Johan Euphrosine from Google gave a short demo of Google Cloud Platform for running Docker containers (mostly around Container Engine, but also briefly App Engine). This was relevant to my interests, but I'd already seen this sort of talk online.
Third, Dan Williams presented his holiday photos featuring a journey on a container ship, which wins points from me for liberal interpretation of the meetup topic, and was genuinely very entertaining/interesting - I just regret having to leave to catch a train halfway through.
In summary, this was worth attending, but as someone just getting started with containers I'd love some sort of smaller meetings with opportunities for interaction/activity. There's such a variety of people/use cases for Docker that I'm not sure how much everyone had in common with each other; it would be interesting to find out.