Planet Debian

Subscribe to Planet Debian feed
Planet Debian - http://planet.debian.org/
Updated: 2 hours 24 min ago

Gunnar Wolf: DebConf17 Key Signing Party: You are here↓

5 August, 2017 - 07:23

I ran my little analysis program written last year to provide a nice map on the DebConf17 key signing party, based on the . What will you find if you go there?

  • A list of all the people that will take part of the KSP
  • Your key's situation relative to the KSP keyring

As an example, here is my location on the map (click on the graph to enlarge):

Its main use? It will help you find what clusters are you better linked with - And who you have not cross-signed with. Some people have signed you but you didn't sign them? Or the other way around? Whom should you approach to make the keyring better connected? Can you spot some attendees who are islands and can get some help getting better connected to our keyring? Please go ahead and do it!

PS— There are four keys that are mentioned in the DebConf17 Keysigning Party Names file I used to build this from: 0xE8446B4AC8C77261, 0x485E1BD3AE76CB72, 0x4618E4C700000173, E267B052364F028D. The public keyserver network does not know about them. If you control one of those keys and you want me to run my script again to include it, please send it to the keyservers and mail me. If your key is not in the keyservers, nobody will be able to sign it!

Daniel Silverstone: USB Device Stacks, on RTFM

4 August, 2017 - 23:05

I have been spending time with Jorge Aparicio's RTFM for Cortex M3 framework for writing Rust to target Cortex-M3 devices from Arm (and particularly the STM32F103 from ST Microelectronics). Jorge's work in this area has been of interest to me ever since I discovered him working on this stuff a while ago. I am very tempted by the idea of being able to implement code for the STM32 with the guarantees of Rust and the language features which I have come to love such as the trait system.

I have been thinking to myself that, while I admire and appreciate the work done on the GNUK, I would like to, personally, have a go at implementing some kind of security token on an STM32 as a USB device. And with the advent of the RTFM for M3 work, and Jorge's magical tooling to make it easier to access and control the registers on an M3 microcontroller, I figured it'd be super-nice to do this in Rust, with all the advantages that entails in terms of isolating unsafe behaviour and generally having the potential to be more easily verified as not misbehaving.

To do this though, means that I need a USB device stack which will work in the RTFM framework. Sadly it seems that, thus-far, only Jorge has been working on drivers for any of the M3 devices his framework supports. And one person can only do so much. So, in my infinite madness, I decided I should investigate the complexity of writing a USB device stack in Rust for the RTFM/M3 framework. (Why I thought this was a good idea is lost to the mists of late night Googling, but hey, it might make a good talk at the next conference I go to). As such, this blog post, and further ones along these lines, will serve as a partial tour of what I'm up to, and a partial aide-memoir for me about learning USB. If I get something horribly wrong, please DO contact me to correct me, otherwise I'll just continue to be wrong. If I've simplified something but it's still strictly correct, just let me know if it's an oversimplification since in a lot of cases there's no point in me putting the full details into a blog posting. I will mostly be considering USB2.0 protocol details but only really for low and full speed devices. (The hardware I'm targetting does low-speed and full-speed, but not high-speed. Though some similar HW does high-speed too, I don't have any to hand right now)

A brief introduction to USB

In order to go much further, I needed a grounding in USB. It's a multi-layer protocol as you might expect, though we can probably ignore the actual electrical layer since any device we might hope to support will have to have a hardware block to deal with that. We will however need to consider the packet layer (since that will inform how the hardware block is implemented and thus its interface) and then the higher level protocols on top.

USB is a deliberately asymmetric protocol. Devices are meant to be significantly easier to implement, both in terms of hardware and software, as compared with hosts. As such, despite some STM32s having OTG ports, I have no intention of supporting host mode at this time.

USB is arranged into a set of busses which are, at least in the USB1.1 case, broadcast domains. As such, each device has an address assigned to it by the host during an early phase called 'configuration'. Once the address is assigned, the device is expected to only ever respond to messages addressed to it. Note that since everything is asymmetric in USB, the device can't send messages on its own, but has to be asked for them by the host, and as such the addressing is always from host toward device.

USB devices then expose a number of endpoints through which communication can flow IN to the host or OUT to the device. Endpoints are not bidirectional, but the in and out endpoints do overlap in numbering. There is a special pair of endpoints, IN0 and OUT0 which, between them, form what I will call the device control endpoints. The device control endpoints are important since every USB device MUST implement them, and there are a number of well defined messages which pass over them to control the USB device. In theory a bare minimum USB device would implement only the device control endpoints.

Configurations, and Classes, and Interfaces, Oh My!

In order for the host to understand what the USB device is, and what it is capable of, part of the device control endpoints' responsibility is to provide a set of descriptors which describe the device. These descriptors form a heirarchy and are then glommed together into a big lump of data which the host can download from the device in order to decide what it is and how to use it. Because of various historical reasons, where a multi-byte value is used, they are defined to be little-endian, though there are some BCD fields. Descriptors always start with a length byte and a type byte because that way the host can parse/skip as necessary, with ease.

The first descriptor is the device descriptor, is a big one, and looks like this:

Device Descriptor Field Name Byte start Byte length Encoding Meaning bLength 0 1 Number Size of the descriptor in bytes (18) bDescriptorType 1 1 Constant Device Descriptor (0x01) bcdUSB 2 2 BCD USB spec version compiled with bDeviceClass 4 1 Class Code, assigned by USB org (0 means "Look at interface descriptors", common value is 2 for CDC) bDeviceSubClass 5 1 SubClass Code, assigned by USB org (usually 0) bDeviceProtocol 6 1 Protocol Code, assigned by USB org (usually 0) bMaxPacketSize 7 1 Number Max packet size for IN0/OUT0 (Valid are 8, 16, 32, 64) idVendor 8 2 ID 16bit Vendor ID (Assigned by USB org) idProduct 10 2 ID 16bit Product ID (Assigned by manufacturer) bcdDevice 12 2 BCD Device version number (same encoding as bcdUSB) iManufacturer 14 1 Index String index of manufacturer name (0 if unavailable) iProduct 15 1 Index String index of product name (0 if unavailable) iSerialNumber 16 1 Index String index of device serial number (0 if unavailable) bNumConfigurations 17 1 Number Count of configurations the device has.

This looks quite complex, but breaks down into a relatively simple two halves. The first eight bytes carries everything necessary for the host to be able to configure itself and the device control endpoints properly in order to communicate effectively. Since eight bytes is the bare minimum a device must be able to transmit in one go, the host can guarantee to get those, and they tell it what kind of device it is, what USB protocol it supports, and what the maximum transfer size is for its device control endpoints.

The encoding of the bcdUSB and bcdDevice fields is interesting too. It is of the form 0xMMmm where MM is the major number, mm the minor. So USB2.0 is encoded as 0x0200, USB1.1 as 0x0110 etc. If the device version is 17.36 then that'd be 0x1736.

Other fields of note are bDeviceClass which can be 0 meaning that interfaces will specify their classes, and idVendor/idProduct which between them form the primary way for the specific USB device to be identified. The Index fields are indices into a string table which we'll look at later. For now it's enough to know that wherever a string index is needed, 0 can be provided to mean "no string here".

The last field is bNumConfigurations and this indicates the number of ways in which this device might function. A USB device can provide any number of these configurations, though typically only one is provided. If the host wishes to switch between configurations then it will have to effectively entirely quiesce and reset the device.

The next kind of descriptor is the configuration descriptor. This one is much shorter, but starts with the same two fields:

Configuration Descriptor Field Name Byte start Byte length Encoding Meaning bLength 0 1 Number Size of the descriptor in bytes (9) bDescriptorType 1 1 Constant Configuration Descriptor (0x02) wTotalLength 2 2 Number Size of the configuration in bytes, in total bNumInterfaces 4 1 Number The number of interfaces in this configuration bConfigurationValue 5 1 Number The value to use to select this configuration iConfiguration 6 1 Index The name of this configuration (0 for unavailable) bmAttributes 7 1 Bitmap Attributes field (see below) bMaxPower 8 1 Number Maximum bus power this configuration will draw (in 2mA increments)

An important field to consider here is the bmAttributes field which tells the host some useful information. Bit 7 must be set, bit 6 is set if the device would be self-powered in this configuration, bit 5 indicates that the device would like to be able to wake the host from sleep mode, and bits 4 to 0 must be unset.

The bMaxPower field is interesting because it encodes the power draw of the device (when set to this configuration). USB allows for up to 100mA of draw per device when it isn't yet configured, and up to 500mA when configured. The value may be used to decide if it's sensible to configure a device if the host is in a low power situation. Typically this field will be set to 50 to indicate the nominal 100mA is fine, or 250 to request the full 500mA.

Finally, the wTotalLength field is interesting because it tells the host the total length of this configuration, including all the interface and endpoint descriptors which make it up. With this field, the host can allocate enough RAM to fetch the entire configuration descriptor block at once, simplifying matters dramatically for it.

Each configuration has one or more interfaces. The interfaces group some endpoints together into a logical function. For example a configuration for a multifunction scanner/fax/printer might have an interface for the scanner function, one for the fax, and one for the printer. Endpoints are not shared among interfaces, so when building this table, be careful.

Next, logically, come the interface descriptors:

Interface Descriptor Field Name Byte start Byte length Encoding Meaning bLength 0 1 Number Size of the descriptor in bytes (9) bDescriptorType 1 1 Constant Interface Descriptor (0x04) bInterfaceNumber 2 1 Number The number of the interface bAlternateSetting 3 1 Number The interface alternate index bNumEndpoints 4 1 Number The number of endpoints in this interface bInterfaceClass 5 1 Class The interface class (USB Org defined) bInterfaceSubClass 6 1 SubClass The interface subclass (USB Org defined) bInterfaceProtocol 7 1 Protocol The interface protocol (USB Org defined) iInterface 8 1 Index The name of the interface (or 0 if not provided)

The important values here are the class/subclass/protocol fields which provide a lot of information to the host about what the interface is. If the class is a USB Org defined one (e.g. 0x02 for Communications Device Class) then the host may already have drivers designed to work with the interface meaning that the device manufacturer doesn't have to provide host drivers.

The bInterfaceNumber is used by the host to indicate this interface when sending messages, and the bAlternateSetting is a way to vary interfaces. Two interfaces with the came bInterfaceNumber but different bAlternateSettings can be switched between (like configurations, but) without resetting the device.

Hopefully the rest of this descriptor is self-evident by now.

The next descriptor kind is endpoint descriptors:

Endpoint Descriptor Field Name Byte start Byte length Encoding Meaning bLength 0 1 Number Size of the descriptor in bytes (7) bDescriptorType 1 1 Constant Endpoint Descriptor (0x05) bEndpointAddress 2 1 Endpoint Endpoint address (see below) bmAttributes 3 1 Bitmap Endpoint attributes (see below) wMaxPacketSize 4 2 Number Maximum packet size this endpoint can send/receive bInterval 6 1 Number Interval for polling endpoint (in frames)

The bEndpointAddress is a 4 bit endpoint number (so there're 16 endpoint indices) and a bit to indicate IN vs. OUT. Bit 7 is the direction marker and bits 3 to 0 are the endpoint number. This means there are 32 endpoints in total, 16 in each direction, 2 of which are reserved (IN0 and OUT0) giving 30 endpoints available for interfaces to use in any given configuration. The bmAttributes bitmap covers the transfer type of the endpoint (more below), and the bInterval is an interval measured in frames (1ms for low or full speed, 125µs in high speed). bInterval is only valid for some endpoint types.

The final descriptor kind is for the strings which we've seen indices for throughout the above. String descriptors have two forms:

String Descriptor (index zero) Field Name Byte start Byte length Encoding Meaning bLength 0 1 Number Size of the descriptor in bytes (variable) bDescriptorType 1 1 Constant String Descriptor (0x03) wLangID[0] 2 2 Number Language code zero (e.g. 0x0409 for en_US) wLangID[n] 4.. 2 Number Language code n ...

This form (for descriptor 0) is that of a series of language IDs supported by the device. The device may support any number of languages. When the host requests a string descriptor, it will supply both the index of the string and also the language id it desires (from the list available in string descriptor zero). The host can tell how many language IDs are available simply by dividing bLength by 2 and subtracting 1 for the two header bytes.

And for string descriptors of an index greater than zero:

String Descriptor (index greater than zero) Field Name Byte start Byte length Encoding Meaning bLength 0 1 Number Size of the descriptor in bytes (variable) bDescriptorType 1 1 Constant String Descriptor (0x03) bString 2.. .. Unicode The string, in "unicode" format

This second form of the string descriptor is simply the the string is in what the USB spec calls 'Unicode' format which is, as of 2005, defined to be UTF16-LE without a BOM or terminator.

Since string descriptors are of a variable length, the host must request strings in two transactions. First a request for 2 bytes is sent, retrieving the bLength and bDescriptorType fields which can be checked and memory allocated. Then a request for bLength bytes can be sent to retrieve the entire string descriptor.

Putting that all together

Phew, this is getting to be quite a long posting, so I'm going to leave this here and in my next post I'll talk about how the host and device pass packets to get all that information to the host, and how it gets used.

Michal Čihař: Changes to Docker container for Weblate

4 August, 2017 - 17:00

I've made several changes to the Weblate Docker container which are worth mentioning today.

First of all if you are still using nijel/weblate, you should switch to weblate/weblate. They both currently share same configuration, but it might happen that some future updates will go to the weblate owned container only.

Now back to the container changes. Since beginning we were using Django built in server. That's fine for development purposes, but it really doesn't work that well in production as it can handle only one request at time. Therefore we've switched to more robust approach using nginx + uwsgi + supervisor.

Thanks to this, the docker-compose no longer needs separate nginx server as everything is now sanely handled within the weblate container itself.

Filed under: Debian English Gammu phpMyAdmin SUSE Weblate

Dirk Eddelbuettel: R for System Adminstration

4 August, 2017 - 10:33

Just getting back from the most fun meetup I have been to in quite some time: episode 23 (by their count) of Open Source Open Mic hosted by Matt Godbolt and Joe Walnes here in Chicago. Nothing but a sequence of lightning talks. Plus beer and pizza. Sounds awesome? It was!

We had fantastic talks across at least half a dozen languages, covering both new-ish (Pony) and interesting ones such (Rust, Go, ...) plus of course some Javascript and some Python, no Java (yay!) and a few batshit crazy things like a self-hosting database in its own (shell) code, a terminal gif viewer (!!), and more. And it gave me an opportunity to quickly (one evening and morning commute) jam out a presentation about what is in the title: R for system administration.

And I am only half-joking. I had used R a couple of years ago when I needed to select, subset, modify, ... a large number of image files given some timestamp and filename patterns. And given how well R works in a vectorised manner with both regular expressions and timestamps, as well as on top of essentially all standard POSIX-style operating system / file-system functions, I picked up that thread again on the problem of ... cleaning up the file storage underlying CRANberries which by now has well over fifty-seven thousand (!!) tarballs of CRAN packages based on now ten years of CRANberries. So I showed how to prune this in essentially half a dozen lines of R (and data.table code), plus some motivation---all just right for a lightning talk. Seemingly the talk went well enough as quite a few folks gave a thumbs up and compliments over beers afterwards.

But see for yourself as the slides are now uploaded to my standard talks page.

My thanks to Matt and Joe for organizing the meetup. I think I will be back.

Joey Hess: home power monitoring

4 August, 2017 - 00:38

For years I've recorded solar panel data by hand. Filled two notebooks with columns of figures. My new charge controller, an EPsolar Tracer-BN, finally let me automate it.

morning activity; by 8 am the sun is still behind the hill but, 16 watts are being produced, and by 11:30 am, the battery bank is full

You can explore my home power data here: http://homepower.joeyh.name/
(click and drag to zoom)

The web interface loads the RRD files into a web browser using javascriptRRD. I wrote a haskell program that drives the epsolar-tracer python library to poll for data, and stores it in RRD files. Could have used collectd or something, but the interface to the charge controller is currently a bit flakey and I have to be careful about retries and polling frequencies. Also I wanted full control over how much data is stored in the RRD files.

Full source code

Daniel Silverstone: Gitano 1.1

3 August, 2017 - 23:34

Today marks the release of Gitano 1.1. Richard(s) and I have spent quite a lot of time and effort on this release, and there's plenty of good stuff in it. We also released new versions of Lace, Supple, Luxio, and Gall to go alongside it, with bugfixes and improvements.

At this point, I intend to take a short break from Gitano to investigate some Rust-on-STM32 stuff, and then perhaps do some NetSurf work too.

Jeremy Bicha: Link: Ubuntu @ GUADEC 2017 and plans for GNOME Shell migration

3 August, 2017 - 22:23

Since Didier Roche’s blog is not on Planet GNOME or Planet Debian and I think his post is of widespread interest, I’m linking to it here. Enjoy!

Ubuntu @ GUADEC 2017 and plans for GNOME Shell migration

Elena 'valhalla' Grandi: Debian Day in Varese

3 August, 2017 - 14:54
Debian Day in Varese

I'm stuck home instead of being able to go to DebConf, but that doesn't mean that Debian Day will be left uncelebrated!

Since many of the locals are away for the holidays, we of @Gruppo Linux Como and @LIFO aren't going to organize a full day of celebrations, but at the very least we are meeting for a dinner in Varese, at some restaurant that will be open on that date.

Everybody is welcome: to join us please add your name (nickname or identifier of any kind, as long as it fits in the box) on https://dudle.inf.tu-dresden.de/debdayvarese2017/ before thursday, August 10th, so that we can
get a reservation at the restaurant.

Michal Čihař: Going to DebConf17

3 August, 2017 - 11:00

After fours years, I will again make it to DebConf, I'm looking forward to meet many great people, so if you want to meet and happen to be in Montreal next week come and say hello to me :-).

It seems I've settled down on four year schedule - I've attended DebConf09 and DebConf13 so far. Let's see if next one will come in 2021 or earlier.

Filed under: Debian English Gammu phpMyAdmin Weblate

Markus Koschany: My Free Software Activities in July 2017

3 August, 2017 - 06:06

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you’re interested in  Java, Games and LTS topics, this might be interesting for you.

Debian Games
  • I backported freeciv, freeorion and minetest to stretch-backports.
  • The bug fix (#866378) for 3dchess also landed in Stretch and Jessie.
  • I sponsored Lugaru for Vincent Prat and Martin Erik Werner, a really cool 3D fighting game featuring a rabbit. The game is dfsg-free now and will replace openlugaru.
  • I uploaded fifechan to unstable and packaged new upstream versions of fife, unknown-horizons, adonthell-data and hyperrogue.
  • I fixed bugs in bloboats (#864534), lordsawar (RC #866988), kraptor (#826423), pathogen (#845991), fretsonfire (#866426), blockout2 (#826416), boswars (#827112), kanatest (RC #868315, fix also backported to Stretch), overgod (#827114), morris (#829948, #721834, #862224), mousetrap (#726842), alsoft-conf (#784052, #562898) and nikwi (#835625)
  • I uploaded a new revision of clanlib and teg fixing Perl transition bugs. The patches were provided by gregor herrmann. I added myself to Uploaders in case of teg because the package was missing a human maintainer.
  • I adopted trackballs after I discovered #868983 where Henrique de Moraes Holschuh called attention to a new fork of Trackballs. The current version was broken and unplayable and it was only a matter of time before the game was removed from Debian. I could fix a couple of bugs, forwarded some issues upstream and I believe a nice game was saved.
  • I uploaded Bullet 2.86.1 to unstable and completed another Bullet transition.
Debian Java Debian LTS

This was my seventeenth month as a paid contributor and I have been paid to work 23,5 hours on Debian LTS, a project started by Raphaël Hertzog. In that time I did the following:

  • From 24. July until 31. July I was in charge of our LTS frontdesk. I triaged bugs in tinyproxy, varnish, freerdp, ghostscript, gcc-4.6, gcc-4.7, fontforge, teamspeak-server, teamspeak-client, qpdf, nvidia-graphics-drivers and sipcrack. I also pinged Diego Biurrun for more information about the next libav update and replied to questions on the debian-lts mailing list and LTS IRC channel.
  • DLA 1034-1. Issued a security update for php5 fixing 5 CVE. I discussed CVE-2017-11362 with the security team. We came to the conclusion that it was no security issue but just a normal bug.
  • DLA 1036-1. Issued a security update for gsoap fixing 1 CVE.
  • DLA 1037-1. Issued a security update for catdoc fixing 1 CVE.
  • DLA 613-2. Issued a regression update for roundcube.
  • DLA 1045-1. Issued a security update for graphicsmagick fixing 10 CVE.
  • DLA 1047-1. Issued a security update for supervisor fixing 1 CVE.
  • DLA-1048-1.  Issued a security update for ghostscript fixing 8 CVE.
Non-maintainer upload
  • I uploaded the security fix for spice to unstable which was already fixed in Stretch and earlier versions.

Thanks for reading and see you next time.

Steve Kemp: So I did a thing, then another thing.

3 August, 2017 - 04:00

So I did start a project, to write a puppet-dashboard, it is functionally complete, but the next step is to allow me to raise alerts based on failing runs of puppet - in real-time.

(i.e. Now that I have a dashboard I wish to not use it. I want to be alerted to failures, without having to remember to go look for them. Something puppet-dashboard can't do ..)

In other news a while back I slipped in a casual note about having a brain scan done, here in Sunny Helsinki a while back.

One of the cool things about that experience, in addition to being told I wasn't going to drop dead that particular day, was that the radiologist told me that I could pay €25 to get a copy of my brain data in DICOM format.

I've not yet played with this very much, but I couldn't resist a brief animation:

  • See my brain.
    • Not the best quality, or the best detail, but damn. It is my brain.
    • I shall do better with more experimentation I think.

Markus Koschany: PDFsam: How to upgrade a Maven application for Debian

3 August, 2017 - 00:34

In the coming weeks and months I intend to write a mini series about packaging Java software for Debian. The following article basically starts in the middle of this journey because the PDFsam upgrade is still fresh in my mind. It requires some preexisting knowledge about build tools like Maven and some Java terminology. But do not fear. Hopefully it will make sense in the end when all pieces fall into place.

A month ago I decided to upgrade PDFsam, a Java application to split, merge, extract, mix and rotate PDF documents. The current version 1.1.4 is already seven years old and uses Ant as its build system. Unfortunately up to now nobody was interested enough to invest the time to upgrade it to the latest version. A quick internet search unveils that the current sources can be found on github.com. Another brief look reveals we are dealing with a Maven project here because we can find a pom.xml file in the root directory and there is no sign of Ant’s typical build.xml file anymore. Here are some general tips how to proceed from this point by using the PDFsam upgrade as an example.

Find out how many new dependencies you really need

The pom.xml file declares its dependencies in the <dependencies> section. It is good practice to inspect the pom.xml file and determine how much work will be required to upgrade the package. A seasoned Java packager will quickly find common dependencies like Hibernate or the Apache Commons libraries. Fortunately for you they are already packaged in Debian because a lot of projects depend on them. If you are unsure what is and what is not packaged for Debian, tracker.debian.org and codesearch.debian.net are useful tools to search for those packages. If in doubt just ask on debian-java@lists.debian.org. There is no automagical tool (yet) to find out what dependencies are really new (we talk about mh_make soon) but if you use the aforementioned tools and websites you will notice that in June 2017 one could not find the following artifacts: fontawesomefx, eventstudio, sejda-* and jackson-jr-objects. There are also jdepend and testFx but notice they are marked as <scope>test</scope> meaning they are only required if you would like to run upstream’s test suite as well. For the sake of simplicity, it is best to ignore them for now and to focus on packaging only dependencies which are really needed to compile the application. Test dependencies can always be added later.

This pom.xml investigation leads us to the following conclusion: PDFsam depends on Sejda, a PDF library. Basically Sejda is the product of a major refactoring that happened years ago and allows upstream to develop PDFsam faster and in multiple directions. For Debian packagers it is quite clear now that the “upgrade” of PDFsam is in reality more like packaging a completely new application. The inspection of Sejda’s pom.xml file (another Maven project) reveals we also have to package imgscalr, Twelvemonkeys and SAMBox. We continue with these pom.xml analyses and end up with these new source packages: jackson-jr, libimgscalr-java, libsambox-java, libsejda-java, libsejda-injector-java, libsejda-io-java, libsejda-eventstudio-java, libtwelvemonkeys-java, fontawesomefx and libpdfbox2-java. Later I also discovered that gettext-maven-plugin was also required.

This was not obvious at first glance if you only check the pom.xml in the root directory but PDFsam and Sejda are multi-module projects! In this case every subdirectory (module) contains another pom.xml with additional information, so ideally you should check those too before you decide to start with your packaging. But don’t worry it is often possible to ignore modules with a simple –ignore  rule inside your debian/*.poms file. The package will have less functionality but it can be still useful if you only need a subset of the modules. Of course in this case ignoring the gettext-maven-plugin artifact would result in a runtime error. C’est la vie.

A brief remark about Java package names: Java library packages must be named like libXXX-java. This is important for binary packages to avoid naming collisions. We are more tolerant when it comes to source package names but in general we recommend to use the exact same name as for the binary package. There are exceptions like prefixing source packages with their well known project name like jackson-XXX or jboss-XXX but this should only be used when there are already existing packages that use such a naming scheme. If in doubt, talk to us.

mh_make or how to quickly generate an initial debian directory

Packaging a Maven library is usually not very difficult even if it consists of multiple modules. The tricky part is to get the maven.rules, maven.IgnoreRules and your *.poms file right but debian/rules often only consists of a single dh line and the rest is finding the build-dependencies and adding them to debian/control.

A small tool called mh_make, which is included in maven-debian-helper, can lend you a helping hand. The tool is not perfect yet. It requires that most build-dependencies are already installed on your local system, otherwise it won’t create the initial debian directory and will only produce some unfinished (but in some cases still useful) files.

A rule of thumb is to start with a package that does not depend on any other new dependency and requires the fewest build-dependencies.  I have chosen libtwelvemonkeys-java because it was the simplest package and met the aforementioned criteria.

Here is how mh_make looks like in action. (The animated GIF was created with Byzanz) First of all download the release tarball, unpack it and run mh_make inside the root directory.

Ok, what is happening here? First you can choose a source and binary package name. Then disable the tests and don’t run javadoc to create the documentation. This will simplify things a little.  Tests and javadoc settings can be added later. Choose the version you want to package and then you can basically follow the default recommendations and confirm them by hitting the Enter key. Throughout the project we choose to transform the upstream version with the symbolic “debian” version. Remember that Java/Maven is version-centric. This will ensure that our Maven dependencies are always satisfied later and we can simply upgrade our Maven libraries and don’t have to change the versions by hand in various pom.xml files; maven-debian-helper will automatically transform them for us to “debian”. Enable all modules. If you choose not to, you can select each module individually. Note that later on some of the required build-dependencies cannot be found because they are either not installed (libjmagick6-java) or they cannot be found in Debian’s Maven repository under /usr/share/maven-repo.  You can fix this by entering a substitution rule or, as I did in this case, you can just ignore these artifacts for now. They will be added to maven.IgnoreRules. In order to successfully compile your program you have to remove them from this file later again, create the correct substitution rule in maven.rules and add the missing build-dependencies to debian/control. For now we just want to quickly create our initial debian directory.

If everything went as planned a complete debian directory should be visible in your root directory. The only thing left is to fix the substitution rule for the Servlet API 3.1. Add libservlet3.1-java to Build-Depends and the following rule to maven.rules:

javax.servlet s/servlet-api/javax.servlet-api/ * s/.*/3.1/ * *
s/javax.servlet/javax.servlet.jsp/ s/jsp-api/javax.servlet.jsp-api/ * s/.*/2.3/ * *

The maven.rules file consists of multiple rows separated by six columns. The values represent groupId, artifactId, type, version number and two fields which I never use. You can just use an asterisk to match any value. Every value can be substituted. This is necessary when the value of upstream’s pom.xml file differs from Debian’s system packages. This happens frequently for API packages which are uploaded to Maven Central multiple times under a different groupId/artifactId but provide the same features. In this case the Twelvemonkeys’ pom requires an older API version but Debian is already at version 3.1. Note that we require a strict version number in this case because libservlet3.1-java does not use a symbolic debian version since we provide more than one Servlet API in the archive and this measure prevents conflicts.

Thanks for reading this far. More articles about Java packaging will follow in the near future and hopefully they will clarify some terms and topics which could only be briefly mentioned in this post.

before

and after

 

 

 

Hideki Yamane: I'm going to DebConf17

2 August, 2017 - 20:57

... No, you're not, my cat.

Jonathan Dowland: Debian on the Raspberry Pi3

2 August, 2017 - 16:36

Back in November, Michael Stapelberg blogged about running (pure) Debian on the Raspberry Pi 3. This is pretty exciting because Raspbian still provide 32 bit packages, so this means you can run a true ARM64 OS on the Pi. Unfortunately, one of the major missing pieces with Debian on the Pi3 at this time is broken video support.

A helpful person known as "SandPox" wrote to me in June to explain that they had working video for a custom kernel build on top of pure Debian on the Pi, and they achieved this simply by enabling CONFIG_FB_SIMPLE in the kernel configuration. On request, this has since been enabled for official Debian kernel builds.

Michael and I explored this and eventually figured out that this does work when building the kernel using the upstream build instructions, but it doesn't work when building using the Debian kernel package's build instructions.

I've since ran out of time to look at this more, so I wrote to request help from the debian-kernel mailing list, alas, nobody has replied yet.

I've put up the dmesg.txt for a boot with the failing kernel, which might offer some clues. Can anyone help figure out what's wrong?

Thanks to Michael for driving efforts for Debian on the Pi, and to SandPox for getting in touch to make their first contribution to Debian. Thanks also to Daniel Silverstone who loaned me an ARM64 VM (from Scaleway) upon which I performed some of my kernel builds.

Paul Wise: FLOSS Activities July 2017

2 August, 2017 - 01:19
Changes Issues Review Administration
  • Debian: fsck/reboot a buildd, reboot a segfaulting buildd, report/fix broken hoster contact, ping hoster about down machines, forcibly reset backup machine, merged cache patch for network-test.d.o, do some samhain dances, fix two stunnel services, update an IP address in LDAP, fix /etc/aliases on one host, reboot 1 non-responsive VM
  • Debian mentors: security updates, reboot
  • Debian wiki: whitelist several email addresses
  • Debian build log scanner: deploy my changes
  • Debian PTS: deploy my changes
  • Openmoko: security updates & reboots
Communication
  • Ping Advogato users on Planet Debian about updating/removing their feeds since it shut down
  • Invite deepin to the Debian derivatives census
  • Welcome Deepin to the Debian derivatives census
  • Inquire about the status of GreenboneOS, HandyLinux
Sponsors

All work was done on a volunteer basis.

Thorsten Alteholz: My Debian Activities in July 2017

1 August, 2017 - 22:12

FTP assistant

This month I am back to normal numbers and accepted 319 packages. I also kept the promise from last month and rejected 26 uploads.

Debian LTS

This was my thirty-seventh month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.

This month my all in all workload went up to 23.5h. During that time I did LTS uploads of:

  • [DLA 1025-1] bind9 security update for two CVEs
  • [DLA 1038-1] libtasn1-3 security update for one CVE
  • [DLA 1025-2] bind9 regression update
  • [DLA 1039-1] rkhunter security update for one CVE
  • [DLA 1040-1] resiprocate security update for one CVE
  • [DLA 1041-1] nasm security update for two CVEs
  • [DLA 1042-1] libquicktime security update for seven CVEs

I could also remove libtorrent-rasterbar and pspp from dla-needed.txt as the affected code was not in the Wheezy version or it was just a simple bug.

Last but not least I also had a few days of frontdesk duties.

Other stuff

This month I uploaded a new version of entropybroker with a revised set of systemd service files. At the moment there is public instance of entropybroker running at eb.debian.net. Its entropy is fed by several Entropy Keys made by Simtec Electronics. Though it is public, it is not yet anonymous, so if you need some entropy please drop me a line. At the moment there are two consumers, but the buffers are still filled.

I also uploaded several new packages, orcania, yder, hoel and ulfius. If everything works as expected, there will be soon an oauth2 server available in Debian.

Last but not least my DOPOM of this month has been ptunnel.

Reproducible builds folks: Reproducible Builds: Weekly report #118

1 August, 2017 - 21:05

Here's what happened in the Reproducible Builds effort between Sunday July 23 and Saturday July 29 2017:

Toolchain development and fixes
  • Chris Lamb sent an experimental patch to apt to make the output of apt-ftparchive reproducible. Thanks to David Kalnischkies for reworking the result. (#869557)
Packages reviewed and fixed, and bugs filed Reviews of unreproducible packages

4 package reviews have been added, 2 have been updated and 24 have been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Aaron M. Ucko (1)
  • Adrian Bunk (35)
  • Helmut Grohne (4)
  • Stefan Tatschner (1)
diffoscope development Misc.

This week's edition was written by Chris Lamb, Mattia Rizzolo & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Russell Coker: QEMU for ARM Processes

1 August, 2017 - 14:13

I’m currently doing some embedded work on ARM systems. Having a virtual ARM environment is of course helpful. For the i586 class embedded systems that I run it’s very easy to setup a virtual environment, I just have a chroot run from systemd-nspawn with the --personality=x86 option. I run it on my laptop for my own development and on a server my client owns so that they can deal with the “hit by a bus” scenario. I also occasionally run KVM virtual machines to test the boot image of i586 embedded systems (they use GRUB etc and are just like any other 32bit Intel system).

ARM systems have a different boot setup, there is a uBoot loader that is fairly tightly coupled with the kernel. ARM systems also tend to have more unusual hardware choices. While the i586 embedded systems I support turned out to work well with standard Debian kernels (even though the reference OS for the hardware has a custom kernel) the ARM systems need a special kernel. I spent a reasonable amount of time playing with QEMU and was unable to make it boot from a uBoot ARM image. The Google searches I performed didn’t turn up anything that helped me. If anyone has good references for getting QEMU to work for an ARM system image on an AMD64 platform then please let me know in the comments. While I am currently surviving without that facility it would be a handy thing to have if it was relatively easy to do (my client isn’t going to pay me to spend a week working on this and I’m not inclined to devote that much of my hobby time to it).

QEMU for Process Emulation

I’ve given up on emulating an entire system and now I’m using a chroot environment with systemd-nspawn.

The package qemu-user-static has staticly linked programs for emulating various CPUs on a per-process basis. You can run this as “/usr/bin/qemu-arm-static ./staticly-linked-arm-program“. The Debian package qemu-user-static uses the binfmt_misc support in the kernel to automatically run /usr/bin/qemu-arm-static when an ARM binary is executed. So if you have copied the image of an ARM system to /chroot/arm you can run the following commands like the following to enter the chroot:

cp /usr/bin/qemu-arm-static /chroot/arm/usr/bin/qemu-arm-static
chroot /chroot/arm bin/bash

Then you can create a full virtual environment with “/usr/bin/systemd-nspawn -D /chroot/arm” if you have systemd-container installed.

Selecting the CPU Type

There is a huge range of ARM CPUs with different capabilities. How this compares to the range of x86 and AMD64 CPUs depends on how you are counting (the i5 system I’m using now has 76 CPU capability flags). The default CPU type for qemu-arm-static is armv7l and I need to emulate a system with a armv5tejl. Setting the environment variable QEMU_CPU=pxa250 gives me armv5tel emulation.

The ARM Architecture Wikipedia page [2] says that in armv5tejl the T stands for Thumb instructions (which I don’t think Debian uses), the E stands for DSP enhancements (which probably isn’t relevant for me as I’m only doing integer maths), the J stands for supporting special Java instructions (which I definitely don’t need) and I’m still trying to work out what L means (comments appreciated).

So it seems clear that the armv5tel emulation provided by QEMU_CPU=pxa250 will do everything I need for building and testing ARM embedded software. The issue is how to enable it. For a user shell I can just put export QEMU_CPU=pxa250 in .login or something, but I want to emulate an entire system (cron jobs, ssh logins, etc).

I’ve filed Debian bug #870329 requesting a configuration file for this [1]. If I put such a configuration file in the chroot everything would work as desired.

To get things working in the meantime I wrote the below wrapper for /usr/bin/qemu-arm-static that calls /usr/bin/qemu-arm-static.orig (the renamed version of the original program). It’s ugly (I would use a config file if I needed to support more than one type of CPU) but it works.

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
  if(setenv("QEMU_CPU", "pxa250", 1))
  {
    printf("Can't set $QEMU_CPU\n");
    return 1;
  }
  execv("/usr/bin/qemu-arm-static.orig", argv);
  printf("Can't execute \"%s\" because of qemu failure\n", argv[0]);
  return 1;
}

Related posts:

  1. SE Linux vs chroot A question that is often asked is whether to use...
  2. Video Mode and KVM I recently changed my KVM servers to use the kernel...
  3. Creating a SE Linux Chroot environment Why use a Chroot environment? A large part of the...

Russ Allbery: Learning Rust

1 August, 2017 - 12:07

I'm obviously not spending much time writing here. It's been a rather busy month at work, and I've been doing other things on the weekend that aren't particularly interesting to write about.

This past week, though, I took advantage of our semi-annual Hack Week to finally learn Rust. I have several co-workers who love the language and have been wanting to stretch my programming language knowledge a bit. I was also profoundly disappointed by Go, which has been touted as the new C-style systems language but which I think is awful. All the reasons why is a topic for another post, but the obnoxiously verbose error handling is probably my biggest complaint. (This is the worst property of C; why would you copy it?) Rust was a favorite of a few people who felt the same way I did about Go, which seemed promising.

I made it through the first thirteen chapters of the second edition Rust book and wrote a not-entirely-trivial program (a tool to filter and search trace logs a Dropbox client) with a co-worker, and I think I'm in love with this language. It reminds me of everything I liked about Perl, except with all the weird bolted-on bits of Perl cleaned up and done properly, and with types. Despite having spent most of my career writing Perl and Python (and C, which is typed but not very well), I love strongly-typed languages. I just usually don't like the rest of the syntax of languages like Java and Go. Rust avoids the garbage collection nonsense (and huge performance issues), gives me the level of fine control that I am used to with C, but gets rid of memory allocation errors and provides a much richer type system and type matching. It feels a bit like an approachable Haskell, and I quickly found myself chaining iterators and pushing myself to write in a more functional style.

The lifetime stuff in Rust can be frustrating, and there are a few limitations that can be hard to deal with (like iterating over one field of a struct while modifying another field of a struct, which comes up a lot and which is the sort of thing you have to avoid in Rust). But I like a language with a very picky compiler. And I love programming in a language where my first attempt is clunky and verbose, and then I think about the problem a bit and rewrite it in half as many lines of code, and then I sleep on it and come back and can delete half of the code again.

I sadly don't have a lot of work projects right now where Rust is the right answer. I'm mostly maintaining existing code bases in Python, and Python is more accessible and more maintainable in most situations. But I now would love to find the time to rewrite a bunch of my personal C projects in Rust, and I'm watching for any new opportunity to use Rust.

If you like new programming languages, but you don't have the time or inclination to live on the bleeding edge, Rust has gotten more stable and is at a good point to start. The documentation is fantastic, support for generating documentation is built into the language, the Rust book is a great teaching research, and Rust is available as Debian packages (so you don't have to do the horrifying curl | bash nonsense in the official Rust documentation).

Recommended, particularly for people who love Perl or functional languages (or functional Perl heavy on map and grep), want a more modern language with fewer odd corners, and want low-level control and native speed.

Russ Allbery: Review: The Fifth Season

1 August, 2017 - 11:44

Review: The Fifth Season, by N.K. Jemisin

Series: The Broken Earth #1 Publisher: Orbit Copyright: 2015 ISBN: 0-316-22930-X Format: Kindle Pages: 497

The world of The Fifth Season is one of near-constant seismic activity. Volcanoes, massive earthquakes, and all the catastrophes that follow them are a constant threat. Civilization barely survives the turmoil, and only because of two things: strict cultural rules about how to handle a "fifth season" of heavy seismic activity and its aftermath (called stonelore), and the orogenes.

Orogenes are humans (well, there is some debate about that) who have an organ that others don't, a biological ability to manipulate the seismic activity and the earth itself. They can protect others by damping down activity, smoothing faults, and redirecting seismic shock waves, but they can also destroy: pull earth out of shape, set off quakes, and create paths for magma to surface. And, to gather the power to manipulate the earth, they draw energy from everything around them, including from other people, often fatally. Orogenes are feared and hated by the typical person.

The Stillness, the ironically-named continent on which this book is set, is very old and has had numerous civilizations destroyed by some seismic catastrophe. The landscape is scattered with useless or dangerous remnants of previous forgotten civilizations; the history, likewise, with only the stonelore and some muddled mythology available to most people. The current rulers have kept their empire for a surprising length of time, however, due mostly to the stable ground beneath their centrally-located capital. That stability comes from Fulcrum-trained orogenes, who are taken from their family as children and trained harshly to serve their society by suppressing or fixing dangerous seismic events. Fulcrum orogenes don't have an awful life (well, most of them; for some, it is pure torture), but they're effectively slaves, kept under the watchful eye of Guardians who have mysterious powers of their own.

Against this background, The Fifth Season tells three interwoven stories. Essun lives in a small village (comm) at the start of the book, leading a quiet life, until one of her children is beaten to death by her husband following a seismic event that he thinks the child stopped. He's taken their other child and left. Essun, severely traumatized, heads after him to attempt a rescue, or at least revenge. Damaya is a child from another comm who is sold to the Guardians by her parents when she demonstrates orogenic ability, and who goes through Fulcrum training. And Syenite is a Fulcrum orogene, assigned to a field mission with a difficult but very senior orogene named Alabaster.

All of these stories eventually interweave, and eventually reveal where they fit in the somewhat unobvious chronology of the story, but it takes some time to get there. It also takes some time for the primary characters to have much in the way of agency. Essun starts with the most, once she recovers her senses enough to start her hunt for revenge. Syenite is ambitious but junior, and Damaya is a child, trying to navigate an unknown world of student politics and strict rules. And all three of the main characters are orogenes, rogga when one is being insulting, and this world does not like orogenes. At all.

The Fifth Season starts with an unusual narrative style: a conversational narrator who begin with some of the world background and some mysterious scenes that didn't make sense until much later in the book (late enough that I didn't remember them or make sense of them until I re-read them for this review). The book then focuses on Essun, whose scenes are written in second person present. Normally I think second person feels weirdly intrusive and off-putting, but once I got used to it here, I think it works as well as I've seen it work anywhere. I also see why Jemisin did it: Essun starts the story so traumatized that she's partly disassociating. First person wouldn't have worked, and the second-person voice gives that trauma some immediacy and emotional heft that would have been hard to achieve in third person.

The story starts slowly, and builds slowly, as the world is introduced and Jemisin lays down the texture and history of the world. The world-building is ambitious in tracing down the ramifications of the seismic chaos and the implications of orogene ability (although it's best to think of it as pure magic, despite the minor science fiction trappings). But through that world-building, what this story is building is a deep, powerful, frustrated rage. The Fifth Season is an angry book. It's a book about outcasts, about slaves. About people who, even if they're succeeding within the parameters they're given, are channeled and stymied and controlled. It's a story about smiling, kind paternalism hiding lies, control, and abuse, about how hard it is to find enough space from the smothering destructiveness of a totalitarian culture to let yourself relax. It's a story about the horrible things people are willing to do to those they don't consider fully human, and all the ways in which safety, expediency, tradition, culture, and established social roles conspire to keep people within the box where they belong. And it's a story about how being constantly on edge, constantly dreading the next abuse, breaking under it, and being left wanting to burn the whole world to the ground.

I struggled at the start of this book, but it grew on me, and by about halfway through it had me hooked completely. At first, Syenite's part of the story (the most traditionally told) was my favorite, but the coming-of-age stories of her and Damaya were overtaken by Essun's far more complex, cautious, and battle-weary tale. And I loved Jemisin's world-building. There's a lot of depth here, a lot of things going on that are unexplained by clearly important, and a restraint and maturity in how the world is revealed that makes it feel older and more layered than Jemisin's The Hundred Thousand Kingdoms.

The major drawback of this book is that it is very much the first book of a series, and it doesn't so much have an ending as a hard stop. It's not quite a cliff-hanger, but it's nearly as unsatisfying as one. Most of the major questions of the book — who the stone eaters are and what they want, and the fate of Essun's husband and child, just to name two — are still unresolved at the end of the story. There is a bit of emotional closure, but not a true moment of catharsis for all of the rage. Hopefully that will be coming in a future book.

This is a very unusual story, mixing fantasy and a sort of magic (orogeny) with some science fiction elements and a deep history. It's gritty, textured, emotional, and furious, and very much worth reading. I'm looking forward to the next book in the series.

Followed by The Obelisk Gate.

Rating: 8 out of 10

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้