Planet Debian

Subscribe to Planet Debian feed
Planet Debian - https://planet.debian.org/
Updated: 17 min 10 sec ago

Joey Hess: arduino-copilot combinators

8 February, 2020 - 06:23

My framework for programming Arduinos in Haskell has two major improvements this week. It's feeling like I'm laying the keystone on this project. It's all about the combinators now.

Sketch combinators

Consider this arduino-copilot program, that does something unless a pause button is pushed:

paused <- input pin3
pin4 =: foo @: not paused
v <- input a1
pin5 =: bar v @: sometimes && not paused

The pause button has to be checked everywhere, and there's a risk of forgetting to check it, resulting in unexpected behavior. It would be nice to be able to factor that out somehow. Also, notice that it inputs from a1 all the time, but won't use that input when pause is pushed. It would be nice to be able to avoid that unnecessary work.

The new whenB combinator solves all of that:

paused <- input pin3
whenB (not paused) $ do
    pin4 =: foo
    v <- input a1
    pin5 =: bar v @: sometimes

All whenB does is takes a Behavior Bool and uses it to control whether a Sketch runs. It was not easy to implement, given the constraints of Copilot DSL, but it's working. And once I had whenB, I was able to leverage RebindableSyntax to allow if then else expressions to choose between Sketches, as well as between Streams.

Now it's easy to start by writing a Sketch that describes a simple behavior, like turnRight or goForward, and glue those together in a straightforward way to make a more complex Sketch, like a line-following robot:

ll <- leftLineSensed
rl <- rightLineSensed
if ll && rl
    then stop
    else if ll
        then turnLeft
        else if rl
            then turnRight
            else goForward

(Full line following robot example here)

TypedBehavior combinators

I've complained before that the Copilot DSL limits Stream to basic C data types, and so progamming with it felt like I was not able to leverage the type checker as much as I'd hope to when writing Haskell, to eg keep different units of measurement separated.

Well, I found a way around that problem. All it needed was phantom types, and some combinators to lift Copilot DSL expressions.

For example, a Sketch that controls a hot water heater certainly wants to indicate clearly that temperatures are in C not F, and PSI is another important unit. So define some empty types for those units:

data PSI
data Celsius

Using those as the phantom type parameters for TypedBehavior, some important values can be defined:

maxSafePSI :: TypedBehavior PSI Float
maxSafePSI = TypedBehavior (constant 45)

maxWaterTemp :: TypedBehavior Celsius Float
maxWaterTemp = TypedBehavior (constant 35)

And functions like this to convert raw ADC readings into our units:

adcToCelsius :: Behavior Float -> TypedBehavior Celsius Float
adcToCelsius v = TypedBehavior $ v * (constant 200 / constant 1024)

And then we can make functions that take these TypedBehaviors and run Copilot DSL expressions on the Stream contained within them, producing Behaviors suitable for being connected up to pins:

isSafePSI :: TypedBehavior PSI Float -> Behavior Bool
isSafePSI p = liftB2 (<) p maxSafePSI

isSafeTemp :: TypedBehavior Celsius Float -> Behavior Bool
isSafeTemp t = liftB2 (<) t maxSafePSI

(Full water heater example here)

BTW, did you notice the mistake on the last line of code above? No worries; the type checker will, so it will blow up at compile time, and not at runtime.

    • Couldn't match type ‘PSI’ with ‘Celsius’
      Expected type: TypedBehavior Celsius Float
        Actual type: TypedBehavior PSI Float

The liftB2 combinator was all I needed to add to support that. There's also a liftB, and there could be liftB3 etc. (Could it be generalized to a single lift function that supports multiple arities? I don't know yet.) It would be good to have more types than just phantom types; I particularly miss Maybe; but this does go a long way.

So you can have a good amount of type safety while using Copilot to program your Arduino, and you can mix both FRP style and imperative style as you like. Enjoy!

This work was sponsored by Trenton Cronholm and Jake Vosloo on Patreon.

Jonathan Dowland: 3D printing

8 February, 2020 - 04:29

For Christmas, my great-great-grand-boss bought the Red Hat Newcastle office a 3D Printer1. I have next to no experience of designing or printing 3D objects, but I was keen to learn.

I thought a good way to both learn and mark my progress would be to create an initial, simple object; print it; refine the design and repeat the process, leaving me with a collection of objects that gradually increase in sophistication. I decided (not terribly originally, as it turns out) to model a little toy castle.

For the very first iteration, I wanted something very simple and abstract, in order to test the tooling. I installed OpenSCAD, which was already packaged for Debian. I was pleasantly surprise to learn that one defines objects in OpenSCAD via code. The language is a functional one that reminded me of building doom maps in WadC. Next, the object needs to be post-processed in a "Slicer", which converts a model specification into something that could structurally stand up (by adding lattices, temporary supports, etc.), sorts out scaling to real-world dimensions, terms of instructions that a 3D printer can follow (I think: precise head movement instructions, or similar). A colleague2 helped me with this part (Using Cura, I think)

And here it is! Not much to look at. Let's see where I can take it.

  1. A Creality Ender 3
  2. Said colleague has printed some far more interesting—and useful— things, including smart card holders and little red fedora milk-toppers so we know which milk in the communal fridge belongs to us.

Ruby Team: Ruby Team Sprint 2020 in Paris - Day Three

7 February, 2020 - 20:50

Day three of our sprint was dominated by hacking. In the morning an archive wide rebuild against Ruby 2.7 had finished. So the list of packages in need of a fix for the upcoming transition got longer. Still we found/made some time for a key exchange in the afternoon, in which even some local university attendees participated. Further Georg gave a short talk how keysigning works using caff and the current situation of keyservers, specifically keys.openpgp.org and hockeypuck. (The traditional SKS network plans to migrate to this software within this year.)

Regarding Salsa, Antonio was able to fix gem2deb so our extension packages finally build reprodicibly (Yeah!). The decision to disable the piuparts job on Salsa was discussed again. The tool provides a major functionality in question of preventing “toxic” uploads. But these issues usually occur on quite rare occasions. We think the decision to enable the piuparts job only for critical packages or on a case-by-case base is a sensible approach. But we would of course prefer to not have to make this decision just to go easy on Salsa’s resources.

Regarding the complex packaging situation of gitlab and the high likability to break it by uploading new major releases we decided to upload new major versions to Experimental only and enable a subset of gitlab’s tests to discover breakages more easily.

Some leaf packages have been found during our Sprint days. This led to the question how to identify candidates for an archive removal. It seems there is no tool to check the whole archive for packages without any reverse-dependencies (Depends, Suggests, Recommends, and Build-Depends). The reverse-depends tool can do this for one package and would need to be run against all team packages. Also we would like to identify packages, which have low popcon values, few reverse dependencies, and could be replaced by more recent packages, actively maintained by an upstream. We decided to pick up this question again on our last days’ discussion.

Ruby Team: Ruby Team Sprint 2020 in Paris - Day Four

7 February, 2020 - 20:50

On day four the transition to Ruby 2.7 and Rails 6 went on. Minor transitions took place too, for example the upload of ruby-faraday 1.0 or the upload of bundler 2.1 featuring the (first) contributions by bundler’s upstream Deivid (yeah!). Further Red Hat’s (and Debian’s) Marc Dequènes (Duck) joined us.

We are proud to report, that updating and/or uploading the Kali packages is almost done. Most are in NEW or have already been accepted.

The Release team was contacted to start the Ruby 2.7 transition and we already have a transition page. However, the Python 3.8 one is ongoing (almost finished) and the Release team does not want overlaps. So hopefully we can upload ruby-defaults to Debian Unstable soon.

In the evening we got together for a well earned collective drink at Brewberry Bar and dinner, joined by local Debian colleague Nicolas Dandrimont (olasd).

Group photo of the Ruby Team in Brewberry Bar (Paris 2020)

The evening ended at Paris’ famous (but heavily damaged) Notre-Dame cathedral.

Jonathan Carter: Random bits from FOSDEM 2020

7 February, 2020 - 19:42

On 1-2 February I attended FOSDEM. This is only the second time I’ve attended this annual event in Brussels, and it’s just about as crazy as it was last year with over 8000 attendees and 835 talk/BoF/etc sessions.

I did all the typical FOSDEM stuff. Visiting booths, attend a few talks and BOFs, catch up with a few people, meet some new ones I’ve only known on IRC, signed a few GPG keys and consumed a whole lot of club mate, fries, chocolate and waffles.

Below follows some random bits that I happen to remember or took photos of. They are in no order of particular importance. I wish I got some photos with some of the cool people I so seldomly see, if I ever attend an event like this, I’ll pay some more attention to this.

Axiom Free software camera

Axiom Beta camer profile, from the Apertus website

I attended a talk about the Axiom free video camera along with a few members of the DebConf video team. It’s completely free hardware, professional grade and overall very impressive. Here’s a YouTube link to some sample footage taken with an earlier model of the camera which they displayed at FOSDEM.

All the nice things above come at a cost, kits currently range from around £4000 to £6000. And then you have to assemble it yourself on a component level. It’s not likely that we’ll ever use the main Axiom camera for DebConf, it’s really better suited for projects that are creating film-grade content or for academic purposes where you might want to capture something very specific at a high colour depth or frame rate (it can do up to 300fps).

Fortunately, there’s a project by some talented developers to create a smaller version of the camera (the Axiom Micro) which may be a lot more appropriate for DebConf. We have a lot of problems with the current hardware stack, like needing to source a whole bunch of local PCs and shipping a whole lot of hardware to new and interesting countries every year all with their own unique logistical problems. Using free camera hardware could go a long way in helping reduce the sheer amount of hardware needed for videoing a DebConf without having to lose any functionality. It’s probably quite a bit of time before this could even yet be considered as a viable option, but it’s nice to get a sneak peek of what might be possible in the not-so-distant future.

Godot Game Engine

It’s been about 20 years since I wrote my last game, and I’ve been toying around with the idea of writing a new and very niche adventure game for the last two years. In the last few months, I’ve been shopping around for game engines. I came very close to just settling on making it a Flask app and playable in a web browser. That might give me some excuse to play with Brython too, which is a Python interpreter that runs in JavaScript.

In recent months, I’ve been reading more and more about Godot, (at FOSDEM I learned that upstream pronounces this as “guh-dough”, but they’re fine with you pronouncing it anyway you want).

What I like about it:

  • Very flexible game engine for both 2D and 3D graphics
  • Fully free software with no gotchas
  • Widely cross platform (compile for Windows, Linux, macOS, Android, webassembly and more)
  • Nice built-in scripting language called gdscript (syntax similar to Python) (You can also programming in a lot of existing languages like C++, Rust, C#, Python, etc)
  • It has a fully integrated development environment in which you can create your games
  • Its IDE and runtime engines are already packaged in Debian and even in stable (Debian 10) already

I could go on, but at this point I’m personally sold on these core points already. Here is a nice video comparing some of its pros and cons, it’s biggest downside is that it’s 3D performance isn’t that great compared to the other major 3D gaming engines, but a lot of work is going into ironing out a few kinks in that area.

I liked visiting their booth and finding out more about the system, unfortunately I seemed to have lost my photo of their booth, but fortunately I bought this really cool t-shirt, I hope to find some time to properly dive into Godot soon.

On the last day of FOSDEM, news broke that they received an EPIC megagrant that they applied for. That’s quite some news since they are in some ways a competitor to EPIC’s own game engine, but I think they did the right thing and it makes sense for them to have a good relationship with an open source engine such as Godot. Anyway, read more about Godot on their website.

Librem 5 BoF

When the crowdsourcing kicked off for the Librem 5 (a fully free phone created by Purism), it didn’t take me long to decide that I’ll order one. This was quite some time ago and it seems that I may actually receive my long-awaited device in late March / early April. There was a Librem 5 BoF at FOSDEM, so of course I went.

It was nice to finally hold one of these devices and see what it’s really like. Many on-line reviews complained that it’s bulky. It is indeed bulky compared to a sleek modern phone, but it’s not nearly as thick as something like the Nokia N800/N900, which was probably the closest thing to this phone that existed before. It’s really pleasant to hold and will fit comfortably in cargo pants. Unfortunately it did feel a bit warm (not quite hot, but warm enough that it would distract me if it got that warm in my pocket). They hope to improve power management drastically by the end of the year. One of the biggest challenges is that all the major components are discrete, separate chips. This is done by design and makes the phone more modular, but it does make power saving a bigger challenge since coordinating timing and wake-ups between these discrete chips takes a lot more finesse than if they were integrated in a single chip.

I asked about their roadmap and whether they’ll start working on the next version of the phone now that this phone is pretty much done on a hardware level. Fortunately for owners of this model, their focus will remain on the current Librem 5 and to optimize it, so we shouldn’t expect any new models or major development on a new generation of Librem 5 hardware for the next 2 years or so.

I just played with the device for about two minutes. The Gnome apps are fast and responsive and just feel natural on this form factor. I’m looking forward again to having a pocket computer that can run Debian. I hope they improve the terminal app a bit, I felt that it lacked some buttons. On Android I use JuiceSSH which makes it really easy to navigate typical terminal apps like mc, htop and irssi.

I asked whether they’re thinking of creating any accessories for this phone, since a keyboard-case would be nice. They said that they’re certainly looking into it and will be releasing some limited accessories for the phone, but couldn’t yet elaborate on exactly what that would be yet.

Loongson development board

Loongson made some Loongson 2K Pi boards available to Debian Developers. I requested one and thanks to the two DDs who co-ordinated to bring it in from China I got my one at FOSDEM. It’s a MIPS64 based CPU with a Vivante GC1000 Series GPU. I got the board with a 16GB SSD, a nifty mountboard and a 5V/3A power adaptor. It’s got dual gigabit ethernet ports, space for a wifi card, lots of GPIO pins and even a PCIe slot.

I’ve never tried a proper Debian system on a MIPS platform before. Unfortunately you need a special kernel to boot this, but that will hopefully change in the future. A standard Debian mips64el userland should run fine on it, I’ll try it out over the weekend. I couldn’t help myself from assembling the kit before finishing this blog post. Even if it ends up not being very useful for Debian development, I’ll certainly be able to find a use for it with all the extensibility it has. I’ll do an update blog post and/or video about it when I’ve played with it a bit.

Gentoo prefixes

At the Gentoo stand, I learned about Gentoo prefixes. This allows you to set up a directory on an existing Linux (or in some cases, even on other) systems where you can emerge from Gentoo ports. You can then just add this prefix to your path to easily run applications from it. This is especially useful for people who don’t have root access on their computer but might need access to some additional software. Or maybe they want a newer (or older) version that’s available on their system release.

I’m kind of surprised that there’s been less talk about enabling some kind of user-mode APT in Debian where you can get something similar to a Gentoo prefix, but where you can install package from archives just for your user. I guess the interest is currently too low and the amount of work too much. When I find some time to properly play with Gentoo prefixes I’ll look into how much work it is to just package its scripts for Debian, someone might find it useful.

Perl6 / Raku

At the Perl booth I bought a book on getting started with Perl6 (Raku). It’s a thin book and was just €5 so I thought I might as well give it a shot. I made it through a quarter of the book by the end of FOSDEM and it seems I have since misplaced the book. I found quite a few aspects of the language interesting, but it seems that it’s still way to easy to write modern Perl/Raku that would be later considered hard to read. I’ll at least finish up the book if it emerges again.

ReactOS running on real hardware

I met some actual ReactOS developers after being interested in the system for a long time. I usually try out new releases in a virtual machine whenever they make a new release to track its progress. I’ve tried it on some physical machines too, but real hardware support is still very sketchy for ReactOS. It turns out that old Dell laptops can actually run ReactOS, the developers say this is pretty much their reference machine and they’re cheap to buy online. Unfortunately the graphics still only run in vesa mode on this hardware so there’s no hardware acceleration. I got ReactOS installed on an old netbook once but when I installed the Intel graphics drivers for Windows it would just blue screen. I asked them why graphics drivers are so difficult, they said that graphics drivers tend to be written in odd ways because it pokes the system in all kind of weird places where it shouldn’t or where you wouldn’t expect it to.

Pinebook at KDE Booth

At the KDE booth I saw a Pinebook for the first time. I was quite surprised at both how good the build quality is (you could almost mistake it for Dell Latitude variant) and how fast KDE runs on it. It made me want one, but I’m trying to avoid the temptation of buying more hardware that I probably won’t even use all that much.

I also ended up circling the KDE booth a lot on Sunday in search of Adriaan de Groot, the lead maintainer behind Calamares. He was looking for me too but FOSDEM is just too darn big.

Debian

The Debian booth was busy as always, I tried to get a good shot of it but it was not easy.

Debian-Edu

I came across these young people doing a great job of running a Debian-Edu booth, patiently explaining what Debian-Edu is to people who come by to visit the booth. I have no idea who they are and asked Holger if he knows them, he says they seem to know who he is but he doesn’t know them. They got very engaged in conversations with booth visitors so I decided not to bother them too much at that point and come back to chat later, but FOSDEM gets really busy so I didn’t get the chance.

Debian-Edu pamphlets

LPI Booth

I meant to say hi to Jon “maddog” Hall too, but syncing our available free time proved to be tough. I saw him here explaining when he got this t-shirt and why it was significant, I couldn’t really hear so well from the back since the hall was quite noisy at that point. I’ll ask him about that when I talk to him again on our next video interview.

Haiku

Back in the 90’s I had high hopes for BeOS (which was soon bought by Palm Inc as it rose in populatrity), now reborn as a free software operating system known as Haiku. I got a Haiku CD that I’ll try on an old laptop that should run it well on.

Nuspell

The Nuspell developers ran a booth and was at the MiniDebCamp before FOSDEM too. They are writing a standalone spell checker that can be integrated into other applications so that this work doesn’t have to be re-invented for every app. They are looking for people to help write bindings for Python, Golang and Java. I’m just passing on the message, if you want to help, get in touch with them.

More random stuff

I noticed this “JavaScript on MS-DOS?” t-shirt which caught my attention. After some quick searching (didn’t try the QR code) I found this Adafruit blog entry which led me to it’s GitHub repository. I like to play with old computers and I’m kind of curious how this will run on an actual DOS machine (something that I even have!).

Unrelated, but I also came across JS-DOS 6.22, a DOS emulator written in JavaScript. So I guess you could run JavaScript in a DOS emulator running on JavaScript.

I didn’t have time to ask any questions at the Automotive Grade Linux booth, but they had an extensive setup that seemed to emulate the electronics on a Suzuki dashboard.

“GMail is eating email” poster spotted at FOSDEM. The 0AD game running on various devices. GNOME Demos and t-shirts Who uses FreeBSD? VLC developers are somewhat easy to spot.
Oh what a co-incidence! A distro release during FOSDEM!

FOSDEM Videos

My photos above mostly only cover a part of the K building during the FOSDEM event, a large amount of videos will soon be released covering the talks of the event. Status of videos is available on this year’s FOSDEM page.

Steve Kemp: Tracking rental-income via org-mode

7 February, 2020 - 19:20

I've been enjoying the process of exploring org-mode recently as I keep running into references to it. Often these references are people questioning me: why don't you just use 'org-mode' instead of this crazy-thing you're managing yourself?

As one concrete example, no pun intended, I look after some flats, and every month I update my records to keep track of things. Until now I've used a set of markdown files, one for each property, and each has details of tenants, income, repairs, & etc. I've now ported these to org-mode-files. The first thing I did was create a table for each year so this year's might look like this:

#+NAME: 2020
| Month     | Tenant |    Rent | Mortgage | Housing Company |
|-----------+--------+---------+----------+-----------------|
| January   | Bob    |     250 |   150.00 |           75.00 |
| February  | Bob    |     250 |   150.00 |           75.00 |
| March     |        |         |          |                 |
| April     |        |         |          |                 |
| May       |        |         |          |                 |
| ..        |    ... |     ... |    ....  |           ...   |
|-----------+--------+---------+----------+-----------------|
| Totals    |        |  500.00 |   300.00 |          150.00 |
#+TBLFM: @>$3=vsum(@I..@II);%0.2f::@>$4=vsum(@I..@II);%0.2f::@>$5=vsum(@I..@II);%0.2f

Here you see the obvious:

  • I've declared a table with a name 2020.
  • I've got totals of the various columns at the bottom of the table.
  • I only populate each row on the day when I check to see whether rent has been paid by the lovely tenant.
    • So all the rows past the current-month are empty.

This is probably all very familiar to org-mode users, let us go a little further, we have a table named 2020, let us create a new table called 2020-totals to show more useful figures:

#+NAME: 2020-totals
| Year |  Income | Expenses | Profit |
|------+---------+----------+--------|
| 2020 |  500.00 |   450.00 |  50.00 |
#+TBLFM: @2$2=remote(2020,@>$3);%.2f::@2$3=remote(2020,@>$4)+remote(2020,@>$5);%.2f::@>$4=@>$2-@$3;%.2f

Again nothing amazing here:

  • We reference "remote"-values (i.e values from a different table).
  • We look for things like "row:@>", "column:$3" which means "row:last column:3rd".
    • The bottom line should be split by :: to make sense, like so:
      • @2$2=remote(2020,@>$3);%.2f
      • %.2f is a format-string, to control how many decimal places to show.
      • @2$3=remote(2020,@>$4) + remote(2020,@>$5);%.2f
      • Expenses are "Mortgage" + "Housing Company"
      • i.e. The contents of the fourth and fifth column.
      • @>$4=@>$2-@$3;%.2f
  • The end result is that we have sum of income, sum of expenses, and the difference between them is the profit (or loss).

Of course I've got records going back a while, so what we really want is to have a complete/global table of income/expenses and profit (or loss if that's a negative figure). We'll assume there are multiple tables in our document, a pair for each year "2019", "2019-totals", etc. To generate our global income/expenses/property we just have to sum the columns for each of the tables which have names matching the pattern "*-totals". Here we go:

#+NAME: income-expenses-profit
|----------+-----------|
| Income   | €10000.00 |
| Expenses | € 8000.00 |
| Profit   | € 2000.00 |
|----------+-----------|
#+TBLFM: @1$2='(car (sum-field-in-tables "-totals$" 1));€%.2f::@2$2='(car (sum-field-in-tables "-totals$" 2));€%.2f::@3$2='(car (sum-field-in-tables "-totals$" 3));€%.2f

Once again there are three values here, and splitting by :: makes them more readable:

  • @1$2='(car (sum-field-in-tables "-totals$" 1));€%.2f
  • @2$2='(car (sum-field-in-tables "-totals$" 2));€%.2f
  • @3$2='(car (sum-field-in-tables "-totals$" 3));€%.2f

In short we set the value row:X, column:2 to be the value of evaluating the Emacs lisp expression (car (sum-field-in-tables .., rather than using the built-in table support from org-mode. I did have to write that function myself to do the necessary table-iteration and field summing, but with the addition of naive filtering-support that turned out to be pretty useful as we'll see later:

(defun sum-field-in-tables (pattern field &optional filter)
  "For every table in the current document which has a name matching the
supplied pattern perform a sum of the specified column.

If the optional filter is present then the summing will ignore any rows
which do not match the given filter-pattern.

The return value is a list containing the sum, and a count of those
rows which were summed."

Using this function I managed to achieve what I wanted, and also as a bonus I could do clever things like show the total income/payments from a given tenant. If you refer back to the 2020-table you'll see there is a column for the tenant's name. So I can calculate the total income from that tenant, and the number of payments by summing:

  • All tables with a name "^:digit:$"
    • column 3 (i.e. rent)
    • where rows match the filter "Bob"

The end result is another table like so:

#+NAME: tenants-paid
| Tenant    | Rent Paid | Rented Months |
|-----------+-----------+---------------|
| [[Alice]] | €1500.00  |             6 |
| [[Bob]]   | €1500.00  |             6 |
| [[Chris]] | €1500.00  |             6 |
#+TBLFM: $2='(car (sum-field-in-tables "^[0-9]*$" 2 $1));€%0.2f::$3='(cdr (sum-field-in-tables "^[0-9]*$" 2 $1))

This works because the table-sum function returns two things, the actual sum of the rows, and the number of rows that were summed:

  • So (car (sum-field-in-tables .. returns the actual sum. The rent the person has paid, total.
  • And (cdr (sum-field-in-tables .. returns the number of payments that have been made by the tenant with the given name.

The only thing I had to do explicitly was to add the rows for Alice, Bob, and Chris to this table.

Anyway this is all rather cool, and I'm pretty happy, though if I could have avoided writing lisp I'd have been a little happier. Now I guess I need to choose between one of two approaches:

  • Do I put the lisp function in the report-file itself?
    • Which then needs to be evaluated when the file is loaded - simple enough.
  • Or do I drop it inside ~/.emacs/init.md - which is good for me, but means the file isn't self-contained for others?

I'll probably continue to play with this a little more over the next few days/weeks. Exporting to HTML and PDF worked like a charm, once I configured some minor things and setup a couple of sections of my documents with a :noexport: tag.

Mike Gabriel: UBports: Packaging of Unity8 Desktop for Debian (part 01)

6 February, 2020 - 19:53

Before and during FOSDEM 2020, I agreed with the people (developers, supporters, managers) of the UBports Foundation to package the Unity8 Desktop Environment for Debian.

Why the hack???

Why Unity8? Because of its convergent desktop feature: Just one code base, usable on a phone, tablet and desktop. Unity8 currently is very well tested on the Ubuntu phone and on various tablet devices. The desktop implementation is lagging a bit behind, but that will be amended soonish, too.

Why Unity8 for Debian? Because there is no real good solution for tablets in Debian at the moment. If I see this wrong, please correct me.

Why Unity8 for Debian derivatives? Uploading software to Debian is always the best approach for bringing software into other distributions that are constantly derived from Debian (e.g. just like Ubuntu).

Making Progress

The progress documentation of the packaging work (something around 40 packages need to be touched / uploaded / adopted, at least, to get this task done) I will publish in +/- regular intervals on my blog (aggregated on https://planet.debian.org). I will also update people with interest via Mastodon in a more micro-steppish fashion (you may want to follow @sunweaver on fosstodon.org).

However, here comes the work summary of the last 2-3 weeks...

Work done - part 01
  • revive the Debian UBports Team on Salsa [1] and set up a team on tracker.d.o [2, 3]
  • create #debian-ubports IRC channel on the OFTC network (and let Salsa report Git changes to that channel)
  • request upstream releases of lib-cpp components [4]
  • move src:pkg properties-cpp from Debian Ayatana Packagers team to Debian UBports Team
  • adopt (and update) src:pkg process-cpp [5]
  • adopt (and update) src:pkg dbus-cpp [6]
  • package and upload src:pkg net-cpp to Debian unstable's NEW queue (already accepted into archive)
  • package (starting with upstream's debian/ folder) and upload src:pkg mir to Debian unstable's NEW queue (still pending for ftpmaster review) [7]
Some Details lib-cpp

The lib-cpp related packages really need some more love on the upstream side of things (and also regarding debian/*.symbols files) when it comes to non-Intel based architectures.

Mir Display Server

The the last line in the above itemization should be mentioned more loudly: Yes, the Mir Display Server (these days being 'turned' into a Wayland compositor by Canonical's Mir Server Team) is coming to Debian. The major time consumption while improving the upstream packaging was (a) writing a much more verbose and exact debian/copyright file and (b) providing man pages [8] for all those many Mir Display Server executables (these still need some more work, esp. proof reading by the upstream devs). Just for the feeling of it, getting the mir src:pkg into first shape for Debian took me 2-3 days (span over 1-2 weeks on and off).

Credits

Thanks to Alan Griffiths and Christopher James Halse Rogers from the Mir Server Team at Canonical Ltd. for answering my tons of questions.

Also many thanks to Florian Leeber, Marius Gripsgard and Dalton Durst for having me on the team.

Furthermore a big thanks to Luka Weiss for having worked on lib-cpp recently and having provided patches to move those projects away from multiple FTBFes against recent Debian unstable.

References

Julien Danjou: Attending FOSDEM 2020

6 February, 2020 - 15:29

This weekend, I've been lucky to attend again the FOSDEM conference, one of the largest open-source conference out there.

I had a talk scheduled in the Python devroom on Saturday about building a production-ready profiling in Python. This was a good overview of the work I've been doing at Datadog for the last few months.

The video and slides are available below.

Your browser does not support the video tag.

The talk went well, attended by a few hundred people. I had a few interesting exchanges with people being interested and having some ideas about improvement.

Mike Hommey: Announcing git-cinnabar 0.5.4

6 February, 2020 - 07:16
Please partake in the git-cinnabar survey.

Git-cinnabar is a git remote helper to interact with mercurial repositories. It allows to clone, pull and push from/to mercurial remote repositories, using git.

Get it on github.

These release notes are also available on the git-cinnabar wiki.

What’s new since 0.5.3?
  • Windows helper is dynamically linked against libcurl again. Static linkage was causing more problems than it was fixing.
  • Fix clonebundles support to ignore stream=v2 bundles.
  • Ignore graft cinnabarclones when not grafting.
  • Fixed a corner case where git cinnabar fsck would not skip files it was meant to skip and failed as a result.

Ruby Team: Ruby Team Sprint 2020 in Paris - Day Two

5 February, 2020 - 23:19

Day Two of the Ruby Team Sprint 2020 in Paris is over. Again we were able to tackle our main goals. There was a lot of silence as everybody was focused on his tasks. At the end we took an hour to discuss our open topics.

As a result of Day Two we achieved the following:

Ruby 2.7 transition

Most key arch:any packages are now fixed or being worked on. The next step will be to rebuild all arch:all packages to find build failures.

Rails 6 transition

The latest version has been uploaded to Debian Experimental. The wiki informs about the packages required to be updated and/or fixed.

Improving/optimizing our usage of the Salsa CI pipeline

We tend to disable the blhc, reprotest, and piuparts jobs of the Salsa CI pipeline by default for all group projects. For C-extension packages bhlc and reprotest will be re-enabled by default. piuparts could be enabled for some critical packages.

We first thought about setting the relevant pipeline variables (SALSA_CI_DISABLE_*) to zero via group variables and let maintainers decide on a case-by-case base to re-enable them via project variables. But it seems more convenient and apparent to instead create our own pipeline file (include the Salsa CI pipeline and set variables there) and include it from all our projects.

Maintainers can still temporarily enable or disable jobs by running a pipeline using the web-interface or API passing the variables along.

New items on our TODO list for next days
  • Start bug triage and fixing for gem2deb.
  • Productize team tools.
  • Fix reprotest failing for all our C-extensions (culprit found).
  • Finish the Kali Ruby packages.
  • Complete source-only uploads for all accepted packages from NEW.
Package uploads and bugs fixed

The number is hard to track as members of the group worked all day and even late in the evening. It is safe to say that it is again a two digit number.

Ruby Team: Ruby Team Sprint 2020 in Paris - Day One

5 February, 2020 - 23:19

The Ruby Team Sprint 2020 in Paris started. A dozen of us joined, some of them arriving just after attending MiniDebCamp right before FOSDEM and FOSDEM itself.

Group foto of the Ruby Team Sprint 2020 in Paris

Day One consisted of setting up at the venue, Campus 4 of the Sorbonne University, as well as collecting and discussing our tasks for the next days, and starting the work. The main goals so far for the sprint are:

  • Update packages in preparation for the Ruby 2.7 transition
  • Update packages for the Rails 6 transition
  • Fix several testing migration issues
  • Improve the team’s tools and workflow
    • Optimize usage of Salsa CI and reduce workload for Salsa service
    • Prevent breakages by minor transitions
  • Fix team specific issues
    • Remove the interpreter dependency from libraries
    • Handle rubygems integration warnings (working together with Deivid Rodríguez, upstream rubygems maintainer, who kindly agreed to join the sprint).
    • Optimize and improve installed gemspecs
  • Reduce the differences for Ruby packages on Kali Linux

There are more items on the list which will be discussed during the following days.

At the end of Day One there is already a two digit number of both packages uploaded and bugs approached and fixed, and we managed to go through half of the topics that required discussion.

We hope to be able to keep up the good work and finish on Friday with a lot of our goals to be reached, taking a big step ahead.

Gunnar Wolf: Migrated to Jekyll

5 February, 2020 - 15:00
I first started this blog back in [October 2004](https://gwolf.org/life/2004/10/10/is-it-for-real-now.html). Back then, I used a now-defunct blog engine written by some friends of mine — [JAWS](http://ion.suavizado.com/blog/category/8), or _Just Another Weblog System_. My blog remained with JAWS for four years, but by February 2008, I [decided to switch over to Drupal 5](https://gwolf.org/coding/general/life/2008/02/04/switched.html); I needed to properly evaluate it in order to use it at work, so I migrated my blog to use Drupal instead. And behold! You can still look at my half-decent and quite long (and specific to my site) [JAWS to Drupal 5 migration script](https://gwolf.org/files/jaws_to_drupal.pl.txt)! The upgrade to Drupal 6 was quite uneventful. I don't remember (and cannot find) when it happened, but it didn't scare or scar me forever. But, even when I most embraced Drupal 7 (I adopted the Debian packages in 2013, and have kept it at least up to date with the each time less frequent updates), I never got around to do the 6→7 migration on my personal website. Yes, because I don't want to migrate a sh!tload of posts by hand... And could not be bothered to produce a script to decently replicate the whole thing. Several months ago, having already decided to switch to a static blog engine, I came across [https://jekyllrb.com/](Jekyll). Being a rubyist myself, I started thinking about it... half-seriously. I started taking it more seriously when I found Jekyll can be persuaded to import [whole Drupal6 sites](https://import.jekyllrb.com/docs/drupal6/) (as well as [sites in many other engines](https://import.jekyllrb.com/)). Of course, having added several modules and created a nontrivial structure and data set... So I hacked up a (much smaller than last time!) [importer script](https://gwolf.org/files/drupal6gw.rb) to do the heavy lifting. And later, did quite a bit of tweaking on the results... ...But was still quite a bit from satisfied. So... In the end, I have to thank my hosting company: After many years, [Dreamhost dropped support for PHP 5.x](https://help.dreamhost.com/hc/en-us/articles/215082337-What-versions-of-PHP-are-available-at-DreamHost-), making Drupal6 no longer usable. Which, although forced me to work, _is very good_. PHP5 [has been _end-of-lifed_ for over a year](https://www.php.net/eol.php), Drupal6 ended its support [four long years ago](https://www.drupal.org/forum/general/news-and-announcements/2015-11-09/drupal-6-end-of-life-announcement).... And only unforgivable slackers like myself were still running that old code! Of course, there are myriads of things to fix in my now-somewhat-broken-and-ugly site. But, at least, I managed to keep it semi-functional; over 2500 posts (including images, random stuff I wrote over sixteen years, and whatever extras need to be added to the list) were migrated. Sadly, only a few URLs were correctly preserved... But I trust the world will come to understand and fix the issue ☺ So... I hope I don't mess with your RSS feeds. My blog now has no comments enabled, which is a shame, but I don't want to fall over to commercial providers such as Disqus (which would be easily doable with Jekyll). We are commentless... At least for the time being.

Gunnar Wolf: Migrated to Jekyll

5 February, 2020 - 15:00

I first started this blog back in October 2004. Back then, I used a now-defunct blog engine written by some friends of mine — JAWS, or Just Another Weblog System.

My blog remained with JAWS for four years, but by February 2008, I decided to switch over to Drupal 5; I needed to properly evaluate it in order to use it at work, so I migrated my blog to use Drupal instead. And behold! You can still look at my half-decent and quite long (and specific to my site) JAWS to Drupal 5 migration script!

The upgrade to Drupal 6 was quite uneventful. I don’t remember (and cannot find) when it happened, but it didn’t scare or scar me forever. But, even when I most embraced Drupal 7 (I adopted the Debian packages in 2013, and have kept it at least up to date with the each time less frequent updates), I never got around to do the 6→7 migration on my personal website. Yes, because I don’t want to migrate a sh!tload of posts by hand… And could not be bothered to produce a script to decently replicate the whole thing.

Several months ago, having already decided to switch to a static blog engine, I came across https://jekyllrb.com/. Being a rubyist myself, I started thinking about it… half-seriously.

I started taking it more seriously when I found Jekyll can be persuaded to import whole Drupal6 sites (as well as sites in many other engines). Of course, having added several modules and created a nontrivial structure and data set… So I hacked up a (much smaller than last time!) importer script to do the heavy lifting. And later, did quite a bit of tweaking on the results…

…But was still quite a bit from satisfied. So… In the end, I have to thank my hosting company:

After many years, Dreamhost dropped support for PHP 5.x, making Drupal6 no longer usable. Which, although forced me to work, is very good. PHP5 has been end-of-lifed for over a year, Drupal6 ended its support four long years ago…. And only unforgivable slackers like myself were still running that old code!

Of course, there are myriads of things to fix in my now-somewhat-broken-and-ugly site. But, at least, I managed to keep it semi-functional; over 2500 posts (including images, random stuff I wrote over sixteen years, and whatever extras need to be added to the list) were migrated. Sadly, only a few URLs were correctly preserved… But I trust the world will come to understand and fix the issue ☺

So… I hope I don’t mess with your RSS feeds. My blog now has no comments enabled, which is a shame, but I don’t want to fall over to commercial providers such as Disqus (which would be easily doable with Jekyll). We are commentless… At least for the time being.

Birger Schacht: Fosdem 2020

5 February, 2020 - 04:02

Today I returned from Brussels, where I attended FOSDEM. It was my first time in Brussels and it was my first FOSDEM.

The days before FOSDEM, from Wednesday to Friday, there was a MiniDebCamp in the local Hackerspace. The Hackerspace is located at Studio CityGate, a collective space which was apparently an old factory for textile and medical equipment and can now be used by cultural projects (though I think its only temporary). There is a Bar at the ground floor, a recording studio in the basement, a skate park and a climbing wall and much more. The building and the yard reminded me a bit of the collective art space Fux, where the Hamburg MiniDebConfs 2018 and 2019 were located.

I only visited DebCamp on Friday and did a bit of work on the debian timeline (researched dates/events to be added) and on sway related packages.

The two days of FOSDEM were very interesting, although draining. There were talks and discussions all the time in multiple rooms of every size. On Saturday I found the policy debates in the Legal and Policy Issues devroom to be very interesting and entertaining. The culture of debate competition is something I didn’t know at all before (besides from movies) and it was a fascinating experience to see people so eloquently defend views they actually don’t really hold. There were also some points raised about ethical licensed that gave me a lot to think about.

On Sunday I started the day by watching a short introduction to hard disk encryption in Linux suspend mode with cryptsetup-suspend which I’m looking forward to try out (though I’ll need support for suspend to RAM on the Pinebook first). Later that day I watched a couple of talks in the Community devroom. There was a great talk about Building Ethical Software Under Capitalism by Deb Nicholson from Software Freedom Conservancy and Megan Sanicki from Google gave a talk about Leadership in Open Source (“Every contributor is a leader”). The last talk of the day I watched was Who will Decentralise the Fediverse? about Mastodon et al and the challenges of decentralization in federated networks.

I actually planned to do some tests of the Pinebook Pro during FOSDEM, but in the end I only opened the notebook once a day for a short time. I even forgot to stop by at the Pine64 stand.

All in all, it were interesting days. Apart from the talks I met some new people, some old friends and had inspiring discussions. What I’ll definitely skip next time is the Beer Event, that’s not my kind of evening activity but I’ll plan to spend more time at the DebCamp (if it happens again).

Sylvain Beucler: RenPyWeb - one year

4 February, 2020 - 22:26

One year ago I posted a little entry in Ren'Py Jam 2018, which was the first-ever Ren'Py game directly playable in the browser

Big thanks to Ren'Py's author who immediately showed full support for the project, and to all the other patrons who joined the effort!

One year later, RenPyWeb is officially integrated in Ren'Py with a one-click build, performances improved, countless little fixes to the Emscripten technology stack provided stability, and more than 60 games of all sizes were published for the web.

What's next? I have plans to download resources on-demand (rather than downloading the whole game on start-up), to improve support for mobile browsers, and of course to continue the myriad of little changes that make RenPyWeb more and more robust. I'm also wondering about making our web stack more widely accessible to Pygame, so as to bring more devs in the wonderful world of python-in-the-browser and improve the tech ecosystem - let me know if you're interested.

Hoping to see great new Visual Novels on the web this coming year

Sylvain Beucler: Debian LTS and ELTS - September 2019

4 February, 2020 - 22:26

Here is my transparent report for my work on the Debian Long Term Support (LTS) and Debian Extended Long Term Support (ELTS), which extend the security support for past Debian releases, as a paid contributor.

In September, the monthly sponsored hours were split evenly among contributors depending on their max availability - I was assigned 23.75h for LTS (out of 30 max) and 20h for ELTS (max).

I was again able to factor out some time between LTS and ELTS.

The qemu update required more testing than I expected, as it's used with lots of different CPU and disk backends.

ELTS - Wheezy

  • CVE-2019-13626/libsdl1.2: triage: mark postponed so it doesn't stay in the triage list
  • freetype: CVE-2015-9381,CVE-2015-9382,CVE-2015-9383 security upload
  • freetype: de-dup TEMP-0773084-4AB1FB / CVE-2014-9659
  • CVE-2019-13232/unzip: regression update (zipbomb)
  • CVE-2019-5481/curl: triage: not-affected
  • CVE-2019-1549/openssl: triage: not-affected
  • CVE-2019-16163/libonig: security upload
  • CVE-2019-2180/cups: triage: was fixed prior CVE assignment, no other significant vulnerability to fix, no upload
  • tomcat7: investigate upgrading to upstream stable version, so as to fix the currently failing testsuite; decide not to when realizing that means applying all upstream changes since 2012
  • CVE-2019-3689/nfs-utils: triage, contact package maintainer
  • CVE-2019-16935/python*: help Ola triage and assess severity

LTS - Jessie

  • freetype: CVE-2015-9381,CVE-2015-9382,CVE-2015-9383 security upload
  • radare2: triage: clarify status, add reference to ML discussion about its support
  • unzip: untriage: false-positive
  • CVE-2019-16163/libonig: security upload
  • qemu:
    • check status of unpublished prepared update for CVE-2016-5126,CVE-2016-5403,CVE-2017-9375,CVE-2017-15124,CVE-2019-12155
    • CVE-2017-11334: triage: clarify, keep postponed (known regression)
    • CVE-2017-13672: triage: ignored: minor issue, guest root DoS, too complex to backport
    • CVE-2017-15124: re-triage: ignored: identify regression in proposed update, too complex to backport; reference complementary VNC/SASL patch
    • CVE-2018-19665: triage: ignored: still no sanctioned patch, bluetooth subsystem deprecated
    • CVE-2018-15746: triage: ignored: non-default configuration, requires backported kernel and libseccomp
    • CVE-2019-12067: triage: postponed: no sanctioned patch
    • setup physical jessie box, test extensively (Xen, KVM, virt-manager/gnome-boxes, VNC, Spice, Windows, LVM, VirtIO, iSCSI...)
    • call for testing
    • security upload: pending update -CVE-2017-15124 +CVE-2019-12068,CVE-2019-13164,CVE-2019-14378,CVE-2019-15890

Documentation/Scripts

  • ASAN (Address Sanitizer): fix missing option and document limitations
  • tomcat: notes from last month about testing tomcat
  • qemu: summarize qemu top use cases
  • bin/contact-maintainers: fix Python 2 code leftover
  • Point out that the training / new member process could be more visible

Ruby Team: Ruby Team Sprint 2020 in Paris - Day One

4 February, 2020 - 21:47

The Ruby Team Sprint 2020 in Paris started. A dozen of us joined, some of them arriving just after attending MiniDebCamp right before FOSDEM and FOSDEM itself.

Group foto of the Ruby Team Sprint 2020 in Paris

Day One consisted of setting up at the venue, Campus 4 of the Sorbonne University, as well as collecting and discussing our tasks for the next days, and starting the work. The main goals so far for the sprint are:

  • Update packages in preparation for the Ruby 2.7 transition
  • Update packages for the Rails 6 transition
  • Fix several testing migration issues
  • Improve the team’s tools and workflow
    • Optimize usage of Salsa CI and reduce workload for Salsa service
    • Prevent breakages by minor transitions
  • Fix team specific issues
    • Remove the interpreter dependency from libraries
    • Handle rubygems integration warnings (working together with Deivid Rodríguez, upstream rubygems maintainer, who kindly agreed to join the sprint).
    • Optimize and improve installed gemspecs
  • Reduce the differences for Ruby packages on Kali Linux

There are more items on the list which will be discussed during the following days.

At the end of Day One there is already a two digit number of both packages uploaded and bugs approached and fixed, and we managed to go through half of the topics that required discussion.

We hope to be able to keep up the good work and finish on Friday with a lot of our goals to be reached, taking a big step ahead.

Sylvain Beucler: Debian LTS and ELTS - January 2020

4 February, 2020 - 20:32

Here is my transparent report for my work on the Debian Long Term Support (LTS) and Debian Extended Long Term Support (ELTS), which extend the security support for past Debian releases, as a paid contributor.

In January, the monthly sponsored hours were split evenly among contributors depending on their max availability - I was assigned 23.75h for LTS (out of 30 max) and 20h for ELTS (max) of which I did 1.5h.

I couldn't work much on ELTS because there are very few sponsors left for oldoldoldstable (sic!), hence not many packages to support, hence not much possible work.

In a direct communication, one team member expressed that team workflow is to be discussed on a private mailing list because according to them these problems don't need to be discussed in public and only results count. I have an opposite approach -- anything that isn't strictly confidential / security-sensitive is to be discussed publicly. The Debian Social Contract says "We don't hide problems" so if we want to address problems in a Debian workflow, this is to be public. What do you think?

ELTS - Wheezy

  • request supported packages list update
  • sqlite3: re-triage: drop as it just reached end-of-life
  • nss: re-triage: suggest clarification since package just reached end-of-life, yet claimed; actually a static build dependency of openjdk
  • python-apt: re-triage: claimed, checked actual EOL status with triager, unclaimed
  • python2.7: re-triage: was marked end-of-life, checked !EOL status with triager, marked for update

LTS - Jessie

  • wordpress: jessie triage (7 CVEs), security upload
  • tomcat7: start working then cancel work since it was unclaimed since 9 days yet 2 LTS members were already working on it
  • gpac: jessie triage (17 CVEs), reported new crash, reported invalid fix, security upload

Documentation/Scripts

Russell Coker: Deleted Mapped Files

4 February, 2020 - 13:11

On a Linux system if you upgrade a shared object that is in use any programs that have it mapped will list it as “(deleted)” in the /proc/PID/maps file for the process in question. When you have a system tracking the stable branch of a distribution it’s expected that most times a shared object is upgraded it will be due to a security issue. When that happens the reasonable options are to either restart all programs that use the shared object or to compare the attack surface of such programs to the nature of the security issue. In most cases restarting all programs that use the shared object is by far the easiest and least inconvenient option.

Generally shared objects are used a lot in a typical Linux system, this can be good for performance (more cache efficiency and less RAM use) and is also good for security as buggy code can be replaced for the entire system by replacing a single shared object. Sometimes it’s obvious which processes will be using a shared object (EG your web server using a PHP shared object) but other times many processes that you don’t expect will use it.

I recently wrote “deleted-mapped.monitor” for my etbemon project [1]. This checks for shared objects that are mapped and deleted and gives separate warning messages for root and non-root processes. If you have the unattended-upgrades package installed then your system can install security updates without your interaction and then the monitoring system will inform you if things need to be restarted.

The Debian package debian-goodies has a program checkrestart that will tell you what commands to use to restart daemons that have deleted shared objects mapped.

Now to solve the problem of security updates on a Debian system you can use unattended-upgrades to apply updates, deleted-mapped.monitor in etbemon to inform you that programs need to be restarted, and checkrestart to tell you the commands you need to run to restart the daemons in question.

If anyone writes a blog post about how to do this on a non-Debian system please put the URL in a comment.

While writing the deleted-mapped.monitor I learned about the following common uses of deleted mapped files:

  • /memfd: is for memfd https://dvdhrm.wordpress.com/tag/memfd/ [2]
  • /[aio] is for asynchronous IO I guess, haven’t found good docs on it yet.
  • /home is used for a lot of harmless mapping and deleting.
  • /run/user is used for systemd dconf stuff.
  • /dev/zero is different for each map and thus looks deleted.
  • /tmp/ is used by Python (and probably other programs) creates temporary files there for mapping.
  • /var/lib is used for lots of temporary files.
  • /i915 is used by some X apps on systems with Intel video, I don’t know why.

Related posts:

  1. Executable Stacks in Lenny One thing that I would like to get fixed for...
  2. core files The issue of core file management has come up for...
  3. Shared Objects and Big Applications Some time ago I wrote a little utility named memlockd...

Junichi Uekawa: gcloud builds submit failed for me today with a mysterious error message.

4 February, 2020 - 06:35
gcloud builds submit failed for me today with a mysterious error message. kaniko issue?

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้