Planet Debian

Subscribe to Planet Debian feed
Planet Debian - https://planet.debian.org/
Updated: 2 hours 18 min ago

Gunnar Wolf: LIDSOL: teaching privacy and anonymity concepts and tools to social scientists

24 June, 2019 - 10:46

I have been working on several privacy/anonymity topics in the past couple of years. And I am very happy, as we just achieved one of our most important stated goals.

I am coordinating LIDSOL, Laboratorio de Investigación y Desarrollo de Software Libre, at the Engineering Faculty, UNAM. LIDSOL is a very interesting and very open lab regularly inhabited by ≈7 bright students, most of them from Computer Engineering (but some from other careers in the faculty), and with over twenty years of history. And I have worked with several of them in my PAPIME project for privacy and anonymity. This time, the task was –after working a year on the broad topic– for the students to plan and present a course titled Privacidad y anonimato para un manejo seguro de mi información en redes» — Privacy and anonymity for safely handling my online information, as part of the Political and Social Sciences Faculty's intersemestral courses on technological updating.

The covered program was quite ambitious; I'm not translating it, you can look at it in Spanish in the course's information. The LIDSOL instructors (please, a round of applause for them!) were:

  • Diego Barriga>
  • Emilio Cabrera
  • Marco Ruano


My friend Lourdes Reséndiz, who works at FCPyS and got us the space to present the course, also gave a module.

Lourdes, during the famous three envelopes dynamic for explaining onion routing

I felt the course to be a great success, and we were asked to repeat it in the future. As any course presenting anonymization technologies, it was of course not without its controversy and discussion — which was great! I think we got many concepts clarified for the attendees. I will later report on any measurable accounts we got, of course!

AttachmentSize grupo_priv.1500.jpg835.55 KB 2019-II Cedula - Privacidad y anonimato para un manejo seguro de mi información en redes.pdf96.53 KB introd.1500.jpg365.73 KB tres-sobres.1500.jpg382.81 KB

Dirk Eddelbuettel: binb 0.0.4: Several nice improvements

23 June, 2019 - 22:49

The fourth release of the binb package just arrived on CRAN. binb regroups four rather nice themes for writing LaTeX Beamer presentations much more easily in in (R)Markdown. As a teaser, a quick demo combining all four themes follows; documentation and examples are in the package.

This release brings a few nice pull requests. Rob Hyndman, improved the Monash theme. Johan Larsson added support for slide notes, pgfpages and beamer options. Joseph Stachelek corrected the date setting in the Presento theme to reflect the date from the YAML header.

Changes in binb version 0.0.4 (2018-06-23)
  • The Monash theme now has improved color theme handling (Rob Hyndman in #15)

  • The Monash them has a demo, a new tighttoc option and other small improvements (Rob Hyndman in #16).

  • A slide notes example was added to Metropolis, pgfpages can be used if present, beameroption was added to three templates (Johan Larsson in #17).

  • The Presento them now use the date from the yaml header (Joseph Stachelek in #18)

CRANberries provides the usual summary of changes to the previous version.

For questions or comments, please use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Simon Josefsson: OpenPGP smartcard under GNOME on Debian 10 Buster

22 June, 2019 - 01:09

Debian buster is almost released, and today I celebrate midsummer by installing (a pre-release) of it on my Lenovo X201 laptop. Everything went smooth, except for the usual issues with smartcards under GNOME. I use a FST-01G running Gnuk, but the same issue apply to all OpenPGP cards including YubiKeys. I wrote about this problem for earlier releases, read Smartcards on Debian 9 Stretch and Smartcards on Debian 8 Jessie. Some things have changed – now GnuPG‘s internal ccid support works, and dirmngr is installed by default when you install Debian with GNOME. I thought I’d write a new post for the new release.

After installing Debian and logging into GNOME, I start a terminal and attempt to use the smartcard as follows.

jas@latte:~$ gpg --card-status
gpg: error getting version from 'scdaemon': No SmartCard daemon
gpg: OpenPGP card not available: No SmartCard daemon
jas@latte:~$ 

The reason is that the scdaemon package is not installed. Install it as follows.

jas@latte:~$ sudo apt-get install scdaemon

After this, gpg --card-status works. It is now using GnuPG’s internal CCID library, which appears to be working. The pcscd package is not required to get things working any more — however installing it also works, and you might need pcscd if you use other applications that talks to the smartcard.

jas@latte:~$ gpg --card-status
Reader ...........: 234B:0000:FSIJ-1.2.14-67252015:0
Application ID ...: D276000124010200FFFE672520150000
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 67252015
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Sex ..............: male
URL of public key : https://josefsson.org/key-20190320.txt
Login data .......: jas
Signature PIN ....: not forced
Key attributes ...: ed25519 cv25519 ed25519
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 658
KDF setting ......: off
Signature key ....: A3CC 9C87 0B9D 310A BAD4  CF2F 5172 2B08 FE47 45A2
      created ....: 2019-03-20 23:40:49
Encryption key....: A9EC 8F4D 7F1E 50ED 3DEF  49A9 0292 3D7E E76E BD60
      created ....: 2019-03-20 23:40:26
Authentication key: CA7E 3716 4342 DF31 33DF  3497 8026 0EE8 A9B9 2B2B
      created ....: 2019-03-20 23:40:37
General key info..: sub  ed25519/51722B08FE4745A2 2019-03-20 Simon Josefsson <simon@josefsson.org>
sec#  ed25519/D73CF638C53C06BE  created: 2019-03-20  expires: 2019-10-22
ssb>  ed25519/80260EE8A9B92B2B  created: 2019-03-20  expires: 2019-10-22
                                card-no: FFFE 67252015
ssb>  ed25519/51722B08FE4745A2  created: 2019-03-20  expires: 2019-10-22
                                card-no: FFFE 67252015
ssb>  cv25519/02923D7EE76EBD60  created: 2019-03-20  expires: 2019-10-22
                                card-no: FFFE 67252015
jas@latte:~$ 

As before, using the key does not work right away:

jas@latte:~$ echo foo|gpg -a --sign
gpg: no default secret key: No public key
gpg: signing failed: No public key
jas@latte:~$ 

This is because GnuPG does not have the public key that correspond to the private key inside the smartcard.

jas@latte:~$ gpg --list-keys
jas@latte:~$ gpg --list-secret-keys
jas@latte:~$ 

You may retrieve your public key from the clouds as follows. With Debian Buster, the dirmngr package is installed by default so there is no need to install it. Alternatively, if you configured your smartcard with a public key URL that works, you may type “retrieve” into the gpg --card-edit interactive interface. This could be considered slightly more reliable (at least from a self-hosting point of view), because it uses your configured URL for retrieving the public key rather than trusting clouds.

jas@latte:~$ gpg --recv-keys B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE
gpg: key D73CF638C53C06BE: 1 signature not checked due to a missing key
gpg: key D73CF638C53C06BE: public key "Simon Josefsson <simon@josefsson.org>" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1
jas@latte:~$ 

Now signing with the smart card works! Yay! Btw: compare the output size with the output size in the previous post to understand the size advantage with Ed25519 over RSA.

jas@latte:~$ echo foo|gpg -a --sign
-----BEGIN PGP MESSAGE-----

owGbwMvMwCEWWKTN8c/ddRHjaa4khlieP//S8vO5OkpZGMQ4GGTFFFkWn5nTzj3X
kGvXlfP6MLWsTCCFDFycAjARscUM/5MnXTF9aSG4ScVa3sDiB2//nPSVz13Mkpbo
nlzSezowRZrhn+Ky7/O6M7XljzzJvtJhfPvOyS+rpyqJlD+buumL+/eOPywA
=+WN7
-----END PGP MESSAGE-----

As before, encrypting to myself does not work smoothly because of the trust setting on the public key. Witness the problem here:

jas@latte:~$ echo foo|gpg -a --encrypt -r simon@josefsson.org
gpg: 02923D7EE76EBD60: There is no assurance this key belongs to the named user

sub  cv25519/02923D7EE76EBD60 2019-03-20 Simon Josefsson <simon@josefsson.org>
 Primary key fingerprint: B1D2 BD13 75BE CB78 4CF4  F8C4 D73C F638 C53C 06BE
      Subkey fingerprint: A9EC 8F4D 7F1E 50ED 3DEF  49A9 0292 3D7E E76E BD60

It is NOT certain that the key belongs to the person named
in the user ID.  If you *really* know what you are doing,
you may answer the next question with yes.

Use this key anyway? (y/N) 
gpg: signal Interrupt caught ... exiting

jas@latte:~$

You update the trust setting with the gpg --edit-key command.

jas@latte:~$ gpg --edit-key simon@josefsson.org
gpg (GnuPG) 2.2.12; Copyright (C) 2018 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret subkeys are available.

pub  ed25519/D73CF638C53C06BE
     created: 2019-03-20  expires: 2019-10-22  usage: SC  
     trust: unknown       validity: unknown
ssb  cv25519/02923D7EE76EBD60
     created: 2019-03-20  expires: 2019-10-22  usage: E   
     card-no: FFFE 67252015
ssb  ed25519/80260EE8A9B92B2B
     created: 2019-03-20  expires: 2019-10-22  usage: A   
     card-no: FFFE 67252015
ssb  ed25519/51722B08FE4745A2
     created: 2019-03-20  expires: 2019-10-22  usage: S   
     card-no: FFFE 67252015
[ unknown] (1). Simon Josefsson <simon@josefsson.org>

gpg> trust
pub  ed25519/D73CF638C53C06BE
     created: 2019-03-20  expires: 2019-10-22  usage: SC  
     trust: unknown       validity: unknown
ssb  cv25519/02923D7EE76EBD60
     created: 2019-03-20  expires: 2019-10-22  usage: E   
     card-no: FFFE 67252015
ssb  ed25519/80260EE8A9B92B2B
     created: 2019-03-20  expires: 2019-10-22  usage: A   
     card-no: FFFE 67252015
ssb  ed25519/51722B08FE4745A2
     created: 2019-03-20  expires: 2019-10-22  usage: S   
     card-no: FFFE 67252015
[ unknown] (1). Simon Josefsson <simon@josefsson.org>

Please decide how far you trust this user to correctly verify other users' keys
(by looking at passports, checking fingerprints from different sources, etc.)

  1 = I don't know or won't say
  2 = I do NOT trust
  3 = I trust marginally
  4 = I trust fully
  5 = I trust ultimately
  m = back to the main menu

Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y

pub  ed25519/D73CF638C53C06BE
     created: 2019-03-20  expires: 2019-10-22  usage: SC  
     trust: ultimate      validity: unknown
ssb  cv25519/02923D7EE76EBD60
     created: 2019-03-20  expires: 2019-10-22  usage: E   
     card-no: FFFE 67252015
ssb  ed25519/80260EE8A9B92B2B
     created: 2019-03-20  expires: 2019-10-22  usage: A   
     card-no: FFFE 67252015
ssb  ed25519/51722B08FE4745A2
     created: 2019-03-20  expires: 2019-10-22  usage: S   
     card-no: FFFE 67252015
[ unknown] (1). Simon Josefsson <simon@josefsson.org>
Please note that the shown key validity is not necessarily correct
unless you restart the program.

gpg> quit
jas@latte:~$

Confirm gpg --list-keys indicate that the key is now trusted, and encrypting to yourself should work.

jas@latte:~$ gpg --list-keys
/home/jas/.gnupg/pubring.kbx
----------------------------
pub   ed25519 2019-03-20 [SC] [expires: 2019-10-22]
      B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE
uid           [ultimate] Simon Josefsson <simon@josefsson.org>
sub   ed25519 2019-03-20 [A] [expires: 2019-10-22]
sub   ed25519 2019-03-20 [S] [expires: 2019-10-22]
sub   cv25519 2019-03-20 [E] [expires: 2019-10-22]

jas@latte:~$ gpg --list-secret-keys
/home/jas/.gnupg/pubring.kbx
----------------------------
sec#  ed25519 2019-03-20 [SC] [expires: 2019-10-22]
      B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE
uid           [ultimate] Simon Josefsson <simon@josefsson.org>
ssb>  ed25519 2019-03-20 [A] [expires: 2019-10-22]
ssb>  ed25519 2019-03-20 [S] [expires: 2019-10-22]
ssb>  cv25519 2019-03-20 [E] [expires: 2019-10-22]

jas@latte:~$ echo foo|gpg -a --encrypt -r simon@josefsson.org
gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2019-10-22
-----BEGIN PGP MESSAGE-----

hF4DApI9fuduvWASAQdA4FIwM27EFqNK1I5eZERaZVDAXJDmYLZQHjZD8TexT3gw
7SDaeTLm7s0QSyKtsRugRpex6eSVhfA3WG8fUOyzbNv4o7AC/TQdhZ2TDtXZGFtY
0j8BRYIjVDbYOIp1NM3kHnMGHWEJRsTbtLCitMWmLdp4C98DE/uVkwjw98xEJauR
/9ZNmmvzuWpaHuEJNiFjORA=
=tAXh
-----END PGP MESSAGE-----
jas@latte:~$ 

The issue with OpenSSH and GNOME Keyring still exists as in previous releases.

jas@latte:~$ ssh-add -L
The agent has no identities.
jas@latte:~$ echo $SSH_AUTH_SOCK 
/run/user/1000/keyring/ssh
jas@latte:~$ 

The trick we used last time still works, and as far as I can tell, it is still the only recommended method to disable the gnome-keyring ssh component. Notice how we also configure GnuPG’s gpg-agent to enable SSH daemon support.

jas@latte:~$ mkdir ~/.config/autostart
jas@latte:~$ cp /etc/xdg/autostart/gnome-keyring-ssh.desktop ~/.config/autostart/
jas@latte:~$ echo 'Hidden=true' >> ~/.config/autostart/gnome-keyring-ssh.desktop 
jas@latte:~$ echo enable-ssh-support >> ~/.gnupg/gpg-agent.conf 

Log out of GNOME and log in again. Now the environment variable points to gpg-agent’s socket, and SSH authentication using the smartcard works.

jas@latte:~$ echo $SSH_AUTH_SOCK 
/run/user/1000/gnupg/S.gpg-agent.ssh
jas@latte:~$ ssh-add -L
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILzCFcHHrKzVSPDDarZPYqn89H5TPaxwcORgRg+4DagE cardno:FFFE67252015
jas@latte:~$ 

Topics for further discussion and research this time around includes:

  1. Should scdaemon (and possibly pcscd) be pre-installed on Debian desktop systems?
  2. Could gpg --card-status attempt to import the public key and secret key stub automatically? Alternatively, some new command that automate the bootstrapping of a new smartcard.
  3. Should GNOME keyring support smartcards?
  4. Why is GNOME keyring used by default for SSH rather than gpg-agent?
  5. Should gpg-agent default to enable the SSH daemon?
  6. What could be done to automatically infer the trust setting for a smartcard based private key?

Thanks for reading and happy smartcarding!

Candy Tsai: Outreachy Week 5: What is debci?

21 June, 2019 - 16:33

The theme for this week in Outreachy is “Think About Your Audience”. So I’m currently thinking about you.

Or not?

After being asked sooo many times what am I doing for this internship, I think I never explained it well enough so that others could understand. Let me give it a try here.

debci is short for “Debian Continuous Integration”, so I’ll start with a short definition of what “Continuous Integration” is then!

Continuous Integration (CI)

Since there should be quite some articles talking about this topic, here is a quick explanation that I found on Microsoft Azure (link):

Continuous Integration (CI) is the process of automating the build and testing of code every time a team member commits changes to version control.

A scenario would be whenever I push code to Debian Salsa Gitlab, it automatically runs the tests we have written in our code. This is to make sure that the new code changes doesn’t break the stuff that used to work.

The debci Project

Before Debian puts out a new release, all of the packages that have tests written in them need to be tested. debci is a platform for testing packages and provides a UI to see if they pass or not. The goal is to make sure that the the packages will pass their tests before a major Debian release. For example, when the ruby-defaults package is updated, we not only want to test ruby-defaults but also all the packages that depend on it. In short, debci helps make sure the packages are working correctly.

For my internship, I am working on improving the user experience through the UI of the debci site. The most major task is to let developers easily test their packages with packages in different suites and architectures.

The terms that keep popping up in debci are:

  • suite
  • architecture
  • pin-packages
  • trigger

There are three obvious suites on the debci site right now, namely unstable, testing and stable. There is also an experimental suite that a user can test their packages upon. And architecture is like amd64 or arm64.

The life of a package is something like this:

  1. When a package is updated/added, it goes into unstable
  2. After it has stayed 2-10 days in unstable without any big issues, it moves into testing
  3. It becomes stable after a release
Normally a package moves from unstable > testing > stable

Let’s say, a user wants to test the ruby-defaults package in unstable on amd64 along with a package C from experimental. Here package C would be a pin-package, which means the package the user wants to test along with. Last but not least, trigger is just the name of the test job, one can choose to use it or not.

Currently, there is an API that you make this request with curl or something similar, but it’s not very friendly since not everyone is familiar with how the request should look like. Therefore, not a lot of people are willing to use it and I have also seen requests for an improvement on this in the #debci channel. An easy to use UI might be the solution to make requesting these tests easier. Knowing what I am working on is useful for others is an important key to keep myself motivated.

The debci Community

The debci community is very small but active. The people that are directly involved are my mentors: terceiro and elbrus. Sometimes people drop by the IRC channel to ask questions, but basically that’s it. This works pretty good for me, because I’m usually not at ease and will keep a low profile if the community gets too large.

I’m not familiar with the whole Debian community, but I have also been hanging around the #debian-outreach channel. It felt warm to know that someone realized there was an intern from Taiwan for this round of Outreachy. As far as I have experienced, everyone I have chatted with were nice and eager to share which Debian-related communities were close to me.

Week 5: Modularizing & Testing

This week I worked on adding tests and tried pulling out the authentication code to make the code a bit more DRY.

  • Learned how to setup tests in Ruby
  • Came up with test cases
  • Learned more about how classes work in Ruby
  • Separated the authentication code

And… probably also writing this blog post! I found that blogging takes up more time than I thought it should.

Sven Hoexter: logstash json filter error

21 June, 2019 - 15:08

If you've a logstash filter that contains a json filter/decoding step like

filter { json { source => "log" } }

this, and you end up with an error message like that:

[2019-06-21T09:47:58,243][WARN ][logstash.filters.json ] Error parsing json {:source=>"log", :raw=>{"file"=>{"path"=>"/var/lib/docker/containers/abdf3db21fca8e1dc17c888d4aa661fe16ae4371355215157cf7c4fc91b8ea4b/abdf3db21fca8e1dc17c888d4aa661fe16ae4371355215157cf7c4fc91b8ea4b-json.log"}}, :exception=>java.lang.ClassCastException}

It might be just telling you that the field log actually does contain valid json, and no decoding is required.

Mathieu Parent: Announcing GitLabracadabra 0.2.1

20 June, 2019 - 22:13

Hello all,

Mid-October I started at work a tool in Python to create and update our projects hosted in our GitLab instance.

I have now reworked and published this tool called GitLabracadabra.

This tool is still very young and documentation is sparse, but following the “release early, release often” motto I think it is ready for general usage.

Please report any bug or suggestion in the bug tracker. that I just enabled :

$ gitlabracadabra --verbose
2019-06-20 16:59:19,144 [29120] INFO gitlabracadabra.objects.object: [gitlabracadabra/gitlabracadabra] Changing param issues_enabled: False -> True

Louis-Philippe Véronneau: membernator -- validate membership cards

20 June, 2019 - 10:45

I currently work part-time for student unions in Montreal and they often have large general assemblies (more than 2000 people). As you can likely figure out by yourself, running through paper lists to validate people's identity is a real PITA and takes quite a long time.

For example, even if you have 4 people checking names, if validating someone's identity takes 5 seconds on average (that's pretty fast), it takes around 40 minutes to go through 2000 people.

Introducing membernator, a python program written using pygame that validates membership cards against a CSV database! The idea is to use barcode scanners to scan people's school ID cards and see if they are in our digital lists. Hopefull, it will make our GA process easier for everyone.

I want to thank Jonathan Carter who provided the inspiration (and a codebase) for this project. membernator is a heavily-modified fork of ToeTally, a program currently in developpement for the DebConf Video Team.

membernator will eventually be packaged in Debian (I've started packaging stuff!), but for now you can either install it manually or get it from PyPi.

Here's a quick video of what running membernator looks like. I'm typing the IDs by hand since I left my barcode scanner at work. Excuse the weird screen glitches, it seems I'm somewhat bad a screen recording.

Debian GSoC Kotlin project blog: Finished converting all the buildfiles to groovy and downgraded to gradle 4.4.1; week 3+ update

20 June, 2019 - 00:49
Converting build files to groovy

During the third week I mainly spent my time converting all the buildfiles in the "dist" task graph to groovy from kotlin-dsl.

I finished converting all the build files from kotlin-dsl to groovy. I then proceeded to build the entire project with only the subprojects required for the dist task so that we can avoid converting all the uneeded subproject buildfiles to groovy. Ran tests on the binary obtained from the newly onverted project and compared it to the test result on an original unconverted project. Since the new project only contains the needed subprojects this new project is unable to run all the needed tests. So inorder to overcome this we copy the binaries built by our new project and run the tests using the original unaltered projects. The compiler test task we need is "compilerTest"; this is the only aplicalbe test for out build binary from the "dist" task. I have run "distTest" for the unaltered project and uploaded it here; "distTest" task encompasses compilerTest task within it. Here is the log of the "compilerTest" run on the geenrated binaries.

Downgrading the Project to gradle 4.4.1

I also have downgraded the project to gradle 4.4.1 decrementaly from 5.1 to 4.7 to 4.4.1. Since gradle 4.4.1 isn't defaulty compatible with Openjdk-11 (defualt Java version in Debian Sid) I used an external JDK-9 to build the project with an external gradle 4.4.1 and confirmed it has indeed been downgraded. Next step would be begining to make this project buildable with Debian Sid gradle.

Closing notes and things to note

I figure that to those unfamiliar with Debian packaging it would be useful to know that we are going to upload a non-free prebuilt pacakge containing only the necessary binaries for bootstraping. Then using this we can build this project in a more Debian way. Also I am posting the enviornment vraibles followed by build command to build this project as of this blog. 1. export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64" 2. export JDK_16="/usr/lib/jvm/java-8-openjdk-amd64" 3. export JDK_17="/usr/lib/jvm/java-8-openjdk-amd64" 4. export JDK_18="/usr/lib/jvm/java-8-openjdk-amd64" 5. export JDK_9="/home/Java/jdk-9.0.4_linux-x64_bin/jdk-9.0.4" 6. ./gradlew -Pteamcity=true dist

Here is a link to the work I have done so
far. You can find me as m36 or m36[m] on #debian-mobile and #debian-java in OFTC.

I ll try to maintain this blog and post the major updates weekly.

Neil McGovern: GNOME ED Update – April/May

19 June, 2019 - 17:04

It’s time for another update on what the GNOME Foundation has been up to, this time in April and May.

Events

We’ve been to a number of events in the last couple of months. April saw myself, Kristi, Bastian, Anisa and Stefano at FOSS-North in Sweden. Zeeshan Ali presented a talk on Open Source Geolocation.

At the end of April, Molly de Blanc and Sri Ramkrishna were at Linux Fest North West. Additionally, Molly delivered a talk related to community guideline enforcement, which was featured on the LFNW web page.

We also had a couple of hackfests in may – Rust+GNOME Hackfest #5 in Berlin at the start of the month, and the GStreamer Spring Hackfest 2019 in Oslo at the end of May.

Coming up in July, we’ll be attending OSCON and having a West Coast Hackfest – a combined 3-in-1 hackfest bringing in GTK, Documentation and Engagement teams!

GUADEC and GNOME.Asia

GUADEC and GNOME.Asia planning is now very much underway, and they’ve now both announced their venues and dates – GUADEC will be in Thessaloniki, Greece at the end of August, and GNOME.Asia will be in Gresik, Indonesia mid October! As always, we’re after sponsors for both of these, so if you know of any organisations who can help, please pass along our sponsorship brochure.

Staffing

For those that didn’t see my announcement, Molly de Blanc joined the Foundation as our Strategic Initiatives Manager! Molly comes from the Free Software Foundation where she was the Campaigns Manager, working on community organising around digital rights issues.
She’s also the President of the Board of Directors of the Open Source Initiative, and on the Debian Outreach and Anti-harassment teams. Regularly speaking at conferences around the world, she has represented multiple projects in community and corporate contexts.

Discourse

We’ve also been trying something new – we moved the gtk lists away from Mailman and over to discourse.gnome.org. The uptake has been rather impressive – we’re now seeing more topics on Discourse then all gtk-* lists grouped together, and more and more people engaged. We also moved over builder, and are looking at other lists, with a possible goal of eventually retiring mailman all together for general purpose discussions. If you’re interested, let me know!

Google Summer of Code

The Google Summer of Code internships are now underway, and we have a total of 10 students working for GNOME:

  • Sajeer Ahamed Riyaf – Converting GStreamer plugins to Rust
  • Srestha Srivastava – Improve GNOME Boxes express-installations by adding support to tree-based installations
  • AJ Jordan – Implement a Migration Assistant
  • Andrei Lişiţă – Add a saved states manager for GNOME Games
  • Xiang Fan – Update of gtk-rs to support GTK4
  • Stefanos Dimos – Adding the ability to preview links in Polari
  • Mayank_Sharma – Improve Google-Drive support for Gvfs
  • Ravgeet Dhillon – Rework the GTK website
  • Sumaid – Full stack MusicBrainz integration for GNOME Music
  • Gaurav Agrawal – Implement side by side diffs in Gitg

This is a fantastic set of projects, and I’m sure all students will be welcomed warmly!

Petter Reinholdtsen: Jami/Ring, finally functioning peer to peer communication client

19 June, 2019 - 13:45

Some years ago, in 2016, I wrote for the first time about the Ring peer to peer messaging system. It would provide messaging without any central server coordinating the system and without requiring all users to register a phone number or own a mobile phone. Back then, I could not get it to work, and put it aside until it had seen more development. A few days ago I decided to give it another try, and am happy to report that this time I am able to not only send and receive messages, but also place audio and video calls. But only if UDP is not blocked into your network.

The Ring system changed name earlier this year to Jami. I tried doing web search for 'ring' when I discovered it for the first time, and can only applaud this change as it is impossible to find something called Ring among the noise of other uses of that word. Now you can search for 'jami' and this client and the Jami system is the first hit at least on duckduckgo.

Jami will by default encrypt messages as well as audio and video calls, and try to send them directly between the communicating parties if possible. If this proves impossible (for example if both ends are behind NAT), it will use a central SIP TURN server maintained by the Jami project. Jami can also be a normal SIP client. If the SIP server is unencrypted, the audio and video calls will also be unencrypted. This is as far as I know the only case where Jami will do anything without encryption.

Jami is available for several platforms: Linux, Windows, MacOSX, Android, iOS, and Android TV. It is included in Debian already. Jami also work for those using F-Droid without any Google connections, while Signal do not. The protocol is described in the Ring project wiki. The system uses a distributed hash table (DHT) system (similar to BitTorrent) running over UDP. On one of the networks I use, I discovered Jami failed to work. I tracked this down to the fact that incoming UDP packages going to ports 1-49999 were blocked, and the DHT would pick a random port and end up in the low range most of the time. After talking to the developers, I solved this by enabling the dhtproxy in the settings, thus using TCP to talk to a central DHT proxy instead of peering directly with others. I've been told the developers are working on allowing DHT to use TCP to avoid this problem. I also ran into a problem when trying to talk to the version of Ring included in Debian Stable (Stretch). Apparently the protocol changed between beta2 and the current version, making these clients incompatible. Hopefully the protocol will not be made incompatible in the future.

It is worth noting that while looking at Jami and its features, I came across another communication platform I have not tested yet. The Tox protocol and family of Tox clients. It might become the topic of a future blog post.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Dirk Eddelbuettel: anytime 0.3.4

18 June, 2019 - 18:28

A new minor release of the anytime package is arriving on CRAN. This is the fifteenth release, and first since the 0.3.3 release in November.

anytime is a very focused package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, … format to either POSIXct or Date objects – and to do so without requiring a format string. See the anytime page, or the GitHub README.md for a few examples.

This release is mostly internal and switches to the excellent new tinytest package, a tweak the iso8601() format helper which now uses T between date and time (which is a breaking change with the usual addition of a option to get the old behaviour back) and a little more. The full list of changes follows.

Changes in anytime version 0.3.4 (2019-06-18)
  • Documentation was updated about a 'Europe/London' conversion issue (#84, inter alia).

  • The package is now compiled under the C++11 standard.

  • The package now uses tinytest for unit tests.

  • The iso8601() function now places a ‘T’ between date and time; an option switches to prior format using a space.

  • The vignette is now pre-made and included as-is in a Sweave document reducing the number of suggested packages.

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. The issue tracker tracker off the GitHub repo can be use for questions and comments.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Steinar H. Gunderson: 0 bytes left

18 June, 2019 - 05:45

Around 2003–2004, a friend and I wrote a softsynth that was used in a 64 kB intro. Now, 14 years later, cTrix and Pselodux picked it up and made a really cool 32 kB tune with it! Who would have thought.

(For the record, the synth plus the original Nemesis tune fit under 16 kB given the right packer and some squeezing, even with some LPC samples. But there's heck of a lot more notes in this one :-) )

Emmanuel Kasper: Normalize a bunch of audio files to the same loudness

17 June, 2019 - 20:36
I had a bunch of audio files in a directory, each recorded live with different devices, and it proved very ear-painful to hear the audio files in a playlist because of the difference of loudness.
To normalize audio filesm  you can find a number of tool working with ID3 tags, but after testing with vlc, mplayer, and the pogo mp3 player none of them did produce a measurable change. So I converted everything to wav, normalized the wav files, then converted back to mp3.

delete funny chars and spaces in file names
detox music_dir
converting files to wav is just a matter of
# this uses zsh recusrive globbing
for file in **/*.mp3 ; do ffmpeg -i $file  "$(basename $file .mp3).wav"; done
normalizing files with the normalize-audio program, from the debian package of the same name.
# this uses zsh recusrive globbing
normalize-audio **/*.wav
converting back to mp3
for f in $(ls); do ffmpeg -b:a 192k -acodec libmp3lame -i $f $(basename $f .wav).mp3; done

Manuel A. Fernandez Montecelo: Debian GNU/Linux riscv64 port in mid 2019

17 June, 2019 - 09:00

It's been a while since last post (Talk about the Debian GNU/Linux riscv64 port at RISC-V workshop), and sometimes things look very quiet from outside even if the people on the backstage never stop working. So this is an update on the status of this port before the release of buster, which should happen in a few weeks and which it will open the way for more changes that will benefit the port.

The Big Picture

First, the big picture(s):

Debian-Ports All-time Graph, 2019-06-17

As it can be seen in the first graph, perhaps with some difficulty, is that the percent of arch-dependent packages built for riscv64 (grey line) has been around or higher than 80% since mid 2018, just a few months after the port was added to the infrastructure.

Given than the arch-dependent packages are about half of the Debian['s main, unstable] archive and that (in simple terms) arch-independent packages can be used by all ports (provided that the software that they rely on is present, e.g. a programming language interpreter), this means that around 90% of packages of the whole archive has been available for this architecture from early on.

Debian-Ports Quarter Graph, 2019-06-17

The second graph shows that the percentages are quite stable (for all architectures, really: the peaks and dips in the graph only represent <5% of the total). This is in part due to the freeze for buster, but it usually happens at other times as well (except in the initial bring-up or in the face of severe problems), and it really shows that even the second-class ports are in quite good health in broad terms.


Note: These graphs are for architectures in the debian-ports infrastructure (which hosts architectures not as well supported as the main ones, the ones present in stable releases). The graphs are taken from the buildd stats page, which also includes the main supported architectures.

A little big Thank You

Together, both graphs are also testament that there are people working on ports at all times, keeping things working behind the scenes, and that's why from a high level view it seems that things “just work”.

More in general, aside from the work of porters themselves, there are also people working on bootstrapping issues that make bringing up ports easier than in the past, or coping better when toolchain support or other issues deal an important blow to some ports. And, of course, all other contributors of Debian help by keeping good tools and building rules that work across architectures, patching the upstream software for needs of several architectures at the same time (endianness, width of basic types), many upstream projects are generic enough that they don't need specific porting, etc.

Thanks to all of you!

Next Steps Installation on hardware, VMs, etc.

Due to several reasons, among them the limited availability of hardware able to run this Debian port and the limited options to use bootloaders during all this time, the instructions to get Debian running on RISC-V are not the best, easiest, more elegant or very up to date. This is an area to improve in the next months.

Meanwhile, there's a Debian RISC-V's wiki page with instructions to get a chroot working in a HiFive Unleashed board as shipped, without destroying the initial factory set-up.

Specially Vagrant Cascadian and Karsten Merker have been working on the area of booting the system, and there are instructions to set-up a riscv64 Qemu VM and boot it with u-boot and opensbi. Karsten is also working to get support in debian-installer, the main/canonical way to install Debian systems (perhaps less canonical nowadays with the use of OS images, but still hugely important).

Additionally, it would be nice to have images publicly available and ready to use, for both Qemu and hardware available like the HiFive Unleashed (or others that might show up in time), but although there's been some progress on that, it's still not ready and available for end users.

The last 10%+ of the archive

So, what's the remaining work left to have almost all of the archive built for this architecture? What's left to port, as such?

The main blockers to get closer to 100% of packages built are basically LLVM and Rust (which, in turn, depends on LLVM).

Currently there are more than 500 packages from the Rust ecosystem in the archive (which is about 4%), and they cannot be built and used until Rust has support for the architecture. And Rust needs LLVM, there's no Rust compiler based on GCC or other toolchains (as it's the case of Go, for example, in which there's a gcc-go compiler in addition to their own golang-go), so this is the only alternative.

Firefox is the main high-level package that depends on Rust, but many packages also depend on librsvg2 to render SVG images, and this library has been converted to Rust. We're still using the C version for that, but it cannot be sustained in the long term.

Aside from Rust, other packages directly depend or use LLVM to some extent, and this is not fully working for riscv64 at the moment, but it is expected that during 2019 the support of LLVM for riscv64 will be completed.

There are other programming language ecosystems that need attention, but they represent a really low percentage (only dozens of packages, of more than 12 thousand; and with no dependencies outside that set). And then, of course, there is long tail of packages that cannot be built due to a missing dependency, lack of support for the architecture or random failures -- together they make a substantial number of the total, but they need to be looked at and solved almost on a case-by-case basis.

Finally, when the gates of the unstable suite open again after the freeze for the stable release of buster, we will see tools with better support and patches can be accepted again to support riscv64, so we can hope that things will improve at a faster rate soon :-)

Russ Allbery: Review: Abaddon's Gate

16 June, 2019 - 12:17

Review: Abaddon's Gate, by James S.A. Corey

Series: The Expanse #3 Publisher: Orbit Copyright: 2013 ISBN: 0-316-23542-3 Format: Kindle Pages: 540

Abaddon's Gate is the third book in the Expanse series, following Caliban's War. This series tells a single long story, so it's hard to discuss without spoilers for earlier books although I'll try. It's a bad series to read out of order.

Once again, solar system politics are riled by an alien artifact set up at the end of the previous book. Once again, we see the fallout through the eyes of multiple viewpoint characters. And, once again, one of them is James Holden, who starts the book trying to get out of the blast radius of the plot but is pulled back into the center of events. But more on that in a moment.

The other three viewpoint characters are, unfortunately, not as strong as the rest of the cast in Caliban's War. Bull is the competent hard-ass whose good advice is repeatedly ignored. Anna is a more interesting character, a Methodist reverend who reluctantly leaves her wife and small child to join an interfaith delegation (part of a larger delegation of artists and philosophers, done mostly as a political stunt) to the alien artifact at the center of this book. Anna doesn't change that much over the course of the book, but her determined, thoughtful kindness and intentional hopefulness was appealing to read about. She also has surprisingly excellent taste in rich socialite friends.

The most interesting character in the book is the woman originally introduced as Melba. Her obsessive quest for revenge drives much of the plot, mostly via her doing awful things but for reasons that come from such a profound internal brokenness, and with so much resulting guilt, that it's hard not to eventually feel some sympathy. She's also the subject of the most effective and well-written scene in the book: a quiet moment of her alone in a weightless cell, trying to position herself in its exact center. (Why this is so effective is a significant spoiler, but it works incredibly well in context.)

Melba's goal in life is to destroy James Holden and everything he holds dear. This is for entirely the wrong reasons, but I had a hard time not feeling a little bit sympathetic to that too.

I had two major problems with Abaddon's Gate. The first of them is that this book (and, I'm increasingly starting to feel, this series) is about humans doing stupid, greedy, and self-serving things in the face of alien mystery, with predictably dire consequences. This is, to be clear, not in the slightest bit unrealistic. Messy humans being messy in the face of scientific wonder (and terror), making tons of mistakes, but then somehow muddling through is very in character for our species. But realistic doesn't necessarily mean entertaining.

A lot of people die or get seriously injured in this book, and most of that is the unpredictable but unsurprising results of humans being petty assholes in the face of unknown dangers instead of taking their time and being thoughtful and careful. The somewhat grim reputation of this series comes from being relatively unflinching about showing the results of that stupidity. Bad decisions plus forces that do not care in the slightest about human life equals mass casualties. The problem, at least for me personally, is this is not fun to read about. If I wanted to see more of incompetent people deciding not to listen to advice or take the time to understand a problem, making impetuous decisions that make them feel good, and then turning everything to shit, I could just read the news. Bull as a viewpoint character doesn't help, since he's smart enough to see the consequences coming but can't stop them. Anna is the one character who manages to reverse some of the consequences by being a better person than everyone else, and that partly salvages the story, but there wasn't enough of that.

The other problem is James Holden. I was already starting to get annoyed with his self-centered whininess in Caliban's War, but in Abaddon's Gate it turns into eye-roll-inducing egomania. Holden seems convinced that everything that happens is somehow about him personally, and my tolerance for self-centered narcissists is, shall we say, at a historically low ebb. There's a point late in this book when Holden decides to be a sexist ass to Naomi (I will never understand what that woman sees in him), and I realized I was just done. Done with people pointing out to Holden that he's just a wee bit self-centered, done with him going "huh, yeah, I guess I am" and then making zero effort to change his behavior, done with him being the center of the world-building for no good reason, done with plot armor and the clear favor of the authors protecting him from consequences and surrounding him with loyalty he totally doesn't deserve, done with his supposed charisma which is all tell and no show. Just done. At this point, I actively loathe the man.

The world-building here is legitimately interesting, if a bit cliched. I do want to know where the authors are going with their progression of alien artifacts, what else humanity might make contact with, and what the rest of the universe looks like. I also would love to read more about Avasarala, who sadly didn't appear in this book but is the best character in this series so far. I liked Anna, I ended up surprising myself and liking Melba (or at least the character she becomes), and I like most of Holden's crew. But I may be done with the series here because I'm not sure I can take any more of Holden. I haven't felt this intense of dislike for a main series character since I finally gave up on The Wheel of Time.

Abaddon's Gate has a lot of combat, a lot of dead people, and a lot of gruesome injury, all of which is belabored enough that it feels a bit padded, but it does deliver on what it promises: old-school interplanetary spaceship fiction with political factions, alien artifacts, some mildly interesting world-building, and, in Melba, some worthwhile questions about what happens after you've done something unforgivable. It doesn't have Avasarala, and therefore is inherently far inferior to Caliban's War, but if you liked the previous books in the series, it's more of that sort of thing. If Holden has been bothering you, though, that gets much worse.

Followed by Cibola Burn.

Rating: 6 out of 10

Erich Schubert: Chinese Citation Factory

16 June, 2019 - 05:02

RetractionWatch published in Feburary 2018 an article titled “A journal waited 13 months to reject a submission. Days later, it published a plagiarized version by different authors”, indicating that in the journal Multimedia Tools and Applications (MTAP) may have been manipulated in the editorial process.

Now, more than a year later, Springer apparently has retracted additional articles from the journal, as mentioned in the blog For Better Science. On the downside, Elsevier has been publishing many of these in another journal now instead…

I am currently aware of 22 retractions associated with this incident. One would have expected to see a clear pattern in the author names, but they seem to have little in common except Chinese names and affiliations, and suspicious email addresses (also, usually only one author has an email at all). It almost appears as if the names may be made up. And these retracted papers clearly contained citation spam: they cite a particular author very often, usually in a single paragraph.

The retraction notices typically include the explanation “there is evidence suggesting authorship manipulation and an attempt to subvert the peer review process”, confirming the earlier claims by Retraction Watch.

So I used the CrossRef API to get the citations from all the articles (I tried SemanticScholar first, but for some of the retracted papers it only had the self-cite of the retraction notice), and counted the citations in these papers.

Essentially, I am counting how many citations authors lost by the retractions.

Here is the “high score” with the top 5 citation losers:

Author Citations lost Cited in papers Citation share Retractions L. Zhang 385 20 60.6% 1 M. Song 68 20 10.9% 0 C. Chen 65 19 11.1% 0 X. Liu 65 19 11.0% 0 R. Zimmermann 60 18 10.8% 0

Now this is a surprisingly clear pattern. In 20 of the retracted papers, L. Zhang was cited on average 19.25 times. In these papers, also 60% of the references were co-authored by him. In one of the remaining two papers, he was an author. The next authors seem to be mostly in this list because of co-authoring with L. Zhang earlier. In fact, if we ignore all citations to papers co-authored by L. Zhang, no author receives more than 5 citations anymore.

So this very clearly suggests that L. Zhang manipulated the MTAP journal to boost his citation index. And it is quite disappointing how long it took until Springe retracted those articles! Judging by the For Better Science article, there may be even more affected papers.

Joey Hess: hacking water

15 June, 2019 - 23:02

From water insecurity to offgrid, solar pumped, gravity flow 1000 gallons of running water.

I enjoy hauling water by hand, which is why doing it for 8 years was not really a problem. But water insecurity is; the spring has been drying up for longer periods in the fall, and the cisterns have barely been large enough to get through.

And if I'm going to add storage, it ought to be above the house, so it can gravity flow. And I have these old 100 watts of solar panels sitting unused after my solar upgrade. And a couple of pumps for a pressure tank system that was not working when I moved in. And I stumbled across an odd little flat spot halfway up the hillside. And there's an exposed copper pipe next to the house's retaining wall; email to Africa establishes that it goes down and through the wall and connects into the plumbing.

So I have an old system that doesn't do what I want. Let's hack the system..

(This took a year to research and put together, including learning a lot about plumbing.)

Run a cable from the old solar panels 75 feet over to the spring. Repurpose an old cooler as a pumphouse, to keep the rain off the Shurflow pump, and with the opening facing so it directs noise away from living areas. Add a Shurflow 902-200 linear current booster to control the pump.

Run a temporary pipe up to the logging road, and verify that the pump can just manage to push the water up there.

Sidetrack into a week spent cleaning out and re-sealing the spring's settling tank. This was yak shaving, but it was going to fail. Build a custom ladder because regular ladders are too wide to fit into it. Flashback to my tightest squeezes from caving. Yuurgh.

Install water level sensors in the settling tank, cut a hole for pipe, connect to pumphouse.

Now how to bury 250 feet of PEX pipe a foot deep up a steep hillside covered in rock piles and trees that you don't want to cut down to make way for equipment? Research every possibility, and pick the one that involves a repurposed linemans's tool resembling a medieval axe.

Dig 100 feet of 1 inch wide trench in a single afternoon by hand. Zeno in on the rest of the 300 foot run. Gain ability to bury underground cables without raising a sweat as an accidental superpower. Arms ache for a full month afterwards.

Connect it all up with a temporary water barrel, and it works! Gravity flow yields 30 PSI!

Pressure-test the copper pipe going into the house to make sure it's not leaking behind the retaining wall. Fix all the old leaky plumbing and fixtures in the house.

Clear a 6 foot wide path through the woods up the hill and roll up two 550 gallon Norwesco water tanks. Haul 650 pounds of sand up the hill, by hand, one 5 gallon bucket at a time. Level and prepare two 6 foot diameter pads.

Build a buried manifold with valves turned by water meter key. Include a fire hose outlet just in case.

Begin filling the tanks, unsure how long it will take as the pump balances available sunlight and spring flow.

François Marier: OpenSUSE 15 LXC setup on Ubuntu Bionic 18.04

15 June, 2019 - 10:15

Similarly to what I wrote for Fedora, here is how I was able to create an OpenSUSE 15 LXC container on an Ubuntu 18.04 (bionic) laptop.

Setting up LXC on Ubuntu

First of all, install lxc:

apt install lxc
echo "veth" >> /etc/modules
modprobe veth

turn on bridged networking by putting the following in /etc/sysctl.d/local.conf:

net.ipv4.ip_forward=1

and applying it using:

sysctl -p /etc/sysctl.d/local.conf

Then allow the right traffic in your firewall (/etc/network/iptables.up.rules in my case):

# LXC containers
-A FORWARD -d 10.0.3.0/24 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 10.0.3.0/24 -j ACCEPT
-A INPUT -d 224.0.0.251 -s 10.0.3.1 -j ACCEPT
-A INPUT -d 239.255.255.250 -s 10.0.3.1 -j ACCEPT
-A INPUT -d 10.0.3.255 -s 10.0.3.1 -j ACCEPT
-A INPUT -d 10.0.3.1 -s 10.0.3.0/24 -j ACCEPT

and apply these changes:

iptables-apply

before restarting the lxc networking:

systemctl restart lxc-net.service
Creating the container

Once that's in place, you can finally create the OpenSUSE 15 container:

lxc-create -n opensuse15 -t download -- -d opensuse -r 15 -a amd64

To see a list of all distros available with the download template:

lxc-create -n foo --template=download -- --list
Logging in as root

Start up the container and get a login console:

lxc-start -n opensuse15 -F

In another terminal, set a password for the root user:

lxc-attach -n opensuse15 passwd

You can now use this password to log into the console you started earlier.

Logging in as an unprivileged user via ssh

As root, install a few packages:

zypper install vim openssh sudo man
systemctl start sshd
systemctl enable sshd

and then create an unprivileged user:

useradd francois
passwd francois
cd /home
mkdir francois
chown francois:100 francois/

and give that user sudo access:

visudo  # uncomment "wheel" line
groupadd wheel
usermod -aG wheel francois

Now login as that user from the console and add an ssh public key:

mkdir .ssh
chmod 700 .ssh
echo "<your public key>" > .ssh/authorized_keys
chmod 644 .ssh/authorized_keys

You can now login via ssh. The IP address to use can be seen in the output of:

lxc-ls --fancy

Eddy Petri&#537;or: How to generate a usable map file for Rust code - and related (f)rustrations

15 June, 2019 - 07:24
Intro
Cargo does not produce a .map file, and if it does, mangling makes it very unusable. If you're searching for the TLDR, read from "How to generate a map file" on the bottom of the article.
MotivationAs a person with experience in embedded programming I find it very useful to be able to look into the map file.

Scenarios where looking at the map file is important:
  • evaluate if the code changes you made had the desired size impact or no undesired impact - recently I saw a compiler optimize for speed an initialization with 0 of an array by putting long blocks of u8 arrays in .rodata section
  • check if a particular symbol has landed in the appropriate memory section or region
  • make an initial evaluation of which functions/code could be changed to optimize either for code size or for more readability (if the size cost is acceptable)
  • check particular symbols have expected sizes and/or alignments
Rustrations  Because these kind of scenarios  are quite frequent in my work and I am used to looking at the .map file, some "rustrations" I currently face are:
  1. No map file is generated by default via cargo and information on how to do it is sparse
  2. If generated, the symbols are mangled and it seems each symbol is in a section of its own, making per section (e.g. .rodata, .text, .bss, .data) or per file analysys more difficult than it should be
  3. I haven't found a way disable mangling globally, without editing the rust sources. - I remember there is some tool to un-mangle the output map file, but I forgot its name and I find the need to post-process suboptimal
  4. no default map file filename or location - ideally it should be named as the crate or app, as specified in the .toml file.
How to generate a map fileGenerating map file for linux (and possibly other OSes)Unfortunately, not all architectures/targets use the same linker, or on some the preferred linker could change for various reasons.

Here is how I managed to generate a map file for an AMD64/X86_64 linux target where it seems the linker is GLD:

Create a .cargo/config file with the following content:

.cargo/config: [build]
    rustflags = ["-Clink-args=-Wl,-Map=app.map"]
This should apply to all targets which use GLD as a linker, so I suspect this is not portable to Windows integrated with MSVC compiler.

Generating a map file for thumb7m with rust-lld
On baremetal targets such as Cortex M7 (thumbv7m where you might want to use the llvm based rust-lld, more linker options might be necessary to prevent linking with compiler provided startup code or libraries, so the config would look something like this:
.cargo/config: 
[build]
target = "thumbv7m-none-eabi"
rustflags = ["-Clink-args=-Map=app.map"]
The thins I dislike about this is the fact the target is forced to thumbv7m-none-eabi, so some unit tests or generic code which might run on the build computer would be harder to test.

Note: if using rustc directly, just pass the extra options
Map file generation with some readable symbolsAfter the changes above ae done, you'll get an app.map file (even if the crate is of a lib) with a predefined name, If anyone knows ho to keep the crate name or at least use lib.map for libs, and app.map for apps, if the original project name can't be used.

The problems with the generated linker script are that:
  1. all symbol names are mangled, so you can't easily connect back to the code; the alternative is to force the compiler to not mangle, by adding the #[(no_mangle)] before the interesting symbols.
  2. each symbol seems to be put in its own subsection (e.g. an initalized array in .data.
Dealing with manglingFor problem 1, the fix is to add in the source #[no_mangle] to symbols or functions, like this:

#[no_mangle]
pub fn sing(start: i32, end: i32) -> String {
    // code body follows
}Dealing with mangling globallyI wasn't able to find a way to convince cargo to apply no_mangle to the entire project, so if you know how to, please comment. I was thinking using #![no_mangle] to apply the attribute globally in a file would work, but is doesn't seem to work as expected: the subsection still contains the mangled name, while the symbol seems to be "namespaced":

Here is a some section from the #![no_mangle] (global) version:
.text._ZN9beer_song5verse17h0d94ba819eb8952aE
                0x000000000004fa00      0x61e /home/eddy/usr/src/rust/learn-rust/exercism/rust/beer-song/target/release/deps/libbeer_song-d80e2fdea1de9ada.rlib(beer_song-d80e2fdea1de9ada.beer_song.5vo42nek-cgu.3.rcgu.o)
                0x000000000004fa00                beer_song::verse
 When the #[no_mangle] attribute is attached directly to the function, the subsection is not mangled and the symbol seems to be global:

.text.verse    0x000000000004f9c0      0x61e /home/eddy/usr/src/rust/learn-rust/exercism/rust/beer-song/target/release/deps/libbeer_song-d80e2fdea1de9ada.rlib(beer_song-d80e2fdea1de9ada.beer_song.5vo42nek-cgu.3.rcgu.o)
                0x000000000004f9c0                verseI would prefer to have a cargo global option to switch for the entire project, and code changes would not be needed, comment welcome.
Each symbol in its sectionThe second issue is quite annoying, even if the fact that each symbol is in its own section can be useful to control every symbol's placement via the linker script, but I guess to fix this I need to custom linker file to redirect, say all constants "subsections" into ".rodata" section.

I haven't tried this, but it should work.

Utkarsh Gupta: GSoC Bi-Weekly Report - Week 1 and 2

15 June, 2019 - 07:04

Hello there.
The last two weeks have been adventurous. Here’s what happened.
My GSoC project is to package a software called Loomio. A little about Loomio:
Loomio is a decision-making software, designed to assist groups with the collaborative decision-making process.
It is a free software web-application, where users can initiate discussions and put up proposals.

Loomio is mostly written in Ruby, but also includes some CoffeeScript, Vue, JavaScript, with a little HTML, CSS.
The idea is to package all the dependencies of Loomio and get Loomio easily installable on the Debian machines.

The phase 1, that is, the first 4 weeks, were planned to package the Ruby and the Node dependencies. When I started off, I hit an obstacle. Little did we know about how to go about packaging complex applications like that.
I have been helping out in packages like gitlab, diaspora, et al. And towards the end of the last week, we learned that loomio needs to be done like diaspora.
First goes the loomio-installer, then would come the main package, loomio.

Now, the steps that are to be followed for loomio-installer are as follows:
» Get the app source.
» Install gem dependencies.
» Create database.
» Create tables/run migrations.
» Precomiple assets (scss -> css, et al).
» Configure nginx.
» Start service with systemd.
» In case of diaspora, JS front end is pulled via wrapper gems and in case of gitlab, it is pulled via npm/yarn.
» Loomio would be done with the same way we’re doing gitlab.

Thus, in the last two weeks, the following work has been done:
» Ruby gems’ test failures patched.
» 18 gems uploaded.
» Looked into loomio-installer’s setup.
» Basic scripts like nginx configuration, et al written.

My other activities in Debian last month:
» Updated and uploaded gitlab 11.10.4 to experimental (thanks to praveen).
» Uploaded gitaly, gitlab-workhorse.
» Sponsored a couple of packages (DM access).
» Learned Perl packaging and packaged 4 modules (thanks to gregoa and yadd).
» Learned basic Python packaging.
» Helping DC19 Bursary team (thanks to highvoltage).
» Helping DC19 Content team (thanks to terceiro).

Plans for the next 2 weeks:
» Get the app source via wget (script).
» Install gem and node dependencies via gem install and npm/yarn install (script).
» Create database for installer.
» Precomiple assets (scss -> css, et al).

I hope the next time I write a report, I’ll have no twists and adventures to share.

Until next time.
:wq for today.

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้