Planet Debian

Subscribe to Planet Debian feed
Planet Debian - http://planet.debian.org/
Updated: 2 hours 47 min ago

Dirk Eddelbuettel: RcppClassic 0.9.8

24 September, 2017 - 02:06

A bug-fix release RcppClassic 0.9.8 for the very recent 0.9.7 release which fixes a build issue on macOS introduced in 0.9.7. No other changes.

Courtesy of CRANberries, there are changes relative to the previous release.

Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Iain R. Learmonth: VM on bhyve not booting

23 September, 2017 - 16:45

Last night I installed updates on my FreeNAS box and rebooted it. As expected my network died, but then it never came back, which I hadn’t expected.

My FreeNAS box provides backup storage space, a local Debian mirror and a mirror of talks from recent conferences. It also runs a couple of virtual machines and one of these provides my local DNS resolver.

I hooked up the VNC console to the virtual machine and the problem looked to be that it was booting from the Debian installer CD. I removed the CD from the VM and rebooted, thinking that would be the end of it, but nope:

The EFI shell presented where GRUB should have been

I put the installer CD back and booted in “Rescue Mode”. For some reason, the bootloader installation wasn’t working, so I planned to reinstall it. The autopartition layout for Debian with EFI seems to use /dev/sda2 for the root partition. When you choose this it will see that you have an EFI partition and offer to mount it for you too.

When I went to install the bootloader, I saw another option that I didn’t know about: “Force GRUB installation in removable media path”. In the work I did on live-wrapper I had only ever dealt with this method of booting, I didn’t realise that there were other methods. The reasoning behind this option can be found in detail in Debian bug #746662. I also found mjg59’s blog post from 2011 useful in understanding this.

Suffice is to say that this fixed the booting issue for me in this case. I haven’t investigated this much futher so I can’t be certain of any reproducable steps to this problem, but I did also stumble across this forum post which essentially gives the manual steps that are taken by that Rescue Mode option in order to fix the problem. I think the only reason I hadn’t run into this before now is that the VMs hadn’t been rebooted since their installation.

Russell Coker: Converting Mbox to Maildir

23 September, 2017 - 10:52

MBox is the original and ancient format for storing mail on Unix systems, it consists of a single file per user under /var/spool/mail that has messages concatenated. Obviously performance is very poor when deleting messages from a large mail store as the entire file has to be rewritten. Maildir was invented for Qmail by Dan Bernstein and has a single message per file giving fast deletes among other performance benefits. An ongoing issue over the last 20 years has been converting Mbox systems to Maildir. The various ways of getting IMAP to work with Mbox only made this more complex.

The Dovecot Wiki has a good page about converting Mbox to Maildir [1]. If you want to keep the same message UIDs and the same path separation characters then it will be a complex task. But if you just want to copy a small number of Mbox accounts to an existing server then it’s a bit simpler.

Dovecot has a mb2md.pl script to convert folders [2].

cd /var/spool/mail
mkdir -p /mailstore/example.com
for U in * ; do
  ~/mb2md.pl -s $(pwd)/$U -d /mailstore/example.com/$U
done

To convert the inboxes shell code like the above is needed. If the users don’t have IMAP folders (EG they are just POP users or use local Unix MUAs) then that’s all you need to do.

cd /home
for DIR in */mail ; do
  U=$(echo $DIR| cut -f1 -d/)
  cd /home/$DIR
  for FOLDER in * ; do
    ~/mb2md.pl -s $(pwd)/$FOLDER -d /mailstore/example.com/$U/.$FOLDER
  done
  cp .subscriptions /mailstore/example.com/$U/ subscriptions
done

Some shell code like the above will convert the IMAP folders to Maildir format. The end result is that the users will have to download all the mail again as their MUA will think that every message had been deleted and replaced. But as all servers with significant amounts of mail or important mail were probably converted to Maildir a decade ago this shouldn’t be a problem.

Related posts:

  1. Why Cyrus Sucks I’m in the middle of migrating a mail server away...
  2. Moving a Mail Server Nowadays it seems that most serious mail servers (IE mail...
  3. Mail Server Training Today I ran a hands-on training session on configuring a...

Enrico Zini: Systemd on the command line

23 September, 2017 - 05:00

These are the notes of a training course on systemd I gave as part of my work with Truelite.

Exploring the state of a system
  • systemctl status [unitname] show status of a unit or of the whole system
  • systemctl list-units or just systemctl show a table with all units, their status and their description
  • systemctl list-sockets lists listening sockets managed by systemd and what they activate
  • systemctl list-timers lists timer-activated units, with information about when they last ran and when they will run again
  • systemctl is-active [pattern] checks if one or more units are in active state
  • systemctl is-enabled [pattern] checks if one or more units are enabled
  • systemctl is-failed [pattern] checks if one or more units are in failed state
  • systemctl list-dependencies [unitname] lists the dependencies of a unit, or a system-wide dependency tree
  • systemctl is-system-running check if the system is running correctly, or if some unit is in a failed state
  • systemd-cgtop like top but processes are aggregated by unit
  • systemd-analyze produces reports on boot time, per-unit boot time charts, dependency graphs, and more
Start and stop services

Similar to the System V service command, systemctl provides commands to start/stop/restart/reload units or services:

  • start: starts a unit if it is not already started
  • stop: stops a unit
  • restart: starts or restarts a unit
  • reload: tell a unit to reload its configuration (if it supports it)
  • try-restart: restarts a unit only if it is already active, otherwise do nothing, to prevent accidentally starting a service
  • reload-or-restart: tell a unit to reload its configuration if supported, otherwise restart it
  • try-reload-or-restart: tell a unit to reload its configuration if supported, otherwise restart it. If the unit is not already active, do nothing to prevent accidentally starting a service.
Changing global system state

systemctl has halt, poweroff, reboot, suspend, hybernate, and hybrid-sleep commands to tell systemd to reboot, power off, suspend and so on. kexec and switch-root also work.

The rescue and emergency commands switch the system to rescue and emergency mode (see man systemd.special. systemctl default switches to the default mode, which also happens when exiting the rescue or emergency shell.

Run services at boot

systemd does not implement runlevels, and services start at boot based on their dependencies.

To start a service at boot, you add to its .service file a WantedBy= dependency on a well-known .target unit.

At boot, systemd brings up the whole chain of dependency started from a default unit, and that will eventually activate also your service.

See systemctl get-default for what unit is currently the default in your system. You can change it via the systemd.unit= kernel command line, so you can configure multiple entries in the boot loader that boot the system running different services. For example systemd.unit=rescue.target for a rescue mode, systemd.unit=multi-user.target for a non-graphical mode, or add your own .target file to implement new system modes.

See systemctl list-units -t target --all for a list of all currently available targets in your system.

  • systemctl enable unitname enables the unit to start at boot, by creating symlinks to it in the .wants directory of the units listed in its WantedBy= configuration
  • systemctl disable unitname removes the symlinks created by enable
  • systemctl reenable unitname removes and readds the symlinks for when you changed WantedBy=

Notes: systemctl start activates a unit right now, but does not automatically enable it at boot systemctl enable enables a unit at boot, but does not automatically start it right now * a disabled unit can still be activated if another unit depends on it

To disable a unit so that it will never get started even if another unit depends on it, use systemctl mask unitname. Use systemctl unmask unitname to undo the masking.

Reloading / restarting systemd

systemctl daemon-reload tells systemd to reload its configuration.

systemctl daemon-reexec tells systemd to restart iself.

Iain R. Learmonth: It Died: An Update

22 September, 2017 - 14:15

Thanks for all the responses I’ve received about the death of my desktop PC. As I updated in my previous post, I find it unlikely that I will have to orphan any of my packages as I believe that I should be able to get a new workstation soon.

The responses I’ve had so far have been extremely uplifting for me. It’s very easy to feel that no one cares or appreciates your work when your hardware is dying and everything feels like it’s working against you.

I’ve already received two donations towards a new workstation. If you feel you can help then please contact me. I’m happy to accept donations by PayPal or you can contact me for BACS/SWIFT/IBAN information.

I’m currently looking at an HP Z240 Tower Workstation starting with 8GB RAM and then perhaps upgrading the RAM later. I’ll be transplanting my 3TB hybrid HDD into the new workstation as that cache is great for speeding up pbuilder builds. I’m hoping for this to work for me for the next 10 years, just as the Sun had been going for the last 10 years.

Somebody buy this guy a computer. But take the Sun case in exchange. That sucker's cool: It Died @iainlearmonth http://ow.ly/oLEI30fk0yN
-- @BrideOfLinux - 11:00 PM - 21 Sep 2017

For the right donation, I would be willing to consider shipping the rebooty Sun if you like cool looking paperweights (send me an email if you like). It’s pretty heavy though, just weighed it at 15kg. (:

You can send me a donation via PayPal here.

Clint Adams: PTT

22 September, 2017 - 05:32

“Hello,” said Adrian, but Adrian was lying.

“My name is Adrian,” said Adrian, but Adrian was lying.

“Today I took a pic of myself pulling a train,” announced Adrian.

Spaniard pulling a train

Posted on 2017-09-21 Tags: bgs

Enrico Zini: Systemd Truelite course

22 September, 2017 - 05:00

These are the notes of a training course on systemd I gave as part of my work with Truelite.

There is quite a lot of material, so I split them into a series of posts, running once a day for the next 9 days.

Units

Everything managed by systemd is called a unit (see man systemd.unit), and each unit is described by a configuration in ini-style format.

For example, this unit continuously plays an alarm sound when the system is in emergency or rescue mode:

[Unit]
Description=Beeps when in emergency or rescue mode
DefaultDependencies=false
StopWhenUnneeded=true

[Install]
WantedBy=emergency.target rescue.target

[Service]
Type=simple
ExecStart=/bin/sh -ec 'while true; do /usr/bin/aplay -q /tmp/beep.wav; sleep 2; done'

Units can be described by configuration files, which have different extensions based on what kind of thing they describe:

  • .service: daemons
  • .socket: communication sockets
  • .device: hardware devices
  • .mount: mount points
  • .automount: automounting
  • .swap: swap files or partitions
  • .target: only dependencies, like Debian metapackages
  • .path: inotify monitoring of paths
  • .timer: cron-like activation
  • .slice: group processes for common resource management
  • .scope: group processes for common resource management

System unit files can be installed in:

  • /lib/systemd/system/: for units provided by packaged software
  • /run/systemd/system/: runtime-generated units
  • /etc/systemd/system/: for units provided by systemd administrators

Unit files in /etc/ override unit files in /lib/. Note that while Debian uses /lib/, other distributions may use /usr/lib/ instead.

If there is a directory with the same name as the unit file plus a .d suffix, any file *.conf it contains is parsed after the unit, and can be used to add or override configuration options.

For example:

  • /lib/systemd/system/beep.service.d/foo.conf can be used to tweak the contents of /lib/systemd/system/beep.service, so it is possible for a package to distribute a tweak to the configuration of another package.
  • /etc/systemd/system/beep.service.d/foo.conf can be used to tweak the contents of /lib/systemd/system/beep.service, so it is possible a system administrator to extend a packaged unit without needing to replace it entirely.

Similarly, a unitname.wants/ or unitname.requires/ directory can be used to extend Wants= and Requires= dependencies on other units, by placing symlinks to other units in them.

See also:

Iain R. Learmonth: It Died

21 September, 2017 - 16:10

On Sunday, in my weekly report on my free software activities, I wrote about how sustainable my current level of activites are. I had identified the risk that the computer that I use for almost all of my free software work was slowly dying. Last night it entered an endless reboot loop and subsequent efforts to save it have failed.

I cannot afford to replace this machine and my next best machine has half the cores, half the RAM and less than half of the screen real estate. As this is going to be a serious hit to my productivity, I need to seriously consider if I am able to continue to maintain the number of packages I currently do in Debian.

The Sun Ultra 24

Steve Kemp: Retiring the Debian-Administration.org site

21 September, 2017 - 04:00

So previously I've documented the setup of the Debian-Administration website, and now I'm going to retire it I'm planning how that will work.

There are currently 12 servers powering the site:

  • web1
  • web2
  • web3
  • web4
    • These perform the obvious role, serving content over HTTPS.
  • public
    • This is a HAProxy host which routes traffic to one of the four back-ends.
  • database
    • This stores the site-content.
  • events
    • There was a simple UDP-based protocol which sent notices here, from various parts of the code.
    • e.g. "Failed login for bob from 1.2.3.4".
  • mailer
    • Sends out emails. ("You have a new reply", "You forgot your password..", etc)
  • redis
    • This stored session-data, and short-term cached content.
  • backup
    • This contains backups of each host, via Obnam.
  • beta
    • A test-install of the codebase
  • planet
    • The blog-aggregation site

I've made a bunch of commits recently to drop the event-sending, since no more dynamic actions will be possible. So events can be retired immediately. redis will go when I turn off logins, as there will be no need for sessions/cookies. beta is only used for development, so I'll kill that too. Once logins are gone, and anonymous content is disabled there will be no need to send out emails, so mailer can be shutdown.

That leaves a bunch of hosts left:

  • database
    • I'll export the database and kill this host.
    • I will install mariadb on each web-node, and each host will be configured to talk to localhost only
    • I don't need to worry about four database receiving diverging content as updates will be disabled.
  • backup
  • planet
    • This will become orphaned, so I think I'll just move the content to the web-nodes.

All in all I think we'll just have five hosts left:

  • public to do the routing
  • web1-web4 to do the serving.

I think that's sane for the moment. I'm still pondering whether to export the code to static HTML, there's a lot of appeal as the load would drop a log, but equally I have a hell of a lot of mod_rewrite redirections in place, and reworking all of them would be a pain. Suspect this is something that will be done in the future, maybe next year.

Dirk Eddelbuettel: pinp 0.0.2: Onwards

20 September, 2017 - 20:00

A first update 0.0.2 of the pinp package arrived on CRAN just a few days after the initial release.

We added a new vignette for the package (see below), extended a few nice features, and smoothed a few corners.

The NEWS entry for this release follows.

Changes in tint version 0.0.2 (2017-09-20)
  • The YAML segment can be used to select font size, one-or-two column mode, one-or-two side mode, linenumbering and watermarks (#21 and #26 addressing #25)

  • If pinp.cls or jss.bst are not present, they are copied in ((#27 addressing #23)

  • Output is now in shaded framed boxen too (#29 addressing #28)

  • Endmatter material is placed in template.tex (#31 addressing #30)

  • Expanded documentation of YAML options in skeleton.Rmd and clarified available one-column option (#32).

  • Section numbering can now be turned on and off (#34)

  • The default bibliography style was changed to jss.bst.

  • A short explanatory vignette was added.

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the tint page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Iain R. Learmonth: Easy APT Repository

20 September, 2017 - 14:30

The PATHspider software I maintain as part of my work depends on some features in cURL and in PycURL that have only just been mereged or are still awaiting merge. I need to build a docker container that includes these as Debian packages, so I need to quickly build an APT repository.

A Debian repository can essentially be seen as a static website and the contents are GPG signed so it doesn’t necessarily need to be hosted somewhere trusted (unless availability is critical for your application). I host my blog with Netlify, a static website host, and I figured they would be perfect for this use case. They also support open source projects.

There is a CLI tool for netlify which you can install with:

sudo apt install npm
sudo npm install -g netlify-cli

The basic steps for setting up a repository are:

mkdir repository
cp /path/to/*.deb repository/
cd repository
apt-ftparchive packages . > Packages
apt-ftparchive release . > Release
gpg --clearsign -o InRelease Release
netlify deploy

Once you’ve followed these steps, and created a new site on Netlify, you’ll be able to manage this site also through the web interface. A few things you might want to do are set up a custom domain name for your repository, or enable HTTPS with Let’s Encrypt. (Make sure you have apt-transport-https if you’re going to enable HTTPS though.)

To add this repository to your apt sources:

gpg --export -a YOURKEYID | sudo apt-key add -
echo "deb https://SUBDOMAIN.netlify.com/ /" | sudo tee -a /etc/apt/sources.list
sudo apt update

You’ll now find that those packages are installable. Beware of APT pinning as you may find that the newer versions on your repository are not actually the preferred versions according to your policy.

Gunnar Wolf: Call to Mexicans: Open up your wifi #sismo

20 September, 2017 - 04:52

Hi friends,

~3hr ago, we just had a big earthquake, quite close to Mexico City. Fortunately, we are fine, as are (at least) most of our friends and family. Hopefully, all of them. But there are many (as in, tens) damaged or destroyed buildings; there have been over 50 deceased people, and numbers will surely rise until a good understanding of the event's strength are evaluated.

Mainly in these early hours after the quake, many people need to get in touch with their families and friends. There is a little help we can all provide: Provide communication.

Open up your wireless network. Set it up unencrypted, for anybody to use.

Refrain from over-sharing graphical content — Your social network groups don't need to see every video and every photo of the shaking moments and of broken buildings. Download of all those images takes up valuable time-space for the saturated cellular networks.

This advice might be slow to flow... The important moment to act is two or three hours ago, even now... But we are likely to have replicas; we are likely to have panic moments again. Do a little bit to help others in need!

Sylvain Beucler: dot-zed extractor

20 September, 2017 - 02:29

Following last week's .zed format reverse-engineered specification, Loïc Dachary contributed a POC extractor!
It's available at http://www.dachary.org/loic/zed/, it can list non-encrypted metadata without password, and extract files with password (or .pem file).
Leveraging on python-olefile and pycrypto, only 500 lines of code (test cases excluded) are enough to implement it

Sylvain Beucler: dot-zed extractor

20 September, 2017 - 02:29

Following last week's .zed format reverse-engineered specification, Loïc Dachary contributed a POC extractor!
It's available at http://www.dachary.org/loic/zed/, it can list non-encrypted metadata without password, and extract files with password (or .pem file).
Leveraging on python-olefile and pycrypto, only 500 lines of code (test cases excluded) are enough to implement it

Reproducible builds folks: Reproducible Builds: Weekly report #125

20 September, 2017 - 00:45

Here's what happened in the Reproducible Builds effort between Sunday September 10 and Saturday September 16 2017:

Upcoming events Reproduciblity work in Debian

devscripts/2.17.10 was uploaded to unstable, fixing #872514. This adds a script to report on reproducibility status of installed packages written by Chris Lamb.

#876055 was opened against Debian Policy to decide the precise requirements we should have on a build's environment variables.

Bugs filed:

Non-maintainer uploads:

  • Holger Levsen:
Reproduciblity work in other projects

Patches sent upstream:

  • Bernhard M. Wiedemann:
Reviews of unreproducible packages

16 package reviews have been added, 99 have been updated and 92 have been removed in this week, adding to our knowledge about identified issues.

1 issue type has been updated:

diffoscope development
  • Juliana Oliveira Rodrigues:
    • Fix comparisons between different container types not comparing inside files. It was caused by falling back to binary comparison for different file types even for unextracted containers.
    • Add many tests for the fixed behaviour.
    • Other code quality improvements.
  • Chris Lamb:
    • Various code quality and style improvements, some of it using Flake8.
  • Mattia Rizzolo:
    • Add a check to prevent installation with python < 3.4
reprotest development
  • Ximin Luo:
    • Split up the very large __init__.py and remove obsolete earlier code.
    • Extend the syntax for the --variations flag to support parameters to certain variations like user_group, and document examples in README.
    • Add a --vary flag for the new syntax and deprecate --dont-vary.
    • Heavily refactor internals to support > 2 builds.
    • Support >2 builds using a new --extra-build flag.
    • Properly sanitize artifact_pattern to avoid arbitrary shell execution.
trydiffoscope development

Version 65 was uploaded to unstable by Chris Lamb including these contributions:

  • Chris Lamb:
    • Packaging maintenance updates.
    • Developer documentation updates.
Reproducible websites development tests.reproducible-builds.org
  • Vagrant Cascadian and Holger Levsen:
    • Added two armhf boards to the build farm. #874682
  • Holger also:
    • use timeout to limit the diffing of the two build logs to 30min, which greatly reduced jenkins load again.
Misc.

This week's edition was written by Ximin Luo, Bernhard M. Wiedemann, Chris Lamb, Holger Levsen and Daniel Shahaf & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Carl Chenet: The Github threat

19 September, 2017 - 05:00

Many voices arise now and then against risks linked to the Github use by Free Software projects. Yet the infatuation for the collaborative forge of the Octocat Californian start-ups doesn’t seem to fad away.

These recent years, Github and its services take an important role in software engineering as they are seen as easy to use, efficient for a daily workload with interesting functions in enterprise collaborative workflow or amid a Free Software project. What are the arguments against using its services and are they valid? We will list them first, then we’ll examine their validity.

1. Critical points 1.1 Centralization

The Github application belongs to a single entity, Github Inc, a US company which manage it alone. So, a unique company under US legislation manages the access to most of Free Software application code sources, which may be a problem with groups using it when a code source is no longer available, for political or technical reason.

The Octocat, the Github mascot

 

This centralization leads to another trouble: as it obtained critical mass, it becomes more and more difficult not having a Github account. People who don’t use Github, by choice or not, are becoming a silent minority. It is now fashionable to use Github, and not doing so is seen as “out of date”. The same phenomenon is a classic, and even the norm, for proprietary social networks (Facebook, Twitter, Instagram).

1.2 A Proprietary Software

When you interact with Github, you are using a proprietary software, with no access to its source code and which may not work the way you think it is. It is a problem at different levels. First, ideologically, but foremost in practice. In the Github case, we send them code we can control outside of their interface. We also send them personal information (profile, Github interactions). And mostly, Github forces any project which goes through the US platform to use a crucial proprietary tools: its bug tracking system.

Windows, the epitome of proprietary software, even if others took the same path

 

1.3 The Uniformization

Working with Github interface seems easy and intuitive to most. Lots of companies now use it as a source repository, and many developers leaving a company find the same Github working environment in the next one. This pervasive presence of Github in free software development environment is a part of the uniformization of said developers’ working space.

Uniforms always bring Army in my mind, here the Clone army

2 – Critical points cross-examination 2.1 Regarding the centralization 2.1.1 Service availability rate

As said above, nowadays, Github is the main repository of Free Software source code. As such it is a favorite target for cyberattacks. DDOS hit it in March and August 2015. On December 15, 2015, an outage led to the inaccessibility of 5% of the repositories. The same occurred on November 15. And these are only the incident reported by Github itself. One can imagine that the mean outage rate of the platform is underestimated.

2.1.2 Chain reaction could block Free Software development

Today many dependency maintenance tools, as npm for javascript, Bundler for Ruby or even pip for Python can access an application source code directly from Github. Free Software projects getting more and more linked and codependents, if one component is down, all the developing process stop.

One of the best examples is the npmgate. Any company could legally demand that Github take down some source code from its repository, which could create a chain reaction and blocking the development of many Free Software projects, as suffered the Node.js community from the decisions of Npm, Inc, the company managing npm.

2.2 A historical precedent: SourceForge

Github didn’t appear out of the blue. In his time, its predecessor, SourceForge, was also extremely popular.

Heavily centralized, based on strong interaction with the community, SourceForge is now seen as an aging SAAS (Software As A Service) and sees most of its customers fleeing to Github. Which creates lots of hurdles for those who stayed. The Gimp project suffered from spams and terrible advertising, which led to the departure of the VLC project, then from installers corrupted with adwares instead of the official Gimp installer for Windows. And finally, the Project Gimp’s SourceForge account was hacked by… SourceForge team itself!

These are very recent examples of what can do a commercial entity when it is under its stakeholders’ pressure. It is vital to really understand what it means to trust them with data and exchange centralization, where it could have tremendous repercussion on the day-to-day life and the habits of the Free Software and open source community.

2.3. Regarding proprietary software 2.3.1 One community, several opinions on proprietary software

Mostly based on ideology, this point deals with the definition every member of the community gives to Free Software and open source. Mostly about one thing: is it viral or not? Or GPL vs MIT/BSD.

Those on the side of the viral Free Software will have trouble to use a proprietary software as this last one shouldn’t even exist. It must be assimilated, to quote Star Trek, as it is a connected black box, endangering privacy, corrupting for profit our uses and restrain our freedom to use as we’re pleased what we own, etc.

Those on the side of complete freedom have no qualms using proprietary software as their very existence is a consequence of freedom without restriction. They even agree that code they developed may be a part of proprietary software, which is quite a common occurrence. This part of the Free Software community has no qualm using Github, which is well within their ideology parameters. Just take a look at the Janson amphitheater during Fosdem and check how many Apple laptops running on macOS are around.

FreeBSD, the main BSD project under the BSD license

2.3.2 Data loss and data restrictions linked to proprietary software use

Even without ideological consideration, and just focusing on Github infrastructure, the bug tracking system is a major problem by itself.

Bug report builds the memory of Free Software projects. It is the entrance point for new contributors, the place to find bug reporting, requests for new functions, etc. The project history can’t be limited only to the code. It’s very common to find bug reports when you copy and paste an error message in a search engine. Not their historical importance is precious for the project itself, but also for its present and future users.

Github gives the ability to extract bug reports through its API. What would happen if Github is down or if the platform doesn’t support this feature anymore? In my opinion, not that many projects ever thought of this outcome. How could they move all the data generated by Github into a new bug tracking system?

One old example now is Astrid, a TODO list bought by Yahoo a few years ago. Very popular, it grew fast until it was closed overnight, with only a few weeks for its users to extract their data. It was only a to-do list. The same situation with Github would be tremendously difficult to manage for several projects if they even have the ability to deal with it. Code would still be available and could still live somewhere else, but the project memory would be lost. A project like Debian has today more than 800,000 bug reports, which are a data treasure trove about problems solved, function requests and where the development stand on each. The developers of the Cpython project have anticipated the problem and decided not to use Github bug tracking systems.

Issues, the Github proprietary bug tracking system

Another thing we could lose if Github suddenly disappear: all the work currently done regarding the push requests (aka PRs). This Github function gives the ability to clone one project’s Github repository, to modify it to fit your needs, then to offer your own modification to the original repository. The original repository’s owner will then review said modification, and if he or she agrees with them will fuse them into the original repository. As such, it’s one of the main advantages of Github, since it can be done easily through its graphic interface.

However reviewing all the PRs may be quite long, and most of the successful projects have several ongoing PRs. And this PRs and/or the proprietary bug tracking system are commonly used as a platform for comment and discussion between developers.

Code itself is not lost if Github is down (except one specific situation as seen below), but the peer review works materialized in the PRs and the bug tracking system is lost. Let’s remember than the PR mechanism let you clone and modify projects and then generate PRs directly from its proprietary web interface without downloading a single code line on your computer. In this particular case, if Github is down, all the code and the work in progress is lost.

Some also use Github as a bookmark place. They follow their favorite projects’ activity through the Watch function. This technological watch style of data collection would also be lost if Github is down.

Debian, one of the main Free Software projects with at least a thousand official contributors

2.4 Uniformization

The Free Software community is walking a thigh rope between normalization needed for an easier interoperability between its products and an attraction for novelty led by a strong need for differentiation from what is already there.

Github popularized the use of Git, a great tool now used through various sectors far away from its original programming field. Step by step, Git is now so prominent it’s almost impossible to even think to another source control manager, even if awesome alternate solutions, unfortunately not as popular, exist as Mercurial.

A new Free Software project is now a Git repository on Github with README.md added as a quick description. All the other solutions are ostracized? How? None or very few potential contributors would notice said projects. It seems very difficult now to encourage potential contributors into learning a new source control manager AND a new forge for every project they want to contribute. Which was a basic requirement a few years ago.

It’s quite sad because Github, offering an original experience to its users, cut them out of a whole possibility realm. Maybe Github is one of the best web versioning control systems. But being the main one doesn’t let room for a new competitor to grow. And it let Github initiate development newcomers into a narrow function set, totally unrelated to the strength of the Git tool itself.

3. Centralization, uniformization, proprietary software… What’s next? Laziness?

Fight against centralization is a main part of the Free Software ideology as centralization strengthens the power of those who manage it and who through it control those who are managed by it. Uniformization allergies born against main software companies and their wishes to impose a closed commercial software world was for a long time the main fuel for innovation thirst and intelligent alternative development. As we said above, part of the Free Software community was built as a reaction to proprietary software and their threat. The other part, without hoping for their disappearance, still chose a development model opposite to proprietary software, at least in the beginning, as now there’s more and more bridges between the two.

The Github effect is a morbid one because of its consequences: at least centralization, uniformization, proprietary software usage as their bug tracking system. But some years ago the Dear Github buzz showed one more side effect, one I’ve never thought about: laziness. For those who don’t know what it is about, this letter is a complaint from several spokespersons from several Free Software projects which demand to Github team to finally implement, after years of polite asking, new functions.

Since when Free Software project facing a roadblock request for clemency and don’t build themselves the path they need? When Torvalds was involved in the Bitkeeper problem and the Linux kernel development team couldn’t use anymore their revision control software, he developed Git. The mere fact of not being able to use one tool or functions lacking is the main motivation to seek alternative solutions and, as such, of the Free Software movement. Every Free Software community member able to code should have this reflex. You don’t like what Github offers? Switch to Gitlab. You don’t like it Gitlab? Improve it or make your own solution.

The Gitlab logo

Let’s be crystal clear. I’ve never said that every Free Software developers blocked should code his or her own alternative. We all have our own priorities, and some of us even like their beauty sleep, including me. But, to see that this open letter to Github has 1340 names attached to it, among them some spokespersons for major Free Software project showed me that need, willpower and strength to code a replacement are here. Maybe said replacement will be born from this letter, it would be the best outcome of this buzz.

In the end, Github usage is just another example of Internet usage massification. As Internet users are bound to go to massively centralized social network as Facebook or Twitter, developers are following the same path with Github. Even if a large fraction of developers realize the threat linked this centralized and proprietary organization, the whole community is following this centralization and uniformization trend. Github service is useful, free or with a reasonable price (depending on the functions you need) easy to use and up most of the time. Why would we try something else? Maybe because others are using us while we are savoring the convenience? The Free Software community seems to be quite sleepy to me.

The lion enjoying the hearth warm

About Me

Carl Chenet, Free Software Indie Hacker, founder of the French-speaking Hacker News-like Journal du hacker.

Follow me on social networks

Translated from French by Stéphanie Chaptal. Original article written in 2015.

Russ Allbery: Consolidation haul

18 September, 2017 - 07:34

My parents are less fond than I am of filling every available wall in their house with bookshelves and did a pruning of their books. A lot of them duplicated other things that I had, or didn't sound interesting, but I still ended up with two boxes of books (and now have to decide which of my books to prune, since I'm out of shelf space).

Also included is the regular accumulation of new ebook purchases.

Mitch Albom — Tuesdays with Morrie (nonfiction)
Ilona Andrews — Clean Sweep (sff)
Catherine Asaro — Charmed Sphere (sff)
Isaac Asimov — The Caves of Steel (sff)
Isaac Asimov — The Naked Sun (sff)
Marie Brennan — Dice Tales (nonfiction)
Captain Eric "Winkle" Brown — Wings on My Sleeve (nonfiction)
Brian Christian & Tom Griffiths — Algorithms to Live By (nonfiction)
Tom Clancy — The Cardinal of the Kremlin (thriller)
Tom Clancy — The Hunt for the Red October (thriller)
Tom Clancy — Red Storm Rising (thriller)
April Daniels — Sovereign (sff)
Tom Flynn — Galactic Rapture (sff)
Neil Gaiman — American Gods (sff)
Gary J. Hudson — They Had to Go Out (nonfiction)
Catherine Ryan Hyde — Pay It Forward (mainstream)
John Irving — A Prayer for Owen Meany (mainstream)
John Irving — The Cider House Rules (mainstream)
John Irving — The Hotel New Hampshire (mainstream)
Lawrence M. Krauss — Beyond Star Trek (nonfiction)
Lawrence M. Krauss — The Physics of Star Trek (nonfiction)
Ursula K. Le Guin — Four Ways to Forgiveness (sff collection)
Ursula K. Le Guin — Words Are My Matter (nonfiction)
Richard Matheson — Somewhere in Time (sff)
Larry Niven — Limits (sff collection)
Larry Niven — The Long ARM of Gil Hamilton (sff collection)
Larry Niven — The Magic Goes Away (sff)
Larry Niven — Protector (sff)
Larry Niven — World of Ptavvs (sff)
Larry Niven & Jerry Pournelle — The Gripping Hand (sff)
Larry Niven & Jerry Pournelle — Inferno (sff)
Larry Niven & Jerry Pournelle — The Mote in God's Eye (sff)
Flann O'Brien — The Best of Myles (nonfiction)
Jerry Pournelle — Exiles to Glory (sff)
Jerry Pournelle — The Mercenary (sff)
Jerry Pournelle — Prince of Mercenaries (sff)
Jerry Pournelle — West of Honor (sff)
Jerry Pournelle (ed.) — Codominium: Revolt on War World (sff anthology)
Jerry Pournelle & S.M. Stirling — Go Tell the Spartans (sff)
J.D. Salinger — The Catcher in the Rye (mainstream)
Jessica Amanda Salmonson — The Swordswoman (sff)
Stanley Schmidt — Aliens and Alien Societies (nonfiction)
Cecilia Tan (ed.) — Sextopia (sff anthology)
Lavie Tidhar — Central Station (sff)
Catherynne Valente — Chicks Dig Gaming (nonfiction)
J.E. Zimmerman — Dictionary of Classical Mythology (nonfiction)

This is an interesting tour of a lot of stuff I read as a teenager (Asimov, Niven, Clancy, and Pournelle, mostly in combination with Niven but sometimes his solo work).

I suspect I will no longer consider many of these books to be very good, and some of them will probably go back into used bookstores after I've re-read them for memory's sake, or when I run low on space again. But all those mass market SF novels were a big part of my teenage years, and a few (like Mote In God's Eye) I definitely want to read again.

Also included is a random collection of stuff my parents picked up over the years. I don't know what to expect from a lot of it, which makes it fun to anticipate. Fall vacation is coming up, and with it a large amount of uninterrupted reading time.

Sean Whitton: Debian Policy call for participation -- September 2017

18 September, 2017 - 06:04

Here’s a summary of the bugs against the Debian Policy Manual. Please consider getting involved, whether or not you’re an existing contributor.

Consensus has been reached and help is needed to write a patch

#172436 BROWSER and sensible-browser standardization

#273093 document interactions of multiple clashing package diversions

#299007 Transitioning perms of /usr/local

#314808 Web applications should use /usr/share/package, not /usr/share/doc/…

#425523 Describe error unwind when unpacking a package fails

#452393 Clarify difference between required and important priorities

#476810 Please clarify 12.5, “Copyright information”

#484673 file permissions for files potentially including credential informa…

#491318 init scripts “should” support start/stop/restart/force-reload - why…

#556015 Clarify requirements for linked doc directories

#568313 Suggestion: forbid the use of dpkg-statoverride when uid and gid ar…

#578597 Recommend usage of dpkg-buildflags to initialize CFLAGS and al.

#582109 document triggers where appropriate

#587991 perl-policy: /etc/perl missing from Module Path

#592610 Clarify when Conflicts + Replaces et al are appropriate

#613046 please update example in 4.9.1 (debian/rules and DEB_BUILD_OPTIONS)

#614807 Please document autobuilder-imposed build-dependency alternative re…

#628515 recommending verbose build logs

#664257 document Architecture name definitions

#682347 mark ‘editor’ virtual package name as obsolete

#685506 copyright-format: new Files-Excluded field

#685746 debian-policy Consider clarifying the use of recommends

#688251 Built-Using description too aggressive

#749826 [multiarch] please document the use of Multi-Arch field in debian/c…

#757760 please document build profiles

#759316 Document the use of /etc/default for cron jobs

#761219 document versioned Provides

#767839 Linking documentation of arch:any package to arch:all

#770440 policy should mention systemd timers

#773557 Avoid unsafe RPATH/RUNPATH

#780725 PATH used for building is not specified

#793499 The Installed-Size algorithm is out-of-date

#810381 Update wording of 5.6.26 VCS-* fields to recommend encryption

#823256 Update maintscript arguments with dpkg >= 1.18.5

#833401 virtual packages: dbus-session-bus, dbus-default-session-bus

#835451 Building as root should be discouraged

#838777 Policy 11.8.4 for x-window-manager needs update for freedesktop menus

#845715 Please document that packages are not allowed to write outside thei…

#853779 Clarify requirements about update-rc.d and invoke-rc.d usage in mai…

#874019 Note that the ’-e’ argument to x-terminal-emulator works like ’–’

#874206 allow a trailing comma in package relationship fields

Wording proposed, awaiting review from anyone and/or seconds by DDs

#515856 remove get-orig-source

#542288 Versions for native packages, NMU’s, and binary only uploads

#582109 document triggers where appropriate

#610083 Remove requirement to document upstream source location in debian/c…

#645696 [copyright-format] clearer definitions and more consistent License:…

#649530 [copyright-format] clearer definitions and more consistent License:…

#662998 stripping static libraries

#682347 mark ‘editor’ virtual package name as obsolete

#683222 say explicitly that debian/changelog is required in source packages

#688251 Built-Using description too aggressive

#737796 copyright-format: support Files: paragraph with both abbreviated na…

#756835 Extension of the syntax of the Packages-List field.

#786470 [copyright-format] Add an optional “License-Grant” field

#810381 Update wording of 5.6.26 VCS-* fields to recommend encryption

#835451 Building as root should be discouraged

#845255 Include best practices for packaging database applications

#850729 Documenting special version number suffixes

#874090 Clarify wording of some passages

#874095 copyright-format: Use the “synopsis” term established in the de…

Merged for the next release

#661928 recipe for determining shlib package name

#679751 please clarify package account and home directory location in policy

#683222 say explicitly that debian/changelog is required in source packages

#870915 [5.6.30] Testsuite: There are much more defined values

#872893 Chapters, sections, appendices and numbering

#872895 Include multi-page HTML in package

#872896 An html.tar.gz has leaked into the .deb?

#872900 Very generic info file name

#872950 Too much indirection in info file menus

#873819 upgrading-checklist.txt: typo pgpsignurlmangle in section 4.11 of V…

#874411 missing line breaks in summary of ways maintainers scripts are call…

Russ Allbery: Free software log (July and August 2017)

18 September, 2017 - 03:08

I've wanted to start making one of these posts for a few months but have struggled to find the time. But it seems like a good idea, particularly since I get more done when I write down what I do, so you all get a rather belated one. This covers July and August; hopefully the September one will come closer to the end of September.

Debian

August was DebConf, which included a ton of Policy work thanks to Sean Whitton's energy and encouragement. During DebConf, we incorporated work from Hideki Yamane to convert Policy to reStructuredText, which has already made it far easier to maintain. (Thanks also to David Bremner for a lot of proofreading of the result.) We also did a massive bug triage and closed a ton of older bugs on which there had been no forward progress for many years.

After DebConf, as expected, we flushed out various bugs in the reStructuredText conversion and build infrastructure. I fixed a variety of build and packaging issues and started doing some more formatting cleanup, including moving some footnotes to make the resulting document more readable.

During July and August, partly at DebConf and partly not, I also merged wording fixes for seven bugs and proposed wording (not yet finished) for three more, as well as participated in various Policy discussions.

Policy was nearly all of my Debian work over these two months, but I did upload a new version of the webauth package to build with OpenSSL 1.1 and drop transitional packages.

Kerberos

I still haven't decided my long-term strategy with the Kerberos packages I maintain. My personal use of Kerberos is now fairly marginal, but I still care a lot about the software and can't convince myself to give it up.

This month, I started dusting off pam-krb5 in preparation for a new release. There's been an open issue for a while around defer_pwchange support in Heimdal, and I spent some time on that and tracked it down to an upstream bug in Heimdal as well as a few issues in pam-krb5. The pam-krb5 issues are now fixed in Git, but I haven't gotten any response upstream from the Heimdal bug report. I also dusted off three old Heimdal patches and submitted them as upstream merge requests and reported some more deficiencies I found in FAST support. On the pam-krb5 front, I updated the test suite for the current version of Heimdal (which changed some of the prompting) and updated the portability support code, but haven't yet pulled the trigger on a new release.

Other Software

I merged a couple of pull requests in podlators, one to fix various typos (thanks, Jakub Wilk) and one to change the formatting of man page references and function names to match the current Linux manual page standard (thanks, Guillem Jover). I also documented a bad interaction with line-buffered output in the Term::ANSIColor man page. Neither of these have seen a new release yet.

Dirk Eddelbuettel: RcppClassic 0.9.7

18 September, 2017 - 02:28

A rather boring and otherwise uneventful release 0.9.7 of RcppClassic is now at CRAN. This package provides a maintained version of the otherwise deprecated first Rcpp API; no new projects should use it.

Once again no changes in user-facing code. But this makes it the first package to use the very new and shiny pinp package as the backend for its vignette, now converted to Markdown---see here for this new version. We also updated three sources files for tabs versus spaces as the current g++ version complained (correctly !!) about misleading indents. Otherwise a file src/init.c was added for dynamic registration, the Travis CI runner script was updated to using run.sh from our r-travis fork, and we now strip the library after they have been built. Again, no user code changes.

And no iterate: nobody should use this package. Rcpp is so much better in so many ways---this one is simply available as we (quite strongly) believe that APIs are contracts, and as such we hold up our end of the deal.

Courtesy of CRANberries, there are changes relative to the previous release.

Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้