Planet Debian

Subscribe to Planet Debian feed
Planet Debian -
Updated: 2 hours 1 min ago

Russ Allbery: Review: Enchanters' End Game

15 January, 2017 - 03:18

Review: Enchanters' End Game, by David Eddings

Series: The Belgariad #5 Publisher: Del Rey Copyright: December 1984 Printing: February 1990 ISBN: 0-345-33871-5 Format: Mass market Pages: 372

And, finally, the conclusion towards which everything has been heading, and the events for which Castle of Wizardry was the preparation. (This is therefore obviously not the place to start with this series.) Does it live up to all the foreshadowing and provide a satisfactory conclusion? I'd say mostly. The theology is a bit thin, but Eddings does a solid job of bringing all the plot threads together and giving each of the large cast a moment to shine.

Enchanters' End Game (I have always been weirdly annoyed by that clunky apostrophe) starts with more of Garion and Belgarath, and, similar to the end of Castle of Wizardry, this feels like them rolling on the random encounter table. There is a fairly important bit with Nadraks at the start, but the remaining detour to the north is a mostly unrelated bit of world-building. Before this re-read, I didn't remember how extensive the Nadrak parts of this story were; in retrospect, I realize a lot of what I was remembering is in the Mallorean instead. I'll therefore save my commentary on Nadrak gender roles for an eventual Mallorean re-read, since there's quite a lot to dig through and much of it is based on information not available here.

After this section, though, the story leaves Garion, Belgarath, and Silk for nearly the entire book, returning to them only for the climax. Most of this book is about Ce'Nedra, the queens and kings of the west, and what they're doing while Garion and his small party are carrying the Ring into Mordor— er, you know what I mean.

And this long section is surprisingly good. We first get to see the various queens of the west doing extremely well managing the kingdoms while the kings are away (see my previous note about how Eddings does examine his stereotypes), albeit partly by mercilessly exploiting the sexism of their societies. The story then picks up with Ce'Nedra and company, including all of the rest of Garion's band, being their snarky and varied selves. There are some fairly satisfying set pieces, some battle tactics, some magical tactics, and a good bit of snarking and interplay between characters who feel like old friends by this point (mostly because of Eddings's simple, broad-strokes characterization).

And Ce'Nedra is surprisingly good here. I would say that she's grown up after the events of the last book, but sadly she reverts to being awful in the aftermath. But for the main section of the book, partly because she's busy with other things, she's a reasonable character who experiences some actual consequences and some real remorse from one bad decision she makes. She's even admirable in how she handles events leading up to the climax of the book.

Eddings does a good job showing every character in their best light, putting quite a lot of suspense (and some dramatic rescues) into this final volume, and providing a final battle that's moderately interesting. I'm not sure I entirely bought the theological ramifications of the conclusion (the bits with Polgara do not support thinking about too deeply), but the voice in Garion's head continues to be one of the better characters of the series. And Errand is a delight.

After the climax, the aftermath sadly returns to Eddings's weird war between the sexes presentation of all gender relationships in this series, and it left me with a bit of a bad taste in my mouth. (There is absolutely no way that some of these relationships would survive in reality.) Eddings portrays nearly every woman as a manipulative schemer, sometimes for good and sometimes for evil, and there is just so much gender stereotyping throughout this book for both women and men. You can tell he's trying with the queens, but women are still only allowed to be successful at politics and war within a very specific frame. Even Polgara gets a bit of the gender stereotyping, although she remains mostly an exception (and one aspect of the ending is much better than it could have been).

Ah well. One does not (or at least probably should not) read this series without being aware that it has some flaws. But it has a strange charm as well, mostly from its irreverence. The dry wise-cracking of these characters rings more true to me than the epic seriousness of a lot of fantasy. This is how people behave under stress, and this is how quirky people who know each other extremely well interact. It also keeps one turning the pages quite effectively. I stayed up for several late nights finishing it, and was never tempted to put it down and stop reading.

This is not great literature, but it's still fun. It wouldn't sustain regular re-reading for me, but a re-read after twenty years or so was pretty much exactly the experience I was hoping for: an unchallenging, optimistic story with amusing characters and a guaranteed happy ending. There's a place for that.

Followed, in a series sense, by the Mallorean, the first book of which is The Guardians of the West. But this is a strictly optional continuation; the Belgariad comes to a definite end here.

Rating: 7 out of 10

Sven Hoexter: moto g falcon reactivation and exodus mod

14 January, 2017 - 20:43

I started to reactivate my old moto g falcon during the last days of CyanogenMod in December of 2016. First step was a recovery update to TWRP 3.0.2-2 so I was able to flash CM13/14 builds. While CM14 nightly builds did not boot at all the CM13 builds did, but up to the last build wifi connections to the internet did not work. I could actually register with my wifi (Archer C7 running OpenWRT) but all apps claim the internet connection check failed and I'm offline. So bummer, without wifi a smartphone is not much fun.

I was pretty sure that wifi worked when I last used that phone about 1.5 years ago with CM11/12, so I started to dive into the forums of xda-developers to look for alternatives. Here I found out about Exodus. I've a bit of trouble trusting stuff from xda-developer forums but what the hell, the phone is empty anyway so nothing to loose and I flashed the latest falcon build.

To flash it I had to clean the whole phone, format all partitions via TWRP and then sideloaded the zip image file via adb (adb from the Debian/stretch adb package works like a charm, thank you guys!). Booted and bäm wifi works again! Now Exodus is a really striped down mod, to do anything useful with it I had to activate the developer options and allow USB debugging. Afterwards I could install the f-droid and Opera apk via "adb install foo.apk".

Lineage OS

As I could derive from another thread on xda-developers Lineago OS has the falcon still on the shortlist for 14.x nightly builds. Maybe that will be an alternative again in the future. For now Exodus is a bit behind the curve (based on Android 6.0.1 from September 2016) but at least it's functional.

Jonathan McDowell: Cloning a USB LED device

14 January, 2017 - 18:53

A month or so ago I got involved in a discussion on IRC about notification methods for a headless NAS. One of the options considered was some sort of USB attached LED. DealExtreme had a cheap “Webmail notifier”, which was already supported by mainline kernels as a “Riso Kagaku” device but it had been sold out for some time.

This seemed like a fun problem to solve with a tinyAVR and V-USB. I had my USB relay board so I figured I could use that to at least get some code to the point that the kernel detected it as the right device, and the relay output could be configured as one of the colours to ensure it was being driven in roughly the right manner. The lack of a full lsusb dump (at least when I started out) made things a bit harder, plus the fact that the Riso uses an output report unlike the relay code, which uses a control message. However I had the kernel source for the driver and with a little bit of experimentation had something which would cause the driver to be loaded and the appropriate files in /sys/class/leds/ to be created. The relay was then successfully activated when the red LED was supposed to be on.

hid-led 0003:1294:1320.0001: hidraw0: USB HID v1.01 Device [MAIL  MAIL ] on usb-0000:00:14.0-6.2/input0
hid-led 0003:1294:1320.0001: Riso Kagaku Webmail Notifier initialized

I subsequently ordered some Digispark clones and modified the code to reflect the pins there (my relay board used pins 1+2 for USB, the Digispark uses pins 3+4). I then soldered a tricolour LED to the board, plugged it in and had a clone of the Riso Kaguku device for about £1.50 in parts (no doubt much cheaper in bulk). Very chuffed.

In case it’s useful to someone, the code is released under GPLv3+ and is available at;a=summary or on GitHub at I’m seeing occasional issues on an older Dell machine that only does USB2 with enumeration, but it generally is fine once it gets over that.

(FWIW, Jon, who started the original discussion, ended up with a BlinkStick Nano which is a neater device with 2 LEDs but still based on an Tiny85.)

Jamie McClelland: What's Up with WhatsApp?

14 January, 2017 - 09:03

Despite my jaded feelings about corporate Internet services in general, I was suprised to learn that WhatsApp's end-to-end encryption was a lie. In short, it is possible to send an encrypted message to a user that is intercepted and effectively de-crypted without the sender's knowledge.

However, I was even more surprised to read Open Whisper Systems critique of the original story, claiming that it is not a backdoor because the WhatsApp sender's client is always notified when a message is de-crypted.

The Open Whisper Systems post acknowledges that the WhatsApp sender can choose to disable these notifications, but claims that is not such a big deal because the WhatsApp server has no way to know which clients have this feature enabled and which do not, so intercepting a message is risky because it could result in the sender realizing it.

However, there is a fairly important piece of information missing, namely: as far as I can tell, the setting to notify users about key changes is disabled by default.

So, using the default installation, your end-to-end encrypted message could be intercepted and decrypted without you or the party you are communicating with knowing it. How is this not a back door? And yes, if the interceptor can't tell whether or not the sender has these notifications turned on, the interceptor runs the risk of someone knowing they have intercepted the message. Great. That's better than nothing. Except that there is strong evidence that many powerful governments on this planet routinely risk exposure in their pursuit of compromising our ability to communicate securely. And... not to mention non-governmental (or governmental) adversaries for whom exposure is not a big deal.

Furthermore a critical reason for end-to-end encrption is so that your provider does not have the technical capacity to intercept your communications. That's simply not true of WhatsApp. It is true of Signal and OMEMO, which requires the active participation of the sender to compromise the communication.

Why in the world would you distribute a client that not only has the ability to surpress such warnings, but has it enabled by default?

Some may argue that users regularly dismiss notifications like "fingerprint has changed" and that this problem is the achilles heal of secure communications. I agree. But... there is still a monumental difference between a user absent-mindedly dismissing an important security warning and never seeing the warning in the first place.

This flaw in WhatsApp is a critical reminder that secure communications doesn't just depend on a good protocol or technology, but on trust in the people who design and maintain our systems.

Elena 'valhalla' Grandi: Modern XMPP Server

13 January, 2017 - 19:59
Modern XMPP Server

I've published a new HOWTO on my website '': already wrote about the Why (and the What, Who and When), so I'll just quote his conclusion and move on to the How.

I now have an XMPP setup which has all the features of the recent fancy chat systems, and on top of that it runs, client and server, on Free Software, which can be audited, it is federated and I can self-host my own server in my own VPS if I want to, with packages supported in Debian.


I've decided to install, mostly because it was recommended by the RTC QuickStart Guide; I've heard that similar results can be reached with and other servers.

I'm also targeting stable (+ backports); as I write this is jessie; if there are significant differences I will update this article when I will upgrade my server to stretch. Right now, this means that I'm using prosody 0.9 (and that's probably also the version that will be available in stretch).

Installation and prerequisites

You will need to enable the repository and then install the packages prosody and prosody-modules.

You also need to setup some TLS certificates (I used Let's Encrypt; and make them readable by the prosody user; you can see Chapter 12 of the RTC QuickStart Guide for more details.

On your firewall, you'll need to open the following TCP ports:

  • 5222 (client2server)

  • 5269 (server2server)

  • 5280 (default http port for prosody)

  • 5281 (default https port for prosody)

The latter two are needed to enable some services provided via http(s), including rich media transfers.

With just a handful of users, I didn't bother to configure LDAP or anything else, but just created users manually via:

prosodyctl adduser

In-band registration is disabled by default (and I've left it that way, to prevent my server from being used to send spim

prosody configuration

You can then start configuring prosody by editing /etc/prosody/prosody.cfg.lua and changing a few values from the distribution defaults.

First of all, enforce the use of encryption and certificate checking both for client2server and server2server communications with:

c2s_require_encryption = true
s2s_secure_auth = true

and then, sadly, add to the whitelist any server that you want to talk to and doesn't support the above:

s2s_insecure_domains = { "" }


For each virtualhost you want to configure, create a file /etc/prosody/conf.avail/ with contents like the following:

VirtualHost ""
enabled = true
ssl = {
key = "/etc/ssl/private/";
certificate = "/etc/ssl/public/";

For the domains where you also want to enable MUCs, add the follwing lines:

Component "" "muc"
restrict_room_creation = "local"

the "local" configures prosody so that only local users are allowed to create new rooms (but then everybody can join them, if the room administrator allows it): this may help reduce unwanted usages of your server by random people.

You can also add the following line to enable rich media transfers via http uploads (XEP-0363):

Component "" "http_upload"

The defaults are pretty sane, but see for details on what knobs you can configure for this module

Don't forget to enable the virtualhost by linking the file inside /etc/prosody/conf.d/.

additional modules

Most of the other interesting XEPs are enabled by loading additional modules inside /etc/prosody/prosody.cfg.lua (under modules_enabled); to enable mod_something just add a line like:


Most of these come from the prosody-modules package (and thus from ) and some may require changing when prosody 0.10 will be available; when this is the case it is mentioned below.

  • mod_carbons (XEP-0280)
    To keep conversations syncronized while using multiple devices at the same time.

    This will be included by default in prosody 0.10.

  • mod_privacy + mod_blocking (XEP-0191)
    To allow user-controlled blocking of users, including as an anti-spim measure.

    In prosody 0.10 these two modules will be replaced by mod_privacy.

  • mod_smacks (XEP-0198)
    Allow clients to resume a disconnected session before a customizable timeout and prevent message loss.

  • mod_mam (XEP-0313)
    Archive messages on the server for a limited period of time (default 1 week) and allow clients to retrieve them; this is required to syncronize message history between multiple clients.

    With prosody 0.9 only an in-memory storage backend is available, which may make this module problematic on servers with many users. prosody 0.10 will fix this by adding support for an SQL backed storage with archiving capabilities.

  • mod_throttle_presence + mod_filter_chatstates (XEP-0352)
    Filter out presence updates and chat states when the client announces (via Client State Indication) that the user isn't looking. This is useful to reduce power and bandwidth usage for "useless" traffic.

@Gruppo Linux Como @LIFO

Ben Hutchings: Debian 8 kernel security update

13 January, 2017 - 05:41

There are a fair number of outstanding security issues in the Linux kernel for Debian 8 "jessie", but none of them were considered serious enough to issue a security update and DSA. Instead, most of them are being fixed through the point release (8.7) which will be released this weekend. Don't forget that you need to reboot to complete a kernel upgrade.

This update to linux (version 3.16.39-1) also adds the perf security mitigation feature from Grsecurity. You can disable unprivileged use of perf entirely by setting sysctl kernel.perf_event_paranoid=3. (This is the default for Debian "stretch".)

Ben Hutchings: Debian LTS work, December 2016

13 January, 2017 - 05:30

I was assigned 13.5 hours of work by Freexian's Debian LTS initiative and carried over 2 from November. I worked only 10 hours, so I carry over 5.5 hours.

As for the last few months, I spent all of this time working on the linux (kernel) package. I backported several security fixes and did some testing of the more invasive changes.

I also added the option to mitigate security issues in the performance events (perf) subsystem by disabling use by unprivileged users. This feature comes from Grsecurity and has been included in Debian unstable and Android kernels for a while. However, for Debian 7 LTS it has to be explicitly enabled by setting sysctl kernel.perf_event_paranoid=3.

I uploaded these changes as linux 3.2.84-1 and then (on 1st January) issued DLA 722-1.

Ritesh Raj Sarraf: Laptop Mode Tools 1.71

12 January, 2017 - 15:54

I am pleased to announce the 1.71 release of Laptop Mode Tools. This release includes some new modules, some bug fixes, and there are some efficiency improvements too. Many thanks to our users; most changes in this release are contributions from our users.

A filtered list of changes in mentioned below. For the full log, please refer to the git repository. 

Source tarball, Feodra/SUSE RPM Packages available at:

Debian packages will be available soon in Unstable.

Mailing List:


1.71 - Thu Jan 12 13:30:50 IST 2017
    * Fix incorrect import of os.putenv
    * Merge pull request #74 from Coucouf/fix-os-putenv
    * Fix documentation on where we read battery capacity from
    * cpuhotplug: allow disabling specific cpus
    * Merge pull request #78 from aartamonau/cpuhotplug
    * runtime-pm: refactor listed_by_id()
    * wireless-power: Use iw and fallback to iwconfig if it not available
    * Prefer available AC supply information over battery state to determine ON_AC
    * On startup, we want to force the full execution of LMT.
    * Device hotplugs need a forced execution for LMT to apply the proper settings
    * runtime-pm: Refactor list_by_type()
    * kbd-backlight: New module to control keyboard backlight brightness
    * Include Transmit power saving in wireless cards
    * Don't run in a subshell
    * Try harder to check battery charge
    * New module: vgaswitcheroo
    * Revive bluetooth module. Use rfkill primarily. Also don't unload (incomplete list of) kernel modules


What is Laptop Mode Tools
Description: Tools for Power Savings based on battery/AC status
 Laptop mode is a Linux kernel feature that allows your laptop to save
 considerable power, by allowing the hard drive to spin down for longer
 periods of time. This package contains the userland scripts that are
 needed to enable laptop mode.
 It includes support for automatically enabling laptop mode when the
 computer is working on batteries. It also supports various other power
 management features, such as starting and stopping daemons depending on
 power mode, automatically hibernating if battery levels are too low, and
 adjusting terminal blanking and X11 screen blanking
 laptop-mode-tools uses the Linux kernel's Laptop Mode feature and thus
 is also used on Desktops and Servers to conserve power
Categories: Keywords: Like: 

Steinar H. Gunderson: 3G-SDI signal support

12 January, 2017 - 02:03

I had to figure out what kinds of signal you can run over 3G-SDI today, and it's pretty confusing, so I thought I'd share it.

For the reference, 3G-SDI is the same as 3G HD-SDI, an extension of HD-SDI, which is an extension of the venerable SDI standard (well, duh). They're all used for running uncompressed audio/video data of regular BNC coaxial cable, possibly hundreds of meters, and are in wide use in professional and semiprofessional setups.

So here's the rundown on 3G-SDI capabilities:

  • 1080p60 supports 10-bit 4:2:2 Y'CbCr. Period.
  • 720p60/1080p30/1080i60 supports a much wider range of formats: 10-bit 4:4:4:4 RGBA (alpha optional), 10-bit 4:4:4:4 Y'CbCrA (alpha optional), 12-bit 4:4:4 RGB, 12-bit 4:4:4 Y'CbCr or finally 12-bit 4:2:2 Y'CbCr (seems rather redundant).
  • There's also a format exclusively for 1080p24 (actually 2048x1080) that supports 12-bit X'Y'Z. Digital cinema, hello. Apart from that, it supports pretty much what 1080p30 does. There's also a 2048x1080p30 (no interlaced version) mode for 12-bit 4:2:2:4 Y'CbCrA, but it seems rather obscure.

And then there's dual-link 3G-SDI, which uses two cables instead of one—and there's also Blackmagic's proprietary “6G-SDI”, which supports basically everything dual-link 3G-SDI does. But in 2015, seemingly there was also a real 6G-SDI and 12G-SDI, and it's unclear to me whether it's in any way compatible with Blackmagic's offering. It's all confusing. But at least, these are the differences from single-link to dual-link 3G-SDI:

  • 1080p60 supports essentially everything that 720p60 supports on single-link: 10-bit 4:4:4:4 RGBA (alpha optional), 10-bit 4:4:4:4 Y'CbCrA (alpha optional), 12-bit 4:4:4 RGB, 12-bit 4:4:4 Y'CbCr and the redundant 12-bit 4:2:2 Y'CbCr.
  • 2048x1080 4:4:4 X'Y'Z' now also supports 1080p25 and 1080p30.

4K? I don't know. 120fps? I believe that's also a proprietary extension of some sort.

And of course, having a device support 3G-SDI doesn't mean at all it's required to support all of this; in particular, I believe Blackmagic's systems don't support alpha at all except on their single “12G-SDI” card, and I'd also not be surprised if RGB support is rather limited in practice.

Sven Hoexter: Failing with F5: using experimental mv feature on a pool causes tmm to segfault

12 January, 2017 - 00:36

Just a short PSA for those around working with F5 devices:

TMOS 11.6 introduced an experimental "mv" command in tmsh. In the last days we tried it for the first time on TMOS 12.1.1. It worked fine for a VirtualServer but a mv for a pool caused a sefault in tmm. We're currently working with the F5 support to sort it out, they think it's a known issue. Recommendation for now is to not use mv on pools. Just do it the old way, create a new pool, assign the new pool to the relevant VS and delete the old pool.

Possible bug ID at F5 is ID562808. Since I can not find it in the TMOS 12.2 release notes I expect that this issue also applies to TMOS 12.2, but I did not verify that.

Reproducible builds folks: Reproducible Builds: week 89 in Stretch cycle

11 January, 2017 - 22:04

What happened in the Reproducible Builds effort between Sunday January 1 and Saturday January 7 2017:

GSoC and Outreachy updates Toolchain development
  • #849999 was filed: "dpkg-dev should not set SOURCE_DATE_EPOCH to the empty string"
Packages reviewed and fixed, and bugs filed

Chris Lamb:


Reviews of unreproducible packages

13 package reviews have been added, 4 have been updated and 6 have been removed in this week, adding to our knowledge about identified issues.

2 issue types have been added/updated:

Upstreaming of reproducibility fixes



Weekly QA work

During our reproducibility testing, the following FTBFS bugs have been detected and reported by:

  • Chris Lamb (4)
diffoscope development

diffoscope 67 was uploaded to unstable by Chris Lamb. It included contributions from :

[ Chris Lamb ]

* Optimisations:
  - Avoid multiple iterations over archive by unpacking once for an ~8X
    runtime optimisation.
  - Avoid unnecessary splitting and interpolating for a ~20X optimisation
    when writing --text output.
  - Avoid expensive diff regex parsing until we need it, speeding up diff
    parsing by 2X.
  - Alias expensive Config() in diff parsing lookup for a 10% optimisation.

* Progress bar:
  - Show filenames, ELF sections, etc. in progress bar.
  - Emit JSON on the the status file descriptor output instead of a custom

* Logging:
  - Use more-Pythonic logging functions and output based on __name__, etc.
  - Use Debian-style "I:", "D:" log level format modifier.
  - Only print milliseconds in output, not microseconds.
  - Print version in debug output so that saved debug outputs can standalone
    as bug reports.

* Profiling:
  - Also report the total number of method calls, not just the total time.
  - Report on the total wall clock taken to execute diffoscope, including

* Tidying:
  - Rename "NonExisting" -> "Missing".
  - Entirely rework diffoscope.comparators module, splitting as many separate
    concerns into a different utility package, tidying imports, etc.
  - Split diffoscope.difference into diffoscope.diff, etc.
  - Update file references in debian/copyright post module reorganisation.
  - Many other cleanups, etc.

* Misc:
  - Clarify comment regarding why we call python3(1) directly. Thanks to Jérémy
    Bobbio <>.
  - Raise a clearer error if trying to use --html-dir on a file.
  - Fix --output-empty when files are identical and no outputs specified.

[ Reiner Herrmann ]
* Extend .apk recognition regex to also match zip archives (Closes: #849638)

[ Mattia Rizzolo ]
* Follow the rename of the Debian package "python-jsbeautifier" to

[ siamezzze ]
* Fixed no newline being classified as order-like difference.
reprotest development

reprotest 0.5 was uploaded to unstable by Chris Lamb. It included contributions from:

[ Ximin Luo ]

* Stop advertising variations that we're not actually varying.
  That is: domain_host, shell, user_group.
* Fix auto-presets in the case of a file in the current directory.
* Allow disabling build-path variations. (Closes: #833284)
* Add a faketime variation, with NO_FAKE_STAT=1 to avoid messing with
  various buildsystems. This is on by default; if it causes your builds
  to mess up please do file a bug report.
* Add a --store-dir option to save artifacts.

Other contributions (not yet uploaded): website development
  • Debian arm64 architecture was fully tested in all three suites in just 15 days. Thanks again to for their support!
  • Log diffoscope profiling info. (lamby)
  • Run pg_dump with -O --column-inserts to make easier to import our main database dump into a non-PostgreSQL database. (mapreri)
  • Debian armhf network: CPU frequency scaling was enabled for three Firefly boards, enabling the CPUs to run at full speed. (vagrant)
  • Arch Linux and Fedora tests have been disabled (h01ger)
  • Improve mail notifications about daily problems. (h01ger)

This week's edition was written by Chris Lamb, Holger Levsen and Vagrant Cascadian, reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Dirk Eddelbuettel: R / Finance 2017 Call for Papers

11 January, 2017 - 18:44

Last week, Josh sent the call for papers to the R-SIG-Finance list making everyone aware that we will have our nineth annual R/Finance conference in Chicago in May. Please see the call for paper (at the link, below, or at the website) and consider submitting a paper.

We are once again very excited about our conference, thrilled about upcoming keynotes and hope that many R / Finance users will not only join us in Chicago in May 2017 -- but also submit an exciting proposal.

We also overhauled the website, so please see R/Finance. It should render well and fast on devices of all sizes: phones, tablets, desktops with browsers in different resolutions. The program and registration details still correspond to last year's conference and will be updated in due course.

So read on below, and see you in Chicago in May!

Call for Papers

R/Finance 2017: Applied Finance with R
May 19 and 20, 2017
University of Illinois at Chicago, IL, USA

The ninth annual R/Finance conference for applied finance using R will be held on May 19 and 20, 2017 in Chicago, IL, USA at the University of Illinois at Chicago. The conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure, and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management, portfolio construction, and trading.

Over the past eight years, R/Finance has included attendees from around the world. It has featured presentations from prominent academics and practitioners, and we anticipate another exciting line-up for 2017.

We invite you to submit complete papers in pdf format for consideration. We will also consider one-page abstracts (in txt or pdf format) although more complete papers are preferred. We welcome submissions for both full talks and abbreviated "lightning talks." Both academic and practitioner proposals related to R are encouraged.

All slides will be made publicly available at conference time. Presenters are strongly encouraged to provide working R code to accompany the slides. Data sets should also be made public for the purposes of reproducibility (though we realize this may be limited due to contracts with data vendors). Preference may be given to presenters who have released R packages.

Financial assistance for travel and accommodation may be available to presenters, however requests must be made at the time of submission. Assistance will be granted at the discretion of the conference committee.

Please submit proposals online at

Submissions will be reviewed and accepted on a rolling basis with a final deadline of February 28, 2017. Submitters will be notified via email by March 31, 2017 of acceptance, presentation length, and financial assistance (if requested).

Additional details will be announced via the conference website as they become available. Information on previous years' presenters and their presentations are also at the conference website. We will make a separate announcement when registration opens.

For the program committee:

Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich

Enrico Zini: Modern and secure instant messaging

11 January, 2017 - 18:43

Conversations is a really nice, actively developed, up to date XMPP client for Android that has the nice feature of telling you what XEPs are supported by the server one is using:

Some days ago, me and Valhalla played the game of trying to see what happens when one turns them all on: I would send her screenshots from my Conversations, and she would poke at her Prosody to try and turn things on:

Valhalla eventually managed to get all features activated, purely using packages from Jessie+Backports:

The result was a chat system in which I could see the same conversation history on my phone and on my laptop (with gajim)(, and have it synced even after a device has been offline,

We could send each other rich media like photos, and could do OMEMO encryption (same as Signal) in group chats.

I now have an XMPP setup which has all the features of the recent fancy chat systems, and on top of that it runs, client and server, on Free Software, which can be audited, it is federated and I can self-host my own server in my own VPS if I want to, with packages supported in Debian.

Valhalla has documented the whole procedure and will publish details.

If you make a client for a protocol with lots of extension, do like Conversations and implement a status page with the features you'd like to have on the server, and little green indicators showing which are available: it is quite a good motivator for getting them all supported.

Dirk Eddelbuettel: nanotime 0.1.0: Now on Windows

11 January, 2017 - 07:49

Last month, we released nanotime, a package to work with nanosecond timestamps. See the initial release announcement for some background material and a few first examples.

nanotime relies on the RcppCCTZ package for high(er) resolution time parsing and formatting: R itself stops a little short of a microsecond. And it uses the bit64 package for the actual arithmetic: time at this granularity is commonly represented at (integer) increments (at nanosecond resolution) relative to an offset, for which the standard epoch of Januar 1, 1970 is used. int64 types are a perfect match here, and bit64 gives us an integer64. Naysayers will point out some technical limitations with R's S3 classes, but it works pretty much as needed here.

The one thing we did not have was Windows support. RcppCCTZ and the CCTZ library it uses need real C++11 support, and the g++-4.9 compiler used on Windows falls a little short lacking inter alia a suitable std::get_time() implementation. Enter Dan Dillon who ported this from LLVM's libc++ which lead to Sunday's RcppCCTZ 0.2.0 release.

And now we have all our ducks in a row: everything works on Windows too. The next paragraph summarizes the changes for both this release as well as the initial one last month:

Changes in version 0.1.0 (2017-01-10)
  • Added Windows support thanks to expanded RcppCCTZ (closes #6)

  • Added "mocked up" demo with nanosecond delay networking analysis

  • Added 'fmt' and 'tz' options to output functions, expanded format.nanotime (closing #2 and #3)

  • Added data.frame support

  • Expanded tests

Changes in version 0.0.1 (2016-12-15)
  • Initial CRAN upload.

  • Package is functional and provides examples.

We also have a diff to the previous version thanks to CRANberries. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Bálint Réczey: Debian Developer Game of the Year

11 January, 2017 - 05:03

I have just finished level one, fixing all RC bugs in packages under my name, even in team-maintained ones.

Next level is no unclassified bug reports, which gonna be harder since I have just adopted shadow with 70+ open bugs. :-\

Luckily I can still go on bonus tracks which is fixing (RC) bugs in others’ packages, but one should not spend all the time on those track before finishing level 1!

PS: Last time I tried playing a conventional game I ended up fixing it in a few minutes instead.

Vincent Fourmond: Version 2.1 of QSoas is out

10 January, 2017 - 14:47
I have just released QSoas version 2.1. It brings in a new solve command to solve arbitrary non-linear equations of one unknown. I took advantage of this command in the figure to solve the equation for . It also provides a new way to reparametrize fits using the reparametrize-fit command, a new series of fits to model the behaviour of an adsorbed 1- or 2-electrons catalyst on an electrode (these fits are discussed in great details in our recent review (DOI: 10.1016/j.coelec.2016.11.002), improvements in various commands, the possibility to now compile using Ruby 2.3 and the most recent version of the GSL library, and sketches for an emacs major mode, which you can activate (for QSoas script files, ending in .cmds) using the following snippet in $HOME/.emacs:

(autoload 'qsoas-mode "$HOME/Prog/QSoas/misc/qsoas-mode.el" nil t)
(add-to-list 'auto-mode-alist '("\\.cmds$" . qsoas-mode))

Of course, you'll have to adapt the path $HOME/Prog/QSoas/misc/qsoas-mode.el to the actual location of qsoas-mode.el.

As before, you can download the source code from our website, and purchase the pre-built binaries following the links from that page too. Enjoy !

Sean Whitton: jan17vcspkg

10 January, 2017 - 03:14

There have been a two long threads on the debian-devel mailing list about the representation of the changes to upstream source code made by Debian maintainers. Here are a few notes for my own reference.

I spent a lot of time defending the workflow I described in dgit-maint-merge(7) (which was inspired by this blog post). However, I came to be convinced that there is a case for a manually curated series of patches for certain classes of package. It will depend on how upstream uses git (rebasing or merging) and on whether the Debian delta from upstream is significant and/or long-standing. I still think that we should be using dgit-maint-merge(7) for leaf or near-leaf packages, because it saves so much volunteer time that can be better spent on other things.

When upstream does use a merging workflow, one advantage of the dgit-maint-merge(7) workflow is that Debian’s packaging is just another branch of development.

Now consider packages where we do want a manually curated patch series. It is very hard to represent such a series in git. The only natural way to do it is to continually rebase the patch series against an upstream branch, but public branches that get rebased are not a good idea. The solution that many people have adopted is to represent their patch series as a folder full of .diff files, and then use gbp pq to convert this into a rebasing branch. This branch is not shared. It is edited, rebased, and then converted back to the folder of .diff files, the changes to which are then committed to git.

One of the advantages of dgit is that there now exists an official, non-rebasing git history of uploads to the archive. It would be nice if we could represent curated patch series as branches in the dgit repos, rather than as folders full of .diff files. But as I just described, this is very hard. However, Ian Jackson has the beginnings of a workflow that just might fit the bill.

Shirish Agarwal: The Great Indian Digital Tamasha

9 January, 2017 - 20:38

This is an extension to last month’s article/sharing where I had shared the changes that had transpired in the last 2-3 months. Now am in a position to share the kind of issues a user can go through in case he is looking for support from IRCTC to help him/her go cashless. If you a new user to use IRCTC services you wouldn’t go through this trouble.

For those who might have TL;DR issues it’s about how hard it can become to get digital credentials fixed in IRCTC (Indian Railway Catering and Tourism Corporation) –

a. 2 months back Indian Prime Minister gave a call incentivizing people to use digital means to do any commercial activities. One of the big organizations which took/takes part is IRCTC which handles the responsibility for e-ticketing millions of Rail tickets for common people. In India, a massive percentage moves by train as it’s cheaper than going by Air.

A typical fare from say Pune – Delhi (capital of India) by second class sleeper would be INR 645/- for a distance of roughly 1600 odd kms and these are monopoly rates, there are no private trains and I’m not suggesting anything of that sort, just making sure that people know.

An economy class ticket by Air for the same distance would be anywhere between INR 2500-3500/- for a 2 hour flight between different airlines. Last I checked there are around 8 mainstream airlines including flag-carrier Air India.

About 30% of the population live on less than a dollar and a half a day which would come around INR 100/-.

There was a comment some six months back on getting more people out of the poverty line. But as there are lots of manipulations in numbers for who and what denotes above poor and below poor in India and lot of it has to do with politics it’s not something which would be easily fixable.

There are lots to be said in that arena but this article is not an appropriate blog-post for that.

All in all, it’s only 3-5% of the population at the most who can travel via Air if situation demands and around 1-2% who might be frequent, business or leisure travellers.

Now while I can thankfully afford an Air Ticket if the situation so demands, my mother gets motion sickness so while together we can only travel by train.

b. With the above background, I had registered with IRCTC few years ago with another number (dual-SIM) I had purchased and was thinking that I would be using this long-term (seems to my first big mistake, hindsight 50:50) . This was somewhere in 2006/2007.

c. Few months later I found that the other service provider wasn’t giving good service or was not upto mark. I was using IDEA (the main mobile operator) throughout those times.

d. As I didn’t need the service that much, didn’t think to inform them that I want to change to another service provider at that point in time (possibly the biggest mistake, hindsight 50:50)

e. In July 2016 itself IRCTC cut service fees,

f. This was shared as a NEW news item/policy decision at November-end 2016 .

g. While I have done all that has been asked by irctc-care haven’t still got the issues resolved IRCTC’s e-mail id –

Now in detail –

This is my first e-mail sent to IRCTC in June 2016 –

Dear Customer care,

I had applied and got username and password sometime back . The
number I had used to register with IRCTC was xxxxxxxxxx (BSNL mobile number not used anymore) . My mobile was lost and along with that the number was also lost. I had filed a complaint with the police and stopped that number as well. Now I have an another mobile number but have forgotten both the password and the security answer that I had given when I had registered . I do have all the conversations I had both with the as well as if needed to prove my identity.

The new number I want to tie it with is xxxxxxxxxx (IDEA number in-use for last 10 years)

I see two options :-

a. Tie the other number with my e-mail address

b. Take out the e-mail address from the database so that I can fill in
as a new applicant.

Looking forward to hear from you.

There was lot of back and forth with various individuals on IRCTC and after a lot of back and forth, this is the final e-mail I got from them somewhere in August 2016, he writes –

Dear Customer,

We request you to send mobile bill of your mobile number if it is post paid or if it is prepaid then contact to your service provider and they will give you valid proof of your mobile number or they will give you in written on company head letter so that we may update your mobile number to update so that you may reset your password through mobile OTP.
and Kindly inform you that you can update your profile by yourself also.

1.login on IRCTC website
2.after login successfully move courser on “my profile” tab.
3.then click on “update profile” your password then you can update your profile on user-profile then email id.
6. click on update.

Still you face any problem related to update profile please revert to us with the screen shots of error message which you will get at the time of update profile .

Thanks & Regards

Parivesh Patel
Executive, Customer Care

IRCTC’s response seemed responsible, valid and thought it would be a cake-walk as private providers are supposed to be much more efficient than public ones. The experience proved how wrong was I trust them with doing the right thing –

1. First I tried the twitter handle to see how IDEA uses their twitter handle.

2. The idea customer care twitter handle was mild in its response.

3. After sometime I realized that the only way out of this quagmire would perhaps be to go to a brick-mortar shop and get it resolved face-to-face. I went twice or thrice but each time something or the other would happen.

On the fourth and final time, I was able to get to the big ‘Official’ shop only to be told they can’t do anything about this and I would have to the appellate body to get the reply.

The e-mail address which they shared (and I found it later) was wrong. I sent a somewhat longish e-mail sharing all the details and got bounce-backs. The correct e-mail address for the IDEA Maharashtra appellate body is –

I searched online and after a bit of hit and miss finally got the relevant address. Then finally on 30th December, 2016 wrote a short email to the service provider as follows –

Dear Sir,
I have been using prepaid mobile connection –

number – xxxxxxx

taken from IDEA for last 10 odd years.

I want to register myself with IRCTC for online railway booking using
my IDEA mobile number.

Earlier, I was having a BSNL connection which I discontinued 4 years back,

For re-registering myself with IRCTC, I have to fulfill their latest
requirements as shown in the email below .

It is requested that I please be issued a letter confirming my
credentials with your esteemed firm.

I contacted your local office at corner of Law College Road and
Bhandarkar Road, Pune (reference number – Q1 – 84786060793) who
refused to provide me any letter and have advised me to contact on the
above e-mail address, hence this request is being forwarded to you.

Please do the needful at your earliest.

Few days later I got this short e-mail from them –

Dear Customer,

Greetings for the day!

This is with reference to your email regarding services.

Please accept our apologies for the inconvenience caused to you and delay in response.

We regret to inform you that we are unable to provide demographic details from our end as provision for same is not available with us.

Should you need any further assistance, please call our Customer Service help line number 9822012345 or email us at by mentioning ten digit Idea mobile number in subject line.

Thanks & Regards,

Javed Khan

Customer Service Team

IDEA Cellular Limited- Maharashtra & Goa Circle.

Now I was at almost my wit’s end. Few days before, I had re-affirmed my e-mail address to IDEA . I went to the IDEA care site, registered with my credentials. While the https connection to the page is weak, but let’s not dwell on that atm.

I logged into the site, I went through all the drop-down menus and came across My Account > Raise a request link which I clicked on . This came to a page where I could raise requests for various things. One of the options given there was Bill Delivery. As I wasn’t a postpaid user but a prepaid user didn’t know if that would work or not I still clicked on it. It said it would take 4 days for that to happen. I absently filed it away as I was somewhat sure that nothing would happen from my previous experience with IDEA. But this time the IDEA support staff came through and shared a toll-free SMS number and message format that I could use to generate call details from the last 6 months.

The toll-free number from IDEA is 12345 and the message format is EBILL MON (short-form for month so if it’s January would be jan, so on and so forth).

After gathering all the required credentials, sent my last mail to IRCTC about a week, 10 days back –

Dear Mr. Parivesh Patel,

I was out-of-town and couldn’t do the needful so sorry for the delay.
Now that I’m back in town, I have been able to put together my prepaid
bills of last 6 months which should make it easy to establish my

As had shared before, I don’t remember my old password and the old
mobile number (BSNL number) is no longer accessible so can’t go
through that route.

Please let me know the next steps in correcting the existing IRCTC
account (which I haven’t operated ever) so I can start using it to
book my tickets.

Look forward to hearing from you.

Haven’t heard anything them from them, apart from a generated token number, each time you send a reply happens. This time it was #4763548

The whole sequence of events throws a lot of troubling questions –

a. Could IRCTC done a better job of articulating their need to me instead of the run-around I was given ?

b. Shouldn’t there be a time limit to accounts from which no transactions have been done ? I hadn’t done a single transaction since registering. When cell service providers including BSNL takes number out after a year of not using a number, why is that account active for so long ?

c. As that account didn’t have OTP at registration, dunno if it’s being used for illegal activities or something.

Update – This doesn’t seem to be a unique thing at all. Just sampling some of the tweets by people at @IRCTC_LTD , all of this just goes to show how un-unique the situation really is.

Filed under: Miscellenous Tagged: #customer-service, #demonetization, #IDEA-aditya birla, #IRCTC, #web-services, rant

Petter Reinholdtsen: Where did that package go? — geolocated IP traceroute

9 January, 2017 - 18:20

Did you ever wonder where the web trafic really flow to reach the web servers, and who own the network equipment it is flowing through? It is possible to get a glimpse of this from using traceroute, but it is hard to find all the details. Many years ago, I wrote a system to map the Norwegian Internet (trying to figure out if our plans for a network game service would get low enough latency, and who we needed to talk to about setting up game servers close to the users. Back then I used traceroute output from many locations (I asked my friends to run a script and send me their traceroute output) to create the graph and the map. The output from traceroute typically look like this:

traceroute to (, 30 hops max, 60 byte packets
 1 (  0.447 ms  0.486 ms  0.621 ms
 2 (  0.467 ms  0.578 ms  0.675 ms
 3 (  0.385 ms  0.373 ms  0.358 ms
 4 (  1.174 ms  1.172 ms  1.153 ms
 5 (  2.627 ms (  3.172 ms (  2.857 ms
 6 (  0.662 ms  0.637 ms (  0.622 ms
 7 (  0.931 ms  0.917 ms  0.955 ms
 8  * * *
 9  * * *

This show the DNS names and IP addresses of (at least some of the) network equipment involved in getting the data traffic from me to the server, and how long it took in milliseconds for a package to reach the equipment and return to me. Three packages are sent, and some times the packages do not follow the same path. This is shown for hop 5, where three different IP addresses replied to the traceroute request.

There are many ways to measure trace routes. Other good traceroute implementations I use are traceroute (using ICMP packages) mtr (can do both ICMP, UDP and TCP) and scapy (python library with ICMP, UDP, TCP traceroute and a lot of other capabilities). All of them are easily available in Debian.

This time around, I wanted to know the geographic location of different route points, to visualize how visiting a web page spread information about the visit to a lot of servers around the globe. The background is that a web site today often will ask the browser to get from many servers the parts (for example HTML, JSON, fonts, JavaScript, CSS, video) required to display the content. This will leak information about the visit to those controlling these servers and anyone able to peek at the data traffic passing by (like your ISP, the ISPs backbone provider, FRA, GCHQ, NSA and others).

Lets pick an example, the Norwegian parliament web site It is read daily by all members of parliament and their staff, as well as political journalists, activits and many other citizens of Norway. A visit to the web site will ask your browser to contact 8 other servers:,,,,,, and I extracted this by asking PhantomJS to visit the Stortinget web page and tell me all the URLs PhantomJS downloaded to render the page (in HAR format using their netsniff example. I am very grateful to Gorm for showing me how to do this). My goal is to visualize network traces to all IP addresses behind these DNS names, do show where visitors personal information is spread when visiting the page.

When I had a look around for options, I could not find any good free software tools to do this, and decided I needed my own traceroute wrapper outputting KML based on locations looked up using GeoIP. KML is easy to work with and easy to generate, and understood by several of the GIS tools I have available. I got good help from by NUUG colleague Anders Einar with this, and the result can be seen in my kmltraceroute git repository. Unfortunately, the quality of the free GeoIP databases I could find (and the for-pay databases my friends had access to) is not up to the task. The IP addresses of central Internet infrastructure would typically be placed near the controlling companies main office, and not where the router is really located, as you can see from the KML file I created using the GeoLite City dataset from MaxMind.

I also had a look at the visual traceroute graph created by the scrapy project, showing IP network ownership (aka AS owner) for the IP address in question. The graph display a lot of useful information about the traceroute in SVG format, and give a good indication on who control the network equipment involved, but it do not include geolocation. This graph make it possible to see the information is made available at least for UNINETT, Catchcom, Stortinget, Nordunet, Google, Amazon, Telia, Level 3 Communications and NetDNA.

In the process, I came across the web service GeoTraceroute by Salim Gasmi. Its methology of combining guesses based on DNS names, various location databases and finally use latecy times to rule out candidate locations seemed to do a very good job of guessing correct geolocation. But it could only do one trace at the time, did not have a sensor in Norway and did not make the geolocations easily available for postprocessing. So I contacted the developer and asked if he would be willing to share the code (he refused until he had time to clean it up), but he was interested in providing the geolocations in a machine readable format, and willing to set up a sensor in Norway. So since yesterday, it is possible to run traces from Norway in this service thanks to a sensor node set up by the NUUG assosiation, and get the trace in KML format for further processing.

Here we can see a lot of trafic passes Sweden on its way to Denmark, Germany, Holland and Ireland. Plenty of places where the Snowden confirmations verified the traffic is read by various actors without your best interest as their top priority.

Combining KML files is trivial using a text editor, so I could loop over all the hosts behind the urls imported by and ask for the KML file from GeoTraceroute, and create a combined KML file with all the traces (unfortunately only one of the IP addresses behind the DNS name is traced this time. To get them all, one would have to request traces using IP number instead of DNS names from GeoTraceroute). That might be the next step in this project.

Armed with these tools, I find it a lot easier to figure out where the IP traffic moves and who control the boxes involved in moving it. And every time the link crosses for example the Swedish border, we can be sure Swedish Signal Intelligence (FRA) is listening, as GCHQ do in Britain and NSA in USA and cables around the globe. (Hm, what should we tell them? :) Keep that in mind if you ever send anything unencrypted over the Internet.

PS: KML files are drawn using the KML viewer from Ivan Rublev, as it was less cluttered than the local Linux application Marble. There are heaps of other options too.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Guido Günther: Debian Fun in December 2016

9 January, 2017 - 15:24
Debian LTS

November marked the 20th month I contributed to Debian LTS under the Freexian umbrella. I had 8 hours allocated which I used by:

  • some rather quiet frontdesk days
  • updating icedove to 45.5.1 resulting in DLA-752-1 fixing 7 CVEs
  • looking whether Wheezy is affected by xsa-202, xsa-203, xsa-204 and handling the communication with credativ for these (update not yet released)
  • Assessing cURL/libcURL CVE-2016-9586
  • Assessing whether Wheezy's QEMU is affeced by security issues in 9pfs "proxy" and "handle" code
  • Releasing DLA-776-1 for samba fixing CVE-2016-2125
Other Debian stuff Some other Free Software activites


Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้