Planet Debian

Subscribe to Planet Debian feed
Planet Debian - http://planet.debian.org/
Updated: 52 min 38 sec ago

Norbert Preining: Fixing vim in Debian

16 October, 2017 - 08:18

I was wondering for quite some time why on my server vim behaves so stupid with respect to the mouse: Jumping around, copy and paste wasn’t possible the usual way. All this despite having

  set mouse=

in my /etc/vim/vimrc.local. Finally I found out why, thanks to bug #864074 and fixed it.

The whole mess comes from the fact that, when there is no ~/.vimrc, vim loads defaults.vim after vimrc.local and thus overwriting several settings put in there.

There is a comment (I didn’t see, though) in /etc/vim/vimrc explaining this:

" Vim will load $VIMRUNTIME/defaults.vim if the user does not have a vimrc.
" This happens after /etc/vim/vimrc(.local) are loaded, so it will override
" any settings in these files.
" If you don't want that to happen, uncomment the below line to prevent
" defaults.vim from being loaded.
" let g:skip_defaults_vim = 1

I agree that this is a good way to setup vim on a normal installation of Vim, but the Debian package could do better. The problem is laid out clearly in the bug report: If there is no ~/.vimrc, settings in /etc/vim/vimrc.local are overwritten.

This is as counterintuitive as it can be in Debian – and I don’t know any other package that does it in a similar way.

Since the settings in defaults.vim are quite reasonable, I want to have them, but only fix a few of the items I disagree with, like the mouse. At the end what I did is the following in my /etc/vim/vimrc.local:

if filereadable("/usr/share/vim/vim80/defaults.vim")
  source /usr/share/vim/vim80/defaults.vim
endif
" now set the line that the defaults file is not reloaded afterwards!
let g:skip_defaults_vim = 1

" turn of mouse
set mouse=
" other override settings go here

There is probably a better way to get a generic load statement that does not depend on the Vim version, but for now I am fine with that.

Pau Garcia i Quiles: FOSDEM 2018 Retrocomputing DevRoom Call for Participation

16 October, 2017 - 07:11

FOSDEM is a free software event that offers open source communities a place to meet, share ideas and collaborate. It is renown for being highly developer-oriented and brings together 8000+ participants from all over the world. It is held in the city of Brussels (Belgium).

FOSDEM 2018 will take place during the weekend of February 3rd-4th 2018. More details about the event can be found at http://www.fosdem.org

Call for Participation

The Retrocomputing DevRoom is a first-timer at FOSDEM, with talks about use of older computing hardware and software in modern times.

Presentation topics could include but are not limited to:

  • Emulation of old systems to run videogames, legacy software, etc
  • Open source software emulation/simulation
  • Open hardware
  • Operating systems/executives for retrocomputers/retrosystems
  • Uses of retrocomputing today
  • Other retrosystems topics
  • Opportunities in retrocomputing
  • Round tables

You are not limited to slide presentations, of course. Be creative. However, FOSDEM is an open source conference, therefore we ask you to stay clear of marketing presentations. We are not afraid of technical stuff: devrooms are a place for development teams to meet, discuss, hack and publicly present their project’s latest improvements and future directions.

If you will have special needs for your talk (e. g. because you will need to plug some sort of a system), please note that clearly in your proposal so that we can provide it.

You can use the Wikipedia definition of retrocomputing as a reference definition to see if you talk qualifies, although it is not exclusive: https://en.wikipedia.org/wiki/Retrocomputing

Important dates
  • 25 Nov 2017: submission deadline for talk proposals
  • 15 Dec 2017: announcement of the final schedule
  • 4 Feb 2018: Retrocomputing dev room
Useful information

Use the FOSDEM Pentabarf tool to submit your proposal: https://penta.fosdem.org/submission/FOSDEM18

If necessary, create a Pentabarf account and activate it. Please reuse your account from previous years if you have already created it.

Details on submission:

  • In the “Person” section, provide First name, Last name (in the “General” tab), Email (in the “Contact” tab) and Bio (“Abstract” field in the “Description” tab).
  • Submit a proposal by clicking on “Create event”.
  • Important! Select the “Retrocomputing DevRoom” track (on the “General” tab). If you do not select a track, then nobody, from any track, will look at your submission!
  • Provide the title of your talk (“Event title” in the “General” tab).
  • Provide a description of the subject of the talk and the intended audience (in the “Abstract” field of the “Description” tab)
  • Provide a rough outline of the talk or goals of the session (a short list of bullet points covering topics that will be discussed) in the “Full description” field in the “Description” tab
  • Provide an expected length of your talk in the “Duration” field, including discussion. The default duration is 30 minutes.

Please note neither FOSDEM nor the Retrocomputing DevRoom will reimburse any expenses you incur

Recording of talks

The FOSDEM organizers plan to have live streaming and recording fully working, both for remote/later viewing of talks, and so that people can watch streams in the hallways when rooms are full. This requires speakers to consent to being recorded and streamed.

If you plan to be a speaker, please understand that by doing so you implicitly give consent for your talk to be recorded and streamed.

The recordings will be published under the same license as all FOSDEM content (CC-BY).

Contact

The Retrocomputing DevRoom is managed by Pau Garcia Quiles (retrocomputing-devroom-manager@fosdem.org).

A mailing list of speakers, audience and the curious is available, please subscribe at https://lists.fosdem.org/listinfo/retrocomputing-devroom

Hope to hear from you soon! And please forward this announcement.

Iain R. Learmonth: Free Software Efforts (2017W41)

16 October, 2017 - 05:00

Here’s my weekly report for week 41 of 2017. In this week I have explored some Java 8 features, looked at automatic updates in a few Linux distributions and decided that actually I don’t need swap anymore.

Debian

The issue that was preventing the migration of the Tasktools Packaging Team’s mailing list from Alioth to Savannah has now been resolved.

Ana’s chkservice package that I sponsored last week has been ACCEPTED into unstable and since MIGRATED to testing.

Tor Project

I have produced a patch for the Tor Project website to update links to the Onionoo documentation now this has moved (#23802 ). I’ve updated the Debian and Ubuntu relay configuration instructions to use systemctl instead of service where appropriate (#23048 ).

When a Tor relay is less than 2 years old, an alert will now appear on Atlas to link to the new relay lifecycle blog post (#23767 ). This should hopefully help new relay operators understand why their relay is not immediately fully loaded but instead it takes some time to ramp up.

I have gone through the tickets for Tor Cloud and did not find any tickets that contain any important information that would be useful to someone reviving the project. I have closed out these tickets and the Tor Cloud component no longer has any non-closed tickets (#7763, #8544, #8768, #9064, #9751, #10282, #10637, #11153, #11502, #13391, #14035, #14036, #14073, #15821 ).

I’ve continued to work on turning the Atlas application into an integrated part of Tor Metrics (#23518 ) and you can see some progress here.

Finally, I’ve continued hacking on a Twitter bot to tweet factoids about the public Tor network and you can now enjoy some JavaDoc documentation if you’d like to learn a little about its internals. I am still waiting for a git repository to be created (#23799 ) but will be publishing the sources shortly after that ticket is actioned.

Sustainability

I believe it is important to be clear not only about the work I have already completed but also about the sustainability of this work into the future. I plan to include a short report on the current sustainability of my work in each weekly report.

I have not had any free software related expenses this week. The current funds I have available for equipment, travel and other free software expenses remains £60.52. I do not believe that any hardware I rely on is looking at imminent failure.

I’d like to thank Digital Ocean for providing me with futher credit for their platform to support my open source work.

I do not find it likely that I’ll be travelling to Cambridge for the miniDebConf as the train alone would be around £350 and hotel accomodation a further £600 (to include both me and Ana).

Norbert Preining: TeX Live Manager: JSON output

15 October, 2017 - 08:32

With the development of TLCockpit continuing, I found the need for and easy exchange format between the TeX Live Manager tlmgr and frontend programs like TLCockpit. Thus, I have implemented JSON output for the tlmgr info command.

While the format is not 100% stable – I might change some thing – I consider it pretty settled. The output of tlmgr info --data json is a JSON array with JSON objects for each package requested (default is to list all).

[ TLPackageObj, TLPackageObj, ... ]

The structure of the JSON object TLPackageObj reflects the internal Perl hash. Guaranteed to be present keys are name (String) and avilable (Boolean). In case the package is available, there are the following further keys sorted by their type:

  • String type: name, shortdesc, longdesc, category, catalogue, containerchecksum, srccontainerchecksum, doccontainerchecksum
  • Number type: revision, runsize, docsize, srcsize, containersize, srccontainersize, doccontainersize
  • Boolean type: available, installed, relocated
  • Array type: runfiles (Strings), docfiles (Strings), srcfiles (Strings), executes (Strings), depends (Strings), postactions (Strings)
  • Object type:
    • binfiles: keys are architecture names, values are arrays of strings (list of binfiles)
    • binsize: keys are architecture names, values or numbers
    • docfiledata: keys are docfile names, values are objects with optional keys details and lang
    • cataloguedata: optional keys aare topics, version, license, ctan, date, values are all strings

A rather long example showing the output for the package latex, formatted with json_pp and having the list of files and the long description shortened:

[
   {
      "installed" : true,
      "doccontainerchecksum" : "5bdfea6b85c431a0af2abc8f8df160b297ad73f6a324ca88df990f01f24611c9ae80d2f6d12c7b3767308fbe3de3fca3d11664b923ea4080fb13fd056a1d0c3d",
      "docfiles" : [
         "texmf-dist/doc/latex/base/README.txt",
         ....
         "texmf-dist/doc/latex/base/webcomp.pdf"
      ],
      "containersize" : 163892,
      "depends" : [
         "luatex",
         "pdftex",
         "latexconfig",
         "latex-fonts"
      ],
      "runsize" : 414,
      "relocated" : false,
      "doccontainersize" : 12812184,
      "srcsize" : 752,
      "revision" : 43813,
      "srcfiles" : [
         "texmf-dist/source/latex/base/alltt.dtx",
         ....
         "texmf-dist/source/latex/base/utf8ienc.dtx"
      ],
      "category" : "Package",
      "cataloguedata" : {
         "version" : "2017/01/01 PL1",
         "topics" : "format",
         "license" : "lppl1.3",
         "date" : "2017-01-25 23:33:57 +0100"
      },
      "srccontainerchecksum" : "1d145b567cf48d6ee71582a1f329fe5cf002d6259269a71d2e4a69e6e6bd65abeb92461d31d7137f3803503534282bc0c5546e5d2d1aa2604e896e607c53b041",
      "postactions" : [],
      "binsize" : {},
      "longdesc" : "LaTeX is a widely-used macro package for TeX, [...]",
      "srccontainersize" : 516036,
      "containerchecksum" : "af0ac85f89b7620eb7699c8bca6348f8913352c473af1056b7a90f28567d3f3e21d60be1f44e056107766b1dce8d87d367e7f8a82f777d565a2d4597feb24558",
      "executes" : [],
      "binfiles" : {},
      "name" : "latex",
      "catalogue" : null,
      "docsize" : 3799,
      "available" : true,
      "runfiles" : [
         "texmf-dist/makeindex/latex/gglo.ist",
         ...
         "texmf-dist/tex/latex/base/x2enc.dfu"
      ],
      "shortdesc" : "A TeX macro package that defines LaTeX"
   }
]

What is currently not available via tlmgr info and thus also not via the JSON output is access to virtual TeX Live databases with several member databases (multiple repositories). I am thinking about how to incorporate this information.

These changes are currently available in the tlcritical repository, but will enter proper TeX Live repositories soon.

Using this JSON output I will rewrite the current TLCockpit tlmgr interface to display more complete information.

Lior Kaplan: Debian Installer git repository

15 October, 2017 - 05:15

While dealing with d-i’s translation last month in FOSScamp, I was kinda surprised it’s still on SVN. While reviewing PO files from others, I couldn’t select specific parts to commit.

Debian does have a git server, and many DDs (Debian Developers) use it for their Debian work, but it’s not as public as I wish it to be. Meaning I lack the pull / merge request abilities as well as the review process.

Recently I got a reminder that the D-I’s Hebrew translation needs some love. I asked my local community for help. Receiving a PO file by mail, reminded me of the SVN annoyance. So this time I decided to convert it to git and ask people to send me pull requests. Another benefit would be making the process more transparent as others could see these PRs (and hopefully comment if needed).

For this experiment, I opened a repository on GitHub at https://github.com/kaplanlior/debian-installer I know they aren’t open source as GitLab, but they are a popular choice which is a good start for my experiment. If and when it succeeds, we can discuss the platform.

 


Filed under: Debian GNU/Linux

Petter Reinholdtsen: A one-way wall on the border?

15 October, 2017 - 03:10

I find it fascinating how many of the people being locked inside the proposed border wall between USA and Mexico support the idea. The proposal to keep Mexicans out reminds me of the propaganda twist from the East Germany government calling the wall the “Antifascist Bulwark” after erecting the Berlin Wall, claiming that the wall was erected to keep enemies from creeping into East Germany, while it was obvious to the people locked inside it that it was erected to keep the people from escaping.

Do the people in USA supporting this wall really believe it is a one way wall, only keeping people on the outside from getting in, while not keeping people in the inside from getting out?

Norbert Preining: ScalaFX: ListView with CellFactory

14 October, 2017 - 12:29

I had a bit hard time to get ScalaFX to display a list of items in a scrollable space, and each item can be clicked. I use this in TLCockpit to display the list of documentation files in a TeX Live package, and open it directly from the application. Unfortunately there is not a huge amount of examples using ScalaFX out there in the web, so it took me a bit. My first try was using a VBox with various Labels in there, but this is not scrollable.

In other areas I have used TreeTableView, so in this case using ListView should be fine. What I finally came up is the following code:

import scalafx.application.JFXApp
import scalafx.application.JFXApp.PrimaryStage
import scalafx.collections.ObservableBuffer
import scalafx.geometry.Orientation
import scalafx.scene.control.{ListCell, ListView}
import scalafx.scene.input.MouseEvent
import scalafx.scene.{Cursor, Scene}
import scalafx.scene.paint.Color
import scalafx.Includes._

object ApplicationMain extends JFXApp {

  val SomeStrings: Seq[String] = Seq("Hello", "World", "Enjoy")

  stage = new PrimaryStage {
    title = "ListViewExample"
    scene = new Scene {
      root = {
        new ListView[String] {
          orientation = Orientation.Vertical
          cellFactory = {
            p => {
              val cell = new ListCell[String]
              cell.textFill = Color.Blue
              cell.cursor = Cursor.Hand
              cell.item.onChange { (_, _, str) => cell.text = str }
              cell.onMouseClicked = { me: MouseEvent => println("Do something with " + cell.text.value) }
              cell
            }
          }
          items = ObservableBuffer(SomeStrings)
        }
      }
    }
  }
}

Some comments to the code, at least as far I understand it:

  • line 9: Importing scalafx.Includes._ seems to simplify some things, in particular the event handler routines can be written more straight forward.
  • line 20: Many more properties can be set here, for example the preferred height and max height, both of which I am using.
  • line 23: In my case a ListCell was enough for my needs (changing color, cursor, and allowing for mouse clicks), but if one needs something more complicated here it is bst to create an arbitrary object and asign it to the graphic field.
  • line 26: The essential part to actually fill the cells is the routine cell.item.onChange, which takes three arguments of which the last is the new value. It is used to update the cell text.
  • line 31: Last but not least one needs to assign some Observable to the items, in this case I use ObservableBuffer around the lost of strings.

Once managed, it doesn’t look so complicated, but took me some time.

Alex Muntada: My Free Software Activities in Jul-Sep 2017

14 October, 2017 - 01:47

If you read Planet Debian often, you’ve probably noticed a trend of Free Software activity reports at the beginning of the month. First, those reports seemed a bit unamusing and lengthy, but since I take the time to read them I’ve learnt a lot of things, and now I’m amazed at the amount of work that people are doing for Free Software. Indeed, I knew already that many people are doing lots of work. But reading those reports gives you an actual view of how much it is.

Then, I decided that I should do the same and write some kind of report since I became a Debian Developer in July. I think it’s a nice way to share your work with others and maybe inspire them as it happened to me. So I asked some of the people that have been inspiring me how do they do it. I mean, I was curious to know how they keep track of the work they do and how long it takes to write their reports. It seems that it takes quite some time, it’s mostly manual work and usually starts by the end of the month, reviewing their contributions in mailing lists, bug trackers, e-mail folders, etc.

Here I am now, writing my first report about my Free Software activities since July and until September 2017. I hope you like it:

  • Filed a bug #867068 in nm.debian.org: Cannot claim account after former SSO alioth cert expired.
  • Replied a request in private mail for becoming the maintainer for the Monero Wallet, that I declined suggesting to file an RFP.
  • Attended DebConf17 DebCamp but I missed most of Open Day and the rest of the Debian conference in Montreal.
  • Rebuilt libdbd-oracle-perl after being removed from testing to enable the transition to perl 5.26.
  • Filed bug #870872 in tracker.debian.org: Server Error (500) when using a new SSO cert.
  • Filed bug #870876 in tracker.debian.org: make subscription easier to upstreams with many packages.
  • Filed bug #871767 in lintian: [checks/cruft] use substr instead of substring in example.
  • Filed bug #871769 in reportbug: man page mentions -a instead of -A.
  • Suggested to remove libmail-sender-perl in bug #790727, since it’s been deprecated upstream.
  • Mentioned -n option for dpt-takeover in how to adopt pkg-perl manual.
  • Fixed a broken link to HCL in https://wiki.debian.org/Hardware.
  • Adopted libapache-admin-config-perl into pkg-perl team, upgraded to 0.95-1 and closed bug #615457.
  • Fixed bug #875835 in libflickr-api-perl: don’t add quote marks in SYNOPSIS.
  • Removed 50 inactive accounts from pkg-perl team in alioth as part of our annual membership ping.

Happy hacking!

 


Lisandro Damián Nicanor Pérez Meyer: Qt 4 and 5 and OpenSSL1.0 removal

13 October, 2017 - 21:29
Today we received updates on the OpenSSL 1.0 removal status:

<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=828522#206>
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859671#19>

So those removal bugs' severities will be raised to RC in aproximately a month.

We still don't have any solutions for Qt 4 or 5.

For the Qt 5 case we will probably keep the bug open until Qt 5.10 is in the archive which should bring OpenSSL 1.1 support *or* FTP masters decide to remove OpenSSL1.0. In this last case the fate will be the same as with Qt4, below.

For Qt4 we do not have patches available and there will probably be none in time (remember we do not have upstream support). That plus the fact that we are actively trying to remove it from the archive it means we will remove openssl support. This might mean that apps using Qt4:

- Might cease to work.
- Might keep working:
  - Informing their users that no SSL support is available → programmer did a good job.
  - Not informing their users that no SSL support is available and establishing connections non the less → programmer might have not done a good job.

Trying to inform users as soon as possible,

Lisandro for the Qt/KDE team.

Michal &#268;iha&#345;: Weblate 2.17

13 October, 2017 - 20:00

Weblate 2.17 has been released today. There are quite some performance improvements, improved search, improved access control settings and various other improvements.

Full list of changes:

  • Weblate by default does shallow Git clones now.
  • Improved performance when updating large translation files.
  • Added support for blocking certain emails from registration.
  • Users can now delete their own comments.
  • Added preview step to search and replace feature.
  • Client side persistence of settings in search and upload forms.
  • Extended search capabilities.
  • More fine grained per project ACL configuration.
  • Default value of BASE_DIR has been changed.
  • Added two step account removal to prevent accidental removal.
  • Project access control settings is now editable.
  • Added optional spam protection for suggestions using Akismet.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Turris, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Filed under: Debian English SUSE Weblate

Shirish Agarwal: I need to speak up now X – Economics

13 October, 2017 - 18:58

Dear all,

This would be a longish blog post (as most of mine are) compiled over days but as there is so short a time and so much to share.

I had previously thought to share beautiful photographs of Ganesh mandals taking out the procession at time of immersion of the idol or the last day of Durga Puja recent events around do not make my mood to share photos at this point in time. I may share some of them in a future blog post or two .

Before going further, I would like to offer my sympathies and condolences to people hurt and dislocated in Hurricane Irma , the 2017 Central Mexico Earthquake and lastly the most recent Las Vegas shooting as well as Hurricane Maria in Puerto Rico . I am somewhat non-plussed as to why Americans always want to name, especially hurricanes which destroy people’s lives and livelihood built over generations and why most of the hurricanes are named after women. A look at weather.com site unveiled the answer to the mystery.

Ironically (or not) I saw some of the best science coverage about Earthquakes or anything scientific reporting and analysis after a long time in mainstream newspapers in India.

On another note, I don’t understand or even expect to understand why the gunman did what he did 2 days back. Country music AFAIK is one of the most chilled-out kind of music, in some ways very similar to classical Indian singing although they are worlds apart in style of singing, renditions, artists, the way they emote etc. I seriously wish that the gunman had not been shot but caught and reasons were sought about what he did, he did. While this is certainly armchair thinking as was not at the scene of crime, but if a Mumbai Police constable could do it around a decade ago armed only with a lathi could do it, why couldn’t the American cops who probably are trained in innumerable ways to subdue people without killing them, did. While investigations are on, I suspect if he were caught just like Ajmal Kasab was caught then lot of revelations might have come up. From what is known, the gentleman was upwardly mobile i.e. he was white, rich and apparently had no reason to have beef with anybody especially a crowd swaying to some nice music, all of which makes absolutely no sense.

Indian Economy ‘Slowdown’

Anyways, back to one of the main reasons of writing this blog post. Few days back, an ex-finance Minister of India Yashwant Sinha wrote what was felt by probably millions of Indians, an Indian Express article called ‘I need to speak up now

While there have been many, many arguments made since then by various people. A simple search of ‘I need to speak up’ would lead to lead to many a result besides the one I have shared above. The only exception I have with the article is the line “Forty leading companies of the country are already facing bankruptcy proceedings. Many more are likely to follow suit.” I would not bore you but you ask any entrepreneur trying to set up shop in India i.e. ones who actually go through the processes of getting all the licenses for setting up even a small businesses as to the numerous hurdles they have to overcome and laid-back corrupt bureaucracy which they have to overcome. I could have interviewed some of my friends who had the conviction and the courage to set up shop and spent more than half a decade getting all the necessary licenses and approval to set up but it probably would be too specific for one industry or the other and would lead to the same result.

Co-incidentally, a new restaurant, leaf opened in my vicinity few weeks before. From the looks it looked like a high-brow, high-priced restaurant hence like many others I did not venture in. After a few days, they introduced south-Indian delicacies like Masala Dosa, Uttapam at prices similar to other restaurants around. So I ventured in and bought some south Indian food to consume between mum and me.

Few days later, I became friends with the owner/franchisee and I suggested (in a friendly tone) that why he doesn’t make it like a CCD play where many people including yours truly use the service to share, strategize and meet with clients.

The CCD joints usually serve coffee and snacks (which are over-priced but still run out pretty fast) but people come as they have chilled-out atmosphere and Wi-Fi access which people need for their smartphones, although the Wi-Fi part may soon become reduntant With Reliance Jio making a big play.

I also shared why he doesn’t add more variety and time (the south Indian items are time-limited) as I see/saw many empty chairs there.

Anyways, the shop-owner/franchisee shared his gross costs including salary, stocking, electricity, rent and it doesn’t pan out to be serving Rs.80/- dish (roughly a 1US dollar and 25 cents) then serving INR Rs. 400/- a dish (around 6 $USD). One round of INR 400/- + dishes make his costs for the day, around 12 tables were there. It’s when they have two full rounds of dishes costing INR 400/- or more that he actually has profits and he is predicting loss for at least 6 months to a year before he makes a rebound. He needs steady customers rather than just walk-ins that will make his business work/click. Currently his family is bearing the costs. He didn’t mention the taxes although I know apart from GST there are still some local body taxes that they will have to pay and comply with.

There are a multitude of problems for shutting a shop legally as well as they have to again renavigate the bureaucracy for the same. I have seen more than a few retailers downing their shutters for 6-8 months and then either sell it to new management, let go of the lease or simply sell the property to a competitor. The Insolvency and Bankruptcy Code is probably the first proper exit policy for large companies. So the 40 odd companies that Mr. Sinha were talking about were probably sick for a long time.

In India, there is also an additional shame of being a failed entrepreneur unlike in the west where Entrepreneurs start on their next venture. As seen from Retailing In India only 3.3% of the population or at the most 4% of the population is directly or indirectly linked with the retail trade. Most of the economy still derives its wealth from the agrarian sector which is still reeling under the pressure from demonetization which happened last year. Al jazeera surprisingly portrayed a truer picture of the effects demonetization had on common citizen than many Indian newspapers did at the time. Because of the South African Debconf, I had to resort to debit cards and hence was able to escape standing in long lines in which many a old and women perished.

It is only yesterday that the Government has acknowledged which many prominent Indians have been saying for months now, that we are in a ‘slowdown‘. Be aware of the terms being used for effect by the Prime Minister. There are two articles which outlines the troubles India is in atm. The only bright spot has been e-commerce which so far has eluded GST although the Govt. has claimed regulations to put it in check.

Indian Education System

Interestingly, Ravish Kumar has started a series on NDTV where he is showcasing how Indian education sector, especially public colleges have been left to teachers on contract basis, see the first four episodes on NDTV channel starting with the first one I have shared as a hyperlink. I apologize as the series is in Hindi as the channel is meant for Indians and is mostly limited to Northern areas of the Country (mostly) although he has been honest that it is because they lack resources to tackle the amount of information flowing to them. Ravish started the series with sharing information about the U.S. where the things are similar with some teachers needing to sleep in cars because of high-cost of living to some needing to turn to sex-work . I was shocked when I read the guardian article, that is no way to treat our teachers.I went on to read ‘How the American University was Killed‘ following the breadcrumbs along the way. Reading that it seems Indians have been following the American system playbook from the 1980’s itself. The article talks about HMO as well and that seems to have followed here as well with my own experience of hospital fees and drugs which I had to entail a few weeks/month ago.

Few years ago, when me and some of my friends had the teaching bug and we started teaching in a nearby municipal school, couple of teachers had shared that they were doing 2-3 jobs to make ends meet. I don’t know about others in my group, at least I was cynical because I thought all the teachers were permanent and they make good money only to realize now that the person was probably speaking the truth. When you have to do three jobs to make ends meet from where do you bring the passion to teach young people and that too outside the syllabus ?

Also, with this new knowledge in hindsight, I take back all my comments I made last year and the year before for the pathetic education being put up by the State. With teachers being paid pathetically/underpaid and almost 60% teachers being ad-hoc/adjunct teachers they have to find ways to have some sense of security. Most teachers are bachelors as they are poor and cannot offer any security (either male or female) and for women, after marriage it actually makes no sense for them to continue in this profession. I salute all the professors who are ad-hoc in nature and probably will never get a permanent position in their life.

I think in some way, thanx to him, that the government has chosen to give 7th pay commisson salary to teachers. While the numbers may appear be large, there are lot of questions as to how many people will actually get paid. There needs to be lot of vacancies which need to be filled quickly but don’t see any solution in the next 2-3 years as well. The Government has taken a position to use/re-hire retired teachers rather than have new young teachers as an unwritten policy. In this Digital India context how are retired teachers supposed to understand and then pass on digital concepts is beyond me when at few teacher trainings I have seen they lack even the most basic knowledge that I learnt at least a decade or two ago, the difference is that vast. I just don’t know what to say to that. My own experience with my own mother who had pretty good education in her time and probably would have made a fine business-woman if she knew that she will have a child that she would have to raise by herself alone (along with maternal grand-parents).

After watching the series/episodes and discussing the issue with my mother it was revealed that both her and my late maternal grandfather were on casual/ad-hoc basis till 20-25 years in their service in the defense sector. If Ravish were to do a series on the defense sector he probably would find the same thing there. To add to that, the defense sector is a vital component to a country’s security. If 60% of the defense staff in all defense establishments have temporary staff how do you ensure the loyalty of the people working therein. That brings to my mind ‘Ignorance is bliss’.

Software development and deployment

There is another worry that all are skirting around, the present dispensation/government’s mantra is ‘minimum government-maximum governance’ with digital technologies having all solutions which is leading to massive unemployment. Also from most of the stories/incidents I read in the newspapers it seems most software deployments done in India are done without having any system of internal checks and balances. There is no ‘lintian‘ for software to be implemented. Contracts seem to be given to big companies and there is no mention of what pre-requisities or conditions were laid down by the Government for software development and deployment and if any checks were done to ensure that the software being developed was in according to government specifications or not. Ideally this should all be in public domain so that questions can be asked and responsibility fixed if things go haywire, as currently they do not.

Software issues

As my health been not that great, I have been taking a bit more time and depth while filing bugs. #877638 is a good example. I suspect though that part of the problem might be that mate has moved to gtk3 while guake still has gtk-2 bindings. I also reported the issue upstream both in mate-panel as well as guake . I haven’t received any response from either or/and upstreams .

I also have been fiddling around with gdb to better understand the tool so I can exploit/use this tool in a better way. There are some commands within the gdb interface which seem to be interesting and hopefully I’ll try how the commands perform over days, weeks to a month. I hope we see more more action on the mate-panel/guake bug as well as move of guake to gtk+3 but that what seemingly seemed like wait for eternity seems to have done by somebody in last couple of days. As shared in the ticket there are lots of things still to do but it seems the heavy lifting has been done but seems merging will be tricky as two developers have been trying to update to gtk+3 although aichingm seems to have a leg up with his 3! branch.

Another interesting thing I saw is the below picture.

The firefox version I was using to test the site/wordpress-wp-admin was Mozilla Firefox 52.4.0 which AFAIK is a pretty recentish one and people using Debian stretch would probably be using the same version (firefox stable/LTS) rather than the more recent versions. I went to the link it linked to and it gave no indication as to why it thought my browser is out-of-date and what functionality was/is missing. I have found that wordpress support has declined quite a bit and people don’t seem to use the forums as much as they used to before.

I also filed a few bugs for qalculate. #877716 where a supposedly transitional package removes the actual application, #877717 as the software has moved its repo. to github.com as well as tickets and other things in process and lastly #877733. I had been searching for a calculator which can do currency calculations on the fly (say for e.g. doing personal budgeting for Taiwan debconf) without needing to manually enter the conversion rates and losing something in the middle. While the current version has support for some limited currencies, the new versions promise more as other people probably have more diverse needs for currency conversions (people who do long or short on oil, stocks overseas is just one example, I am sure there are many others) than simplistic mine.


Filed under: Miscellenous Tagged: #American Education System, #bug-filing, #Climate change, #Dignity, #e-commerce, #gtk+3, #gtk2, #Indian Economy 'Slowdown', #Indian Education System, #Insolvency and Bankruptcy Code, #Las Vegas shooting, #Modern Retail in India, #natural-disasters, #planet-debian, #qalculate, Ad-hoc and Adjunct Professors, wordpress.com

Norbert Preining: Scala: parse JSON into nested case classes

13 October, 2017 - 12:24

I was playing around with parsing JSON in Scala, and got spray-json as recommendation from my Senpai. My aim was parsing some nested structure like:

{ 
  "key1" : "value1",
  "key2" : { "subkey1" : "subval1", "subkey2" : "subval2" }
}

into a nested Scala case class:

case class SubClass(subkey1: String, subkey2: String)
case class MainClass(key1: String, key2: SubClass)


To achieve this one needs to define one’s own JsonProtocol that describes the parsing:

import spray.json._
object MyJsonProtocol extends DefaultJsonProtocol {
  implicit val subclassFormat = jsonFormat2(SubClass)
  implicit val mainclassFormat = jsonFormat2(MainClass)
}
import MyJsonProtocol._

Using this parsing can be done in the usual way:

val jsonstr = """
    { 
      "key1" : "value1",
      "key2" : { "subkey1" : "subval1", "subkey2" : "subval2" }
    }
    """
val jsonAst = jsonstr.parseJson
val gotit = jsonAst.convertTo[MainClass]

If the input is from a JSON array of the above arrays:

[
  { 
    "key1" : "value1",
    "key2" : { "subkey1" : "subval1", "subkey2" : "subval2" }
  },
  { 
    "key1" : "value10",
    "key2" : { "subkey1" : "subval10", "subkey2" : "subval20" }
  }
]

the only change necessary is to call

val gotit = jsonAst.convertTo[List[MainClass]]

Michal &#268;iha&#345;: Using Trezor to store cryptocurencies

13 October, 2017 - 11:00

For quite some time I have some cryptocurrencies on hold. These mostly come from times it was possible to mine Bitcoin on the CPU, but I've got some small payments recently as well.

I've been using Electrum wallet so far. It worked quite well, but with increasing Bitcoin value, I was considering having some hardware wallet for that. There are few options which you can use, but I've always preferred Trezor as that device is made by guys I know. Also it's probably device with best support out of these (at least I've heard really bad stories about Ledger support).

In the end what decided is that they are also using Weblate to translate their user interface and offered me the wallet for free in exchange. This is price you can not beat :-). Anyway the setup was really smooth and I'm now fully set up. This also made me more open to accept other cryptocurrencies which are supported by Trezor, so you can now see more options on the Weblate donations page.

Filed under: Debian English SUSE Weblate

Dirk Eddelbuettel: GitHub Streak: Round Four

13 October, 2017 - 09:45

Three years ago I referenced the Seinfeld Streak used in an earlier post of regular updates to to the Rcpp Gallery:

This is sometimes called Jerry Seinfeld's secret to productivity: Just keep at it. Don't break the streak.

and showed the first chart of GitHub streaking

And two year ago a first follow-up appeared in this post:

And a year ago we had a followup last year

And as it October 12 again, here is the new one:

Again, special thanks go to Alessandro Pezzè for the Chrome add-on GithubOriginalStreak.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Joachim Breitner: Isabelle functions: Always total, sometimes undefined

13 October, 2017 - 00:54

Often, when I mention how things work in the interactive theorem prover Isabelle/HOL to people with a strong background in functional programming (whether that means Haskell or Coq or something else), I cause confusion, especially around the issue of what is a function, are function total and what is the business with undefined. In this blog post, I want to explain some these issues, aimed at functional programmers or type theoreticians.

Note that this is not meant to be a tutorial; I will not explain how to do these things, and will focus on what they mean.

HOL is a logic of total functions

If I have a Isabelle function f :: a ⇒ b between two types a and b (the function arrow in Isabelle is ⇒, not →), then – by definition of what it means to be a function in HOL – whenever I have a value x :: a, then the expression f x (i.e. f applied to x) is a value of type b. Therefore, and without exception, every Isabelle function is total.

In particular, it cannot be that f x does not exist for some x :: a. This is a first difference from Haskell, which does have partial functions like

spin :: Maybe Integer -> Bool
spin (Just n) = spin (Just (n+1))

Here, neither the expression spin Nothing nor the expression spin (Just 42) produce a value of type Bool: The former raises an exception (“incomplete pattern match”), the latter does not terminate. Confusingly, though, both expressions have type Bool.

Because every function is total, this confusion cannot arise in Isabelle: If an expression e has type t, then it is a value of type t. This trait is shared with other total systems, including Coq.

Did you notice the emphasis I put on the word “is” here, and how I deliberately did not write “evaluates to” or “returns”? This is because of another big source for confusion:

Isabelle functions do not compute

We (i.e., functional programmers) stole the word “function” from mathematics and repurposed it1. But the word “function”, in the context of Isabelle/HOL, refers to the mathematical concept of a function, and it helps to keep that in mind.

What is the difference?

  • A function a → b in functional programming is an algorithm that, given a value of type a, calculates (returns, evaluates to) a value of type b.
  • A function a ⇒ b in math (or Isabelle/HOL) associates with each value of type a a value of type b.

For example, the following is a perfectly valid function definition in math (and HOL), but could not be a function in the programming sense:

definition foo :: "(nat ⇒ real) ⇒ real" where
  "foo seq = (if convergent seq then lim seq else 0)"

This assigns a real number to every sequence, but it does not compute it in any useful sense.

From this it follows that

Isabelle functions are specified, not defined

Consider this function definition:

fun plus :: "nat ⇒ nat ⇒ nat"  where
   "plus 0       m = m"
 | "plus (Suc n) m = Suc (plus n m)"

To a functional programmer, this reads

plus is a function that analyses its first argument. If that is 0, then it returns the second argument. Otherwise, it calls itself with the predecessor of the first argument and increases the result by one.

which is clearly a description of a computation.

But to Isabelle/HOL, the above reads

plus is a binary function on natural numbers, and it satisfies the following two equations: …

And in fact, it is not so much Isabelle/HOL that reads it this way, but rather the fun command, which is external to the Isabelle/HOL logic. The fun command analyses the given equations, constructs a non-recursive definition of plus under the hood, passes that to Isabelle/HOL and then proves that the given equations hold for plus.

One interesting consequence of this is that different specifications can lead to the same functions. In fact, if we would define plus' by recursing on the second argument, we’d obtain the the same function (i.e. plus = plus' is a theorem, and there would be no way of telling the two apart).

Termination is a property of specifications, not functions

Because a function does not evaluate, it does not make sense to ask if it terminates. The question of termination arises before the function is defined: The fun command can only construct plus in a way that the equations hold if it can find a termination proof – very much like Fixpoint in Coq.

But while the termination check of Fixpoint in Coq is a deep part of the basic logic, in Isabelle it is simply something that this particular command requires. Other commands may have other means of defining a function that do not require a termination proof.

For example, a function specification that is tail-recursive can be turned in to a function, even without a termination proof: The following definition describes a higher-order function that iterates its first argument f on the second argument x until it finds a fixpoint. It is completely polymorphic (the single quote in 'a indicates that this is a type variable):

partial_function (tailrec)
  fixpoint :: "('a ⇒ 'a) ⇒ 'a ⇒ 'a"
where
  "fixpoint f x = (if f x = x then x else fixpoint f (f x))"

We can work with this definition just fine. For example, if we instantiate f with (λx. x-1), we can prove that it will always return 0:

lemma "fixpoint (λ n . n - 1) (n::nat) = 0"
  by (induction n) (auto simp add: fixpoint.simps)

Similarly, if we have a function that works within the option monad (i.e. |Maybe| in Haskell), its specification can always be turned into a function without an explicit termination proof – here one that calculates the Collatz sequence:

partial_function (option) collatz :: "nat ⇒ nat list option"
 where "collatz n =
        (if n = 1 then Some [n]
         else if even n
           then do { ns <- collatz (n div 2);    Some (n # ns) }
           else do { ns <- collatz (3 * n + 1);  Some (n # ns)})"

Note that lists in Isabelle are finite (like in Coq, unlike in Haskell), so this function “returns” a list only if the collatz sequence eventually reaches 1.

I expect these definitions to make a Coq user very uneasy. How can fixpoint be a total function? What is fixpoint (λn. n+1)? What if we run collatz n for a n where the Collatz sequence does not reach 1?2 We will come back to that question after a little detour…

HOL is a logic of non-empty types

Another big difference between Isabelle and Coq is that in Isabelle/HOL, every types is inhabited. Just like the totality of functions, this is a very fundamental fact about what HOL defines to be a type.

Isabelle gets away with that design because in Isabelle, we do not use types for propositions (like we do in Coq), so we do not need empty types to denote false propositions.

This design has an important consequence: It allows the existence of a polymorphic expression that inhabits any type, namely

undefined :: 'a

The naming of this term alone has caused a great deal of confusion for Isabelle beginners, or in communication with users of different systems, so I implore you to not read too much into the name. In fact, you will have a better time if you think of it as arbitrary or, even better, unknown.

Since undefined can be instantiated at any type, we can instantiate it for example at bool, and we can observe an important fact: undefined is not an extra value besides the “usual ones”. It is simply some value of that type, which is demonstrated in the following lemma:

lemma "undefined = True ∨ undefined = False" by auto

In fact, if the type has only one value (such as the unit type), then we know the value of undefined for sure:

lemma "undefined = ()" by auto

It is very handy to be able to produce an expression of any type, as we will see as follows

Partial functions are just underspecified functions

For example, it allows us to translate incomplete function specifications. Consider this definition, Isabelle’s equivalent of Haskell’s partial fromJust function:

fun fromSome :: "'a option ⇒ 'a" where
  "fromSome (Some x) = x"

This definition is accepted by fun (albeit with a warning), and the generated function fromSome behaves exactly as specified: when applied to Some x, it is x. The term fromSome None is also a value of type 'a, we just do not know which one it is, as the specification does not address that.

So fromSome None behaves just like undefined above, i.e. we can prove

lemma "fromSome None = False ∨ fromSome None = True" by auto

Here is a small exercise for you: Can you come up with an explanation for the following lemma:

fun constOrId :: "bool ⇒ bool" where
  "constOrId True = True"

lemma "constOrId = (λ_.True) ∨ constOrId = (λx. x)"
  by (metis (full_types) constOrId.simps)

Overall, this behavior makes sense if we remember that function “definitions” in Isabelle/HOL are not really definitions, but rather specifications. And a partial function “definition” is simply a underspecification. The resulting function is simply any function hat fulfills the specification, and the two lemmas above underline that observation.

Nonterminating functions are also just underspecified

Let us return to the puzzle posed by fixpoint above. Clearly, the function – seen as a functional program – is not total: When passed the argument (λn. n + 1) or (λb. ¬b) it will loop forever trying to find a fixed point.

But Isabelle functions are not functional programs, and the definitions are just specifications. What does the specification say about the case when f has no fixed-point? It states that the equation fixpoint f x = fixpoint f (f x) holds. And this equation has a solution, for example fixpoint f _ = undefined.

Or more concretely: The specification of the fixpoint function states that fixpoint (λb. ¬b) True = fixpoint (λb. ¬b) False has to hold, but it does not specify which particular value (True or False) it should denote – any is fine.

Not all function specifications are ok

At this point you might wonder: Can I just specify any equations for a function f and get a function out of that? But rest assured: That is not the case. For example, no Isabelle command allows you define a function bogus :: () ⇒ nat with the equation bogus () = S (bogus ()), because this equation does not have a solution.

We can actually prove that such a function cannot exist:

lemma no_bogus: "∄ bogus. bogus () = Suc (bogus ())" by simp

(Of course, not_bogus () = not_bogus () is just fine…)

You cannot reason about partiality in Isabelle

We have seen that there are many ways to define functions that one might consider “partial”. Given a function, can we prove that it is not “partial” in that sense?

Unfortunately, but unavoidably, no: Since undefined is not a separate, recognizable value, but rather simply an unknown one, there is no way of stating that “A function result is not undefined”.

Here is an example that demonstrates this: Two “partial” functions (one with not all cases specified, the other one with a self-referential specification) are indistinguishable from the total variant:

fun partial1 :: "bool ⇒ unit" where
  "partial1 True = ()"
partial_function (tailrec) partial2 :: "bool ⇒ unit" where
  "partial2 b = partial2 b"
fun total :: "bool ⇒ unit" where
  "total True = ()"
| "total False = ()"

lemma "partial1 = total ∧ partial2 = total" by auto

If you really do want to reason about partiality of functional programs in Isabelle, you should consider implementing them not as plain HOL functions, but rather use HOLCF, where you can give equational specifications of functional programs and obtain continuous functions between domains. In that setting, ⊥ ≠ () and partial2 = ⊥ ≠ total. We have done that to verify some of HLint’s equations.

You can still compute with Isabelle functions

I hope by this point, I have not scared away anyone who wants to use Isabelle for functional programming, and in fact, you can use it for that. If the equations that you pass to `fun are a reasonable definition for a function (in the programming sense), then these equations, used as rewriting rules, will allow you to “compute” that function quite like you would in Coq or Haskell.

Moreover, Isabelle supports code extraction: You can take the equations of your Isabelle functions and have them expored into Ocaml, Haskell, Scala or Standard ML. See Concon for a conference management system with confidentially verified in Isabelle.

While these usually are the equations you defined the function with, they don't have to: You can declare other proved equations to be used for code extraction, e.g. to refine your elegant definitions to performant ones.

Like with code extraction from Coq to, say, Haskell, the adequacy of the translations rests on a “moral reasoning” foundation. Unlike extraction from Coq, where you have an (unformalized) guarantee that the resulting Haskell code is terminating, you do not get that guarantee from Isabelle. Conversely, this allows you do reason about and extract non-terminating programs, like fixpoint, which is not possible in Coq.

Conclusion

We have seen how in Isabelle/HOL, every function is total. Function declarations have equations, but these do not define the function in an computational sense, but rather specify them. Because in HOL, there are no empty types, many specifications that appear partial (incomplete patterns, non-terminating recursion) have solutions in the space of total functions. Partiality in the specification is no longer visible in the final product.

PS: Axiom undefined in Coq

This section is speculative, and an invitation for discussion.

Coq already distinguishes between types used in programs (Set) and types used in proofs Prop.

Could Coq ensure that every t : Set is non-empty? I imagine this would require additional checks in the Inductive command, similar to the checks that the Isabelle command datatype has to perform3, and it would disallow Empty_set.

If so, then it would be sound to add the following axiom

Axiom undefined : forall (a : Set), a.

wouldn't it? This axiom does not have any computational meaning, but that seems to be ok for optional Coq axioms, like classical reasoning or function extensionality.

With this in place, how much of what I describe above about function definitions in Isabelle could now be done soundly in Coq. Certainly pattern matches would not have to be complete and could sport an implicit case _ ⇒ undefined. Would it “help” with non-obviously terminating functions? Would it allow a Coq command Tailrecursive that accepts any tailrecursive function without a termination check?

  1. At least we do not violate this term as much as the imperative programmers do.

  2. Let me know if you find such an n. Besides n = 0.

  3. Like fun, the constructions by datatype are not part of the logic, but create a type definition from more primitive notions that is isomorphic to the specified data type.

Dirk Eddelbuettel: RcppArmadillo 0.8.100.1.0

12 October, 2017 - 09:13

We are thrilled to announce a new big RcppArmadillo release! Conrad recently moved Armadillo to the 8.* series, with significant improvements and speed ups for sparse matrix operations, and more. See below for a brief summary.

This also required some changes at our end which Binxiang Ni provided, and Serguei Sokol improved some instantiations. We now show the new vignette Binxiang Ni wrote for his GSoC contribution, and I converted it (and the other main vignette) to using the pinp package for sleeker pdf vignettes.

This release resumes our bi-monthly CRAN release cycle. I may make interim updates available at GitHub "as needed". And this time I managed to mess up the reverse depends testing, and missed one sync() call on the way back to R---but all that is now taken care of.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language--and is widely used by (currently) 405 other packages on CRAN.

A high-level summary of changes follows.

Changes in RcppArmadillo version 0.8.100.1.0 (2017-10-05)
  • Upgraded to Armadillo release 8.100.1 (Feral Pursuits)

    • faster incremental construction of sparse matrices via element access operators

    • faster diagonal views in sparse matrices

    • expanded SpMat to save/load sparse matrices in coord format

    • expanded .save(),.load() to allow specification of datasets within HDF5 files

    • added affmul() to simplify application of affine transformations

    • warnings and errors are now printed by default to the std::cerr stream

    • added set_cerr_stream() and get_cerr_stream() to replace set_stream_err1(), set_stream_err2(), get_stream_err1(), get_stream_err2()

    • new configuration options ARMA_COUT_STREAM and ARMA_CERR_STREAM

  • Constructors for sparse matrices of types dgt, dtt amd dst now use Armadillo code for improved performance (Serguei Sokol in #175 addressing #173)

  • Sparse matrices call .sync() before accessing internal arrays (Binxiang Ni in #171)

  • The sparse matrix vignette has been converted to Rmarkdown using the pinp package, and is now correctly indexed. (#176)

Courtesy of CRANberries, there is a diffstat report. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Steve Kemp: A busy week or two

12 October, 2017 - 04:00

It feels like the past week or two has been very busy, and so I'm looking forward to my "holiday" next month.

I'm not really having a holiday of course, my wife is slowly returning to work, so I'll be taking a month of paternity leave, taking sole care of Oiva for the month of November. He's still a little angel, and now that he's reached 10 months old he's starting to get much more mobile - he's on the verge of walking, but not quite there yet. Mostly that means he wants you to hold his hands so that he can stand up, swaying back and forth before the inevitable collapse.

Beyond spending most of my evenings taking care of him, from the moment I return from work to his bedtime (around 7:30PM), I've made the Debian Administration website both read-only and much simpler. In the past that site was powered by a lot of servers, I think around 11. Now it has only a small number of machines, which should slowly decrease.

I've ripped out the database host, the redis host, the events-server, the planet-machine, the email-box, etc. Now we have a much simpler setup:

  • Front-end machine
    • Directly serves the code site
    • Directly serves the SSL site which exists solely for Let's Encrypt
    • Runs HAProxy to route the rest of the requests to the cluster.
  • 4 x Apache servers
    • Each one has a (read-only) MySQL database on it for the content.
      • In case of future-compromise I removed all user passwords, and scrambled the email-addresses.
      • I don't think there's a huge risk, but better safe than sorry.
    • Each one runs the web-application.
      • Which now caches each generated page to /tmp/x/x/x/x/$hash if it doesn't exist.
      • If the request is cached it is served from that cache rather than dynamically.

Finally although I'm slowly making progress with "radio stuff" I've knocked up a simple hack which uses an ultrasonic sensor to determine whether I'm sat in front of my (home) PC. If I am everything is good. If I'm absent the music is stopped and the screen locked. Kinda neat.

(Simple ESP8266 device wired to the sensor. When the state changes a message is posted to Mosquitto, where a listener reacts to the change(s).)

Oh, not final. I've also transfered my mobile phone from DNA.fi to MoiMobile. Which should complete soon, right now my phone is in limbo, active on niether service. Oops.

Michal &#268;iha&#345;: New projects on Hosted Weblate

11 October, 2017 - 23:00

Hosted Weblate provides also free hosting for free software projects. The hosting requests queue has grown too long, so it's time to process it and include new project.

This time, the newly hosted projects include:

  • Hunspell - famous spell checker
  • Eolie - a web browser for GNOME
  • SkyTube - an open-source YouTube app for Android
  • Eventum - issue tracking system

Additionally there were some notable additions to existing projects:

If you want to support this effort, please donate to Weblate, especially recurring donations are welcome to make this service alive. You can do that easily on Liberapay or Bountysource.

Filed under: Debian English SUSE Weblate

Carl Chenet: The Slack Threat

11 October, 2017 - 05:00

During a long era, electronic mail was the main communication tool for enterprises. Slack, which offer public or private group discussion boards and instant messaging between two people, challenge its position, especially in the IT industry.

Not only Slack has features known and used since IRC launch in the late ’80s, but Slack also offers file sending and sharing, code quoting, and it indexing for ulterior searches everything that goes through the application. Slack is also modular with numerous plug-in to easily add new features.

Using the Software-As-A-Service (SAAS) model, Slack basic version is free, and users pay for options. Slack is now considered by the Github generation like the new main enterprise communication tool.

As I did in my previous article on the Github threat, this one won’t promote Slask advantages, as many other articles have already covered all these points ad nauseam, but to show the other side and to warn the companies using this service about its inherent risks. So far, these risks have been ignored, sometimes voluntary in the name of the “It works™” ideology. Neglecting all economic and safety consideration, neglecting all threat to privacy and individual freedom. We’ll see about them below.

Github, a software forge as a SAAS, with all the advantage but also all the risk of its economic model

All your company communication since its creation

When a start-up chooses Slack, all of its internal communication will be stored by Slack. When someone uses this service, the simple fact to chat through it means that the whole communication is archived.

One may point that within the basic Slack offer, only the last 10.000 messages can be read and searched. Bad argument. Slack stored every message and every file shared as it pleases. We’ll see below this application behavior is of capital importance in the Slack threat to enterprises.

And the problem is the same for all other companies which choose Slack at one point or another. If they replace their traditional communication method with it, Slack will have access to capital data, not only in volume, but also because of their value for the company itself… Or anyone interested in this company life.

Search Your Entire Archive

One of the main arguments to use Slack is its “Search your entire archive” feature. One can search almost anything one can think of. Why? Because everything is indexed. Your team chat archive or the more or less confidential documents exchanged with the accountant department; everything is in it in order to provide the most effective search tool.

The search bar, well-known by Slack users

We can’t deny it’s a very attractive feature for everyone inside the company. But it is also a very attractive feature for everyone outside of the company who would want to know more about its internal life. Even more if you’re looking for a specific subject.

If Slack is the main communication tool of your company, and if as I’ve experienced in my professional life, some teams prefer to use it than to go to the office next door or even bugger you to put the information on the dedicated channel, one can easily deduce that nothing—in this type of company—escape Slack. The automatic indexation and the search feature efficiency are excellent tools to get all the information needed, in quantity and in quality.

As such, it’s a great social engineering tool for everyone who has access to it, with a history as old as the use of Slack as a communication tool in the company.

Across borders… And Beyond!

Slack is a Web service which uses mainly Amazon Web services and most specially Cloudfront, as stated by the available information on Slack infrastructure.

Even without a complete study of said infrastructure, it’s easy to state that all the data regarding many innovative global companies around the world (and some of them including for all their internal communication since their creation) are located in the United States, or at least in the hands of a US company, which must follow US laws, a country with a well-known history of large scale industrial espionage, as the whistleblower Edward Snowden demonstrated it in 2013 and where company data access has no restriction under the Patriot Act, as in the Microsoft case (2014) where data stored in Ireland by the Redmond software editor have been given to US authorities.

Edward Snowden, an individual—and corporate—freedom fighter

As such, Slack’s automatic indexation and search tool are a boon for anyone—spy agency or hacker—which get authorized access to it.

To trust a third party with all, or at least most of, your internal corporate communication is a certain risk for your company if the said third party doesn’t follow the same regulations as yours or if it has different interests, from a data security point of view or more globally on its competitiveness. A badly timed data leak can be catastrophic.

What’s the point of secretly preparing a new product launch or an aggressive takeover if all your recent Slack conversations have leaked, including your secret plans?

What if… Slack is hacked?

First let’s remember that even if a cyber attack may appear as a rare or hypothetical scenario to a badly informed and hurried manager, it is far from being as rare as she or he believes it (or wants to believe it).

Infrastructure hacking is quite common, as a regular visit to Hacker News will give you multiple evidence. And Slack itself has already been hacked.

February 2015: Slack is the victim during four days of a cyber attack, which was made public by the company in March. Officially, the unauthorized access was limited to information on the users’ profiles. It is impossible to measure exactly what and who was impacted by this attack. In a recent announcement, Yahoo confessed that these 3 billion accounts (you’ve read well: 3 billions) were compromised … late 2014!

Yahoo, the company which suffered the largest recorded cyberattack regarding the compromised account numbers

Officially, Slack stated that “No financial or payment information was accessed or compromised in this attack.” Which is, and by far, the least interesting of all data stored within Slack! With company internal communication indexed—sometimes from the very beginning of said company—and searchable, Slack may be a potential target for cybercriminal not looking for its users’ financial credentials but more their internal data already in a usable format. One can imagine Slack must give information on a massive data leak, which can’t be ignored. But what would happen if only one Slack user is the victim of said leak?

The Free Alternative Solutions

As we demonstrated above, companies need to find an alternative solution to Slack, one they can host themselves to reduce data leaks and industrial espionage and dependency on the Internet connection. Luckily, Slack success created its own copycats, some of them being also free software.

Rocket.chat is one of them. Its comprehensive service offers chat rooms, direct messages and file sharing but also videoconferencing and screen sharing, and even most features. Check their dedicated page. You can also try an online demo. And even more, Rocket Chat has a very simple extension system and an API.

Mattermost is another service which has the advantages of proximity and of compatibility with Slack. It offers numerous features including the main expected by this type of software. It also offers numerous apps and plug-ins to interact with online services, software forges, and continuous integration tools.

It works

In the introduction, we discussed the “It works™” effect, usually invoked to dispel any arguments about data protection and exchange confidentiality we discussed in this article. True, one single developer can ask: why worry about it? All I want is to chat with my colleagues and send files!

Because Slack service subscription in the long term put the company continuously at risk. Maybe it’s not the employees’ place to worry about it, they just have to do their job the more efficiently possible. On the other side, the company management, usually non-technical, may not be aware of what risks will threaten their company with this technical choice. The technical management may pretend to be omniscient, nobody is fooled.

Either someone from the direction will ask the right question (where are our data and who can access them?) or someone from the technical side alert them officially on these problems. This is this technical audience, even if not always heard by their direction, which is the target of this article. May they find in it the right arguments to be convincing.

We hope that the several points we developed in this article will help you to make the right choice.

About Me

Carl Chenet, Free Software Indie Hacker, founder of the French-speaking Hacker News-like Journal du hacker.

Follow me on social networks

Translated from French by Stéphanie Chaptal. Original article written in October 2016.

 

Yves-Alexis Perez: OpenPGP smartcard transition (part 1)

11 October, 2017 - 03:44

A long time ago, I switched my GnuPG setup to a smartcard based one. I kept using the same master key, but:

  • copied the rsa4096 master key to a “master” smartcard, for when I need to sign (certify) other keys;
  • created rsa2048 subkeys (for signature, encryption and authentication) and moved them to an OpenPGP smartcard for daily usage.

I've been working with that setup for a few years now and it is working perfectly fine. The signature counter on the OpenPGP basic card is a bit north of 5000 which is large but not that huge, all considered (and not counting authentication and decryption key usage).

One very nice feature of using a smartcard is that my laptop (or other machines I work on) never manipulates the private key directly but only sends request to the card, which is a really huge improvement, in my opinion. But it's also not the perfect solution for me: the OpenPGP card uses a proprietary platform from ZeitControl, named BasicCard. We have very few information on the smartcard, besides the fact that Werner Koch trust ZeistControl to not mess up. One caveat for me is that the card does not use a certified secure microcontroler like you would find in smartcard chips found in debit card or electronic IDs. That means it's not really been audited by a competent hardware lab, and thus can't be considered secure against physical attacks. The cardOS software and the application implementing the OpenPGP specification are not public either and have not been audited either, to the best of my knowledge.

At one point I was interested in the Yubikey Neo, especially since the architecture Yubico used was common: a (supposedly) certified platform (secure microcontroler, card OS) and a GlobalPlatform / JavaCard virtual machine. The applet used in the Yubikey Neo is open-source, too, so you could take a look at it and identify any issue.

Unfortunately, Yubico transitioned to a less common and more proprietary infrastructure for Yubikey 4: it's not longer Javacard based, and they don't provide the applet source anymore. This was not really seen as a good move by a lot of people, including Konstantin Ryabitsev (kernel.org administrator). Also, it wasn't possible  even for the Yubico Neo to actually build the applet yourself and inject it on the card: when the Yubikey leaves the facility, the applet is already installed and the smartcard is locked (for obvious security reason). I've tried asking about getting naked/empty Yubikey with developers keys to load the applet myself, but it' was apparently not possible or would have required signing an NDA with NXP (the chip maker), which is not really possible as an individual (not that I really want to anyway).

In the meantime, a coworker actually wrote an OpenPGP javacard applet, with the intention to support latest version of the OpenPGP specification, and especially elliptic curve cryptography. The applet is called SmartPGP and has been released on ANSSI github repository. I investigated a bit, and found a smartcard with correct specification: certified (in France or Germany), and supporting Javacard 3.0.4 (required for ECC). The card can do RSA2048 (unfortunately not RSA4096) and EC with NIST (secp256r1, secp384r1, secp521r1) and Brainpool (P256, P384, P512) curves.

I've ordered some cards, and when they arrived started playing. I've built the SmartPGP applet and pushed it to a smartcard, then generated some keys and tried with GnuPG. I'm right now in the process of migrating to a new smartcard based on that setup, which seems to work just fine after few days.

Part two of this serie will describe how to build the applet and inject it in the smartcard. The process is already documented here and there, but there are few things not to forget, like how to lock the card after provisionning, so I guess having the complete process somewhere might be useful in case some people want to reproduce it.

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้