Planet Debian

Subscribe to Planet Debian feed
Planet Debian - http://planet.debian.org/
Updated: 2 hours 4 min ago

Petter Reinholdtsen: Simpler recipe on how to make a simple $7 IMSI Catcher using Debian

10 August, 2017 - 04:59

On friday, I came across an interesting article in the Norwegian web based ICT news magazine digi.no on how to collect the IMSI numbers of nearby cell phones using the cheap DVB-T software defined radios. The article refered to instructions and a recipe by Keld Norman on Youtube on how to make a simple $7 IMSI Catcher, and I decided to test them out.

The instructions said to use Ubuntu, install pip using apt (to bypass apt), use pip to install pybombs (to bypass both apt and pip), and the ask pybombs to fetch and build everything you need from scratch. I wanted to see if I could do the same on the most recent Debian packages, but this did not work because pybombs tried to build stuff that no longer build with the most recent openssl library or some other version skew problem. While trying to get this recipe working, I learned that the apt->pip->pybombs route was a long detour, and the only piece of software dependency missing in Debian was the gr-gsm package. I also found out that the lead upstream developer of gr-gsm (the name stand for GNU Radio GSM) project already had a set of Debian packages provided in an Ubuntu PPA repository. All I needed to do was to dget the Debian source package and built it.

The IMSI collector is a python script listening for packages on the loopback network device and printing to the terminal some specific GSM packages with IMSI numbers in them. The code is fairly short and easy to understand. The reason this work is because gr-gsm include a tool to read GSM data from a software defined radio like a DVB-T USB stick and other software defined radios, decode them and inject them into a network device on your Linux machine (using the loopback device by default). This proved to work just fine, and I've been testing the collector for a few days now.

The updated and simpler recipe is thus to

  1. start with a Debian machine running Stretch or newer,
  2. build and install the gr-gsm package available from http://ppa.launchpad.net/ptrkrysik/gr-gsm/ubuntu/pool/main/g/gr-gsm/,
  3. clone the git repostory from https://github.com/Oros42/IMSI-catcher,
  4. run grgsm_livemon and adjust the frequency until the terminal where it was started is filled with a stream of text (meaning you found a GSM station).
  5. go into the IMSI-catcher directory and run 'sudo python simple_IMSI-catcher.py' to extract the IMSI numbers.

To make it even easier in the future to get this sniffer up and running, I decided to package the gr-gsm project for Debian (WNPP #871055), and the package was uploaded into the NEW queue today. Luckily the gnuradio maintainer has promised to help me, as I do not know much about gnuradio stuff yet.

I doubt this "IMSI cacher" is anywhere near as powerfull as commercial tools like The Spy Phone Portable IMSI / IMEI Catcher or the Harris Stingray, but I hope the existance of cheap alternatives can make more people realise how their whereabouts when carrying a cell phone is easily tracked. Seeing the data flow on the screen, realizing that I live close to a police station and knowing that the police is also wearing cell phones, I wonder how hard it would be for criminals to track the position of the police officers to discover when there are police near by, or for foreign military forces to track the location of the Norwegian military forces, or for anyone to track the location of government officials...

It is worth noting that the data reported by the IMSI-catcher script mentioned above is only a fraction of the data broadcasted on the GSM network. It will only collect one frequency at the time, while a typical phone will be using several frequencies, and not all phones will be using the frequencies tracked by the grgsm_livemod program. Also, there is a lot of radio chatter being ignored by the simple_IMSI-catcher script, which would be collected by extending the parser code. I wonder if gr-gsm can be set up to listen to more than one frequency?

Simon McVittie: DebConf 17: Flatpak and Debian

10 August, 2017 - 00:06
The indoor garden at Collège de Maisonneuve, the DebConf 17 venue

I'm currently at DebConf 17 in Montreal, back at DebConf for the first time in 10 years (last time was DebConf 7 in Edinburgh). It's great to put names to faces and meet more of my co-developers in person!

On Monday I gave a talk entitled “A Debian maintainer's guide to Flatpak”, aiming to introduce Debian developers to Flatpak, and show how Flatpak and Debian (and Debian derivatives like SteamOS) can help each other. It seems to have been quite well received, with people generally positive about the idea of using Flatpak to deliver backports and faster-moving leaf packages (games!) onto the stable base platform that Debian is so good at providing.

I've now put up my slides in the DebConf git-annex repository, with some small edits to link to more source code: A Debian maintainer's guide to Flatpak. Source code for the slides is also available from Collabora's git server.

The next step is to take my proof-of-concept for building Flatpak runtimes and apps from Debian and SteamOS packages, flatdeb, get it a bit more production-ready, and perhaps start publishing some sample runtimes from a cron job on a Debian or Collabora server. (By the way, if you downloaded that source right after my talk, please update - I've now pushed some late changes that were necessary to fix the 3D drivers for my OpenArena demo.)

I don't think Debian will be going quite as far as Endless any time soon: as Cosimo outlined in the talk right before mine, they deploy their Debian derivative as an immutable base OS with libOSTree, with all the user-installable modules above that coming from Flatpak. That model is certainly an interesting thing to think about for Debian derivatives, though: at Collabora we work on a lot of appliance-like embedded Debian derivatives, with a lot of flexibility during development but very limited state on deployed systems, and Endless' approach seems a perfect fit for those situations.

Joey Hess: unifying OS installation and configuration management

9 August, 2017 - 22:57

Three years ago, I realized that propellor (my configuration management system that is configured using haskell) could be used as an installer for Debian (or other versions of Linux). In propellor is d-i 2.0, I guessed it would take "a month and adding a few thousand lines of code".

I've now taken that month, and written that code, and I presented the result at DebConf yesterday. I demoed propellor building a live Debian installation image, and then handed it off to a volenteer from the audience to play with its visual user interface and perform the installation. The whole demo took around 20 minutes, and ended with a standard Debian desktop installation.

The core idea is to reuse the same configuration management system for several different purposes.

  1. Building a bootable disk image that can be used as both a live system and as an OS installer.
  2. Running on that live system, to install the target system. Which can just involve copying the live system to the target disk and then letting the configuration management system make the necessary changes to get from the live system configuration to the target system configuration.
  3. To support such things as headless arm boards, building customized images tuned for the target board and use case, that can then simply be copied to the board to install.
  4. Optionally, running on the installed system later, to futher customize it. Starting from the same configuration that produced the installed system in the first place.

There can be enourmous code reuse here, and improvements made for one of those will often benefit all the rest as well.

Once everything is handled by configuration management, all user interface requirements become just a matter of editing the configuration. Including:

  • A user interface that runs on the live system and gets whatever input is needed to install to the target system. This is really just a config editor underneath. I built a prototype gamified interface that's as minimal as such an interface could get.
  • With a regular text editor, of course. This is the equivilant of preseeding in d-i, giving advanced users full control over the system that gets built. Unlike with preseeding, users have the full power of a configuration management system, so can specify precisely the system they want installed.
  • A separate user interface for customizing disk images, for arm boards and similar use cases. This would run on a server, or on the user's own laptop.

That's the gist of it. Configuration management reused for installation and image building, and multiple editor interfaces to make it widely usable.

I was glad, sitting in to a BoF session before my talk, that several people in Debian are already thinking along similar lines. And if Debian wanted to take this work and run with it, I'd be glad to assist as propellor's maintainer. But the idea is more important than the code and I hope my elaboration of it helps point a way if not the way.

While what I've built installs Debian, little of it is Debian-specific. It would probably be easy to port it to Arch Linux, which propellor already supports. There are Linux-specific parts, so porting to FreeBSD would be harder, but propellor knows, at the type level which OSs properties support, which will ease porting.

GuixSD and NixOS already use configuration management for installation, and were part of my inspiration. I've extended what they do in some ways (in other ways they remain far ahead).

The code is here. And here are some links to more details about what I built, and ideas encountered along the way:

Junichi Uekawa: reading up on rapidjson.

9 August, 2017 - 07:14
reading up on rapidjson. I was reading the docs for rapidjson performance and I like that source buffer is destroyed for performance. I was wrinting JSON parser myself and performance bottleneck seems to be copying and constructing objects.

Jonathan Dowland: libraries

9 August, 2017 - 03:50

Cover for The Rise Of The Meritocracy

At some point during my Undergraduate years I lost the habit of using Libraries. On reflection this is probably Amazon's fault. In recent years I've tried to get back into the habit of using them.

Using libraries is a great idea if you are trying to lead a more minimalist life. I am registered to use Libraries in two counties: North Tyneside, where I live, and Newcastle, where I work. The union of the two counties' catalogues is pretty extensive. Perhaps surprisingly I have found North Tyneside to offer both better customer service and a more interesting selection of books.

Sometimes there are still things that are hard to get ahold of. After listening to BBC Radio 4's documentary The Rise and Fall of Meritocracy, presented by Toby Young, I became interested in reading The Rise of the Meritocracy: an alarmist, speculative essay that coined the term meritocracy, written by Toby's father, Michael Young.

The book was not on either catalogue. It is out of print, with the price of second hand copies fluctuating but generally higher than I am prepared to pay. I finally managed to find a copy in Newcastle University's Library. As an associate of the School of Computing I have access to the Library services.

It's an interesting read, and I think if it were framed more as a novel than as an essay it might be remembered in the same bracket as Brave New World or 1984.

Wouter Verhelst: DebConf17 first videos published

8 August, 2017 - 19:11

Due to some technical issues, it took a slight bit longer than I'd originally expected; but the first four videos of the currently running DebConf 17 conference are available. Filenames are based on the talk title, so that should be reasonably easy to understand. I will probably add an RSS feed (like we've done for DebConf 16) to that place some time soon as well, but code for that still needs to be written.

Meanwhile, we're a bit behind on the reviewing front, with (currently) 34 talks still needing review. If you're interested in helping out, please join the #debconf-video channel on OFTC and ask what you can do. This is something which you can do from home if you're interested, so don't be shy! We'd be happy for your help.

Ben Hutchings: Debian LTS work, July 2017

8 August, 2017 - 05:35

I was assigned 15 hours of work by Freexian's Debian LTS initiative and worked 14 hours. I will carry over 1 hour to the next month.

I prepared and released an update on the Linux 3.2 longterm stable branch (3.2.91), and started work on the next update. However, I didn't make any uploads to Debian this month.

Raphaël Hertzog: My Free Software Activities in July 2017

7 August, 2017 - 21:49

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donors (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it’s one of the best ways to find volunteers to work with me on projects that matter to me.

Debian LTS

This month I was allocated 12 hours but I only managed to work for 7 hours (due to vacation and unanticipated customer work). I gave back the remaining hours to the pool as I didn’t want to carry them over for August which will be also short due to vacation (BTW I’m not attending Debconf). I spent my 7 hours doing CVE triaging during the week where I was in charge of the LTS frontdesk (I committed 22 updates to the security tracker). I did publish DLA-1010-1 on vorbis-tools but the package update had been prepared by Petter Reinholdtsen.

Misc Debian work

zim. I published an updated package in experimental (0.67~rc2-2) with the upstream bug fixes on the current release candidate. The final version has been released during my vacation and I will soon upload it to unstable.

Debian Handbook. I worked with Petter Reinholdtsen to finalize the paperback version of the Norwegian translation of the Debian Administrator’s Handbook (still covering Debian 8 Jessie). It’s now available.

Bug reports. I filed a few bugs related to my Kali work. #868678: autopkgtest’s setup-testbed script is not friendly to derivatives. #868749: aideinit fails with syntax errors when /etc/debian_version contains spaces.

debian-installer. I submitted a few d-i patches that I prepared for a customer who had some specific needs (using the hd-media image to boot the installer from an ISO stored in an LVM logical volume). I made changes to debian-installer-utils (#868848), debian-installer (#868852), and iso-scan (#868859, #868900).

Thanks

See you next month for a new summary of my activities.

No comment | Liked this article? Click here. | My blog is Flattr-enabled.

Gunnar Wolf: #DebConf17, Montreal • An evening out

7 August, 2017 - 18:49

I have been in Montreal only for a day. Yesterday night, I left DebConf just after I finished presenting the Continuous Key-Signing Party introduction to go out with a long-time friend from Mexico and his family. We went to the Mont Royal park, from where you can have a beautiful city view:

What I was most amazed of as a Mexico City dweller is of the sky, of the air... Not just in this picture, but as we arrived, or later when a full moon rose. This city has beautiful air, and a very beautiful view. We later went for dinner to a place I heartfully recommend to other non-vegetarian attendees:

Portuguese-style grill. Delicious. Of course, were I to go past it, I'd just drive on (as it had a very long queue waiting to enter). The secret: Do your request on the phone. Make a short queue to pick it up. Have somebody in the group wait for a table, or eat at the nearby Parc Lafontaine. And... Thoroughly enjoy :-)

Anyway, I'm leaving for the venue, about to use the Bixi service for the first time. See you guys soon! (if you are at DebConf17, of course. And you should all be here!)

AttachmentSize Montreal1.jpeg112.83 KB Montreal2.jpeg118.2 KB Poule.jpeg118.85 KB

Mario Lang: If your software should be cross platform and accessible, forget about Qt

7 August, 2017 - 15:10

A few years ago, I started to write software which primary audience is going to be blind musicians. I did a small presentation of the UI at DebConf15.

Most of the functionality is in a compiler-alike backend. But eventually, I wanted to create a user interface to improve the interactive experience.

So, the problem again: which toolkit to choose which would be accessible on most platforms? Last time I needed to solve a similar problem, I used Java/Swing. This has its problems, but it actually works on Windows, Linux and (supposedly) Mac. This time around, my implementation language is C++, so Swing didn't look that interesting. It appears there is not much that fullfils these requirements. Qt looked like it could. But since I had my bad experiences already with Qt claiming accessibility they really never implemented, I was at least a bit cautious. Around 10 years ago, when Qt 4 was released, I found that the documentation claimed that Qt4 was accessible on Linux, but it really never was until a very late 4.x release. This information was a blatant lie, trying to lure uninformed programmers into using Qt, much to the disservice of their disabled users. If you ask a random blind Windows user who knows a bit about toolkits, they will readily tell you that they hate every app written in Qt.

With this knowledge, and the spirit of "We can change the world" I wrote a private mail to the person responsible for maintaining Qt accessibility. I explained to them that I am about to choose Qt as the UI platform for my program, and that my primary audience is users that rely on Accessibility. I also explained that cross-platform support (esp. good support on Windows) is a necessary requirement for my project. I basically got a nice marketing speak answer back, but when I read it back then, I didn't fully realize that just yet. The tone basicallly: "No problem. Qt works on Linux, Mac and Windows, and if you find any problems, just report them to us and we are going to fix them." Well, I was aware that I am not a paying customer of Qt Company, so the promise above is probbably a bit vague (I thought), but still, it sounded quite encouraging.

So off I went, and started to learn enough Qt to implement the simple user interface I wanted. First tests on Linux seemed to work, that is nice. After a while, I started to test on Windows. And BANG, of course, there is a "hidden" problem. The most wide-spread (commercial) screen reader used by most blind people somehow does not see the content of text entry widgets. This was and still is a major problem for my project. I have a number of text entry fields in my UI. Actually, the main part of the UI is a simple editor, so you might see the problem already.

So some more testing was done, just to realize that yes, text entry fields indeed do not work with the most widely used screen reader on Windows. While other screen readers seemed to work (NVDA) it is simply not feasable to ask my future users to switch to a different screen reader just for a single program. So I either needed to get JAWS fixed, or drop Qt.

Well, after a lot of testing, I ended up submitting a bug to the Qt tracker. That was a little over a year ago. The turnaround time of private mail was definitely faster.

And now I get a reply to my bug explaining that JAWS was never a priority, still is not, and that my problem will probably go away after a rewrite which has no deadline yet.

Why did I expect this already?

At least now I know why no blind users wants to have any Qt on their machines.

If you want to write cross-platform accessible software: You definitely should not use Qt. And no other Free Software toolkit for that matter, because they basically all dont give a shit about accessibility on non-Linux platforms. Yes, GTK has a Windows port, but that isn't accessible at all. Yes, wxWindows has a Windows port, but that has problems with, guess what, text entry fields (at least last time I checked).

Free Software is NOT about Accessibility or equality. I see evidence for that claim since more then 15 years now. It is about coolness, self-staging, scratch-your-own-itchness and things like that. When Debian released Jessie, I was told that something like Accessibility is not important enough to delay the release. If GNOME just broke all the help system by switching to not-yet-accessible webkit, that is just bad luck, I was told. But it is outside of the abilities of package maintainers to ensure that what we ship is accessible.

I hereby officially give up. And I admit my own stupidity. Sorry for claiming Free Software would be a good thing for the world. It is definitely not for my kin. If Free Software ever takes over, the blind will be unable to use their computers.

Don't get me wrong. I love my command-line. But as the well-known saying goes: "Free Software will be ready for the desktop user, perhaps, next year?"

The scratch-your-own-itch philosophy simply doesn't work together with a broad list of user requirements. If you want to support users with disabilities, you probably should not rely on hippie coders right now.

I repeat: If you want to write compliant software, that would be also useable to people with disabilities, you can not use Qt. For now, you will need to write a native UI for every platform you want to support. Oh, and do not believe Qt Company marketing texts, your users will suffer if you do.

Francois Marier: Time Synchronization with NTP and systemd

7 August, 2017 - 03:10

I recently ran into problems with generating TOTP 2-factor codes on my laptop. The fact that some of the codes would work and some wouldn't suggested a problem with time keeping on my laptop.

This was surprising since I've been running NTP for a many years and have therefore never had to think about time synchronization. After looking into this though, I realized that the move to systemd had changed how this is meant to be done.

The new systemd time synchronization daemon

On a machine running systemd, there is no need to run the full-fledged ntpd daemon anymore. The built-in systemd-timesyncd can do the basic time synchronization job just fine.

However, I noticed that the daemon wasn't actually running:

$ systemctl status systemd-timesyncd.service 
● systemd-timesyncd.service - Network Time Synchronization
   Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/systemd-timesyncd.service.d
           └─disable-with-time-daemon.conf
   Active: inactive (dead)
Condition: start condition failed at Thu 2017-08-03 21:48:13 PDT; 1 day 20h ago
     Docs: man:systemd-timesyncd.service(8)

referring instead to a mysterious "failed condition". Attempting to restart the service did provide more details though:

$ systemctl restart systemd-timesyncd.service 
$ systemctl status systemd-timesyncd.service 
● systemd-timesyncd.service - Network Time Synchronization
   Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/systemd-timesyncd.service.d
           └─disable-with-time-daemon.conf
   Active: inactive (dead)
Condition: start condition failed at Sat 2017-08-05 18:19:12 PDT; 1s ago
           └─ ConditionFileIsExecutable=!/usr/sbin/ntpd was not met
     Docs: man:systemd-timesyncd.service(8)

The above check for the presence of /usr/sbin/ntpd points to a conflict between ntpd and systemd-timesyncd. The solution of course is to remove the former before enabling the latter:

apt purge ntp
systemctl enable systemd-timesyncd.service
systemctl restart systemd-timesyncd.service
Enabling time synchronization with NTP

Once the ntp package has been removed, it is time to enable NTP support in timesyncd.

Start by choosing the NTP server pool nearest you and put it in /etc/systemd/timesyncd.conf. For example, mine reads like this:

[Time]
NTP=ca.pool.ntp.org

before restarting the daemon:

systemctl restart systemd-timesyncd.service 

That may not be enough on your machine though. To check whether or not the time has been synchronized with NTP servers, run the following:

$ timedatectl status
...
 Network time on: yes
NTP synchronized: no
 RTC in local TZ: no

If NTP is not enabled, then you can enable it by running this command:

timedatectl set-ntp true

Once that's done, everything should be in place and time should be kept correctly:

$ timedatectl status
...
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

Daniel Silverstone: STM32 and RTFM

6 August, 2017 - 23:23

I have been working with STM32 chips on-and-off for at least eight, possibly closer to nine years. About as long as ST have been touting them around. I love the STM32, and have done much with them in C. But, as my previous two posts may have hinted, I would like to start working with Rust instead of C. To that end, I have been looking with great joy at the work which Jorge Aparicio has been doing around Cortex-M3 and Rust. I've had many comments in person at Debconf, and also several people mention on Twitter, that they're glad more people are looking at this. But before I can get too much deeper into trying to write my USB stack, I need to sort a few things from what Jorge has done as demonstration work.

Okay, this is fast, but we need Ludicrous speed

All of Jorge's examples seem to leave the system clocks in a fairly default state, excepting turning on the clocks to the peripherals needed during the initialisation phase. Sadly, if we're going to be running the USB at all, we need the clocks to run a tad faster. Since my goal is to run something moderately CPU intensive on the end of the USB too, it makes sense to try and get our STM32 running at maximum clock speed. For the one I have, that's 72MHz rather than the 8MHz it starts out with. Nine times more cycles to do computing in makes a lot of sense.

As I said above, I've been doing STM32 in C a lot for many years; and fortunately I have built systems with the exact chip that's on the blue-pill before. As such, if I rummage, I can find some old C code which does what we need...

    /* Enable HSE */
    RCC_HSEConfig(RCC_HSE_ON);

    /* Wait till HSE is ready */
    HSEStartUpStatus = RCC_WaitForHSEStartUp();

    if (HSEStartUpStatus == SUCCESS)
    {
      /* Enable Prefetch Buffer */
      FLASH_PrefetchBufferCmd(FLASH_PrefetchBuffer_Enable);
      /* Flash 2 wait state */
      FLASH_SetLatency(FLASH_Latency_2);

      /* HCLK = SYSCLK */
      RCC_HCLKConfig(RCC_SYSCLK_Div1);
      /* PCLK2 = HCLK */
      RCC_PCLK2Config(RCC_HCLK_Div1);
      /* PCLK1 = HCLK/2 */
      RCC_PCLK1Config(RCC_HCLK_Div2);
      /* ADCCLK = PCLK2/6 */
      RCC_ADCCLKConfig(RCC_PCLK2_Div6);
      /* PLLCLK = 8MHz * 9 = 72 MHz */
      RCC_PLLConfig(RCC_PLLSource_HSE_Div1, RCC_PLLMul_9);

      /* Enable PLL */
      RCC_PLLCmd(ENABLE);
      /* Wait till PLL is ready */
      while (RCC_GetFlagStatus(RCC_FLAG_PLLRDY) == RESET)
      {}

      /* Select PLL as system clock source */
      RCC_SYSCLKConfig(RCC_SYSCLKSource_PLLCLK);
      /* Wait till PLL is used as system clock source */
      while (RCC_GetSYSCLKSource() != 0x08)
      {}
    }

This code, rather conveniently, uses an 8MHz external crystal so we can almost direct-port it to the blue-pill Rust code and see how we go. If you're used to the CMSIS libraries for STM32, then you won't completely recognise the above since it uses the pre-CMSIS core libraries to do its thing. Library code from 2008 and it's still good on today's STM32s providing they're in the right family :-)

A direct conversion to Rust, using Jorge's beautifully easy to work with crates made from svd2rust results in:

    fn make_go_faster(rcc: &RCC, flash: &FLASH) {
        rcc.cr.modify(|_, w| w.hseon().enabled());
        while !rcc.cr.read().hserdy().is_ready() {}
        flash.acr.modify(|_, w| w.prftbe().enabled());
        flash.acr.modify(|_, w| w.latency().two());
        rcc.cfgr.modify(|_, w| w
                        .hpre().div1()
                        .ppre2().div1()
                        .ppre1().div2()
                        // .adcpre().bits(8)
                        .pllsrc().external()
                        .pllxtpre().div1()
                        .pllmul().mul9()
        );
        rcc.cr.modify(|_, w| w.pllon().enabled());
        while rcc.cr.read().pllrdy().is_unlocked() {}
        rcc.cfgr.modify(|_,w| w.sw().pll());
        while !rcc.cfgr.read().sws().is_pll() {}
    }

Now, I've not put the comments in which were in the C code, because I'm being very lazy right now, but if you follow the two together you should be able to work it through. I don't have timeouts for the waits, and you'll notice a single comment there (I cannot set up the ADC prescaler because for some reason the SVD is missing any useful information and so the generated crate only carries an unsafe function (bits()) and I'm trying to steer clear of unsafe for now. Still, I don't need the ADC immediately, so I'm okay with this.

By using this function in the beginning of the init() function of the blinky example, I can easily demonstrate the clock is going faster since the LED blinks more quickly.

This function demonstrates just how simple it is to take bit-manipulation from the C code and turn it into (admittedly bad looking) Rust with relative ease and without any of the actual bit-twiddling. I love it.

Mess with time, and you get unexpected consequences

Sadly, when you mess with the clock tree on a microcontroller, you throw a lot of things out of whack. Not least, by adjusting the clock frequency up we end up adjusting the AHB, APB1, and APB2 clock frequencies. This has direct consequences for peripherals floating around on those busses. Fortunately Jorge thought of this and while the blue-pill crate hard-wires those frequencies to 8MHz, they are, at least, configurable in code in some sense.

If we apply the make_go_faster() function to the serial loopback example, it simply fails to work because now the bus which the USART1 peripheral is connected to (APB2) is going at a different speed from the expected power-on default of 8MHz. If you remember from the function, we did .hpre().div1() which set HCLK to 72MHz, then .ppre1().div2() which sets the APB1 bus clock to be HCLK divided by 2, and .ppre2().div1() which sets APB2 bus clock to be HCLK. This means that we'd need to alter src/lib.rs to reflect these changes in the clock frequences and in theory loopback would start working once more.

It'd be awkward to try and demonstrate all that to you since I only have a phone camera to hand, but if you own a blue-pill then you can clone Jorge's repo and have a go yourself and see that I'm not bluffing you.

With all this done, it'll be time to see if we can bring the USB peripheral in the STM32 online, and that will be the topic of my next post in this discovery series.

Joachim Breitner: Communication Failure

6 August, 2017 - 22:14

I am still far from being a professor, but I recently got a glimps of what awaits you in that role…

From: Sebastian R. <…@gmail.com>
To: joachim@cis.upenn.edu
Subject: re: Errors

I've spotted a basic error in your course on Haskell (https://www.seas.upenn.edu/~cis194/fall16/). Before I proceed, it's cool if you're not receptive to errors being indicated; I've come across a number of professors who would rather take offense than admit we're all human and thus capable of making mistakes... My goal is to find a resource that might be useful well into the future, and a good indicator of that is how responsive the author is to change.

In your introduction note you have written:

n contrast to a classical intro into Haskell, we do not start with numbers, booleans, tuples, lists and strings, but we start with pictures. These are of course library-defined (hence the input CodeWorld) and not part of “the language”. But that does not make them less interesting, and in fact, even the basic boolean type is library defined – it just happens to be the standard library.

Howeverm there is no input CodeWorld in the code above. Have you been made aware of this error earlier?

Regards, ...

Nice. I like when people learn from my lectures. The introduction is a bit werid, but ok, maybe this guy had some bad experiences.

Strangley, I don’t see a mistake in the material, so I respond:

From: Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript>
To: Sebastian R. <…@gmail.com>
Subject: Re: Errors

Dear Sebastian,

thanks for pointing out errors. But the first piece of code under “Basic Haskell” starts with

{-# LANGUAGE OverloadedStrings #-}
import CodeWorld

so I am not sure what you are referring to.

Note that these are lecture notes, so you have to imagine a lecturer editing code live on stage along with it. If you only have the notes, you might have to infer a few things.

Regards, Joachim

A while later, I receive this response:

From: Sebastian R. <…@gmail.com>
To: Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript>
Subject: Re: Errors

Greetings, Joachim.

Kindly open the lecture slides and search for "input CodeWorld" to find the error; it is not in the code, but in the paragraph that implicitly refers back to the code.

You might note that I quoted this precisely from the lectures... and so I repeat myself... this came from your lectures; they're not my words!

In contrast to a classical intro into Haskell, we do not start with numbers, booleans, tuples, lists and strings, but we start with pictures. These are of course library-defined (hence the input CodeWorld) and not part of “the language”. But that does not make them less interesting, and in fact, even the basic boolean type is library defined – it just happens to be the standard library.

This time around, I've highlighted the issue. I hope that made it easier for you to spot...

Nonetheless, I got my answer. Don't reply if you're going to fight tooth and nail about such a basic fix; it's simply a waste of both of our time. I'd rather learn from somewhere else...

On Tue, Aug 1, 2017 at 11:19 PM, Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript> wrote:

I am a bit reminded of Sean Spicer … “they’re not my words!” … but clearly I am missing something. And indeed I am: In the code snippet, I wrote – correctly – import CodeWorld, but in the text I had input CodeWorld. I probably did write LaTeX before writing the lecture notes. Well, glad to have that sorted out. I fixed the mistake and wrote back:

From: Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript>
To: Sebastian R. <…@gmail.com>
Betreff: Re: Errors

Dear Sebastian,

nobody is fighting, and I see the mistake now: The problem is not that the line is not in the code, the problem is that there is a typo in the line and I wrote “input” instead of “import”.

Thanks for the report, although you did turn it into quite a riddle… a simple “you wrote import when it should have been import” would have been a better user of both our time.

Regards, Joachim

Am Donnerstag, den 03.08.2017, 13:32 +1000 schrieb Sebastian R.:

(And it seems I now made the inverse typo, writing “import“ instead of “input”. Anyways, I did not think of this any more until a few days later, when I found this nice message in my mailbox:

From: Sebastian R. <…@gmail.com> To: Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript> Subject:: Re: Errors

a simple “you wrote import when it should have been import” would have been a better user of both our time.

We're both programmers. How about I cut ALL of the unnecessary garbage and just tell you to s/import/input/ on that last quotation (the thing immediately before this paragraph, in case you didn't know).

I blatantly quoted the error, like this:

In your introduction note you have written:

n contrast to a classical intro into Haskell, we do not start with numbers, booleans, tuples, lists and strings, but we start with pictures. These are of course library-defined (hence the input CodeWorld) and not part of “the language”. But that does not make them less interesting, and in fact, even the basic boolean type is library defined – it just happens to be the standard library.

Howeverm there is no input CodeWorld in the code above.

Since that apparently wasn't clear enough, in my second email to you I had to highlight it like so:

You might note that I quoted this precisely from the lectures... and so I repeat myself... this came from your lectures; they're not my words!

In contrast to a classical intro into Haskell, we do not start with numbers, booleans, tuples, lists and strings, but we start with pictures. These are of course library-defined (hence the input CodeWorld) and not part of “the language”. But that does not make them less interesting, and in fact, even the basic boolean type is library defined – it just happens to be the standard library.

This time around, I've highlighted the issue. I hope that made it easier for you to spot...

I'm not sure if you're memeing at me or not now, but it seems either your reading comprehension, or your logical deduction skills might be substandard. Unfortunately, there isn't much either of us can do about that, so I'm happy to accept that some people will be so stupid; after all, it's to be expected and if we don't accept that which is to be expected then we live our lives in denial.

Happy to wrap up this discusson here, Seb...

On Fri, Aug 4, 2017 at 12:22 AM, Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript> wrote:

Well, I chose to be amused by this, and I am sharing my amusement with you.

Foteini Tsiami: Internationalization, part four: localization

6 August, 2017 - 22:13

Now, I am working in the fourth part of my Outreachy project which is the localization of the just-internationalized LTSP Manager software. Specifically, I am translating every message of the application’s GUI from English to Greek (the reverse task from part 1), using the “Translations” environment of Launchpad, that my mentors pointed out to me. I am writing this post from Montreal, where I have traveled in order to attend the 18th DebConf and present my Outreachy project to the Debian community. My mentor and I are giving a joined presentation titled: LTSP Manager: how 1000+ Greek schools switched to Debian-based distributions.

Last but not least, I would like to mention that today, tο my great surprise,  when I logged in to the launchpad translating environment, I saw that a Czech translation to the LTSP Manager software had started!  The internationalization of the LTSP Manager software progresses well: it is already available in English and very soon in Greek and Czech!


Bits from Debian: DebConf17 starts today in Montreal

6 August, 2017 - 21:00

DebConf17, the 18th annual Debian Conference, is taking place in Montreal, Canada from August 6 to August 12, 2017.

Debian contributors from all over the world have come together at Collège Maisonneuve during the preceding week for DebCamp (focused on individual work and team sprints for in-person collaboration developing Debian), and the Open Day on August 5th (with presentations and workshops of interest to a wide audience).

Today the main conference starts with nearly 400 attendants and over 120 activities scheduled, including 45- and 20-minute talks and team meetings, workshops, a job fair, talks from invited speakers, as well as a variety of other events.

The full schedule at https://debconf17.debconf.org/schedule/ is updated every day, including activities planned ad-hoc by attendees during the whole conference.

If you want to engage remotely, you can follow the video streaming of the events happening in the three talk rooms: Buzz (the main auditorium), Rex, and Bo, or join the conversation about what is happening in the talk rooms: #debconf17-buzz, #debconf17-rex and #debconf17-bo, and the BoF (discussions) rooms: #debconf17-potato and #debconf17-woody (all those channels in the OFTC IRC network).

DebConf is committed to a safe and welcome environment for all participants. See the DebConf Code of Conduct and the Debian Code of Conduct for more details on this.

Debian thanks the commitment of numerous sponsors to support DebConf17, particularly our Platinum Sponsors Savoir-Faire Linux, Hewlett Packard Enterprise, and Google.

Bits from Debian: Google Platinum Sponsor of DebConf17

6 August, 2017 - 04:30

We are very pleased to announce that Google has committed support to DebConf17 as a Platinum sponsor.

Google is one of the largest technology companies in the world, providing a wide range of Internet-related services and products as online advertising technologies, search, cloud computing, software, and hardware.

Google has been supporting Debian by sponsoring DebConf since more than ten years, and at gold level since DebConf12.

With this additional commitment as Platinum Sponsor for DebConf17, Google contributes to make possible our annual conference, and directly supports the progress of Debian and Free Software helping to strengthen the community that continues to collaborate on Debian projects throughout the rest of the year.

Thank you very much Google, for your support of DebConf17!

DebConf17 is starting!

Many Debian contributors are already taking advantage of DebCamp and the Open Day to work individually or in groups developing and improving Debian. DebConf17 will officially start on August 6, 2017. Visit the DebConf17 website at https://debconf17.debconf.org to know the schedule, live streaming and other details.

Lars Wirzenius: Enabling TRIM/DISCARD on Debian, ext4, luks, and lvm

6 August, 2017 - 01:37

I realised recently that my laptop isn't set up to send TRIM or DISCARD commands to its SSD. That means the SSD firmware has a harder time doing garbage collection (see whe linked Wikipedia page for more details.)

After some searching, I found two articles by Christopher Smart: one, update. Those, plus some addition reading of documentation, and a little experimentation, allowed me to do this. Since the information is a bit scattered, here's the details, for Debian stretch, as much for my own memory as to make sure this is collected into one place.

  • Append ,discard to the fourth column on relevant lines in /etc/crypttab. For me, this means the fourth column should be luks,discard.
  • Change in /etc/lvm/lvm.conf that says issue_discards to enable it (assign 1 instead of 0).
  • Append rd.luks.options=discard to the GRUB_CMDLINE_LINUX_DEFAULT value in /etc/default/grub
  • Run sudo update-grub
  • Run sudo update-initramfs -u
  • Reboot.
  • Run sudo fstrim -av - if this works, you're good! If it gives you errors, then you get to debug. I have no idea what I'm talking about.
  • Copy /usr/share/doc/util-linux/examples/fstrim.* to /etc/systemd/system and run sudo systemctl enable fstrim.timer. This will tell systemd to run fstrim every week. (If you don't use systemd you'll have to adapt the systemd bits mentioned here. I've no idea how.)

Note that it seems to be a possible information leak to TRIM encryped devices. I don't know the details, but if that bothers you, don't do it.

I don't know of any harmful effects for enabling TRIM for everything, except the crypto bit above, so I wonder if it wouldn't make sense for the Debian installer to do this by default.

Daniel Silverstone: USB Device Stacks, on RTFM, part 2

5 August, 2017 - 23:08

Previously we talked about all the different kinds of descriptors which USB devices use to communicate their capability. This is important stuff because to write any useful USB device firmware we need to be able to determine how to populate our descriptors. However, having that data on the device is entirely worthless without an understanding of how it gets from the device to the host so that it can be acted upon. To understand that, let's look at the USB wire protocol.

Note, I'll again be talking mostly about USB2.0 low- and full-speed. I believe that high speed is approximately the same but with faster wires, except not quite that simple.

Down to the wire

I don't intend to talk about the actual electrical signalling, though it's not un-reasonable for you to know that USB is a pair of wires forming a differentially signalled bidirectional serial communications link. The host is responsible for managing all the framing and timing on the link, and for formatting the communications into packets.

There are a number of packet types which can appear on the USB link:

Packet type Purpose Token Packet When the host wishes to send a message to the Control endpoint to configure the device, read data IN, or write data OUT, it uses this to start the transaction. Data(0/1) Packet Following a Setup, In, or Out token, a Data packet is a transfer of data (in either direction). The 0 and 1 alternate to provide a measure of confidence against lost packets. Handshake Packet Following a data packet of some kind, the other end may ACK the packet (all was well), NAK the packet (report that the device cannot, temporarily, send/receive data, or that an interrupt endpoint isn't triggered), or STALL the bus in which case the host needs to intervene. Start of Frame Every 1ms (full-speed) the host will send a SOF packet which carries a frame number. This can be used to help keep time on very simple devices. It also divides the bus into frames within which bandwidth is allocated.

As an example, when the host wishes to perform a control transfer, the following packets are transacted in turn:

  1. Setup Token - The host addresses the device and endpoint (OUT0)
  2. Data0 Packet - The host transmits a GET_DESCRIPTOR for the device descriptor
  3. Ack Packet - The device acknowledges receipt of the request

This marks the end of the first transaction. The device decodes the GET_DESCRIPTOR request and prepares the device descriptor for transmission. The transmission occurs as the next transaction on the bus. In this example, we're assuming 8 byte maximum transmission sizes, for illustrative purposes.

  1. In Token - The host addresses the device and endpoint (IN0)
  2. Data1 Packet - The device transmits the first 8 bytes of the descriptor
  3. Ack Packet - The host acknowledges the data packet
  4. In Token - The host addresses the device and endpoint (IN0)
  5. Data0 Packet - The device transmits the remaining 4 bytes of the descriptor (padded)
  6. Ack Packet - The host acknowledges the data packet

The second transaction is now complete, and the host has all the data it needs to proceed. Finally a status transaction occurs in which:

  1. Out Token - The host addresses the device and endpoint (OUT0)
  2. Data1 Packet - The host transmits a 0 byte data packet to indicate successful completion
  3. Ack Packet - The device acknowledges the completion, indicating its own satisfaction

And thus ends the full control transaction in which the host retrieves the device descriptor.

From a high level, we need only consider the activity which occurs at the point of the acknowledgement packets. In the above example:

  1. On the first ACK the device prepares IN0 to transmit the descriptor, readying whatever low level device stack there is with a pointer to the descriptor and its length in bytes.
  2. On the second ACK the low levels are still thinking.
  3. On the third ACK the transmission from IN0 is complete and the endpoint no longer expects to transfer data.
  4. On the fourth ACK the control transaction is entirely complete.
Thinking at the low levels of the control interface

Before we can build a high level USB stack, we need to consider the activity which might occur at the lower levels. At the low levels, particularly of the device control interface, work has to be done at each and every packet. The hardware likely deals with the token packet for us, leaving the data packets for us to process, and the resultant handshake packets will be likely handled by the hardware in response to our processing the data packets.

Since every control transaction is initiated by a setup token, let's look at the setup requests which can come our way...

Setup Packet (Data) Format Field Name Byte start Byte length Encoding Meaning bmRequestType 0 1 Bitmap Describes the kind of request, and the target of it. See below. bRequest 1 1 Code The request code itself, meanings of the rest of the fields vary by bRequest wValue 2 2 Number A 16 bit value whose meaning varies by request type wIndex 4 2 Number A 16 bit value whose meaning varies by request type but typically encodes an interface number or endpoint. wLength 6 2 Number A 16 bit value indicating the length of the transfer to come.

Since bRequest is essentially a switch against which multiple kinds of setup packet are selected between, here's the meanings of a few...

GET_DESCRIPTOR (Device) setup packet Field Name Value Meaning bmRequestType 0x08 Data direction is IN (from device to host), recipient is the device bRequest 0x06 GET_DESCRIPTOR (in this instance, the device descriptor is requested) wValue 0x0001 This means the device descriptor wIndex 0x0000 Irrelevant, there's only 1 device descriptor anyway wLength 12 This is the length of a device descriptor (12 bytes) SET_ADDRESS to set a device's USB address Field Name Value Meaning bmRequestType 0x00 Data direction is OUT (from host to device), recipient is the device bRequest 0x05 SET_ADDRESS (Set the device's USB address) wValue 0x00nn The address for the device to adopt (max 127) wIndex 0x0000 Irrelevant for address setting wLength 0 There's no data transfer expected for this setup operation

Most hardware blocks will implement an interrupt at the point that the Data packet following the Setup packet has been receive. This is typically called receiving a 'Setup' packet and then it's up to the device stack low levels to determine what to do and dispatch a handler. Otherwise an interrupt will fire for the IN or OUT tokens and if the endpoint is zero, the low level stack will handle it once more.

One final thing worth noting about SET_ADDRESS is that it doesn't take effect until the completion of the zero-length "status" transaction following the setup transaction. As such, the status request from the host will still be sent to address zero (the default for new devices).

A very basic early "packet trace"

This is an example, and is not guaranteed to be the packet sequence in all cases. It's a good indication of the relative complexity involved in getting a fresh USB device onto the bus though...

When a device first attaches to the bus, the bus is in RESET state and so the first event a device sees is a RESET which causes it to set its address to zero, clear any endpoints, clear the configuration, and become ready for control transfers. Shortly after this, the device will become suspended.

Next, the host kicks in and sends a port reset of around 30ms. After this, the host is ready to interrogate the device.

The host sends a GET_DESCRIPTOR to the device, whose address at this point is zero. Using the information it receives from this, it can set up the host-side memory buffers since the device descriptor contains the maximum transfer size which the device supports.

The host is now ready to actually 'address' the device, and so it sends another reset to the device, again around 30ms in length.

The host sends a SET_ADDRESS control request to the device, telling it that its new address is nn. Once the acknowledgement has been sent from the host for the zero-data status update from the device, the device sets its internal address to the value supplied in the request. From now on, the device shall respond only to requests to nn rather than to zero.

At this point, the host will begin interrogating further descriptors, looking at the configuration descriptors and the strings, to build its host-side representation of the device. These will be GET_DESCRIPTOR and GET_STRING_DESCRIPTOR requests and may continue for some time.

Once the host has satisfied itself that it knows everything it needs to about the device, it will issue a SET_CONFIGURATION request which basically starts everything up in the device. Once the configuration is set, interrupt endpoints will be polled, bulk traffic will be transferred, Isochronous streams begin to run, etc.

Okay, but how do we make this concrete?

So far, everything we've spoken about has been fairly abstract, or at least "soft". But to transfer data over USB does require some hardware. (Okay, okay, we could do it all virtualised, but there's no fun in that). The hardware I'm going to be using for the duration of this series is the STM32 on the blue-pill development board. This is a very simple development board which does (in theory at least) support USB device mode.

If we view the schematic for the blue-pill, we can see a very "lightweight" USB interface which has a pullup resistor for D+. This is the way that a device signals to the host that it is present, and that it wants to speak at full-speed. If the pullup were on D- then it would be a low-speed device. High speed devices need a little more complexity which I'm not going to go into for today.

The USB lines connect to pins PA11 and PA12 which are the USB pins on the STM32 on the board. Since USB is quite finicky, the STM32 doesn't let you remap that function elsewhere, so this is all looking quite good for us so far.

The specific STM32 on the blue-pill is the STM32F103C8T6. By viewing its product page on ST's website we can find the reference manual for the part. Jumping to section 23 we learn that this STM32 supports full-speed USB2.0 which is convenient given the past article and a half. We also learn it supports up to eight endpoints active at any one time, and offers double-buffering for our bulk and isochronous transfers. It has some internal memory for packet buffering, so it won't use our RAM bandwidth while performing transfers, which is lovely.

I'm not going to distill the rest of that section here, because there's a large amount of data which explains how the USB macrocell operates. However useful things to note are:

  • How IN OUT and SETUP transfers work.
  • How the endpoint buffer memory is configured.
  • That all bus-powered devices MUST respond to suspend/resume properly
  • That the hardware will prioritise endpoint interrupts for us so that we only need deal with the most pressing item at any given time.
  • There is an 'Enable Function' bit in the address register which must be set or we won't see any transactions at all.
  • How the endpoint registers signal events to the device firmware.

Next time, we're going to begin the process of writing a very hacky setup routine to try and initialise the USB device macrocell so that we can see incoming transactions through the ITM. It should be quite exciting, but given how complex this will be for me to learn, it might be a little while before it comes through.

Bits from Debian: DebConf17 Open Day

5 August, 2017 - 21:00

Today, the day preceeding the official start of the annual Debian Conference, is the Open Day at DebConf17, at Collège Maisonneuve in Montreal (Canada).

This day is open to the public with events of interest to a wide audience.

The schedule of today's events include, among others:

  • A Newbie's Newbie Guide to Debian
  • Ask Anything About Debian
  • Debian Packaging 101
  • Debian InstallFest
  • Presentations or workshops related to free software projects and local organizations.

Everyone is welcome to attend! It is a great possibility for interested users to meet our community and for Debian to widen our community.

See the full schedule for today's events at https://debconf17.debconf.org/schedule/open-day/.

If you want to engage remotely, you can watch the video streaming of the Open Day events happening in the "Rex" room, or join the conversation in the channels #debconf17-rex, #debconf17-potato and #debconf17-woody in the OFTC IRC network.

DebConf is committed to a safe and welcome environment for all participants. See the DebConf Code of Conduct and the Debian Code of Conduct for more details on this.

Debian thanks the commitment of numerous sponsors to support DebConf17, particularly our Platinum Sponsors Savoir-Faire Linux, Hewlett Packard Enterprise, and Google.

Steinar H. Gunderson: Dear conference organizers

5 August, 2017 - 16:45

Dear conference organizers,

In this day and age, people stream conferences and other events over the Internet. Most of the Internet happens to be in a different timezone from yours (it's crazy, I know!). This means that if you publish a schedule, please say which timezone it's in. We've even got this thing called JavaScript now, which allows you to also convert times to the user's local timezone (the future is now!), so you might want to consider using it. :-)

(Yes, this goes for you, DebConf, and also for you, Assembly.)

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้