Planet Debian

Subscribe to Planet Debian feed
Planet Debian - http://planet.debian.org/
Updated: 1 hour 6 min ago

Dirk Eddelbuettel: RcppArmadillo 0.6.700.6.0

6 May, 2016 - 09:18

A second Armadillo release 6.700.6 came out in the 6.700 series, and we uploaded RcppArmadillo 0.6.700.6.0 to CRAN and Debian. This followed the usual thorough reverse-dependecy checking of by now 220 packages using.

This release is a little unusual in that it contains both upstream bugfixes in the same series (see below) but also two nice bug fixes from the RcppArmadillo side. Both were squashed by George G. Vega Yon via two focused pull request. The first ensures that we can now use ARMA_64BIT_WORD (provided C++11 is turned on too) allowing for much bigger Armadillo objects. And the second plugs a small leak in the sparse matrix converter I had added a while back. Nice work, all told!

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab.

Changes in this release are as follows:

Changes in RcppArmadillo version 0.6.700.6.0 (2016-05-05)
  • Upgraded to Armadillo 6.700.6 (Catabolic Amalgamator Deluxe)

    • fix for handling empty matrices by kron()

    • fix for clang warning in advanced matrix constructors

    • fix for false deprecated warning in trunc_log() and trunc_exp()

    • fix for gcc-6.1 warning about misleading indentation

    • corrected documentation for the solve() function

  • Added support for int64_t (ARMA_64BIT_WORD) when required during compilation time. (PR #90 by George G. Vega Yon, fixing #88)

  • Fixed bug in SpMat exporter (PR #91 by George G. Vega Yon, fixing #89 and #72)

Courtesy of CRANberries, there is also a diffstat report for this release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Matthias Klumpp: Adventures in D programming

6 May, 2016 - 01:26

I recently wrote a bigger project in the D programming language, the appstream-generator (asgen). Since I rarely leave the C/C++/Python realm, and came to like many aspects of D, I thought blogging about my experience could be useful for people considering to use D.

Disclaimer: I am not an expert on programming language design, and this is not universally valid criticism of D – just my personal opinion from building one project with it.

Why choose D in the first place?

The previous AppStream generator was written in Python, which wasn’t ideal for the task for multiple reasons, most notably multiprocessing and LMDB not working well together (and in general, multiprocessing being terrible to work with) and the need to reimplement some already existing C code in Python again.

So, I wanted a compiled language which would work well together with the existing C code in libappstream. Using C was an option, but my least favourite one (writing this in C would have been much more cumbersome). I looked at Go and Rust and wrote some small programs performing basic operations that I needed for asgen, to get a feeling for the language. Interfacing C code with Go was relatively hard – since libappstream is a GObject-based C library, I expected to be able to auto-generate Go bindings from the GIR, but there were only few outdated projects available which did that. Rust on the other hand required the most time in learning it, and since I only briefly looked into it, I still can’t write Rust code without having the coding reference open. I started to implement the same examples in D just for fun, as I didn’t plan to use D (I was aiming at Go back then), but the language looked interesting. The D language had the huge advantage of being very familiar to me as a C/C++ programmer, while also having a rich standard library, which included great stuff like std.concurrency.Generator, std.parallelism, etc. Translating Python code into D was incredibly easy, additionally a gir-d-generator which is actively maintained exists (I created a small fork anyway, to be able to directly link against the libappstream library, instead of dynamically loading it).

What is great about D?

This list is just a huge braindump of things I had on my mind at the time of writing

Interfacing with C

There are multiple things which make D awesome, for example interfacing with C code – and to a limited degree with C++ code – is really easy. Also, working with functions from C in D feels natural. Take these C functions imported into D:

extern(C):
nothrow:

struct _mystruct {}
alias mystruct_p = _mystruct*;

mystruct_p = mystruct_create ();
mystruct_load_file (mystruct_p my, const(char) *filename);
mystruct_free (mystruct_p my);

You can call them from D code in two ways:

auto test = mystruct_create ();
// treating "test" as function parameter
mystruct_load_file (test, "/tmp/example");
// treating the function as member of "test"
test.mystruct_load_file ("/tmp/example");
test.mystruct_free ();

This allows writing logically sane code, in case the C functions can really be considered member functions of the struct they are acting on. This property of the language is a general concept, so a function which takes a string as first parameter, can also be called like a member function of string.

Writing D bindings to existing C code is also really simple, and can even be automatized using tools like dstep. Since D can also easily export C functions, calling D code from C is also possible.

Getting rid of C++ “cruft”

There are many things which are bad in C++, some of which are inherited from C. D kills pretty much all of the stuff I found annoying. Some cool stuff from D is now in C++ as well, which makes this point a bit less strong, but it’s still valid. E.g. getting rid of the #include preprocessor dance by using symbolic import statements makes sense, and there have IMHO been huge improvements over C++ when it comes to metaprogramming.

Incredibly powerful metaprogramming

Getting into detail about that would take way too long, but the metaprogramming abilities of D must be mentioned. You can do pretty much anything at compiletime, for example compiling regular expressions to make them run faster at runtime, or mixing in additional code from string constants. The template system is also very well thought out, and never caused me headaches as much as C++ sometimes manages to do.

Built-in unit-test support

Unittesting with D is really easy: You just add one or more unittest { } blocks to your code, in which you write your tests. When running the tests, the D compiler will collect the unittest blocks and build a test application out of them.

The unittest scope is useful, because you can keep the actual code and the tests close together, and it encourages writing tests and keep them up-to-date. Additionally, D has built-in support for contract programming, which helps to further reduce bugs by validating input/output.

Safe D

While D gives you the whole power of a low-level system programming language, it also allows you to write safer code and have the compiler check for that, while still being able to use unsafe functions when needed.

Unfortunately, @safe is not the default for functions though.

Separate operators for addition and concatenation

D exclusively uses the + operator for addition, while the ~ operator is used for concatenation. This is likely a personal quirk, but I love it very much that this distinction exists. It’s nice for things like addition of two vectors vs. concatenation of vectors, and makes the whole language much more precise in its meaning.

Optional garbage collector

D has an optional garbage collector. Developing in D without GC is currently a bit cumbersome, but these issues are being addressed. If you can live with a GC though, having it active makes programming much easier.

Built-in documentation generator

This is almost granted for most new languages, but still something I want to mention: Ddoc is a standard tool to generate code documentation for D code, with a defined syntax for describing function parameters, classes, etc. It will even take the contents of a unittest { } scope to generate automatic examples for the usage of a function, which is pretty cool.

Scope blocks

The scope statement allows one to execute a bit of code before the function exists, when it failed or was successful. This is incredibly useful when working with C code, where a free statement needs to be issued when the function is exited, or some arbitrary cleanup needs to be performed on error. Yes, we do have smart pointers in C++ and – with some GCC/Clang extensions – a similar feature in C too. But the scopes concept in D is much more powerful. See Scope Guard Statement for details.

Built-in syntax for parallel programming

Working with threads is so much more fun in D compared to C! I recommend taking a look at the parallelism chapter of the “Programming in D” book.

D is fast!

D matches the speed of C++ in almost all occasions, so you won’t loose performance when writing D code – that is, unless you have the GC run often in a threaded environment.

Very active and friendly community

The D community is very active and friendly – so far I only had good experience, and I basically came into the community asking some tough questions regarding distro-integration and ABI stability of D. The D community is very enthusiastic about pushing D and especially the metaprogramming features of D to its limits, and consists of very knowledgeable people. Most discussion happens at the forums/newsgroups at forum.dlang.org.

What is bad about D? Proprietary reference compiler

This is probably the biggest issue. Not because the proprietary compiler is bad per se, but because of the implications this has for the D ecosystem.

For the reference D compiler, Digital Mars’ D (DMD), only the frontend is distributed under a free license (Boost), while the backend is proprietary. The FLOSS frontend is what the free compilers, LLVM D Compiler (LDC) and GNU D Compiler (GDC) are based on. But since DMD is the reference compiler, most features land there first, and the Phobos standard library and druntime is tuned to work with DMD first.

Since major Linux distributions can’t ship with DMD, and the free compilers GDC and LDC lack behind DMD in terms of language, runtime and standard-library compatibility, this creates a split world of code that compiles with LDC, GDC or DMD, but never with all D compilers due to it relying on features not yet in e.g. GDCs Phobos.

Especially for Linux distributions, there is no way to say “use this compiler to get the best and latest D compatibility”. Additionally, if people can’t simply apt install latest-d, they are less likely to try the language. This is probably mainly an issue on Linux, but since Linux is the place where web applications are usually written and people are likely to try out new languages, it’s really bad that the proprietary reference compiler is hurting D adoption in that way.

That being said, I want to make clear DMD is a great compiler, which is very fast and build efficient code. I only criticise the fact that it is the language reference compiler.

Phobos (standard library) is deprecating features too quickly

This basically goes hand in hand with the compiler issue mentioned above. Each D compiler ships its own version of Phobos, which it was tested against. For GDC, which I used to compile my code due to LDC having bugs at that time, this means that it is shipping with a very outdated copy of Phobos. Due to the rapid evolution of Phobos, this meant that the documentation of Phobos and the actual code I was working with were not always in sync, leading to many frustrating experiences.

Furthermore, Phobos is sometimes removing deprecated bits about a year after they have been deprecated. Together with the older-Phobos situation, you might find yourself in a place where a feature was dropped, but the cool replacement is not yet available. Or you are unable to import some 3rd-party code because it uses some deprecated-and-removed feature internally. Or you are unable to use other code, because it was developed with a D compiler shipping with a newer Phobos.

This is really annoying, and probably the biggest source of unhappiness I had while working with D – especially the documentation not matching the actual code is a bad experience for someone new to the language.

Incomplete free compilers with varying degrees of maturity

LDC and GDC have bugs, and for someone new to the language it’s not clear which one to choose. Both LDC and GDC have their own issues at time, but they are rapidly getting better, and I only encountered some actual compiler bugs in LDC (GDC worked fine, but with an incredibly out-of-date Phobos). All issues are fixed meanwhile, but this was a frustrating experience. Some clear advice or explanation which of the free compilers is to prefer when you are new to D would be neat.

For GDC in particular, being developed outside of the main GCC project is likely a problem, because distributors need to manually add it to their GCC packaging, instead of having it readily available. I assume this is due to the DRuntime/Phobos not being subjected to the FSF CLA, but I can’t actually say anything substantial about this issue. Debian adds GDC to its GCC packaging, but e.g. Fedora does not do that.

No ABI compatibility

D has a defined ABI – too bad that in reality, the compilers are not interoperable. A binary compiled with GDC can’t call a library compiled with LDC or DMD. GDC actually doesn’t even support building shared libraries yet. For distributions, this is quite terrible, because it means that there must be one default D compiler, without any exception, and that users also need to use that specific compiler to link against distribution-provided D libraries. The different runtimes per compiler complicate that problem further.

The D package manager, dub, does not yet play well with distro packaging

This is an issue that is important to me, since I want my software to be easily packageable by Linux distributions. This is reported as issue #811, with quite positive feedback so far, so this might soon be solved.

The GC is sometimes an issue

The garbage collector in D is quite dated (according to their own docs) and is currently being reworked. While working with asgen, which is a program creating a large amount of interconnected data structures in a threaded environment, I realized that the GC is significantly slowing down the application when threads are used (it also seems to use UNIX signals SIGUSR1 and SIGUSR2 to stop/resume threads, which I still find odd). Also, the GC performed poorly on memory pressure, which did get asgen killed by the OOM killer on some more memory-constrained machines. Triggering a manual collection run after a large amount of these interconnected data structures wasn’t needed anymore solved this problem for most systems, but it would of course have been better to not needing to give the GC any hints. The stop-the-world behavior isn’t a problem for asgen, but it might be for other applications.

These issues are at time being worked on, with a GSoC project laying the foundation for further GC improvements.

The ecosystem is not (yet) mature

In general it can be said that the D ecosystem, while existing for almost 9 years, is not yet that mature. There are various quirks you have to deal with when working with D code on Linux. It’s always nothing major, usually you can easily solve these issues and go on, but it’s annoying to have these papercuts.

This is not something which can be resolved by D itself, this point will solve itself as more people start to use D and D support in Linux distributions gets more polished.

Conclusion

I like to work with D, and I consider it to be a great language – the quirks it has in its toolchain are not that bad to prevent writing great things with it.

At time, if I am not writing a shared library or something which uses much existing C++ code, I would prefer D for that task. If a garbage collector is a problem (e.g. for some real-time applications, or when the target architecture can’t run a GC), I would not recommend to use D. Rust seems to be the much better choice then.

In any case, D’s flat learning curve (for C/C++ people) paired with the smart choices taken in language design, the powerful metaprogramming, the rich standard library and helpful community makes it great to try out and to develop software for scenarios where you would otherwise choose C++ or Java. Quite honestly, I think D could be a great language for tasks where you would usually choose Python, Java or C++, and I am seriously considering to replace quite some Python code with D code. For very low-level stuff, C is IMHO still the better choice.

As always, choosing the right programming language is only 50% technical aspects, and 50% personal taste

Sean Whitton: dh_make_elpa & dh_elpa_test

6 May, 2016 - 01:08

I recently completed and released some work on Debian’s tooling for packaging Emacs Lisp addons to GNU Emacs. Emacs grew a package manager called package.el a few years ago, and last year David Bremner wrote the dh_elpa tool to simplify packaging addons for Debian by leveraging package.el features. Packaging a series of addons for Debian left me with a wishlist of features for dh_elpa and I was recently able to implement them.

Debian tooling generally uses Perl, a language I didn’t know before starting on this project. I was fortunate enough to receive a free review copy of Perl 5 by Example when I attended a meeting of the Bay Area Linux Users Group while I was visiting San Francisco a few months ago. I accepted the book with the intent of doing this work.

dh_make_elpa

dh_make_elpa (at present available from Debian experimental) is a Perl script to convert a git repository cloned from the upstream of an Emacs Lisp addon to a rudimentary Debian package. It performs a lot of guesswork, and its simple heuristics are something I hope to improve on. Since I am new to object-oriented program design in Perl and I wanted to leverage object-oriented Debian tooling library code, I took the structure of my project from dh_make_perl. In this manner I found it easy and pleasant to write a maintainable script.

dh_elpa_test

A lot of Emacs Lisp addon packages use a program called Cask to manage the Emacs Lisp dependencies needed to run their test suites. That meant that dh_auto_test often fails to run Emacs Lisp addon package test suites. Since the Debian packaging toolchain already has advanced dependency management, it’s undesirable to involve Cask in the package build pipeline if it can be avoided. I had been copying and pasting the code needed to make the tests run in our environment to the debian/rules files of each package whose test suite I wanted to run.

dh_elpa_test tries to detect Emacs Lisp addon package test suites and run them with the workarounds needed in our environment. This avoids boilerplate in debian/rules. dh_elpa_test also disables dh_auto_test to avoid a inadvertent Cask invocation.

Future & acknowledgements

My hope for this work was to make it easier and faster to package Emacs Lisp addon packages for Debian, for my own sake and for anyone new who is interested in joining the pkg-emacsen team. In the future, I want to have dh_elpa_test generate an autopkgtest definition so that a Testsuite: pkg-emacsen line in debian/control is enough to have an Emacs Lisp addon package test suite run on Debian CI.

I’m very grateful to David Bremner for reviewing and supporting this work, and also for supporting my Emacs Lisp addon packaging work more generally.

Petter Reinholdtsen: The Pyra - handheld computer with Debian preinstalled

4 May, 2016 - 15:00
A friend of mine made me aware of The Pyra, a handheld computer which will be delivered with Debian preinstalled. I would love to get one of those for my birthday. :)

The machine is a complete ARM-based PC with micro HDMI, SATA, USB plugs and many others connectors, and include a full keyboard and a 5" LCD touch screen. The 6000mAh battery is claimed to provide a whole day of battery life time, but I have not seen any independent tests confirming this. The vendor is still collecting preorders, and the last I heard last night was that 22 more orders were needed before production started.

As far as I know, this is the first handheld preinstalled with Debian. Please let me know if you know of any others. Is it the first computer being sold with Debian preinstalled?

Debian Java Packaging Team: What's new since Jessie?

4 May, 2016 - 14:55

Jessie was released one year ago now and the Java Team has been busy preparing the next release. Here is a quick summary of the current state of the Java packages:

  • A total of 136 packages have been added, 63 removed, 213 upgraded to a new upstream release, and 145 updated. We are now maintaining 892 packages (+12.34%).
  • OpenJDK 8 is now the default Java runtime in testing/unstable. OpenJDK 7 has been removed, as well as several packages that couldn't be upgraded to work with OpenJDK 8 (avian, eclipse).
  • OpenJDK 9 is available in experimental. As a reminder, it won't be part of the next release; OpenJDK 8 will be the only Java runtime supported for Stretch.
  • Netbeans didn't make it into Jessie, but it is now back and up to date.
  • The main build tools are close to their latest upstream releases, especially Maven and Gradle which were seriously lagging behind.
  • Scala has been upgraded to the version 2.11. We are looking for Scala experts to maintain the package and its dependencies.
  • Freemind has been removed due to lack of maintenance, Freeplane is recommended instead.
  • The reproducibility rate has greatly improved, climbing from 50% to 75% in the past year.
  • Backports are continuously provided for the key packages and applications: OpenJDK 8, OpenJFX, Ant, Maven, Gradle, Tomcat 7 & 8, Jetty 8 & 9, OpenJDK 8, OpenJFX, jEdit.
  • The transition to Maven 3 has been completed, and packages are no longer built with Maven 2.
  • We replaced several obsolete libraries and transitioned them to their latest versions - for example, asm2, commons-net1 and commons-net2. Groovy 1.x was replaced with Groovy 2, and we upgraded BND, an important tool to develop with OSGi, and more than thirty of its reverse-dependencies from the 1.x series to version 2.4.1.
  • New packaging tools have been created to work with Gradle (gradle-debian-helper) and Ivy (ivy-debian-helper).
Outlook, goals and request for help
  • We have several difficult transitions ahead: BND 3, Tomcat 7 to 8, Jetty 8 to 9, ASM 5, and of course Java 9. Any help would be welcome.
  • Eclipse is severely outdated and currently not part of testing. We would like to update this important piece of software and its corresponding modules to the latest upstream release, but we need more active people who want to maintain them. If you care about the Eclipse ecosystem, please get in touch with us.
  • We still are in the midst of removing old libraries like asm3, commons-httpclient and the servlet 2.5 API, which is part of the Tomcat 6 source package.
  • Want to see Azureus/Vuze in Stretch again? Packaging is almost complete but we are looking for someone who can clarify remaining licensing issues with upstream and wants to maintain the software for the foreseeable future.
  • Do you have more ideas and want to get involved with the Java Team? Just send your suggestions to debian-java@lists.debian.org or chat with us on IRC at irc.debian.org, #debian-java.
Java and Friends
  • The Java Team is not the only team that maintains Java software in Debian. DebianMed, DebianScience and the Android Tools Maintainers rely heavily on Java. By helping the Java Team and working together, you can improve the Java ecosystem and further the efforts of multiple other fields of endeavor all at once.
Package updates

The packages listed below detail the changes in jessie-backports and testing. Libraries and Debian specific tools have been excluded.

Packages added to jessie-backports:

Packages removed from testing:

Packages added to testing:

Packages upgraded in testing:

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้