Mirage OS Logo
Home Blog Docs API Community

MirageOS Summer 2016 hack retreat roundup

By Gemma Gordon , Anil Madhavapeddy - 2016-07-18

Our first Cambridge-based MirageOS hack retreat took place yesterday - and what a fantastic day it was! The torrential rain may have halted our punting plans, but it didn't stop progress in the Old Library! Darwin College was a fantastic venue, complete with private islands linked by picturesque wooden bridges and an unwavering wifi connection.

People naturally formed groups to work on similar projects, and we had a handful of brand new users keen to get started with OCaml and Mirage. The major tasks that emerged were:

  • new hypervisor target: the integration of the Solo5 KVM-based hypervisor backend, bringing the number of officially supported targets up to 3 (Xen, Unix and KVM)
  • build system template: establishing a new topkg template for MirageOS libraries, to prepare us for building a unified API documentation bundle that works across all the entire project.
  • CPU portability: improving ARM support via a better base OS image.
  • libraries breadth: hacking on all the things to fill in the blanks, such as btree support for bare-metal Irmin, or a peer-to-peer layer for the DataKit.

We'll write about all of this in more detail, but for now here are the hack retreat notes hot off the press...

Solo5/MirageOS integration (KVM-based backend)

Progress on the Solo5 project has been steaming ahead since January, and this was the perfect opportunity to get everyone together to plan its integration with MirageOS. Dan Williams from IBM Research flew over to join us for the week, and Martin Lucina headed to Cambridge to prepare for the upstreaming of the recent Solo5 work. This included deciding on naming and ownership of the repositories, detailing the relationships between repositories and getting ready to publish the mirage-solo5 packages to OPAM. Mindy Preston, our MirageOS 3.0 release manager, and Anil Madhavapeddy and Thomas Gazagnaire (OPAM minions) were on hand to help plan this smoothly.

See their updates from the day on Canopy and related blog posts:

Onboarding new MirageOS/OCaml users

Our tutorials and onboarding guides really needed a facelift and an update, so Gemma Gordon spent the morning with some of our new users to observe their installation process and tried to pinpoint blockers and areas of misunderstanding. Providing the simple, concise instructions needed in a guide together with alternatives for every possible system and version requirement is a tricky combination to get right, but we made some changes to the installation guide that we hope will help. The next task is to do the same for our other popular tutorials, reconfigure the layout for easy reading and centralise the information as much as possible between the OPAM, MirageOS and OCaml guides. Thank you to Marwan Aljubeh for his insight into this process.

Other industrial users are also steaming ahead with their own MirageOS deployments. Amir Chaudhry spent the hackathon blogging about NFV Platforms with MirageOS unikernels, which details how Ericsson Silicon Valley has been using MirageOS to build lightweight routing kernels.

Packaging

Thomas Gazagnaire was frenetically converting functoria, mirage, mirage-types and mirage-console to use topkg, and the feedback prompted fixes and a new release from Daniel Buenzli.

ARM and Cubieboards

Ian Campbell implemented a (slightly hacky) way to get Alpine Linux onto some Cubieboard2 boxes and provided notes on his process, including how to tailor the base for KVM and Xen respectively.

Meanwhile, Qi Li worked on testing and adapting simple-nat and mirage-nat to provide connectivity control for unikernels on ARM Cubieboards to act as network gateways.

Hannes Mehnert recently published a purely functional ARP package and continued refining it (with code coverage via bisect-ppx) during the hackathon.

MirageOS 3.0 API changes

Our MirageOS release manager, Mindy Preston, was on hand to talk with everyone about their PRs in preparation for the 3.0 release along with some patches for deprecating out of date code. There has been a lot of discussion on the development list. One focus was to address time handling properly in the interfaces: Matthew Gray came up from London to finish up his extensive revision of the CLOCK interface, and Hannes developed a new duration library to handle time unit conversions safely and get rid of the need for floating point handling. We are aiming to minimise the dependency on floating point handling in external interfaces to simplify compilation to very embedded hardware that only has soft floats (particularly for something as ubiquitous as time handling).

Error logging

Thomas Leonard continued with the work he started in Marrakech by updating the error reporting patches (also here) to work with the latest version of MirageOS (which has a different logging system based on Daniel Buenzlis Logs). See the original post for more details.

Ctypes 0.7.0 release

Jeremy released the foreign function interface library Ctypes 0.7.0 which, along with bug fixes, adds the following features:

  • Support for bytecode-only architectures (#410)
  • A new sint type corresponding to a full-range C integer and updated errno support for its use (#411)

See the full changelog online.

P2P key-value store over DataKit

KC Sivaramakrishnan and Philip Dexter took on the challenge of grabbing the Docker DataKit release and started building a distributed key-value store that features flexible JSON synching and merging. Their raw notes are in a Gist -- get in touch with them if you want to help hack on the sync system backed by Git.

Developer experience improvements

The OCaml Labs undergraduate interns are spending their summers working on user improvements and CI logs with MirageOS, and used the time at the hackathon to focus on these issues.

Ciaran Lawlor is working on an editor implementation, specifically getting the IOcaml kernel working with the Hydrogen plugin for the Atom editor. This will allow developers to run OCaml code directly in Atom, and eventually interactively build unikernels!

Joel Jakubovic used Angstrom (a fast parser combinator library developed by Spiros Eliopoulos) to ANSI escape codes, usually displayed as colours and styles into HTML for use in viewing CI logs.

Windows Support

Most of the Mirage libraries already work on Windows thanks to lots of work in the wider OCaml community, but other features don't have full support yet.

Dave Scott from Docker worked on ocaml-wpcap: a ctypes binding to the Windows winpcap.dll which lets OCaml programs send and receive ethernet frames on Windows. The ocaml-wpcap library will hopefully let us run the Mirage TCP/IP stack and all the networking applications too.

David Allsopp continued his OPAM-Windows support by fine-tuning the 80 native Windows OCaml versions - these will hopefully form part of OPAM 2.0. As it turns out, he's not the only person still interested in being able to run OCaml 3.07...if you are, get in touch!

General Libraries and utilities

Olivier Nicole is working on an implementation of macros in OCaml and started working on the HTML and XML templates using this system. The objective is to have the same behaviour as the Pa_tyxml syntax extension, but in a type-safe and more maintainable way without requiring PPX extensions. This project could be contributed to the development of Ocsigen once implemented.

Nick Betteridge teamed up with Dave Scott to look at using ocaml-btree as a backend for Irmin/xen and spent the day looking at different approaches.

Anil Madhavapeddy built a Docker wrapper for the CI system and spun up a big cluster to run OPAM bulk builds. Several small utilities like jsontee and an immutable log collection server and bulk build scripts will be released in the next few weeks once the builds are running stably, and be re-usable by other OPAM-based projects to use for their own tests.

Christophe Troestler is spending a month at OCaml Labs in Cambridge this summer, and spent the hack day working on implementing a library to allow seamless application switching from HTTP to FastCGI. Christophe has initiated work on a client and server for this protocol using CoHTTP so that it is unikernel-friendly.