Announcing MirageOS 3.0.0

We're excited to announce MirageOS 3.0! MirageOS is a modern, modular library operating system that allows the creation of small, secure, legacy-free services. MirageOS applications can be compiled to run as self-contained virtual machines (a few MB in size) on Xen or KVM hosts, FreeBSD's bhyve, or even as regular Unix processes (allowing access to regular debugging tools). The system libraries themselves can be reused in traditional applications, just like any other software library.

Full release notes are available on GitHub. If you're interested in getting started with MirageOS 3 right away, you might be interested in the revamped guide to getting started, a small collection of example unikernels, or the porting guide for updating Mirage 2.x unikernels to Mirage 3.

Here's a summary of the things in MirageOS 3 that we're most excited about:

Solo5

MirageOS 3.0 is the first release that integrates the solo5 targets, virtio and ukvm, fully with the mirage front-end tool. Now you can mirage configure -t ukvm, build a unikernel, and run directly with the generated ukvm-bin! We've updated the "hello world" tutorial to reflect our excitement about ukvm -- the ukvm target is considerably easier to interface with and configure than xen was, and for a lot of users this will be a clearer path toward operational deployment of unikernels.

For a lot more information on the Solo5 targets, see the earlier blog post announcing solo5, Unikernel Monitors: Extending Minimalism Outside of the Box, and the very readable solo5 repository README. You can also read how to run solo5 unikernels on FreeBSD via bhyve.

Playing More Nicely with OPAM

MirageOS 3 has a much richer interface for dealing with the package manager and external library dependencies. A user can now specify a version or range of versions for a package dependency, and the mirage front-end tool will construct a custom opam file including both those package dependencies and the ones automatically generated from mirage configure. mirage will also consider version constraints for its own packages -- from now on, opam should notice that releases of mirage are incompatible with your unikernel.

For more information on dealing with packages and dependencies, the documentation for the Functoria.package function will likely be of use. The PRNG device-usage example in mirage-skeleton demonstrates some useful invocations of package.

Amazing Docs

Thanks to a lot of hard work, a fully interlinked set of module documentation is now automatically generated by odig and available for your reading pleasure at the MirageOS central documentation repository. While documentation was previously available for most modules, it was scattershot and often required having several disconnected pages open simultaneously. We hope you'll find the new organization more convenient. The documentation generation system is still in beta, so please report issues upstream if you run across rendering issues or have other feedback.

Result-y Errors

The module types provided by MirageOS 3 replace the previous error paradigm (a combination of exceptions and directly returning polymorphic variants) with one that uses the Result module included in OCaml 4.03 and up. A notable exception is when problems occur during the unikernel's initialization (i.e., in connect functions), where unikernels will now fail hard as soon as they can. The goal of these changes is to surface errors when the application cares about them, and to not present any uninitialized or unstable state to an application at start time.

The MirageOS 3 module types define a core set of likely errors for each module type (see the mirage-flow module type for an example), which can be extended by any given implementation. Module types now specify that each implementation must include a pretty-printer that can handle all emitted error types. Functions that return a success type when they run as expected return a (success, error) Result.t, which the caller can print with pp_error if the value is an Error.

For more background on the result type, see the Rresult library which defines further useful operations on Result.t and is used widely in MirageOS libraries. A more in-depth explanation of errors in Mirage 3 is also available.

Logs Where You Want Them

MirageOS version 2.9.0 included automatic support for logging via the Logs and Mirage_logs library, but by default logs were always printed to the console and changing the log reporter was cumbersome. In MirageOS 3, you can send logs to a consumer of syslog messages with syslog_udp, syslog_tcp, or with the full authentication and encryption provided by ocaml-tls using syslog_tls. For more information, see the excellent writeup at hannes.nqsb.io.

Disaggregated Module Type Definitions

Breaking all of the MirageOS 3.0 APIs showed us that keeping them all in the same place made updates really difficult. There's now an additional set of packages which contain the definitions for each set of module types (e.g. mirage-fs for the FS module type, mirage-block for the BLOCK module type, etc). A few module types had some additional useful code that was nicely functorized over the module type in question, so we've bundled that code in the module type packages as well. Documentation for all of the module type packages is available at the Mirage documentation hub.

We hope that this change combined with the opam workflow changes above will result in much less painful API changes in the future, as it will be possible for unikernel authors to target specific versions more readily.

Clockier Clocks, Timelier Time

In older MirageOS versions, we noticed that we were often having to deduce a span of time from having taken two wall-clock samples of the current time. In MirageOS 3, you have your choice of two types of clock - MCLOCK, which provides a monotonically increasing clock reflecting the time elapsed since the clock started, and PCLOCK, which provides a traditional POSIX wall-clock time. Most previous users of CLOCK were able to migrate to the more-honest, less-complicated MCLOCK. For an example of both clocks, see the speaking clock. You may also be interested in an example of converting existing code from CLOCK to MCLOCK.

MCLOCK provides a nice interface for dealing with time at a nanosecond granularity. The TIME module type has been updated to expect an int64 number of nanoseconds, rather than a float, as an argument to its function sleep. For those of us who don't think in nanoseconds, the Duration library provides convenient functions for translating from and to more familiar units like seconds.

Build System Shift

Mirage 3.0 has many, many more packages than before, and so we turned to OCaml Labs to help us to scale up our package management. In many but not all MirageOS packages, we've replaced oasis with topkg, the "transitory OCaml software packager". topkg is a lighter layer over the underlying ocamlbuild. Using topkg has allowed us to remove several thousand lines of autogenerated code across the MirageOS package universe, and let our release manager automate a significant amount of the MirageOS 3 release process. We hope to continue benefitting from the ease of using topkg and topkg-care.

Not all packages are using topkg yet -- if you see one that isn't, feel free to submit a pull request!

Less Code, Better Behavior

There's more in MirageOS 3 than we can fit in one blog post without our eyes glazing over. The release notes for mirage version 3.0.0 are a nice summary, but you might also be interested in the full accounting of changes for every package released as a part of the MirageOS 3 effort; links for each library are available at the end of this post.

Across the package universe, a net several thousand lines of code were removed as part of MirageOS 3. Many were autogenerated build-time support files removed in the transition from oasis to topkg. Others were small support modules like Result, which had previously been replicated in many places and were replaced by a reference to a common implementation. Some large implementations (like the DHCP client code in mirage-tcpip) were replaced by smaller, better implementations in common libraries (like charrua-core).

For example, ocaml-fat had 1,280 additions and 10,265 deletions for a net of -8,985 lines of code; version 0.12.0 jettisoned a custom in-memory block device in favor of using the in-memory block device provided by Mirage_block_lwt.Mem, removed several thousand lines of autogenerated OASIS code, removed several custom error-case polymorphic variants, and lost a custom result module. The mirage repository itself netted -8,490 lines of code while adding all of the features above!

A number of improvements were made to mirage to limit the number of unnecessary build artifacts and reduce the amount of unnecessary code linked into unikernels. Modules you're unlikely to use like Str are no longer included in the OCaml runtime. MirageOS 3 is also the first to drop support for OCaml 4.02.3, meaning that all supported compilers support the flambda compiler extension and a number of related optimization opportunities.

Very many people were involved in making the MirageOS package universe smaller and better than it was before. We'd like to thank, in a particular alphabetical order, the following people who contributed code, suggestions, bug reports, comments, mailing lists questions and answers, and other miscellaneous help:

• Aaron Cornelius
• Amir Chaudhry
• Andrew Stuart
• Anil Madhavapeddy
• Ashish Agarwal
• Balraj Singh
• Cedric Cellier
• Christiano Haesbaert
• Daniel Bünzli
• Dan Williams
• Dave Scott
• David Kaloper
• David Sheets
• Enguerrand Decorne
• Eugene Bagdasaryan
• Federico Gimenez
• Gabriel de Perthuis
• Gabriel Jaldon
• Gabriel Radanne
• Gemma Gordon
• Hannes Mehnert
• Ian Campbell
• Jochen Bartl
• John P. McDermott
• Jon Ludlam
• Kia
• Leo White
• Leonid Rozenberg
• Liang Wang
• Madhuri Yechuri
• Magnus Skjegstad
• Martin Lucina
• Matt Gray
• Mindy Preston
• Nick Betteridge
• Nicolas Ojeda Bar
• Nik Sultana
• Pablo Polvorin
• Petter A. Urkedal
• Qi LI
• Ramana Venkata
• Ricardo Koller
• Richard Mortier
• Rudi Grinberg
• Sean Grove
• Takayuki Imada
• Thomas Gazagnaire
• Thomas Leonard
• Vincent Bernardoff
• Vittorio Cozzolino
• GitHub user waldyrious
• Wassim Haddad
• Jeremy Yallop

Please let us know if you notice someone (including yourself) is missing so we can add them and apologize! We're happy to remove or change your listed name if you'd prefer as well. Names were taken from metadata on commit messages and e-mail headers.

• For a summary of changes in each repository that released code for MirageOS 3, please see the following list:
• changes in mirage-clock between v1.1 and 1.2.0: 51 files changed, 788 insertions(+), 381 deletions(-)
• changes in ocaml-vchan between v2.2.0 and v2.3.0: 14 files changed, 4384 insertions(+), 3553 deletions(-)
• changes in charrua-core between v0.3 and v0.4: 14 files changed, 760 insertions(+), 231 deletions(-)
• changes in arp between 0.1.1 and 0.2.0: 23 files changed, 302 insertions(+), 430 deletions(-)
• changes in logs-syslog between 0.0.2 and 0.1.0: 25 files changed, 277 insertions(+), 361 deletions(-)
• changes in mirage-vnetif between 0.2.0 and v0.3: 35 files changed, 669 insertions(+), 8551 deletions(-)
• changes in functoria between 1.1.0 and 2.0.1: 46 files changed, 1107 insertions(+), 9666 deletions(-)
• changes in mirage-block between v0.2 and 1.0.0: 42 files changed, 1194 insertions(+), 925 deletions(-)
• changes in mirage-block-ramdisk between v0.2 and v0.3: 18 files changed, 440 insertions(+), 305 deletions(-)
• changes in mirage-block-solo5 between v0.1.1 and v0.2.1: 23 files changed, 187 insertions(+), 7854 deletions(-)
• changes in mirage-block-unix between v2.5.0 and v2.6.0: 9 files changed, 76 insertions(+), 40 deletions(-)
• changes in mirage-block-xen between v1.4.0 and v1.5.0: 11 files changed, 4261 insertions(+), 3440 deletions(-)
• changes in mirage-bootvar-solo5 between v0.1.1 and v0.2.0: 24 files changed, 78 insertions(+), 7842 deletions(-)
• changes in mirage-bootvar-xen between v0.3.2 and 0.4.0: 24 files changed, 98 insertions(+), 8068 deletions(-)
• changes in mirage-channel between v1.1.1 and v3.0.0: 22 files changed, 511 insertions(+), 485 deletions(-)
• changes in mirage-console between v2.1.3 and 2.2.0: 63 files changed, 1364 insertions(+), 9188 deletions(-)
• changes in mirage-console-solo5 between v0.1.1 and v0.2.0: 26 files changed, 164 insertions(+), 7814 deletions(-)
• mirage-device is new in MirageOS 3: 13 files changed, 169 insertions(+)
• changes in mirage-entropy between 0.3.0 and 0.4.0: 34 files changed, 533 insertions(+), 8181 deletions(-)
• changes in mirage-flow between v1.1.0 and 1.2.0: 48 files changed, 1254 insertions(+), 8865 deletions(-)
• changes in mirage-fs between v0.6.0 and 1.0.0: 27 files changed, 476 insertions(+), 244 deletions(-)
• changes in mirage-fs-unix between v1.2.1 and 1.3.0: 41 files changed, 1075 insertions(+), 9477 deletions(-)
• changes in mirage between v2.9.1 and v3.0.0: 77 files changed, 2332 insertions(+), 11037 deletions(-)
• changes in mirage-http between 2.5.3 and 3.0.0: 14 files changed, 75 insertions(+), 49 deletions(-)
• mirage-kv is new in MirageOS 3: 18 files changed, 282 insertions(+)
• changes in mirage-logs between v0.2 and 0.3.0: 30 files changed, 563 insertions(+), 8250 deletions(-)
• mirage-net is new in MirageOS 3: 18 files changed, 345 insertions(+)
• changes in mirage-net-macosx between 1.2.0 and 1.3.0: 15 files changed, 184 insertions(+), 181 deletions(-)
• changes in mirage-net-solo5 between v0.1.1 and v0.2.0: 27 files changed, 266 insertions(+), 7969 deletions(-)
• changes in mirage-net-unix between v2.2.3 and 2.3.0: 26 files changed, 365 insertions(+), 8133 deletions(-)
• changes in mirage-net-xen between v1.4.2 and v1.7.0: 47 files changed, 6279 insertions(+), 4059 deletions(-)
• changes in mirage-platform between v2.6.0 and v3.0.0: 36 files changed, 7897 insertions(+), 7449 deletions(-)
• mirage-protocols is new in MirageOS 3: 18 files changed, 780 insertions(+)
• mirage-random is new in MirageOS 3: 16 files changed, 172 insertions(+)
• changes in mirage-solo5 between v0.1.1 and v0.2.0: 69 files changed, 1411 insertions(+), 9130 deletions(-)
• mirage-stack is new in MirageOS 3: 16 files changed, 254 insertions(+)
• changes in mirage-tcpip between v2.8.1 and v3.0.0: 228 files changed, 15376 insertions(+), 13301 deletions(-)
• mirage-time is new in MirageOS 3: 15 files changed, 198 insertions(+)
• changes in ocaml-conduit between v0.15.0 and v0.14.5: 14 files changed, 59 insertions(+), 76 deletions(-)
• changes in ocaml-crunch between v1.4.1 and 2.0.0: 34 files changed, 576 insertions(+), 8196 deletions(-)
• changes in ocaml-dns between v0.18.1 and v0.19.0: 24 files changed, 5526 insertions(+), 3824 deletions(-)
• changes in ocaml-fat between 0.11.0 and 0.12.0: 80 files changed, 3239 insertions(+), 12224 deletions(-)
• changes in ocaml-freestanding between v0.2.0 and v0.2.1: 3 files changed, 7 insertions(+), 1 deletion(-)
• changes in ocaml-qcow between v0.8.0 and v0.8.1: 2 files changed, 5 insertions(+), 2 deletions(-)
• changes in ocaml-nocrypto between v0.5.3 and v0.5.4: 44 files changed, 513 insertions(+), 576 deletions(-)
• changes in ocaml-tls between 0.7.1 and 0.8.0: 43 files changed, 415 insertions(+), 695 deletions(-)
• changes in mirage-os-shim between v0.0.1 and v3.0.0: 11 files changed, 62 insertions(+), 143 deletions(-)
• changes in ocb-stubblr between v0.1.0 and v0.1.1: 3 files changed, 21 insertions(+), 11 deletions(-)
• changes in solo5 between v0.1.1 and v0.2.0: 102 files changed, 5554 insertions(+), 9421 deletions(-)
• changes in mirage-qubes between v0.3 and 0.4: 28 files changed, 318 insertions(+), 1074 deletions(-)
• changes in charrua-client between 0.0.1 and 0.1.0: 24 files changed, 492 insertions(+), 442 deletions(-)