Index: head/en_US.ISO8859-1/htdocs/news/status/report-2015-07-2015-09.xml =================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2015-07-2015-09.xml (revision 47667) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2015-07-2015-09.xml (revision 47668) @@ -1,3685 +1,3685 @@ July-September 2015
Introduction

The third quarter of 2015, from July to September, was again a period of busy activity for &os;: for the second quarter in a row we have the largest report yet published.

The Foundation continues to play a strong role, bringing both a developer and evangelist presence to conferences, funding much of the hardware that the cluster administration team uses to keep things running, and sponsoring many development projects for &os;. This quarter we also hear from some of the student projects funded by Google Summer of Code 2015, ranging a wide gamut from the bootloader to additional ARM support, but also at a range of completion status. Some of the GSoC output is in the tree already, but others could benefit from additional attention to help out our budding new contributors as their schedules fill with the return to classes.

ZFS and the network stack continue to be strong areas for &os;, with both receiving active maintenance and feature improvements during this quarter. Substantial work continues on arm64, potentially putting it on the path toward a promotion to Tier-1 status, and a new port to the RISC-V architecture has made great headway in a short period of time. But it is not just our strengths and exciting new areas that have seen attention this cycle; there are also some parts of the system that are frequently perceived as unchanging infrastructure that have received attention and improvements, with truss and (k)gdb receiving significant overhauls, new implementations for the man page tools being brought in, the website receiving a new skin, and a brand new system for translating documentation that greatly lowers the barrier to entry.

Nonetheless, despite its record length, this report does not and cannot cover all of the work being done on &os; throughout the reporting period — there are many bug fixes too minor to mention here, and developers too busy working on the next project to write up an entry for the previous project. It is not just the developers committing to Subversion that comprise the ongoing activities of &os;, but also the users testing unreleased code or reporting bugs in released code, and participants on the mailing lists and forums helping each other solve their problems. Even the chats on IRC that wander far from the stated topic of a channel contribute to the community around &os;; it is that community whose effectiveness and helpfulness is a key component of the effectiveness and usefulness of &os; itself. Not just to the developers listed in this report, but to everyone in the community, thank you for making &os; a great operating system.

—Ben Kaduk


Please submit status reports for the fourth quarter of 2015 (from October to December) by January 7, 2016.

team &os; Team Reports proj Projects kern Kernel arch Architectures bin Userland Programs ports Ports doc Documentation gsoc Google Summer of Code misc Miscellaneous <tt>ioat(4)</tt> Driver Import Jim Harris jimharris@FreeBSD.org Conrad Meyer cem@FreeBSD.org Wikipedia article on IOAT Commit importing ioat(4)

A new driver, ioat(4), was added to the tree. ioat(4) supports Intel's I/O Acceleration Technology devices which are found on some Intel server systems.

These devices are DMA offload engines, which can accelerate some I/O-heavy applications by offloading memory copies from the main CPU to the I/OAT unit. This acceleration is not transparent; applications must be adapted to take advantage of the hardware.

Some I/OAT models support more advanced copying modes, such as XOR; these modes are not yet supported in the ioat(4) driver.

Intel Corporation EMC / Isilon Storage Division

Further testing, especially on a range of device models other than BDXDE (looking for volunteers here).

Support for the more advanced copy modes.

IPsec Upgrades George Neville-Neil gnn@FreeBSD.org John-Mark Gurney jmg@FreeBSD.org Ermal Luçi eri@FreeBSD.org

IPsec is now enabled by default in the GENERIC kernel configuration, and work is proceeding to speed things up in various ways. The latest changes are the addition, by &a.jmg;, &a.eri;, and &a.gnn;, of AES modes both in hardware and in software. Part of this work also includes more benchmarks undertaken using Conductor in the netperf project. Results have been reported at BSDCan and vBSDcon with more to come at EuroBSDcon and BSDCon Brasil.

Netgate The FreeBSD Foundation

Performance improvements and other tweaks are ongoing.

DTrace and TCP George Neville-Neil gnn@FreeBSD.org Commit adding trace points replacing TCPDEBUG

With the advent of DTrace we are able to replace many of the internal kernel debugging options, such as TCPDEBUG, with statically defined tracepoints (SDTs). Tracepoints have now been added to the system that replicate the functionality of the TCPDEBUG kernel option. No new kernel options need to be added — they are standard with any kernel that has DTrace, which is included in the default GENERIC kernels in 10.X and HEAD.

Limelight Networks
FreeBSD on the Acer C720 Chromebook Michael Gmelin freebsd@grem.de Blog post on how to get things working Blog post with links to commits in HEAD Backported patch for 10.2-RELEASE

The Acer C720 Chromebook is an affordable (under $200) and powerful little laptop that provides a battery life of up to six hours running &os;. It is a great machine for travelling and coding in general. The machine is fully functional, meaning that all essential devices work: keyboard, trackpad, light sensor, backlight control, display in VESA mode (fast), external Display on HDMI (only VESA mirror mode), sound, USB ports, SD card slot, camera, and Atheros wireless.

This quarter, this project extended previous work on the boot process and keyboard driver as well as the smbus(4) driver. It added three new drivers: ig4(4) for the I2C bus, cyapa(4) for the trackpad, and isl(4), for the ambient light sensor.

Much of the development was originally done in late 2014. Since then, the patches have been massively improved and merged into HEAD, so that all relevant devices work without manual patching.

For those who are unable to run HEAD, there is a backported patch to 10.2-RELEASE.

Thanks to everyone who helped in the process. I couldn't have done it without you (you know who you are).

Cavium LiquidIO Smart NIC Driver Stanislaw Kardach kda@semihalf.com Zyta Racia zr@semihalf.com LiquidIO product page

This project aims to add support for the LiquidIO family of high-performance programmable accelerator 10/40-gigabit Ethernet network adapters. The currently developed kernel driver supports CN6640- and CN6880-based PCIe cards, enabling these features:

The project is currently being developed in house and is being prepared for upstream. We plan on making it available in &os; 11.

Cavium Semihalf Upstream the code to &os; head.
&os; Release Engineering Team &os; Release Engineering Team re@FreeBSD.org &os; 10.2-RELEASE announcement &os; development snapshots &os; 10.3-RELEASE schedule &os; 11.0-RELEASE schedule

The &os; Release Engineering Team is responsible for setting and publishing release schedules for official project releases of &os;, announcing code freezes, and maintaining the respective branches, among other things.

In mid-August, the &os; Release Engineering Team released &os; 10.2-RELEASE, two weeks earlier than the original schedule anticipated.

The &os; Release Engineering Team would like to thank all that have tested the BETA and RC builds and reported issues during the release cycle.

The &os; Release Engineering Team, with approval from the &os; Core Team, appointed &a.marius; as the Deputy Lead.

The FreeBSD Foundation
Porting <tt>bhyve</tt> to ARM-based Platforms Mihai Carabas mihai@FreeBSD.org Peter Grehan grehan@FreeBSD.org Project Wiki page

This summer we have started porting bhyve onto ARMv7 platforms. The low-level routines for ARM processors were rewritten while trying to preserve the hypervisor API originally created for the x86 architectures. We managed to bring up a &os; guest up to the point of initializing interrupts. There is still work to be done in order to virtualize the interrupts and the timer. Our short-term plan after finishing the interrupts and the timer is porting to a real hardware platform (Cubie2).

Virtualize interrupts and timer.

Port to a real hardware platform.

Create SMP support for bhyve-on-arm.

Port to ARMv8.

Bringing GitLab into the Ports Collection Torsten Zühlsdorff ports@toco-domains.de Michael Fausten ports@michael-fausten.de PR for the new port Installation guide GitLab Source Tree

GitLab is a web-based Git repository manager with many features, used by more than 100.000 organizations, including NASA and Alibaba. It also is a very long-standing entry on the "Wanted Ports" list on the &os; Wiki.

In the last month there was steady progress, finally resulting in the PR for adding the new port. In addition to the many dependencies &a.pgollucci; is working on, there was already a large amount of work done. Along with many new or updated rubygems, Rails 4.1 was resurrected. A large group of committers were involved in the process and guided us through the various problems and pitfalls.

Because of the number of dependencies — we nearly hit 100 — making progress takes some time. In the meantime, a new major version of GitLab has already been released, requiring even more dependencies and updates. Work on this version is in progress, but the first goal is to get the latest stable version from the 7.14 branch into the ports tree.

anyMOTION GRAPHICS GmbH, Düsseldorf, Germany

Closing all the PRs of the dependencies

Committing the GitLab port itself

Updating the port to the latest version of the 8.x branch

Xfce on &os; &os; Xfce Team xfce@FreeBSD.org &os; Xfce Project &os; Xfce Repository

Xfce is a free software desktop environment for Unix and Unix-like platforms, such as &os;. It aims to be fast and lightweight, while still being visually appealing and easy to use.

During this quarter, the team has kept these applications up-to-date:

We also follow the unstable releases (available in our experimental repository) of:

In the trunk branch, x11-wm/xfce4-panel contains a patch to support sysutils/xfce4-panel-switch (available through the panel preferences).

Test the new stable release of GLib 2.46.x with the kqueue/kevent backend enabled (it was disabled with revision - r393663. + r393663). Currently several features are broken, especially in Thunar, xfce4-panel, and Xfdashboard.

<tt>Node.js</tt> Modules Olivier Duchateau olivierd@FreeBSD.org Node.js modules Pre-draft documentation

Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. It uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

The goal of this project is to make it easy to install the modules available in the npm package registry.

Currently, the repository contains more than 100 new ports, in particular:

We have also written several helpers for the porting, available in our experimental repository.

Bring in grunt.js (and modules), the JavaScript task runner.

-

Put more effort into support of node-gyp in the +

Put more effort into support for node-gyp in the USES framework

Root Remount Edward Tomasz Napierala trasz@FreeBSD.org Userland code review

A feature long missing from &os; was the ability to boot up with a temporary rootfs, configure the kernel to be able to access the real rootfs, and then replace the temporary root with the real one. In Linux, this functionality is known as pivot_root. The reroot project aims to provide similar functionality in a different, slightly more user-friendly, way. Simply put, from the user's point of view it is as simple as running reboot -r. The system performs a partial shutdown, killing all processes and unmounting the rootfs, and then partial bringup, mounting the new rootfs, running init, and running the startup scripts as usual.

The kernel part of the project has been committed to 11-CURRENT. The userland part is at the "finishing touches" stage, and is expected to be committed soon. A merge to stable/10 is planned and reroot support is planned be included in &os; 10.3.

The FreeBSD Foundation
Clang, <tt>llvm</tt>, <tt>lldb</tt>, <tt>compiler-rt</tt> and <tt>libc++</tt> Updated to 3.7.0 Dimitry Andric dim@FreeBSD.org Ed Maste emaste@FreeBSD.org Roman Divacky rdivacky@FreeBSD.org Davide Italiano davide@FreeBSD.org LLVM 3.7.0 Release Notes Clang 3.7.0 Release Notes PR 201377 Ports exp-run

We have updated clang, llvm, lldb, compiler-rt, and libc++ in base to the 3.7.0 release. These all contain numerous improvements. Please see the linked release notes for more detailed information. This brings us completely up-to-date with the latest upstream versions of these projects. Meanwhile, &a.emaste; is working on importing the llvm.org version of libunwind.

Like the 3.5.x and 3.6.x releases, these components require C++11 support to build. At this point, &os; 10.0 and later provide that support, at least on x86. Currently, there are no solid plans to MFC these versions to any stable branches, due to the difficulties this would introduce for the usual upgrade scenarios.

Thanks to &a.emaste; and &a.andrew; for their help with this import, and thanks to &a.antoine; for several ports exp-runs.

During the first ports exp-run, some major problems were found, one introduced by a clang bug which caused pow() to generate floating point exceptions in some cases. This in turn caused libpng to fail to build, and one bug in libjpeg-turbo, which was caused by undefined behavior. These two problems took some time to fix, after which another exp-run was done, and this resulted in about a dozen newly failed ports. For almost all of these new failures, fixes were submitted and linked to the original PR 201377 for the exp-run.

Commit ports fixes for dependencies of PR 201377. Test and report issues with the new tool chain.
UEFI Boot and Framebuffer Support Ed Maste emaste@freebsd.org Marcel Moolenaar marcel@freebsd.org

A number of UEFI bug fixes were committed over the last quarter, improving compatibility with different UEFI implementations. This includes improvements to EFI's vt(4) framebuffer driver, efifb, to handle systems with high resolution displays and unusual framebuffer stride values. In particular, this improves compatibility with a large number of recent Apple MacBook Pros and other Macs.

Test &os; head and &os;-STABLE snapshots on a variety of UEFI implementations.
The Graphics Stack on &os; &os; Graphics team freebsd-x11@FreeBSD.org Graphics stack roadmap and supported hardware matrix Graphics stack team blog Ports development tree on GitHub

The Mesa ports were updated to 10.6.8. At the same time, the ports received a major overhaul to make sure all ports are correctly configured. Dual version support was removed. There is only one Mesa version for all supported &os; versions. The libosmesa port, which provided the off-screen version of Mesa, was merged into the Mesa framework.

Another big item that was included in the Mesa port is OpenCL. There are two GPU-based OpenCL implementations: lang/clover for supported Radeon cards, and lang/beignet for supported Intel cards (currently only Ivybridge). Thanks go to Johannes Dieterich, O. Hartmann, and Koop Mast for making this happen.

Now that Mesa is up-to-date, we can apply the same update procedure to the X.Org server. It is currently at 1.14, and an update to 1.17 is ready. It will be committed shortly.

On the kernel side, progress has been made with the i915 update. The driver is able to attach. There are some reports that the X.Org server starts but Mesa is unhappy, so acceleration does not work yet. If you want to test, instructions will be posted on the wiki in the i915 update article (see links). At this stage, we can only accept patches, though — we will not be able to provide support.

We attended two conferences: XDC 2015 in Toronto and EuroBSDcon 2015 in Stockholm. Reports will be posted on the blog.

See the Graphics wiki page for up-to-date information.

The FreeBSD Foundation Deb Goodkin deb@FreeBSDFoundation.org Foundation website &os; Journal

The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the &os; Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage development projects, conferences and developer summits, and provide travel grants to &os; developers. The Foundation purchases hardware to improve and maintain &os; infrastructure and publishes &os; white papers and marketing material to promote, educate, and advocate for the &os; Project. The Foundation also represents the &os; Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.

Here are some highlights of what we did to help &os; last quarter:

Anne Dickison and Deb Goodkin attended OSCON to promote &os;.

&a.rwatson; organized and ran the Cambridge &os; Developer Summit 2015 ("BSDCam"). We provided travel grants to two &os; developers to attend the summit. Three Foundation board/staff members attended too.

&a.gnn; attended the ARM Partner Meeting where he met with 15 silicon and systems vendors to present the unique traits and qualities of &os; and work on setting up partnerships with the companies building and deploying ARM hardware.

George and &a.rwatson; collaborated in Cambridge on developing further &os;-based teaching material at undergraduate and masters levels. Part of this project was funded by the Foundation.

George planned and ran the DevSummit at vBSDCon 2015.

We were proud to be a sponsor of vBSDCon 2015, Sept 11-13 in Washington DC. &a.gnn; and &a.emaste; presented "Supporting a BSD Project" at the conference. &a.dru;, &a.gjb;, &a.gnn;, and &a.emaste; attended and represented the Foundation at both vBSDCon and the &os; Developer Summit that preceded it. We had many people stop by our table to make a donation, and it was another great opportunity to talk and work with people face-to-face.

Cheryl Blain and &a.jhb; promoted the Foundation and &os; at the SNIA 2015 Storage Developer Conference, in Santa Clara, California, Sept 21-24. The Foundation was also a sponsor.

We sponsored Andy Turner to attend Linaro Connect in San Francisco, Sept 21-25.

&a.emaste;, our project development director, attended the X.Org Developer's Conference (XDC) in Toronto, Ontario.

We sponsored the 2015 nginx Conference and sent &os; community member &a.jhb;.

George Neville-Neil continued planning the 2015 Silicon Valley Vendor Summit, including securing the venue.

&a.bcr; and &a.erwin; helped plan and organize the EuroBSDCon &os; Developer Summit. This included setting up the working groups, securing the venue, and getting the T-shirts made.

Benedict helped organize, and he and &a.dru; participated in the &os; Hackathon in the Linuxhotel in Essen, Germany. It was a successful weekend of fixing bugs and collaborating with others.

&a.dru; taught a &os; class in Berlin, Germany July 29-31.

We were a sponsor of womENcourage 2015, in Uppsala Sweden, Sept 24-26. Dru was the moderator for a panel on Open Source as a Career Path. All the panelists were &os; contributors including Dan Langille, Allan Jude, Benedict Reuschling, and Deb Goodkin. We also had a table at the job fair and talked to a lot of students and professors about the benefits of working on &os; as an alternative to an internship, teaching about &os; in university classes, and hosting &os; events at their schools. Dan taught a workshop on How to Contribute to an Open Source project. Deb participated in this workshop and started a discussion on offering a similar workshop at BSD and non-BSD conferences. The workshop would be titled "How to Contribute to &os;", and participants would learn how to contribute documentation to the Project.

We continued to publish our monthly newsletters, keeping the community informed on what we are doing, including event recaps, testimonials, project updates, and upcoming events. We received testimonials from Microsoft, NYCBus, and ScaleEngine. We also continued to approach companies to provide us with testimonials to help promote their use of &os;.

Anne Dickison rebooted the Faces of &os; series and is working with &os; contributors on writing their stories. She continued to produce more &os; Swag and literature to promote &os;, as well as advocating for &os; over our social channels and with new partnerships.

We reached our 2015 goal of 10,000 &os; Journal subscribers, and we published a new Open Journal article on our website, to help promote the Journal. We also started offering a new subscription bundle, where you can buy all the 2014 issues. The July/August issue was published.

&a.gibbs; began teaching a semester-long &os; class at a middle school in Boulder, Colorado. We are using the BeagleBone Black (BBB) to run &os; connected to Macs and PCs. We have received a lot of support, both internally, and from the Project, to get the &os; images to work on the BBB with the Macs and PCs. It has been a great collaborative effort with community members, and this will help future classes in being able to support inexpensive platforms for teaching &os;.

Work continued on creating a &os; curriculum for a half day workshop. Hopefully this will be available in late Spring.

We provided legal support for the Project including granting trademark permission for some users and companies who requested permission to put the &os; logo on their websites and marketing literature.

We met with commercial users to get their input on what they would like to see supported in &os;. We also do this to help connect &os; developers with commercial users to help facilitate collaboration.

&os; Foundation employee and Release Engineer &a.gjb; was extremely busy during this quarter, working on a number of exciting areas of the &os; Project. Some of the highlights include:

Multiqueue Testing Tiwei Bie btw@FreeBSD.org Hiren Panchasara hiren@FreeBSD.org George Neville-Neil gnn@FreeBSD.org Robert Watson rwatson@FreeBSD.org Project Wiki Page

The aim of this project is to design and implement infrastructure to validate that a number of the network stack's multiqueue behaviours are functioning as expected.

At present, most of this project has been implemented. It mainly consists of two parts:

  1. A general mechanism to collect the per-ring per-cpu statistics that can be used by all NIC drivers, and extensions to netstat(1) to report these statistics.
  2. A suite of network stack behavior testing programs that consists of:
    • a virtual multiqueue ethernet interface (vme)
    • a UDP packet generator based on vme
    • a UDP server based on socket(2)
    • a TCP client based on lwip and vme
    • a TCP server based on socket(2)

However, it still needs further refinements to make it suitable for committing to &os; head.

Google Summer of Code 2015
GNOME on &os; &os; GNOME Team freebsd-gnome@FreeBSD.org &os; Gnome website Devel repository Upstream build bot USE_GNOME Porter's Handbook chapter

The &os; GNOME Team maintains the GNOME, MATE, and CINNAMON desktop environments and graphical user interfaces for &os;. GNOME 3 is part of the GNU Project. MATE is a fork of the GNOME 2 desktop. CINNAMON is a desktop environment using GNOME 3 technologies but with a GNOME 2 look and feel.

This quarter, GNOME 3.16 and MATE 1.10 were committed to the ports tree, followed up by some incremental improvements. A chapter covering the use of USE_GNOME within individual ports' Makefiles was written and committed to the Porter's Handbook.

GNOME 3.18 has been ported. There are, however, some issues that need to be resolved before it can be committed to the ports tree.

The &os; GNOME website is stale. Work is under way to improve it.

Please give feedback on and suggest improvements to the chapter in the Porter's Handbook on the USE_GNOME functionality.

Continue working on investigating the issues blocking GNOME 3.18.

Atomics Konstantin Belousov kib@FreeBSD.org Alan Cox alc@FreeBSD.org Bruce Evans bde@FreeBSD.org

Atomic operations serve two fundamental purposes. First, they are the building blocks for expressing synchronization algorithms in a single, machine-independent way using high-level languages. In essense, atomics abstract the different building blocks supported by the various architectures on which &os; runs, making it easier to develop and reason about lock-less code by hiding hardware-level details.

Atomics also provide the barrier operations that allow software to control the effects on memory of out-of-order and speculative execution in modern processors as well as optimizations by compilers. This capability is especially important to multithreaded software, such as the &os; kernel, when running on systems where multiple processors communicate through a shared main memory.

Each machine architecture defines a memory model, which specifies the possible effects on memory of out-of-order and speculative execution. More precisely, it specifies the extent to which the machine may visibly reorder memory accesses to optimize performance. Unfortunately, there are almost as many models as architectures. Some architectures, for example IA32 or Sparcv9 TSO, are relatively strongly ordered. In contrast, others, like PowerPC or ARM, are very relaxed. In effect, atomics define a very relaxed abstract memory model for &os;'s machine-independent code that can be efficiently realized on any of these architectures.

Most &os; development and testing still happens on x86 machines, which, when combined with x86's strongly ordered memory model, leads to errors in the use of atomics, specifically, barriers. In other words, the code is not properly written to &os;'s abstract memory model, but the strong ordering of the x86 architecture hides this fact. The architectures impacted by the code that incorrectly uses atomics are less popular or have limited availability, and the resulting bugs from the misuse of atomics are hard to diagnose.

The goal of this project is to audit and upgrade the usage of lockless facilities, hopefully fixing bugs before they are observed in the wild.

&os; defines its own set of atomics operations, like many other operating systems. But unlike other operating systems, &os; models its atomics and barriers on the release consistency model, which is also known as acquire/release model. This is the same model which is used by the C11 and C++11 language standards as well as the new 64-bit ARM architecture. Despite having syntactical differences, C11 and &os; atomics share essentially the same semantics. Consequently, ample tutorials about the C11 memory model and algorithms expressed with C11 atomics can be trivially reused under &os;.

One facility of C11 that was missing from &os; atomics was fences. Fences are bidirectional barrier operations which could not be expressed by the existing atomic+barrier accesses. They were added in r285283.

Due to the strong memory model implemented by x86 processors, atomic_load_acq() and atomic_store_rel() can be implemented by plain load and store instructions with only a compiler barrier. No additional ordering constraints are required. This simplification of atomic_store_rel() was done some time ago in r236456. The atomic_load_acq() change was done in r285934, after careful review of all its uses in the kernel and user-space to ensure that no hidden dependency on a stronger implementation was left.

The only reordering in memory accesses which is allowed on x86 is that loads may be reordered with older stores to different locations. This results from the use of store buffers at the micro-architecural level. So, to ensure sequentially consistent behavior on x86, a store/load barrier needs to be issued, which can be done with an MFENCE instruction or by any locked read-modify-write operation. The latter approach is recommended by the optimization guides from Intel and AMD. It was noted that careful selection of the scratch - memory location, which is modified by the locked RWM - operation, can reduce the cost of barrier by avoiding false + memory location, which is modified by the locked RMW + operation, can reduce the cost of the barrier by avoiding false data dependencies. The corresponding optimization was committed in r284901.

The atomic(9) man page was often a cause of confusion due to both erroneous and ambiguous statements. The most significant of these issues were addressed in changes r286513 and r286784.

Some examples of our preemptive fixes to the misuse of atomics that would only become evident on weakly ordered machines are:

The FreeBSD Foundation (Konstantin Belousov's work)
FreeBSD/RISC-V Port Ruslan Bukin br@FreeBSD.org Arun Thomas arun.thomas@baesystems.com Ed Maste emaste@FreeBSD.org &os; wiki RISC-V Single user boot log

RISC-V is an open source Instruction Set Architecture (ISA) designed at UC Berkeley. It is freely available for all uses without requiring fees or license agreements. The RISC-V team intends to provide freely available BSD licensed CPU designs.

&a.br; (University of Cambridge) now has &os; booting to a single user shell on a RISC-V simulator. The porting effort started only two months ago and is very much a work in progress, requiring significant refactoring and clean up before it reaches a committable state. Nonetheless, this is exceptional progress in a short time. The porting effort also identified a number of proposed ISA improvements.

The port currently uses the GNU tool chain (GCC and binutils), and runs on the Spike simulator. Improved RISC-V support in Clang/LLVM and related tools is highly desired.

DARPA, AFRL
Ports on PowerPC Alexey Dokuchaev danfe@FreeBSD.org

The Ports Collection typically receives less attention on Tier-2 architectures than on Tier-1 architectures, although several build-runs were performed at various points in the past, and broken ports were marked as such at those times.

Some of the Tier-2 platforms, such as PowerPC and ARM, have improved considerably recently, both on &os;'s and the compilers' sides, but as the tree is not rebuilt on the cluster very often, it was suspected that many ports are marked BROKEN while they in fact now build and run correctly.

Over the past several weeks, 26 ports that were indeed broken on at least PowerPC had been fixed, 58 ports that were incorrectly marked as broken (leftovers from the old times) were marked as working, and fewer than 40 ports still have issues requiring further work.

The Ports Collection could benefit a lot from more frequent sweeps targeting Tier-2 systems.

Recent work on QEMU-backed emulators and the much-anticipated cross-building of ports are essential pieces to bring &os; packages on par with the base system's support, architecture-wise.

The &os; Core Team &os; Core Team core@FreeBSD.org

The biggest task handled by the Core Team during this quarter was developing and publishing the new Code of Conduct. The Code of Conduct describes how people are expected to behave on all &os; official communication channels, as well as how developers and other people involved with the project are to behave when representing the project in public.

The Code of Conduct was generally well received and elicited numerous comments and suggestions for improvements from the community, many of which have been integrated.

The next task handled by Core was the restoration of Babak Farrokhi's ports commit bit. Babak resides in Iran. A few years ago, legal advice suggested that allowing contributions from Iranian residents might violate US trade sanctions. After several years, Core was asked to revisit the issue. On the advice of counsel, Core decided that it could restore commit privileges to commmitters residing in Iran.

The CTM service came under security review. Given that the lack of use of routine authenticity checking made the injection of trivial exploit code relatively easy, the service was held to be too risky to continue as an official part of the &os; base system. CTM has very few remaining users but they should be able to install CTM from the Ports Collection in order to continue doing so.

Core learned that ISC was ceasing its hosting service, which has entailed a rapid rework of plans on the movement of significant portions of the &os; cluster to that data center. Cluster administration has taken ownership of the situation and is making progress.

Core fielded an enquiry about NextBSD and whether this should be the future direction for the whole &os; project. Core's position is that NextBSD is an interesting project, and we regard it, like the other BSD projects, as a potential source of good ideas. However, we currently have no plans to adopt NextBSD as the official &os; distribution.

Beyond these issues, Core also spent time in various routine activities. During this quarter we issued three new src commit bits, and took none in for safekeeping. Welcome to &a.allanjude;, &a.araujo;, and &a.avos;.

Ports Collection Frederic Culot portmgr-secretary@FreeBSD.org &os; Ports Management Team portmgr@FreeBSD.org Ports Collection website Contributing to the Ports Collection Port Monitoring service Team Website Blog Twitter feed Facebook page Google+ page

As of the end of Q3 the ports tree holds just over 25,000 ports, and the PR count is above 2,000. The summer period saw less activity on the ports tree than during the previous quarter, with fewer than 7,000 commits performed by 120 active committers. Unfortunately, the number of problem reports closed also decreased significantly, with fewer than 1,500 problem reports fixed during Q3.

In Q3 several commit bits were taken in for safekeeping, following an inactivity period of more than 18 months (fluffy), or on committer's request (xmj, stefan, brix). One new developer was granted a ports commit bit (&a.junovitch.email;), and one returning committer (&a.farrokhi;) had his commit bit reinstated.

On the management side, no changes were made to the portmgr team during Q3.

On the QA side, 25 exp-runs were performed to validate sensitive updates or cleanups. Amongst those, the noticeable changes are the update to pkg 1.6, the automake14 removal, and several important port updates such as doxygen to 1.8.10, gnome3 to 3.16, cmake to 3.3.1, and the Qt4 ports to 4.8.7. The default jdk was also set to openjdk8. Some infrastructure changes included the addition of new options helpers: opt_VARS, opt_VARS_OFF, opt_IMPLIES, and opt_PREVENTS. Some macros were also removed, such as UNIQUENAME and LATEST_LINK.

We would like to remind everyone that the ports tree is built and run by volunteers, and any help is greatly appreciated. This is more important than ever, since the number of problem reports cannot seem to stop increasing. So if you use ports or packages, please consider jumping in and helping! This is also true for existing porters: it would be great if you would consider the next step, which is to share your knowledge and mentor someone more junior with the ports tree internals. And if you already do these tasks, many thanks to you!

&os; Cluster Administration Team &os; Cluster Administration Team clusteradm@

The &os; Cluster Administration Team consists of the people responsible for administering the machines that the project relies on for its distributed work and communications to be synchronised.

Our primary cluster has been hosted as a guest in California for many years. Our ongoing project is relocating the core functionality to a location in New Jersey with a formal hosting arrangement. This is an equipment refresh, consolidation for better use of resources, and for better continuity of service.

There is a significant amount of behind-the-scenes work to make this happen. The original cluster was implemented with a common, shared, assumed-to-be secure network with ubiquitous NFS everywhere. This structure does not lend itself well to being distributed across geographically diverse locations, particularly when Internet transit is required. The bulk of the work is rebuilding services to be portable, stand-alone components that do not depend on shared-network access and are safe enough to use across the insecure Internet.

Highlights this quarter:

Ongoing:

KDE on &os; KDE on &os; team kde@FreeBSD.org KDE on &os; website KDE ports staging area KDE on &os; wiki KDE/&os; mailing list Development repository for integrating KDE 5

Overall, we have updated the following ports this quarter:

In our development repository, we have done the following work:

Work on getting the stuff from plasma5 branch into ports. (This is a major update to nearly all KDE applications, so testers are very welcome.)

Finalize the work on PyQt5.

Port qt5-webengine. Qt-5.5 will probably be the last release shipping a www/webkit-qt5 port.

&os;/arm64 Andrew Turner andrew@FreeBSD.org Ed Maste emaste@FreeBSD.org &os; arm64 wiki page

Numerous cleanups and fixes have been applied to the arm64 kernel. This includes fixes to exception handling, asynchronous signals, ddb, and pmap. ddb has been updated to better handle accessing memory that may be unmapped. The pmap code was made more complete by implementing more functions as needed.

Further work on SMP means that &os; now boots on all 48 cores on the Cavium ThunderX platform. This includes adding support for the ARM GICv3 interrupt controllers and fixing the memory mapping to be shareable between CPUs.

The test suite has been run on both qemu and hardware. Most of the test cases are passing, with around 30 tests either broken or failing. Work on diagnosing the issues with the remaining test cases is ongoing.

The FreeBSD Foundation ABT Systems Ltd

Port to more SoCs.

&os; on the HiKey ARMv8 Board Andrew Turner andrew@FreeBSD.org HiKey wiki entry Hardware description

The HiKey is a low-cost ARMv8 development board from the Linaro 96boards initiative. It contains a HiSilicon Kirin 6220 with eight ARMv8 cores and 1GB of ram.

&os; has been ported to run on the HiKey with a minimal set of drivers. As of this report, &os; supports the micro-SD slot and USB host, and will boot off the SD card to multi-user mode using a recent arm64 snapshot.

The kernel is missing a number of device drivers. However, it is at a usable state for people interested in testing &os; on ARMv8 hardware.

ABT Systems Ltd ARM Ltd

A driver for SDIO and the onboard WiFi.

Fix the MMC driver to access the eMMC.

Support the USB in OTG mode.

Support a display via HDMI.

Add thermal management.

PO Translation Project &os; Documentation Team freebsd-doc@FreeBSD.org &os; Translators freebsd-translators@FreeBSD.org Warren Block wblock@FreeBSD.org PO Translations German translation of the Leap Seconds article Dutch translation of the Explaining BSD article &os; Translators mailing list

The &os; documentation translation process has been in need of modernization for quite some time. The existing process was just too difficult for translators to keep translations up to date.

With help from Benedict Reuschling, Shaun McCance, Ryan Lortie, Hiroki Sato, and many others, the availability of a new PO translation system was announced in August.

PO translations handle most of the overhead of the translation process. Translators do not have to keep track of commits to the upstream English version. The actual work of translating is quicker and easier. PO editors show how much of the document has been translated. If a translation is already available for a given string, it can be easily reused.

Early testing has been very successful. Most issues involve discovering and documenting the new processes rather than fixing bugs. New translations of English documents have already been committed.

There will certainly be additional changes and improvements, but the system works. We will continue to discover how to share work between translation teams and the project as a whole. This work will be much easier now that the initial hurdle of being able to use PO software has been passed.

Continue testing. The system is new to us and there are bound to be bugs and situations with unexpected results.

Improve documentation on using the new PO translations. Much of this involves things that rarely happened with the old system, like adding a completely new language directory.

Add new translations for existing documents. There is much less work to create and update a translated document now. Existing and new translators are working on adding and updating translations of the English documentation.

Figure out how to generate and share translation memory with other members of a language team or translators outside the team.

Test new PO editors like Pootle and Virtaal.

Determine a method to allow translators commit access for translations.

Develop and test code to translate manual pages.

Website CSS Update Warren Block wblock@FreeBSD.org &os; Main Site

The &os; website has remained essentially unchanged in appearance for many years. Like other legacy systems, it is difficult to change. It is heavily used and therefore subject to non-trivial bikeshedding.

The CSS shrunk the reader's font from the size they had requested. It specified hardcoded font and object sizes in pixels. On wide monitors, only the middle third of the screen was used. Hardware has changed from what existed when this version of the site was created. Screens have become larger and wider, and increased in resolution at the same time.

It was time for a change. Font sizes were set to percentages, with none smaller than 100%. The width of the main box was changed to 90%. Other small adjustments were added. These limited changes produced a rendered site that better respects the reader's settings, is much easier to read, and shows more information.

Although no content changed, the appearance was so different that some viewers thought we had redesigned the site. It is gratifying to know that so many people are using it. We would also like to thank people for the response, which was overwhelmingly positive and hardly bikesheddy at all.

Fix other outdated assumptions in the CSS. Alternately, rework the entire site. However, that is a much more complex and ambitious project than it might seem.

<tt>pkg</tt> 1.6 &os; pkg Team pkg@FreeBSD.org

pkg 1.6.0 has been released. Many changes have been made since pkg 1.5:

Add a notion of priority to the list of files to ensure that certain files are the first to be replaced. This was a blocker for packaging base.

Investigate replacing openssl by mbedtls.

&os; Xen Roger Pau Monné royger@FreeBSD.org Julien Grall julien.grall@citrix.com &os; PVH DomU wiki page &os; PVH Dom0 wiki page Porting &os; as a Xen ARM guest

Xen is a hypervisor using a microkernel design, providing services that allow multiple computer operating systems to execute on the same computer hardware concurrently. Xen support for &os; on x86 as a guest was introduced in version 8 and ARM support is currently being worked on. Support for running &os; as an amd64 Xen host (Dom0) is available in HEAD.

On the x86 front, most of the work during this quarter focused on the implementation of PVH inside Xen. Consequently, most of the activity happened inside of the hypervisor. Patches for a clean PVH implementation have been posted, with the aim of having them merged in the next Xen release (4.7). Once that is done, work will continue adding new features to both &os; and Xen to have feature parity with traditional PV guests/hosts.

Apart from this, work is ongoing to import a new netfront from Linux in order to support new features, like split event channel and multiple queue support.

On the ARM front, this quarter's work focused on getting &os;/arm64 booting as a Xen guest. The current activity is to upstream patches preparing Xen drivers to support arm64. This includes a rework of the console driver.

Citrix Systems R&D

Generalize the event channel code so it can be used on ARM.

Improve backend (netback, blkback) performance.

Work with upstream Xen to improve PVH and make it stable.

Improve the generic bounce buffer code for unmapped bios in order to support the alignment requirements of the blkfront driver.

<tt>bhyve</tt> Peter Grehan grehan@FreeBSD.org Neel Natu neel@FreeBSD.org Tycho Nightingale tychon@FreeBSD.org Allan Jude freebsd@allanjude.com Michael Dexter editor@callfortesting.org bhyve FAQ and talks NE2000 device emulation GSoC project Porting bhyve to ARM GSoC project ptnetmap support in bhyve GSoC project Windows support Illumos support

bhyve is a hypervisor that runs on the &os;/amd64 platform. At present, it runs &os; (8.x or later), Linux i386/x64, OpenBSD i386/amd64, NetBSD/amd64, Illumos, and Windows Vista/7/8/10/2008r2/2012r2/2016 x64 guests. Current development is focused on enabling additional guest operating systems and implementing features found in other hypervisors.

A combined bhyve and ZFS BoF was held during vBSDCon 2015, hosted by Michael Dexter and Allan Jude. Questions asked about bhyve included live migration and suspend/resume support, and configurations using ZFS.

Three bhyve-related projects were selected for GSoC 2015: NE2000 device emulation, porting bhyve to ARM, and ptnetmap support.

The major enhancement for bhyve this quarter was support for external firmware, along with a port of the Intel edk2 UEFI firmware. This allows bhyve to run Windows in headless mode, and also Illumos.

Improve the documentation.

bhyveucl is a work-in-progress script for starting bhyve instances based on a libUCL config file. More information at https://github.com/allanjude/bhyveucl.

Add support for virtio-scsi.

Flexible networking backends: wanproxy, vhost-net.

Support running bhyve as non-root.

Add filters for popular VM file formats (VMDK, VHD, QCOW2).

Implement an abstraction layer for video (no X11 or SDL in base system).

Suspend/resume support.

Live migration.

Nested VT-x support (bhyve in bhyve).

Support for other architectures (ARM, MIPS, PPC).

High Availability Clustering in CTL Alexander Motin mav@FreeBSD.org

CAM Target Layer (CTL), when originally developed by Copan/SGI, had support for High Availability clustering. Unfortunately, significant portions of the HA code were never published with the main body of the source code. Now, the missing part has been reimplemented from scratch, fixed, and improved.

This code supports dual-node HA with Asynchronous LUN Unit Access (ALUA) in four modes:

In the case of lost interlink connectivity to primary node, the secondary node falls into the Transitioning state, which is like Unavailable and cannot answer most requests, but makes the initiator wait for recovery or cluster failover.

CTL also got a large number of other improvements, including support for emulation of CD/DVD drives and removable disks, live LUN reconfiguration, and so on.

The code is committed to &os; head and was recently merged to the stable/10 branch.

iXsystems, Inc.
ZFS Code Sync with Latest Illumos Alexander Motin mav@FreeBSD.org

The ZFS codebase received a significant batch of merges, and is now in sync with the latest Illumos. Among other things, this update includes:

The code is committed to &os; head and was recently merged to the stable/10 branch.

<tt>mandoc</tt> and <tt>roff</tt> Toolchain Baptiste Daroussin bapt@FreeBSD.org Heirloom doctools mandoc

mandoc is a suite of tools for compiling mdoc, the roff macro language of choice for BSD manual pages.

mandoc is the default renderer for manpages on &os; head. This quarter, the apropos(1) utility was switched to use mandoc's version, which offers a new database format (in SQLite) bringing more powerful, fine-grained ways to search man pages.

While mandoc is very good for man pages, we also provide lots of other documentation in plain roff format. The Heirloom toolchain is being studied to replace groff in base. The Heirloom nroff toolchain has multiple benefits: it has very good unicode support and very good compatibility with groff.

A great deal of work as been done testing the Heirloom nroff toolchain with all the roff documents in the base system (including man pages), and upstream has been very proactive in fixing reported bugs.

The soelim(1) utility has been replaced with a BSD-licensed version which is good enough to work with all available roff toolchains to ease the transition. This version of the soelim(1) utility, originally written solely for &os;, is now part of the mandoc tool suite.

In coordination with Ingo Schwarze from OpenBSD, the col(1) utility has been cleaned up and updated to recognize both SUSv2-style escape-digit and BSD-style escape-control-char sequences in the input stream.

The checknr(1) utility has been cleaned up and extended to support modern roff(7) macros, including synchronizing code from NetBSD and the Heirloom doctools version.

Many roff fixes were made to documentation and man pages, having been discovered while testing the new toolchain.

Updates to GDB John Baldwin jhb@FreeBSD.org Extend libkvm to support cross-debugging of vmcores

Support for following children after forks for &os; was implemented and merged upstream to GDB's master branch, and was included in GDB 7.10.

Work has continued on porting kgdb to newer gdb. The amd64, i386, powerpc, powerpc64, and sparc64 backends have all been ported and are now available via a new KGDB option in the devel/gdb port.

The MD backends for libkvm have been rewritten to support cross-debugging crashdumps, and the kgdb targets for amd64 and i386 have been reworked to support cross-debugging. Both i386 and amd64 kgdb binaries have been able to cross-debug the other architecture's vmcores with these changes. This changeset for libkvm is not yet in the tree, but is awaiting more testing.

Test the libkvm changes on platforms other than amd64, i386, and powerpc64.

Figure out why the powerpc kgdb targets are not able to unwind the stack past the initial frame.

Add support for more platforms (arm, mips, aarch64) to upstream gdb for both userland and kgdb.

Write a new 1:1-only thread target for &os; that can be sent upstream.

Add support for debugging powerpc vector registers.

<tt>truss(1)</tt> John Baldwin jhb@FreeBSD.org Bryan Drewery bdrewery@FreeBSD.org

The interface between the ABI-specific backends and the truss core was refactored, reducing duplicated code. This prompted additional follow-on work to add support for more ABIs, including aarch64 and CloudABI.

ptrace(2) was extended to return more information about the currently executing system call. This restored behavior that had been present in a previous version of truss: knowing the correct number of arguments for all system calls.

The fork-following support in truss was reworked to use native fork following in ptrace(2) rather than forking a new truss process for each child of a traced process.

Support for decoding more arguments has been added in the last quarter as well.

Create a new libsysdecode library to hold shared code between truss(1) and kdump(1).

Decode more system call arguments.

Add appropriate system call decoding specifications for freebsd32 system calls.

Implement an ABI for 64-bit Linux binaries under &os;/amd64.

<tt>sesutil(8)</tt> Baptiste Daroussin bapt@FreeBSD.org Allan Jude allanjude@FreeBSD.org Wikipedia: SCSI Enclosure Services (SES)

sesutil(8) was originally created as a more universal way to blink the "locate" LEDs on most hot-swappable drive enclosures.

This work is based on the original SES tools created by Matthew Jacob in 2000, which have been available in the share/examples section of the source tree, but were not built by default.

The new utility extends the original code with a number of very useful features:

Gandi ScaleEngine Inc.

Test sesutil(8) against more hardware.

Diagnose an issue where the locate command sometimes needs to be sent twice to activate the LED.

Add support for libxo output types.

&os; on Cavium ThunderX (arm64) Dominik Ermel der@semihalf.com Wojciech Macek wma@semihalf.com Zbigniew Bodek zbb@semihalf.com

Cavium’s ThunderX is a high-performance 64-bit ARMv8 CPU, available in configurations with up to 48 cores per package. ThunderX is the initial reference platform for the &os;/arm64 porting effort.

Additional Semihalf-sponsored work on ThunderX support brought brand new features such as:

Moreover, previously introduced functionalities have been improved and committed to HEAD. This includes:

The remaining features are being reviewed and will be integrated into HEAD soon. However, the GENERIC kernel already supports and runs on ThunderX.

The FreeBSD Foundation ARM Ltd. Cavium Semihalf

Upstream the remaining features: 2-socket support, VNIC driver, and PCIe fixes

Multipath TCP for &os; Nigel Williams njwilliams@swin.edu.au MPTCP for &os; Project Website MPTCP for &os; Source Repository

Multipath TCP (MPTCP) is an extension to TCP that allows for the use of multiple network interfaces on a standard TCP session. The addition of new addresses and scheduling of data across them occurs transparently from the perspective of the TCP application.

The goal of this project is to deliver an MPTCP kernel patch that interoperates with the reference MPTCP implementation, along with additional enhancements to aid network research.

The v0.5 patch was released, which is the first patch of the re-written implementation. We are in the process of documenting the new design and addressing some feedback as provided from the community.

Work has commenced on improved input handling, as the current method of receiving and reassembling segments has been the cause of some instability and stalls during connection shutdown. This will involve re-using the subflow receive buffers and an upcall to enqueue a MP-layer reassembly task without the need to take a lock on the MP control block. The improvements should also allow bypassing mptcp_usrreq for standard TCP connections.

The MPTCP commit history was synchronized with hg-beta.FreeBSD.org, and we have made the repository available on BitBucket (see links). Future patch releases will be tagged there. The tree is now merged with &os; head weekly. An updated v0.51 patch is ready for release.

The FreeBSD Foundation

Release the v0.51 patch.

Populate documentation and the issue tracker on the BitBucket repository.

Improvements to receive-side code before further testing.

Prepare a technical report detailing the design of the current patch.

CloudABI: Pure Capabilities Runtime Environment Ed Schouten ed@FreeBSD.org CloudABI project page CloudABI Ports Collection CloudABI presentation at FrOSCon

CloudABI is a POSIX-like runtime environment that uses Capsicum as its sole access control mechanism. CloudABI allows you to develop software that is better hardened against security vulnerabilities, is easier to test, and is easier to migrate across systems.

As of August, all of the kernel modifications that are needed to run CloudABI programs have been integrated into &os; head. After loading the cloudabi64 kernel module, you can either run CloudABI programs directly from the shell or by using the cloudabi-run tool (sysutils/cloudabi-utils). cloudabi-run allows you to inject sockets, files, and directories into the launched program in a more structured way.

In the meantime, work has started on developing a Ports Collection that contains cross-compiled utilities and libraries for CloudABI. The intent is that this framework generates native packages for a number of operating systems, making it possible to develop CloudABI applications on any operating system, regardless of whether that operating system actually supports CloudABI.

If you are interested in CloudABI, be sure to go to the project page on GitHub, watch recordings of talks at conferences, or wait for the upcoming edition of the FreeBSD Journal, which will feature an article on CloudABI.

Nuxi, the Netherlands

CloudABI is currently only available for amd64. It would make sense to port CloudABI to additional architectures like aarch64.

Support for CloudABI has only been integrated into &os;. If we manage to upstream support for CloudABI into other operating systems, it should be possible to run the same binary on multiple operating systems without recompilation.

The CloudABI Ports Collection currently has only 60 packages. Although these packages already the building blocks of some interesting software, we are always interested in expanding.

ZFS Support for UEFI Boot/Loader Eric McCorkle eric@metricspace.net

UEFI-enabled boot1.efi and loader.efi have been modified to support loading and booting from a ZFS filesystem, as described in the previous report. The ZFS-enabled loader.efi can be treated as a chainloader when using ZFS-enabled GRUB.

During this quarter, several successful tests have been reported on simple ZFS setups, using both boot1.efi with loader.efi as well as GRUB and loader.efi.

Successful tests have been reported for UFS with the patched boot1.efi and loader.efi as well.

Test reports are needed for more complex ZFS setups, such as with log/l2arc vdevs, mirroring, striping, and raidz.

Pending successful reports, the patch needs to be reviewed and committed.

Allwinner A10/A20 Support Luiz Otavio O Souza loos@FreeBSD.org Pratik Singhal ps06756@gmail.com Wiki page

The Allwinner A10 and A20 chips are ARM CPUs found in increasingly common development boards and other devices, such as the Cubieboard/Cubieboard 2 and the Banana Pi.

With the end of a GSoC project by Pratik Singhal, our A10 and A20 support has improved. Pratik helped with the implementation and testing of the SD card and SATA support for the Allwinner chips.

&a.loos; added support for the dwc network interface on the A20, which is capable of gigabit speeds.

&a.gjb; kindly added support for official &os; images for Cubieboard 2 and the Banana Pi.

Google Summer of Code 2015 (partly)

Some drivers are still missing: audio, video/HDMI/framebuffer, IR, I2C, SPI, PWM.

Fix if_dwc for better performance.

The <tt>nosh</tt> Project Jonathan de Boyne Pollard J.deBoynePollard-newsgroups@NTLWorld.COM Introduction and blurb &os; binary packages Installation How-To Roadmap Commands A slightly outdated nosh Guide

The nosh project is a suite of system-level utilities for initializing, running, and shutting down BSD systems, and for managing daemons, terminals, and logging. It supersedes BSD init and the NetBSD rc.d system, drawing inspiration from Solaris SMF for named milestones, daemontools-encore for service control/status mechanisms, UCSPI, and IBM AIX for separated service and system management. It comprises a range of compatibility mechanisms, including shims for familiar commands from other systems, and an automatic import mechanism that takes existing configuration data from /etc/fstab, /etc/rc.conf{,.local}, /etc/ttys, and elsewhere, applying them to its native service definitions and creating additional native services. It is portable (including to Linux) and composable, it provides a migration path from the world of systemd Linux, and it does not require new kernel APIs. It provides clean service environments, orderings and dependencies between services, parallelized startup and shutdown (including fsck), strictly size-capped and autorotated logging, the service manager as a "subreaper", and uses kevent(2) for event-driven parallelism.

The past few months have seen a growth in the import mechanism, with full import of /etc/fstab and /etc/ttys available in version 1.18 in July, and importing PC-BSD Warden and &os; 9 jails, and full import of gbde and geli mount/unmount mechanisms in version 1.21 in October. It has also gained the ability to automatically re-generate host.conf and sysctl.conf whenever their source files change.

Other developments in the past few months include fully independent shutdown support, no longer relying upon an externally provided shutdown command from another toolset, and a full suite of binary packages. As of version 1.20, it became possible to have a fully-nosh-managed system, on both &os; and Linux, using just precompiled binary packages.

The biggest task remaining is one that was set a while ago: the creation of enough native service bundles and ancillary utilities to entirely supplant the rc.d system. A lot of this has been achieved, with the original target list of 157 items now down to just 39 remaining. These are the tricky ones, of course, where help is most needed.

There are still a few rc scripts left that should be easy to convert, such as /etc/rc.d/gptboot and /etc/rc.d/growfs as oneshot services, /etc/rc.d/routing, and /etc/rc.d/kldxref.

&os;'s /etc/rc.d/bluetooth is over 360 lines long. In 2011, Iain Hibbert wrote a "simpler" bluetooth for NetBSD. This can perhaps be used as a simpler basis for a nosh translation.

Add kernel support for passing a -b option to pid 1, and support for a boot_bare variable in the loader, to allow "emergency" (where even no shell dotfiles are loaded) and "rescue" mode bootstraps, akin to Linux. (History: The -b mechanism and idea date back to version 2.57d of Miquel van Smoorenburg's System 5 init clone, dated 1995-12-03, and was already known as "emergency boot" by 1997.)

Add support to &os;'s fsck(8) for outputting machine-readable progress reports to a designated file descriptor, so that nosh can provide progress bars for multiple fscks running in parallel. nosh already provides this functionality on Linux, where fsck(8) does provide machine-readable output.

Identify when the configuration import system needs to be triggered, such as when bsdconfig alters configuration files, and create the necessary hooks to import external configuration changes into nosh.

Investigate how &os;/PC-BSD could be improved by taking advantage of some available nosh package mechanisms.

Adding PCIe Hot-plug Support John-Mark Gurney jmg@FreeBSD.org PCIe Hot-plug Perforce Branch Commit - adding bridge save/restore. + adding bridge save/restore Github branch with patches

PCI Express (PCIe) hot-plug is used on both laptops and servers to allow peripheral devices to be added or removed while the system is running. Laptops commonly include hot-pluggable PCIe as either an ExpressCard slot or a Thunderbolt interface. ExpressCard has built-in USB support that is already supported by &os;, but ExpressCard PCIe devices like Gigabit Ethernet adapters and eSATA cards are only supported when they are present at boot, and their removal may cause &os; to crash.

The goal of this project is to allow these devices to be inserted and removed while &os; is running. This work will provide the basic infrastructure to support adding and removing devices, though it is expected that additional work will be needed to update individual drivers to support hot-plug.

Current testing is focused on getting a simple UART device functional. Basic hot swap is currently functional.

A set of the patches for the work done in this project is now available on github.com.

The FreeBSD Foundation

Get suspend/resume functional by saving and restoring the necessary registers. This should be addressed by r281874.

Make sure that upon suspend, devices are removed so that we are not fooled if they are replaced with different devices while the machine is suspended.

Improve how state transitions are handled, possibly by using a proper state machine.

<tt>automtud</tt>: Better Jumbo Frame Support John-Mark Gurney jmg@FreeBSD.org jmgurney/automtud on github

The automtud script will allow a &os; machine to send jumbo frames to machines that support them, while using normal-sized frames for other machines.

There are various advantages to using jumbo frames, such as reduced protocol overhead. It also means that TCP streams will not be segmented as much, although TSO helps mitigate the disadvantages of such segmentation. In cases where LRO does not work well, fewer packets will be received.

The script currently does not restore the system to its original state when it exits. This means that you must manually change the interface MTU and delete host routes after stopping the script.

Fix up various Ethernet drivers to better support jumbo frames. Most Ethernet drivers, though they support scatter/gather, use a physically contiguous zone to do so, which can cause resource shortages.

More testing is needed to ensure that things behave as expected. This means that when running the script, communication to all machines functions normally, without slowdown or connectivity issues. Check vmstat -z | grep mbuf to ensure that such issues are not due to running out of jumbo_9k or jumbo_16k buffers due to Ethernet driver issues.

Update Ficl in Bootloader Colin Lord clord@FreeBSD.org Wiki Page

The &os; bootloader has used Ficl 3 for quite some time. This project was intended to update the version of Ficl in use to Ficl 4. Ficl 4 is not only faster but also has a smaller memory footprint, both being important advantages for a bootloader.

As part of the Google Summer of Code program, I worked on importing the Ficl 4 sources to get a bootloader running Ficl 4. The first half of the summer consisted of setting up my test environment, as well as arranging the sources in the tree properly and modifying the build files to point to the new locations. Once that was complete, the sources had to be modified to build correctly and to add back in some of the &os;-specific parts from Ficl 3. Unfortunately, after all those tasks were completed, a few bugs in the Ficl project were discovered that delayed the bootloader update, so it is not finished. The Illumos project has faced similar issues with Ficl 4 so I received some good tips from them, but since school has started back up I have not been able to put much work into fixing the bugs.

Google Summer of Code 2015
<tt>mtree</tt> Parsing and Manipulation Library Michal Ratajsky michal@FreeBSD.org Brooks Davis brooks@FreeBSD.org Wiki page

&os; includes several programs that work with file system hierarchy descriptions in the mtree(5) format. These descriptions, also called specifications, have a broad range of uses, from automatically creating directory structures to security auditing.

Each of the programs, namely mtree, bsdtar, install, and makefs, has its own implementation of the mtree format. This not only adds maintenance overhead, but also makes interoperability difficult, as each of the implementations only supports a limited subset of the format.

The goal of this project was to create a new libmtree library, implementing everything the mtree format has to offer, and wrapping it with an expressive API which all the listed programs can use. We also wanted libmtree to be portable, as one of the major users of the mtree format is libarchive, the library implementing most of bsdtar.

Currently, the library is functionally complete, ready to be downloaded and receive everyone's attention. We have also decided to bundle the mtree program along with it. The bundled mtree has also been modified for better portability.

The project included modifying libarchive, install and makefs to use libmtree. These modified versions are also available.

Please see the Wiki page for more information, download locations, and an example of using the libmtree API.

Google Summer of Code 2015

Test and review the library code and API, and the modifications made to the programs.

Fix the known problems that are mentioned on the Wiki page.

ZFSguru Jason Edwards jason@zfsguru.com Home page Forum post on Gnome 3 debugging

ZFSguru started as a front-end to ZFS but has since grown into a multifunctional server appliance with its own unique features. While the project is still in early development, it already offers multiple unique features not found in other projects. Unlike similar projects, nothing is stripped away from the base operating system, meaning ZFSguru behaves as a normal &os; installation and thus is very versatile. The web-interface is designed to unite both novice and advanced users, providing both very easy to use basic functionality as well as features to be appreciated by more experienced users. The modular nature of the project combats the danger of bloat, whilst still allowing extended functionality to be easily deployed.

On the 15th of August, version 0.3 of ZFSguru was released. Some highlights of the new version:

Other progress made in the months July, August, and September:

The new Gnome 3 desktop does not work for everyone and still has issues. Anyone capable of diagnosing these issues can give the Gnome 3 LiveCD a try. Please see the linked forum thread for more information.

Several ports fail to compile with our own build infrastructure, and require bug reports in order to get them fixed upstream.

A 'State of the Project 2015' is due in Q4, providing an overview for future development of the project.