Index: head/en_US.ISO8859-1/htdocs/news/status/report-2016-07-2016-09.xml =================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2016-07-2016-09.xml (revision 49671) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2016-07-2016-09.xml (revision 49672) @@ -1,2259 +1,2259 @@ July-September 2016
Introduction

As focused as we are on the present and what is happening now, it is sometimes useful to take a fresh look at where we have come from, and where we are going. This quarter, we had our newest doc committer working to trace through the tangled history of many utilities, and we also get a glimpse looking forward at what may come in &os; 12.

Though 11.0-RELEASE was not finalized until after the period covered in this report, we can still have some anticipatory excitement for the features that will be coming in 12.0. The possibilities are tantalizing: a base system with no GPL components, arm64 as a Tier-1 architecture, capsicum protection for common utilities, and the CloudABI for custom software are just a few.

The work of the present is no less exciting, with 11.0 making its way out just after the end of Q3, the new core coming into its own, and much more that you'll have to read and find out.

—Benjamin Kaduk

Please submit status reports for the fourth quarter of 2016 by January 7.

team &os; Team Reports proj Projects kern Kernel gsoc Google Summer of Code arch Architectures ports Ports doc Documentation ClonOS: New &os;-Based Free/Open Hosting Platform Oleg Ginzburg olevole@olevole.ru ClonOS Homepage

Currently, &os; is well proven as a base for routers (pfSense, OPNSense, BSDRP) and NAS (FreeNAS, zfsGuru, NAS4Free). However, &os;-based solutions are almost completely absent in the virtualization area, and ClonOS is one of the attempts to change that.

ClonOS is a new free open-source &os;-based platform for virtual environment creation and management. In the core platform are:

We would like to see ClonOS in real-world use. In this regard we are interested in finding more people and companies that use &os; in hosting tasks. In addition, it could be great to work with the developers of existing NAS solutions (zfsGuru, NAS4Free). Google Summer of Code 2016 Gavin Atkinson gavin@FreeBSD.org Pedro Giffuni pfg@FreeBSD.org GSoC 2016 Projects GSoC Ideas page

As in all previous editions of the Google Summer of Code, &os; was an accepted organization, and we had the chance to mentor 15 projects. Huge thanks to all our mentors for keeping the high quality standards that make our community shine.

This year was rather unique in that we accepted for the first time well-known members of the community that are not src committers to co-mentor. We also accepted projects that have a different upstream than &os;. Both are clear signs that &os; is growing and adapting to the wider community.

This year we are also had administrative issues with Perforce and have officially accepted the use of external repositories, in particular github, as requested by students.

12 of 15 projects were successful, which we think is an excellent result for a Google Summer of Code.

Google Inc The FreeBSD Foundation The world is changing and we need fresh project ideas. We need to start looking for those ideas now. The project ideas wiki page has been reset and we need to get it populated before applying for the next Google Summer of Code. Please help unleash the next stream of projects you want to see in &os;. CloudABI: Running Untrusted Programs Directly on top of &os; Ed Schouten ed@FreeBSD.org The CloudABI mailing list cloudabi-devel@googlegroups.com Official CloudABI Website Using CloudABI on &os; Python for CloudABI CloudABI on GitHub

CloudABI is a compact UNIX-like runtime environment inspired by &os;'s Capsicum security framework. It allows you to safely run potentially untrusted programs directly on top of &os;, Linux and macOS, without requiring the use of virtualisation, jails, etc. This makes it a useful building block for cluster/cloud computing.

Over the last couple of months, several new libraries and applications have been ported over to CloudABI, the most important addition being Python 3.6. This means that you can now write strongly sandboxed apps in Python!

Support for different hardware platforms has also improved. In addition to amd64 and arm64, we now support i686 and armv6. The release of LLVM 3.9 was important to us, as it has integrated all the necessary changes to support the first three platforms. Full armv6 support is still blocked on some issues with LLVM's linker, LLD.

Nuxi, the Netherlands Play around with CloudABI and let us know what you think of it! Full support for amd64 and arm64 is part of &os; 11.0. i686 and armv6 support is only available on head, but will be merged to the stable/11 branch in the future. Interested in Python programming? Give our copy of Python a try and share your experiences! Do you maintain pieces of software that could benefit from strong sandboxing? Try building them using the CloudABI cross compiler! &os; on Hyper-V and Azure Sepherosa Ziehau sepherosa@gmail.com Hongjiang Zhang honzhan@microsoft.com Dexuan Cui decui@microsoft.com Kylie Liang kyliel@microsoft.com &os; Virtual Machines on Microsoft Hyper-V Supported Linux and &os; virtual machines for Hyper-V on Windows

This quarter, the Hyper-V storage driver was greatly improved: its performance was increased by a factor of 1.2-2 by applying BUS_DMA and UNMAP_IO, enlarging the request queue, and selecting the outgoing channel with the LUN considered; TRIM/UNMAP was enabled; and some critical bugs (PRs 209443, 211000, 212998) were fixed so that disk hot add/remove and VHDX online resizing should work now.

The VMBus driver also received attention, with enhancements made for the handling of device hot add/remove.

In the Hyper-V network driver, configurable RSS key and dynamic MTU change are now supported.

&os; images on Azure continue to be updated — after publishing the &os; 10.3 VM image on the global Microsoft Azure in June, Microsoft also published the VM image on the Microsoft Azure operated by 21Vianet in China in September.

Patches have been developed to support PCIe pass-through (also known as Discrete Device Assignment); this feature allows physical PCIe devices to be passed through to &os; VMs running on Hyper-V (Windows Server 2016), giving them near-native performance with low CPU utilization. The patch to enable the feature will be posted for review soon.

Microsoft <tt>ptnet</tt> Driver and <tt>bhyve</tt> Device Model Vincenzo Maffione v.maffione@gmail.com &os; Wiki Page for Project Overview Conference Paper Subversion Repository

This project provides:

The ptnet device and driver has been introduced to overcome the performance limitations of TCP/IP networking between bhyve VMs. Prior to this work, the most performant solution for VM-to-VM intra-host TCP communication provided less than 2 Gbps TCP throughput. With ptnet, in the same VM-to-VM TCP communication scenario, it is possible to obtain up to 20 Gbps.

Google Summer of Code Share virtio-net header management code with the if_vtnet driver. In the current code, about 100 lines of code have been copied and pasted from if_vtnet.c. LXQt on &os; Olivier Duchateau olivierd@FreeBSD.org Jesper Schmitz Mouridsen jesper@schmitz.computer &os; LXQt Project LXQt Project LXQt Development Repository

LXQt is the Qt port of and the upcoming version of LXDE, the Lightweight Desktop Environment. It is the product of a merge between the LXDE-Qt and Razor-qt projects.

The porting effort remains very much a work in progress: it requires some components of Plasma 5, the new major KDE workspace.

The porting of the 0.11 branch is now complete, with new ports (compared to the previous release). See our wiki page for a complete list of applications.

We also have updates for:

Improve &os; support in sysutils/lxqt-admin, especially with respect to user management. Add additional panel plugins. Xfce on &os; &os; Xfce Team xfce@FreeBSD.org &os; Xfce Project &os; Xfce Repository

Xfce is a free software desktop environment for Unix and Unix-like platforms such as &os;. It aims to be fast and lightweight, while still being visually appealing and easy to use.

During this quarter, the team has kept these applications up-to-date:

We also follow the unstable releases; the current unstable release brings support for Gtk3 (available in our experimental repository) to:

Currently the unstable releases work fine with our Gtk3 ports available in the ports tree, but in the future support for 3.18 will be removed in preference of 3.20.x.

Continue working on unstable releases. Non-BSM to BSM Conversion Tools Mateusz Piotrowski 0mp@FreeBSD.org Wiki Page GitHub Repository Pull Request With Consolidated Patch

This project was started during Google Summer of Code this year. The aim was to create a library which can convert the audit trail files in Linux Audit format or the format used by Windows to the BSM format used by &os; for its audit logs. Apart from that, I wanted to create a simple command-line tool and extend auditdistd so that it is possible to send non-BSM logs to it over a secure connection and save those audit logs on disk, preferably in the BSM format.

So far, it is possible to reasonably convert some of the most common Linux audit log events to BSM, but it still needs a lot of work. Secondly, I was able to configure auditdistd to communicate with CentOS over an insecure connection. Thirdly, the command-line tool is usable but not perfect.

The present work focuses on configuring the secure TLS connection between CentOS and auditdistd. I have already tried using rsyslogd but was not able to make it work.

Google Summer of Code I need more examples of rare Linux Audit logs; please send me some examples if you have any. It is much easier to improve the conversion process with real-life examples of audit events as you write the code to convert them. Configure auditdistd to be able to communicate with some software on CentOS over TLS in order to receive audit logs. I was not able to come up with a simple solution for that. Additional open tasks are listed on the Wiki page and in the TODO file in the root directory of the project. Improvements to Non-Transparent Bridge Subsystem Alexander Motin mav@FreeBSD.org

Non-Transparent Bridges allow the creation of memory windows between different systems using the regular PCIe links of CPUs as a transport. During the last quarter, the NTB subsystem gained a significant set of improvements and fixes:

The code is committed to the &os; head, stable/11 and stable/10 branches.

iXsystems, Inc Support for the next generation of Intel hardware. Support for non-Intel hardware (AMD, PLX, etc.). Support for I/OAT and other DMA offloads. Creating a more efficient packet transport protocol. Creating a greater variety of NTB applications. ZFS Code Sync with Latest OpenZFS/Illumos Alexander Motin mav@FreeBSD.org Andriy Gapon avg@FreeBSD.org

The ZFS code base in &os; regularly gets merges of new code, staying in sync with the latest OpenZFS/Illumos sources. Among other things, the latest merge included the following improvements:

 &os; on Marvell Armada38x Marcin Wojtas mw@semihalf.com Bartosz Szczepanek bsz@semihalf.com

&os; includes support for the Marvell Armada38x platform, which has been tested and improved in order to gain production quality. Most of this effort has been invested in development and benchmarking of the on-chip Gigabit Ethernet (NETA) functionality. Numerous bug fixes and some new features have been introduced.

Work completed this quarter includes:

Along with support for new boards (SolidRun ClearFog and DB-88F6285-AP), all changes will be submitted upstream.

Stormshield Semihalf Finalize NETA and prepare for submission. Submit remaining fixes and drivers. &os; Driver for the Annapurna Labs ENA Jan Medala jan@semihalf.com Jakub Palider jpa@semihalf.com Amazon AWS Documentation of the ENA

The Elastic Network Adapter (ENA) is a 25G SmartNIC developed by Annapurna Labs based on a custom ARMv8 chip. This is a high-performance networking card that is available to AWS virtual machines. It introduces enhancements in network utilization scalability on EC2 machines running various operating systems, in particular &os;.

The goal of &os; enablement is to provide top performance and a wide range of monitoring and management features such as:

The current state offers stable driver operation with good performance on machines running &os; directly on the hardware.

Annapurna Labs — an Amazon company Optimize performance for virtualized environments. Prepare for submitting the driver as a Phabricator review. &os; on Annapurna Labs Alpine Jan Medala jan@semihalf.com Michal Stanek mst@semihalf.com Wojciech Macek wma@semihalf.com

Alpine is a family of Platform-on-Chip devices, including multi-core 32-bit (first-gen Alpine) and 64-bit (Alpine V2) ARM CPUs, developed by Annapurna Labs.

The primary focus areas of the Alpine platform are high-performance networking, storage, and embedded applications. The network subsystem features 10-, 25-, and 50-Gbit Ethernet controllers with support for virtualization, load-balancing, hardware offload and other advanced features.

A basic patch set has already been committed to head including:

Additional work, such as an MSI-X driver and full Ethernet support, is currently undergoing community review on Phabricator.

The multi-user SMP system is stable and fully working, along with the 1G and 10G Ethernet links.

The interrupt management code has been adjusted to work with the new INTRNG framework on both ARM32 and ARM64.

Annapurna Labs — an Amazon company Semihalf Documenting the History of Utilities in <tt>/bin</tt> and <tt>/sbin</tt> Sevan Janiyan sevan@FreeBSD.org The igor Port BSD Family Tree in Subversion The UNIX Heritage Society Cat-V Manual Library

For EuroBSDcon, I began looking into inconsistencies within components inside our family of operating systems. My workflow consisted of reading the documentation for a given utility and checking the history in the revision control system for missing fixes or functionality in the trees of NetBSD, &os;, OpenBSD, and DragonFly BSD.

One thing which became obvious very quickly was the inconsistency between operating systems about where and/or which version a utility originated in, despite our common heritage.

I began working through the man pages in &os;, verifying the details in pages which already had a history section and making patches for those which did not.

From there, changes were propogated out to NetBSD, OpenBSD, and Dragonfly BSD where applicable (not all utilities originated from the same source or implementation, for example).

This was a good exercise in:

Cover the remaining manuals for userland utilities, and maybe expand into library and syscall APIs, though I say that without estimating the feasibility. The history of components originating from a closed-source operating system is tricky to document, since older versions are not always available. VirtualBox Shared Folders Filesystem Li-Wen Hsu lwhsu@FreeBSD.org Oleksandr Tymoshenko gonzo@FreeBSD.org Project Repository

&os; provides an API for guest operating systems to access shared folders on the host so that the kernel driver can expose them to the guest's userland. This project aims to add such functionality to the VirtualBox Guest Additions driver.

Good progress was made over the last few months. Developers were able to mount a filesystem in read-only mode and, with some limitations, in read-write mode. The implementation still lacks some critical pieces, but the roadmap is clear.

Finish the missing pieces. Implement proper locking. General clean-up and bugfixes. <tt>evdev</tt> Support Vladimir Kondratiev wulf@cicgroup.ru Oleksandr Tymoshenko gonzo@FreeBSD.org evdev WIP Repository Original evdev Proposal

evdev is a portable, API-compatible implementation of the Linux /dev/input/eventX interface. It covers a wide variety of input devices like keyboards, mice, and touchscreens (with multitouch support), and support for it is implemented in a lot of existing userland components like Qt, libinput, and tslib.

evdev support was started by Jakub Klama as a Google SoC 2014 project, and later picked up and finished by Vladimir Kondratiev. General API and evdev support bits for ukbd and ums were committed to head. Support was also added for TI's AM33xx touchstreen controller (the popular BeagleBone is based on the AM33xx) and the - official touschreen for the Raspberry Pi. Multitouch support - for the Raspberry Pi was successfully demonstarted using the + official touchscreen for the Raspberry Pi. Multitouch support + for the Raspberry Pi was successfully demonstrated using the latest Qt development branch.

Documentation. In particular, manual pages are needed for the KPI. Support additional hardware. Enable evdev support in existing ports, and add new evdev-dependent ports. &os;/arm64 Andrew Turner andrew@FreeBSD.org Jared McNeill jmcneill@FreeBSD.org &os; arm64 Wiki Entry Using Crochet to Build &os; Images

Transparent superpage support has been added. This allows &os; to create 2MiB blocks with a single pagetable and TLB entry. This shows a small but significant improvement in the buildworld time on ThunderX machines. Superpages have been enabled in head and merged to stable/11, but they are disabled by default on stable/11 due to a lack of testing there.

Support for the pre-INTRNG interrupt framework has been removed. This means that arm64 requires INTRNG to even build. This has allowed various cleanups within the arm64 drivers that interact with the interrupt controller.

The cortex Strings library from Linaro has been imported. The parts of this that have been shown to be improvements over the previous C code were attached to the libc build.

There is ongoing work to add ACPI support to the kernel. On ThunderX, &os; can get to the mountroot prompt, however, due to incomplete ACPI tables the external PCIe support needed to support the netboot setup in the test cluster is not functional.

Pine64 support has been committed to head. &os; can now boot to multiuser with SMP enabled. This includes support for clocks, secure ID controller, USB Host controller, GPIOs, non-maskable interrupts, AXP81x power management unit, cpu freqency and voltage scaling, MMC, UART, gigabit networking, watchdog, and thermal sensors.

The FreeBSD Foundation ABT Systems Ltd KDE on &os; KDE on &os; Team kde@FreeBSD.org KDE on &os; website KDE ports staging area KDE on &os; wiki KDE/&os; mailing list Development repository for integrating Plasma 5 and KDE Frameworks 5 Development repository for integrating Qt 5.7

The KDE on &os; team focuses on packaging the KDE software and making sure that the experience of KDE and Qt on &os; is as good as possible.

The following big updates were landed in the ports tree this quarter:

In our development repository, we have done this work:

 The &os; Core Team &os; Core Team core@FreeBSD.org

The third quarter started with the handover to the ninth Core team as it took office. With four members returning from the previous core (Baptiste Daroussin, Ed Maste, George Neville-Neil and Hiroki Sato), one returning member after a term away (John Baldwin), and four members new to core (Allan Jude, Kris Moore, Benedict Reuschling and Benno Rice), the new core team represents just about the ideal balance between experience and fresh blood.

Beyond handing over all of the ongoing business, reviewing everything on Core's agenda, and other routine changeover activities, the first action of the new core was to respond to a query from Craig Rodrigues concerning how hardware supplied to the project through donations to the &os; Foundation was being used.

The Foundation does keep records of what hardware has been supplied over time and has some idea of the original purpose that hardware was provisioned for, but does not track the current usage of the project's hardware assets. Cluster administration keeps their own configuration database, but this is not suitable for general publication and covers much more than Foundation supplied equipment. After some discussion it was decided that updated information about the current disposition of Foundation supplied equipment should be incorporated in the Foundation's annual report.

Ensuring that all of the &os; code base is supplied under open and unencumbered licensing terms and that we do not infringe on patent terms or otherwise act counter to any legal requirements are some of Core's primary concerns. During this quarter, there were three items of this nature.

The item that has absorbed the largest portion of Core's attention this quarter concerns the project's handling of security vulnerabilities in bspatch(1), libarchive(3), freebsd-update(8) and portsnap(8). A partial fix was applied in &os;-SA-16:25.bspatch but this lacks fixes to libarchive code that were not yet available from upstream.

SecTeam receives privileged early reports of many vulnerabilities and consequently has a strict policy of not commenting publicly until an advisory and patches have been published. Early access to information about vulnerabilities is contingent on their ability to avoid premature disclosure, and without such, they could not have security advisories and patches ready to go immediately when a vulnerability is published.

However, in this case, vulnerabilities were already public and the lack of any official response from the &os; Project was leading to concern amongst users and some critical press coverage. Core stepped in and published a statement clarifying the situation and the particular difficulties involved in securely modifying the mechanisms used to deliver security patches. Core believes that prompt notification and discussion of the implications and possible workarounds to any public vulnerability should not wait on the availability of formal OS patches.

The OpenSSH project has deprecated DSA keys upstream. &os; had kept DSA keys enabled in the later 10.x releases for compatibility reasons, but with the release of 11.0 the time has come to synchronize again with upstream. Since there are numerous DSA keys in use in the &os; cluster, this necessitated an exercise to get replacement keys installed. Core would like to thank David Wolfskill and the accounts team for handling the surge in key changes with a great deal of aplomb.

During this quarter we welcomed Michael Zhilin, Imre Vadasz, Steve Kiernan and Toomas Soome as new source committers. Over the same period, we said farewell to Martin Wilke and Erwin Lansing who have handed in their commit bits. We wish them well in their future endeavours and hope to see them return as soon as they can.

 Timekeeping Code Improvements Konstantin Belousov kib@FreeBSD.org

Work was done to properly lock the time-keeping code. The existing code correctly interoperated with readers, both kernel- and user-space, giving them lock-less access to the actual data ('timehands') needed to derive the time of day from the timecounter hardware in the presence of updaters. But updates of the timehands, which are performed by periodic clock interrupts, the ntpd-driven sys_ntp_adjtime(2) syscall, the settimeofday(2) syscall, pps sync, and possibly other sources, were not coordinated. Moreso, the NTP code was locked by Giant, which did not really serve any purpose.

As a result of the work, locking was applied to ensure that any timehands adjustments are performed by a single mutator. An interesting case is the parallel modification of the timehands from the top of the kernel, for instance the settimeoday(2) syscall, and a simultaneous clock tick event, where the syscall has already acquired the resources. In this case, it is highly desirable to not block (spin) in the tick handler, and the required adjustments are performed by the already executing call from the top half. There, the typical trylock operation is desired, which was surprisingly lacking in our spinlock implementation. mtx_trylock_spin(9) was implemented and is used for this purpose.

The userspace gettimeofday(2) implementation was enhanced to allow syscall-less operation on machines that use - HPET hardwire for timecounters. The HPET algorithm coexists + HPET hardware for timecounters. The HPET algorithm coexists with older RDTSC-based code, allowing dynamic switching of timecounters. A page with HPET registers is mmap(2)-ed readonly by libc into userland application programs' address space as needed. Measurements demonstrated modest improvements in gettimeofday(2) performance, but, not unexpectedly, even the syscall-less HPET timecounter is slower than invoking a syscall for RDTSC.

Some not strictly intertwined but related code is the time-bound sleep implementation. Handling of races between callouts and the top-half code that sets and processes the timeouts depended on the many fine details of the callout_stop(9) KPI (kernel programming interface). In particular, races or unpunctual KPI changes usually result in the "catch-all" unkillable thread state with the "-" waitchain bug. The sleepqueue timeout code was rewritten to stop depending on the KPI details, which removed the source of recurring bugs, and also surprisingly simplified the code.

The FreeBSD Foundation UEFI Runtime Services Konstantin Belousov kib@FreeBSD.org

UEFI (Unified Extensible Firmware Interface) specifies two kinds of services for use by operating systems. Boot Services are designed for OS loaders to load and initialize kernels, while Runtime Services are meant to be used by kernels during regular system operations. The boot and runtime phases are explicitly separated. During boot, when loaders are executed, the machine configuration is owned by UEFI. During runtime, the kernel manages the configuration, but needs to inform the firmware about any changes that are made.

The model of split boot/runtime configuration makes assumptions about the OS architecture that do not quite apply to the existing &os; codebase. For instance, the firmware notification of the future runtime configuration must be done while the loader is effectively still in control. In technical terms, the SetVirtualAddressMap() call must be made with the 1:1 physical:virtual mapping on amd64 systems, which for &os; means that the call can only be issued by the loader. But the loader needs to know intimate details of the kernel address map to provide the requested information. This creates a new, unfortunate, coupling between loader and kernel.

Reading the publicly available information about the MS Windows boot process explained the UEFI control transfer model. The Windows loader constructs the address map for the kernel, and with such a division of work the UEFI model is reasonable. The &os; kernel constructs its own address space, only relying on a minimal map constructed by the loader, which is enough for the pmap subsystem to bootstrap itself and then to perform machine initialization in common code.

Initial experiments with enabling runtime services were centered around utilizing the direct address map (DMAP) on amd64, which currently always exists and linearly maps at least the lower 4G of physical addresses at some KVA location. It was supposed that the kernel would export the DMAP details like linear base and guaranteed size for loader from its ELF image, and provide the needed overflow map if the DMAP cannot completely serve. Unfortunately, two show-stopper bugs were discovered with this approach.

First, EDK-based firmware apparently requires that the runtime mapping exists simultaneously with the physical mapping for the SetVirtualAddressMap() call. Second, there were references from other open-source projects mentioning that some firmware required the presence of the physical mapping during the runtime call. Effectively, this forces both kernel and loader to provide both mappings for all runtime calls.

With such restrictions, informing the firmware about the details of the kernel address space only adds useless work. We could just as easily establish the 1:1 physical mapping during runtime and get rid of SetVirtualAddressMap() entirely. This approach was coded and the kernel interface to access runtime services is based on it.

During development, particularly when trying to make the loader modifications, it was quickly realized that there were no fault-reporting facilities in loader.efi. Machine exceptions resulted in a silent hang. Curiously, in such a situation the Intel firmware outputs the error code over the serial port over 115200/8/1 settings, regardless of UEFI console configuration, which was discovered by accident. Unfortunately, the error code alone is not enough to diagnose most problems.

A primitive fault reporter was written for loader.efi on amd64, which intercepts exceptions from the firmware IDT and dumps the machine state to the loader console. Due to the complexity of the interception and possible bugs which might do more harm than good there, the dumper is only activated on explicit administrator action.

Note that the described work only provides the kernel interfaces to make calling the EFI runtime services as easy as calling a regular C function. User-visible feature development making use of the new interfaces is being performed right now.

The FreeBSD Foundation &os; Release Engineering Team &os; Release Engineering Team re@FreeBSD.org &os; 11.0-RELEASE schedule

The &os; Release Engineering Team is responsible for setting and publishing release schedules for official project releases of &os;, announcing code freezes, and maintaining the respective branches, among other things.

The &os; Release Engineering Team continued the 11.0-RELEASE cycle which was planned to be released in September, but as a result of several last-minute issues, the final release announcement was delayed.

The FreeBSD Foundation The Graphics Stack on &os; &os; Graphics Team freebsd-x11@FreeBSD.org Matthew Macy mmacy@nextbsd.org Johannes Lundberg yohanesu75@me.com GitHub Repository Graphics Stack Roadmap and Supported Hardware Matrix Ports Development Repository DRM 4.7 Development Repository GSoC 2016: Link /dev Entries to Sysctl Nodes GSoC 2016: Redesign libdevq Wayland Notes Graphics Team Blog

We are sad to report that both GSoc projects failed. The libdevq project was abandoned as the student disappeared. The kernel project was incomplete because the student could not work for personal reasons. He plans to resume work and complete the task, even though GSoC 2016 is finished.

X.org server version 1.18.4 and updates for the xf86-video-ati and xf86-video-intel DDX drivers are ready for wider testing. A CFT will be sent out shortly. These updates are required to use newer DRM versions.

The missing functionality from libdrm that is needed by the amdgpu driver has been added. These changes will be committed to the ports tree shortly after the xorg-server update.

DRM from Linux 4.8 was ported to the drm-next branch. This branch should be used for radeon and amdgpu cards. The drm-next-4.7 branch should be used for i915 cards due to instabilities in the intel driver in the drm-next branch.

Johannes Lundberg has been working on getting the Wayland environment running on &os;. The Wayland ports are in a working state except for the Weston compositor.

The current Weston port (from DragonFlyBSD) might be scrapped and a new port created from scratch based on the upstream source code. With the use of libinput, libudev-devd, and epoll-shim, the diff will not be very large and will be easier to maintain.

Patches for wlc (another Wayland compositor) are being pushed upstream. On the TODO list is refactoring the tty code into selectable backends (linux, &os;, etc), as recommended by the author of wlc. For now, it is running on &os; with patches in the ports tree.

 The FreeBSD Foundation Deb Goodkin deb@FreeBSDFoundation.org FreeBSD Foundation Website

The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the &os; Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to &os; contributors. The Foundation purchases hardware to improve and maintain &os; infrastructure and publishes &os; white papers and marketing material to promote, educate, and advocate for the &os; Project. The Foundation also represents the &os; Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.

Here are some highlights of what we did to help &os; last quarter:

Fundraising Efforts

Our work is 100% funded by your donations. Our spending budget for 2016 is $1,250,000 and we have raised $271,500 so far. Our Q1-Q3 financial reports will be posted in early November. As you can see, we need your donations to continue supporting &os; at our current level. Please consider making a donation at https://www.FreeBSDFoundation.org/donate/.

OS Improvements

The Foundation improves &os; by funding software development projects approved through our proposal submission process, and our internal software developer staff members. Two Foundation-funded projects continued last quarter: one project is to port NetBSD's blacklistd daemon and related elements to &os;, and the second is phase two of the &os;/arm64 port.

Foundation staff members were responsible for many changes over the quarter. Kostik Belousov accomplished this work last quarter:

Ed Maste, our Project Development Director, accomplished this work last quarter:

George Neville-Neil continued hosting a bi-weekly Transport conference call (notes at https://wiki.FreeBSD.org/TransportProtocols) and the bi-weekly DTrace conference call (notes at https://wiki.FreeBSD.org/DTrace).

Release Engineering

Foundation staff member Glen Barber worked with the Release Engineering team to continue finalizing the 11.0-RELEASE cycle, which was delayed to address several last-minute issues.

As part of the Cluster Administration team, Glen worked with the amazing on-site staff at NYI to rack and install two Cavium ThunderX machines, one of which is used for native package builds for the &os;/arm64 architecture, and the other of which is targeted to be used as a reference machine in the &os; infrastructure.

Getting Started with &os; Project

We hired a summer intern, with no experience on &os;, Linux, or any command-line operating system, to figure out on his own how to install and use &os;. He wrote easy-to-follow how-to guides to help make the new-user experience straightforward and positive. He submitted bug reports and reported issues through the appropriate channels, and worked with Glen Barber and Brad Davis to improve the new user information on FreeBSD.org to make it easier for new people to get started with &os;. You can find his how-to guides at https://www.FreeBSDFoundation.org/FreeBSD/how-to-guides/ and check out his interview on BSDNow at http://www.bsdnow.tv/episodes/2016_08_24-the_fresh_bsd_experience.

Supporting &os; Infrastructure

We provide hardware and support for &os; infrastructure. Last quarter we purchased and brought up two 48-core Cavium ThunderX machines to build &os; package sets for the arm64 platform. We also purchased more servers to help with continuous integration efforts.

&os; Advocacy and Education

A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others using &os;, producing advocacy literature to teach people about &os; and ease the path to starting out with &os;, contributing to the Project, and attending and getting other &os; contributors to volunteer to run &os; events, staff &os; tables, and give &os; presentations.

We created new handouts to promote TeachBSD.org (https://www.FreeBSDFoundation.org/wp-content/uploads/2016/08/TeachBSD_half_final.pdf) and the Google Summer of Code program (https://www.FreeBSDFoundation.org/wp-content/uploads/2016/08/GSOC-flyerv2.pdf).

We published the July/August issue of the &os; Journal: https://www.FreeBSDFoundation.org/past-issues/FreeBSD-and-rtems/.

We also published monthly newsletters to highlight work being done to support &os;, tell you about upcoming events, and provide other information to keep you in the loop of what we are doing to support the &os; Project and community: https://www.FreeBSDFoundation.org/news-and-events/newsletter/.

Conferences and Events

The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups.

We support the &os;-focused events to help provide a venue for sharing knowledge, to work together on projects, and facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-&os; events to promote and raise awareness about &os;, to increase the use of &os; in different applications, and to recruit more contributors to the Project.

This quarter, we sponsored and/or attended the following events:

We sponsored three &os; contributors to attend EuroBSDcon.

Legal/&os; IP

The Foundation owns the &os; trademarks, and it is our responsibility to protect them. We continued to review requests and grant permission to use the trademarks.

&os; Community Engagement

Anne Dickison, our Marketing Director, has been overseeing the efforts to rewrite the Project's Code of Conduct to help make this a safe, inclusive, and welcoming community.

Other Stuff We Did

We welcomed Kylie Liang and Philip Paeps to the Board of Directors. More information and interviews can be found at: https://www.FreeBSDFoundation.org/blog/FreeBSD-foundation-welcomes-new-board-members/.

George attended the ARM Partner Meeting in Cambridge.

 Capsicum Update Allan Jude allanjude@FreeBSD.org Baptiste Daroussin bapt@FreeBSD.org Conrad Meyer cem@FreeBSD.org Ed Maste emaste@FreeBSD.org Mariusz Zaborski oshogbo@FreeBSD.org Capsicum Wiki Page

Several developers have undertaken a recent effort to sandbox additional applications in the base system. This work is proceeding nicely and one of the goals is to target basic utilities used in security sensitive applications, like freebsd-update and portsnap.

This work highlighted two longstanding challenges in applying Capsicum. First, there are a number of common constructs shared by many simple programs, such as limiting capability rights on the stdio file descriptors. To address this, a set of capsicum helper routines has been added for these common cases.

Second, a common challenge occurs where applications need to open an arbitrarily large number of files, possibly from various directories, where preopening the file descriptors may not be suitable. Several possible solutions for this are in discussion.

Recently Capsicumized utilities include:

Additional Capsicum changes are in review:

An additional syscall (getdtablesize) and additional sysctls (kern.proc.nfds, kern.hostname, etc.) are now permitted in capability mode.

Capability rights are now propagated to child descriptors on accept(2).

Capsicum is now enabled in the 32-bit compatibility syscall layer.

Per-process (procctl) and global (sysctl) settings have been added to aid in debugging while Capsicumizing existing applications. When enabled, instead of returning ENOTCAPABLE or ECAPMODE for a system call, the kernel will issue a SIGTRAP to generate a core dump or enter the debugger.

Dell EMC Isilon ScaleEngine Inc. The FreeBSD Foundation Using <tt>lld</tt>, the LLVM Linker, to Link &os; Ed Maste emaste@FreeBSD.org LLD Wiki Page

lld is the linker in the LLVM family of projects. It is a high-performance linker that supports the ELF, COFF, and Mach-O object formats. Where possible, lld maintains command-line and functional compatibility with the existing GNU BFD ld and gold linkers. However, the authors of lld are not constrained by strict compatibility where it would hamper performance or desired functionality.

Compared to the GNU ld 2.17.50 currently in the base system, lld will bring:

The upstream lld project has now implemented nearly all of the functionality required to link the amd64 &os; base system, including the kernel. The boot loader components and rescue utilities currently do not build with lld.

Merge lld to &os; head as part of the Clang 3.9.0 import. Request a ports exp-run with lld installed as /usr/bin/ld. Fix building the boot loader with lld. Fix building rescue with lld. Test and iterate making lld fixes for additional architectures. The FreeBSD Foundation Ports Collection René Ladan portmgr-secretary@FreeBSD.org &os; Ports Management Team portmgr@FreeBSD.org &os; Ports Website How to Contribute Ports Monitoring Website Ports Management Team Website Ports Management Team on Twitter Ports Management Team on Facebook Ports Management Team on Google+

The Ports Tree currently contains over 26,300 ports, with the PR count around 2,150. Of these PRs, 516 are unassigned. The last quarter saw 5,295 commits by 117 active committers. Compared to the preceding quarter, there is both a slight increase in the number of PRs and the number of unassigned PRs, and a slight decrease in the number of committers.

In the last quarter, four commits bits were taken in for safe keeping: erwin, miwi, and sem left by their own request and jase was inactive for more than 18 months. We welcomed two new committers: Tobias Berner (tcberner) and Joseph Mingrone (jrm).

On the management side, erwin and miwi left portmgr. bapt also left portmgr but is still the liaison for core.

On the infrastructure side, three new USES (grantlee, kde, linux) and one new Keyword (javavm) were introduced. The default version of the Linux ports is now CentOS 6, with the Fedora 10 ports scheduled for removal at the end of the year. The license framework has been extended with a NONE license to indicate that a port has no clearly defined licensing terms. For those ports, no packages or distribution files are distributed. Also, support for the complete set of Creative Commons licenses has been added.

Some major user-visible ports were updated: Firefox to 49.0 and Firefox Extended Service Release to 45.4.0; Chromium to 52.0.2743.116; the default version of gcc to 4.8.5; and pkg itself to 1.8.7.

Behind the scenes, antoine ran 24 exp-runs to validate various package updates, framework changes, and changes to the base system. bdrewery added two new package building machines, supervised the package builds for 11.0-RELEASE, and added support for building arm64 packages.

At EuroBSDcon, rene visited a presentation by Landry Breuil <landry@openbsd.org> explaining how packages are built in the OpenBSD world and explaining various design decisions.

If you have some spare time, please take up a PR for testing and committing.