Index: head/en_US.ISO8859-1/htdocs/news/status/report-2016-10-2016-12.xml =================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2016-10-2016-12.xml (revision 49847) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2016-10-2016-12.xml (revision 49848) @@ -1,887 +1,891 @@ October-December 2016
Introduction

This is a draft of the October–December 2016 status report. Please check back after it is finalized, and an announcement email is sent to the &os;-Announce mailing list.

This report covers &os;-related projects between October and December 2016. This is the last of four reports planned for 2016.

The last quarter of 2016 was another productive quarter for the &os; project and community. [...]

Thanks to all the reporters for the excellent work!

The deadline for submissions covering the period from January to March 2017 is April 7, 2017.

?>
team &os; Team Reports proj Projects kern Kernel arch Architectures bin Userland Programs ports Ports doc Documentation misc Miscellaneous &os;/EC2 Colin Percival cperciva@FreeBSD.org

This report covers work since the last &os;/EC2 status report (2015Q1).

&os;/EC2 is now part of the regular &os; release build, with snapshots and releases being automatically uploaded and copied to all available regions. Due to legal restrictions this does not currently include the GovCloud or China (Beijing) regions; anyone wishing to use &os; in those regions is encouraged to contact the author.

The AWS Marketplace reports that approximately 800 users are running roughly 2000 &os; EC2 instances. This does not count the likely significantly larger number of EC2 instances launched "directly" through the EC2 API and Console, but at least places a lower bound on usage.

&os; 11.0-RELEASE shipped with support for the "enhanced networking" support in EC2 C3, C4, R3, I2, D2, and M4 (excluding m4.16xlarge) instances; this provides significantly higher network performance than the virtual networking available on older EC2 instances and with older versions of &os;.

&os; 11.0-RELEASE and later also use indirect segment disk I/Os, which yield ~20% higher throughput with equal or lower latency, and support the 128-vCPU x1.32xlarge instance type.

&os; now supports the Amazon Simple Systems Manager service ("run command").

Complete a pending reorganization of the accounts used for &os;/EC2 releases. Support "second generation enhanced networking" via the new Elastic Network Adapter found in P2, R4, X1, and m4.16xlarge instances. Provide tools for improved functionality via the Simple Systems Manager service: Listing installed packages, checking for updates, adding/removing users, [your favourite sysadmin task goes here]. Add support for EC2's IPv6 networking to the default &os;/EC2 configuration. Continue ongoing interoperability testing between &os;'s NFS client and Amazon Elastic File System (NFS-as-a-service). Sysctl Exporter for Prometheus Ed Schouten ed@FreeBSD.org The Prometheus Project Node Exporter Sysctl Exporter

Prometheus is an Open Source monitoring system that was originally built at SoundCloud in 2012. Since 2016, this project is part of the Cloud Native Computing Foundation, together with other projects like Kubernetes.

Prometheus scrapes its targets by periodically sending HTTP GET requests. Targets then respond by sending key-value pairs of metrics and their sample value. Prometheus has a query language, PromQL, that can be used to aggregate sample values and specify alerting conditions. Tools like Grafana can be used to create fancy dashboards using such queries.

The Prometheus project provides a utility called the node_exporter that gathers basic system metrics and serves them over HTTP. This utility tends to be rather complex, as it has to extract metrics from many different sources. On Linux, files in /proc have no uniform format, meaning that for every kernel framework a custom collector needs to be written.

On &os; the sitiuation is better, as the data exported through sysctl is already structured in such a way that it can easily be translated to Prometheus' metrics format. The goal of this project is thus to provide a generic exporter for the entire sysctl tree. Not only does this prevent unnecessary bloat and indirection, it may also make the life of a kernel developer a lot easier. One can easily use Prometheus to graph the occurrence of an event over time by (temporarily) adding a counter to the kernel.

An initial version of the sysctl exporter has been integrated into the &os; base system in December. It can be run through inetd by uncommenting the example provided in inetd.conf. Unfortunately, this exporter cannot be merged back to &os; 10.x/11.x, as it depends on KBI-breaking changes to sysctl(9).

Are you using Prometheus or are you interested in using it? Be sure to give both Prometheus and this sysctl exporter a try! It would be nice if we created a set of useful alerting rules and placed those in /usr/share/examples. For example, how can one use this exporter to monitor the state of GEOM-based RAID arrays? Is such information even exported through sysctl? Prometheus uses a pretty nifty format for exporting histograms. Histograms are useful for expressing the amount of time taken to complete certain events (for example, disk operations). Would it be possible to add histograms as native datatypes to sysctl? If so, is there any chance they can be implemented without picking up any kernel locks? &os; on ARM boards Ganbold Tsagaankhuu ganbold@FreeBSD.org &os; on Allwinner (Sunxi) Systems &os; Commit Adding Support for IR Interfaces

The changes necessary to support the Allwinner Consumer IR interface in &os; have been committed. The receive (RX) side is supported now and the driver is using the evdev framework. It was tested on the Cubieboard2 (A20 SoC) using lirc with dfrobot's simple IR remote controller.

 libarchive Tim Kientzle kientzle@FreeBSD.org Martin Matuska mm@FreeBSD.org Official Libarchive Homepage Libarchive on GitHub

Libarchive is a BSD-licensed archive and compression library originally developed as part of &os;. It supports a wide variety of input and output formats and also includes three command-line tools: bsdcat, bsdcpio and bsdtar. The &os; tar and cpio utilities are taken directly from Libarchive, and many other important utilities like ar, unzip, and the pkg package manager make use of libarchive's functions.

Libarchive development in 2016 has been focusing on bug fixes and code cleanup, including fixing several critical security issues. Automated testing with Travis CI and Jenkins has been introduced and libarchive has been added to the Google OSS-Fuzz project. Fuzzing helped detect several hidden problems like buffer overflows and memory leaks.

Over the last few months, NFSv4 ACL support for the pax and restricted pax (the default for bsdtar) formats has been completed and merged to &os;-CURRENT. NFSv4 ACL entries can now be stored to and restored from tar archives.

More extensive CI testing with &os; on different platforms and releases. Currently only 11.0-RELEASE-amd64 gets tested via an automated Jenkins job. As every commit to libarchive may influence the build process of &os; ports, the ability to trigger a (semi-)automated exp-run for the ports tree would be great. &os; on Hyper-V and Azure Sepherosa Ziehau sepherosa@gmail.com Hongjiang Zhang honzhan@microsoft.com Dexuan Cui decui@microsoft.com Kylie Liang kyliel@microsoft.com &os; Virtual Machines on Microsoft Hyper-V Supported Linux and &os; Virtual Machines for Hyper-V on Windows

Per-ring polling, multi-packet RNDIS messages, and system RSS integration have been implemented, further optimizing the throughput and latency of the Hyper-V network driver.

Live virtual machine backup is implemented (for now, only for UFS), after the VSS (Volume Shadow Copy Service), which it depends on, was implemented.

PCIe pass-through is implemented, and the patches to implement NIC SR-IOV are being reviewed on Phabricator.

vDSO support for speeding up gettimeofday(2) is now implemented.

The &os; 11.0 image on Azure (https://azure.microsoft.com/en-us/marketplace/partners/microsoft/FreeBSD110/) is also available now, in addition to the existing 10.3 image.

We fixed an issue where SCSI disks would sometimes fail to attach, resolving bug 215171 ([Hyper-V] Fail to attach SCSI disk from LUN 8 on Win2008R2/Win2012/Win2012R2).

Microsoft Updates to GDB John Baldwin jhb@FreeBSD.org Luca Pizzamiglio luca.pizzamiglio@gmail.com

The port has been updated to GDB 7.12.

7.12 includes additional fixes related to tracing vfork()s. Some of these fixes depend on changes to ptrace() in the kernel to report a new ptrace stop when the parent of a vfork() resumes.

Support for &os;/mips userland binaries has been committed upstream. These patches, along with support for debugging &os;/mips kernels, should be added to the port soon.

Figure out why the powerpc kgdb targets are not able to unwind the stack past the initial frame. Add support for more platforms (arm, aarch64) to upstream gdb for both userland and kgdb. Add support for debugging powerpc vector registers. Add support for $_siginfo. Implement 'info proc' commands. Implement 'info os' commands. Debug gdb hangs related to the 'kill' command. LXQt on &os; Olivier Duchateau olivierd@FreeBSD.org Jesper Schmitz Mouridsen jesper@schmitz.computer LXQt Project &os; LXQt Project LXQt Development Repository

LXQt is the Qt port of and the upcoming version of LXDE, the Lightweight Desktop Environment. It is the product of a merge between the LXDE-Qt and Razor-qt projects.

The porting effort remains very much a work in progress: LXQt requires some components of Plasma 5, the new major KDE workspace.

We imported some core components (it was necessary to update to x11/qterminal 0.7.0):

Standalone applications:

We also have updates for:

Improve support in sysutils/lxqt-admin (especially date and time settings). Xfce on &os; &os; Xfce Team xfce@FreeBSD.org &os; Xfce Project &os; Xfce Repository

Xfce is a free software desktop environment for Unix and Unix-like platforms such as &os;. It aims to be fast and lightweight, while still being visually appealing and easy to use.

During this quarter, the team has kept these applications up-to-date:

We also follow the unstable releases (available in our experimental repository) of:

Apply the changes discussed in D8416 (simplify the MASTER_SITES macro in port Makefiles) Commit the stable panel plugins OpenBSM Christian Brueffer brueffer@FreeBSD.org Robert Watson rwatson@FreeBSD.org TrustedBSD Audit Mailing Mist trustedbsd-audit@TrustedBSD.org OpenBSM: Open Source Basic Security Module (BSM) Audit Implementation OpenBSM on GitHub &os; Audit Handbook Chapter OpenBSM 1.2 alpha 5 announcement - CADETS project + DARPA CADETS project

OpenBSM is a BSD-licensed implementation of Sun's Basic Security Module (BSM) API and file format. It is the user-space side of the CAPP Audit implementations in &os; and Mac OS X. Additionally, the audit trail processing tools are expected to work on Linux.

This quarter saw increased development activity, fueled by - the CADETS project, resulting in the release of OpenBSM 1.2 + the DARPA CADETS project, resulting in the release of OpenBSM 1.2 alpha 5. Among this release's changes are the ability to specify the kernel's maximum audit queue length, sandboxing support for auditreduce(1) and praudit(1) on &os; and other systems that support Capsicum, as well as the addition of event identifiers for more &os; system calls. The complete list of changes is documented in the NEWS file on GitHub. The new release will be merged into &os; HEAD and the supported STABLE branches shortly.

Test the new release on different versions of &os;, Mac OS X, and Linux. In particular, testing on the latest versions of Mac OS X would be greatly appreciated. Fix problems that have been reported via GitHub and the &os; bug tracker. Implement the features mentioned in the TODO list on GitHub. + + + Portions of this work were sponsored by DARPA/AFRL. + The &os; Core Team &os; Core Team core@FreeBSD.org

The major concern for Core during the last quarter of 2016 has been about maintaining the effectiveness of secteam. The team is primarily in need of better project management, both to improve communication generally and to allow the other team members to concentrate on the technical aspects of handling vulnerabilities.

To that end, there has been agreement in principle for either the FreeBSD Foundation or one of the companies that are major &os; users to employ someone specifically in this role.

Core confirmed that the new support model would go into effect with 11.0-RELEASE despite the postponement of the switch to a packaged base release mechanism. For details of the new support model, please follow the links from the security page of the &os; website.

Core requested the removal of the misc/jive port, on the grounds that it had no function other than to turn text into an offensively racist parody. This proved controversial, with many seeing this as a first step in bowdlerizing the entire ports tree. That is certainly not Core's intention. Core's aim here is to help secure the future of the &os; project by making it welcoming to all contributors, regardless of ethnicity, gender, sexuality or other improper bases for discrimintation. While misc/jive may once have been seen as harmless fun, today the implicit approval implied by having it in the ports tree sends a message at odds with the project's aims.

The Marketing team and the associated marketing@FreeBSD.org mailing list were wound up, due to lack of activity. Messages to marketing@FreeBSD.org will be forwarded to the FreeBSD Foundation's marketing team instead.

Core member Allan Jude, who was already the clusteradm liason, became a full member of clusteradm.

An emergency correction to the 11.0 release notes was authorised, as it was giving the misleading impression that 802.11n wireless support had only just been added, and this misapprehension was being repeated in the press. In reality, &os; has had 802.11n support for many years, and the announcement should have said that support had been added to many additional device drivers.

Discussions about a proposal to improve Unicode support are on-going. &os; is already standards conformant, but the propsal is to switch to a __STDC_ISO_10646_ implementation, similar to what Linux glibc currently uses. Opinions are divided on the technical merits of the new approach.

There were the usual quota of queries about licensing and other legal matters:

During this quarter four new commit bits were awarded. Please welcome Dexuan Cui, David Bright, Konrad Witaszczyk, and Piotr Stefaniak. We were sorry to see Edwin Lansing hang up his commit bits and step down from portmgr.

 Ports Collection René Ladan portmgr-secretary@FreeBSD.org &os; Ports Management Team portmgr@FreeBSD.org About &os; Ports Contributing to Ports &os; Ports Monitoring Ports Management Team &os; portmgr on Twitter (@FreeBSD_portmgr) &os; Ports Management Team on Facebook &os; Ports Management Team on Google+

The Ports Tree has reached the marker of 27,000 ports, with the PR count risen slightly to around 2,250. Of these PRs, 572 are unassigned. The last quarter saw 6871 commits by 176 committers. The number of open and the number of unassigned PRs both increased lightly since last quarter.

Two commit bits were taken in for safe keeping in the last quarter: jmg after 19 months of inactivity, and edwin at his own request. We welcomed three new committers: Nikolai Lifanov (lifanov), Jason Bacon, and Mikhail Pchelin (misha).

On the management side, adamw and feld were elected as new portmgr members, and rene was promoted to full member. feld is already involved in ports-secteam.

On the infrastructure side, two new USES (lxqt and varnish) were introduced. Some default versions were also updated: varnish 4 (new), GCC 4.8 to 4.9, Perl 5.20 to 5.24, and Python 3.4 to 3.5. Two major ports reached their end-of-life at December 31st and were removed: Perl 5.18 and Linux Fedora 10 (the default is Linux CentOS 6). Because &os; 9.3, 10.1, and 10.2 also reached end-of-life, support for those versions was removed from the Ports Tree.

Some major ports were updated to their latest versions: pkg to 1.9.4, Firefox to 50.1.0, Firefox-esr to 45.6.0, Chromium to 54.0.2840.100, and Ruby to 2.1.10 / 2.2.6 / 2.3.3. www/node was updated to version 7; version 6 was split off as www/node6 for long-term support.

Behind the scenes, antoine ran 39 exp-runs to verify package updates, framework changes, and changes to the base system. bdrewery installed new package builders and added builds for &os; 11 for mips, mips64, and armv6. He also improved the balancing, monitoring, automation of the package builders.

If you have some spare time, please take up a PR for testing and committing. I2C, GPIO, and SPI support for MinnowBoard Oleksandr Tymoshenko gonzo@FreeBSD.org Blog Post MinnowBoard Website

The Minnowboard is an Atom-based x86 board (Intel E38xx Series SoC) in a maker-friendly form-factor: it provides convenient access to pins that can be used to connect peripherals using one of the standard buses: GPIO, SPI, or I2C. These buses are more common in the ARM/MIPS world than in x86, so while &os; was able to boot just fine, it lacked support for these buses on the MinnowBoard.

As of r310645, HEAD support all three buses via ig4(4), bytgpio(4), and intelspi drivers. The ig4(4) and bytgpio(4) changes were backported to 11-STABLE; intelspi will be MFCed in couple of weeks.