Index: head/en_US.ISO8859-1/htdocs/news/status/report-2017-01-2017-03.xml =================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2017-01-2017-03.xml (revision 50197) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2017-01-2017-03.xml (revision 50198) @@ -1,1187 +1,1282 @@ January-March 2017
Introduction

This is a draft of the January–March 2017 status report. Please check back after it is finalized, and an announcement email is sent to the &os;-Announce mailing list.

This report covers &os;-related projects between January and March 2017. This is the first of four reports planned for 2017.

The first quarter of 2017 was another productive quarter for the &os; project and community. [...]

Thanks to all the reporters for the excellent work!

The deadline for submissions covering the period from April to June 2017 is July 7, 2017.

?>
team &os; Team Reports proj Projects kern Kernel arch Architectures bin Userland Programs ports Ports doc Documentation misc Miscellaneous The &os; Dutch Documentation Project Rene Ladan rene@FreeBSD.org Remko Lodder remko@FreeBSD.org The Dutch Translation Project

Work has started on an initial translation of the &os; Handbook to the Dutch language via the 'po' system. While we have an (outdated) version of the Handbook available via the older XML files, we are now trying to get back into shape with the po file.

Rene started working on 2 articles already and did some translation strings for the FDP-Primer, while Remko had started working on the Handbook. If you think you can assist with that, please let Rene and Remko know on their email addresses so that we can start coordinating work.

In addition, since we have a translation set already from the XML files, it would be interesting to see and know whether we can merge them easily into the po structure. If you have ideas on that, contact us a.s.a.p.

Part of this work is facilitated by Remko's employer: Snow B.V.

Snow B.V. Identify a way to merge the current XML translations into the nl_NL.po files. Merge the translations into the .po files. Update the remaining/open items into the po files. Remove the old/outdated translation files from the main repo and use the po and book.xml files to generate the dutch handbook and other files. Identify whether we can also translate the htdocs pages via the po system. &os; s390x attempt III Bjoern A. Zeeb bz@FreeBSD.org

A long time ago, in the &os; 5 times, there was an initial port of &os; to s390 (32bit) and s390x (64bit) which booted past init on good days in an emulator.

As an attempt to revive the s390x/systemz efforts I started to get &os; s390x to build with clang/llvm 3.90. At this time it is possible to build world and a GENERIC kernel skeleton (not doing anything yet) using external binutils.

The primary idea of this initial work was to allow to incrementally add the neccessary architecture-specific code. Having the build framework done will allow third-party developers to simply type make, as they are willing to contribute to the port without having to know &os; build specifics. After some cleanup and further updates to a more recent HEAD I am planning to push the current work to a public repo to allow collaboration.

Write a wiki page with per-architecture specific tasks that need to be done based on the current work and the experience from arm64 and riscv. Implement both the user space and kernel per-architecture gaps. Figure out a way to get access to IBM's zPDT or better emulators to ease implementation, testing, and debugging. 64-bit PowerPC Book-E support Justin Hibbits jhibbits@FreeBSD.org

The Book-E platform target now supports 64-bit mode ("powerpc64"). It includes a 63-bit address space split, but the page table directory list uses holes to expand to the full address space, leaving gaps in the address space where page mappings are repeated. This may change in the future.

As with the AIM powerpc64 port, this supports running powerpc (32-bit) binaries as well, and has even been tested with a 32-bit init and 64-bit shell.

Several of the SoC drivers are supported, however, the dTSEC ethernet controller is not yet supported. Work is ongoing to support this.

A QORIQ64 config is included, targeting the P5 and T* series SoCs from Freescale.

Thanks to Juniper Networks for providing patches against an older internally maintained &os; version, which enabled this porting effort, and for providing historical context for quirks of the pmap changes.

Port the dTSEC driver to 64-bit. There are assumptions in the reference driver of operating in a 32-bit environment. It may be easier to port the Linux driver instead, which would also give ARM support. Take advantage of pointer alignment to squeeze more bits out of the page tables; it should be possible to squeeze at least 3 more bits out, one at each level. pNFS server Plan B Rick Macklem rmacklem@FreeBSD.org

Parallel NFS (pNFS) is an extension to the NFSv4 protocol that allows for file accesses within a single logical mount to be performed against multiple NFS file servers, "in parallel". Thus, a pNFS server separats the MetaData operations from the Data operations (Read/Write/Setattr of size N), letting existing NFSv3 servers be used for the bulk data storage while still taking advantage of NFSv4 protocol enhancements, among other things.

My first attempt at a pNFS server using GlusterFS was a dud. It worked, but performance was so poor that it was not usable. This attempt that I call Plan B, only uses &os;, with one &os; server handling the MetaData operations and K &os; servers configured to serve Data. An NFSv4.1 client that supports the pNFS File Layout will be able to Read/Write to the Data servers directly, spreading out the RPC load and allowing growth beyond that of what a single &os; NFS server could achieve.

There is no support for the Flex Files Layout or mirroring at this time. I hope to use the Flex Files Layout to add mirroring support over the next year or so. Striping is also not supported, but I have no plans for implementing it at the moment.

Plan B is working quite well now and should be available for testing by the end of April. I will announce how to do this on the freebsd-fs@FreeBSD.org mailing list when it is available.

Testing by others will be needed, once it is available. OpenBSM Christian Brueffer brueffer@FreeBSD.org Robert Watson rwatson@FreeBSD.org TrustedBSD audit mailing list trustedbsd-audit@TrustedBSD.org OpenBSM: Open Source Basic Security Module (BSM) Audit Implementation OpenBSM on GitHub &os; Audit Handbook Chapter DTrace Audit Provider DARPA CADETS project TODO List on GitHub

OpenBSM is a BSD-licensed implementation of Sun's Basic Security Module (BSM) API and file format. It is the user space side of the CAPP Audit implementations in &os; and Mac OS X. Additionally, the audit trail processing tools are expected to work on Linux.

During this quarter, experimental support for UUIDs in BSM trails was added to OpenBSM. A DTrace audit provider using this functionality has been developed as part of the DARPA CADETS project and is in review (https://reviews.FreeBSD.org/D10149). In the OpenBSM GitHub repository, support for Coverity static analysis was added via TravisCI. Additionally, the OpenBSM 1.2-alpha5 release has been merged into the &os; HEAD branch.

Test the latest release on different versions of &os;, Mac OS X and Linux. Testing on the latest versions of Mac OS X would be particularly appreciated. Fix problems that have been reported via GitHub and the &os; bug tracker. Implement features mentioned in the TODO list on GitHub. DARPA/AFRL (in part) TrustedBSD Christian Brueffer brueffer@FreeBSD.org Robert Watson rwatson@FreeBSD.org TrustedBSD announce mailing list trustedbsd-announce@TrustedBSD.org TrustedBSD Website TrustedBSD on GitHub

The TrustedBSD Project is an open source community developing advanced security features for the open source &os; operating system. Started in April 2000, the project developed support for extended attributes, access control lists (ACLs), UFS2, OpenPAM, security event auditing, OpenBSM, a flexible kernel access control framework, mandatory access control, and the GEOM storage layer. The results of this work may be found not just in &os;, but also NetBSD, OpenBSD, Linux, and Apple's Mac OS X and iOS operating systems. Today, the project continues to maintain and enhance these mature features in &os;.

During this quarter, the TrustedBSD project transitioned from the &os; Perforce server to GitHub. This was made possible by Alexis Sarghel, who owned the user "trustedbsd" on GitHub and graciously transferred this account to the TrustedBSD project. To date, the repositories hosting the TrustedBSD website and the SEBSD repository have been moved.

 MySQL Mahdi Mokhtari mmokhi@FreeBSD.org Mark Felder feld@FreeBSD.org MySQL80 Overview MySQL80 InnoDB New Features

This quarter a new -dev version of MySQL landed in the port tree, MySQL 8.0. It introduces many new features, though we had to (re)-patch parts of it which were merged by MySQL from MySQL5.7.

We also updated MySQL 5.6 to latest version and closed bunch of PRs related to it, mostly about using &os;-provided ports for librariess instead of the bundled copies. And of course there were plenty of security updates.

We can also report that the problem of having to specify ${mysql_optfile}, which some people encountered while using MySQL, is now considered to be solved in all MySQL versions: 5.6, 5.7, and 8.0. Now the init script will search all default locations, for backwards compatibility with the variety of locations used for configuration files before it gives up and reports an error.

Test the new version and report problems. Linuxulator Dimitry Chagin dchagin@FreeBSD.org Edward Tomasz Napiera trasz@FreeBSD.org Mahdi Mokhtari mmokhi@FreeBSD.org

In this quarter, we are pleased to announce two (of many) works achieved in the Linuxulator.

We added a new placeholder marker UNIMPLEMENTED to accompany the previously existing DUMMY, for distinguishing syscalls that the Linux kernel itself does not implement from those that we currently do not implement. Now our linux_dummy.c is clearer for the newcomers to follow, and they will quickly know which areas they can start working on.

Support for two new syscalls, preadv and pwritev, was added to the Linuxulator.

We plan to implement the execveat syscall for the native &os; syscall table and then port/wrap it for use in the Linuxulator. Intel 10G and 40G Network Driver Updates Jeb Cramer jeb.j.cramer@intel.com Eric Joyner eric.joyner@intel.com Krzysztof Galazka krzysztof.galazka@intel.com Commit adding X553 ix/ixv Support for iflib Commit converting ixgbe to iflib

This driver update is for the Intel ix/ixv and ixl/ixlv network drivers, and includes support for several new hardware releases.

ix/ixv:

ixl/ixlv:

ix/ixv iflib support is currently under review in Phabricator. It will be refactored to include D5213. Initial work for ixl/ixlv iflib support is in progress. &os; on Hyper-V and Azure Sepherosa Ziehau sepherosa@gmail.com Hongjiang Zhang honzhan@microsoft.com Dexuan Cui decui@microsoft.com Kylie Liang kyliel@microsoft.com &os; Virtual Machines on Microsoft Hyper-V Supported Linux and &os; virtual machines for Hyper-V on Windows

SR-IOV support for NICs is implemented. So far, we have only tested with the Mellanox ConnectX-3 VF card, which works despite some issues (Bug 216493: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=216493).

Updates for UEFI VMs (i.e., Hyper-V Generation 2 VM):

  1. After the loader issue (Bug 211746) is fixed, UEFI VMs can now boot with Secure Boot disabled;
  2. A synthetic keyboard driver has been added. Currently it is only in HEAD, but MFCs to stable/10 and stable/11 are planned to occur soon;
  3. A SCSI DVD detection issue (Bug 218248) was fixed. Without the fix, the VM would fail to boot.
Microsoft Porting Software to CloudABI: Sandboxed Bitcoin! Ed Schouten ed@FreeBSD.org How to use CloudABI on &os; LevelDB for CloudABI Memcached for CloudABI Bitcoin for CloudABI

CloudABI is a framework that allows you to develop strongly sandboxed applications a lot more easily. It is a programming environment that exclusively uses &os;'s Capsicum facilities. Any features incompatible with Capsicum have been removed entirely, which means that it is easier to determine how code needs to be adjusted to behave correctly while sandboxed. In essence, you only need to patch up the code until it builds.

Last year we've managed to port a lot of exciting libraries over to CloudABI. Highlights include sandboxing aware versions of Boost and LevelDB. Now that these libraries are readily available, we're at the point where we can shift focus towards porting full applications.

Late February one of the lead developers of the Bitcoin reference implementation got in touch, as he is very interested in creating a copy of Bitcoin that is better protected against security bugs. You do not want a security bug in the networking/consensus code allowing an attacker to steal coins from your local wallet.

As I think that this is a use case that demonstrates the strength of CloudABI well, I've made addressing any issues reported by the Bitcoin developers a top priority. Once the Bitcoin port is complete, we want to provide binary packages of it as well.

Nuxi, the Netherlands Though getting Bitcoin to work is pretty awesome, don't let that distract us from porting other pieces of software over as well! Are you the maintainer of a piece of software that could benefit from sandboxing? Be sure to try building it using the CloudABI toolchain! One of the pieces of software that got ported over to CloudABI some time ago is Memcached. Are you a user of Memcached? If so, feel free to give the sandboxed version of Memcached for CloudABI a try! So far CloudABI can be used to run software written in C, C++ and Python. Would you like to see any other programming language work on CloudABI as well? Be sure to help out! Ports Collection René Ladan portmgr-secretary@FreeBSD.org &os; Ports Management Team portmgr@FreeBSD.org About &os; Ports Contributing to Ports &os; Ports Monitoring Ports Management Team &os; portmgr on Twitter (@FreeBSD_portmgr) &os; Ports Management Team on Facebook &os; Ports Management Team on Google+

The number of ports is currently just 500 short of 30,000. The current number of PRs is close to 2,400, of which 620 are unassigned. The last quarter saw 6656 commits from 167 comitters. Both the number of ports and the number of unassigned PRs have increased in the last quarter.

In the last quarter, we welcomed 7 new committers: Eugene Grosbein (eugen), Johannes Dieterich (jmd), Larry Rosenman (ler), Mahdi Mokhtari (mmohki), Matthew Rezny (rezny), Tobias Kortkamp (tobik), and Vladimir Krstulja (?). dumbbell@ was already a src committer and got an extension for the Ports Tree. We also welcomed back krion@ and miwi@. We took 6 bits in for safe-keeping: itetcu@, leeym@, mva@, olivierd@, pgollucci@, and sanpei@.

There were no changes to the membership of portmgr.

antoine@ worked on USES=samba to prepare for the removal of the long-outdated Samba 3.6 ports and replace them with modern versions. The new default versions are: FreePascal 3.0.2, Ruby 2.3, and Samba 4.4. A new variable USE_LOCALE was createdto add the LANG and LC_ALL environment variables to all builds. Out-of-tree patches can now be added with the new EXTRA_PATCH_TREE variable. The error messages for invalid SINGLE options were improved.

Some of the major port updates last quarter were: pkg 1.10.1, linux c6_64, Firefox 52.0.2, Chromium 57.0.2987.110, GCC 4.9.4, Gnome 3.18.0, Xorg 1.18.4, Qt 4.8.7 and 5.7.1, and PHP 7.1.

antoine@ ran 31 exp-runs to test version updates and under-the-hood changes.

The number of unassigned and open PRs is still growing, so if you have some spare time, please close some of those. Rust Jean-Sébastien Pédron dumbbell@FreeBSD.org Thomas Zander riggs@FreeBSD.org Wiki Portal Guide to Bootstrap Rust on &os; Bug Report to Track Progress on Bootstrapping

In the Ports Collection, Rust was updated to 1.16.0 and Cargo to 0.17.0, the latest versions at the time of this writing.

lang/rust-nightly was also updated to a snapshot from February and it is now enabled on i386. It is lagging a bit behind upstream, but Rustup works nicely on &os; if you need to try any versions/channels of Rust.

Work has started to bootstrap Rust on non-x86 architectures. Patches to add &os;/aarch64 support were submitted and accepted upstream. &os;/sparc64 is in progress. The lang/rust-nightly port is also being adapted to compile natively on &os;/aarch64. This work is critical, in particular because Firefox will shortly require Rust. If you want to help, please refer to the guide linked above.

The compiler, rustc, is crashing sometimes when there is a compilation error. Therefore, there is a bit of work to do to improve stability.

There is some code duplication between lang/rust* and devel/cargo. Those Makefiles deserve a bit of cleanup. It might be useful to create a USES=rust Makefile helper.

Bootstrap Rust on more platforms. Investigate compiler crashes. Create a USES=rust Makefile helper and simplify Rust and Cargo ports. Investigate how to speed up lang/rust* compilation time. &os; Release Engineering Team &os; Release Engineering Team re@FreeBSD.org &os; 11.1-RELEASE Schedule &os; development Snapshots

The &os; Release Engineering Team is responsible for setting and publishing release schedules for official project releases of &os;, announcing code freezes and maintaining the respective branches, among other things.

The &os; Release Engineering Team continued producing weekly development snapshots for the 12-CURRENT, 11-STABLE, and 10-STABLE branches.

In addition, the &os; 11.1-RELEASE schedule was added to the Project website. Please note, however, the schedule on the website is still subject to change.

The &os; Foundation &os; on Marvell Armada38x Marcin Wojtas mw@semihalf.com Zbigniew Bodek zbb@semihalf.com

Final testing and productionization of support for the Marvell Armada38x platform is underway. The rebase and cleanup is going well, with support on top of HEAD and ready for upstreaming.

Specific tasks completed include:

Stormshield Semihalf Submit the remaining fixes and drivers. Support for eMMC flash and Faster SD Card Modes Marius Strobl marius@FreeBSD.org

In r315430, support for eMMC partitions has been added to mmc(4) and mmcsd(4) in &os; 12. Besides the user data area, i.e., the default partition, eMMC v4.41 and later devices can additionally provide up to:

Apart from simply subdividing eMMC flash devices or having UEFI code in the boot partition, as is done on some Intel NUCs, another use case for partition support is the activation of pseudo-SLC mode, which manufacturers of eMMC chips typically associate with the enhanced user data area and/or the enhanced attribute of general purpose partitions.

In order to be able to partition eMMC devices, r315430 also added a Linux-compatible IOCTL interface to mmcsd(4). This allows for using the GNU mmc-utils (found in ports as sysutils/mmc-utils) on &os;. Besides partitioning eMMC devices, the mmc tool can also be used to query for life time estimate and pre-EOL information of eMMC flash, as well as to query some basic information from SD cards.

CAVEAT EMPTOR: Partitioning eMMC devices is a one-time operation.

Additionally, in order to make eMMC flash devices more usable, support for DDR (Dual Data Rate) bus speed mode at a maximum of 52 MHz (DDR52) has been added to mmc(4) and sdhci(4) in r315598, i.e., in &os; 12. Compared to high speed mode (the previous maximum) at 52 MHz, DDR52 mode increases the performance of the tested eMMC chips from ~45 MB/s to ~80 MB/s.

So far, support for DDR52 mode has been enabled for the eMMC controllers found in Intel Apollo Lake, Bay Trail and Braswell chipsets. Note, however, that the eMMC and SDHCI controllers of the Apollo Lake variant occasionally lock up due to a silicon bug (which is independent of running in DDR52 mode). The only viable workaround for that problem appears to be the implementation of support for ADMA2 mode in sdhci(4) (currently, sdhci(4) supports the encumbered SDMA mode only, or no DMA at all).

However, r315598 also already brought in infrastructure and a fair amount of code for using even faster transfer modes with eMMC devices and SD cards respectively, i.e., up to HS400ES with eMMC and the UHS-I modes up to SDR104 with SD cards.

The intent is to merge these changes back to &os; 10 and 11.

Add support for eMMC HS200, HS400 and HS400ES transfer modes. Add support for SD card UHS-I transfer modes (SDR12 to SDR104). Make mmcsd(4) more robust and correctly follow the relevant specifications for existing features, e.g., calculate and handle erase timeouts, do a SEND_STATUS when CMD6 is invoked with an R1B response to the extent not already fixed as part of r315430, get the remainder of the existing code to properly check and handle return codes, etc. Ceph on &os; Willem Jan Withagen wjw@digiware.nl Ceph Main Site Main Repository My &os; Fork

Ceph is a distributed object store and file system designed to provide excellent performance, reliability and scalability.

I started looking into Ceph, because the HAST solution with CARP and ggate did not really do what I was looking for. But I aim to run a Ceph storage cluster of storage nodes that are running ZFS. User stations would be running bhyve on RBD disk that are stored in Ceph.

The &os; build will build most of the tools in Ceph.

The most notable progress since the last report:

To get things running on a &os; system, run pkg install net/ceph-devel or clone https://github.com/wjwithagen/ceph and build manually by running ./do_freebsd.sh in the checkout root.

Parts not (yet) included:

Run integration tests to see if the &os; daemons will work with a Linux Ceph platform. Compile and test the userspace RBD (Rados Block Device). This currently works but testing has been limitted. Investigate and see if an in-kernel RBD device could be developed akin to ggate. Investigate the keystore, which can be embedded in the kernel on Linux and currently prevents building Cephfs and some other parts. The first question is whether it is really required, or only KRBD requires it. Scheduler information is not used at the moment, because the schedulers work rather differently between Linux and &os;. But at a certain point in time, this will need some attention (in src/common/Thread.cc). Improve the &os; initscripts in the Ceph stack, both for testing purposes and for running Ceph on production machines. Work on ceph-disk and ceph-deploy to make it more &os;- and ZFS- compatible. Build a test cluster and start running some of the teuthology integration tests on it. Teuthology wants to build its own libvirt and that does not quite work with all the packages &os; already has in place. There are many details to work out here. Design a vitual disk implementation that can be used with bhyve and attached to an RBD image. + + + The &os; Core Team + + + + &os; Core Team + core@FreeBSD.org + + + + +

Core's primary function is to ensure the long-term viability of + the &os; project. A very large part of that is to ensure that the + interactions between developers remain cordial, and consequently + that the project appears welcoming to newcomers.

+ +

Normally, most of Core's activities around this are done in + private -- a quiet word in the right ear, some discrete + peacemaking, occasional reading of the riot act. Most of the time, + this is all that is necessary.

+ +

Unfortunately, this quarter we had an instance where such private + measures failed to achieve the desired result, and we ended up + ejecting a developer. John Marino is an extremely talented + programmer and has made significant contributions to the Ports + Collection. Despite this, portmgr found him to be + sufficiently disruptive and abrasive that in their judgement, + the project was better off overall to sever his connection to + itself, and core backed them up in that. We are sorry that + events came to this sad conclusion, but we remain convinced + that this was a necessary step to safeguard the character of + our community.

+ +

In a more positive light, Core has been working on a proposal + to recognise notable contributors to the &os; project who are not + (or perhaps not yet) suitable to be put forward as new + committers. In addition to the usual routes of recognising people + that write numbers of good bug reports or that supply patches or + that volunteer to maintain ports, this will also allow recognition + of people who contribute by such things as organising &os; events + or who promote &os; through social media. A formal announcement + of Core's proposal is imminent.

+ +

During January, the core secretary held an exercise to contact + all source committers who had been inactive for more than 18 + months and persuade them to hand in their commit bits if they + weren't planning to resume working on &os; in the near future. + This is meant to be a routine function -- the "grim reaper" -- + that aims to keep the list of people with the ability to commit + pretty much in synchrony with the list of people that are actively + committing. The regular process had fallen out of activity + several years ago, and we needed to clear the decks before + restarting. Ultimately, this resulted in some 20 + developers-emeritus handing in their commit bits.

+ +

No new commit bits were awarded during this quarter.

+ +

Core is also taking soundings on producing a 10.4-RELEASE. + This is not in the current plan, but a number of developers and + important &os; users would be keen to see it happen, given some of + the work that has gone into the stable/10 branch since + 10.3-RELEASE. On the other hand, this would require an additional + support burden for SecTeam, including maintaining versions of + software that have been declared obsolete upstream, in particular + OpenSSL. As an even-numbered release, 10.4-RELEASE would have a + "normal" rather than an "extended" lifetime which means it should + not result in extending the support lifetime of the stable/10 + branch.

+ +

In other news, Core arranged for the old and largely inactive + marketing@FreeBSD.org mailing list to be wound up, and for any + remaining activities to be transferred to the &os; Foundation.

+ +

Core also asked clusteradm to turn off Internet-wide access to + the finger server on freefall.freebsd.org. Many + developers have included details such as phone numbers into + the GECOS field of their &os; password database entries, and + these would be revealed by the finger server -- + details which are nowadays generally felt inadvisable to + expose publicly. finger is still available + internally within freefall.freebsd.org. Core recommends that + GECOS data is limited to just your full name, and we have + updated the standard "new committer" e-mail template + to reflect that.

+ +

Core is looking for new volunteers to help out with several of + the teams that manage various aspects of the project. In + particular, Postmaster and SecTeam are in need of new blood. + Recruiting a new member of SecTeam is well underway, but anyone + interested in joining any of the teams is encouraged to make + themselves known either to Core, or directly to the teams + concerned.

+ +