Introduction

Index: head/en_US.ISO8859-1/htdocs/news/status/Makefile =================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/Makefile +++ head/en_US.ISO8859-1/htdocs/news/status/Makefile @@ -88,6 +88,7 @@ XMLDOCS+= report-2019-10-2019-12 XMLDOCS+= report-2020-01-2020-03 XMLDOCS+= report-2020-04-2020-06 +XMLDOCS+= report-2020-07-2020-09 XSLT.DEFAULT= report.xsl # Install a sample entry. Index: head/en_US.ISO8859-1/htdocs/news/status/report-2020-07-2020-09.xml =================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2020-07-2020-09.xml +++ head/en_US.ISO8859-1/htdocs/news/status/report-2020-07-2020-09.xml @@ -0,0 +1,2172 @@ + + + + + + + + + + 07-09 + + 2020 + + +

+ Introduction +

This report covers FreeBSD related projects for the period between +July and September, and is the third of four planned reports for 2020. +

This quarter brings a good mix of additions and changes to the FreeBSD +Project and community, from a diverse number of teams and people covering +everything from architectures, continuous integration, wireless networking +and drivers, over drm, desktop and third-party project work, as well as +several team reports, along with many other interesting subjects too +numerous to mention. +

As the world is still affected by the epidemic, we hope that this report +can also serve as a good reminder that there is good work that can be done +by people working together, even if we're apart. +

We hope you'll be as interested in reading it, as we've been in making it. +Daniel Ebdrup Jensen, on behalf of the quarterly team. +

+ +FreeBSD Foundation + + + +Deb Goodkin +deb@FreeBSDFoundation.org + + + +

The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to +supporting and promoting the FreeBSD Project and community worldwide. Funding +comes from individual and corporate donations and is used to fund and manage +software development projects, conferences and developer summits, and provide +travel grants to FreeBSD contributors. The Foundation purchases and supports +hardware to improve and maintain FreeBSD infrastructure and provides resources +to improve security, quality assurance, and release engineering efforts; +publishes marketing material to promote, educate, and advocate for the FreeBSD +Project; facilitates collaboration between commercial vendors and FreeBSD +developers; and finally, represents the FreeBSD Project in executing contracts, +license agreements, and other legal arrangements that require a recognized +legal entity. +

Here are some highlights of what we did to help FreeBSD last quarter: +

COVID-19 Impact to the Foundation

+ +

Like other organizations, we put policies in place for all of our staff members +to work from home. We also put a temporary ban on travel for staff members. +We are continuing our work supporting the community and Project, but some of +our work and responses may be delayed because of changes in some of our +priorities and the impact of limited childcare for a few of our staff members. +

Partnerships and Commercial User Support

+ +

We help facilitate collaboration between commercial users and FreeBSD +developers. We also meet with companies to discuss their needs and bring that +information back to the Project. Not surprisingly, the stay at home orders, +combined with our company ban on travel during Q3 made in-person meetings +non-existent. However, the team was able to continue meeting with our partners +and commercial users virtually. These meetings help us understand some of the +applications where FreeBSD is used. +

We are currently scheduling Zoom company meetings for Q4, please reach out if +you would like to schedule a meeting with us. +

Fundraising Efforts

+ +

Last quarter we raised $192,874.43! Thank you to the individuals and +organizations that stepped in, to help fund our efforts. We'd like to thank +Arm for their large contribution last quarter, which helped bring our 2020 +fundraising effort to $521k. We hope other organizations will follow their +lead and give back to help us continue supporting FreeBSD. +

These are trying times, and we deeply appreciate every donation that has come +in from $5 to $150,000. We're still here giving 110% to supporting FreeBSD! +

We are 100% funded by donations, and those funds go towards software +development work to improve FreeBSD, FreeBSD advocacy around the world, keeping +FreeBSD secure, continuous integration improvements, sponsoring BSD-related and +computing conferences (even the virtual events!), legal support for the +Project, and many other areas. +

Please consider making a +donation to help us continue and increase our support for FreeBSD. +

We also have the Partnership Program, to provide more benefits for our larger +commercial donors. Find out more information about the +partnership program +and share with your companies! +

OS Improvements

+ +

A number of FreeBSD Foundation grant recipients started, continued working on, +or completed projects during the third quarter. These include: +

Ongoing WiFi and Linux KPI layer improvements. +
Linuxulator application compatibility. +
DRM / Graphics driver updates. +
Zstd compression for OpenZFS. +
Online RAID-Z expansion. +
Modernized LLDB target support for FreeBSD. +
+

+You can find more details about most of these projects in other quarterly +

reports. +

Staff members also worked on a number of larger projects, including: +

Run-Time Dynamic Linker (rtld) and kernel ELF loader improvements. +
Rewritten UNIX domain socket locking. +
Build infrastructure. +
Open system call path handling support for O_BENEATH, O_RESOLVE_BENEATH. +
arm64 support. +
Migration to a Git repository. +
+

+Many of these projects also have detailed entries in other quarterly report +

entries. +

Staff members also put in significant effort in many ways other than larger, +individual projects. These include assisting with code reviews, bug report +triage, security report triage and advisory handling, addressing syzkaller +reports, and ongoing maintenance and bug fixes in functional areas such as the +tool chain, developer tools, virtual memory kernel subsystem, low-level x86 +infrastructure, sockets and protocols, and others. +

University of Waterloo Co-op

+ +

With the transition to working from home, the Foundation decided to again take +on three University of Waterloo Co-op students for the Fall 2020 term +(September to December). Tiger returns for a second term, joined by new +students Yang and Zac. Projects for the term include more work on +ELF Tool Chain, application of Capsicum to additional utilities, testing and +integration of FreePBX and Asterisk VOIP software, pkgbase, and exploring +containerization tooling. +

Continuous Integration and Quality Assurance

+ +

The Foundation provides a full-time staff member and funds projects on +improving continuous integration, automated testing, and overall quality +assurance efforts for the FreeBSD project. +

During the third quarter of 2020, Foundation staff continued improving and +monitoring the Project's CI infrastructure, and working with experts to fix +the failing builds and the regressions found by tests. The setting up of +dedicated VM host for running tests is completed. New feature developments +and the CI staging environment is in progress. We are also working with +other teams in the Project for their testing needs. For example, tests of +non-x86 architectures now run periodically, and improve the CI of the +embedded systems. We are also working with many external projects and +companies to improve the CI between their products and FreeBSD. +

See the FreeBSD CI section of this report for completed work items and detailed +information. +

Supporting FreeBSD Infrastructure

+ +

The Foundation provides hardware and support to improve the FreeBSD +infrastructure. Last quarter, we continued supporting FreeBSD hardware located +around the world. We coordinated efforts between the new NYI Chicago facility +and clusteradm to start working on getting the facility prepared for some of +the new FreeBSD hardware we are planning on purchasing. NYI generously +provides this for free to the Project. We also worked on connecting with the +new owners of the Bridgewater site, where most of the FreeBSD infrastructure is +located. +

Some of the purchases we made for the Project last quarter to support +infrastructure includes: +

Spamhaus spam filtering software to limit the amount of spam on the mailing + lists. +
5 application servers to run tasks like bugzilla, wiki, website, cgi, + Phabricator, host git, etc. +
1 server to replace the old pkg server and provide a lot more IOPS to + avoid the slowdowns seen during peak times of the day where the disks just + cannot keep up with the request volume. +
1 server for exp-runs to make them faster. +
1 server to build packages more frequently. +
+

FreeBSD Advocacy and Education

+ +

A large part of our efforts are dedicated to advocating for the Project. This +includes promoting work being done by others with FreeBSD; producing advocacy +literature to teach people about FreeBSD and help make the path to starting +using FreeBSD or contributing to the Project easier; and attending and getting +other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD +tables, and give FreeBSD presentations. +

The FreeBSD Foundation sponsors many conferences, events, and summits around +the globe. These events can be BSD-related, open source, or technology events +geared towards underrepresented groups. We support the FreeBSD-focused events +to help provide a venue for sharing knowledge, to work together on projects, +and to facilitate collaboration between developers and commercial users. This +all helps provide a healthy ecosystem. We support the non-FreeBSD events to +promote and raise awareness of FreeBSD, to increase the use of FreeBSD in +different applications, and to recruit more contributors to the Project. As is +the case for most of us in this industry, COVID-19 has put our in-person events +on hold. In addition to attending virtual events, we are continually working +on new training initiatives and updating our selection of how-to guides to +facilitate getting more folks to try out FreeBSD. +

Check out some of the advocacy and education work we did last quarter: +

Launched our FreeBSD Fridays series of 101 classes. Topics included an + Introduction to FreeBSD, FreeBSD Installfest, Introduction to Security, + Introduction to ZFS and more. Videos of the past sessions and a schedule of + upcoming events can be found here. +
Attended and presented at OSI's State of the Source conference. The event + was held virtually, September 9-11, 2020. +
Launched the + redesign + of the FreeBSD Foundation Website. +
Announced + the 20th Anniversary of the FreeBSD Foundation. +
Participated as an Admin for Google Summer of Code 2020 +
Continued to promote the FreeBSD Office Hours series including holding our + own Foundation led office hours. Videos from the one hour sessions can be + found on the Project's + YouTube Channel. You can watch + ours here. +
Interviewed + members of the outgoing FreeBSD Core Team to get their thoughts on their + term. +
Began working with the FreeBSD Vendor Summit planning committee on the + November 2020 Vendor Summit. +
Promoted the Foundation's 20th Anniversary and our work to support the + FreeBSD Project in the It's FOSS Article. + FreeBSD Foundation Celebrates 20 Years of Promoting and Supporting FreeBSD Project. +
Authored a Beginners Guide to FreeBSD for Fosslife. +
Committed to sponsoring All Things Open as a media Sponsor. +
Committed to sponsoring the OpenZFS Developers Summit at the Bronze level. +
Became an International RISC-V Member. +
Committed to giving a FreeBSD talk at the nerdear.la conference on + October 20th. +
+

+Keep up to date with our latest work in our +

monthly newsletters. +

Netflix provided an update on how and why they use FreeBSD in our latest +Contributor Case Study. +

We help educate the world about FreeBSD by publishing the professionally +produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is +now a free publication. Find out more and access the latest issues at +https://www.FreeBSDfoundation.org/journal/. +

You can find out more about events we attended and upcoming events at +https://www.FreeBSDfoundation.org/news-and-events/. +

Legal/FreeBSD IP

+ +

The Foundation owns the FreeBSD trademarks, and it is our responsibility to +protect them. We also provide legal support for the core team to investigate +questions that arise. We updated our +Trademark Usage Terms and Conditions +on July 1, 2020. +

Go to the FreeBSD Foundation's web site to +find out how we support FreeBSD and how we can help you! +

### Other +

We welcomed Andrew Wafaa and Kevin Bowling to our board of directors, to help +govern the Foundation and guide us with our strategic direction. We have +more information about our new board members +on our website. +

+ +FreeBSD Release Engineering Team + + + +FreeBSD Release Engineering Team +re@FreeBSD.org + + + + +FreeBSD 12.2-RELEASE schedule +FreeBSD 12.2 test builds +FreeBSD development snapshots + + +

The FreeBSD Release Engineering Team is responsible for setting +and publishing release schedules for official project releases +of FreeBSD, announcing code freezes and maintaining the respective +branches, among other things. +

During the third quarter of 2020, the Release Engineering Team started +work on the 12.2-RELEASE cycle, the third release from the stable/12 +branch. +

As of this writing, two BETA builds have been released, with the +expectation there will be a third BETA build currently remaining on the +schedule. +

The 12.2-RELEASE cycle will continue throughout October, with two RC +builds currently planned, and RC3 scheduled on an as-needed basis. The +12.2-RELEASE is so far scheduled for final release on October 27. +

In addition to the 12.2-RELEASE, Glen Barber of the Release Engineering +Team finished work to the release build tools and scripts to prepare for +the conversion from Subversion to Git for the 13.0-RELEASE cycle. There +are no plans to merge these changes to stable branches at this time; as +discussed within the Git working group, we feel such a change on a stable +branch would be too intrusive to our user base as well as downstream +FreeBSD consumers. Development snapshot builds for 13.0-CURRENT have +recently been built from the Git tree within the project, and further +snapshot builds for 12.x and 11.x will continue to be built from Subversion. +

Additionally throughout the quarter, several development snapshots builds +were released for the head, stable/12, and stable/11 branches. +

Finally, the Release Engineering Team would like to thank Marius Strobl +for his time serving on the team; he had recently stepped down from the +Deputy RE Lead role due to constraints on his time. The Team welcomes +Colin Percival, who has accepted fulfilling this role. +

Much of this work was sponsored by Rubicon Communications, LLC (netgate.com) +and the FreeBSD Foundation. +

+ +Cluster Administration Team + + + +Cluster Administration Team +clusteradm@FreeBSD.org + + + + +Cluster Administration Team members + + +

The FreeBSD Cluster Administration Team consists of the people responsible for +administering the machines that the Project relies on for its distributed work + and communications to be synchronised. In this quarter, the team has worked +on the following: +

Work with the FreeBSD Foundation on hardware update for web services, mirror and package building servers. +
Disable directory indexing on the package mirrors to resolve performance issues of the machine. +
- This was later relaxed to allow indexing of the parent directories but still disallow the large package directories. +
+
Ongoing systems administration work: +
- Accounts management for committers. +
- Backups of critical infrastructure. +
- Keeping up with security updates in 3rd party software. +
  +
+

+Work in progress: + +

Setup Malaysia (KUL) mirror. +
Setup Brazil (BRA) mirror. +
Review the service jails and service administrators operation. +
Infrastructure of building aarch64 and powerpc64 packages. +
- NVMe issues on PowerPC64 POWER9 blocking dual socket machine from being used as pkg builder. +
- Drive upgrade test for pkg builders (SSDs) courtesy of the FreeBSD Foundation. +
- Boot issues with Aarch64 reference machines. +
+
New NYI.net sponsored colocation space in Chicago-land area. +
Work with git working group for the git repository. +
Searching for more providers that can fit the requirements for a generic mirrored layout or a tiny mirror. +

+ + +Continuous Integration + + +FreeBSD Jenkins Instance +FreeBSD Hardware Testing Lab +FreeBSD CI artifact archive +FreeBSD CI weekly report +FreeBSD Jenkins wiki +Hosted CI wiki +3rd Party Software CI +Tickets related to freebsd-testing@ +FreeBSD CI Repository + + + + +Jenkins Admin +jenkins-admin@FreeBSD.org + + +Li-Wen Hsu +lwhsu@FreeBSD.org + + +

Contact: freebsd-testing Mailing List
+Contact: IRC #freebsd-ci channel on EFNet
+

The FreeBSD CI team maintains the continuous integration system +of the FreeBSD project. The CI system firstly checks the committed changes +can be successfully built, then performs various tests and analysis over the +newly built results. +The artifacts from those builds are archived in the artifact server for +further testing and debugging needs. The CI team members examine the +failing builds and unstable tests and work with the experts in that area to +fix the codes or adjust test infrastructure. The details of these efforts +are available in the weekly CI reports. +

During the third quarter of 2020, we continued working with the contributors and +developers in the project to fulfill their testing needs and also keep +collaborating with external projects and companies to improve their products +and FreeBSD. +

Important changes: +

All !x86 -test builds now trigger a new build on 22:00 UTC daily; this was + not running very often because running all the tests in qemu takes lots + of time. The work on improving the test execution speed and parallelism is + in progress. The following is a list of the jobs affected: +
+
The build and test results will be sent to the + dev-ci mailing list + soon. Feedback and help with analysis is very appreciated! +
+
- A builder dedicated to run jobs using provisioned VMs is setup, this + improves the stableness and reduces the execution time. +
  +
- The result of FreeBSD-head-amd64-test_zfs + is changed after OpenZFS importing; we encourage everyone to check and fix the + failing and skipped test cases. +
  +
+

+New jobs added: +

CI build for FreeBSD HEAD on PowerPC64LE. +
+

+Work in progress: +

Collecting and sorting CI tasks and ideas + here. +
Testing and merging pull requests in the + the FreeBSD-ci repo. +
Designing and implementing pre-commit CI building and testing, +
Reduce the procedures of CI/test environment setting up for contributors and + developers. +
Setting up the CI stage environment and putting the experimental jobs on it. +
Setting up public network access for the VM guest running tests. +
Implementing automatic tests on bare metal hardware. +
Adding drm ports building tests against -CURRENT. +
Planning to run ztest and network stack tests. +
Adding more external toolchain related jobs. +
Improving the hardware lab to be more mature and adding more hardware. +
Helping more 3rd software get CI on FreeBSD through a hosted CI solution. +
Working with hosted CI providers to have better FreeBSD support. +
+

+Please see freebsd-testing@ related tickets for more WIP information, and don't hesitate to join the effort! + +

Sponsor: The FreeBSD Foundation +

+ +Ports Collection + + +About FreeBSD Ports +Contributing to Ports +FreeBSD Ports Monitoring +Ports Management Team + + + + +René Ladan +portmgr-secretary@FreeBSD.org + + +FreeBSD Ports Management Team +portmgr@FreeBSD.org + + + +

The Ports Management Team is responsible for overseeing the +overall direction of the Ports Tree, building packages, and +personnel matters. Below is what happened in the last quarter. +

We passed the landmark of 40,000 ports in the Ports Collection +and are now around 40,400 ports. The last quarter saw 9335 +commits to the HEAD branch and 481 commits to the 2020Q3 branch +by respectively 167 and 63 committers. There are currently 2525 +open problem reports of which 595 are unassigned. Compared to +last quarter, this means a slight decrease in activity and also +a slight increase in open PRs. +

During the last quarter we welcomed Rainer Hurling (rhurlin@) and +said goodbye to Kevin Lo (kevlo@) and Grzegorz Blach (gblach@). +

The last three months saw new default versions for Perl (5.32), +PostgreSQL (12) and PHP (7.4). Various packages also got updated: +Firefox to 81.0.1, Chromium to 84.0.4147.135, Gnome to 3.36, +Xorg to 1.20.9, Qt5 to 5.15.0, Emacs to 27.1, KDE Frameworks to +5.74.0 and pkg itself to 1.15.8. +

Never tired, antoine@ ran 30 exp-runs to test port version updates, +on such diverse matters as: +

Updating byacc in base to 20200330. +
Check balancing of sed "y" command. +
Use of brackets. +
Removing the now redundant "port" argument from USES=readline. +

+ + +FreeBSD Office team - 3rd quarter 2020 report + + +The FreeBSD Office project + + + + +FreeBSD Office team ML +office@FreeBSD.org + + +Dima Panov +fluffy@FreeBSD.org + + +Li-Wen Hsu +lwhsu@FreeBSD.org + + + + +

The FreeBSD Office team works on a number of office-related software suites +and tools such as OpenOffice and LibreOffice.
+

Work during this quarter focused on providing the latest stable release of +LibreOffice suite and companion apps to all FreeBSD users. +

Alongside with updating old stable branch to latest 6.4.x releases, + current ports-tree now have a full-featured cutting-edge 7.0.1 bundle.
+
Conservative users can keep 6.4.x stable version by switching to use + all-in-one editors/libreoffice6 port and even with i18n language pack (off by default). + It will be kept updated at least till 7.1.0 version is released.
+
+

+We are looking for people to help the project. +

All unstable work with LibreOffice snapshots is staged in our WIP repository.
+The open bugs list +contains all filed issues which need some attention. +Patches, comments and objections are always welcome in the mailing list and bugzilla. +

+ + +FreeBSD Graphics Team status report + + +Project GitHub page + + + + +FreeBSD Graphics Team +x11@freebsd.org + + +Niclas Zeising +zeising@freebsd.org + + + +

The FreeBSD X11/Graphics team maintains the lower levels of the FreeBSD graphics +stack. +This includes graphics drivers, graphics libraries such as the +MESA OpenGL implementation, the X.org xserver with related libraries and +applications, and Wayland with related libraries and applications. +

There have been several updates to the FreeBSD graphics stack and related +libraries since the last report. +

Most notably, MESA related ports were changed to use the meson build system, +instead of the autotools based one. +This was needed since mesa upstream has deprecated and removed the autotools +build system, and this paved the way for further mesa updates. +While there was a need for a few minor corrections after the initial update, +this update has been successful and made it possible to further update and +improve the FreeBSD mesa port. +

There have also been several security fixes for xorg-server and libX11, so +these ports have been updated to fix these issues. +

During the period, FreeBSD 12 was changed to improve the compatibility with +input devices using udev/evdev and libinput. +This change removes the need for local configuration and makes most mice, +touchpads and keyboards work out of the box. +This change will be in the upcoming FreeBSD 12.2 release. +

There have also been several updates to various libraries, both in the graphics +and input stacks, and several userland drivers have been updated. +Libraries such as libdrm and libevdev have been updated to include new +FreeBSD support, developed by team members and added upstream. +

There has also been ongoing work to keep the various drm-kmod ports and packages +up to date, mostly in response to changes in various FreeBSD versions. +

We have also continued our regularly scheduled bi-weekly meetings. +

People who are interested in helping out can find us on the x11@FreeBSD.org +mailing list, or on our gitter chat. +We are also available in #freebsd-xorg on EFNet. +

We also have a team area on GitHub where our work repositories can be found. +

+ +FreeBSD on Microsoft HyperV and Azure + + +Microsoft Azure article on FreeBSD wiki +Microsoft HyperV article on FreeBSD wiki + + + + +FreeBSD Integration Services Team +bsdic@microsoft.com + + +Wei Hu +whu@FreeBSD.org + + +Li-Wen Hsu +lwhsu@FreeBSD.org + + + +

Li-Wen is working on the FreeBSD release code related to Azure for +the -CURRENT, 12-STABLE and 11-STABLE branches. +The work-in-progress is available here. +The 11.4-RELEASE image on Azure Marketplace is published. +We are testing the releng/12.2 branch and 12.2-RELEASE image will be +published to Azure Marketplace soon after released. +

This project is sponsored by The FreeBSD Foundation, with resources provided by Microsoft. +

+ +Building FreeBSD on non-FreeBSD hosts + + +Wiki + + + + +Alex Richardson +arichardson@freebsd.org + + + +

Until recently FreeBSD could only be built on a FreeBSD host. +However, many popular free CI tools only allow building on Linux or macOS and +therefore can not be used for building the FreeBSD base system. Furthermore, it +is sometimes useful to cross-build FreeBSD for a remote machine or an emulator +even if the build machine is not running FreeBSD. +The goal of this project is to allow building the base system on Linux and macOS +hosts. +

I started this project in 2017 to allow building CheriBSD on the Linux servers +and desktops that many of us working on the CHERI project use. +The first few patches were upstreamed in 2018 (see the 2018q3 report) and +I merged the full set of patches to CheriBSD shortly after. Over the past two +years I have slowly been upstreaming the remaining patches and finally committed +the last required change in time for this report. +

As of September 2020 it should be possible to use the buildworld and +buildkernel make targets to build a fully-functional FreeBSD installation +on macOS and Linux hosts. We use this in our continuous integration system to +build and test CheriBSD disk images for multiple architectures. +I have also committed a GitHub Actions configuration upstream +that takes approximately 10 minutes to build an amd64 kernel. +This will ensure that changes that break crossbuilding from Linux/macOS +can be detected easily. +

Upstreaming the crossbuilding changes has resulted in various build system +cleanups. For example, we now no longer need to use lorder.sh +when building libraries which speeds up the linking step a bit. +The portability and bootstrapping changes should also make it easier +to upgrade from older versions since we no longer rely on host headers in +/usr/include matching those of the target system (e.g. when bootstrapping +localedef, etc.). +

While this support for building on Linux and macOS should still be considered +experimental, it should work in many cases. If you would like to give it a try, +the following command line should successfully build an amd64 world on Linux +and macOS systems that have packages for LLVM 10 (or newer) installed: +MAKEOBJDIRPREFIX=/somewhere ./tools/build/make.py TARGET=amd64 TARGET_ARCH=amd64 buildworld +Builds must be performed using the ./tools/build/make.py wrapper script since +most Linux and macOS systems do not ship an appropriate version of bmake. +Please let me know if you encounter any issues. +

Sponsor: DARPA +

+ +Git Migration Working Group + + +Git conversion tooling repo +FreeBSD-git mailing list +Beta doc git repo +Beta ports git repo +Beta src git repo + + + + +Ed Maste +emaste@FreeBSD.org + + +Warner Losh +imp@FreeBSD.org + + +Ulrich Spörlein +uqs@FreeBSD.org + + + +

Work continues on FreeBSD's migration from Subversion to Git. Ulrich has +addressed all known issues with svn2git and has been able to work around the +inconsistent metadata and forced commit issues in the Subversion history. +

We still have additional documentation to write, and need to finish installing +commit hooks (e.g. restricting branch creation, or ensuring appropriate data +exists on cherry-pick commits). +

We expect to open the beta repository to test commits before the end of +October. This is to allow testing of the commit hooks, and to allow developers +to test access and become familiar with git operation. Commits in this +repository will be deleted and the repository will be recreated at least once +prior to the final migration. +

Those with an interest in the migration to Git are encouraged to subscribe +to the +FreeBSD-git mailing list +and test out the beta src, ports, and/or doc repositories. +

You are also welcome check out the wiki, issues, README and other documentation +at the Git conversion tooling repo. +

We currently expect to transition the src and doc repositories in mid-November. +Additional investigation and experimentation with the ports repository is still +underway. +

Sponsor: The FreeBSD Foundation (in part) +

+ +Linux compatibility layer update + + + +Edward Tomasz Napierala +trasz@FreeBSD.org + + +Mark Johnston +markj@FreeBSD.org + + + +

Earlier Linuxulator work focused on code cleanups and improving +diagnostic tools. +Work has now shifted from cleanups to fixing actual applications. +Current status is being tracked at Linux app status Wiki page. +Initial focus was on applications that don't involve X11, mostly +because they tend to be easier to test and debug, and the bug fixes +are not application-specific. +

Foundation-sponsored work during this quarter included implementing +a devfs(5) workaround to fix gettynam(3) inside jail/chroot, and +workaround for the missing splice(2) syscall, which caused problems +for grep and autotools. The Linux version reported to userspace was bumped +to 3.10.0, which matches the kernel shipped with RHEL 7 and is neccessary +for IBM's DB2 database installation to succeed. The BLKPBSZGET ioctl neccessary for +Oracle database is supported now. There is now support for kcov(4), +neccessary for syzcaller; as well as a number of fixes for issues +reported by syzcaller, such as futex lock leaks. +There were also more cleanups, including moving +some Linuxulator-specific functionality related to error handling off +from the syscall's fast code paths. The sysutils/debootstrap port, +which provides an easy way to create Debian or Ubuntu jail, was updated +to version 1.0.123. Finally there were some improvements +to the documentation. +

Most of those changes have been merged to FreeBSD 12-STABLE, in order +to ship with 12.2-RELEASE. +

There is increased involvement from other developers; this includes termios +performance fixes, improved memfd support, implementing CLOCK_MONOTONIC_RAW +required for Steam, madvise improvements, new compat.linux.use_emul_path +sysctl. There is also ongoing work +on tracking down the causes of failures related to Steam and WebKit, with +fixes being first implemented in linuxulator-steam-utils. +

Sponsor: The FreeBSD Foundation +

+ +LLDB Debugger Improvements + + +Moritz Systems Project Description +Git Repository + + + + +Kamil Rytarowski +kamil@moritz.systems + + +Michał Górny +mgorny@moritz.systems + + + +

FreeBSD includes LLDB, the debugger in the LLVM family, in the base +system. At present it has some limitations in comparison with the GNU +GDB debugger, and does not yet provide a complete replacement. It +relies on an obsolete plugin model in LLDB that causes growing +technical debt. This project aims to bring LLDB closer to a fully +featured replacement for GDB, and therefore for FreeBSD to feature a +modern debugger for software developers. +

The legacy monolithic target supports the executed application being +debugged in the same process space as the debugger. The modern LLDB +plugin approach, used on other supported targets, executes the +target process under a separate lldb-server process. This improves +reliability and simplifies the process / thread model in LLDB itself. +In addition, remote and local debugging will both be performed using +the same approach. +

After the migration to the new process model is complete, the project +will include reviewing the results of LLDB's test suite and fixing +tests as time permits. The work is expected to be complete in 2020. +

The project schedule is divided into three milestones, each taking approximately +one month: +

1. Introduce new FreeBSD Remote Process Plugin for x86_64 with basic support and upstream to LLVM. + 2. Ensure and add the mandated features in the project (process launch, process attach (pid), process attach (name), userland core files, breakpoints, watchpoints, threads, remote debugging) for FreeBSD/amd64 and FreeBSD/i386. + 3. Iterate over the LLDB tests. Detect, and as time permits, fix bugs. Ensure bug reports for each non-fixed and known problem. Add missing man pages and update the FreeBSD Handbook. +

We are nearing the completion of the first milestone. The new plugin is getting into +shape, and it can already run simple single-threaded programs. The supported features +include single-stepping, breakpoints, memory and register I/O on amd64. +Both plugins are supported simultaneously. The new plugin is used if +FREEBSD_REMOTE_PLUGIN environment variable is set to any value, or if lldb-server is +spawned directly. Otherwise, the old plugin is used for compatibility. Once the new +plugin matures, we are planning to enable it unconditionally on the architectures that +it is ported to. +

Sponsor: The FreeBSD Foundation
+

+ +Lua usage in FreeBSD + + + +Ed Maste +emaste@FreeBSD.org + + +Kyle Evans +kevans@FreeBSD.org + + +Ryan Moeller +freqlabs@FreeBSD.org + + + +

During this quarter, flua (FreeBSD Lua) was taught +where to find base .lua modules in order to support require of .lua modules +to be provided by the base system. flua also gained support +for require of binary modules. +

A review for libjail bindings has also +been submitted, pending review. libjail is an essential component if one wants +to be able to write jail management utilities in flua. +

People interested in working with Lua in FreeBSD are welcome to get in +contact to discuss other project ideas. To name a couple of potential +projects, some interesting modules that have not been started but could +prove useful (listed in no particular order): +

libcrypt +
libexpat +
libnv +
libxo +
+

+There is also a small list of scripts that would do well with a port to flua: + +

certctl(8) +

+ + +NFS over TLS implementation + + + +Rick Macklem +rmacklem@freebsd.org + + + +

In an effort to improve NFS security, an internet draft +which I expect will become an RFC soon specifies the +use of TLS 1.3 to encrypt all data traffic on a Sun RPC +connection used for NFS. +

Although NFS has been able to use sec=krb5p to encrypt data +on the wire, this requires a Kerberos environment and, as +such, has not been widely adopted. It also required that +encryption/decryption be done in software, since only the +RPC message NFS arguments are encrypted. +Since Kernel TLS is capable of using hardware assist to +improve performance and does not require Kerberos, NFS +over TLS may be more widely adopted, once implementations +are available. +

The coding for this project has now been completed. +All required changes to the NFS and kernel RPC code have +been committed to -CURRENT. +The daemons are now believed to be complete, but will +remain in base/projects/nfs-over-tls until -CURRENT +has an OpenSSL library with the kernel TLS support +incorporated in it. +If this does not happen for FreeBSD-13, hopefully the +patched OpenSSL and the daemons can become ports. +

To support clients such as laptops, the daemons that perform the TLS +handshake may optionally handle client X.509 certificates from a +site local CA. There are now exports(5) options to require client(s) to +provide a valid X.509 certificate. +

While setting up system(s) for testing is still a little awkward, +the documentation is now available for those who want to help with testing. +

The main limitation in the current implementation is that it uses TLS1.2 +and not TLS1.3. This should change once the KERN_TLS rx patch includes +TLS1.3 support. +

Third party testing would be appreciated. +

+ + +syzkaller on FreeBSD + + + +Mark Johnston +markj@FreeBSD.org + + + +

See the syzkaller entry in the 2019q1 quarterly report for an +introduction to syzkaller. +

syzkaller, especially the public syzbot instance, continues to find bugs +in the FreeBSD kernel. A number of these bugs have been fixed in +subsystems such as the VFS name cache, the TCP and SCTP stacks, pf(4), +the unix domain socket implementation, and the Linuxulator. +

The FreeBSD Foundation sponsored some work to enable cross-OS fuzzing. +This makes it possible to fuzz the Linuxulator using syzkaller's Linux +target. This effort quickly found several bugs; once the support is +committed upstream we will hopefully be able to leverage syzbot to gain +continuous testing of the Linux system call interface in addition to the +native and 32-bit compatibility interfaces. +

Some work was also done to enable running syzkaller in a FreeBSD jail, +with the eventual aim of making it easy to distribute binary images +containing everything required to immediately start running syzkaller on +a new host. Currently a number of setup steps are required, making +deployment somewhat painful. +

Sponsor: The FreeBSD Foundation +

+ +DRM Drivers Update + + +drm-kmod + + + + +Emmanuel Vadot +manu@FreeBSD.Org + + + +

The drm drivers for FreeBSD 13-CURRENT have been updated to match Linux 5.4.62 +Then graphics/drm-current-kmod have been updated to follow this LTS release of Linux. +

For now graphics/drm-devel-kmod is also tracking this release but will be updated +to a later revision of Linux drm drivers in the near future. +

A lot of linuxkpi code was removed from the ports or replaced with a BSD +licenced implementation. +

Sponsor: The FreeBSD Foundation +

+ +DTS Update + + + +Emmanuel Vadot +manu@FreeBSD.org + + + +

DTS files (Device Tree Sources) were updated to be on par with Linux 5.8 for +HEAD and 5.6 for the 12-STABLE branch. +

+ +DesignWare Ethernet adapter driver improvements + + +WIP branch + + + + +Oleksandr Tymoshenko +gonzo@FreeBSD.org + + + +

DesignWare Ethernet adapter IP is used in Rockchip and Allwinner SoCs. +The driver was updated with following fixes: +

Initialize clocks instead of relying on u-boot to do the right thing. +
Sense media type and adjust controller configuration accordingly. +
Add support for RMII PHY mode. +
+

+Yet uncommitted changes include performance optimisation by adding +

support for multi-segment mbuf transmission. The next step is to +try to get more performance boost by using interrupt coalescence. +

+ +Google Summer of Code’20 Project - eBPF XDP Hooks + + +Github diff link +Project wiki + + + + +Ankur Kothiwal +ankur@freebsd.org + + + +

The eBPF eXpress Data Path (XDP) allows eBPF programs to be run to filter +received packets as early as possible, avoiding unnecessary processing +overhead before the filter is run. The goal of this project is to extend an +existing FreeBSD network driver (a virtual NIC like a VirtIO if_vtnet) to +be able to call into an eBPF program when processing a newly received +packet. In short, with XDP the driver must PASS (accept and process normally), DROP, +TX or REDIRECT the packet as specified by the program. eBPF helper +functions and maps for aiding in packet filtering will also be +implemented. +

Implemented: +

Register a eBPF probe when an interface is registered with pfil. +
Activating eBPF probe. +
Create hooks and link them to the pfil head when the eBPF XDP probe is + activated and successfully list the XDP probes. +
Create a xdp_rx function which will pass the received packets to the + eBPF program where the packets can be further processed. This function will + return XDP actions: DROP and PASS. +
Register the xdp hook and link it to the pfil head. +
Write an eBPF program to process (currently drop and pass) ICMP traffic - + This is to test that the hook is working properly. +
Write a loader function to load the ICMP filter program to the kernel. +
+

+Future Work: +

Currently we can only attach the XDP hook to PASS and DROP the packets - + The work on detaching the hook is left. +
The XDP action to “TX” and “REDIRECT” the packets. +
+

+Final Deliverables: +

Implemented XDP hook to pass and drop packets. +
Created a loader program to attach the eBPF program to the kernel. +
A test program to DROP ICMP filter. +
+

+This code was done under the Google Summer of Code 2020 under the guidance +

of Ryan Stone (rstone@). The eBPF implementation for FreeBSD +is still a work in progress and FreeBSD doesn’t support eBPF yet. The +basic implementation for eBPF was a GSoC’18 project, +and is still under development. This project is based on that implementation so the XDP +implementation for FreeBSD can only be merged into the FreeBSD source code +once it supports eBPF. +

Currently this code is a work in progress and is merged to Ryan Stone’s +branch with support for the eBPF implementation. +

Sponsor: Google Summer of Code +

+ +ENA FreeBSD Driver Update + + +ENA README + + + + +Michal Krawczyk +mk@semihalf.com + + +Artur Rojek +ar@semihalf.com + + +Marcin Wojtas +mw@semihalf.com + + + +

ENA (Elastic Network Adapter) is the smart NIC available in the +virtualized environment of Amazon Web Services (AWS). The ENA +driver supports multiple transmit and receive queues and can handle +up to 100 Gb/s of network traffic, depending on the instance type +on which it is used. +

Completed since the last update: +

Fix ENA compilation in case it is integrated into the kernel binary. +
MFC of the ENA v2.2.0 driver to the FreeBSD 12.2. +
+

+Work in progress: +

Add feature that allows reading extra ENI (Elastic Network Interface) + metrics about exceeding BW/pps limits. +
Introduce full kernel RSS API support. +
Allow reconfiguration of the RSS indirection table and hash key. +
Evaluation and prototyping of the driver port to the iflib framework. +
+

+Sponsor: Amazon.com Inc + + +IPSec Extended Sequence Number (ESN) support + + + +Grzegorz Jaszczyk +jaz@semihalf.com + + +Patryk Duda +pdk@semihalf.com + + +Marcin Wojtas +mw@semihalf.com + + + +

Extended Sequence Number (ESN) is IPSec extension defined in RFC4303 Section 2.2.1. +It makes possible to implement high-speed IPSec implementations where standard, 32-bit sequence number is not sufficient. +A key feature of the ESN is that only low order 32 bits of sequence number are transmitted over the wire. +High-order 32 bits are maintained by sender and receiver. Additionally high-order bits are included in the computation of Integrity Check Value (ICV) field. +

Extended Sequence Number support contains following: +

Modification of existing anti-replay algorithm to fulfil ESN requirements. +
Trigger soft lifetime expiration at 80% of UINT32_MAX when ESN is disabled. +
Implement support for including ESN into ICV in cryptosoft engine in both + encrypt and authenticate mode (eg. AES-CBC and SHA256 HMAC) and combined + mode (eg. AES-GCM). +
Implement support for including ESN into ICV in AES-NI engine in both + encrypt and authenticate mode and combined mode. +
+

+Completed since the last update: + +

Adjust implementation of crypto part to the reworked Open Crypto Framework. +
Move the core ESN implementation from the crypto drivers to netipsec layer. +
Make use of the newly introduced crp_aad mechanism for combined modes. +
Introduce minor fixes and improvements. +
+

+TODO: + +

Complete review process in Phabricator and merge patches in the tree. +
+

+Sponsor: Stormshield + + + +NXP ARM64 SoC support + + + +Marcin Wojtas +mw@semihalf.com + + +Artur Rojek +ar@semihalf.com + + +Dawid Gorecki +dgr@semihalf.com + + + +

The Semihalf team initiated working on FreeBSD support for the +NXP LS1046A SoC +

LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with +integrated packet processing acceleration and high speed peripherals +including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide +range of networking, storage, security and industrial applications. +

Completed since the last update: +

Upstreaming of the QorIQ SDHCI driver (r365054). +
+

+With above the current Semihalf upstreaming activity is complete. + +

The major out-of-tree supported components: +

DPAA network controller support. +
QSPI controller support. +
+

+They work on 11.2-RELEASE, but still require significant +

effort to adopt to FreeBSD-CURRENT. +

Sponsor: Alstom Group +

+ +Addition of PowerPC64LE Architecture + + +Early notes +Announcement + + + + +Brandon Bergren +bdragon@freebsd.org + + + +

As of r366063, experimental support for little-endian PowerPC64 (PowerPC64LE) +is available in -CURRENT for POWER8 and POWER9 machines. +

In 2010, when FreeBSD was ported to PowerPC64, the average +user would have been using a G5 PowerMac, a purely big-endian +machine. +

While, at the time, a 32-bit PowerPC machine could run in little-endian, +as well as POWER6 and POWER7, in practice, the complexities involved +in managing it at the kernel level and lack of firmware support made it +infeasible to support. +

When IBM designed POWER8, one main focus was to improve little-endian support, +and bring it up to parity with big-endian. +

This improved support makes it practical to support a little-endian operating +environment on what is traditionally a primarily big-endian platform. +

In 2020, with POWER9 being affordable for many users thanks to the Raptor +Blackbird, semi-easy access to surplus POWER8 hardware, IBM having +a major future focus on POWER little-endian, and the decay of big-endian +support in modern video cards and graphical environments, there is demand for +a little-endian version of FreeBSD on POWER. +

With FreeBSD/PowerPC64's transition in 2019 to the ELFv2 ABI as part of the +2019q4 PowerPC on Clang effort, the last major barrier to a little-endian +port was eliminated. +

Since nobody else was working on it, and I had the skillset required to do +the port, I decided to experiment one weekend with a little-endian kernel +to see how difficult it would be to port. +

It turned out to be a lot more trivial than I was expecting. Three days later +I had console support in qemu, and after another week of debugging, I had it +fully up and running on hardware. +

FreeBSD PowerPC64LE is now an experimental MACHINE_ARCH in base, and is +continuing to evolve at a rapid pace. +

Big-endian PowerPC64 is still the preferred platform for the foreseeable +future, and will not be deprecated. +

Sponsor: Tag1 Consulting, Inc. +

+ +ure - USB 3.0 Gigabit Ethernet Driver update + + +svn commit: r365648 +FreeBSD-SA-20:27.ure +D25809 major update to if_ure + + + + +John-Mark Gurney +jmg@FreeBSD.org + + + +

The ure is a driver for handling the RealTek ethernet adapters, +including the RTL8153 USB 3.0 Gigabit ethernet adapters. It is used +in many ethernet dongles and docking stations. +

Previous to this update, the driver was limited in speed. In my +testing, I was only able to get ~91Mbps. This limit was due to one +packet per USB transfer. USB has a limit of 8000 transfers per +second (1500 bytes/pkt * 8000 pkts/sec * 8bits/byte == 96 Mbps). +This was acceptable for fast ethernet (RTL8152, 100Mbps), but with +the additional support for Gigabit ethernet, it became a bottleneck. +

The updates add sending and receiving multiple packets in a single +USB transfer, VLAN hardware tagging, and enable TCP and UDP +checksum offloading. This increased the speed on gigabit ethernet +to ~940 Mbps. +

In doing this work, a security vulnerability was discovered in the +driver. Due to improper setting of a device register, on some +devices, it caused packets to be fragmented when they shouldn't be +and the driver was unable to handle them correctly. This allowed an +attacker, who could generate large frames (say, ping packets, or +large TCP transfers), to inject arbitrary packets into the network +stack. This could allow the attacker to spoof traffic from other +machines, and bypass VLAN protections. See the SA for more +information. +

As part of this work, a script was created to run tests to +validate that basic functionality of the driver (w/o options) work +properly, and then iterate over each option to make sure that they +function properly. This will be released at some point in the +future. +

If you're interested in helping out, or testing it, let me know. +

+ +Stateless hardware offloads for VXLANs + + +r365867 +r365868 +r365869 +r365870 +r365871 +RFC6935 + + + + +Navdeep Parhar +np@FreeBSD.org + + + +

VXLAN (Virtual eXtensible LAN) is a tunneling protocol in which Layer 2 +traffic for a virtual LAN is encapsulated in UDP and transferred over +Layer 3 networks between VTEPs (VXLAN Tunnel End Points). Traffic on +the wire has two sets of networking headers: the headers for the +encapsulation and the headers of the traffic being encapsulated. VXLANs +are supported by if_vxlan(4) on FreeBSD. +

Modern NICs commonly support header checksum insertion and verification, +TSO (TCP Segmentation Offload) on transmit, and RSS for load +distribution on receive. But the default is to operate on the outermost +headers. Some NICs can operate on the inner encapsulated frames as +well. The commits listed above allow if_vxlan(4) to take advantage of +such NICs. +

r365867 and r365868 add new mbuf checksum flags and ifnet capabilities. +r365870 implements the kernel parts of the new capabilities and updates +if_vxlan(4) to make use of them. r365871 implements driver support for +the new capabilities in cxgbe(4). +

VXLAN and other tunneling protocols that use UDP explicitly allow zero +checksum in the outer UDP header, even with IPv6. r365869 adds support +for configuring one UDP/IPv6 port where zero checksums are allowed. +

This work was sponsored by Chelsio Communications and was implemented +and tested using T6 (Terminator 6) NICs supported by cxgbe(4). It is +available in 13.0-CURRENT (head) right now and will be available in +12-STABLE in the future. +

VXLANs can be created as usual and will automatically have checksum and +TSO capabilities if the underlying physical interface supports VXLAN +stateless offloads. Use ifconfig to list, disable, and enable checksum +capabilities on the VXLAN interface. Use https://bugs.freebsd.org/bugzilla/ +to report bugs. +

Future work: +

Direct call into a vxlan input routine from the driver's receive routine. +
LRO support in if_vxlan(4). +
GENEVE support. +
+

+Sponsor: Chelsio Communications + + +Wireless updates + + +The freebsd-wireless mailing list +athp github repository + + + + +Adrian Chadd +adrian@FreeBSD.org + + +Bjoern A. Zeeb +bz@FreeBSD.org + + + +

net80211 and infrastructure, driver updates, and upcoming 12.2 Release

+ +

The following works happened in FreeBSD HEAD (some already in Q2) and were +merged for 12.2-BETA2 and include net80211 and driver updates for better 11n +and upcoming 11ac support. +

In more detail, this includes an ath(4) update, some run(4) 11n support, 11n for otus(4), +A-MPDU, A-MSDU, A-MPDU+A-MSDU and Fast frames options, scanning fixes, +enhanced PRIV checks for jails, restored parent device name printing, +improvements for upcoming VHT support, lots of under-the-hood infrastructure +improvements, new device IDs, and debug tools updates. +

If you have a chance please test before the release. +

Atheros 11ac driver athp

+ +

In the last three months the athp(4) port of the ath10k driver has progressed +well. Adrian reports the following important changes: +

Per-node transmit buffering was implemented, required for correct hostap + and QCA6174 behaviour. +
Issues with ignoring sending some management frames got fixed; null-data + frames were being filtered out and this caused undesirable hostap behaviour. +
Transmit path refactoring reduced code duplication. +
A fix on firmware start / VAP running tracking no longer stops + the first VAP from coming active after VAP creation / ifconfig up. +
Correcting hostap mode PHY configuration now allows non-VHT stations to + associate and correctly exchange data with a VHT AP. +
Addition of a crypto key configuration cache in the driver ensures the + ieee80211_key details are available after the key is deleted; net80211 + would reuse or free the state before the driver task would finish the + firmware command. +
+

Newer Intel Wireless device support

+ +

Initial work was done to integrate net80211 support in the LinuxKPI compat +layer to get the wireless parts going. +In addition, upstreaming code changes and working through problems and review +started on two sides. One was trying to get mostly compile time changes +upstreamed to the iwlwifi driver. The other is sorting out conflicting +LinuxKPI changes to not break the DRM graphics drivers. +Bjoern hopes that with some of that sorted out, he can soon go back to focus +on the wireless parts and produce a new snapshot. +

rtw88 and brcmfmac

+ +

As the Intel driver port and LinuxKPI advance, both the rtw88, and to a lower +degree the brcmfmac, ports benefit from that. +Bjoern lately also got a brcmfmac PCIe card and started to port support for +that. +This for the moment remains a free-time project. +

Work by Bjoern was sponsored by: Rubicon Communications, LLC (d/b/a "Netgate") and The FreeBSD Foundation +

+ +ZSTD Compression in ZFS + + + +Allan Jude +allanjude@freebsd.org + + + +

Zstandard (ZSTD) is a modern high-performance compression +algorithm designed to provide the compression ratios of gzip +while offering much better performance. ZSTD has been adopted +in FreeBSD for a number of other uses, including compressing +kernel crash dumps, as a replacement for gzip or bzip for +compressing log files, and for future versions of pkg(8). +

This effort to complete the integration of ZSTD into ZFS is +funded by the FreeBSD Foundation. +

During the third quarter the integrating of ZSTD into OpenZFS +was completed in the upstream OpenZFS repository, and the new +OpenZFS 2.0 codebase was imported into 13-CURRENT. +Completed milestones in this project: +

Importing ZSTD 1.4.5 into OpenZFS, using the recent upstream zstd features that make it easier to embed zstd in other projects. +
Changing the way compression levels are tracked and inherited. +
Save and restore the compression level via an embedded block header. +
Also store the version of zstd used in the embedded block header, for future-proofing. The checksum of a block may not match if zstd is upgraded, since it may compress the block more. +
Add tests to ensure zstd compression and metadata survive ZFS replication. +
Resolve possible negative interactions with L2ARC and ZFS Native Encryption. +
Fix bug with L2ARC if the Compressed ARC feature is disabled. +
Improve the ZFS feature activation code, so that zstd cannot create pools that will panic older versions of ZFS. +
+

+With these changes, upgraded pools can compress data with zstd +

or zstd-fast, across a wide range of different compression levels. +This will allow the storage administrator to select the +performance-vs-compression tradeoff that best suits their needs. +

Tasks remaining to be completed: +

Add a section to the FreeBSD Handbook ZFS chapter about zstd +
Create more documentation around selecting a suitable compression level +
Finish support for ZSTD in the FreeBSD boot loader (Warner Losh imp@freebsd.org) +
+

+Sponsor: The FreeBSD Foundation + + +CheriBSD 2020 Q3 + + +http://www.cheri-cpu.org +https://github.com/CTSRD-CHERI/cheribsd +https://fett.darpa.mil +https://www.morello-project.org +https://developer.arm.com/architectures/cpu-architecture/a-profile/morello + + + + +Alex Richardson +arichardson@FreeBSD.org + + +Andrew Turner +andrew@FreeBSD.org + + +Brooks Davis +brooks@FreeBSD.org + + +Edward Tomasz Napierala +trasz@FreeBSD.org + + +George Neville-Neil +gnn@FreeBSD.org + + +Jessica Clarke +jrtc27@FreeBSD.org + + +John Baldwin +jhb@FreeBSD.org + + +Robert Watson +rwatson@FreeBSD.org + + +Ruslan Bukin +br@FreeBSD.org + + + +

CheriBSD Status

+ +

CheriBSD extends FreeBSD to implement memory protection and software +compartmentalization features supported by the CHERI instruction-set +extensions. There are three architectural implementations of the +CHERI protection model: CHERI-MIPS, CHERI-RISC-V, and Arm's forthcoming +experimental Morello processor (due late 2021). CheriBSD is a research +operating system with a stable baseline implementation into which +various new research features have been, or are currently being, merged: +

Arm Morello - We are preparing to open source our adaptation of + CheriBSD to Arm's Morello architecture. The Morello branch is being + updated to the most recent CheriBSD baseline, and patches are in review + for upstreaming to our open-source repository. CheriBSD currently boots + and runs statically linked CheriABI binaries on the Morello simulator, + and dynamic linking support is in progress, with OS and toolchain bugs + being worked on. We aim to make a first CheriBSD/Morello snapshot + available alongside other open-source Morello software in mid-October + 2020, however, our target for a more mature and usable implementation is + December 2020. +
+
Kernel spatial memory safety (pure-capability kernel) - The current + CheriBSD kernel is a hybrid C program where only pointers to userspace + are CHERI capabilities. This ensures that the kernel follows the + intent of the application runtime and cannot be used to defeat + bounds on application pointers. We have developed and will soon + merge a pure-capability kernel where all pointers in the kernel are + appropriately bounded capabilities. This vastly reduces the opportunity + for buffer overflows. This spatial memory safety lays the + groundwork for future work such as device driver compartmentalization + and kernel temporal safety. +
+
Userspace heap temporal memory safety (Cornucopia) - CHERI + capabilities provide the necessary features to enable + robust and efficient revocation of freed pointers. With Cornucopia + we have implemented a light-weight revocation framework providing + protection from use-after-reallocation bugs with an average cost below + 2%. We aim to bring these overheads down further over the next year and + merge this functionality into the mainline CheriBSD. +
+
We have been working on updating the arm64 bhyve from Politehnica + University of Bucharest to have it committed to FreeBSD. We have been + upstreaming initial changes to help support this. +
+
Baseline FreeBSD improvements - We are upstreaming (to FreeBSD) various + bug fixes and tweaks for PCIe support, and support for the System MMU (SMMU) + that will be present on the N1SDP and Morello SoCs. We have upstreamed + support for cross-building FreeBSD from macOS and Linux (with some + limitations; see separate entry on crossbuilding). We have also fixed + implementation bugs in the RISC-V ABI. +
+

CHERI Documentation and Exercises

+ +

We have released [Capability Hardware Enhanced RISC Instructions: CHERI + Instruction-Set Architecture (Version 8)](https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-951.pdf). + Notable changes include promotion of CHERI-RISC-V to non-experimental + and discussion of Arm's Morello prototype. +
+
We have developed a set of [Adversarial CHERI Exercises and + Missions](https://ctsrd-cheri.github.io/cheri-exercises) to introduce security + researchers to CHERI protections. +

+ + +FreeBSD/RISC-V Project + + +Wiki + + + + +Mitchell Horne +mhorne@FreeBSD.org + + +

Contact: freebsd-riscv Mailing List
+Contact: IRC #freebsd-riscv on freenode
+

The FreeBSD/RISC-V project is providing support for running FreeBSD on the +RISC-V Instruction Set Architecture. +

This quarter saw several important bug fixes. A number of hangs in the system +were identified and addressed, and a bug in QEMU's implementation of the +Platform Level Interrupt Controller was fixed. This fix is included in the new +devel/qemu50 and devel/qemu-devel ports. +

The end result of these fixes is that the test suite can now be reliably run to +completion in QEMU. The entire run takes several hours, so CI has been +configured to run the job once a day. There is active effort into reducing the +time it takes to run the entire test suite. +

A new u-boot port was created: sysutils/u-boot-qemu-riscv64. This variant can +be used as a secondary bootloader alongside OpenSBI to load and launch FreeBSD's +loader(8) from an EFI System Partition. +

Next quarter will likely bring further fixes to address some of the failing test +cases. +

+ +DOCNG on FreeBSD + + +DOCNG Website Repo +DOCNG Documentation Repo +DOCNG Share Repo + + + + +Sergio Carlavilla +carlavilla@FreeBSD.org + + + +

The Doc New Generation project aims to convert the website and all +existing documentation to Hugo/AsciiDoctor. Right now almost +everything is converted as you can see in the repositories. +

The objective of using Hugo and AsciiDoctor is to reduce the +learning curve and let people to start quickly with our documentation +system. Other benefits of using Hugo is that we can use other +technologies aside from AsciiDoctor, like MarkDown, RST, Pandoc, etc. +

The remaining tasks include: +

Finish the conversion of some books to AsciiDoctor. +
Get some tweaks in the CSS to be responsive. +
Add AsciiDoctor extensions to create an index of tables and figures. +
Make a general review. +
+

+The dates for making the migration have yet to be discussed. + +

Patches, comments and objections are always welcome. +

+ +Update to grub-bhyve + + +grub-bhyve Git Repository + + + + +Chuck Tuffli +chuck@freebsd.org + + + +

bhyve is the hypervisor included in FreeBSD and other operating systems +used to run virtual machines. When not using a boot ROM (i.e. UEFI), the +user must load the guest operating system for bhyve. For non-FreeBSD +guests, the loader is a version of GNU GRUB (a.k.a the GNU GRand Unified +Bootloader) modified to interface with bhyve. This work is an effort to +both update the base GRUB code to the latest version as well as improve +the usability on FreeBSD. +

The current grub-bhyve +is based on an older version of GRUB (circa 2015) and thus is missing +more recent additions such as XFS file system and +syslinux support. With the update, +installing CentOS, for example, now does not require the extra step of +changing the default file system to something other than XFS. +

Internally, the code has been restructured to be its own "platform" +which should make it easier to keep in sync with upstream development. +The major improvement is the ability to automatically find and load the +GRUB configuration file from the guest disk image. With this change, it +is not necessary to create a device map file or specify which Linux +kernel or initrd image to use. More importantly, if the guest image +updates its GRUB configuration, for example after updating the kernel, +no changes are needed when invoking grub-bhyve. Note, this feature +requires a new "disk" option: +

# grub-bhyve --disk=/zroot/vms/u18-mini/disk0.img --vm=u18-mini +

The automatic configuration file detection works with both GRUB +configuration files (e.g. CentOS, Ubuntu) as well as syslinux +configuration (e.g. Alpine). For the adventurous, there is experimental +support for Fedora's BootLoaderSpec (a.k.a. blscfg) on the blscfg +branch of the grub-bhyve Git repository. +

The code has been tested on a few Linux variants, but it would benefit +from wider testing (and bug reports!). The new version does not have a +Port but is easily built on FreeBSD. After cloning / downloading the +source, run: +

$ PYTHON=python3.7 ./bootstrap + $ MAKE=gmake ./configure --with-platform=bhyve + $ gmake +

The resulting binary, grub-bhyve, will be in the grub-core/ +directory. If you have success or troubles with it, please let me know. +

+ +KDE on FreeBSD + + +KDE FreeBSD +KDE Community FreeBSD + + + + +Adriaan de Groot +kde@FreeBSD.org + + + +

The KDE on FreeBSD project aims to package all of the software +produced by the KDE Community for the FreeBSD ports tree. +The software includes a full desktop environment called KDE Plasma, +an IDE with the name KDevelop, +a PIM suite known as Kontact +and hundreds of other applications that can be used on +any FreeBSD machine. +

With the continuation of the ever-so-peculiar era of +almost-only-online, the KDE community has shifted gears +and also gone for online events. The yearly conference, +Akademy, +was conducted online over video calls. +Meanwhile, software continues to be released, +so this quarter the kde@ team: +

Put the beta of the next version of KDE Plasma, scheduled for + official release in October 2020, into the + Area51 development tree. + Area51 is a fork of the FreeBSD ports tree where new development for + KDE ports happens. +
The monthly regular updates to the KDE Plasma desktop landed on-time + and safely. +
With three months in a quarter, there were also three releases of + KDE Frameworks 5, including a new framework for handling DAV jobs. +
The June applications update and its .1 release landed a bit late, + but brings with it the usual raft of updates to KDE applications and libraries, +
A new Digikam release, which arrived in + the ports tree on the day of its release. +
A new KDevelop release arrived a day + after its release. This update contains a number of crash fixes + for refactoring support. +
Qt was updated to Qt 5.15, the last in the Qt5 series and an LTS + version. Bugfix releases are expected, but the next major Qt will + be Qt 6. +
+

+On the infrastructure front, August saw some minor updates to CMake and ninja. +

As usual, kde@ continues to support the work of xorg@ and gnome@ in maintaining +the Free Desktop stack on FreeBSD, including XOrg, poppler, and xdg-utils. +A new MAINTAINER group, desktop@, has been created to provide +shared ownership of that shared stack. +

With Python2 deprecation looming, the build system for QtWebEngine -- +itself a fork of Chromium -- is becoming a pressing issue in Q4 +and will no doubt chew up a lot of time in the coming months. +

+ +Potluck - Flavour & Image Repository for pot + + +Potluck Repository & Project +Potluck on github +pot project + + + + + + + + + +

pot is a jail management tool that also supports orchestration through nomad. +

Potluck aims to be to FreeBSD and pot what Dockerhub is to Linux and Docker: A repository of pot flavours and complete images for usage with pot. +

In the last quarter, an initial set of Nomad, Consul and Traefik images has been created that are sufficient to run a simple virtual datacenter out of the box.
+A three-part article series explaining how to set this up is also available now. +

Furthermore, ready-made images suitable for scheduling via Nomad and Consul in such an environment have been created, e.g. a BackupPC or a Postfix Backup MX service. +

Future plans include additional images and exposing more configuration options in the existing images to allow a more flexible usage. +

Beside general feedback and tests, additional flavours and patches are very welcome! +

Sponsors: Honeyguide GmbH & Honeyguide Group (Pty) Ltd +## Puppet +

+ +Puppet +Puppet's FreeBSD slack channel +Bolt +Choria + + + + +Puppet Team +puppet@FreeBSD.org + + + +

Since out last status report a few years ago, the puppet@ team regularly +updated the various Puppet ports to follow upstream releases of Puppet +4, Puppet 5 and Puppet 6. Puppet 4 was removed when it reached EOL. +

More recently, an effort was made to enhance Facter 4 so that it can be +used as a drop-in replacement of Facter 3 on FreeBSD. Facter 4 is a +Ruby rewrite of Facter 3, the C++ rewrite of Facter 2 which was +initially in Ruby. As a consequence we have two ports for Facter: +sysutils/facter is the C++ implementation (Facter 3) and +sysutils/rubygems-facter is the Ruby implementation (updated from Facter +2 to Facter 4 a few weeks ago). The Puppet 5 and Puppet 6 ports already +allow to choose which version of Facter to use. Facter 4 will be the +default version of Facter with Puppet 7 which is expected to be released +soon. +

We are getting ready to add a port for Puppet 7 as sysutils/puppet7 +when it is available, along with PuppetServer 7 (sysutils/puppetserver7), +and PuppetDB 7 (databases/puppetdb7). +

Regarding orchestration, most Marionette Collective ports have been +deprecated for a long time, and the last component sysutils/mcollective +is expected to be deprecated soon: Marionette Collective was not shipped +anymore with Puppet 6 and Bolt has been made available as a lightweight +replacement. +

Bolt is already available in the ports tree as sysutils/rubygems-bolt), +but if you are using Marionette Collective, you are invited to look into +Choria which will reach the ports tree soon as sysutils/choria. Choria +is a direct evolution of Marionette Collective allowing a smooth +transition from MCollective. Once Choria is available in the ports +tree, Marionette Collective will be deprecated. +

+ + team + + &os; Team Reports + +

Entries from the various official and semi-official teams, + as found in the Administration + Page.

+ + + + proj + + Projects + +

Projects that span multiple categories, from the kernel and userspace + to the Ports Collection or external projects.

+ + + + kern + + Kernel + +

Updates to kernel subsystems/features, driver support, + filesystems, and more.

+ + + + arch + + Architectures + +

Updating platform-specific features and bringing in support + for new hardware platforms.

. + + + ports + + Ports + +

Changes affecting the Ports Collection, whether sweeping + changes that touch most of the tree, or individual ports + themselves.

+ + + + doc + + Documentation + +

Noteworthy changes in the documentation tree, in manpages, or in + external books/documents.

+ + + + third + + Third-Party Projects + +

Many projects build upon &os; or incorporate components of + &os; into their project. As these projects may be of interest + to the broader &os; community, we sometimes include brief + updates submitted by these projects in our quarterly report. + The &os; project makes no representation as to the accuracy or + veracity of any claims in these submissions.

+ + +