Index: head/en_US.ISO8859-1/htdocs/news/status/report-2020-01-2020-03.xml
===================================================================
--- head/en_US.ISO8859-1/htdocs/news/status/report-2020-01-2020-03.xml (revision 54052)
+++ head/en_US.ISO8859-1/htdocs/news/status/report-2020-01-2020-03.xml (revision 54053)
@@ -1,1910 +1,1910 @@
Welcome, to the quarterly reports, of the future! Well, at least the first
quarterly report from 2020. The new timeline, mentioned in the last few
reports, still holds, which brings us to this report, which covers the period
of January 2020 - March 2020.
As you will see from this report, we've had quite an active quarter with big
changes to both kernel, userland, documentation, ports, and third-party
projects in the form of everything from bug and security fixes over new features
to speed improvements and optimizations.
As this report also covers the start of the epidemic, it's also interesting to
note that a quick glance at the svn logs reveal that there has been no overall
drop in number of source commits, that docs commits have also stayed constant,
and that ports have seen an upwards trend.
We hope that all of you are and yours are as safe as can be managed, and that
we get through this together by working together.
-- Daniel Ebdrup Jensen, debdrup@freebsd.org
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to
supporting and promoting the FreeBSD Project and community worldwide. Funding
comes from individual and corporate donations and is used to fund and manage
software development projects, conferences and developer summits, and provide
travel grants to FreeBSD contributors. The Foundation purchases and supports
hardware to improve and maintain FreeBSD infrastructure and provides resources
to improve security, quality assurance, and release engineering efforts;
publishes marketing material to promote, educate, and advocate for the FreeBSD
Project; facilitates collaboration between commercial vendors and FreeBSD
developers; and finally, represents the FreeBSD Project in executing contracts,
license agreements, and other legal arrangements that require a recognized
legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
We moved! Our new address is:
The FreeBSD Foundation
3980 Broadway St. STE #103-107
Boulder, CO 80304
USA
In February, the board of directors had an all-day board meeting in Berkely, CA,
where FreeBSD began! We put together our strategic plans for the next 2 years,
which includes software developments projects we want to support and some
educational initiatives.
COVID-19 impacts the Foundation. We put policies in place for all of our staff
members to work from home. We also put a temporary ban on travel for staff
members. We are continuing our work supporting the community and Project, but
some of our work and responses are delayed because of changes in some of our
priorities and the impact of limited childcare for a few of our staff members.
We help facilitate collaboration between commercial users and FreeBSD
developers. We also meet with companies to discuss their needs and bring that
information back to the Project. In Q1, Deb Goodkin met with commercial users
at LinuxConfAu in Australia, FOSDEM in Belgium, and SCALE18x in the US. These
venues provide an excellent opportunity to meet with commercial and individual
users and contributors to FreeBSD. It's not only beneficial for the above, but
it also helps us understand some of the applications where FreeBSD is used.
In addition to meeting with commercial users at conferences, we continued
discussions over email or on calls over the quarter.
Last quarter we raised $57,000! Thank you to everyone who came through,
especially in this economic crisis we have found ourselves in. It heartens us
deeply that individuals and organizations have supported our efforts, when there
are so many people, animals, and businesses in need right now. We also want to
extend a big thank you to Tarsnap, VMWare, and Stormshield for leading the way
with Silver level donations. We hope other organizations will follow their lead
and give back to help us continue supporting FreeBSD.
We are 100% funded by donations, and those funds go towards software development
work to improve FreeBSD, FreeBSD advocacy around the world, keeping FreeBSD
secure, continuous integration improvements, sponsoring BSD-related and
computing conferences, legal support for the Project, and many other areas.
Please consider making a donation to help us continue and increase our support
for FreeBSD: https://www.FreeBSDfoundation.org/donate/.
We also have the Partnership Program, to provide more benefits for our larger
commercial donors. Find out more information at
https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and
share with your companies!
The Foundation supports software development projects to improve the FreeBSD
operating system through our full time technical staff, contractors, and project
grant recipients. They maintain and improve critical kernel subsystems, add new
features and functionality, and fix problems.
Over the last quarter there were 273 commits to the FreeBSD base system source
repository tagged with FreeBSD Foundation sponsorship, about 12% of base system
commits over the quarter. Many of these are part of sponsored or staff projects
that have their own entries in this FreeBSD Quarterly Report, but Foundation
staff and contractors (Ed Maste, Konstantin Belousov, Mark Johnston, Li-Wen Hsu)
also support the project with an ongoing series of bug fixes, build fixes, and
miscellaneous improvements that don't warrant a separate entry.
Ed committed miscellaneous improvements to various parts of FreeBSD's build
infrastructure, largely prompted by the work to retire the obsolete GCC 4.2.1.
This included removal of the LLVM_LIBUNWIND option (now always set), and
the removal of unused gperf, gcov, and the GPL devicetree compiler (dtc). Ed
committed sendfile support for the Linuxulator, submitted by previous intern
Bora Özarslan, and tested and committed a number of submitted bug fixes for
the Microchip USB-Ethernet controller if_muge driver. Ed also updated the
copy of OpenSSH in the base system to 7.9p1, with additional updates in
progress, and worked on a number of security advisories released during the
quarter.
Konstantin Belousov and Mark Johnston both performed a large number of code
reviews during the quarter under Foundation sponsorship. This work helps
developers in the FreeBSD community and those working at companies using
FreeBSD to integrate their work into FreeBSD.
In addition to work described elsewhere in this report Konstantin also
continued his usual series of bug fixes and improvements. This quarter this
included low-level x86 support, fixing sendfile bugs, file system and vfs
bug fixes, and dozens of other miscellaneous improvements. Additional work
included a variety of commits to support Hygon x86 CPUs and improvements to
the runtime linker (rtld)'s direct execution mode.
Mark Johnston continued his work on the
Syzkaller system-call fuzzer, and
committed fixes for many issues reported by Syzkaller. Mark triaged a large
number of submitted bug reports and in many cases committed attached patches
or developed fixes. Mark also addressed dozens of Coverity Scan reports.
Mark's other changes included arm64 Large System Extensions (LSE) atomic
operations, low-level arm64 and x86 work, virtual memory (VM) work, and bug
fixes or other improvements to syslog, the lagg(4) link aggregation driver,
and build reproducibility.
Li-Wen Hsu committed many changes to tests in the base system, such as turning
off known failing tests tracked by PRs, test-related pkgbase fixes, and other
improvements.
The Foundation provides a full-time staff member who is working on improving
our automated testing, continuous integration, and overall quality assurance
efforts.
During the first quarter of 2020, Foundation staff continued to improve the
Project's CI infrastructure, worked with contributors to fix the failing build
and test cases. The building of a CI staging environment is in progress on the
new machine purchased by the Foundation. We are also working with other teams
in the Project for their testing needs. For example, we added a new job for
running LTP (Linux Testing Project) on the Linuxulator, to validate improvements
in the Foundation's sponsored Linux emulation work. We are also working with
many external projects and companies to improve their support of FreeBSD.
See the FreeBSD CI section of this report for completed work items and detailed
information.
The Foundation provides hardware and support to improve the FreeBSD
infrastructure. Last quarter, we continued supporting FreeBSD hardware located
around the world. We purchased one server for a mirror in Malaysia, and signed
the MOU for the new NYI colocation facility in Illinois. NYI generously
provides this as an in-kind donation to the Project.
A large part of our efforts are dedicated to advocating for the Project. This
includes promoting work being done by others with FreeBSD; producing advocacy
literature to teach people about FreeBSD and help make the path to starting
using FreeBSD or contributing to the Project easier; and attending and getting
other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD
tables, and give FreeBSD presentations.
The FreeBSD Foundation sponsors many conferences, events, and summits around the
globe. These events can be BSD-related, open source, or technology events
geared towards underrepresented groups. We support the FreeBSD-focused events
to help provide a venue for sharing knowledge, to work together on projects, and
to facilitate collaboration between developers and commercial users. This all
helps provide a healthy ecosystem. We support the non-FreeBSD events to promote
and raise awareness of FreeBSD, to increase the use of FreeBSD in different
applications, and to recruit more contributors to the Project.
Check out some of the advocacy and education work we did last quarter:
Organized and presented at the first ever FreeBSD Mini-Conf LinuxConfAu 2020,
in Gold Coast, Australia in addition to sponsoring the conference itself.
The recap can be found here.
Presented BSD Dev Room at FOSDEM '20, in Brussels, Belgium and represented
FreeBSD at a stand along with other members of the community. [Find out more
here](https://www.freebsdfoundation.org/blog/fosdem-2020-conference-recap/).
Represented FreeBSD at Apricot 2020 in Melbourne, Australia and sponsored the
event.
Industry Partner Sponsor for USENIX FAST '20 in Santa Clara, CA
Sponsored FOSSASIA 2020, in Singapore
Committed to hold FreeBSD Day at SCALE 18x, in Pasadena, CA
Held a "Getting Started with FreeBSD Workshop" at SCALE 18x in addition to
giving a talk, representing FreeBSD at the Expo and holding a "Why FreeBSD is
Me" BoF. Check out the conference recap.
Learn more about our efforts in 2019 to advocate for FreeBSD.
In addition to the information found in the Development Projects update section
of this report, take a minute to check out the latest update blogs:
reports in our monthly newsletters.
We help educate the world about FreeBSD by publishing the professionally
produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is
now a free publication. Find out more and access the latest issues.
You can find out more about events we attended and upcoming events here.
As is the case for most of
us in this industry, SCALE was the last event we will be attending for a few
months. However, we're already working on how we can make more on-line
tutorials and how-to guides available to facilitate getting more folks to try
out FreeBSD. In the meantime, please check out the how-to guides we already
have available!
We have continued our work with a new website developer to help us improve our
website. Work has begun to make it easier for community members to find
information more easily and to make the site more efficient.
The Foundation owns the FreeBSD trademarks, and it is our responsibility to
protect them. We also provide legal support for the core team to investigate
questions that arise.
Go to http://www.FreeBSDfoundation.org/ to find out how we support FreeBSD and
how we can help you!
The FreeBSD Core Team is the governing body of FreeBSD.
Core approved a source commit bit for Alfredo Dal'Ava Júnior. Alfredo has
been working on powerpc64 support. Justin Hibbits (jhibbits) will mentor
Alfredo.
Core approved a source commit bit for Ryan Moeller. Ryan has been working on
porting ZoL to FreeBSD. Alexander Motin (mav) and Matt Macy (mmacy) will
mentor Ryan.
Core approved a source commit bit for Nick O'Brien. Nick has been working on
RISC-V at Axiado. Kristof Provost (kp) and Philip Paeps (philip) will mentor
Nick.
Core approved a source commit bit for Richard Scheffenegger. Richard has been
contributing TCP work. Michael Tuexen (tuexen) will mentor Richard and Rodney
Grimes (rgrimes) will act as co-mentor.
Core approved a source commit bit for Aleksandr Fedorov. Aleksandr has been
testing and reviewing bhyve networking code. Vincenzo Maffione (vmaffione)
will mentor Aleksandr and John Baldwin (jhb) will act as co-mentor.
Core requested that the freebsd-mobile@ list be retired as it was almost
- exclusively receiving spam. postmater@ completed core's request.
+ exclusively receiving spam. postmaster@ completed core's request.
Core approved third party authentication for some project services with
certain conditions. For example, for authentication with Google, users must
be using a FreeBSD.org account with two-factor authentication enabled. For
GitHub, we will enable and force multi-factor authentication for our
organization.
The Core-initiated Git Transition Working Group continued to meet over the
first quarter of 2020. Their report is still forthcoming.
The FreeBSD Release Engineering Team is responsible for setting
and publishing release schedules for official project releases
of FreeBSD, announcing code freezes and maintaining the
respective branches, among other things.
The FreeBSD Release Engineering Team published the schedules for the upcoming
11.4-RELEASE and 12.2-RELEASE cycles.
Much time was spent by Glen Barber working on updates to the various build
tools adding support for builds from both Subversion and Git. This is very
much a work in progress, as there are a number of inter-connected moving
parts.
Additionally throughout the quarter, several development snapshots builds
were released for the head, stable/12, and stable/11 branches.
Much of this work was sponsored by Rubicon Communications, LLC (netgate.com)
and the FreeBSD Foundation.
The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the Project relies on for its distributed work and communications to be synchronised. In this quarter, the team has worked on the following:
Upgrade all ref- and universe- machines
South Africa mirror (JINX) is online
Package service of Seattle, USA mirror (TUK) is online
Ongoing systems administration work:
Creating accounts for new committers.
Backups of critical infrastructure.
Keeping up with security updates in 3rd party software.
Setup Malaysia (KUL) mirror
Setup Brazil (BRA) mirror
Setup Amsterdam (PKT) mirror
Review the service jails and service administrators operation.
Infrastructure of building aarch64 and powerpc64 packages
NVME issues on PowerPC64 Power9 blocking dual socket machine from being used as pkg builder.
Drive upgrade test for pkg builders (SSDs) courtesy of the FreeBSD Foundation.
Boot issues with Aarch64 reference machines.
New NYI.net sponsored colocation space in Chicago-land area.
Prepare resource for git working group
Searching for more mirror providers
https://wiki.freebsd.org/Teams/clusteradm/generic-mirror-layout
https://wiki.freebsd.org/Teams/clusteradm/tiny-mirror
Contact: freebsd-testing Mailing List
Contact: IRC #freebsd-ci channel on EFNet
The FreeBSD CI team maintains the continuous integration system and related tasks
for the FreeBSD project. The CI system regularly checks the committed changes
can be successfully built, then performs various tests and analysis of the
results. The artifacts from the build jobs are archived in the artifact server for
further testing and debugging needs. The CI team members examine the
failing builds and unstable tests and work with the experts in that area to
fix the codes or adjust test infrastructure. The details of these efforts
are available in the weekly CI reports.
During the first quarter of 2020, we continue working with the contributors and developers in the project for their testing needs and also keep working with external projects and companies to improve their support of FreeBSD.
Important changes:
All the -head jobs are using clang/lld toolchain
All the -head test are using kyua in the base
RISC-V jobs now generate full disk image and run tests in QEMU with OpenSBI
freebsd-doc job also checks building of www.freebsd.org
https://ci.freebsd.org/job/FreeBSD-head-amd64-test_ltp/
https://ci.freebsd.org/job/FreeBSD-head-powerpc64-images/
https://ci.freebsd.org/job/FreeBSD-head-powerpc64-testvm/
Collecting and sorting CI tasks and ideas here
Setup the CI stage environment and put the experimental jobs on it
Implementing automatic tests on bare metal hardware
Adding drm ports building test against -CURRENT
Testing and merging pull requests in the FreeBSD-ci repo
Planning for running ztest and network stack tests
Helping more 3rd software get CI on FreeBSD through a hosted CI solution
Adding non-x86 test jobs.
Adding external toolchain related jobs.
Adding more hardware to the hardware lab
Sponsor: The FreeBSD Foundation
The Ports Management Team is responsible for overseeing the
overall direction of the Ports Tree, building packages, and
personnel matters. Below is what happened in the last quarter.
During the last quarter the number of ports settled in at 39,000.
There are currently just over 2,400 open PRs of which 640 are
unassigned. The last quarter saw 8146 commits by 173 committers
to the HEAD branch and 357 commits by 52 committers to the
2020Q1 branch. This means the number of PRs grew although the
committer activity remained more or less constant.
As always, people come and go. This time we welcomed Loïc
Bartoletti (lbartoletti@), Mikael Urankar (mikael@), Kyle Evans
(kevans@, who is already a src committer), and Lorenzo Salvadore
(salvadore@, who we already know for compiling these reports you
are reading right now). We said goodbye to dbn@ and theraven@,
who we hope to see back in the future.
On the infrastructure side, USES=qca was added and USES=zope was
removed. The latter was also due to it was incompatible with
Python 3, and portmgr is in the process of removing Python 2.7 from
the Ports Tree. This means that all ports that currently rely on
Python 2.7 need to be updated to work with Python 3 or be removed.
After a long period of work by multiple people, Xorg got updated
from the 1.18 to the 1.20 release series. Also, the web browsers
were updated: Firefox to version 75.0, Firefox ESR to 68.7.0, and
Chromium to 80.0.3987.149. The package manager itself got updated
to version 1.13.2.
antoine@ ran 29 exp-runs during the last quarter for various updates
to KDE, poppler, pkg and build tools; and test compatibility with src
changes: removing procfs-based debugging, fixing TLS alignment, and
only including libssp_nonshared.a in libc for the i386 and Power
architectures.
The FreeBSD X11/Graphics team maintains the lower levels of the FreeBSD graphics
stack.
This includes graphics drivers, graphics libraries such as the
MESA OpenGL implementation, the X.org xserver with related libraries and
applications, and Wayland with related libraries and applications.
The biggest highlight by far during the previous quarter was the long awaited
update of These updates caused some fallout when they first were comitted, most notably
issues with keyboards.
But with help from Michael Gmelin and others on the mailing lists, most issues
were sorted fast.
Unfortunately version 304 of the nVidia graphics driver is no longer supported
as of this release.
Since this update, xorg-server has also been bumped to 1.20.8, which is the
latest upstream release.
Apart from this update, there has also been ongoing work to keep the various
drm-kmod ports and packages up to date, mostly in response to changes in FreeBSD
CURRENT and to security issues found in the Intel i915 driver.
We have also done updates as needed to keep the graphics and input stack up to
date and working, and deprecated and removed several old and no longer used
drivers, applications and libraries.
We have also continued our regularly scheduled bi-weekly meetings.
People who are interested in helping out can find us on the x11@FreeBSD.org
mailing list, or on our gitter chat: (https://gitter.im/FreeBSDDesktop/Lobby).
We are also available in #freebsd-xorg on EFNet.
We also have a team area on GitHub where our work repositories can be found:
(https://github.com/FreeBSDDesktop)
In an effort to improve NFS security, an internet draft
which I expect will become and RFC soon specifies the
use of TLS 1.3 to encrypt all data traffic on a Sun RPC
connection used for NFS.
Although NFS has been able to use sec=krb5p to encrypt data
on the wire, this requires a Kerberos environment and, as
such, has not been widely adopted. It also required that
encryption/decryption be done in software, since only the
RPC message NFS arguments are encrypted.
Since Kernel TLS is capable of using hardware assist to
improve performance and does not require Kerberos, NFS
over TLS may be more widely adopted, once implementations
are available.
Since FreeBSD's kernel TLS requires that data be in ext_pgs
mbufs for transmission, most of the work so far has been
modifying the NFS code that builds the protocol arguments
to optionally use ext_pgs mbufs.
Coding changes to handle received ext_pgs mbufs has also
been done, although this may not be required by the receive
kernel TLS.
The kernel RPC has also been modified to do the STARTTLS
Null RPC and to do upcalls to userland daemons that
perform the SSL_connect()/SSL_accept(), since the kernel
TLS does not do this initial handshake.
So far only a self signed certificate on the server,
with no requirement for the client to have a certificate
has been implemented.
Work is still needed to be done for the case where the NFS
client is expected to have a signed certificate. In particular,
it is not obvious to me what the correct solution is for
clients that do not have a fixed IP address/DNS name.
The code now is about ready for testing, but requires that
the kernel TLS be able to support receive as well as transmit.
Patches to the kernel TLS for receive are being worked on
by jhb@freebsd.org.
Once receive side kernel TLS becomes available, the code in
subversion under base/projects/nfs-over-tls will need third
party testing and a security evaluation by someone familiar
with TLS.
The FreeBSD test suite uses the Kyua test framework to run tests.
Historically Kyua has been installed from the ports collection
( We hope that this inclusion will spur testing of embedded platforms
and simplify the process of testing within continuous integration
systems.
We currently plan to retain the Sponsor: DARPA
Work during this quarter focused on source code cleanup and making
it easier to debug missing functionality. There were, however,
some user-visible changes: added support for The Linux Test Project tests that are being run as part of the
the FreeBSD Continuous Integration infrastructure
now include the Open POSIX test suite.
There's still a lot to do:
There are pending reviews for patches that add
extended attributes support,
and fexecve(2) syscall, and
they require wrapping up and committing
There are over 400 failing LTP tests.
Some of them are false positives, some are easy to fix bugs, and some require adding
new system calls. Any help is welcome.
See the syzkaller entry in the 2019q1 quarterly report for an
introduction to syzkaller.
A number of kernel bugs have been found by syzkaller and fixed this
quarter, mostly in the network stack and file descriptor table code.
Bug investigations have led to improvements in debugging facilities and
assertions, for example in the SCTP stack. Syzkaller reproducers have
been added to Peter Holm's stress2 suite, helping ensure that
regressions are found quickly.
The syzkaller instance hosted by backtrace.io (see the 2019q3 report)
has been very useful in testing syzkaller improvements and finding bugs.
Though Google runs a dedicated syzkaller instance targeting FreeBSD,
it has proved fruitful to run multiple instances since they end up
building different corpuses and thus discover different, though
overlapping, sets of bugs.
Support for fuzzing a number of new system calls has been added,
including the new copy_file_range() and __realpathat() system calls,
and the Capsicum system calls. Some work was also done to audit
existing system call definitions to ensure that FreeBSD-specific
extensions of POSIX system calls are covered. Work is ongoing to target
the Linux emulation layer, and to collect kernel dumps so that one-off
crashes with no reproducer have a chance at being diagnosed and fixed.
Sponsor: backtrace.io
Sponsor: The FreeBSD Foundation
The current implementation of if_bridge uses a single mutex to protect its
internal data structures. As a result it’s nowhere near as fast as it could be.
This is relevant for users who want to run many vnet jails or virtual machines
bridged together, for example.
As part of this project several new tests have already been added for
if_bridge. These are generally very useful for validating any locking changes,
and will also help to prevent regressions for other future changes. These
tests live in /usr/tests/sys/net/if_bridge_test.
The current work is concentrating on investigating if it's possible to leverage
the ConcurrencyKit epoch code for the datapath (i.e. Sponsor: The FreeBSD Foundation
Rtld services need to be async signal safe. This is needed, for
instance, to provide working symbol bindings in signal handlers.
For threaded processes, libthr interposes all user-installed signal
handlers and saves the signals and related context if signal is
delivered while rtld or libthr are in protected section of code.
In non-threaded processes, the async safety is provided by changing
signal mask for the thread. It is actually better than the
interposing done by libthr, since signals are delivered in the right
context, instead of libthr attempt of recreate it later. But the
unfortunate side-effect is that each rtld entry requires two syscalls,
one to set mask, and one to restore it. Typically this adds around 40
or more syscalls on each process startup. Worse, rtld services used
by typical language runtime exception handling systems also have the
cost of signal mask manipulation.
The new sigfastblock(2) syscall was added that allows thread to
designate a memory location as fast signal block. If this word
contains non-zero value, kernel interprets the thread state same as if
all blockable signals are blocked. The facility drastically improves
exception handling speed on FreeBSD.
Since signals might abort interruptible sleeps, initial implementation
read the blocking word on each syscall entry. This is needed to
ensure that userspace does not see spurious EINTR/ERESTART if the
signals are blocked by the word. Since if kernel cached outdated
value for the block word, it would abort sleep, but then ast sees the
correct mask and does not deliver the pending signal.
There were concerns that this read of the word causes slowdown in
syscalls microbenchmarks, esp. on machines with SMAP. The reason is
that SMAP requires all userspace access bracketed by STAC/CLAC pair of
instructions, which are de-facto serializing (this is not
architectural, but all current microarchitectures do it). The
decision was made to eliminate the word read, at the cost of possibly
returning spurious EINTR. The impact should be minimal, since
sigfastblock(2) is not supposed to be the service available to users,
it is only assumed for rtld and libthr implementations.
Sponsor: The FreeBSD Foundation
An investigation of some performance oddities on EC2 Graviton 2
instances resulted in support for the use of Large System Extension
(LSE) atomic instructions in the FreeBSD kernel.
LSE is an mandatory ISA extension specified in ARMv8.1. It consists of
a number of new atomic instructions, superseding the
Load-Linked/Store-Conditional (LL/SC) instruction pairs use when LSE is
not implemented. The extension is present in a number of ARMv8 server
platforms, including the Cavium ThunderX2 and AWS Graviton 2. The new
instructions provide significantly better scalability.
A recent set of patches modified the FreeBSD kernel to detect support
for LSE and dynamically select an atomic(9) implementation based on
the new instructions when all CPUs implement the extension. The initial
atomic(9) implementations were provided by Ali Saidi. Some benchmarking
on a 64-vCPU Graviton 2 instance shows a ~4% reduction in wall clock
time for a kernel build, and a ~15% reduction in system CPU time.
Some ARMv8 multi-processor systems implement a heterogenous CPU
architecture, referred to as big.LITTLE, in which multiple processor
types are used. Surprisingly, such systems may implement the LSE on
only a subset of its CPUs, in which case LSE instructions cannot be used
by the kernel. As a result, FreeBSD currently waits until all
processors are online before selecting the atomic(9) implementation,
which precludes the use of ifuncs to provide dynamic selection.
Currently atomic(9)'s use of LSE is limited to the kernel. A future
project would extend this to userspace, so that FreeBSD system libraries
can leverage the LSE instructions when they are available.
Sponsor: The FreeBSD Foundation
Sponsor: Amazon
Wei is working on HyperV Socket support for FreeBSD. HyperV Socket provides a way for the HyperV host and guest to communicate using a common socket interface without networking required. Some features in Azure require HyperV Socket support in the guest.
Details of HyperV Socket is available here.
The work-in-progress is available here
This project is sponsored by Microsoft.
Li-Wen is working on the FreeBSD release code related to Azure for the -CURRENT and 12-STABLE branches. The release of 12.1-RELEASE on Azure is also in progress.
The work-in-progress is available here
This project is sponsored by The FreeBSD Foundation.
CHERI (Capability Hardware Enhanced RISC Instructions) extends
conventional hardware Instruction-Set Architectures (ISAs) with new
architectural features to enable fine-grained memory protection and
highly scalable software compartmentalization. The CHERI
memory-protection features allow historically memory-unsafe programming
languages such as C and C++ to be adapted to provide strong, compatible,
and efficient protection against many currently widely exploited
vulnerabilities. The CHERI scalable compartmentalization features enable
the fine-grained decomposition of operating-system (OS) and application
code, to limit the effects of security vulnerabilities in ways that are
not supported by current architectures. CHERI is a hybrid capability
architecture in that it is able to blend architectural capabilities with
conventional MMU-based architectures and microarchitectures, and with
conventional software stacks based on virtual memory and C/C++. This
approach allows incremental deployment within existing ecosystems, which
we have demonstrated through hardware and software prototyping.
On 18 October 2019, Arm announced Morello, an experimental
CHERI-extended, multicore, superscalar ARMv8-A processor, System-on-Chip
(SoC), and prototype board to be available from late 2021. Morello is a
part of the UKRI £187M Digital Security by Design Challenge (DSbD)
supported by the UK Industrial Strategy Challenge Fund, including a
commitment of over £50M commitment by Arm. The aim is to test and
validate CHERI extensions to the Arm ISA at scale with the idea that
"successful concepts are expected to be carried forward into the
architecture." The Morello board is scheduled to ship in the third
quarter of 2021.
Over the past decade we have developed CheriBSD, a version of FreeBSD
supporting CHERI. Our public facing work has been performed on MIPS64
and more recently on RISC-V. Andrew has also developed a port to an
earlier version of the Morello ISA which we will be merging into
our public repository as simulators and compilers become available.
The Morello board is based on the Arm Neoverse N1 platform and derived
from the N1SDP development platform. (The AWS Graviton2 systems are
also based on the N1 core.) Ruslan and Andrew are currently
working to enable all relevant features of the N1 and the N1SDP to give
us a solid baseline for work on Morello. These features include the
PCI root complex, system memory management unit (SMMU), and CoreSight.
To the extent practical we are upstreaming these features to FreeBSD.
Sponsor: DARPA, UKRI
The Semihalf team initiated working on FreeBSD support for the
NXP LS1046A SoC
LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with
integrated packet processing acceleration and high speed peripherals
including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide
range of networking, storage, security and industrial applications.
Completed since the last update:
Clean-up and rebase support on top of FreeBSD-HEAD. Prepare features
for the upstream submission:
QorIQ platform clockgen driver
LS1046A clockgen driver
GPIO support for QorIQ boards
QorIQ LS10xx AHCI driver
VF610 I2C controller support
TCA6416 GPIO expander
Epson RX-8803 RTC
QorIQ LS10xx SDHCI driver
Upstreaming of developed features. This work is expected to
be submitted/merged to HEAD in the Q2 of 2020.
ENA (Elastic Network Adapter) is the smart NIC available in the
virtualized environment of Amazon Web Services (AWS). The ENA
driver supports multiple transmit and receive queues and can handle
up to 100 Gb/s of network traffic, depending on the instance type
on which it is used.
Completed since the last update:
Upstream of the driver to v2.1.1, introducing:
Bug fix for LLQ mode which was causing race when multiple IO queues were
used
Last touches for ENA v2.2.0 release, introducing:
Add driver support for the upcoming HW features (like Rx offsets,
reporting Tx drops)
Add sysctl tuneables for IO queue number
Create IO queues with optional size backoff
Rework the way of configration of drbr and Rx ring size to be more robust
and stable
New HAL version
Other minor fixes and improvements
The FreeBSD/powerpc project continues to mature.
In addition to the above listed people, we want to acknowledge
contributions from adalava, bdragon, luporl, and mikael, among
others.
Key points:
On -CURRENT, all platforms have been switched to the
LLVM 10.0 compiler and lld10. Thus, ld.bfd has been removed
from base.
On powerpc64, -CURRENT has been switched to the ELFv2 ABI.
Older versions of -CURRENT that either used GCC, or LLVM with
the ELFv1 ABI, are no longer supported.
On powerpc64 FreeBSD-STABLE (11 and 12), the platforms still
remain on the antique gcc4.2.1 in base. Note: that version of
GCC has been removed from the -CURRENT src tree. Support for
this configuration is now a "best-effort" status.
On powerpc (32-bit), the ABI did not change as with powerpc64,
so upgrading should be easier than with powerpc64.
The aacraid(4) driver has been been fixed for big-endian, thanks
to luporl. This means that Talos customers who got the SAS option
can now use the onboard SAS.
The ixl(4) driver has also been fixed for big-endian, also thanks
to luporl.
As a result of -CURRENT switching to LLVM/ELFv2, ifuncs became
available, meaning that we now have optimized memcpy/bcopy and
strncpy functions when running on processors that supports VSX
instructions.
powerpc64 is now able to run on QEMU without the need of
Huge Pages support.
The virtio drivers have been fixed.
kernel minidump has been fixed.
A FreeBSD.org package set is available for powerpc64/12
(quarterly). The -quarterly build has just been rebased
from 12.0 to 12.1, per the desupport of the older 12.0.
The first rebased build has been completed, with 29776
packages being available.
We are currently working on the upgrade of the package
builder to a recent -CURRENT. Therefore, the available
packages for -CURRENT are still ELFv1, which are not useful.
Please contact Mark Linimon for more information.
mesa has been switched to llvm90, which fixes certain
problems.
Work continues on firefox and related ports.
More ports fixes are being committed every day.
POWER9 machines, and Oregon State University (OSU) for providing the
hosting. As well, we would like to thank the clusteradm team for
keeping the Tyan POWER8 machines online that are hosted at
NYI.
Also, Piotr would like to thank the FreeBSD Foundation for
funding his personal Talos, and Raptor (via its IntegriCloud
subsidiary) for loaning a server on which talos.anongoth.pl runs.
Contact: freebsd-riscv Mailing List
Contact: IRC #freebsd-riscv channel on freenode
It has been a year since the RISC-V project's last status report. In that time,
the RISC-V port has benefited from increased attention, and received
improvements of all kinds.
The RISC-V project has brought in two new src committers. We'd like to welcome
Jessica Clarke (jrtc27@), who is a member of CheriBSD, and Nick O'Brien (nick@)
of Axiado to the team.
Some highlights from last year:
Bring-up on SiFive's Hifive Unleashed board
Support for the OpenSBI firmware and version 0.2 of the SBI specification
Addition of the UART, SPI, and PRCI device drivers for the HiFive Unleashed
required a small number of integration changes on our side, but was mainly
enabled by the upstream improvements to the RISC-V LLVM back-end. LLVM's RISC-V
support became "official" with LLVM 9, and LLVM 10 has brought further
improvements. The LLVM back-end is expected to continue to mature, as there are
now many parties actively involved in its development. GCC remains supported as
an external toolchain for RISC-V.
The CI job for HEAD
has been updated to use the clang/lld toolchain, and a GCC job will be added in the future.
The RISC-V disk image built in the CI system now contains the full base system and
is available on the CI artifact server for
further testing. The CI test job was updated to use OpenSBI in qemu. Work on
running the FreeBSD test suite for RISC-V in the CI system is in progress.
Some progress has been made on supporting the ports framework on RISC-V, which
was mostly untested until recently. First,
The CheriBSD project saw an initial port to RISC-V this quarter. Preliminary
support for the CHERI ISA has been added to the Spike and QEMU emulators, as
well as the necessary changes on the CheriBSD side. Currently, the CheriBSD
RISC-V kernel boots, and most statically compiled CHERI binaries run without
issue.
Although real RISC-V hardware is still scarce, any users with an interest
trying out or contributing to the RISC-V port are encouraged to do so. Please
visit the recently updated wiki page for information on getting set up, or check
out "Getting Started with FreeBSD/RISC-V" in the January/February edition of The
FreeBSD Journal.
Sponsor: DARPA, AFRL, Axiado, the FreeBSD Foundation
In 2007 the GNU Compiler Collection (GCC) migrated to GPLv3, which
prompted discussions about the future of the FreeBSD tool chain. We held
a Tool Chain Summit at
BSDCan 2010. Roman Divacky gave an update on the ClangBSD project, building
FreeBSD using the new and rapidly improving Clang compiler.
Since that time Clang was imported into the FreeBSD base system and was used
more and more widely - first being installed but not the default GCC 4.2.1 was kept in the tree for a few FreeBSD targets that hadn't migrated
to Clang, such as MIPS and Sparc64. By early this year all remaning targets
had migrated to external toolchain (contemporary GCC from ports or packages),
or had been deprecated.
With no in-tree consumers remaining, GCC 4.2.1 was removed from FreeBSD in
r358454 on February 29,
2020.
Sponsor: The FreeBSD Foundation
In r340076 Ed added the
NT_FREEBSD_FEATURE_CTL ELF note, used to allow binaries to opt out of,
or in to, vulnerability mitigation and other features. FreeBSD Foundation
intern Bora Özarslan later added a tool to decode and modify the ELF note,
but it had yet to be installed by default.
In the previous quarter Ed renamed the tool to elfctl, and installed it
in /usr/bin. Ed also committed a number of minor bug fixes, code style
improvements, etc.
Usage examples - list known feature flags:
General
Partnerships and Commercial User Support
Fundraising Efforts
OS Improvements
Continuous Integration and Quality Assurance
Supporting FreeBSD Infrastructure
FreeBSD Advocacy and Education
We continued producing FreeBSD advocacy material to help people promote FreeBSD.
Legal/FreeBSD IP
Work in progress:
New jobs added:
Work in progress:
Please see freebsd-testing@ related tickets for more WIP information, and join the efforts
xorg-server
to version 1.20.
After years of work by many people, this update finally landed in the form of
xorg-server
1.20.7.
With this update came a couple of new things, most notably, FreeBSD 12 and later
was switched to use the udev/evdev backend by default for handling input
devices, such as mice and keyboards.
Together with this release, the OpenGL library implementation mesa
was
switched to use DRI3 by default, instead of the older DRI2.
devel/kyua
). While this is fine for mainstream architectures,
it can pose bootstrapping issues on new architectures and package
installation is quite slow under emulation or on FPGA based systems.
By including it in the FreeBSD base system we can avoid these issues.
devel/kyua
port to serve FreeBSD
versions without and to serve as a development version.
TCP_CORK
as required by Nginx,
added support MAP_32BIT
flag, which fixes Mono binaries from Ubuntu Bionic,
and a fix for DNS resolution with glibc newer than 2.30, which affected
CentOS 8.
Sponsor: The FreeBSD Foundation
bridge_input()
,
bridge_output()
, bridge_forward()
, ...).
Todo:
Sponsor: Alstom Group
Work in progress:
Sponsor: Amazon.com Inc
Hardware status:
Software status:
Package status:
The team would like to thank IBM for the loan of two POWER8 and one
Last quarter, the default compiler and linker was switched to clang/lld. This
emulators/qemu-user-static-devel
received an update adding support for the
RISC-V 64-bit ABI, allowing ports to be cross-compiled via poudiere(8)
.
Second, improvements were made to the detection of the soft-float ABI,
riscv64sf. Systems running either of the hard-float or soft-float ABIs can now
compile and run ports natively. At the moment a small subset of ports can be
built successfully, and in the coming months we will look to improve that to
include a base set of crucial ports (e.g. python or perl).
cc
, then
used by default on i386 and amd64, and later used on more and more targets.
In the years since Dimitry Andric has been keeping our copy of Clang
up-to-date.
$ elfctl -l
Known features are:
aslr Disable ASLR
protmax Disable implicit PROT_MAX
stackgap Disable stack gap
wxneeded Requires W+X mappings
List feature tags set on a binary:
$ elfctl /bin/ls
File '/bin/ls' features:
aslr 'Disable ASLR' is unset.
protmax 'Disable implicit PROT_MAX' is unset.
stackgap 'Disable stack gap' is unset.
wxneeded 'Requires W+X mappings' is unset.
Indicate that a binary requests to opt-out of address randomization:
$ elfctl -e +aslr binary
Sponsor: The FreeBSD Foundation
A number of performance and functional improvements were committed to ELF Tool Chain tools over the last quarter.
FreeBSD Foundation intern Tiger Gao added DWARF Debug Information Entry (DIE) caching to addr2line which provided a substantial improvement when translating many entries (even surpassing GNU addr2line with a large list).
Tiger also rebased and updated an upstream ELF Tool Chain submission to handle DW_AT_ranges and addressed two elfcopy/objcopy bugs: setting the OS/ABI field correctly when converting a binary file to ELF, and correctly adding new sections when there is no .shstrtab section.
Ed committed several readelf improvements, including decoding the PROTMAX_DISABLE, STKGAP_DISABLE, and WXNEEDED ELF feature control flags, decoding Xen and GNU Build-ID ELF notes, and improved input validation.
Mark Johnston addressed many memory and file descriptor leaks and similar issues reported by Coverity Scan.
Sponsor: The FreeBSD Foundation
As announced on January, The FreeBSD Project is adopting Weblate as its web-based continuous localization platform.
We are getting new volunteers to the effort and so far these are the numbers:
10 languages
47 registered users
Chinese (Simplified) (zh_CN)
Chinese (Traditional) (zh_TW) - Added
French (fr_FR) - Added
German (de_DE) - Added
Italian (it_IT) - Added
Norwegian Bokmål - Added - New language on FreeBSD
Persian (fa_IR) - Added - New language on FreeBSD
Portuguese (Brazil)
Spanish
Turkish (tr-TR) [1] - Added - New language on FreeBSD
We want to thank everyone that contributed, translating or reviewing documents.
And please, help promote this effort on your local user group, we always need more volunteers.
I am currently working on an overhaul for the FreeBSD manpages by updating the HISTORY and STANDARDS sections and while here creating new manpages for parts of the system that missing documentation. FreeBSD has already one of the best documentation available for an UNIX-like operation system, but there are parts that could be improved.
For the parts that have been already improved you can have a look at my Phabricator account.
If you would like to help on improving the documentation effort, please contact Benedict Reuschling bcr@freebsd.org or me at gbergling@gmail.com.
The KDE on FreeBSD project packages the software produced by the KDE Community for FreeBSD. The software includes a full desktop environment KDE Plasma, the art application Krita, video editor Kdenlive and hundreds of other applications that can be used on any FreeBSD desktop machine.
The quarter opened with a new kstars (amateur astronomy application) release landing in ports, and then had the usual regular updates:
three KDE Frameworks releases (on a monthly schedule),
three bugfix releases to the collection of KDE software from the KDE release service (formerly KDE Applications, but it was always more that only-applications),
three bugfix releases to the KDE Plasma desktop.
devel/cmake
, and regular work all over the ports tree.
The SDDM login manager was updated to a much newer -- by over a year -- release and patched to support more FreeBSD features.
One update to devel/qca
dropped compatibility with FreeBSD 11
because upstream no longer supports older OpenSSL versions.
There is infrastructure in the ports tree now that adds a USES=qca
for Qt applications needing crypto support.
The open bugs list remains stable around 28 open issues, with some interesting xkb issues as a highlight. We welcome detailed bug reports and patches. KDE packaging updates are prepared in a copy of the ports repository on GitHub and then merged in SVN. We welcome pull requests there as well.
After the XFCE update to 4.14 a regression was observed in the XFCE window manager xfwm4. It caused window decorations to be drawn wrong or missing with certain graphic hardware setups. It has been reported that the recent update to Xorg server in the ports tree fixes this issue. The updated Xorg server will be available in the next qurterly branch.
The standard Wine port has moved from Wine 4.0.3 to Wine 5.0 which represents over 7,400 individual changes including built-in modules in PE format, multi-monitor support, Vulkan 1.1 support, and an XAudio2 re-implementation.
After our request for help in the last quarterly report the i386 wine ports have been adopted by salvadore who immediately started resolving existing bugs and improving the ports. Most of this work is ready and we began committing first pieces in March. Since it takes more time than initially expected, we will also update the i386-wine-devel port during this process so that users needing a more recent version can easily get it from the ports tree (or binary packages). On the other hand, we plan on backporting these improvements to i386-wine after i386-wine-devel is done and only then update that port, so that we always guarantee a stable version of i386-wine.
Starting from the recently released version 1.14, Go now officially supports 64-bit ARM architecture on FreeBSD 12.0 or later. This porting effort was initially started by Greg V (aka myfreeweb) and resumed by Shigeru Yamamoto, Dmitri Goutnik and Mikaël Urankar. Dmitry has set up a CI builder to catch regression on FreeBSD aarch64 (it's required by the golang policy for adding a new port to the main Go repository)
Work in progress:
a lot of ports use an old version of golang.org/x/sys or golang.org/x/net (to name a few) that doesn't contain the FreeBSD aarch64 bits, work is being done to fix these ports (details are in the bug tracker entry
In the previous third and fouth quarterly status reports 2019, the sysctlinfo interface and an extension to improve the sysctlbyname() syscall were described, they can access to the sysctl MIB and pass the properties of an object to the userland, but both are quite low level and kernel related.
The sysctlmibinfo2 library provides an API to explore the sysctl MIB, to convert an object name in its corresponding Object Identifier and to find an object to get its properties, therefore it is useful to handle an object correctly and to build a sysctl-like utility.
Primarily sysctlmibinfo2 wraps the low level interface to provide an easy API, some example: sysctlmif_desc() retrieves the description of an object, sysctlmif_kind() gets the type (string, integer, etc) and sysctlmif_fmt() specifies the format (an integer could represent a deciKelvin, milliKelvin, etc), then it is possible to print properly an object value.
Moreover sysctlmibinfo2 provides a high level API: a struct sysctlmif_object definition and functions to build data structures of objects. Example, let's say we want to manage the sound system, sysctlmif_grouplistbyname("hw.snd") returns the list of the Sound Driver objects and sysctlmif_treebyname("dev.pcm") returns a tree where "dev.pcm" is the root node and each subtree represents an audio device.
Obviously sysctlmibinfo2 benefits of the features of sysctlinfo: handles OIDs up to CTL_MAXNAME levels, supports capability mode, can seek an object by its name (avoiding to explore the MIB just to find the corresponding OID), gets all info about an object in a time, manages a name with a NULL level or expanded with an input for the sysctl handler.
The library can be installed via the devel/libsysctlmibinfo2 port, a manual page and examples in the Public Domain are available for getting started your projects.
An initial effort to write proper documentation and guides for the pot project has started. The documentation, even if incomplete, is available at here. A F.A.Q. page is available and waiting for users to submit their questions.
During the last quarter, some bugs were reported on pot and on the nomad-pot-driver. Both projects released a new bug fix version. Many thanks to 'grembo' and 'Crest' that reported issues, tested and tried our solutions. Thanks also to Mateusz (0mp) for his Pull Requests!
pot will have a new release soon (0.11.0), focused on network:
network stack support: ipv4 only, ipv6 only, dual stack.
flexible network setup for alias: adding the ability to use an arbitrary network setup for alias network type
NomadBSD is a persistent live system for USB flash drives, based on FreeBSD. Together with automatic hardware detection and setup, it is configured to be used as a desktop system that works out of the box, but can also be used for data recovery, for educational purposes, or testing FreeBSD's hardware compatibility.
In March we released a new minor version 1.3.1 which improves the configuration of the network interfaces, fixed some bugs and added nomadbsd-chusr and nomadbsd-sysinfo. Further some new features found their way into the release.
Some days later the channel explainingcomputers on YouTube released a review video of NomadBSD. The explainingcomputers has almost 600,000 followers and the review was positive so we saw the highest peak in downloads ever! Along with it came a lot of people looking for help on our mailing list and on Twitter so we decided to set up a new support forum.
We are looking for people to help the project. Help is much appreciated in all areas:
Translation of program interfaces
Design artwork
Programming new tools, extend existing ones
Tests and Bug reports / UX and feature suggestions
Mirrors outside of Europe
Support installation on disk partitions and add a partition editor GUI.
Complete disk encryption
Add a user-friendly network manager
Entries from the various official and semi-official teams, as found in the Administration Page.
Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects.
Updates to kernel subsystems/features, driver support, filesystems, and more.
Updating platform-specific features and bringing in support for new hardware platforms.
.Changes affecting the base system and programs in it.
Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.
Noteworthy changes in the documentation tree, in manpages, or in external books/documents.
Many projects build upon &os; or incorporate components of &os; into their project. As these projects may be of interest to the broader &os; community, we sometimes include brief updates submitted by these projects in our quarterly report. The &os; project makes no representation as to the accuracy or veracity of any claims in these submissions.