Changeset View
Standalone View
en_US.ISO8859-1/htdocs/news/status/report-2018-01-2018-09.xml
- This file was added.
<?xml version="1.0" encoding="utf-8" ?> | |||||
<!DOCTYPE report PUBLIC "-//FreeBSD//DTD FreeBSD XML Database for | |||||
Status Report//EN" | |||||
"http://www.FreeBSD.org/XML/share/xml/statusreport.dtd" > | |||||
<!-- $FreeBSD: head/en_US.ISO8859-1/htdocs/news/status/report-template.xml 51132 2017-10-19 01:38:11Z bjk $ --> | |||||
<!-- This file was generated with https://github.com/trasz/md2docbook --> | |||||
<!-- | |||||
Variables to replace: | |||||
%%START%% - report month start | |||||
%%STOP%% - report month end | |||||
%%YEAR%% - report year | |||||
%%NUM%% - report issue (first, second, third, fourth) | |||||
%%STARTNEXT%% - report month start | |||||
%%STOPNEXT%% - report month end | |||||
%%YEARNEXT%% - next report due year (if different than %%YEAR%%) | |||||
%%DUENEXT%% - next report due date (i.e., June 6) | |||||
--> | |||||
<report> | |||||
<date> | |||||
<month>%%START%%-%%STOP%%</month> | |||||
<year>%%YEAR%%</year> | |||||
</date> | |||||
<section> | |||||
<title>Introduction</title> | |||||
<p>With &os; having gone all the way to 12, it is perhaps | |||||
useful to take a look back at all the things that have been | |||||
accomplished, in terms of many visible changes, as well as all | |||||
the things that happen behind the scenes to ensure that &os; | |||||
continues to offer an alternative in both design, | |||||
implementation, and execution.</p> | |||||
<p>The things you can look forward to reading about are too | |||||
numerous to summarize, but cover just about everything from | |||||
finalizing releases, administrative work, optimizations | |||||
and depessimizations, features added and fixed, and many areas | |||||
of improvement that might just surprise you a little.</p> | |||||
<p>Please have a cup of coffee, tea, hot cocoa, or other beverage | |||||
of choice, and enjoy this culmulative set of reports covering | |||||
everything that's been done since October, 2017.</p> | |||||
allanjude: It looks like we never issued the 2017-10 - 2017-12 report, and there was very little in it… | |||||
Done Inline ActionsWell, tbh it's been described a few times as applying to the past three quarters. I'd rather leave it as it is. trasz: Well, tbh it's been described a few times as applying to the past three quarters. I'd rather… | |||||
<p>—Daniel Ebdrup</p> | |||||
</section> | |||||
<category> | |||||
<name>team</name> | |||||
<description>&os; Team Reports</description> | |||||
<p>Entries from the various official and semi-official teams, | |||||
as found in the <a href="&enbase;/administration.html">Administration | |||||
Page</a>.</p> | |||||
</category> | |||||
<category> | |||||
<name>proj</name> | |||||
<description>Projects</description> | |||||
<p>Projects that span multiple categories, from the kernel and userspace | |||||
to the Ports Collection or external projects.</p> | |||||
</category> | |||||
<category> | |||||
<name>arch</name> | |||||
<description>Architectures</description> | |||||
<p>Updating platform-specific features and bringing in support | |||||
for new hardware platforms.</p>. | |||||
</category> | |||||
<category> | |||||
<name>ports</name> | |||||
<description>Ports</description> | |||||
<p>Changes affecting the Ports Collection, whether sweeping | |||||
changes that touch most of the tree, or individual ports | |||||
themselves.</p> | |||||
</category> | |||||
<category> | |||||
<name>doc</name> | |||||
<description>Documentation</description> | |||||
<p>Noteworthy changes in the documentation tree or new external | |||||
books/documents.</p> | |||||
</category> | |||||
<category> | |||||
<name>third</name> | |||||
<description>Third-Party Projects</description> | |||||
<p>Many projects build upon &os; or incorporate components of | |||||
&os; into their project. As these projects may be of interest | |||||
to the broader &os; community, we sometimes include brief | |||||
updates submitted by these projects in our quarterly report. | |||||
The &os; project makes no representation as to the accuracy or | |||||
veracity of any claims in these submissions.</p> | |||||
</category> | |||||
<project cat='team'> | |||||
<title>Release Engineering Team</title> | |||||
<contact> | |||||
<person> | |||||
<name>FreeBSD Release Engineering Team</name> | |||||
<email>re@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://www.FreeBSD.org/releases/10.4R/announce.html">FreeBSD 10.4-RELEASE announcement</url> | |||||
<url href="https://www.FreeBSD.org/releases/11.2R/announce.html">FreeBSD 11.2-RELEASE announcement</url> | |||||
<url href="https://www.FreeBSD.org/releases/12.0R/schedule.html">FreeBSD 12.0-RELEASE schedule</url> | |||||
<url href="https://download.FreeBSD.org/ftp/snapshots/ISO-IMAGES/">FreeBSD development snapshots</url> | |||||
</links> | |||||
<body> | |||||
<p>The FreeBSD Release Engineering Team responsibilities | |||||
include:</p> | |||||
<ul> | |||||
<li>setting and publishing release schedules for official | |||||
project releases</li> | |||||
</ul> | |||||
<p>of FreeBSD</p> | |||||
<ul> | |||||
<li>announcing code slushes, freezes, and thaws</li> | |||||
<li>maintaining the respective branches for all supported | |||||
releases</li> | |||||
</ul> | |||||
<p> | |||||
The FreeBSD Release Engineering Team, led by Marius | |||||
Strobl, | |||||
completed the 10.4-RELEASE in early October 2017. FreeBSD | |||||
10.4-RELEASE was the | |||||
fifth release from the <tt>stable/10</tt> branch, which | |||||
built on the | |||||
stability and reliability of 10.3-RELEASE.</p> | |||||
<p>The FreeBSD 11.2-RELEASE cycle started April 20, 2018 with | |||||
the | |||||
announcement of the code slush. The first stage progress | |||||
was | |||||
continued throughout the rest of the quarter with the code | |||||
freeze, | |||||
followed by three BETA builds, three RC builds, and the | |||||
final release | |||||
build was announced June 27, 2018.</p> | |||||
<p>The FreeBSD Release Engineering Team started the | |||||
12.0-RELEASE cycle | |||||
August 10, 2018 with the announcement of the code slush. | |||||
The code | |||||
freeze followed on August 24, 2018. The tentative date for | |||||
the <tt>stable/12</tt> branch was expected to be September 21, | |||||
2018.</p> | |||||
<p> | |||||
Done Inline Actionsthis markup in the middle of the paragraph seems off allanjude: this markup in the middle of the paragraph seems off | |||||
Due to unforeseen circumstances with upstream code that | |||||
was necessary | |||||
to include in 12.0-RELEASE, the tentative release schedule | |||||
needed | |||||
to be adjusted several times. The API changes in the | |||||
updated version | |||||
of the upstream code required changes to be made for all | |||||
base system | |||||
utilities that linked with the upstream code. By the end | |||||
of the | |||||
2018Q3 quarter, the <tt>stable/12</tt> branch had not been | |||||
created due to | |||||
this delay.</p> | |||||
<p>Throughout the remainder of 2018Q3, several development | |||||
snapshots builds | |||||
were released for the <tt>head</tt>, <tt>stable/11</tt>, | |||||
and <tt>stable/10</tt> branches.</p> | |||||
<p>Much of this work was sponsored by the FreeBSD Foundation.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='team'> | |||||
<title>Ports Collection</title> | |||||
<contact> | |||||
<person> | |||||
<name>René Ladan</name> | |||||
<email>portmgr-secretary@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://www.FreeBSD.org/ports/">About FreeBSD Ports</url> | |||||
<url href="https://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html">Contributing to ports</url> | |||||
<url href="http://portsmon.FreeBSD.org/index.html">FreeBSD ports monitoring</url> | |||||
<url href="https://www.FreeBSD.org/portmgr/index.html">Ports Management Team</url> | |||||
<url href="@freebsd_portmgr)](https://twitter.com/freebsd_portmgr/">FreeBSD portmgr (@freebsd_portmgr)</url> | |||||
<url href="Facebook)](https://www.facebook.com/portmgr">FreeBSD Ports Management Team (Facebook)</url> | |||||
<url href="Google+)](https://plus.google.com/communities/108335846196454338383">FreeBSD Ports Management Team (Google+)</url> | |||||
</links> | |||||
Done Inline Actionss/management/Management/ bcr: s/management/Management/
(it's capitalized in the entry below) | |||||
<body> | |||||
<p>During the first quarter of 2018, the number of ports grew | |||||
to almost 32,000. | |||||
In 2018Q1, there were | |||||
2,100 open PRs with fewer than 600 unassigned. There were | |||||
7,900 commits from 169 committers. Compared to last | |||||
quarter, the number | |||||
of commits grew by 18% and the number of PRs dropped by | |||||
25%. Those are | |||||
some good numbers!</p> | |||||
<p>During the 2018Q2 and 2018Q3 quarters, the number of ports | |||||
grew to just under | |||||
34,000. The number of open PR grew to almost 2,500 with | |||||
fewer than 600 | |||||
of those unassigned. A total of 175 committers made almost | |||||
14,200 commits. | |||||
Compared to the first quarter, the number of commits | |||||
dropped by 10% and | |||||
the number of PRs grew by 19%.</p> | |||||
<p>During the last three quarters, portmgr took twelve commit | |||||
bits in for | |||||
safekeeping: daichi@, deichen@, ian@, junovitch@, kevlo@, | |||||
maho@, nemysis@, | |||||
pawel@, rea@, tabthorpe@, vg@, and wxs@.</p> | |||||
<p>Portmgr welcomed thirteen new committers in 2018Q2 and | |||||
2018Q3:</p> | |||||
<ul> | |||||
<li>Devin Teske (dteske@)</li> | |||||
<li>Eric Turgeon (ericbsd@)</li> | |||||
<li>Fernando Apesteguía (fernape@)</li> | |||||
<li>Fukang Chen (loader@)</li> | |||||
<li>Gleb Popov (arrowd@)</li> | |||||
<li>Jesper Schmitz Mouridsen (jsm@)</li> | |||||
<li>John Hixson (jhixson@)</li> | |||||
<li>Kevin Bowling (kbowling@)</li> | |||||
<li>Koichiro IWAO (meta@)</li> | |||||
<li>Mateusz Piotrowski (0mp@)</li> | |||||
<li>Matthias Fechner (mfechner@)</li> | |||||
<li>Sergey Kozlov (skozlov@)</li> | |||||
</ul> | |||||
<p> | |||||
The following committers returned after a hiatus:</p> | |||||
<ul> | |||||
<li>Ion-Mihai Tetcu (itetcu@)</li> | |||||
<li>Kevin Lo (kevlo@)</li> | |||||
<li>Sean Chittenden (seanc@)</li> | |||||
</ul> | |||||
<p> | |||||
During the last three quarters, Antoine Brodin (antoine@) | |||||
ran no | |||||
fewer than 113 exp-runs against the ports tree. These runs | |||||
were | |||||
executed to test updates, perform cleanups, and make | |||||
improvements | |||||
to the framework and the base system. Most of the runs | |||||
were for | |||||
port upgrades, but others include LLD progress, changes to | |||||
the | |||||
default port versions, improved support for armv6, armv7, | |||||
and RISC-V | |||||
architectures, removed old base system functionality, new | |||||
USES, and | |||||
better matching pkg-plist with Makefile options (DOCS and | |||||
EXAMPLES).</p> | |||||
<p>Five new USES values were introduced:</p> | |||||
<ul> | |||||
<li>apache: handle dependencies on the Apache web server and | |||||
modules</li> | |||||
<li>eigen: automatically depend on math/eigen2 or math/eigen3</li> | |||||
<li>emacs: handle dependencies on the Emacs editor and | |||||
modules.</li> | |||||
<li>gl replaces the old USE_GL from bsd.port.mk</li> | |||||
<li>qt-dist, qt:4 and qt:5 replace the old USE_QT from | |||||
bsd.qt.mk</li> | |||||
</ul> | |||||
<p> | |||||
The EXTRA_PATCHES functionality has been extended to | |||||
support | |||||
directories, where it will automatically apply all | |||||
patch-\* files to the port.</p> | |||||
<p>Ports using USES=php:phpize, php:ext, php:zend, and | |||||
php:pecl have | |||||
been flavored and packages are now automatically built | |||||
for all | |||||
versions of PHP that are supported (5.6, 7.0, 7.1 or 7.2).</p> | |||||
<p>2018Q3 had updates of major ports: pkg 1.10.5, Chromium | |||||
Done Inline Actionss/automatically be built/automatically built/ bcr: s/automatically be built/automatically built/ | |||||
65.0.3325.181, Firefox 59.0.2, Firefox-ESR 52.7.3, Ruby | |||||
2.3.7/2.5.1 | |||||
and Qt5 5.9.4. | |||||
The default version of PHP was changed from 5.6 to 7.1. | |||||
The former | |||||
version of PHP is no longer supported by the developers. | |||||
The | |||||
default versions of Samba and GCC are now respectively 4.7 | |||||
and 7. The | |||||
Xorg ports have been reorganized and there have been | |||||
changes to | |||||
net/openntpd. Please review the UPDATING file for relevant | |||||
details.</p> | |||||
<p>Open tasks:</p> | |||||
<ul> | |||||
<li>The number of commits dropped somewhat over the last three | |||||
quarters, | |||||
leaving more PRs unresolved. If possible, please pick up | |||||
some PRs | |||||
and improve everyone's experience.</li> | |||||
</ul> | |||||
</body> | |||||
</project> | |||||
<project cat='team'> | |||||
<title>Core Team</title> | |||||
<contact> | |||||
<person> | |||||
<name>FreeBSD Core Team</name> | |||||
<email>core@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>Much of Core's focus for the past months has been on three | |||||
items:</p> | |||||
<p>1. Coordination between different groups to support the | |||||
upcoming 12.0 release. The timing of the OpenSSL | |||||
1.1.1 release posed challenges, the new OpenSSL | |||||
version included API changes, many components of | |||||
the base system and ports required changes. | |||||
Staying with the older OpenSSL in 12.0 was not a | |||||
feasible option, because it would have meant | |||||
backporting many changes to a version of OpenSSL | |||||
that would be unmaintained by the upstream source.</p> | |||||
<p>2. Discussions with the release engineering team and Scott | |||||
Long about updating the FreeBSD release process. | |||||
Topics for exploration include:</p> | |||||
<ul> | |||||
<li>having more frequent point releases</li> | |||||
<li>changing the support model</li> | |||||
<li>revising and improving the tooling used to manage the tree | |||||
and releases</li> | |||||
<li>additional topics as they are discovered</li> | |||||
</ul> | |||||
<p>3. Gathering information to make decisions more | |||||
data-driven. For example, we are planning | |||||
developer and user surveys. If there are questions | |||||
that you think should be added to the survey, | |||||
please discuss them on freebsd-arch@. We are | |||||
exploring ways for automated user-driven hardware | |||||
usage data to understand the changing ways our | |||||
Done Inline ActionsYou can write text directly after the <p> tag, no need to keep it on a line of its own. bcr: You can write text directly after the <p> tag, no need to keep it on a line of its own. | |||||
software is used and to target better hardware | |||||
support.</p> | |||||
<p>Here are other noteworthy events (in chronological order) | |||||
since the last quarterly report.</p> | |||||
<p>2017 Q4</p> | |||||
<ul> | |||||
<li>Sean Eric Fagan's (sef@) commit bit was reactivated with a | |||||
period of re-mentoring under Alexander Motin | |||||
(mav@).</li> | |||||
<li>The MIPS architecture was promoted to tier 2 status.</li> | |||||
Done Inline ActionsShould these be headings? allanjude: Should these be headings? | |||||
Done Inline ActionsThose are "level 3 headings", and were marked as "<p>" in the previous report; I've decided to follow that markup for now. trasz: Those are "level 3 headings", and were marked as "<p>" in the previous report; I've decided to… | |||||
<li>Core approved changes to the Code of Conduct.</li> | |||||
<li>All fortune data files, except freebsd-tips, were removed | |||||
in r325828.</li> | |||||
<li>Core approved the adoption of a policy requiring any | |||||
license exceptions to be recorded alongside code.</li> | |||||
<li>Gordon Tetlow (gordon@) became the new security officer.</li> | |||||
<li>Core approved the use of SPDX tags.</li> | |||||
</ul> | |||||
<p> | |||||
2018 Q1</p> | |||||
<ul> | |||||
<li>Jeb Cramer (jeb@) was awarded a src commit bit under the | |||||
mentorship of Sean Bruno (sbruno@) and Eric Joyner | |||||
(erj@).</li> | |||||
<li>Members of the CoC Review Team were approved. The | |||||
membership is to be reviewed once per year.</li> | |||||
<li>A vendor commit bit was awarded to Slava Shwartsman | |||||
(slavash@) of Mellanox Technologies under the | |||||
mentorship of Konstantin Belousov (kib@) and Hans | |||||
Petter Selasky (hselasky@).</li> | |||||
<li>Walter Schwarzenfeld was awarded project membership.</li> | |||||
<li>Brad Davis (brd@) was awarded a src commit bit under the | |||||
mentorship of Allan Jude (allanjude@) with | |||||
Baptiste Daroussin (bapt@) as co-mentor.</li> | |||||
<li>Vincenzo Maffione (vmaffione@) was awarded a src commit | |||||
bit under the mentorship of Hiroki Sato (hrs@).</li> | |||||
<li>Ram Kishore Vegesna (ram@) was awarded a src commit bit | |||||
under the mentorship of Kenneth D. Merry (ken@) | |||||
and Alexander Motin (mav@).</li> | |||||
</ul> | |||||
<p>2018 Q2</p> | |||||
<ul> | |||||
<li>Tom Jones (thj@) was awarded a src commit bit under the | |||||
mentorship of Jonathan T. Looney (jtl@).</li> | |||||
<li>Matt Macy's (mmacy@) commit bit was restored under the | |||||
mentorship of Sean Bruno (sbruno@).</li> | |||||
Done Inline ActionsThis could easily be one line: <p>2018 Q2</p> bcr: This could easily be one line: <p>2018 Q2</p> | |||||
<li>Breno Leitao (leitao@) was awarded a src commit bit under | |||||
the mentorship of Justin Hibbits (jhibbits@) with | |||||
Nathan Whitehorn (nwhitehorn@) as co-mentor.</li> | |||||
<li>Leandro Lupori (luporl@) was awarded a src commit bit | |||||
under the mentorship of Justin Hibbits (jhibbits@) | |||||
with Nathan Whitehorn (nwhitehorn@) as co-mentor.</li> | |||||
<li>The handover from the ninth to the tenth elected Core team | |||||
took place. The tenth Core members are: Allan Jude | |||||
(allanjude@), Benedict Reuschling (bcr@), Brooks | |||||
Davis (brooks@), Hiroki Sato (hrs@), Warner Losh | |||||
(imp@), Jeff Roberson (jeff@), John Baldwin | |||||
(jhb@), Kris Moore (kmoore@), and Sean Chittenden | |||||
(seanc@).</li> | |||||
<li>Joseph Mingrone (jrm@) was appointed the Core secretary | |||||
under mentorship of the retiring Core secretary, | |||||
Matthew Seaman (matthew@).</li> | |||||
<li>The new team liaisons were decided. portmgr: Sean, doceng: | |||||
Hiroki, secteam: Brooks, re: John, clusteradm: | |||||
Allan, CoC: Warner, Foundation: Benedict, | |||||
bugmeister: John, CI: Sean.</li> | |||||
<li>David Maxwell (dwm@) was awarded project membership.</li> | |||||
<li>Daichi Goto's (daichi@) commit bit was reactivated with a | |||||
period of re-mentoring under George Neville-Neil | |||||
(gnn@).</li> | |||||
<li>A vendor commit bit was awarded to Ben Widawsky | |||||
(bwidawsk@) of Intel under the mentorship of Ed | |||||
Maste (emaste@).</li> | |||||
</ul> | |||||
<p> | |||||
2018 Q3</p> | |||||
<ul> | |||||
<li>Core decided to begin meeting twice per month in an | |||||
attempt to catch up with many new agenda items.</li> | |||||
<li>Li-Wen Hsu (lwhsu@) was awarded a src commit bit under the | |||||
mentorship of Mark Johnston (markj@) with Ed Maste | |||||
(emaste@) as co-mentor.</li> | |||||
<li>Samy al Bahra was awarded project membership.</li> | |||||
<li>George Neville-Neil (gnn@) was approved to begin | |||||
co-mentoring Vincenzo Maffione (vmaffione@).</li> | |||||
</ul> | |||||
</body> | |||||
</project> | |||||
<project cat='team'> | |||||
<title>The FreeBSD Foundation</title> | |||||
<contact> | |||||
<person> | |||||
Done Inline ActionsIs this one really necessary? bcr: Is this one really necessary? | |||||
<name>Deb Goodkin</name> | |||||
<email>deb@FreeBSDFoundation.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>The FreeBSD Foundation is a 501(c)(3) non-profit | |||||
organization dedicated to supporting and promoting | |||||
the FreeBSD Project and community worldwide. | |||||
Funding comes from individual and corporate | |||||
donations and is used to fund and manage software | |||||
development projects, conferences and developer | |||||
summits, and provide travel grants to FreeBSD | |||||
contributors. The Foundation purchases and | |||||
supports hardware to improve and maintain FreeBSD | |||||
infrastructure and provides resources to improve | |||||
security, quality assurance, and release | |||||
engineering efforts; publishes marketing material | |||||
to promote, educate, and advocate for the FreeBSD | |||||
Project; facilitates collaboration between | |||||
commercial vendors and FreeBSD developers; and | |||||
finally, represents the FreeBSD Project in | |||||
executing contracts, license agreements, and other | |||||
legal arrangements that require a recognized legal | |||||
entity.</p> | |||||
<p>Here are some highlights of what the FreeBSD Foundation | |||||
did to help FreeBSD last quarter:</p> | |||||
<p>Partnerships and Commercial User Support</p> | |||||
<p>As a 501(c)(3) non-profit, we don't directly support | |||||
commercial users, but we do work with them to | |||||
understand their needs and help facilitate | |||||
collaboration with the community. Last quarter we | |||||
met with a few key FreeBSD users and supporters, | |||||
to discuss pain points, how they can contribute | |||||
back to FreeBSD, and what technologies they would | |||||
like to see supported, to support FreeBSD over | |||||
more of their technologies and products.</p> | |||||
<p>As many of you know, we formed a partnership with Intel | |||||
around one and a half years ago. Since then the | |||||
people we worked directly with left the company, | |||||
but it moved us into a new relationship with their | |||||
Open Source Technology Center (OTC).</p> | |||||
<p>We are very encouraged that Intel has dedicated additional | |||||
resources from the OTC to work on FreeBSD in | |||||
addition to existing resources from the networking | |||||
group and other technologies such as QuickAssist. | |||||
Much of the work has been focused on security and | |||||
OS mitigations but we're also focusing on other | |||||
areas such as power management and persistent | |||||
memory. In May and again in July we traveled to | |||||
Intel's Hillsboro campus to meet with management | |||||
and engineers from OTC and the networking team. We | |||||
presented an overview of the project and | |||||
Foundation and also discussed key markets and | |||||
vendors who use FreeBSD in their products or | |||||
services and their future requirements.</p> | |||||
<p>Intel was also interested in learning more about who | |||||
contributes to FreeBSD. Along those lines we've | |||||
done some work with OTC to create scripts and | |||||
organizational mappings to answer that question. | |||||
Note that we do need developers | |||||
to help us update and maintain the organizational | |||||
mappings as we understand that developers do tend | |||||
to move around and contractors are often working | |||||
on behalf of multiple organizations.</p> | |||||
<p>Fundraising Efforts</p> | |||||
<p>Our work is 100% funded by your donations. As of September | |||||
30, we raised $328,482. Our 2018 fundraising goal | |||||
is $1,250,00 and we are continuing to work hard to | |||||
meet and exceed this goal! Please consider making | |||||
Not Done Inline Actionsmissing link allanjude: missing link | |||||
Not Done Inline ActionsThere is no link here at (here). I'd just remove it or find the original link (blog post?). bcr: There is no link here at (here). I'd just remove it or find the original link (blog post?). | |||||
a donation to help us continue and increase our | |||||
support for FreeBSD: <a | |||||
Done Inline ActionsThe colon (:) can be removed here. bcr: The colon (:) can be removed here. | |||||
href="https://www.FreeBSDfoundation.org/donate/">https://www.FreeBSDfoundation.org/donate/</a>.</p> | |||||
<p>We also have a new Partnership Program, to provide more | |||||
benefits for our larger commercial donors. Find | |||||
out more information at <a | |||||
href="https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/">https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/</a> | |||||
and share with your companies!</p> | |||||
<p>OS Improvements</p> | |||||
<p>The Foundation improves the FreeBSD operating system by | |||||
employing technical staff to maintain and improve | |||||
critical kernel subsystems, add features and | |||||
functionality, and fix problems. This also | |||||
includes funding separate project grants like the | |||||
arm64 port, porting the blacklistd access control | |||||
daemon, and the integration of VIMAGE support, to | |||||
make sure that FreeBSD remains a viable solution | |||||
for research, education, computing, products and | |||||
more.</p> | |||||
<p>We kicked off or continued the following projects last | |||||
quarter:</p> | |||||
<ul> | |||||
<li>OpenZFS RAID-Z Expansion project</li> | |||||
<li>Headless mode out-of-the-box for embedded ARM boards like | |||||
the Beaglebone Black</li> | |||||
<li>Performance and scalability improvements</li> | |||||
</ul> | |||||
<p> | |||||
Having software developers on staff has allowed us to jump | |||||
in and work directly on projects to improve | |||||
FreeBSD such as:</p> | |||||
<ul> | |||||
<li>ZFS improvements</li> | |||||
Done Inline Actionss/Arm/ARM/ (acronym) bcr: s/Arm/ARM/ (acronym) | |||||
<li>New Intel server support</li> | |||||
<li>kqueue(2) updates</li> | |||||
<li>64-bit inode support</li> | |||||
<li>Stack guard</li> | |||||
<li>Kernel Undefined Behavior Sanitizer</li> | |||||
<li>Toolchain projects</li> | |||||
<li>i915 driver investigation</li> | |||||
<li>NVDIMM support in acpiconf(8)</li> | |||||
<li>Continuous integration dashboard (web page and physical | |||||
hardware)</li> | |||||
<li>FAT filesystem support in makefs(8)</li> | |||||
</ul> | |||||
<p> | |||||
Continuous Integration and Quality Assurance</p> | |||||
<p>The Foundation provides a full-time staff member who is | |||||
working on improving our automated testing, | |||||
continuous integration, and overall quality | |||||
assurance efforts.</p> | |||||
<p>Foundation employee Li-Wen Hsu set up new CI servers to | |||||
speed up amd64 build and test jobs, to reduce the | |||||
latency between changes being committed and | |||||
results being available. Li-Wen also set up a | |||||
staging / development server in order to test | |||||
changes to the CI system itself without affecting | |||||
production results. We have also started a small | |||||
hardware test lab, currently connected to the | |||||
staging server, that tests the full boot and test | |||||
cycle on physical hardware. In the near future | |||||
additional hardware devices will be added, and | |||||
this will migrate to the production CI server.</p> | |||||
<p>Release Engineering</p> | |||||
<p>The Foundation provides a full-time staff member to lead | |||||
the release engineering efforts. This has provided | |||||
timely and reliable releases over the last five | |||||
years.</p> | |||||
<p>Foundation employee Glen Barber continued leading the | |||||
efforts on the upcoming 12.0-RELEASE. For details | |||||
surrounding the work involved and progress thus | |||||
far on 12.0-RELEASE, please see the FreeBSD | |||||
Release Engineering Team section of this quarterly | |||||
status report.</p> | |||||
<p>Supporting FreeBSD Infrastructure</p> | |||||
<p>The Foundation provides hardware and support to improve | |||||
the FreeBSD infrastructure. Last quarter, we | |||||
continued supporting FreeBSD hardware located | |||||
around the world.</p> | |||||
Done Inline ActionsThe comma here can go. bcr: The comma here can go. | |||||
<p>FreeBSD Advocacy and Education</p> | |||||
<p>A large part of our efforts are dedicated to advocating | |||||
for the Project. This includes promoting work | |||||
being done by others with FreeBSD; producing | |||||
advocacy literature to teach people about FreeBSD | |||||
and help make the path to starting using FreeBSD | |||||
or contributing to the Project easier; and | |||||
attending and getting other FreeBSD contributors | |||||
to volunteer to run FreeBSD events, staff FreeBSD | |||||
tables, and give FreeBSD presentations.</p> | |||||
<p>The FreeBSD Foundation sponsors many conferences, events, | |||||
and summits around the globe. These events can be | |||||
BSD-related, open source, or technology events | |||||
geared towards underrepresented groups. We support | |||||
the FreeBSD-focused events to help provide a venue | |||||
for sharing knowledge, to work together on | |||||
projects, and to facilitate collaboration between | |||||
developers and commercial users. This all helps | |||||
provide a healthy ecosystem. We support the | |||||
non-FreeBSD events to promote and raise awareness | |||||
of FreeBSD, to increase the use of FreeBSD in | |||||
different applications, and to recruit more | |||||
contributors to the Project.</p> | |||||
<p>Check out some of the advocacy and education work we did | |||||
last quarter:</p> | |||||
<ul> | |||||
<li>Organized and ran the Essen FreeBSD Hackathon in Essen, | |||||
Germany</li> | |||||
<li>Participated in the FreeBSD Developer Summit BSDCam, in | |||||
Cambridge, England</li> | |||||
<li>Represented FreeBSD at the ARM Partner Meeting</li> | |||||
<li>Presented and taught about FreeBSD at SdNOG 5 in Khartoum, | |||||
Sudan</li> | |||||
<li>Exhibited and gave a talk at OSCON 2018 in Portland, OR</li> | |||||
<li>Exhibited at the 2018 Grace Hopper Celebration and | |||||
sponsored as a Silver Non-Profit Sponsor</li> | |||||
<li>Exhibited at COCON 2018 in Taipei, Taiwan</li> | |||||
<li>Sponsored and gave presentations and tutorials at | |||||
EuroBSDCon in Bucharest, Romania</li> | |||||
<li>Organized and ran the Bucharest FreeBSD Developer Summit</li> | |||||
<li>Sponsored the 2018 USENIX Security Symposium in Baltimore, | |||||
MD as an Industry Partner</li> | |||||
<li>Provided FreeBSD advocacy material</li> | |||||
<li>Sponsored the 2018 USENIX Annual Technical Conference in | |||||
Boston, MA as an Industry Partner</li> | |||||
<li>Sponsored the OpenZFS Developer Summit as a Silver Sponsor</li> | |||||
<li>Presented and taught about FreeBSD at SANOG32 in Dhaka, | |||||
Bangladesh</li> | |||||
<li>Sponsored the SNIA Storage Developer Conference 2018 as an | |||||
Association Partner</li> | |||||
<li>Provided 11 travel grants to FreeBSD contributors to | |||||
attend many of the above events.</li> | |||||
</ul> | |||||
<p> | |||||
We continued producing FreeBSD advocacy material to help | |||||
people promote FreeBSD around the world.</p> | |||||
<p>Read more about our conference adventures in the | |||||
conference recaps and trip reports in our monthly | |||||
newsletters: <a | |||||
href="https://www.freebsdfoundation.org/news-and-events/newsletter/">https://www.freebsdfoundation.org/news-and-events/newsletter/</a></p> | |||||
<p>We help educate the world about FreeBSD by publishing the | |||||
professionally produced FreeBSD Journal. Last | |||||
quarter we published the July/August issue that | |||||
you can find at <a | |||||
href="https://www.FreeBSDfoundation.org/journal/">https://www.FreeBSDfoundation.org/journal/</a>.</p> | |||||
<p>You can find out more about events we attended and | |||||
upcoming events at <a | |||||
href="https://www.FreeBSDfoundation.org/news-and-events/">https://www.FreeBSDfoundation.org/news-and-events/</a>.</p> | |||||
<p>Legal/FreeBSD IP</p> | |||||
<p>The Foundation owns the FreeBSD trademarks, and it is our | |||||
responsibility to protect them. We also provide | |||||
legal support for the core team to investigate | |||||
questions that arise.</p> | |||||
<p>Go to <a | |||||
href="http://www.FreeBSDfoundation.org">http://www.FreeBSDfoundation.org</a> | |||||
to find out how we support FreeBSD and how we can | |||||
help you!</p> | |||||
</body> | |||||
</project> | |||||
<project cat='team'> | |||||
<title>Continuous Integration</title> | |||||
<contact> | |||||
<person> | |||||
<name>Jenkins Admin</name> | |||||
<email>jenkins-admin@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Li-Wen Hsu</name> | |||||
<email>lwhsu@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://ci.FreeBSD.org">FreeBSD Jenkins Instance</url> | |||||
<url href="https://artifact.ci.FreeBSD.org/">FreeBSD CI artifact archive</url> | |||||
<url href="https://wiki.FreeBSD.org/Jenkins">FreeBSD Jenkins wiki</url> | |||||
<url href="https://lists.FreeBSD.org/mailman/listinfo/freebsd-testing">freebsd-testing Mailing List</url> | |||||
<url href="https://github.com/freebsd/freebsd-ci">freebsd-ci Repository</url> | |||||
</links> | |||||
<body> | |||||
<p>The FreeBSD CI team maintains continuous integration tasks | |||||
for FreeBSD. The CI | |||||
system regularly checks the changes committed to the | |||||
project's Subversion | |||||
repository can be successfully built, and performs various | |||||
tests and analysis | |||||
with the build results. The CI team also maintains the | |||||
archive of the artifact | |||||
built by the CI system, for the further testing and | |||||
debugging needs.</p> | |||||
<p>Starting from June 2018, the project is sponsored by the | |||||
FreeBSD Foundation in | |||||
hardware and staff. For more details of the sponsored | |||||
projects, please refer | |||||
to:</p> | |||||
<p><a | |||||
href="https://www.freebsdfoundation.org/news-and-events/newsletter/freebsd-foundation-update-september-2018/">https://www.freebsdfoundation.org/news-and-events/newsletter/freebsd-foundation-update-september-2018/</a></p> | |||||
<p>In addition to that, we also helped checking regressions | |||||
for OpenSSL 1.1.1 | |||||
update and test continuously for 12-STABLE branch.</p> | |||||
<p>We had meetings and working groups at two developer | |||||
summits during 2018Q3:</p> | |||||
<ul> | |||||
<li><a | |||||
href="https://wiki.FreeBSD.org/DevSummit/201808/Testing">BSDCam | |||||
2018</a></li> | |||||
<li><a | |||||
href="https://wiki.FreeBSD.org/DevSummit/201809">EuroBSDCon | |||||
2018</a></li> | |||||
</ul> | |||||
<p> | |||||
Work in progress:</p> | |||||
<ul> | |||||
<li>Fixing the failing test cases and builds</li> | |||||
<li><a | |||||
href="https://ci.FreeBSD.org/job/FreeBSD-head-amd64-dtrace_test/lastCompletedBuild/testReport/">DTrace | |||||
test</a></li> | |||||
<li><a | |||||
href="https://ci.FreeBSD.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/">ZFS | |||||
test</a></li> | |||||
<li><a | |||||
href="https://ci.FreeBSD.org/job/FreeBSD-head-amd64-gcc/">GCC | |||||
build</a></li> | |||||
<li>Adding drm ports building test against -CURRENT</li> | |||||
<li>Adding tests for selected project branches, e.g.: | |||||
clang700-import</li> | |||||
<li>Adding new hardware to the embedded testbed</li> | |||||
<li>Implementing automatic tests on bare metal hardware</li> | |||||
<li>Planning running ztest and network stack tests</li> | |||||
</ul> | |||||
</body> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>4G/4G address space split for i386</title> | |||||
<contact> | |||||
<person> | |||||
<name>Konstantin Belousov</name> | |||||
<email>kib@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
Done Inline ActionsLeftover? These can go... bcr: Leftover? These can go... | |||||
<body> | |||||
<p>Most 32-bit FreeBSD architectures, including i386, started | |||||
to suffer | |||||
from the rapid growth of the size of software during the | |||||
past decade. | |||||
When a 32-bit address space is enough space for a given | |||||
task, 32-bit | |||||
mode still has an intrinsic advantage over 64-bit mode, | |||||
due to less | |||||
memory traffic and more economical use of caches. It has | |||||
grown | |||||
harder to provide the self-hosting i386 system build due | |||||
to the | |||||
increase in size of the build tools.</p> | |||||
<p>The FreeBSD i386 kernel, prior to the 12.0-RELEASE | |||||
version, split | |||||
the 4GB address space of the platform into 3GB (minus 4MB) | |||||
accessible | |||||
to userspace accesses and 1GB for kernel accesses. | |||||
Neither kernel nor userspace could access a full 4GB | |||||
address space. | |||||
Programs that require very large virtual address spaces, | |||||
such as | |||||
clang when compiling or lld when linking, could run out of | |||||
address | |||||
space: 3GB of address space was insufficient for their | |||||
operation. | |||||
The kernel also had trouble fitting into the traditional | |||||
1GB | |||||
limitation of address space with the modern sizing for | |||||
network | |||||
buffers, ZFS and other KVA-hungry in-kernel subsystems.</p> | |||||
<p>In FreeBSD 12, the i386 architecture has been changed to | |||||
provide | |||||
dedicated separate address spaces for userspace and | |||||
kernel, giving | |||||
each mode full access to 4GB (minus 8MB) of usable address | |||||
space. | |||||
The userspace on the i386 architecture now has access to | |||||
the same | |||||
amount of address space as the compat32 subsystem in the | |||||
amd64 | |||||
architecture kernel. The increase in kernel address space | |||||
enables | |||||
further growth and maintainability of the i386 | |||||
architecture.</p> | |||||
<p>The split 4GB/4GB user/kernel implementation uses two page | |||||
directory | |||||
entries (PDEs) shared between modes: one for mapping the | |||||
page table, | |||||
another for the mode switching trampoline and other | |||||
required system | |||||
tables. The required system tables, which must always be | |||||
mapped, | |||||
regardless of kernel or user mode, includes such things as | |||||
the | |||||
GDT/IDT/TSS entries. Significant changes were made to the | |||||
locore | |||||
code. The page table creation portion of the code was | |||||
completely | |||||
rewritten from assembly to C, improving readability and | |||||
maintainability | |||||
of the code.</p> | |||||
<p>Because the user address space is no longer shared with | |||||
the kernel, | |||||
the copyout(9) functions were rewritten to make a | |||||
transient mapping | |||||
of userspace pages for the duration of any needed | |||||
accesses. The initial | |||||
implementation used the vm_fault_quick_hold_pages() | |||||
framework, but | |||||
this was later optimized by temporarily switching to user | |||||
mode | |||||
mappings from a trampoline, and then using hand-written | |||||
assembler | |||||
routines to perform a faster small block copy operation.</p> | |||||
<p>Future plans for the ongoing maintenance of i386 include | |||||
making the i386 pmap | |||||
capable of runtime selection of the PAE or non-PAE page | |||||
table format | |||||
and supporting NX (no execute) mappings for regular i386 | |||||
kernel.</p> | |||||
</body> | |||||
<sponsor> | |||||
The FreeBSD Foundation | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>FreeBSD/DTrace</title> | |||||
<contact> | |||||
<person> | |||||
<name>George Neville-Neil</name> | |||||
<email>gnn@FreeBSD.org</email> | |||||
</person> | |||||
Done Inline ActionsAnother one bites the dust... bcr: Another one bites the dust... | |||||
<person> | |||||
<name>Domagoj Stolfa</name> | |||||
<email>domagoj.stolfa@cl.cam.ac.uk</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>DTrace is a whole-system debugging tool in FreeBSD and is | |||||
one of the | |||||
actively supported projects during the past year.</p> | |||||
<p>A research prototype of a distributed version of DTrace | |||||
and a version | |||||
of DTrace that can trace bhyve virtual machines from the | |||||
host FreeBSD | |||||
system are currently under development at the University | |||||
of Cambridge | |||||
as a part of the CADETS project. Recent developments | |||||
include the | |||||
creation of dlog, an in-kernel DTrace consumer which is | |||||
able to | |||||
publish to Kafka, and improvements to early boot and | |||||
shutdown tracing. | |||||
On the virtualisation front, | |||||
improvements were made in the ability to dereference and | |||||
follow | |||||
pointers inside guests from the host in the probe context | |||||
by | |||||
implementing a nested page table walk inside DTrace for | |||||
Intel | |||||
architectures. The CADETS project has started formalizing | |||||
DTrace in HOL4 which enables automated test generation, | |||||
high assurance | |||||
of DTrace implementations in terms of adherence to the | |||||
specification | |||||
and exploration of all allowable behaviors for a given D | |||||
script. Currently, the formal model contains most of DIF | |||||
instructions | |||||
and a code generator for them, providing the ability to | |||||
run DIF | |||||
programs specified using the model using FreeBSD's DTrace | |||||
implementation.</p> | |||||
<p>As a result of all of this, a number of changes were | |||||
upstreamed to the | |||||
FreeBSD auditing subsystem and new variables such as | |||||
<tt>jid</tt> and | |||||
<tt>jailname</tt> were added to DTrace which can be | |||||
accessed from D scripts.</p> | |||||
<p>OpenDTrace Specification 1.0 has been published which | |||||
covers the | |||||
internal workings of DTrace in general, and its adaptation | |||||
to various | |||||
operating systems in particular. This work was sponsored | |||||
by | |||||
AFRL/DARPA through the CADETS project.</p> | |||||
<p>Ruslan Bukin (br@) has added C-compressed ISA extension | |||||
support to the | |||||
RISC-V FBT provider as a part of the ECATS project.</p> | |||||
<p>Mark Johnston (markj@) has done some work to fix | |||||
interactions between | |||||
FBT and ifuncs. ifuncs are a toolchain feature which allow | |||||
programmers | |||||
to select a function's implementation at boot-time, rather | |||||
than at | |||||
compile-time. For instance, on amd64, memcpy() is an ifunc | |||||
and may be | |||||
implemented by either memcpy_erms() or memcpy_std(). FBT | |||||
created | |||||
probes for the implementation functions, but we needed | |||||
some extra | |||||
support to ensure that fbt::memcpy:entry continues to work | |||||
as | |||||
expected. Similar work is needed for the PID provider, but | |||||
is still | |||||
pending.</p> | |||||
<p>Microsoft showed a <a | |||||
href="https://youtu.be/tG8R5SQGPck?t=891">working | |||||
demo of DTrace</a>, | |||||
which was ported from FreeBSD.</p> | |||||
<p>Added to FreeBSD base in 11.2, dwatch is a new DTrace | |||||
tool, developed | |||||
by Devin Teske (dteske@), for automating complex queries | |||||
for data and | |||||
surgically tapping the kernel. In base there are 85 | |||||
profiles for | |||||
interpreting domain-specific data with another 17 | |||||
available from ports | |||||
making a total of over 100 different pipelines from which | |||||
you can | |||||
extract data in multiple formats. dwatch also simplifies | |||||
observation | |||||
of over 100,000 probe points available in FreeBSD, making | |||||
it easy to | |||||
find any process, thread, or jail triggering any probe. | |||||
On top of all | |||||
that, dwatch profiles can leverage higher-level languages | |||||
such as | |||||
python, perl, sh, and many more.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>ACPI NVDIMM driver</title> | |||||
<contact> | |||||
<person> | |||||
<name>Konstantin Belousov</name> | |||||
<email>kib@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
Done Inline ActionsI think the dash between "On" and "top" here is not needed. bcr: I think the dash between "On" and "top" here is not needed. | |||||
<body> | |||||
<p>NVDIMM is a technology which provides non-volatile memory | |||||
with | |||||
access characteristics similar to regular DRAM, which is | |||||
the | |||||
technology that implements the normal memory address space | |||||
of a host. | |||||
There are ACPI and UEFI specifications that define | |||||
platform independent | |||||
ways to detect and enumerate the presence of NVDIMMs. | |||||
These | |||||
specifications allow the retrieval of most of the data | |||||
needed to | |||||
allow proper application use of the NVDIMM storage.</p> | |||||
<p>A new FreeBSD driver parses the ACPI NFIT table which | |||||
lists NVDIMMs, | |||||
their operational characteristics, and the physical | |||||
address space | |||||
where the NVDIMM memory is accessible. The driver presents | |||||
each | |||||
address region as two devices: One device allows userspace | |||||
to | |||||
open(2) a devfs node, which can be read/written/mapped | |||||
from the | |||||
application. This mapping is zero-copy. The second device | |||||
is a | |||||
geom disk(9), which makes it possible to use NVDIMM for | |||||
the backing | |||||
storage for a normal FreeBSD filesystem, such as UFS, ZFS, | |||||
or msdosfs. | |||||
Note that buffer cache/mapping of files from a traditional | |||||
filesystem | |||||
created over NVDIMM causes an unneeded double-buffering.</p> | |||||
<p>Empirically, on typical modern hardware, NVDIMM regions | |||||
are located | |||||
far from the regular DRAM backed memory in the address | |||||
space, and | |||||
have attributes that are not compatible with regular DRAM | |||||
memory. | |||||
This makes it unfeasible to extend the kernel's direct map | |||||
to provide | |||||
the kernel mappings for the NVDIMM regions. | |||||
A new pmap KPI was designed, pmap_large_map(9), | |||||
which allows efficient mapping of very large physical | |||||
regions into | |||||
the KVA. The new code has some optimizations to the cache | |||||
flushing | |||||
operations over the mapped regions, which is needed to | |||||
efficiently | |||||
support bio flushes from a filesystems using the NVDIMM | |||||
storage. | |||||
The NVDIMM driver is the first user of the new KPI, | |||||
but the new KPI might also be useful for the NTB driver.</p> | |||||
<p> | |||||
TODO:</p> | |||||
<ul> | |||||
<li>Intel is currently working on extending the driver to | |||||
support | |||||
UEFI namespaces.</li> | |||||
</ul> | |||||
<ul> | |||||
<li>A DAX-capable filesystem is needed, which solves the issue | |||||
of | |||||
double-buffering. Our tmpfs already provides VM facilities | |||||
which | |||||
allows it to avoid double-buffering for mmap, which can be | |||||
reused | |||||
there.</li> | |||||
</ul> | |||||
</body> | |||||
<sponsor> | |||||
The FreeBSD Foundation | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>SMAP</title> | |||||
<contact> | |||||
<person> | |||||
<name>Konstantin Belousov</name> | |||||
<email>kib@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>Support for SMAP (Supervisor-Mode Access Prevention), has | |||||
been added | |||||
to the amd64 kernel. The SMAP feature makes any access | |||||
from the | |||||
supervisor mode to the pages accessible to user mode cause | |||||
a fault, | |||||
unless the %eflags.AC bit is set at the time of the | |||||
access.</p> | |||||
<p>The SMAP implementation uses the ifunc framework to avoid | |||||
checking | |||||
for the SMAP capability of hardware on each call to | |||||
copyout(9) and | |||||
other functions.</p> | |||||
<p>In the amd64 architecture, FreeBSD has a common address | |||||
space between | |||||
the kernel space and user space. Enabling SMAP virtually | |||||
splits | |||||
the shared address space into two disjoint address spaces, | |||||
which | |||||
have different access criteria. This splitting of the | |||||
address space | |||||
provides a relatively low-overhead way of catching direct | |||||
accesses | |||||
from kernel to usermode, when not using the copyout(9) | |||||
family of | |||||
functions. The copyout(9) family of functions are | |||||
permitted direct | |||||
access to user space. Any direct access from kernel mode | |||||
to user | |||||
address space that isn't performed through the copyout(9) | |||||
family | |||||
of functions indicates a potential programming error.</p> | |||||
<p>It is interesting that very few bugs were found in the | |||||
FreeBSD | |||||
kernel after the SMAP feature was enabled. One issue that | |||||
was | |||||
identified existed in the pci(9) user driver. Enabling the | |||||
SMAP | |||||
feature identified at least two ports, VBox and acpi_call, | |||||
which | |||||
appeared to access userspace in an unsafe manner.</p> | |||||
</body> | |||||
<sponsor> | |||||
The FreeBSD Foundation | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>Large scale package building</title> | |||||
<contact> | |||||
<person> | |||||
<name>Mateusz Guzik</name> | |||||
<email>mjg@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>Building packages on a 128-thread machine with poudriere | |||||
exhibits some | |||||
bottlenecks.</p> | |||||
<p>See the October FreeBSD Foundation Newsletter for a short | |||||
write-up: | |||||
<a | |||||
href="https://www.freebsdfoundation.org/news-and-events/newsletter/freebsd-foundation-update-october-2018/">https://www.freebsdfoundation.org/news-and-events/newsletter/freebsd-foundation-update-october-2018/</a></p> | |||||
<p>One encountered problem stems from process handling.</p> | |||||
<p>The standard process life cycle on UNIX-like systems looks | |||||
like this:</p> | |||||
<ul> | |||||
<li>a process is created with fork(2)</li> | |||||
<li>it can do regular work or execve(2) a new binary</li> | |||||
<li>it exits, becoming a zombie</li> | |||||
<li>the parent collects the exit code and removes the zombie</li> | |||||
</ul> | |||||
<p> | |||||
There are other variations (e.g. vfork(2) can be used | |||||
instead of | |||||
fork).</p> | |||||
<p>When you type 'ls' into your shell, it will typically | |||||
vfork a new process | |||||
which will then execve /bin/ls.</p> | |||||
<p>All this is guarded with several global kernel locks, but | |||||
the granularity | |||||
can be significantly improved.</p> | |||||
<p>A different problem stems from pipes.</p> | |||||
<p>Pipes are used all the time, e.g. "du -s | sort -n" | |||||
creates a pipe whose | |||||
one endpoint is standard output for du and another is | |||||
standard input for sort.</p> | |||||
<p>By default the pipe can hold up to 16KB before it gets | |||||
filled up.</p> | |||||
<p>The kernel dedicates part of its virtual address space to | |||||
hold pipe buffers | |||||
and allocates/deallocates physical pages as pipes get | |||||
created/destroyed. | |||||
This induces TLB invalidation requests to other CPUs, | |||||
which causes an | |||||
unnecessary slowdown.</p> | |||||
<p>An easy way out is to cache a certain number of buffers.</p> | |||||
</body> | |||||
<sponsor> | |||||
The FreeBSD Foundation | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>String functions on the amd64 architecture</title> | |||||
<contact> | |||||
<person> | |||||
<name>Mateusz Guzik</name> | |||||
<email>mjg@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>Functions like memset, memmove and memcpy are very | |||||
Done Inline ActionsThe "is" here is superfluous. bcr: The "is" here is superfluous. | |||||
frequently used by virtually | |||||
all programs. They can be optimized in various ways, but | |||||
FreeBSD uses very | |||||
rudimentary implementations using rep movsq/stosq. rep | |||||
prefix has high startup | |||||
latency which is overly expensive when dealing with small | |||||
sizes.</p> | |||||
<p>Short term goal of this project is to implement faster | |||||
variants for the kernel | |||||
and import them into libc. The main speed up comes from | |||||
not using rep for small | |||||
sizes (< 256) and from aligning target buffers to 16 | |||||
bytes when rep is used. | |||||
On top of that runtime detection of the Enhanced REP | |||||
MOVSB/STOSB extention can | |||||
be used to only use rep movsb/stosb.</p> | |||||
<p>Mid term goal extends userspace. SIMD extensions can be | |||||
used to make these functions | |||||
faster. They can't easily be used in the kernel: SIMD | |||||
registers are not saved on | |||||
transitions user<->kernel for performance reasons. | |||||
Thus any use would have to | |||||
take care of saving these registers, which can consume any | |||||
advantage from using them in | |||||
the first place. This is not a concern for userspace code.</p> | |||||
<p>There is a BSD-licensed implementation in bionic: | |||||
<a | |||||
href="https://android.googlesource.com/platform/bionic/+/master/libc/arch-x86_64/string/">https://android.googlesource.com/platform/bionic/+/master/libc/arch-x86_64/string/</a></p> | |||||
<p>which with some modifications can be used in libc later | |||||
on.</p> | |||||
<p>See the Intel Optimization Manual for reference: | |||||
<a | |||||
href="https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-manual.pdf">https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-manual.pdf</a></p> | |||||
</body> | |||||
<sponsor> | |||||
The FreeBSD Foundation | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>32-bit compatibility and other ABI cleanups</title> | |||||
<contact> | |||||
<person> | |||||
<name>Brooks Davis</name> | |||||
<email>brooks@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>As part of maintaining an external ABI (application binary | |||||
interface) | |||||
compatibility layer, I've been improving FreeBSD | |||||
infrastructure, | |||||
primarily the 32-bit compatibility layer. One of FreeBSD's | |||||
strengths is | |||||
that we can easily support many ABIs. This includes | |||||
support for a.out | |||||
format executables (vs the standard ELF), support for i386 | |||||
on amd64, the | |||||
Linux emulation layer, etc.</p> | |||||
<p>This infrastructure has existed for decades and not every | |||||
design | |||||
decision has stood the test of time. Support has also been | |||||
incomplete | |||||
in a number of areas (e.g. network management under 32-bit | |||||
emulation).</p> | |||||
<p>Committed improvements include:</p> | |||||
<ul> | |||||
<li>Improved <tt>ioctl</tt> and <tt>sysctl</tt> support to | |||||
allow <tt>ifconfig</tt> and | |||||
<tt>netstat</tt> to work in 32-bit compat mode.</li> | |||||
</ul> | |||||
<ul> | |||||
<li>Migration from a model of translating <tt>ioctl</tt> | |||||
commands and data | |||||
structures at the kernel boundary to translating where the | |||||
commands | |||||
are processed. This is a correctness improvement (`ioctl` | |||||
commands | |||||
do not have meaning outside the specific file descriptor | |||||
in question) | |||||
and improves code reusability (my out-of-tree work will | |||||
soon include | |||||
a 64-bit compatibility layer.)</li> | |||||
</ul> | |||||
<ul> | |||||
<li>Simplifications of the generic ELF process execution path | |||||
by Ed | |||||
Maste, John Baldwin, and myself. A number of | |||||
simplifications including | |||||
minor speedups have been committed.</li> | |||||
</ul> | |||||
<p> | |||||
Portions of this work were developed by SRI International | |||||
and the | |||||
University of Cambridge Computer Laboratory (Department of | |||||
Computer | |||||
Science and Technology) under DARPA/AFRL contract | |||||
FA8750-10-C-0237 | |||||
("CTSRD"), as part of the DARPA CRASH research programme | |||||
and under DARPA | |||||
contract HR0011-18-C-0016 ("ECATS"), as part of the DARPA | |||||
SSITH research | |||||
programme.</p> | |||||
<p>Work in progress includes cleanups to the APIs used by the | |||||
kernel when | |||||
creating processes and continued <tt>ioctl</tt> | |||||
improvements. Work is also | |||||
underway to generate the 32-bit system call list from the | |||||
"default" | |||||
list.</p> | |||||
<p>The remaining <tt>ioctl</tt> commands handled in | |||||
<a | |||||
href="https://svnweb.FreeBSD.org/base/head/sys/compat/freebsd32/freebsd32_ioctl.c?view=log">sys/compat/freebsd32/freebsd32_ioctl.c</a> | |||||
need to be migrated to the point of implementation. Help | |||||
with the latter | |||||
would be appreciated.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>Save/Restore/Migration support in bhyve</title> | |||||
<contact> | |||||
<person> | |||||
<name>Elena Mihailescu</name> | |||||
<email>elenamihailescu22@gmail.com</email> | |||||
</person> | |||||
<person> | |||||
<name>Darius Mihai</name> | |||||
<email>dariusmihaim@gmail.com</email> | |||||
</person> | |||||
<person> | |||||
<name>Sergiu Weisz</name> | |||||
<email>sergiu121@gmail.com</email> | |||||
</person> | |||||
<person> | |||||
<name>Mihai Carabas</name> | |||||
Done Inline Actionss/Works/Work/ bcr: s/Works/Work/ | |||||
<email>mihai@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://github.com/FreeBSD-UPB/freebsd/tree/projects/bhyve_migration">Github repository for the save/restore and migration features</url> | |||||
<url href="https://github.com/FreeBSD-UPB/freebsd/wiki/Save-and-Restore-a-virtual-machine-using-bhyve">Github wiki - How to Save and Restore a bhyve guest</url> | |||||
<url href="https://github.com/FreeBSD-UPB/freebsd/wiki/Virtual-Machine-Migration-using-bhyve">Github wiki - How to Migrate a bhyve guest</url> | |||||
<url href="https://github.com/FreeBSD-UPB/freebsd/wiki/Suspend-Resume-test-matrix">Github wiki - Suspend/resume test matrix</url> | |||||
</links> | |||||
<body> | |||||
<p>The Save/Restore feature is a facility to suspend and | |||||
resume guest | |||||
virtual images that has been added to the FreeBSD/amd64's | |||||
hypervisor, | |||||
bhyve. The bhyvectl tool is used to save the guest virtual | |||||
machine | |||||
into three files:</p> | |||||
<ul> | |||||
<li>a file for the guest memory</li> | |||||
<li>a file for state of each device / CPU state</li> | |||||
<li>a file that has metadata that is used in the restore | |||||
process</li> | |||||
</ul> | |||||
<p> | |||||
To suspend a bhyve guest, the bhyvectl tool must be run | |||||
with the <tt>--suspend | |||||
<state_file_name></tt> | |||||
option followed by the guest name.</p> | |||||
<p>To restore a bhyve guest from a checkpoint, one simply has | |||||
to add the <tt>-r</tt> option | |||||
followed by the main state file (the same file that was | |||||
given to the <tt>--suspend</tt> | |||||
option for bhyvectl) when starting the VM.</p> | |||||
<p>The Migration feature uses the Save/Restore implementation | |||||
to migrate a bhyve guest | |||||
from one FreeBSD host to another FreeBSD host. To migrate | |||||
a bhyve guest, | |||||
one needs to start an empty guest on the destination host | |||||
from a shared guest | |||||
image using the bhyve tool with the <tt>-R</tt> option | |||||
followed by the source host | |||||
IP and the port to listen to migration request. On the | |||||
source host, the | |||||
migration is started by executing the bhyvectl command | |||||
with the <tt>--migrate</tt> | |||||
option, followed by the destination host IP and the port | |||||
to send to the messages.</p> | |||||
<p>New features added:</p> | |||||
<ul> | |||||
<li>Create the socket connection between source and | |||||
destination hosts</li> | |||||
<li>Migrate the guest state via sockets</li> | |||||
<li>Separate the suspend/resume/migration code from the | |||||
bhyverun.c and bhyvectl.c and added two new files | |||||
for them: migration.c and migration.h</li> | |||||
<li>Added save/restore state for xhci</li> | |||||
<li>Added save/restore state for fbuf</li> | |||||
<li>Fix vhpet restore state issues (timers related)</li> | |||||
<li>Add partially support for suspending and resuming a Linux | |||||
guest</li> | |||||
</ul> | |||||
<p>Future tasks:</p> | |||||
<ul> | |||||
<li>Check if live migration can be implemented using the | |||||
FreeBSD's Copy-on-Write mechanism</li> | |||||
<li>Add live migration support by using EPT (Intel)</li> | |||||
<li>Add live migration support by using NPT (AMD)</li> | |||||
<li>Add suspend/resume support for nvme</li> | |||||
<li>Add suspend/resume support for virtio-console</li> | |||||
<li>Add suspend/resume support for virtio-scsi</li> | |||||
<li>Fix restore timers issues</li> | |||||
<li>Fix suspending bhyve - threads issues</li> | |||||
<li>Fix suspending bhyve - mutexes issues</li> | |||||
<li>Add suspend/resume support for Windows guests</li> | |||||
</ul> | |||||
</body> | |||||
<sponsor> | |||||
Matthew Grooms | |||||
</sponsor> | |||||
<sponsor> | |||||
iXsystems | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>Building FreeBSD on non-FreeBSD hosts</title> | |||||
<contact> | |||||
<person> | |||||
<name>Alex Richardson</name> | |||||
<email>arichardson@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://wiki.FreeBSD.org/BuildingOnNonFreeBSD">Wiki</url> | |||||
<url href="https://github.com/arichardson/freebsd/tree/crossbuild-aug2018">GitHub project</url> | |||||
</links> | |||||
<body> | |||||
<p>Currently FreeBSD can only be built on a FreeBSD host. | |||||
However, most free | |||||
CI tools only allow building on Linux or macOS and | |||||
therefore can not be used | |||||
for building the FreeBSD base system. It is sometimes | |||||
useful to | |||||
cross-build FreeBSD for a remote machine or an emulator | |||||
even if the build | |||||
machine is not running FreeBSD. | |||||
The goal of this project is to allow building FreeBSD on | |||||
both Linux and macOS hosts | |||||
and in the future it may be extended to allow compiling on | |||||
a Windows host. | |||||
This work originates from the CHERI project and was | |||||
motivated by multiple cases of | |||||
people wanting to try out CheriBSD but not being able to | |||||
compile it since they did | |||||
not have a FreeBSD system available for compiling. | |||||
Once completed this project will also allow developers to | |||||
contribute to FreeBSD | |||||
even if they don't have access to a FreeBSD build system.</p> | |||||
<p>The current set of patches for this project can be found | |||||
on | |||||
<a | |||||
href="https://github.com/arichardson/freebsd/tree/crossbuild-aug2018">GitHub</a>. | |||||
With the current prototype it is possible to compile both | |||||
world and kernel for | |||||
architectures that use the clang compiler and for MIPS64, | |||||
which uses gcc. However, some options such as | |||||
LOCALES are | |||||
not supported yet and require further changes before the | |||||
bootstrap tools can be built | |||||
on Linux/macOS.</p> | |||||
<p>Some changes required for building on non-FreeBSD have | |||||
already been merged to | |||||
HEAD but there are still a rather large number of changes | |||||
that need review.</p> | |||||
<p>If you are interested in getting this into HEAD and would | |||||
like to help, please | |||||
try the current prototype and report any issues to | |||||
arichardson@FreeBSD.org. | |||||
If you can help with reviewing the changes please contact | |||||
arichardson@FreeBSD.org | |||||
to be added to any pending Phabricator reviews.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>ENA FreeBSD Driver Update</title> | |||||
<contact> | |||||
<person> | |||||
<name>Michał Krawczyk</name> | |||||
<email>mk@semihalf.com</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README">ENA README</url> | |||||
</links> | |||||
<body> | |||||
<p>ENA (Elastic Network Adapter) is the smart NIC which is | |||||
used in the virtualised | |||||
environment of Amazon Web Services (AWS). It supports | |||||
multiple queues and can handle up to 25 Gb/s, | |||||
depending on the instance type on which it is | |||||
used.</p> | |||||
<p>Since last report, ENA versions v0.8.0 and v0.8.1 have | |||||
been released, which introduced | |||||
many bug fixes, new features, optimization, stability and | |||||
error recovery | |||||
improvements. The last is especially important on the AWS, | |||||
where the instances | |||||
have to be reliable as they may be running very sensitive | |||||
functions and the | |||||
down time of the VM should be reduced to minimum.</p> | |||||
<p>The v0.8.0 and v0.8.1 release patches included:</p> | |||||
<ul> | |||||
<li>Upgrade of the HAL to version v1.1.4.3</li> | |||||
<li>Improvement to the reset routine - the driver is now | |||||
triggering reset from | |||||
more fault points and is passing the reset reason to the | |||||
device, which can | |||||
perform the reset adequately to the encountered error.</li> | |||||
<li>Device statistics (like global Tx and Rx counters) are no | |||||
longer read directly from the device. | |||||
The only exception is Rx drops, which are still read using | |||||
the AENQ | |||||
descriptor.</li> | |||||
<li>The RX Out Of Order completion feature was added, which | |||||
enabled to cleanup the | |||||
RX descriptors out of order by keeping trace of all free | |||||
descriptors.</li> | |||||
<li>RX ring is now being monitored, to prevent the ring from | |||||
stalling.</li> | |||||
<li>Error handling paths were reworked and fixed.</li> | |||||
<li>Driver was covered with branch prediction statements, to | |||||
make the most | |||||
of this CPU feature in the hot paths.</li> | |||||
<li>Fix handling of the DF flag in the IP packets.</li> | |||||
<li>Add dynamic logging and reduce number of messages being | |||||
printed by the driver.</li> | |||||
<li>MTU configuration now is being verified using the device | |||||
capabilities instead | |||||
of a constant value.</li> | |||||
<li>Do not pass packet header length hint to the device, | |||||
because for the chained | |||||
mbufs it may be problematic to determine header length, if | |||||
the header is split | |||||
into multiple segments.</li> | |||||
</ul> | |||||
</body> | |||||
<sponsor> | |||||
Amazon.com Inc. | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>FreeBSD Graphics Team</title> | |||||
<contact> | |||||
<person> | |||||
<name>FreeBSD Graphics Team</name> | |||||
<email>x11@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Niclas Zeising</name> | |||||
<email>zeising@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://github.com/FreeBSDDesktop">Project GitHub page</url> | |||||
</links> | |||||
<body> | |||||
<p>The FreeBSD X11/Graphics team is responsible for the lower | |||||
levels of the FreeBSD | |||||
graphics stack. This includes graphics drivers, graphics | |||||
libraries such as the | |||||
MESA OpenGL implementation, the X.org xserver with related | |||||
libraries and | |||||
applications, and Wayland with related libraries and | |||||
applications.</p> | |||||
<p>There have been a lot of changes since the last report. | |||||
The most important one | |||||
is the change of driver distribution and updates. On | |||||
FreeBSD 11.2 and later | |||||
modern graphics drivers using the Linux KPI subsystem are | |||||
found in ports. These | |||||
give much improved support for Intel and AMD graphics | |||||
hardware, however, they | |||||
are currently only available for amd64.</p> | |||||
<p>The legacy drivers available in the FreeBSD base system | |||||
are also available in | |||||
the ports tree, since they cause issues with the new | |||||
drivers. They will remain | |||||
in tree for 11.2 and 12, but work is on-going to have them | |||||
removed for 13, | |||||
except for arm.</p> | |||||
<p>The easiest way to install the new drivers is to install | |||||
graphics/drm-kmod which | |||||
will install the correct driver depending on your | |||||
architecture and FreeBSD | |||||
version.</p> | |||||
<p>There have been changes to the ports as well. Most notably | |||||
is the changed | |||||
handling of OpenGL dependencies, which has moved to USES | |||||
instead of being | |||||
handled directly in bsd.port.mk. Other big infrastructure | |||||
changes is the move | |||||
from individual \*proto packages to xorgproto, and turning | |||||
that into a build | |||||
time dependency. Many thanks to portmgr for help with | |||||
exp-runs for these | |||||
changes.</p> | |||||
<p>There have been updates to applications and libraries as | |||||
needed.</p> | |||||
<p>On the project management side, there is ongoing work to | |||||
set up a more efficient | |||||
way of working, including bi-weekly conference calls to | |||||
discuss the current | |||||
works in progress. Notes from these conference calls will | |||||
be posted on the | |||||
mailing list.</p> | |||||
<p>Looking forward, the current major work in progress is to | |||||
update the graphics | |||||
driver to be on par with Linux 4.17. The code is merged, | |||||
but patching and bug | |||||
fixing is ongoing.</p> | |||||
<p>There is also work to port the VMware guest graphics | |||||
driver, vmwgfx, to FreeBSD | |||||
and to the Linux KPI, to get better graphics support in | |||||
VMware.</p> | |||||
<p>Lastly, on the driver side is to get the new graphics | |||||
drivers to work on i386 as | |||||
well. Experimental support for this exists in the code | |||||
repository, but is not | |||||
yet merged to the FreeBSD ports tree.</p> | |||||
<p>In userland, the biggest things happening is the update of | |||||
the input stack, | |||||
including libinput and supporting libraries.</p> | |||||
<p>Work is also ongoing on updating MESA libraries.</p> | |||||
<p>On the project management side, the most important tasks | |||||
is to continue to work | |||||
on the team, and how we work internally.</p> | |||||
<p>We are also working on setting up a list of requirements | |||||
for testing, so that we | |||||
can be reasonably assured that updates won't cause | |||||
regressions.</p> | |||||
<p>People who are interested in helping out can find us on | |||||
the x11@FreeBSD.org | |||||
mailing list, or on our Gitter chat: <a | |||||
href="https://gitter.im/FreeBSDDesktop/Lobby">https://gitter.im/FreeBSDDesktop/Lobby</a>. | |||||
We | |||||
are also available in #freebsd-xorg on EFNet.</p> | |||||
<p>We also have a team area on GitHub where our work | |||||
repositories can be found: | |||||
<a | |||||
href="https://github.com/FreeBSDDesktop">https://github.com/FreeBSDDesktop</a></p> | |||||
</body> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>ifuncs</title> | |||||
<contact> | |||||
<person> | |||||
<name>Konstantin Belousov</name> | |||||
<email>kib@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Ed Maste</name> | |||||
<email>emaste@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Mark Johnston</name> | |||||
<email>markj@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Mateusz Guzik</name> | |||||
<email>mjg@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>An ifunc is a special construct in an ELF object, which | |||||
allows for | |||||
the selection of the implementation for the given symbol | |||||
at runtime, | |||||
when the ELF module gets the final relocations applied. | |||||
The selection | |||||
of which code to use is governed by the small piece of | |||||
user provided | |||||
code, attached to the symbol, the so called resolver | |||||
function. | |||||
Ifuncs provide a convenient way to select between | |||||
different | |||||
machine-specific implementations of the parts of the code, | |||||
without | |||||
the ugliness and unsafety of the alternative approach, | |||||
which is | |||||
runtime patching.</p> | |||||
<p>Ifuncs require support both from the static linker ld(1), | |||||
and from the | |||||
runtime linker for the corresponding execution | |||||
environment. On | |||||
FreeBSD, with the switch from the ancient GPLv2 licensed | |||||
BFD-based | |||||
ld(1) to either in-tree LLD or external modern BFD ld, the | |||||
use of | |||||
ifuncs become possible. Runtime linkers for ifunc support | |||||
exists for | |||||
the following environments:</p> | |||||
<ul> | |||||
<li>i386, amd64, and arm64 kernels</li> | |||||
<li>usermode dynamic linker ld-elf.so.1 on i386 and amd64</li> | |||||
<li>static binaries startup code for i386 and amd64</li> | |||||
</ul> | |||||
<p> | |||||
The use of ifuncs were previously applied for optimization | |||||
of the | |||||
following areas of the amd64 kernel:</p> | |||||
<ul> | |||||
<li>context switching code, instead of huge number of runtime | |||||
checks | |||||
(PTI vs non-PTI, PCID or not, is INVPCID instruction | |||||
supported for | |||||
PCID) now uses set of mode-specific routines, see | |||||
pmap_activate_sw(). Besides removing checks at runtime, it | |||||
also | |||||
makes the code much more cleanly structured and readable.</li> | |||||
</ul> | |||||
<ul> | |||||
<li>TLB and cache flush implementation.</li> | |||||
</ul> | |||||
<ul> | |||||
<li>memcpy/memmove, copyin/copyout variants selection for ERMS | |||||
and SMAP.</li> | |||||
</ul> | |||||
<ul> | |||||
<li>FPU state save and restore, depending on the support for | |||||
AVX or not, | |||||
this is also used on i386.</li> | |||||
</ul> | |||||
<p> | |||||
For amd64 userspace, we currently use ifunc for | |||||
optimization of the | |||||
architecture dependent TLS base set and get functions.</p> | |||||
</body> | |||||
<sponsor> | |||||
The FreeBSD Foundation | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>Intel Work on Core Enabling and Security</title> | |||||
<contact> | |||||
<person> | |||||
<name>Ben Widawsky</name> | |||||
<email>bwidawsk@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>A new team has been formed within Intel to help with side | |||||
channel security | |||||
mitigations as well as core enabling. They are evaluating | |||||
work from all areas | |||||
except networking. The team is currently focusing on two | |||||
areas:</p> | |||||
<ol><li>Power Management improvements</li> | |||||
<li>NVDIMM namespace support</li></ol> | |||||
<p>The ultimate goal of the power management work is to get | |||||
runtime power | |||||
management to hit "opportunistic idle". What this means is | |||||
when devices are | |||||
idle, the OS will power them down, and when everything | |||||
goes idle certain SoCs | |||||
will allow you to hit very low power states across the | |||||
platform. FreeBSD | |||||
currently doesn't have any notion of runtime power | |||||
management, and many devices | |||||
don't properly implement suspend and resume. In addition, | |||||
some preliminary work | |||||
is in process as it was thought to help when eventually | |||||
enabling opportunistic | |||||
idle. That preliminary work has been happening and is now | |||||
up for review:</p> | |||||
<ul> | |||||
<li><a | |||||
href="https://reviews.FreeBSD.org/D17675">https://reviews.FreeBSD.org/D17675</a></li> | |||||
<li><a | |||||
href="https://reviews.FreeBSD.org/D17676">https://reviews.FreeBSD.org/D17676</a></li> | |||||
</ul> | |||||
<p> | |||||
NVDIMM namespace support has also been put up for review. | |||||
ACPI spec defines | |||||
namespaces as a way of partitioning the device into | |||||
separate labels. The current | |||||
work will integrate with geom(4). How these are used is | |||||
application dependent. | |||||
This work is up for review as well: <a | |||||
href="https://reviews.FreeBSD.org/D17619">https://reviews.FreeBSD.org/D17619</a></p> | |||||
<p>The team has additionally taken on smaller tasks like | |||||
porting turbostat(8), | |||||
working on git svn init scripts, some small modifications | |||||
to acpi tooling, and | |||||
an effort to create a port PMDK.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>LLVM 7.0 - Sanitizers support improvements / Static code analysis</title> | |||||
<contact> | |||||
<person> | |||||
<name>David Carlier</name> | |||||
<email>devnexen@gmail.com</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="http://releases.llvm.org/7.0.0/docs/ReleaseNotes.html">Release notes</url> | |||||
</links> | |||||
<body> | |||||
<p>In order to increase the FreeBSD tooling to uncover | |||||
code bugs in the userland, further compiler-rt components | |||||
support and static code analysis improvements had been | |||||
added | |||||
since the last 6.0 version.</p> | |||||
<p>Starting with the sanitizers, Memory Sanitizer (for amd64) | |||||
mainly to | |||||
detect unitialized pointers. There is also a simple W^X | |||||
paging | |||||
requests detection available from most of sanitizers.</p> | |||||
<p>Also libFuzzer support finally had been possible. | |||||
It allows code to be tested with random values from corpus | |||||
inputs. | |||||
Mutation and combination algorithms of those random inputs | |||||
can be overwritten. Can also be used in addition to ubsan, | |||||
asan, msan and so on.</p> | |||||
<p>At last, the X-Ray instrumentation feature is also | |||||
supported. | |||||
It is mainly about performance profiling purposes for a | |||||
reasonable performance runtime cost.</p> | |||||
<p>In the static code analysis department, reliable strlcpy | |||||
(unfortunately strlcat | |||||
did not get merged in due time for the release) wrong | |||||
usage | |||||
cases are now covered and W^X code detection tooling had | |||||
been added.</p> | |||||
<p>At the moment, this 7.0 version is imported by Dimitry | |||||
Andric, within | |||||
its own git branch available only for FreeBSD after 12 | |||||
releases.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>Boot Loader</title> | |||||
<contact> | |||||
<person> | |||||
<name>Warner Losh</name> | |||||
<email>imp@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Kyle Evans</name> | |||||
<email>kevans@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Toomas Soome</name> | |||||
<email>tsoome@Freebsd.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>The FreeBSD boot loader lives in src/stand (prior releases | |||||
had it in | |||||
sys/boot and lib/libstand). It covers all the code that | |||||
the project | |||||
provides that interacts with the hardware before the | |||||
kernel starts.</p> | |||||
<p>The LUA interpreter added earlier in 2018 was made default | |||||
in 2018Q3. | |||||
Due to undiagnosed booting issues, the LUA interpreter has | |||||
been | |||||
disabled on sparc64 and all powerpc. The LUA interpreter | |||||
is scheduled | |||||
to replace the FORTH interpreter entirely in FreeBSD 13, | |||||
although the | |||||
FORTH interpreter will remain available as a build option | |||||
in FreeBSD | |||||
12. The plans are not to remove the FORTH loader for about | |||||
a year | |||||
after 12.0 release, or approximately January 2020. | |||||
Platforms not | |||||
currently working with the LUA interpreter | |||||
have until that date to resolve the issues.</p> | |||||
<p>At this point, the LUA scripts implement everything that | |||||
the FORTH scripts | |||||
did. Where there was ambiguity in the spec, or where the | |||||
FORTH scripts | |||||
were more forgiving than was strictly documented, every | |||||
effort has | |||||
been made to improve the documentation and follow the old | |||||
FORTH | |||||
behavior, or document the new behavior where the old | |||||
behavior was | |||||
clearly a bug.</p> | |||||
<p>It's anticipated that no further changes to the FORTH | |||||
loader or the | |||||
FORTH scripts will happen. They are quite mature and | |||||
bullet proof at | |||||
this point and it's unlikely that an undiscovered bug will | |||||
need to be | |||||
fixed before retirement.</p> | |||||
<p>Other work in progress includes Toomas Soome's port to | |||||
OpenIndiana. In | |||||
addition to porting, he's enhanced the code in a number of | |||||
ways (both | |||||
in the block layer, and UEFI). Many of his improvements | |||||
have been | |||||
committed to FreeBSD, though a few remain and hopefully | |||||
will be | |||||
entering the tree soon after the freeze lifts.</p> | |||||
<p>UEFI booting has been greatly enhanced. There are still | |||||
some | |||||
machines that have issues with the default BootXXXX | |||||
variables or | |||||
something else in the environment that are being | |||||
investigated. We hope | |||||
to understand the problems well enough to provide a fix | |||||
for FreeBSD | |||||
12.0.</p> | |||||
<p>Ian Lepore has reworked the GELI support so that it is | |||||
architecture independent and can be | |||||
used on any architecture we support.</p> | |||||
<p>There are also efforts underway to support booting signed | |||||
images, improved | |||||
crypto booting options, and implement Multiboot 2.0 | |||||
support.</p> | |||||
</body> | |||||
</project> | |||||
Done Inline Actionsmore forgiving than was allanjude: more forgiving **than** was | |||||
<project cat='proj'> | |||||
<title>Usermode mapping of PCI BARs</title> | |||||
<contact> | |||||
<person> | |||||
<name>Konstantin Belousov</name> | |||||
<email>kib@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>Modern PCI(e) devices typically define memory-mapped BARs | |||||
(Base Address Registers) to make a memory region available | |||||
to the device. | |||||
Each BAR | |||||
has a separate page-aligned boundary and memory region | |||||
associated with it. | |||||
This is enforced by the need | |||||
of hypervisors to provide the pass-through using VT-d, | |||||
which operates | |||||
with memory and has the granularity of one page for access | |||||
control. | |||||
As is, it | |||||
also means that the BARs have a suitable configuration for | |||||
providing | |||||
access to usermode, controlling access by the normal page | |||||
tables.</p> | |||||
<p>Linux already gives a way for userspace mapping of BARs | |||||
using sysfs.</p> | |||||
<p>Of course, if a userspace program has enough privileges, | |||||
it can read a BAR, | |||||
determine the physical address of the mapping as seen by | |||||
CPU, and use | |||||
Done Inline ActionsI might be worth expanding MI or even saying 'architecture independent' allanjude: I might be worth expanding MI or even saying 'architecture independent' | |||||
mem(4) (aka /dev/mem) to mmap() that region of memory. | |||||
This is very cumbersome, and leaves many unresolved | |||||
issues. | |||||
For example, a BAR might be not activated, which would | |||||
require | |||||
involvement of the IOMMU on some architectures. Also this | |||||
rude approach | |||||
makes it very hard to create mappings with the correct | |||||
caching | |||||
attributes.</p> | |||||
<p>The FreeBSD pci(4) driver was enhanced to support such | |||||
mappings, and pciconf(8) utility was extended to use the | |||||
new support. | |||||
See pci(4) | |||||
for PCIOCBARMMAP ioctl(2) request description for details, | |||||
and | |||||
pciconf(8) for the -D switch.</p> | |||||
<p>TODO: automatically activate the BAR on mapping, this is | |||||
not done yet. | |||||
There is a problem with avoiding the resource conflicts on | |||||
possible future attachmens of the kernel driver.</p> | |||||
</body> | |||||
<sponsor> | |||||
The FreeBSD Foundation | |||||
</sponsor> | |||||
<sponsor> | |||||
Mellanox Technologies | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>Device Mode USB</title> | |||||
<contact> | |||||
<person> | |||||
<name>Edward Tomasz Napierala</name> | |||||
<email>trasz@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/usb-device-mode.html">Handbook chapter</url> | |||||
</links> | |||||
<body> | |||||
<p>Many embedded boards include hardware which supports | |||||
device | |||||
side USB - the ability for the board to present itself to | |||||
another | |||||
system as a USB drive, network adapter, or a virtual | |||||
serial port. | |||||
The FreeBSD USB stack has supported this functionality for | |||||
quite some | |||||
time, but it has not been used to its fullest extent.</p> | |||||
<p>The goal of this project was to fix that - to document the | |||||
functionality, possibly fix some bugs, and to make it easy | |||||
to use, automating it as much as possible.</p> | |||||
<p>Starting with FreeBSD 12.0, this functionality is enabled | |||||
out of the box. This means you can connect your BeagleBone | |||||
Black's (using its USB client socket) or a Raspberry Pi 0 | |||||
(using the On-The-Go (OTG) port) to your laptop, and | |||||
you'll get a virtual | |||||
USB serial port, which serves as a system console, with | |||||
getty(8) | |||||
waiting for you to log in. This means you no longer need | |||||
to | |||||
look for a keyboard and a screen, or mess with the console | |||||
cables just to configure your system. You can also switch | |||||
it to provide network interface, or present itself as a | |||||
USB | |||||
drive - it's all documented in the FreeBSD Handbook.</p> | |||||
</body> | |||||
<sponsor> | |||||
The FreeBSD Foundation | |||||
</sponsor> | |||||
</project> | |||||
<project cat='proj'> | |||||
<title>Performance improvements</title> | |||||
<contact> | |||||
<person> | |||||
<name>Matthew Macy</name> | |||||
<email>mmacy@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>FreeBSD 12 saw the introduction of a number of performance | |||||
improvements:</p> | |||||
<ul> | |||||
<li>The introduction of the new synchronization primitive | |||||
epoch(9) to | |||||
replace the use of reader locks for providing existence | |||||
guarantees | |||||
for data structures.</li> | |||||
<li>epoch(9) was used to provide an 85+% reduction in the | |||||
overhead of | |||||
pcb lookup in high core count systems.</li> | |||||
<li>epoch(9) was used to provide an 85+% reduction in UDP send | |||||
overhead | |||||
on high core count systems. See the link for a bit more | |||||
detail: | |||||
<a | |||||
href="http://scalebsd.org/blog/2018/06/16/UDP-and-epoch-for-liveness-guarantees">http://scalebsd.org/blog/2018/06/16/UDP-and-epoch-for-liveness-guarantees</a></li> | |||||
<li>System call overhead is now half that of FreeBSD 11.</li> | |||||
<li>UNIX sockets now scale near linearly (previously maxed out | |||||
at 3-4 threads).</li> | |||||
<li>The NUMA work has lead to a 20x-80x improvement in the | |||||
scalability | |||||
of page fault handling.</li> | |||||
</ul> | |||||
</body> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>Armada 38x FreeBSD support</title> | |||||
<contact> | |||||
<person> | |||||
<name>Marcin Wojtas</name> | |||||
<email>mw@semihalf.com</email> | |||||
</person> | |||||
<person> | |||||
<name>Patryk Duda</name> | |||||
<email>pdk@semihalf.com</email> | |||||
</person> | |||||
<person> | |||||
<name>Rafał Kozik</name> | |||||
<email>rk@semihalf.com</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://www.marvell.com/documents/egrkpyqzpoebxblyeept/">PRODUCT BRIEF</url> | |||||
</links> | |||||
<body> | |||||
<p>The Marvell Armada 38x is a very poplular ARMv7-based dual | |||||
core SoC. | |||||
Thanks to the multiple low and high speed interfaces | |||||
the platform is used in a wide range of products, such | |||||
as Network-Attached Storage (NAS), Wi-Fi Access Point | |||||
(WAP) and others.</p> | |||||
<p>Since last report, remaining Armada 38x support was | |||||
integrated to HEAD, which can now compile with the | |||||
armv7 | |||||
GENERIC config and use unmodified sys/gnu/dts device | |||||
trees. The details are as follows:</p> | |||||
<ul> | |||||
<li>GENERIC config</li> | |||||
<li>Introduce a vast rework of the sys/arm/mv directory for | |||||
arm and armv7 platforms.</li> | |||||
<li>Enable PLATFORM support for Marvell ARMv7 SoCs, which can | |||||
now can boot with GENERIC kernel.</li> | |||||
<li>Base on dynamic detection of SoC type and device tree | |||||
instead of using ifdefs | |||||
and enable more flexible environment for maintaining | |||||
Marvell platforms.</li> | |||||
<li>sys/gnu/dts device trees</li> | |||||
<li>Improve platform code and the drivers (e.g. CESA, PCIE, | |||||
GPIO) to properly work with original | |||||
Linux device trees.</li> | |||||
<li>GPIO</li> | |||||
<li>Add multiple fixes and improvements to the | |||||
sys/arm/mv/gpio.c</li> | |||||
<li>Rework driver to properly integrate with HEAD GPIO | |||||
frameworks (main and gpioled)</li> | |||||
<li>Enable support for both old and Linux GPIO device tree | |||||
bindings, so that multiple controllers | |||||
can be used.</li> | |||||
</ul> | |||||
</body> | |||||
<sponsor> | |||||
Stormshield | |||||
</sponsor> | |||||
<sponsor> | |||||
Semihalf | |||||
</sponsor> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>PINE64-LTS Image</title> | |||||
<contact> | |||||
<person> | |||||
<name>Emmanuel Vadot</name> | |||||
<email>manu@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>We now produce an image for the PINE64-LTS. | |||||
This image works on the PINE64-LTS and the Sopine with | |||||
Baseboard.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>PocketBeagle Support</title> | |||||
<contact> | |||||
<person> | |||||
<name>Emmanuel Vadot</name> | |||||
<email>manu@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Tom Jones</name> | |||||
<email>thj@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://www.beagleboard.org/pocket">Pocket Beagle</url> | |||||
</links> | |||||
<body> | |||||
<p>The Pocket Beagle is the latest member of the BeagleBoard | |||||
family. | |||||
Support for it was added and the Beaglebone image can be | |||||
used on it directly.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>Allwinner SoC Support</title> | |||||
<contact> | |||||
<person> | |||||
<name>Emmanuel Vadot</name> | |||||
<email>manu@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<ul> | |||||
<li>SPI driver added for A64 SoC</li> | |||||
<li>Thermal driver added/fixed for A64/H3/H5 SoCs</li> | |||||
<li>Lot of bugs where fixed in the mmc driver, stability | |||||
should be better</li> | |||||
<li>New driver for AXXP803 which is the power chip companion | |||||
of the A64 SoC</li> | |||||
<li>Add overlays to use another timer controller as the | |||||
default one in A64 if faulty | |||||
These overlay is enabled in the PINE64/LTS images by | |||||
default</li> | |||||
</ul> | |||||
</body> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>ARMv6 and ARMv7 image now use EFI loader</title> | |||||
<contact> | |||||
<person> | |||||
<name>Emmanuel Vadot</name> | |||||
<email>manu@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>Instead of using the ubldr version of the loader which | |||||
uses the U-Boot | |||||
API, all images now use loader.efi as their primary | |||||
FreeBSD loader. | |||||
This allow us to have a common boot path for all arm and | |||||
arm64 images.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>DTS Update</title> | |||||
<contact> | |||||
<person> | |||||
<name>Emmanuel Vadot</name> | |||||
<email>manu@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>DTS files (Device Tree Sources) were updated to be on par | |||||
with Linux 4.18 for | |||||
the 12.0 release.</p> | |||||
<p>The DTS are now compiled for some arm64 boards as the one | |||||
present in U-Boot are | |||||
now always up-to-date.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>RPI Firmware/DTB/U-Boot Update</title> | |||||
<contact> | |||||
<person> | |||||
<name>Emmanuel Vadot</name> | |||||
<email>manu@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>U-Boot mailing list</name> | |||||
<email>uboot@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>The RaspberryPi firmware loads the DTB from the FAT | |||||
partition based on | |||||
the model. U-Boot now uses this DTB and passes it to the | |||||
FreeBSD loader/kernel | |||||
instead of using the DTS embedded in U-Boot. | |||||
This allow the FreeBSD Kernel to use the RaspberryPi | |||||
Foundation provided DTB overlays to enable | |||||
HATs. | |||||
The overlays can be obtained by installing the | |||||
rpi-firmware package.</p> | |||||
<p>A new U-Boot port for the W variant of the RPI0 was | |||||
committed as | |||||
u-boot-rpi-0-w. Some experiments started by Edward Tomasz | |||||
Napierala | |||||
(trasz@) have shown that we could possibly produce a | |||||
generic image | |||||
for all armv6 RPI (RPI-B, RPI0 and RPI0W).</p> | |||||
</body> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>FreeBSD on PowerNV (ppc64)</title> | |||||
<contact> | |||||
<person> | |||||
<name>Patryk Duda</name> | |||||
<email>pdk@semihalf.com</email> | |||||
</person> | |||||
<person> | |||||
<name>Wojciech Macek</name> | |||||
<email>wma@FreeBSD.org</email> | |||||
</person> | |||||
Done Inline Actionss/compile/compiled/ bcr: s/compile/compiled/ | |||||
<person> | |||||
<name>Michal Stanek</name> | |||||
<email>mst@semihalf.com</email> | |||||
</person> | |||||
<person> | |||||
<name>Nathan Whitehorn</name> | |||||
<email>nw@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>Semihalf is happy to announce that FreeBSD is now running | |||||
on IBM POWER8. This project is a continuation of | |||||
work done by Nathan Whitehorn who provided basic | |||||
support for a PowerNV emulator.</p> | |||||
<p>The IBM POWER8 family of CPUs offers superior performance | |||||
compared to previous Power series. It provides | |||||
complete NUMA support with up to 192 cores in a | |||||
two socket system (up to 8 threads per core). All | |||||
IO communication is handled by integrated PCIe | |||||
interface equipped with multiple IOMMU engines.</p> | |||||
<p>The support for POWER8 system running FreeBSD in | |||||
Non-Virtualized environment contains:</p> | |||||
<ul> | |||||
<li>Generic driver for OPAL hypervisor</li> | |||||
<li>kboot loader modifications to allow Little-Endian loader | |||||
to load a Big-Endian kernel ELF</li> | |||||
<li>skiboot update for ELF-parser allowing it to understand | |||||
FreeBSD kernel file format</li> | |||||
<li>Basic support for PowerNV architecture, including modes of | |||||
operation, MMU, interrupt controller</li> | |||||
<li>SMP operation (tested with 128 CPU configuration)</li> | |||||
<li>PHB subsystem driver, including IOMMU mapping for external | |||||
buses</li> | |||||
<li>PCIe host controller driver</li> | |||||
<li>USB-3.0 XHCI driver</li> | |||||
<li>Reworked drivers to be Big-Endian compatible:</li> | |||||
<li>Chelsio cxgbe(4) 10/25G network adapter</li> | |||||
<li>NVMe SSD drive</li> | |||||
</ul> | |||||
<p> | |||||
All work has been merged into HEAD and will be included in | |||||
FreeBSD 12.0-RELEASE.</p> | |||||
<p>Sponsors: IBM, FreeBSD Foundation, QCM Technologies, | |||||
Semihalf, Limelight Networks.</p> | |||||
<p>The project is kindly initiated and supported by Limelight | |||||
Networks (Kevin Bowling).</p> | |||||
</body> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>FreeBSD on POWER9</title> | |||||
<contact> | |||||
<person> | |||||
<name>Matthew Macy</name> | |||||
<email>mmacy@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>Once Justin Hibbits largely stabilized the powerpc64 port | |||||
on the POWER9 | |||||
based Talos II I decided to procure one. I've been slowly | |||||
working towards | |||||
taking powerpc64 to parity with amd64. I've been working | |||||
in an out of tree | |||||
GitHub project - in part to eliminate the need to continue | |||||
to support the 11 | |||||
year old in tree gcc4.</p> | |||||
<p>Progress so far:</p> | |||||
<ul> | |||||
<li>Adapted lock_delay to use POWER's SMT scheduling hints | |||||
rather than | |||||
using the yield hint from an older ISA</li> | |||||
<li>Added ifunc support</li> | |||||
<li>Ported the amd64 pmap so FreeBSD can use POWER9's new | |||||
radix tree | |||||
page tables. This will give us superpages mostly for free.</li> | |||||
<li>Reduced the overhead of copyinout primitives for radix.</li> | |||||
<li>Converted the copyinout primitives to ifuncs to switch | |||||
between the | |||||
old and the new at initialization time.</li> | |||||
<li>Converted pmap to use ifuncs to eliminate the overhead of | |||||
kobj dispatch.</li> | |||||
<li>Hot patch out trap handling paths only needed by the older | |||||
hashed page | |||||
table support.</li> | |||||
</ul> | |||||
<p> | |||||
Work in Progress:</p> | |||||
<ul> | |||||
<li>NMI semantics: NMIs need to be emulated by only soft | |||||
disabling interrupts, | |||||
disabling interrupts blocks all interrupts except machine | |||||
check exceptions | |||||
and system resets.</li> | |||||
<li>Superpage support: Superpages work in the functional | |||||
simulator running SMT4 | |||||
but currently crash on real hardware due to incomplete | |||||
page walk cache / | |||||
TLB invalidation.</li> | |||||
<li>Kernel minidump - with radix MMU enabled minidump support | |||||
was a fairly | |||||
straightforward port but time needs to be spent on test / | |||||
debug.</li> | |||||
<li>EARLY_AP_STARTUP - preliminary patches exist, but this is | |||||
more work on !x86 | |||||
architectures due to IPI setup not being tied to the CPU | |||||
code.</li> | |||||
</ul> | |||||
<p> | |||||
A list of the other items needed to achieve kernel feature | |||||
parity with a | |||||
(wishful) list of milestones can be found at: | |||||
<a | |||||
href="https://github.com/POWER9BSD/freebsd/projects/1">https://github.com/POWER9BSD/freebsd/projects/1</a></p> | |||||
</body> | |||||
</project> | |||||
<project cat='arch'> | |||||
<title>FreeBSD on RISC-V</title> | |||||
<contact> | |||||
<person> | |||||
<name>Ruslan Bukin</name> | |||||
<email>br@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>FreeBSD/RISC-V has been one of the actively supported | |||||
projects during the past year.</p> | |||||
<p>On a compiler front we have upstreamed FreeBSD | |||||
OS-dependent bits for GNU toolchain. It was | |||||
updated to GCC 8.1 and Binutils 2.30. FreeBSD | |||||
packages are available.</p> | |||||
<p>FreeBSD Testsuite and required dependencies were | |||||
successfully built for RISC-V and we did a test | |||||
run: 152 tests failed out of 5186, which | |||||
demonstrates a very good result for initial run | |||||
and reveals areas to work on.</p> | |||||
<p>We have added support for compressed ISA extension to KDB | |||||
debugger and DTrace FBT provider enabling | |||||
C-compressed kernel and userland by default. The | |||||
output of disassembling instructions in KDB is | |||||
looking similar to objdump.</p> | |||||
<p>QEMU has updated to latest privilege spec allowing us to | |||||
bring up FreeBSD on it. The emulation is quite | |||||
fast: it takes one second only to boot FreeBSD to | |||||
single-user mode in QEMU: <a | |||||
href="https://www.youtube.com/watch?v=FnWpRBaWF18">https://www.youtube.com/watch?v=FnWpRBaWF18</a></p> | |||||
<p>Platform-Level Interrupt Controller (PLIC) driver was | |||||
added. Interrupt support was converted to INTRNG. | |||||
PLIC is used in QEMU for virtio network and block devices. | |||||
With these changes, a full FreeBSD distribution | |||||
can now be booted in QEMU.</p> | |||||
<p>Network virtualization support (VIMAGE) was fixed and | |||||
enabled by default now.</p> | |||||
<p>In order to support RocketChip and derivatives we had to | |||||
work on A(accessed), D(dirty) PTE (page table | |||||
entry) bits management. | |||||
We have successfully tested this on a lowRISC board and it | |||||
is booting to multiuser just fine. lowRISC UART | |||||
driver was added.</p> | |||||
<p>Superuser-User-Modify (SUM) bit in status register is now | |||||
used: kernel can access userspace only within | |||||
certain functions that explicitly handle crossing | |||||
user/kernel boundary.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='ports'> | |||||
<title>KDE on FreeBSD</title> | |||||
<contact> | |||||
<person> | |||||
<name>Adriaan de Groot</name> | |||||
<email>adridg@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Tobias C. Berner</name> | |||||
<email>tcberner@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://FreeBSD.kde.org/">KDE FreeBSD</url> | |||||
</links> | |||||
<body> | |||||
<p>KDE FreeBSD is responsible for the ports of the Plasma5 | |||||
and KDE4 desktops, and | |||||
all associated applications. Further we also manage the | |||||
Qt4 and Qt5 ports, as | |||||
well as CMake.</p> | |||||
<p>We also care for the FreeBSD builders for KDE's upstream | |||||
CI on build.kde.org.</p> | |||||
<p>Since the last status report a lot of things have changed. | |||||
First and foremost, | |||||
the Plasma5 Desktop and the Qt5 based KDE Applications | |||||
have finally made their | |||||
way into the official ports tree after lingering for | |||||
multiple years in our | |||||
development repository.</p> | |||||
<p>Secondly KDE4 has been marked deprecated and will be | |||||
removed at the end of the | |||||
year. With Qt4 following no later than the next year (due | |||||
to the exponentially | |||||
increasing burden of maintenance).</p> | |||||
<p>On a more technical side, bsd.qt.mk has been replaced by | |||||
qt.mk and qt-dist.mk. | |||||
The porter's handbook is being updated (with thanks to | |||||
Tobias Kortkamp).</p> | |||||
<p>Further we have been keeping CMake and Qt5 and almost | |||||
every other port under our | |||||
control up to date. SDDM has been updated to the | |||||
next-to-latest release with | |||||
backported security fixes.</p> | |||||
<p>One big issue we have is www/qt5-webengine, which requires | |||||
too much time to keep | |||||
up to date, as the underlying chromium is in need of many | |||||
patches, which change | |||||
with every release. Another upcoming issue is the way in | |||||
which FreeBSD's libinput | |||||
lags behind. This blocks future updates to KDE Plasma as | |||||
well as Wayland | |||||
improvements. Thankfully x11@ is looking at this issue | |||||
already, so it should be | |||||
fixed soon -- for the meantime people who want to give the | |||||
latest KDE Plasma | |||||
Desktop a try can use the appropriate branch from our | |||||
GitHub.</p> | |||||
<p>People who are willing to contribute can find us on | |||||
#kde-freebsd on freenode, | |||||
the kde@FreeBSD.org mailing list. Further, we accept | |||||
pull-requests and | |||||
contributions on github.com/freebsd/freebsd-ports-kde.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='ports'> | |||||
<title>Puppet</title> | |||||
<contact> | |||||
<person> | |||||
<name>Puppet Team</name> | |||||
<email>puppet@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://puppetcommunity.slack.com/messages/C6CK0UGB1/">PuppetLab's FreeBSD slack channel</url> | |||||
<url href="https://www.bsdcan.org/2018/schedule/events/930.en.html">BSDCan 2018: IT automation with Puppet</url> | |||||
</links> | |||||
<body> | |||||
<p>Since our last status report last year, the puppet@ team | |||||
regularly updated the | |||||
Puppet ports to catch on upstream releases. We have also | |||||
held a Puppet talk at | |||||
BSDCan.</p> | |||||
<p>More recently, Puppet 6 was released, and a bunch of new | |||||
ports appeared in the | |||||
FreeBSD ports tree: sysutils/puppet6, | |||||
sysutils/puppetserver6, | |||||
databases/puppetdb6 being (obviously) the main ones. In | |||||
this update, the Puppet | |||||
language has not been heavily modified. As a consequence, | |||||
upgrading from Puppet | |||||
5 to Puppet 6 is an easy task compared to the experience | |||||
you may have | |||||
encountered from previous major version bumps. If you are | |||||
still using Puppet 4, | |||||
we recommend to schedule an upgrade soon: Puppet 4 is | |||||
expected to be EOL by the | |||||
end of 2018.</p> | |||||
<p>Because distributing Marionette Collective modules via | |||||
Puppet is more efficient | |||||
than using packages, the | |||||
sysutils/mcollective-\*-{agent,client} ports have | |||||
been | |||||
deprecated. Marionette Collective itself being phased out | |||||
by PuppetLabs, the | |||||
sysutils/mcollective port is expected to be deprecated at | |||||
some point in the | |||||
future, but we plan to keep it until an alternative is | |||||
available. This | |||||
alternative, called Choria, is in active development by | |||||
R.I.Pienaar the | |||||
original author of Marionette Collective. We are actively | |||||
Done Inline Actionss/Further/Further,/ bcr: s/Further/Further,/ | |||||
working with him to | |||||
support FreeBSD out of the box, and will commit | |||||
sysutils/choria to the tree as | |||||
soon as it is considered a drop-in replacement for | |||||
Marionette Collective.</p> | |||||
</body> | |||||
</project> | |||||
<project cat='ports'> | |||||
<title>scarab: CLI tool for Bugzilla-related workflows</title> | |||||
<contact> | |||||
<person> | |||||
<name>Oleksandr Tymoshenko</name> | |||||
<email>gonzo@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://github.com/gonzoua/scarab">GitHub repo</url> | |||||
</links> | |||||
<body> | |||||
<p>scarab is a CLI tool that makes some of Bugzilla | |||||
functionality | |||||
available from the command line. Normally users interact | |||||
with the | |||||
<a | |||||
href="https://bugs.FreeBSD.org/bugzilla/">bugtracker</a> | |||||
using a web browser | |||||
but for certain workflows, Web UI may be more of an | |||||
obstacle | |||||
than help requiring to perform more steps compared to CLI | |||||
tool.</p> | |||||
<p>Bugzilla provides XML-RPC interfaces that can be used for | |||||
automation/integration and there are several CLI tools | |||||
like | |||||
<a href="https://github.com/williamh/pybugz">pybugz</a> | |||||
that can be used | |||||
with bugs.FreeBSD.org as-is. They are generic | |||||
one-size-fits-all | |||||
tools which mean they can do a lot of thing at the cost of | |||||
more complex CLI.</p> | |||||
<p>scarab was created to be more specialized and less complex | |||||
with following | |||||
principles in mind:</p> | |||||
<ul> | |||||
<li>Be an auxiliary tool, not a replacement for the web UI</li> | |||||
<li>Move complexity to a configuration file, keep arguments as | |||||
simple as possible</li> | |||||
<li>Optimize for most common/tedious tasks</li> | |||||
</ul> | |||||
<p> | |||||
Based on my experience with Bugzilla following tasks were | |||||
identified as | |||||
candidates for inclusion in the first release of the tool:</p> | |||||
<ul> | |||||
<li>Downloading attachment on host machine and copying it to | |||||
devbox</li> | |||||
<li>Creating a file on the devbox and copying it to a host | |||||
machine to be attached through Web UI</li> | |||||
<li>Creating PRs with common fields' values</li> | |||||
</ul> | |||||
<p> | |||||
First two operations were implemented as <tt>files</tt>, | |||||
<tt>fetch</tt>, <tt>fetchall</tt>, <tt>attach</tt> | |||||
commands of the tool.</p> | |||||
<p>The third operation was implemented by introducing PR | |||||
templates, set of | |||||
predefined field/value pairs, that can be combined | |||||
run-time to provide higher | |||||
flexibility. More information and usage examples can be | |||||
found in the | |||||
<a | |||||
href="https://github.com/gonzoua/scarab/blob/master/examples/scarabrc">config | |||||
file example</a></p> | |||||
</body> | |||||
Done Inline Actionss/using web browser/using a web browser/ bcr: s/using web browser/using a web browser/ | |||||
</project> | |||||
<project cat='doc'> | |||||
<title>Quarterly Reports</title> | |||||
<contact> | |||||
<person> | |||||
<name>Edward Tomasz Napierała</name> | |||||
<email>trasz@FreeBSD.org</email> | |||||
</person> | |||||
<person> | |||||
<name>Mateusz Piotrowski</name> | |||||
<email>0mp@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>The Quarterly Reports have been resurrected after almost a | |||||
year long hiatus. | |||||
The old workflow, which consisted of users submitting | |||||
XML-formatted entries, | |||||
which would then get hand-assembled into DocBook, was | |||||
replaced with a new | |||||
one, using Markdown instead. The XML submission form was | |||||
replaced with | |||||
GitHub Pull Requests. This should make submissions and | |||||
editing much easier | |||||
and user-friendly.</p> | |||||
</body> | |||||
Done Inline Actionss/as a candidates/as candidates/ bcr: s/as a candidates/as candidates/ | |||||
</project> | |||||
<project cat='doc'> | |||||
<title>Cleaning up the Wiki</title> | |||||
<contact> | |||||
<person> | |||||
<name></name> | |||||
<email>wiki-admin@FreeBSD.org</email> | |||||
</person> | |||||
</contact> | |||||
<links> | |||||
<url href="https://wiki.FreeBSD.org/WikiFixitGroup/">Wiki Fixit Group Website</url> | |||||
<url href="irc://freebsd-wiki@irc.freenode.net">IRC channel</url> | |||||
</links> | |||||
<body> | |||||
<p>The FreeBSD Wiki used to be a scratch pad for the FreeBSD | |||||
developers to | |||||
organize projects, store notes and publish articles that | |||||
were about to be added | |||||
to the handbook. Recently, however, the FreeBSD wiki | |||||
started to attract more | |||||
and more people from the wider FreeBSD community, which | |||||
resulted in a change of | |||||
the character of the wiki.</p> | |||||
<p>As a result we decided to discuss the future of the tools | |||||
we want to use for | |||||
documentation in FreeBSD (one of such discussions was held | |||||
during BSDCam 2018, | |||||
you may see some notes | |||||
<a | |||||
href="https://wiki.FreeBSD.org/DevSummit/201808/DeveloperTools">here</a>). | |||||
The general | |||||
conclusion is that wiki is a great tool for what it was | |||||
meant for: organizing | |||||
projects and notes in the community of developers. We | |||||
should not move all our | |||||
documentation (especially handbooks) to Wiki as the | |||||
quality and maintainability | |||||
would suffer. On the other hand, the current workflow of | |||||
submitting | |||||
documentation patches, which involves checking out the doc | |||||
tree and patching | |||||
XML files is not ideal for many end users. This is why we | |||||
are trying to approach the problem from various | |||||
directions:</p> | |||||
<ol><li>The wiki is being cleaned up of old content. We are | |||||
trying to define a clear | |||||
hierarchy of subpages and categories to make navigating | |||||
the wiki easier.</li> | |||||
<li>Some articles from the wiki are going to be migrated to | |||||
either the doc tree | |||||
or manual pages.</li></ol> | |||||
</body> | |||||
</project> | |||||
<project cat='third'> | |||||
<title>HardenedBSD 2018Q3 Update</title> | |||||
<contact> | |||||
<person> | |||||
<name>Shawn Webb</name> | |||||
<email>shawn.webb@hardenedbsd.org</email> | |||||
</person> | |||||
</contact> | |||||
<body> | |||||
<p>Our last report was | |||||
<a | |||||
href="https://www.FreeBSD.org/news/status/report-2017-04-2017-06.html#HardenedBSD">June | |||||
2017</a>. | |||||
A lot has transpired since then. In this status report, we | |||||
will | |||||
attempt to briefly cover all the progress we've made, | |||||
including the | |||||
few commits that made it upstream to FreeBSD.</p> | |||||
<p>On 01 Jul 2018, we switched back to | |||||
<a | |||||
href="https://hardenedbsd.org/article/shawn-webb/2018-04-30/hardenedbsd-switching-back-openssl">OpenSSL</a> | |||||
as the crypto library provider in base. We did this | |||||
because we lack | |||||
the resources and the documentation for properly | |||||
supporting LibreSSL | |||||
in base. We still maintain LibreSSL in base; however, | |||||
OpenSSL is | |||||
simply the default crypto library (aka, | |||||
<tt>WITHOUT_LIBRESSL</tt> is the | |||||
default). We look forward to building a development | |||||
community around | |||||
LibreSSL in HardenedBSD such that we can re-enable | |||||
LibreSSL by | |||||
default, providing enhanced security for our users through | |||||
the | |||||
rejection of software monocultures.</p> | |||||
<p>Cross-DSO Control Flow Integrity (Cross-DSO CFI) is an | |||||
exploit | |||||
mitigation from llvm that provides forward-edge | |||||
protections across | |||||
shared library and application boundaries. With | |||||
HardenedBSD 12-STABLE, | |||||
we launched non-Cross-DSO CFI support in base. Meaning, | |||||
CFI is only | |||||
applied to applications and not shared libraries. Along | |||||
with | |||||
SafeStack, which provides backward-edge protections, | |||||
Cross-DSO CFI | |||||
requires both ASLR and W^X for effectiveness as they store | |||||
crucial | |||||
metadata needing protection. HardenedBSD expertly, | |||||
efficiently, and | |||||
robustly fulfill those requirements through its PaX ASLR | |||||
and PaX | |||||
NOEXEC implementations.</p> | |||||
<p>Over the past two years, we have slowly worked on | |||||
Cross-DSO CFI | |||||
support in HardenedBSD. In mid-2018, we made enough | |||||
progress that we | |||||
could publish an alpha | |||||
<a | |||||
href="https://hardenedbsd.org/article/shawn-webb/2018-07-16/preliminary-call-testing-cross-dso-cfi">Call-For-Testing | |||||
(CFT)</a>. | |||||
We need to integrate the Cross-DSO CFI support with the | |||||
RTLD such that | |||||
function pointers resolved through | |||||
<tt>dlopen(3)</tt>/`dlsym(3)` work properly | |||||
with the cfi-icall scheme. We also need to perform | |||||
experimental | |||||
package builds, find breakages, and fix those breakages. | |||||
We hope to | |||||
officially debut Cross-DSO CFI in the latter half of 2019 | |||||
with the | |||||
possibility of pushing back to 2020. HardenedBSD remains | |||||
the first and | |||||
only enterprise operating system to use CFI across the | |||||
base set of | |||||
applications.</p> | |||||
<p>On 20 Aug 2018, we launched a new tool called | |||||
<tt>hbsdcontrol(8)</tt> to | |||||
toggle exploit mitigations on a per-application basis. | |||||
<tt>hbsdcontrol(8)</tt> uses filesystem extended | |||||
attributes and is the | |||||
preferred method for exploit mitigation toggling for those | |||||
filesystem | |||||
that support extended attributes (UFS, ZFS). Our original | |||||
utility, | |||||
<tt>secadm</tt>, should be used with filesystems that do | |||||
not support extended | |||||
attributes (NFS).</p> | |||||
<p>In <a | |||||
href="https://hardenedbsd.org/article/shawn-webb/2018-09-17/announcing-hardenedbsd-foundation">September | |||||
2018</a>, | |||||
the HardenedBSD Foundation Corp became a 501(c)(3) | |||||
tax-exempt, | |||||
not-for-profit organization in the USA. This means that | |||||
donations by | |||||
US persons are eligible for tax deductions. The creation | |||||
of the | |||||
HardenedBSD Foundation will ensure that HardenedBSD | |||||
remains successful | |||||
long-term. We look forward to working with the BSD | |||||
community to | |||||
provide an open source, clean-room reimplementation of the | |||||
grsecurity | |||||
patchset based on publicly-available documentation.</p> | |||||
<p>We assisted Kyle Evans with the new <tt>bectl(8)</tt> | |||||
utility, primarily | |||||
enhancing jail support and fixing regressions. We are | |||||
grateful for | |||||
Kyle Evans' assistance in landing the enhancements | |||||
upstream in FreeBSD | |||||
and his overall responsiveness and helpfulness.</p> | |||||
<p>Relevant commits for the <tt>bectl(8)</tt> are:</p> | |||||
<ul> | |||||
<li><a | |||||
href="https://svnweb.FreeBSD.org/base?view=revision&revision=339047">r339047</a></li> | |||||
<li><a | |||||
href="https://svnweb.FreeBSD.org/base?view=revision&revision=338221">r338221</a></li> | |||||
<li><a | |||||
href="https://svnweb.FreeBSD.org/base?view=revision&revision=337993">r337993</a></li> | |||||
<li><a | |||||
href="https://svnweb.FreeBSD.org/base?view=revision&revision=337947">r337947</a></li> | |||||
</ul> | |||||
<p> | |||||
We taught <tt>bhyve(8)</tt> how to live in a | |||||
<a href="https://reviews.FreeBSD.org/rS337023">jailed | |||||
environment</a>, allowing users to | |||||
jail the hypervisor. We hardened the virtual address space | |||||
of | |||||
<tt>bhyve(8)</tt> by using <a | |||||
href="https://reviews.FreeBSD.org/rS338511">guard | |||||
pages</a>. | |||||
This work made it upstream to FreeBSD. We are grateful to | |||||
those in | |||||
FreeBSD who provided insight to increase the quality and | |||||
efficiency | |||||
of our patches.</p> | |||||
</body> | |||||
</project> | |||||
</report> |
It looks like we never issued the 2017-10 - 2017-12 report, and there was very little in it
https://www.freebsd.org/news/status/report-2017-10-2017-12.html
Should this file be named 2017-10-2018-09.xml ?