One of the benefits of the FreeBSD development model is a focus on centralized design and implementation, in which the operating system is maintained in a central repository, and discussed on centrally maintained lists. This allows for a high level of coordination between authors of various components of the system, and allows policies to be enforced over the entire system, covering issues ranging from architecture to style. However, as the FreeBSD developer community has grown, and the rate of both mailing list traffic and tree modifications has increased, making it difficult even for the most dedicated developer to remain on top of all the work going on in the tree.
The FreeBSD Monthly Development Status Report attempts to address this problem by providing a vehicle that allows developers to make the broader community aware of their on-going work on FreeBSD, both in and out of the central source repository. This is the first issue, and as such is an experiment. For each project and sub-project, a one paragraph summary is included, indicating progress since the last summary (in this case, simply recent progress, as there have been no prior summaries).
This status report may be reproduced in whole or in part, as long as the source is clearly identified and appropriate credit given.
Assuming there is some positive feedback on this idea, and that future submissions get made such that there is content for future issues, the goal is to release a development status report once a month. As such, the next deadline will be July 31, 2001, with a scheduled publication date in the first week of August. This will put the status report on a schedule in line with the calendar, as well as providing a little over a month until the next deadline, which will include a number of pertinent events, including the Annual USENIX Technical Conference in Boston, MA. Submissions should be e-mailed to:
robert+freebsd.monthly@cyrus.watson.org
Many submitters will want to wait until the last week of July so as to provide the most up-to-date status report; however, submissions will be accepted at any time prior to that date.
-- Robert Watson < rwatson@FreeBSD.org >
The FreeBSD Binary Updater Project aims to provide a secure mechanism for the distribution of binary updates for FreeBSD. This project is complementary to the Open Packages and libh efforts and there should be very little overlap with those projects. The system uses a client / server mechanism that allows clients to install any known "profile" or release of FreeBSD over the network. Where a specific profile might contain a specific set of FreeBSD software to install, additional packages, and configuration actions that make it more ideal for a specific environment (ie FreeBSD 4.3 Secure Web Server Profile)
The system can currently be used to install a FreeBSD system or perform the most simple of upgrades but many features are absent. In particular, the client is in its infancy and much work remains to be done. We need additional developers so please get in touch with us at updater@osd.bsdi.com if you are interested in spending some cycles on this.
Poul-Henning Kamp kicked off a drive to get our GNATS PR database cleaned up so the wheat can be sorted from the chaff. Progress is good, but there is still a lot of work to do. Give a hand if you can. Remember: every unhandled PR is a pissed off contributor or user.
I'm in the process of rewriting the CVSROOT/scripts to make them more clean and configurable. A lot of other projects also use these and so it makes sense to make them as easy to use in other environments as possible.
Status: work in progress. There is now a configuration file, but not all the scripts use it yet.
Work is progressing on implementing true cloning devices in DEVFS. Brian Somers and Poul-Henning Kamp are working to make if_tun the first truly cloning driver in the system. Next will be the pty driver and the bpf driver.
From July 1st DEVFS will be standard in -current.
Added the digi driver. Initial work was done by John Prince <johnp@knight-trosoft.com>, but all the modular stuff was done by me and initial work on supporting Xe and Xi cards (ala dgb) was done by me. I'm now awaiting an Xe card being sent from joerg@ (almost a donation) so that I can get that side of things working properly.
Ben Smithurst has written a "diskcheckd" daemon which will read all sectors on the disks over a configured period. With recent increases in disksizes it is by no means a given that disk read errors will be discovered before they are fatal. This daemon will hopefully result in the drive firmware being able to relocate bad sectors before they become unreadable. This code is now committed to 5.0-CURRENT.
In the last month (May-June), the new fxp driver was brought into -stable. This new driver uses the common MII code, so support for new PHYs is easy to add. Support for the new Intel 82562 chips was added. The driver was updated to add VLAN support and a workaround for a bug affecting Intel 815-based boards.
The FreeBSD Java Project has continued its "behind the scenes" work over the last month. Progress was made both technically, with the help of Bill Huey (of Wind River), on a port of JDK 1.3.1 and legally, with Nate Williams continuing negotiations with Sun on a mutually acceptable license to release a binary Java 2 SDK under. The JDK 1.2.2 port has also seen some development, with a new patchset likely to be released soon which includes JPDA and NetBSD support (the latter courtesy of Scott Bartram).
The Kernel Graphics Interface project has worked for several years to provide a framework for graphic drivers under Linux receiving input from other groups like the UDI project. Currently the KGI core implementation is quite settled, as is the driver coding model as a whole. Work is being done to newbussify KGI and produce a kld, as part of a future redesign of the graphics subsystem in FreeBSD. KGI will be an alternative for graphic card producers that don't accept the XFree86 model of userland graphic adapters and will also provide accelerated support for any other graphic alternative.
The libh project is a next generation sysinstall. It is written in C++ using QT for its graphical frontend and tvision for its console support. The menus are scriptable via an embedded tcl interpreter. It has been growing functionality quite a bit lately, including a new disklabel editor. Current work is on installation scripts for CDROM, FTP, ... installs as well as a fully functional standalone disk-partition and label editor. The GUI API was extended a little and many bugs were fixed. There seems to be some interest in i18n work.
Maxime Henrion is working on implementing a new and more extensible mount(2) systemcall, mainly to overcome the 32 bits for mountoptions limit, secondary goal to make it possible to mount filesystems from inside the kernel.
In the last two months, the OLDCARD pccard implemenation was +
In the last two months, the OLDCARD pccard implementation was rototilled to within an inch of its life. Many new pci cardbus bridges were added. Power handling was improved. PCI Card cardbus bridges are nearly supported and should be committed in early June to the tree. This will likely be the last major work done on OLDCARD. After pci cards are supported, work will shift to improving NEWCARD.
The PowerPC port is proceeding well. All seems to be working in pmap.c after a number of problems encountered where FreeBSD passes a vm_page_t to a NetBSD-derived function that expects a vm_offset_t. Then after debugging the atomic operations code, I'm - now at the point where VM appears to be initialised and it's now + now at the point where VM appears to be initialized and it's now hanging while in sys/kern/kern_malloc.c:kmeminit(). Progress continues. =)
Developing full MPPE support for Andre Opperman @ Monzoon in Switzerland. Work is now complete and will eventually be brought into -current, but no dates are yet known.
Pseudofs is a framework for pseudo-filesystems, like procfs and linprocfs. The goal of pseudofs is twofold:
Pseudofs has reached the point where it is sufficiently functional and stable that linprocfs has been almost fully reimplemented on top of it; the only bit that's missing is the proc/<pid>/mem file.
The primary to-do item for pseudofs right now is to add support for writeable files (which are required for procfs, and are quite a bit less trivial to handle than read-only files). In addition, pseudofs needs either generic support for raw (non-sbuf'ed, possibly mmap'able) files, or failing that, special-case code to handle proc/<pid>/mem.
RELNOTESng is the name I've given to the rewrite of the *.TXT files that typically accompany a FreeBSD release. The information from these files (which include, among other things, the release notes and the supported hardware list) have been reorganized and converted to SGML. This helps us produce the documentation in - various formats, as well as facilitating the maintainence of + various formats, as well as facilitating the maintenance of documentation for multiple architectures. This work was recently committed to -CURRENT, and I intend to MFC it to 4-STABLE before 4.4-RELEASE.
The SMPng project aims to provide multithreaded support for the FreeBSD kernel. Currently the kernel still runs almost exclusively under the Giant kernel lock. Recently, progress has been made in locking the process group and session structures as well as file descriptors by Seigo Tanimura-san. Alfred Perlstein has also added in a giant lock around the entire virtual memory (VM) subsystem which will eventually be split up into several smaller locks. The locking of the VM subsystem has proved tricky, and some of the current effort is focused on finding and fixing a few remaining bugs in on the alpha architecture.
mb_alloc is a new specialized allocator for mbufs and mbuf clusters. Presently, it offers various important advantages over - the old (status quo) mbuf allocator, particularily for MP + the old (status quo) mbuf allocator, particularly for MP machines. Additionally, it is designed with the possibility of - future enchancements in mind.
+ future enhancements in mind.Presently in initial review & testing stages, most of the code is already written.
Work has (re)started on a port of FreeBSD to the UltraSPARC architecture, specifically targeting PCI based workstations. Jake Burkholder will be porting the kernel, and Ade Lovett has expressed an interest in working on userland. Recent work on the project includes:
At this point the kernel can be net-booted and prints the FreeBSD copyright before calling code that is not yet implemented. I am currently working on a design for the pmap module and plan to begin implementation in the next few days.
The TrustedBSD Project seeks to improve the security of the FreeBSD operating system by adding new security features, many derived from common trusted operating system requirements. This includes Access Control Lists (ACLs), Fine-grained Event Logging (Audit), Fine-grained Privileges (Capabilities), Mandatory Access Control (MAC), and other architecture features, including file system extended attributes, and improved object labeling.
-Individual feature status reports are documented seperately +
Individual feature status reports are documented separately below; in general, basic features (such as EAs, ACLs, and kernel support for Capabilities) will be initially available in 5.0-RELEASE, conditional on specific kernel options. A - performance-enhanced version of EAs is currently being targetted + performance-enhanced version of EAs is currently being targeted at 6.0-RELEASE, along with an integrated capability-aware userland, and MAC support.
Patches are now available to add ACL support to cp(1) and mv(1) along with preliminary support for install(1). Ilmar's i18n patches for getfacl(1) and setfacl(1) need to be updated for the last set of changes and committed. Some other functional improvements are also in the pipeline.
The kernel part of the capability implementation is mostly finished; all uses of suser() and suser_xxx() and nearly all comparisons of uid's with 0 have been converted to use the newly introduced cap_check() call. Some details still need clarification. More documentation for this needs to be done.
POSIX.2c-compatible getfcap and setfcap programs have been written. Experimental capability support in su(1), login(1), install(1) and bsd.prog.mk is being tested.
Support for capabilities, ACL's, capabilities and MAC labels in tar(1) is being developed; only the capability part is tested right now. Generic support for extended attributes is planned, this will require extensions to the current EA interface, which are written and will probably be committed to -CURRENT in a few weeks. A port of these features to pax(1) is planned.
An initial prototype of a Mandatory Access Control implementation was completed earlier this year, supporting Multi-Level Security, Biba Integrity protection, and a more general jail-based access control model. Based on that implementation, I'm now in the process of improving the FreeBSD security abstractions to simplify both the implementation and integration of MAC support, as well as increase the number of kernel objects protected by both discretionary and mandatory protection schemes. Generic object labeling introduces a structure not dissimilar in properties to the kernel ucred structure, only it is intended to be associated with kernel objects, rather than kernel subjects, permitting the creation of generic security protection routines for objects. This would allow the easy extension of procfs and devfs to support ACLs and MAC, for example. A prototype is underway, with compiling and running code and simple protections now associated with sysctl's.
Last month's status report was apparently a great success: I received countless e-mails with comments, questions, and suggestions. I've tried to incorporate any suggestions and address any problems from these e-mails in this month's report, which captures a far more extensive snapshot of FreeBSD activity in the last month. Unlike last month's report, it does a better job of reflecting non-development activity, such as on-going conference planning, documentation, and so on. This is a trend I hope to see improve in future months as well.
On the topic of conferences, in the future I'd like to report more on publication activities relating to FreeBSD, including online journals with articles relating to FreeBSD, paper journals, conference papers, and so on. Likewise, I would be interested in including references to Call for Papers relating to FreeBSD. I'll take this opportunity to plug both registration and paper submission for BSDCon Europe in November, which has status included in this report, and for the general BSD Conference being hosted by USENIX in February. Your attendance and submissions make these conferences "happen", and promote FreeBSD as a platform for new research, feature development, and application products. Work of extremely high calibre is performed on FreeBSD, and we need to get the word out.
Next month, we're maintaining much the same submission requirements: reports should be one or two paragraphs long, sent by e-mail, and approximate the layout of the entries this month (Project, Contact, URL, and text). I'll send out reminders again over the week before the deadline, with more specific instructions. An area where I'd like to explore improvement lies in the coordination of related status reports for larger projects, such as new architectural work or platform ports. This might even have the effect of encouraging communication within these projects :-). I'd like to continue to focus on pulling in a broader range of groups and their activities, including the Security Officer, Release Engineer, and Core Team.
-- Robert Watson < rwatson@FreeBSD.org >
ACPI (Advanced Configuration and Power Interface) is an industry standard which obsoletes APM, Intel MPS, PnPBIOS, and other Intel PC firmware interface standards. It is also used on the IA64 platform. More information on ACPI is available at
http://developer.intel.com/technology/iapc/acpiThe FreeBSD ACPI subsystem project is based heavily on the Intel ACPI Component Architecture. This status report outlines the current state of the project; future updates will focus on changes as they occur.
The Intel ACPI interpreter is fully integrated, although bugs are still coming out of the woodwork occasionally.
Work is ongoing in the following areas:
The ARM port is currently going pretty well. The kernel is compiling and is able to boot to the point where it panics trying to initialize the network subsystem. The current reference platform is the Netwinder but this may change as many people expressed interest in a more broadly available platform. Things that need to be done before it can get further includes adding footbridge, timer and interrupt supports. The pmap module is not completed yet either.
Now that BIND 8.2.4 is finally imported the time has come to look at getting BIND 9 imported into CURRENT. The current idea is to have it imported alongside BIND 8 so that people can play with either one until all import problems have been taken care of and people have tested it a bit.
Although gaining a new name, the project has been at a standstill due to both resource availability during the move between BSDi and Wind River, and other commitments of the developers. The project should obtain an official mailing list, as well as return to an active state after the dust settles.
The conference will take place at the Thistle Hotel, Brighton, UK from 9-11 November 2001.
The aim of the conference is to provide a focal point for European users and developers of all the BSD derived operating systems. The format will be similar to other conferences, with 2 days of technical sessions over the Saturday and Sunday.
-We'll be finalising the schedule towards the end of the month +
We'll be finalizing the schedule towards the end of the month and anybody who is interested in doing a talk should contact us - asap. There are no restrictions on the use of talks, if it's been + ASAP. There are no restrictions on the use of talks; if it's been done before we may still be interested in having it presented to an European audience, and we make no claims to the talks so speakers are free to present the talks again at other conferences.
We're also still looking for sponsors.
We had 80 pre-registrations in the first week so we're expecting a good turnout.
The new CAM transport code is starting to get supported in more HBAs and to get refined so that it does the intended - per-protocol support. No progress on doing any SMPNG work for CAM + per-protocol support. No progress on doing any SMPng work for CAM has been made yet. This is a fairly high priority.
Thanks to various outstanding individual efforts, we are now down to just below 2300 open bug-reports. This means that we have fought our way back to the level we had around march 2000.
Work continues (in large part sponsored by WRS) on updating the Handbook ready for the second print edition. There has been a flurry of activity in this area recently, and the ToDo list can be seen at
http://www.freebsd.org/docproj/handbook.html
Dima and others are doing a stellar job of keeping up with the steady flow of incoming PRs relating to the documentation project.
The Developers' Handbook,
http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/index.html
is a year old; it contains a wealth of useful content for developers developing on, or for, FreeBSD. As ever, more contributions are always required, not only for the developers' handbook, but for all of the FreeBSD documentation set.
The basic design hasn't changed and this project mainly is in the phase of continued hardening and test case development. The next major feature will be to fully integrate into the new CAM TRAN code and to fully support on the fly device addition and removal. The only HBA supported is QLogic at this time. Future support for the QLogic line is planned to have 2300 (2Gb) and IP support before October.
Hardware watchpoints are now available for kernel debugging on the IA32 (i386) architecture. One can now set hardware watchpoints using the new ddb command 'hwatch', which is analogous to the existing 'watch' command. Alternatively, if greater flexibility is required, direct access to the debug registers is available using the ddb 'set' command which allows complete control over the processor hardware debug facilities. Hardware watchpoints are very useful in tracking down those elusive memory overwrite bugs in the kernel. Hardware watchpoints can even be used to set a code breakpoint in ROM, which is commonly found in embedded systems.
Support for configuring IEEE 802.11 wireless devices via ifconfig has been committed to -current and -stable. It contains most of the functionality needed to configure an wireless device. Some missing features are being worked on including integrated support for DHCP so a single entry in /etc/rc.conf can be used to fully configure a wireless device on a DHCP lan and setting the CTS/RTS threshold. Currently the an(4) and wi(4) drivers are supported in -current and -stable with the awi(4) device supported in -current. Further work is needed to support Frequency Hopping devices such as ray(4).
jailNG is a from-scratch rewrite of the popular jail(8) - service, focussing on improved management functions, as well as + service, focusing on improved management functions, as well as more fine-grained configurability. An initial prototype has been written, based on explicitly named and configured jails, and work is proceeding on userland integration. Currently, it's not clear if the timeline for this will be 5.0-RELEASE, or 5.1-RELEASE.
The main development in the FreeBSD Java Project over the last month was the release of an initial "Developers Only" patchset for the JDK 1.3.1. Since that release progress had been made - towards a much more useable alpha quality patchset which is + towards a much more usable alpha quality patchset which is likely to be turned into a port, as per the current JDK 1.2.2 patchset. This new patchset will feature a number of bugfixes, which essentially get the JDK to a working state for early adopters, and an initial implementation of "native threads" based on FreeBSD's userland pthreads. Unfortunately this implementation - isn't fully functional, but is included in the hope of more - getting more eyesballs on the code (particularly experience + isn't fully functional, but is included in the hope of + getting more eyeballs on the code (particularly experienced pthread programmers). We'd also like to welcome Fuyuhiko Maruyama-san as a new committer, the usual punishment for too many good patches.
We have been working to provide Japanese version of FreeBSD online manuals, since 1996. Currently, RELENG_4 manuals are based. Translated versions are placed on doc/ja_JP.eucJP/man and provided to users using ports/japanese/man-doc. Also, we discuss about related commands (e.g. ports/japanese/man and ports/japanese/groff).
The first FreeBSD kernel summit meeting was held June 29-30, 2001 in Boston, MA at the Usenix 2001 Annual Technical Conference. Links to a variety of files are posted on the web site.
Note: I (jhb) am still working on writing up a general summary of the meeting. When that is completed it will be posted here and mailed to the -hackers mailing list.
I'm working on multithreading the kernel. So far I have over 400KB of diffs relative to todays -current (I'm keeping my tree updated with changes as they occur rather than get hit with a big - updte at the end).
+ update at the end).I have split the proc structure and am changing most of the kernel to pass around a thread identifier instead of a proc structure.
The following interfaces have been changed so far:
I have still a lot of work to go with a lot of "dumb editing" (s/struct proc \*p/struct thread \*td/) usually I change a few items and then fix everything that breaks when I try compile it. I'd like to check it in on a branch so others can help the editing but haven't worked out the best way to do it yet.
-I have implemented changes to the scheduler so that kse's are +
I have implemented changes to the scheduler so that KSE's are scheduled instead of processes, and threads sleep, letting the - kse pick up a new thread. but it's not anywhere ready yet (heck + KSE pick up a new thread. but it's not anywhere ready yet (heck it doesn't compile yet :-)
Note that I have not yet updated the document listed above.. everywhere it mentions "ksec" or "KSE-context", the code uses the word "thread". I will update it soon as Jason has sent me the source.
The FreeBSD Monthly Development Status Report aims to keep users and developers up-to-date on the latest goings-on in the FreeBSD project by providing summaries of each project and its status. At the time of this writing, the July 2001 status report is being prepared and is very near release. The FreeBSD Web site now has a Status Reports section, which, when the July 2001 report is released, will be updated to include a link to an HTML-ified version.
The NetBSD rc.d port aims to improve the FreeBSD startup process by porting Luke Mewburn's rc.d work from NetBSD to FreeBSD. This will score FreeBSD startup and shutdown dependencies without losing the traditional and much loved - monolothic configuration filesystem.
+ monolithic configuration filesystem.Luke Mewburn's USENIX paper and slides on the system as implemented in NetBSD are available here:
http://groups.yahoo.com/group/FreeBSD-rc/message/3
Interested parties are urged to study this material before joining the discussion list.
The intention at this stage is to decide on an approach that will ensure that the differences between the NetBSD rc.d system and the system as ported to FreeBSD will be kept to a minimum. This will probably involve discussions with Luke around those areas of the system that are identified as areas for potential improvement.
The goal of this project is the implementation of ATM signalling and other ATM protocols by means of the netgraph(4) - framework. This should provide an easily extendable architecture + framework. This should provide an easily extensible architecture for using ATM on FreeBSD. Currently the full UNI4.0 stack (except for the LIJ capability) has been implemented, including ILMI and a first version of the ATM Forum API for UNI. An implementation of Classical IP over ATM is also available. Drivers have been implemented for the Fore PCA200E and Fore HE-155 cards.
Network device cloning support has been imported from NetBSD. This allows virtual devices to be allocated on demand rather then - being staticly allocated at compile time. Our implementation + being statically allocated at compile time. Our implementation differs slightly from that of NetBSD's in that we allow both the creation of specific devices (i.e. gif0) and arbitrary devices instead of just allowing specific devices. Currently, the only device in the tree which has been converted is the gif(4) device which has been converted in both -current and -stable. Work is ongoing to convert all other virtual network devices with work in progress on faith, stf, and vlan interfaces. In general this conversion is accompanied by appropriate modifications to make these devices fully modular.
NGPT is an effort led by IBM engineers to implement MxN threads (also known as many user threads to one kernel thread mapping) on Linux. I have ported it to FreeBSD to use rfork(2).
The port is right here:
Funded by: Monzoon Networking, LLC
This month has been a month of conventration and consolidation. Much of the changes from current have been migrating into stable. I've improved power support, suspend/resume interactions, interrupt handling, and ability to work after windows/NEWCARD has run. Interrupt routing continues to be a locking issue for a complete MFC. Current patches are available at the above website. I'm racing to get this done before 4.4 is released.
Information on Intel ORP - a BSD licensed Java VM is right here:
http://www.intel.com/research/mrl/orp/
A FreeBSD patch has been tested to work with NGPT and submitted to the ORP project. The patch is available here:
http://www.sharma-home.net/~adsharma/projects/orp/orp-freebsd-1.0.5.patch.txt.gz
There are some issues to be ironed out to make it work with FreeBSD's default (user level) pthread implementation.
OpenPackages intends to create a software packaging system that will allow third-party programs to be installed, without operating system dependent changes, on as many platforms as are feasible. OpenPackages was originally based on code from the BSD ports systems, and has been improved and extended by developers of many heritages.
The OpenPackages Project is pleased to release the Milestone 2 codebase. This release contains a working package building system and a single test package. OP currently is known to build on certain instances of the following operating systems: FreeBSD, HP/UX, IRIX, Linux (Debian, Red Hat, Suse, Mandrake, TurboLinux, Caldera, etc.), NetBSD, OpenBSD, Solaris
(First report)
Large cleanup and extension of FreeBSD PAM modules. All - modules are to be documented, consistant in style (style(9) used) + modules are to be documented, consistent in style (style(9) used) and as complete as possible WRT functionality. Mostly done.
We now have the rudiments of device support. We have a nexus driver for OpenFirmware machines, along with support for the Apple UniNorth PCI/AGP host bridge. I'm currently trying to get the USB hardware working so that I can get closer to having a - console driver independant of OpenFirmware, then I'll be trying + console driver independent of OpenFirmware, then I'll be trying to get the system to get to single-user mode using NFS.
Work has begun, but nothing has yet been committed. The NCP addresses used by ppp have been abstracted and initial support has been added to the filter set for ipv6 addresses. NCP negotiation hasn't yet been started.
Patches have been submitted to get ppp working under HURD, and mostly under Linux. There are GPL copyright problems that need to be addressed.
Making pppoed function in a production environment. Most of the work is complete and committed. Additional work includes adding a -l option where ``-l label'' is shorthand for ``-e exec ppp -direct label'' and discovering why rogue child processes are being left around.
PRFW is a set of hooks which I have integrated into the FreeBSD kernel. This allows modules to easily intercept system calls with less overhead. It also supports per-pid restrictions, which means, one process may not be able to use X function in Y manner, but another process may.
Progress: I was working on this in 4.3-RELEASE, but now I'm merging it into current. I will be submitting a patch to the mailing lists in about a week.
This driver is currently not working well under -current and is undergoing some work at this time. No major design or feature changes are planned. There was some notion of adding TapeAlert support, but HP supports that as a binary product via a user library and it was felt that it'd be more politically prudent to leave it alone.
In the 'smpng' p4 branch there is code to make the ast() function loop to close the race when an AST is triggered while we are handling previously triggered AST's.
In the 'jhb_preemption' p4 branch work is being done to make the kernel fully preemptive. It is reportedly stable on UP x86, but SMP x86 locks up, UP alpha has problems during shutdown and can recurse indefinitely until it exhausts its stack.
We are using a perforce repository for live development work, - which can track multiple seperate long-lived works-in-progress + which can track multiple separate long-lived works-in-progress and collaborate between multiple developers at the same time on the same change set.
FreeBSD-current is being imported into p4 hourly, for easy tracking of the moving -current tree.
I haven't written up a good primer yet, but we're able to open this up to the general developer community. NEWCARD work looks like it will be done here too. Perforce is ideal for tracking this sort of long-lived project without having to resort to passing patches around.
KSE work is now being checked into a kse p4 branch - thanks Julian!
KSE work is focusing on getting the main API changes into the base tree well before 5.0.
mb_alloc is a specialized allocator for mbufs and mbuf clusters. It offers various important advantages over the old - mbuf allocator, particularily for MP machines. Additionally, it + mbuf allocator, particularly for MP machines. Additionally, it is designed with the possibility of important future - enchancements in mind.
+ enhancements in mind.The mb_alloc code has been committed to -CURRENT a month ago and appears to be holding up well. Prior to committing it, preliminary performance measurements were done merely to ensure that it is not significantly worse than the old allocator, even with Giant still in place. Results were promising [http://people.freebsd.org/~bmilekic/code/mb_alloc/results.html] - also see jlemon's results (link at the bottom of accompanying text). Since the commit, Matt Jacob has provided useful feedback and bugfixes. Work is now being done to re-enable mbtypes statistics and make appropriate changes to netstat(1) and systat(1).
The sparc64 port has been committed to the FreeBSD repository. As such further development will occur in cvs, rather than as a separately maintained patch set. Significant progress has been made since the last status report, including; support for kernel debugging with ddb, much more complete pmap support, support for context switching and process creation, and filling out of important machine dependent data structures. Thomas Moestl has shown a strong interest in working on the port and is in the process of implementing support for saving and restoring a process's floating point context. I look forward to working with him and any other developers that happen to fall out of the wood works.
The sparc64 loader is functional enough to boot an ELF binary from an UFS filesystem using the existent openfirmware library, which has been revised to work flawlessly on 32-bit and 64-bit architectures. Support for netbooting and modules will be implemented next, followed by a better openfirmware mapping strategy.
This project brings a SYN cache implementation to FreeBSD, in order to make it more robust to DoS attacks. A SYN cookie - approach was considered, but ultimately rejected becuase it does + approach was considered, but ultimately rejected because it does not conform to the TCP protocol. The SYN cache will work with T/TCP, IPV6 and IPSEC, and the size of each cache element is currently is less than 1/5th the size of a normal TCP control block.
It's been a busy month, with a number of relevant news items. Not least important is that NAI Labs was awarded a $1.2M contract from the US Defense Advanced Research Projects Agency (DARPA) to work on a variety of components relevant to the TrustedBSD Project, including support for pluggable security models, and supporting features such as improving the extended attributes implementation, simple crypto support for swap and filesystems, documentation, and much more.
On the features side, progress continues on Mandatory Access Control, object labeling, and improving the consistency of kernel access control mechanisms--in particular, with regard to inter-process authorization and credential management. Work has begun on porting LOMAC, NAI Labs' Low-Watermark Mandatory Access Control scheme, from Linux to FreeBSD, and it has been re-licensed under a BSD license. We hope to have an initial port complete in time for 5.0-RELEASE later this year.
The FreeBSD Project made substantial progress in the month of August, 2001, both on continuing the development of the RELENG_4 line (4.x-STABLE and 4.x-RELEASE), and on 5.0-CURRENT, the main development branch. During this month, the decision was made to push the release of 5.0-CURRENT back so that KSE (support for fine-grained user threads) could be completed in time for the release, rather than postponing that support for 6.0. As such, the lifespan of the RELENG_4 line will be extended, with new features continuing to be backported to that branch. 4.4-RELEASE went into final beta during this month, and will also be available shortly.
This month's edition of the status report has been written with the assistance of Nik Clayton and Chris Costello.
For next month, the submission procedures remain the same: reports should be between one and two paragraphs long, sent by e-mail, and in a format approximately that of this month's submissions (Project, Contact, URL, and text). Reminders will be mailed to the hackers@FreeBSD.org and developers@FreeBSD.org mailing lists at least a week before the deadline; complete submission instructions may be found in those reminders.
-- Robert Watson
2 Gigabit support was integrated on 8/31/2001 (QLogic 2300/2312 cards). Because of the author's shrinking time commitment for FreeBSD, the previously planned "next step" which would have been more complete new CAM Transport integration is now probably just the addition of an FC-IP adjunct (as this can benefit many platforms simultaneously).
A major update to error handling was done on 8/28/2001 which should correct most of the EOM detection problems that have been around for a while. There are several things to fix. The principle thing to fix next is the establishment of a loader(8) mediated device quirks method.
No change since last status. Some discussion amongst all of us occurred, but lack of time and commitment to FreeBSD has meant - little has actually been committed to the tree. SMPNG work will + little has actually been committed to the tree. SMPng work will be left to those who seem to have a notion about what needs to be done.
No new status to report. This driver will be worked on again soon and cleaned up to work better.
Work in adding supporting infrastructure to the kernel for KSE threading support has reached "milestone 2".
Milestone 2 is where the kernel source consistently refers to its resources in terms of per-thread and per-process resources, in the way that it will need to when there are > 1 threads per process, but the LOGICAL changes to such things as the scheduler, and fork and exit, have not yet been made to allow more than one thread to be created. (nor have new threading syscalls been added yet). This is an important milestone as it represents the last point where the kernel has only "mechanical" changes. To go - further we must start adding new algorythms and functions.
+ further we must start adding new algorithms and functions. -The kernel for milestone 2 is reliable and has no noticable - performance degradations when compared to a matchung -current +
The kernel for milestone 2 is reliable and has no noticeable + performance degradations when compared to a matching -current kernel. (the differences are less than the margin of error, so that sometimes the new kernel actually fractionally beats the unaltered kernel).
We hope that by the time this is published, the KSE patches will have been committed. The Major effect for most developers will be only that the device driver interface requires a 'thread' pointer instead of a Proc pointer in the open, close and ioctl entrypoints.
I'm sure there will be small teething problems but we are not expecting great problems at the commit.
The position of Core Secretary was filled by Alan Clegg <abc@freebsd.org> The first core-secretary report should be available the second week in September and will cover the issues discussed by core during August 2001.
Development is continuing; pam_unix has gained the ability to change passwords, login(1) has had PAM made compulsory (and is going to have more PAM-capable features handed over to PAM).
The ATM stack has been tested with a number of FreeBSD machines and a Marconi ATM switch and seems to be quite stable running CLIP. Multi port support for the native ATM API has been implemented but needs some testing.
PRFW is a set of hooks for the FreeBSD kernel. It allows users to insert code into system calls, for such purposes as creating extended security features. Last week, PRFW reached 0.1.0, with many bugfixes and cleaning. I urge anyone who is interested to please visit the site, join the mailing list. Also take a peek at lsm.immunix.org, the Linux hooks. It will be a good contrast.
Work is still progressing to make all of the perl scripts run using perl's 'strict' mode, and to migrate all FreeBSD specific options into the configuration file (CVSROOT/cfg.pm). I'll be looking for help soon to write a guide on how to make use of these scripts for use in your own repository. Anyone interested in helping should contact me at the above email address.
The software has been committed to -current and seems functional. Outstanding issues include dealing with IPV6CP events (linkup & linkdown scripts) and allocating site-local and global addresses (currently, ``iface add'' is the only way to actually use the link).
Status is unchanged since last month. Patches have been submitted to get ppp working under HURD, and mostly under Linux. There are GPL copyright problems that need to be addressed. Many conflicts are expected after the commit of IPv6 support in ppp.
Making pppoed function in a production environment. All known problems have been fixed and committed.
I looked at bringing PPPoA into the base system, but could not because of an overly restrictive distribution license on the Alcatel Speedtouch modem firmware. It has been committed as a port instead and is running live at a FreeBSD Services client site.
The OLDCARD improvements have been completed, except for a few edge cases for older laptops with CL-PD6729/30 chips and some pci bios issues. Some minor work will continue, but after 4.4R is released, only a few remaining bugs will be fixed before the author moves on to greener fields of NEWCARD development.
Targeting 4.4-RELEASE, one team has been translating newly MFC'ed section [125678] manpages. The other team has been updating section 3 since May and one third (1/3) is finished. The port ja-groff is updated to be groff-1.17.2 based, and now it has the same functionality as base system does. The port ja-man is updated to have the search capability under an architecture subdirectory, as base system does. The doc/ja_JP.eucJP/man hierarchy update (adding architecture subdirectories) is planned after 4.4-RELEASE.
Basic footbridge support is now functional and the kernel is now able to probe the pci bus. Access primitives for the bus are still missing so I can't attach any drivers yet.
The syncache implementation is completed, and currently under testing and review. The code should be committed to -current in the near future, and a patchset for -stable made available.
State information for TCP connections is primarily kept in the TCP/IP control blocks in the kernel. Not all of the TCP states make use of the entire structure, and significant memory savings can be had by using a cut-down version of the state in some cases. The first phase of this project will address connections that are in the TIME_WAIT state by moving them into a smaller structure.
This project has completed the initial research and rough design phases, with actual code development starting immediately.
For 5.0, the goal is for the network stack to run without the Giant lock. Initial development in this area may focus on partitioning the code and data structures into distinct areas of responsibilities. A first pass of locking may involve using a several smaller mini-giant code locks in order to reduce the problem to a manageable size.
Progress for this month includes the creation of a perforce repository to officially track the locking changes, and the initial submission of locks for the &ifnet list. Some code cleanup has also been done to the main tree in order to better support future locking additions.
Currently, all network devices (fxp0, lo0, etc) exist in their own namespace, and are accessed through a socket interface. This project creates device nodes in /dev for network devices, and allows control and access in that fashion.
This is experimental work, and suggestions for APIs and functionality are strongly encouraged and welcomed. In is not clear whether it will be possible (or desirable) to provide the exact same set of operations that can be done through the socket interface.
Benefits of approach include the fact that a kqueue filter can be attached to a network device for monitoring purposes. Initial - code exists to send a kq event whever the network link status + code exists to send a kq event whenever the network link status changes. Other benefits may include better access control by using filesystem ACLs to control access to the device.
RELNOTESng, the DocBook-ified set of release documentation files, has been merged to the RELENG_4 branch. 4.4-RELEASE will be the first release of FreeBSD with the new-style release notes, hardware list, etc. Some of these documents are being translated by the Japanese and Russian translation teams.
Snapshots of RELNOTESng for CURRENT and 4-STABLE in HTML, text, and PDF are available at the above URL and are updated irregularly but frequently. Dima Dorfman <dd@FreeBSD.org> and Nik Clayton <nik@FreeBSD.org> have been working to have automatically-generated snapshots on the main FreeBSD web site.
On my TODO list: 1) Resynchronize the FreeBSD installation document with the installation chapter in the Handbook. 2) Update the hardware lists (with particular emphasis on PCCARD and USB devices). 3) Update the infrastructure to allow the architecture-dependent parts of RELNOTESng to scale to more hardware platforms.
Sparc64 development is still continuing rapidly and we're making some excellent progress. Of note, some problems with the way the pmap module implements copy-on-write mappings have been fixed and fork() now works as expected, support for signals has - been added, and the port has been updated for kse in the perforce + been added, and the port has been updated for KSE in the perforce repository. Thomas Moestl has begun work on pci bus support, and a basic nexus bus for sparc64 has been written. The driver for the Sun `Psycho' and `Sabre' UPA-to-PCI bridges and associated code has been ported from NetBSD (the Sabre is the on-chip version found in the UltraSparc IIi and IIe). PCI configuration, I/O and memory space accesses do already work, as well as interrupt assignment and delivery for devices attached directly to the bridge, and the first PCI device drivers can attach and seem to work mostly. Interrupt routing and busdma support still need much work.
The Handbook has been the main focus of activity this month. Due to go to the printers on the 15th a vast amount of new content has been submitted and committed. This includes a complete rewrite of the "Installing FreeBSD", which massively expands the amount of information available to people new to FreeBSD. It even includes screenshots.
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/install.html
Comments, and contributions are, of course, welcome.
FreeBSD's IP Multicast Routing support was recently updated in several ways. One big change is that it's now able to be loaded as a KLD instead of statically compiled into the kernel; this is especially useful for experimentation or updating of an existing system. It also now coexists nicely with the kernel IP encapsulation infrastructure, so that multicast tunnels can better coexist with MobileIP, certain IPSec tunnels and generic IPv4-in-IPv4 tunnels.
The allocator appears to be stable. Mbtypes statistics have been re-activated thanks, in part, to Jiangyi Liu <jyliu@163.net> although the diff has not yet been committed (I'm just in the process of cleaning it up a little and final testing). More work to come: cleanups, follow TODO from the original commit, and perhaps an eventual generalization of the allocator for various network-related allocations (in a more distant future).
After two months of little progress, RAIDframe work is gearing up again. The port to -stable has some known bugs but is fairly stable. The port to -current was recently completed and patches will be released soon. RAIDframe is a multi-platform RAID subsystem designed at CMU. This is a port of the NetBSD version by Greg Oster.
The aac driver has been given a lot of attention lately and is now nearly feature complete. Changes include crashdump support, correct handling of controller initiated commands, and more complete management interface support. The Linux RAID management tool available from Dell and HP now fully works; a FreeBSD native version of the tool is also in the works. These changes have been checked into -current, and will appear in -stable once 4.4 has been released.
We are making some progress, we are now down to 2170 open PR's down from an all time high of 3270 just 3 months ago. The aim is still to get rid of all the dead-wood in the PR database so only relevant PRs in the database. A big thanks from me to the people who have made this happen!
Support for cloning vlan devices via ifconfig has been committed to -current and will be MFC'd after further testing. - Additionaly, Maksim Yevmenkin submitted code to allow cloning of + Additionally, Maksim Yevmenkin submitted code to allow cloning of tap and vmnet devices on devfs systems. Code for faith and stf should be committed shortly.
Current status is that the ia64 kernel builds and runs in a simulator environment up to single user mode and has been tested lightly in that environment. My current focus is on completing the ia64 loader so that I can start to get kernels working on the real hardware. The loader is coming along well and I expect to be able to load kernels (but not necessary execute them) soon.
I have access to the libh CVS repo again and am testing a new, OBJDIR capable build structure at the moment. Done that, I'm going to continue testing the package library and implement the missing functionality. Currently, import of libh into the base system is under discussion (arch mailinglist). Now that 5.0-RELEASE has been shifted, I want 5.0 ship with a libh installer and package system. We can really need people who are good in C++, are able to understand what the current implementation does and also feel that working on libh is fun and thus are willing to help.
Getting GNOME Fifth-Toe metaport ready for 4.4-RELEASE was the main focus of activity this month. In the process many components were updated, many bugs were tracked down and solved, which allowed to make this 97-component meta-package building and working properly.
-Next month the project will be focused on organising work of +
Next month the project will be focused on organizing work of the FreeBSD GNOME Team as well as on attempts to increase amount of people participating in the team (anybody who is willing to participate is welcome to drop a note to gnome@FreeBSD with a short explanation of how he/she could help).
NVIDIA Corporation releases Linux drivers by using a combination of binary object files and source (under a constrictive license). The FreeBSD NVIDIA driver project aimed to completely replace the source component of the driver using code - targetting FreeBSD 4.3 and released under the BSD license. The + targeting FreeBSD 4.3 and released under the BSD license. The binary module provided is supposedly the same module used on Windows, BeOS, and OS/2, so it should be portable between different i80x86 based OS's.
The project is currently on indefinite hold. Our contact at NVIDIA seemed enthusiastic about the project, and was fairly quick about returning email, but when we discovered issues that prevented porting without changes to the binary component or - error codes we needed decyphered, Nick (the contact) said he'd + error codes we needed deciphered, Nick (the contact) said he'd look into it and never got back. The first major problem was the - ioctl interface, the nvidia driver passes a pointer and depends + ioctl interface, the NVIDIA driver passes a pointer and depends on the kernel side to copyout the right amount, where FreeBSD expect the parameters to be correct and the copyout is performed by the subsystem. This was worked around using Dave Rufinos "ioctl tunnel" idea. After that, we found that X refused to load and traced it down to an ioctl defined in the binary component erroring. We cannot tell what that ioctl is, were told that we - could not sign an nda for source to that component, and have been + could not sign an NDA for source to that component, and have been waiting a month for Nick to "look into it". Therefore progress is impossible (without breaking the license) and we believe that the flaws make the driver unportable to any *nix other than Linux.
The FreeBSD release engineering process for FreeBSD 4.4 started to ramp up around August 1st when the "code slush" took affect. During this time all commits to the RELENG_4 branch were reviewed by re@FreeBSD.org (over 250 code snippets had to be reviewed). After the first release candidate on August 15th, all submissions were scrutinized under a more strict potential risk vs benefit curve. The best way to help get involved with the release engineering process is to simply follow the low volume freebsd-qa mailing list, help out with the neverending supply of PRs related to our installation tools (sysinstall), or to work on a possible next-generation replacement for our installation technology, such as the libh or OpenPackages projects.
Many companies donated equipment, network access, or paychecks to finance these activities. Including Compaq, Yahoo!, Wind River Systems, and many more.
In mid March, 2001, Tim Newsham of Guardent identified an attack possible against the initial sequence number generation scheme of FreeBSD (and other OSes.) In order to guard against this threat, a randomized sequence number generation scheme was ported over from OpenBSD and included in 4.3-release. Unfortunately, non-monotonic generation was found to cause major problems with applications which initiate continuous, rapid connections to a single host.
In order to restore proper operation under such circumstances while still providing strong resistance against sequence number prediction, FreeBSD 4.4 uses the algorithm specified in RFC 1948. This algorithm hashes together host and port information with a piece of secret data to generate a unique sequence number space for each connection. As a result, outgoing initial sequence numbers are again monotonic, but also unguessable by an attacker.
The port of LOMAC to FreeBSD is progressing well, and already has a very high level of stability (no known outstanding bugs!). Aspects which have already been implemented include a stacking filesystem overlay with fully-functional access controls (for files and directories) based on path names, access controls for sending signals, and file-backed-memory revocation for processes.
Updates to things from last month:
New stuff since last month:
Most of the work this month has focused on development of the native JDK 1.3.1 patchset. The 3rd patchset is out and has been accompanied with the creation of a FreeBSD "port". This has allowed early adopters much easier access to the code and naturally resulted in a number of bugs being found. Development work has mostly focused on fixing these problems and the project is now set to release fourth patchset over the weekend, which - should see the JDK in a reasonable useable state. One of the big + should see the JDK in a reasonably usable state. One of the big challenges left is producing a working HotSpot JVM, which looks like it will require some heavy hacking.
We also welcome OpenBSD's Heikki Korpela to the porting team :)
As part of some ongoing development activity, the floppy driver (fdc(4)) enjoyed some overhaul in the past which is part of an ongoing process. Automatic density selection will come next, something i meant to implement for years now. As part of that, the entire density selection stuff has been rewritten. 2.88 MB floppies are on the wishlist as well, but I need a working 2.88 drive before attempting to implement that.
sppp(4) should be merged with the ISDN4BSD offspring variant. This will merge some features and bugfixes from the i4b branch (like VJ compression), and eventually end up in a single sppp(4) in the tree. While being at that, incorporating many changes and bugfixes from NetBSD is considered as well.
The KAME project (http://www.kame.net/) has merged its IPv6 and IPsec implementation as of July 2001 to FreeBSD CURRENT and STABLE, in cooperation with some contributors of the project. The latest code includes a number of bug fixes, has been fully tested in FreeBSD STABLE, and will appear in FreeBSD 4.4 RELEASE. Thus, the new RELEASE version will be quite stable in terms of IPv6 and IPsec.
The project has assigned a talented guy to be responsible for merge from KAME to FreeBSD, so future merge efforts will be smoother.
The TrustedBSD project continues to move ahead, with progress made in the ACL, Capability, and MAC implementations. In addition, support from DARPA is permitting new work to improve the extended attribute code, improve security abstractions, and work on security documentation. Due to the push-back of the FreeBSD 5.0 release, it should now be possible to include a complete MAC implementation in that release. Specific status reports appear for components where substantial progress is being made.
Capabilities support is currently being comitted to the base +
Capabilities support is currently being committed to the base FreeBSD tree--userland libraries are now fully committed, and kernel infrastructure is being integrated.
Planning for BSDCon Europe is going well. We're still accepting proposals for talks but the schedule is starting to fill up so we may not be for much longer.
An update of the site that includes accommodation information, a preliminary schedule, a list of speakers and an online payment page will be launched on Wednesday 19 September.
The fee will be £150 for individuals and £250 for corporations. The individual pricing is valid only until the end of September, the price will rise to £200 for October and late registrations in November will be £250.
The updated website will include a list of sponsorship options, we're still looking for more sponsorship.
In the month of September, the FreeBSD Project continued its investment in long-term projects, including continuing work on a fine-grained SMP implementation, support for Kernel Schedulable Entities (KSE) supporting highly efficient threading, and broadening support for modern hardware platforms, including Intel's new IA64 architecture, UltraSparc, and PowerPC. Additional focus was placed on the release process, including work on the release notes infrastructure, support for DVD releases, and work on a binary updating tool.
Due to the delay in getting the September report out the door, the November status report will also cover October. During the month of November, we look forward to BSDCon Europe, the first such event outside the continental United States. The USENIX conference paper submission deadlines are also in November, and FreeBSD users and developers are encouraged to submit to the general and FREENIX tracks. Please see www.usenix.org for more information.
PRFW provides hooks in the FreeBSD kernel, allowing users to insert their own checks in system calls and various kernel functions. PRFW is nearing 0.5, which will incorporate numerous structural changes such as, much faster per-process hooks, kernel function hooks, plus, a new way of adding hooks which would enable users to reference hooks by a string.
The build process is now creating four different versions of the libs, which include support for TVision, Qt, both or none. I created some first packages from existing ports and installed those libh packages on my system only using libh's tools, including registering all the files in the package database, recording their checksums etc. Patches to the disk editor have been submitted, which include functionality to write the changes in the fdisk part and initial support for a disk label editor. We'll soon have a new committer.
FreeBSD 4.4-RELEASE was the first release of FreeBSD with its new-style release documentation. Both English and Japanese versions of these documents were created. Regularly-built snapshots of -CURRENT and 4-STABLE release documentation are now available on the Web site, but they require a little HTML infrastructure to make them viewer-friendly. I intend to continue updating my snapshot site at the URL above, at least for a little while.
-Call for help: The hardware compatability lists need to be +
Call for help: The hardware compatibility lists need to be updated in the areas of the Alpha architecture, USB devices, and PCCARD devices. I'm looking for volunteers to help; interested parties should contact me at the email address above. DocBook experience is not required; familiarity with the hardware above would be very helpful.
Bug fixing and move to -STABLE of 2Gb support.
Quite a lot of cleanup of this driver. Bug fixes and some performance enhancements. However, this driver is likely to be removed shortly and replaced by one from Intel itself.
As you know, in march 2001 the version 2.3 of TIRPC has been - comitted together with many userland changes. Alfred Perlstein + committed together with many userland changes. Alfred Perlstein and Ian Dowse have helped me a lot with the porting effort and if I had problems with understanding the code.
Most bugs are now fixed, some remaining areas to fix are secure RPC (keyserv) and unix domain support. I've patches for these area available. Ian Dowse fixed a lot of outstanding bugs in the rpcbind binary itself. Thank you Ian !
The plan is now to migrate slowly towards TIRPC 2.8, which is threadsafe for the server- and clientside. One first patch I've made available on my URL. TIRPC 2.8 is licensed under the "Sun Standards License Version 1.0" and we have to add some license lines and the license itself to all modified files.
A example is timed_clnt_create.diff which can be found on the homepage.
The project has gained a mailing list, freebsd-binup@FreeBSD.org - and the source tree has been moved into the projects/ directory in the FreeBSD CVS repository. Current work is focusing on extending the FreeBSD package framework, and the client library should be rewritten and completed by the end of the year.
TODO: make the projects/ hierarchy into a cvsup distribution and add it to cvs-all. Then update distrib.self.
Status is unchanged since last month. Patches have been submitted to get ppp working under HURD, and mostly under Linux. There are GPL copyright problems that need to be addressed. Many conflicts are expected after the commit of IPv6 support in ppp.
The software has been committed to -current and seems functional. Outstanding issues include dealing with IPV6CP events (linkup & linkdown scripts) and allocating site-local and global addresses (currently, ``iface add'' is the only way to actually use the link). A bug exists in -stable (running the not-yet-MFC'd ppp code) whereby routing entries are disappearing after a time (around 12 or 24 hours). No further details are yet available.
A two disc set has been mastered and sent for pressing. There are a few surprises with this release - details will be given in the official announcement (at BSDConEurope).
ATM-Forum LAN-emulation version 2.0 without support for QoS has been implemented and tested. The ILMI daemon has been modularized into a general mini-SNMP daemon, an ILMI module and a not yet finished IPOA (IP over ATM) module.
We have finished updating section [125678] manpages to 4.4-RELEASE based, 1 week after 4.4-RELEASE is announced. To finish this update, OKAZAKI Tetsurou has imported Ex/Rv macro support on ja-groff-1.17.2_1. SUZUKI Koichi did most Ex/Rv changes on Japanese manpages. He also find some issues of these - macro usage on some original manapges and filed a PR. For + macro usage on some original manpages and filed a PR. For post-4.4-RELEASE, now we target 4.5-RELEASE. Section 3 update is also in progress.
We've made some good progress now, and the new nmount(2) syscall is nearly finished. There is still some work to do to have a working kernel_mount() and to convert all filesystems to use this new API for their VFS_MOUNT() functions.
I am pleased to announce that as of 1 AM Friday October 19th, the sparc64 port boots to single user mode. A few binaries from the base system have been built and verified to work properly. Much of this work is still in review for commit, but will be integrated into the cvs tree as soon as possible. EBus support has been ported from NetBSD, and ISA support has been written. The PCI host bridge code has stabilized, and busdma seems to work correctly now. The sio driver has had EBus support added, and the ATA driver has been modified so that it works on big-endian systems and uses the busdma API. With these changes, a root file system can now be successfully mounted from ATA disks on sparc64, even in DMA mode. The gem driver, which supports Sun GEM and ERI and Apple GMAC and GMAC2 ethernet adaptor, has been ported from - NetBSD but has not yet had sufficent testing.
+ NetBSD but has not yet had sufficient testing.No new status to report, the code is still waiting to be committed. It is likely that this code will be expanded to include syn cookies as a further fallback mechanism.
Development on this project has been slowed, pending the commit of the syncache code, as this builds on part of that work.
Not much progress has been made this month, with other projects occupying most of my time. However, reviewing all the code and data structures had a side benefit; a hash table for inet addresses has been added. This will significantly speed up interface address lookups in the case where there are a larger number of interface aliases.
Currently, a single device may act as a console at any time, which requires the user to choose the console device at boot time. With the upcoming network console support, it is desirable to allow multiple console devices which behave identically, and to alter consoles while the kernel is running.
The code is completed, and needs some final polishing to clean up the rough edges. Console output can be sent to both syscons and sio, (as well as the network) and when in ddb, input can be taken from any input source. A small control program allows adding and removing consoles on the fly.
This project's goal is to add low level network functionality to FreeBSD. The initial target is to make a network console available for remote debugging with ddb or gdb. A secondary target is to utilize the code to perform network crash dumps. The design assumes that the network card and driver are working, but does not rely on other parts of the kernel.
Initial development has been fairly rapid, and a minimal TCP/IP stack has been written. It is currently possible to telnet to a machine which is at the ddb> prompt and interact with the debugger.
Network devices now support aliases in the form of /dev/netN, where N is the interface index. Devices may be wired down to a specific index number by entries in /boot/device.hints of either:
hint.net.<ifindex>.dev="devname" hint.net.<ifindex>.ether="ethernet address"
Additionally, ifconfig has been updated so that it will accept the alias name when configuring a device.
The gx driver has finally been committed to the tree. The driver provides support for the Intel PRO/1000 cards, both fiber and copper variants. The driver supports VLAN tagging and TCP/IP checksum offload.
In the last month, not a lot has happenned other than settling +
In the last month, not a lot has happened other than settling in of the big August commit. Largely due to me having a sudden increased workload at work, and a need for increased time to be spent elsewhere. However some design work has proceeded. The API has firmed up somewhat and several people have been reading through what has been done already in order to be able to help in the next phase.
Milestone 3 will be to have the ability to generate and remove multiple threads/KSEs per process. Milestone 3 will NOT require that doing so will be safe. (especially in SMP systems), i.e. - locking issues will not be fully addressed, so while some testign + locking issues will not be fully addressed, so while some testing will be possible, it will not be possible to actually run in this mode with any load.
This will require allocators and destructors for the new - structures. Creation of the syscalls. Generation of an acurate + structures. Creation of the syscalls. Generation of an accurate written API for the userland crew. Writing of the upcall launch code. Production of a userland test program (not a full thread scheduler). Resolution of some of the more glaring incompatibilities (e.g. the scheduler) in a backwards compatible manner. (i.e. if there are no multi threaded processes on a system it should behave the same as now (and be as reliable)).
-Criterea for knowing when we have reached Milestione 3 is the +
Criteria for knowing when we have reached Milestone 3 is the ability for a simple process on an unloaded system to perform a series of blocking syscalls reliably. e.g. open 2 sockets, and - send data on one, after having done a read on another,and then + send data on one, after having done a read on another, and then 'respond' in like manner..
There have been a few major successes in the PowerPC port this month. Mark Peek has succeeded in getting the FreeBSD/PowerPC kernel cross compiled on FreeBSD and booting under the PSIM simulator (now in /usr/ports/emulators/psim-freebsd). I have succeeded in getting the FreeBSD loader to load and execute kernels using the OpenFirmware found on Apple Macintosh hardware. Mark is now working on completing some of the startup and pmap code, while I am taking advantage of the simulator to work on some interrupt and device issues.
The project has moved forward on JDK 1.3.1 development this month, with the release of two more patchsets. The team is reasonably confident that the latest patchset is a stable release of the core JDK 1.3.1 tools and classes, when the default "green" threads subsystem is used. This is mostly thanks to hard work by - Fuyuhiko Maruyama to stabilise and fix the code. Bill Huey has + Fuyuhiko Maruyama to stabilize and fix the code. Bill Huey has also been progressing with his work on the "native" threads subsystem, although this hasn't yet reached the stability of "green" threads. Another (arguably the) major highlight of the latest patchset was the integration of NetBSD support by Scott Bartram and Alistair Crooks (the latter of NetBSD packages fame). Hopefully OpenBSD support will follow, making it truly a united BSD Java Project.
This group is for discussion about the startup scripts in FreeBSD, primarily the scripts in /etc/rc*. Primary focus will be on improvements and importation of NetBSD's excellent work on this topic.
Alright folks, I finally got off my butt last night and put together a roadmap for the migration to the new rc.d init scripts that were imported from NetBSD a long time ago and just sat in the tree.
M1 (Patch included)
Setup infrastructure
Make rcorder compile
Hook rc.subr into the distribution (and mergemaster)
Hook rcorder into the world
Add toggle in rc.conf to switch between rc_ng and current boot
scripts
M2
Get FreeBSD to boot with the new boot scripts
Rewrite the /etc/rc.d scripts to work with FreeBSD
M3
Add some FreeBSD specific support into rc.subr
M4
Add true dependency checking to the infrastructure so that
starting nfsd will start mountd and rpcbind
add support into rc.subr
Add dependencies into rc.d scripts
I'd like a couple of people to take a look at this and then I'll submit a pr for it if there aren't too many objections. I'm expecting M2 to run into quite a bikeshed, but hey, I got my nice shiny asbestos back from the cleaners.
The FreeBSD C99/POSIX Conformance Project aims to implement all requirements of the C99 Standard and the latest 1003.1-200x POSIX draft (currently Draft 7). In cases where aspects of the standard cannot be followed, those aspects will be documented in the c99(7) or posix(7) manuals. It is also an aim of this project to implement regression tests to ensure correctness whenever possible.
Patches that implement the <stdint.h> and <inttypes.h> headers, and modifications to printf(3) have been developed and will be committed shortly. They will allow us to use some of the new types C99 introduces, such as intmax_t and the printf(3) conversion specifier "%j".
Some progress has been made on the proc locking this month. Also, a new LOCK_DEBUG macro was defined to allow some locking infrastructure to be more efficient. Kernels now only include the filenames of files calling mutex, sx, or semaphore lock operations if the filenames are needed. Also, mutex operations are no longer inlined if any debugging options are turned on. The ucred API was also overhauled to be more locking friendly. A group has also started investigating the tty subsystem to design and possibly implement a locking strategy.
This months report covers activity during the second half of October, and the month of November. During these months, substantial work was performed to improve system performance and stability, in particular addressing concerns regarding regressions in network performance for the TCP protocol, and via the introduction of polled network device driver support. Work continues on long-term architectural projects for 5.0, including KSEs, NEWCARD, and TrustedBSD, as well as the cleaning up of long-standing problems in FreeBSD, such as PAM integration. Administrative changes are also documented, including work to redefine and formalize the release engineering process, and the approval of a new portmgr group which will administer the ports collection.
FreeBSD users and developers are strongly encouraged to attend the USENIX BSD Conference in February of next year; it is expected that this will be a useful forum both for learning about FreeBSD and on-going work, as well as providing an opportunity for developers to work more closely and act as a vehicle for discussion and round-the-clock hacking. More information is available at the USENIX web site.
Robert Watson
A number of serious TCP bugs effecting throughput snuck into the system over the last few releases and have finally been fixed. TCP performance should be greatly improved for a number of cases, including TCP/NFS.
The wx driver is desupported and removed from -current. No further support for wx in -stable is planned. Newer and better drivers are now in the tree.
Ongoing bug fixes. Work is underway, to be integrated shortly, that makes the cross platform endian support easier and will prepare the FreeBSD version for eventual sparc64 and PowerPC usage.
Currently, we are exploring a variety of strategies to learn about the implementation and performance issues in order to have a solid design. One of our main goals will be to use a standardized interface to the system, whether it be POSIX.1e, or another of the other standards, because as they say "Standards are great because you have so many to choose from." Hopefully within the next month or so, we will populate the perforce TrustedBSD tree with an agreed upon framework that is ready for serious final work.
On the code side, a number of libpam bugs have been fixed; a new PAM module, pam_self(8) , has been written; and preparations have been made for - transitioning from + the transition from /etc/pam.conf to /etc/pam.d .
On the documentation side, new manual pages have been written for pam_ssh(8) , pam_get_item(3) and pam_set_item(3) , and work has started on a longer article about PAM which is expected to be finished by the end of the year.
A lot of work still remains to be done to integrate PAM more tightly with the FreeBSD base system—particularly the passwd(1) , chpass(1) etc. utilities—and ports collection.
Presently re-style(9)ing mbuf code with the help of Bruce (bde). The next larger step is approaching: to better performance, as initially planned, not have reference counters for clusters allocated separately via malloc(9). Rather, use some of the [unused] space at the end of each cluster as a counter; since this space is totally unused and since ref. counter <--> mbuf cluster is a one-to-one relationship, this is most convenient.
Release engineering activities for FreeBSD 4.5 have begun. An overview of the entire process has been added to the FreeBSD web site, along with a specific schedule for 4.5. The code freeze is scheduled to start on December 20. The team responsible for responding to MFC requests sent to re@FreeBSD.org for this release is: Murray Stokely, Robert Watson, and John Baldwin. Some of our many goals for this release include closing more installation-related problem reports, being more conservative with our approval of changes during the code freeze, and continuing to document the entire process. For suggestions or questions about FreeBSD 4.5 release activities, please subscribe to the public freebsd-qa@FreeBSD.org mailing list.
Work is (slowly) progressing on converting the web site to use pages marked up in a simple XML schema, and then generating HTML and other output formats using XSLT style sheets. The work so far can be tested by doing "cvs checkout -r XML_XSL_XP www" and then "cd www/en; make index.html". Take a look at index.page in the same directory to see the source XML. The CVS logs for index.page contain detailed instructions explaining how index.page was generated from its earlier form.
The FreeBSD in Bulgarian project aims to bring a more comfortable working environment to Bulgarian users of the FreeBSD OS. This includes, but is not limited to, font, keymap and locale support, translation of the FreeBSD documentation into Bulgarian, local user groups and various forms of on-line help channels and discussion forums to help Bulgarians adopt and use FreeBSD.
Bulgarian locale support has been committed to FreeBSD 5.0-CURRENT (and later merged into 4.x-STABLE on December 10th). A local CVS repository for the translation of the FreeBSD documentation into Bulgarian has been created.
There is now some code ready for the new mount API, which has to be reviewed and tested. If it is adopted, we will probably start converting all the filesystems, as well as other code in the kernel, to make them use it. If you want to play with it, the patch is available at the above URL.
Support for VLAN cloning has been merged from current and will - ship with 4.5-RELEASE. Additionaly, new rc.conf support for + ship with 4.5-RELEASE. Additionally, new rc.conf support for cloning interfaces at boot has been MFD'd. Work is ongoing to MFC stf and faith cloning as well as adding cloning for ppp devices and enhancing VLAN modularity.
This work uses a mixed interrupt-polling architecture to handle network device drivers, giving the system substantial improvements in terms of stability and robustness to overloads, as well as the ability to control the sharing of CPU between network-related kernel processing and other user/kernel tasks. Last not least, you might even see a moderate (up to 20-30%, machine dependent) performance improvement.
I've been working on making the Hardware Notes less i386-centric. This will be especially important for -CURRENT as the ia64 and sparc ports reach maturity; most of this work should be completed in time to be MFC-ed for FreeBSD 4.5-RELEASE. I encourage any interested parties to review the release documentation and send me comments or patches.
The port of the driver is around 90% feature complete. AGP support and "Registry" support via sysctl need to be finished/implemented. The NVIDIA guys are working on a build of the X11 libs and extensions for FreeBSD; once this is done hardware accelerated direct rendering should work. The previous version this driver is no longer available. I'm planning on making a snapshot of my code once I chase out a few more bugs.
Please note that development is taking place under -CURRENT right now; a port to -STABLE will be available at some later time.
jp.FreeBSD.org daily SNAPSHOTs project is yet another snapshots server that provides latest 4-stable and 5-current distribution. You also find installable ISO image, live filesystem, HTMLed source code with search engine, and more; please check project webpage for more details.
Modest gains have been made on the UDF filesystem since the last report. Reading of files from DVD-ROM now works (and is fast, according to some reports), and there is preliminary support for reading from CD-RW media. The CD-RW support has only been tested against CD's created with Adaptec/ Roxio DirectCD, and much, much more testing is needed. Once this support is solid, I plan to check it into the tree and start work on making the filesystem writable.
Not much to report. A number of minor bugs in OLDCARD have been corrected. A larger number of machines now work. Additional work on ToPIC support has been committed, but continued lack of a suitable ToPIC machine has left the author unable to do much work. A few stubborn machines still need to be supported (the author has an example of one such machine, so there is hope for it being fixed. Some pci related issues remain for both OLDCARD and NEWCARD.
NEWCARD work is ramping up, while OLDCARD work is ramping down. A number of things remain to be done for NEWCARD, including suspend/ resume support, generic device arrival/removal daemon and hopefully automatic loading of drivers. A number of current pccard drivers still need to be converted to NEWBUS. Several Chipset issues remain, as does the merging of isa pccard bridge code with the pccbb code.
This project is now finally underway, thanks to DARPA and NAI getting a sponsorship lined up. The infrastructure code and data structures are currently taking form inside a userland simulation harness.
Targeting 4.5-RELEASE, we continued to revising doc/ja_JP.eucJP/man/man[1256789] to catch up with RELENG_4. Section 3 updating has 45% finished.
A FreeBSD -CURRENT snapshot with LOMAC is currently being prepared, with aid of Perforce on the "green_lomac" branch. Very soon there should be a working demonstration installation CD of FreeBSD with LOMAC, including the ability to enable LOMAC in rc.conf with sysinstall, being a legitimate "out-of-the-box" FreeBSD experience. Actual release build is pending debugging issues with program start-up (especially xdm).
Work is underways to support failing mirror disks better and handle hotswapping in a new replacement disk and have it rebuild automagically.
Support for the Promise TX4 is now working in my lab, seems they did the PCI-PCI bridging in the not so obvious way.
Plans are in the works to backport the -current ATA driver to -stable with hotswap and the works. Now that -current is delayed I'm working on ways to give me time to get this done, since I've had lots of requests lately and we really can't let down our customers :).
SMART support is being worked on, but no timelines yet.
Although not strictly ATA, Promise has equipped me with a couple SuperTrak sx6000 RAID controllers, they take 6 ATA disks and does RAID0-5 in hardware. I have done a driver (its an I2O - device) for both -current and -stable and it works butifully with + device) for both -current and -stable and it works beautifully with hotswap the works. It will enter the tree when it is more mature, and I have an agreement with Promise on how we handle userland control util etc. BTW it seems it can also be used as a normal 6 channel PCI ATA controller, a bit on the expensive side maybe...
Extending camcontrol's page definition file format to include both modepage and logpage definitions; adding support to camcontrol to query and reset log page parameters. Consideration is being made to possibly include support for diagnostic and vital product data pages, but that is outside the current project scope. New page definition file format includes capability to conditionally include page definitions based on SCSI INQUIRY results allowing vendor-specific pages to be described also. Approximately 80% complete.
Work on the FreeBSD C99 & POSIX Conformance Project is progressing nicely. Since the last status report, two new headers have been added [<stdint.h> and <inttypes.h>], several new functions implemented [atoll(3), imaxabs(3), imaxdiv(3), llabs(3), lldiv(3), strerror_r(3), strtoimax(3), and strtoumax(3)], and changes to assert(3) and printf(3) were made to support C99. More printf(3) changes are in the works to support the remaining C99 and POSIX requirements. Additionally, research was done into our POSIX Utility conformance and a list of tasks was derived from that research.
Several other interesting events occurred during November and the beginning of December. The project mailing list was moved to the FreeBSD.org domain, and is now available at standards@FreeBSD.org. On December 6, 2001, the IEEE Standards Board approved the Austin Group Specification as IEEE Std 1003.1-2001, thus making the work we're doing ever more important.
This group is for discussion about the startup scripts in FreeBSD, primarily the scripts in /etc/rc*. Primary focus will be on improvements and importation of NetBSD's excellent work on this topic.
<-- from Gordon Tetlow's ranting -->Due to personal commitments by the folks working on this project we have been unable to spend much time porting the rc.d infrastructure into the FreeBSD boot framework.
Currently, the system will boot (with a little fudging) just before network utilization. There are patches floating around for this (see the -arch list from September).
I have been working behind the scenes on design rather than programming for this last month. I have been working however in the p4 tree to make the system run with the thread structure NOT - a part of the proc structure (a prerequisite fo threading)
+ a part of the proc structure (a prerequisite for threading)After a discussion with the Core Team about our status regarding the ports collection, we heard from them that they'd decided to recognize us as the final authority for approving ports committers. We've spent the last few weeks working on our ports build cluster (see the link) and trying to find ways to improve it for the ports development community. We've also handled a few minor issues in the ports collection.
The TrustedBSD Project continued focussing development efforts +
The TrustedBSD Project continued focusing development efforts on fine-grained Capabilities and Mandatory Access Control this month. Kernel support for capabilities is essentially complete, and efforts are underway to adapt userland applications to use Capabilities. The login process has been updated to allow users to run with additional privilege based on /etc/capabilities. The MAC implementation work has also been active, with improved support for the labeling of IPC objects, including better integration into the network stack. Both development trees have been updated to work with recent KSE-related developments, as well as exist more happily in a fine-grained SMP kernel. Initial - audit-related work appears in a seperate entry.
+ audit-related work appears in a separate entry.Development of TrustedBSD source code was moved to the FreeBSD Perforce repository, permitting better source code management. As such, the TrustedBSD development trees will now be available via cvsup.
October ended up being a bit busier than November for SMPng. During October, Peter Wemm finally finished the ambitious task of unwinding all the macros in NFS and splitting it up into two halves: client and server. Andrew Reiter also submitted some code to add locks to taskqueues, and the folks working on the TTY subsystem designed the locking strategy they will be using. Per-thread ucred references were also added for user traps and syscalls. Once the necessary locking on the process ucred references is committed, this will allow kernel code to access the credentials of the current thread without needing locks while also ensuring that a thread has constant credentials for the lifetime of a syscall. November only saw a few small bug fixes unfortunately, but December is already shaping up to be a very active month, so next month's report should be a bit more interesting.
In non-coding news, the website for the SMPng project has moved from its old location to the new location above. Also, I have completed a paper I am presenting for BSDCon regarding the SMPng project. The paper will be available in the conference proceedings and will be available online after the conference as well.
This bi-monthly report covers development activities on the FreeBSD Project for December 2001 and January 2002. A variety of accomplishments have been made over the last couple of months, including strong progress relating to the KSE project, which brings Scheduler Activations to the FreeBSD kernel, as well as less visible infrastructure projects such as improvements to the mount interface, PAM integration work, and translation efforts. Shortly following the deadline for this status report, the BSD Conference and FreeBSD Developer Summit were held, and will be covered in the next bi-monthly report at the end of March. Plans are already under way for the USENIX Annual Technical Conference in Monterey, CA, later this year, and all and sundry are encouraged to attend to get further insight in FreeBSD development.
Robert Watson
I've been working to integrate recent improvements in the NetBSD usb stack to FreeBSD -current. Both NetBSD and OpenBSD currently share the same source, as FreeBSD did too at once point before it diverged. The goal is to get back to that state, but there are many improvements on both sides that need to be merged before this is complete.
I'm currently looking for someone to help maintain usb in -stable. Please let me know if you're interested.
Patches for cp(1), ls(1), and mv(1) to bring in POSIX.1e-compliant Access Control List support have been updated to patch against builds of -CURRENT. Other system utilities are currently being evaluated for ACL support including install(1) (patch available) and mtree(8). Work is in progress to verify the native getfacl(1), setfacl(1), and other utilities build and work correctly on other ACL-enabled systems (e.g. Linux w/ACL patches) and to help verify POSIX-compliance of the continuing TrustedBSD work along with other systems. Finally, experimental Perl and PHP modules are available allowing limited access to native ACLs for languages other than C.
The project is making progress. The goal is to design and implement Host Controller Interface (HCI) and Link Layer Control and Adaptation Protocol (L2CAP) layers using Netgraph framework. More distant goal is to write support for Service Discovery Protocol (SDP) and RFCOMM protocol (Serial port emulation over Bluetooth link) . All information was obtained from Bluetooth Specification Book v1.1.
Project status: In progress. 1) Design: mostly complete, there are some minor issues to be resolved. 2) Implementation: Kernel - HCI and L2CAP Netgraph nodes have been implemented; 3) User space (API, library, utilities) - in progress. 4) Testing: In progress. I do not have real Bluetooth hardware at this point, so i wrote some tools that allow me to test the code. Some of them will be used as foundation for future user space utilities.
Issues: 1) Bluetooth hardware; I do not have real Bluetooth hardware, so if people can donate hardware/specs it would be great. I promise to write all required drivers and make them available. I also promise to return hardware/specs on first request. 2) Project name; I would like to see the name that reflects the following: it is a Bluetooth stack, implementation is for FreeBSD and implementation is based on Netgraph framework
This project is now finally underway, thanks to DARPA and NAI getting a sponsorship lined up. The infrastructure code and data structures are currently taking form inside a userland simulation harness. Basic MBR and BSD methods have been written and device attach/taste/dettach algorithms been implemented and validated.
I've update OS of buildboxes to the latest FreeBSD 5-current and 4-stable. Everything goes fine. From January 2002, I've started a webzine, SNAPSHOTS Notes (only Japanese version is available). SNAPSHOTs Notes pickups tips and information especially for the people living with FreeBSD 5-current/4-stable. Article or idea for SNAPSHOTs notes are always welcome (you don't need to write in Japanese :-).
Robert Watson created the TrustedBSD audit perforce tree, which is a branch from the TrustedBSD base tree, in order to start pushing development efforts towards using a revision control system. Andrew Reiter started to merge in some framework related code for generation of audit records, enqueueing writes, and handling data writing. There is a great deal of work to be done with updates and discussion on the trustedbsd-discuss@TrustedBSD.org mailing list.
The KSE project (an attempt to support scalable thread in FreeBSD using kernel support), has reached What I call "milestone 3". At this milestone it is possible to run a multithreaded - program on a single CPU but with full concurrancy of threads on + program on a single CPU but with full concurrency of threads on that CPU. In other words the kernel supports the fact that one thread can block by allowing another thread to run in its place. A test program that demonstrates this is available at the above website.
Milestone 4 will be to allow threads from the same program to - run on multiple CPUS but may require more input from the SMPNG + run on multiple CPUs but may require more input from the SMPng project. I am at the moment (Feb 6) getting ready to commit a first set of changes for milestone 3, that have no real effect but serve to drastically reduce the complexity of the remaining diff so that others can read it more easily. After changes to libkvm to support this diff have been added it should be possible to run 'ps' and look at multiple threads in a treaded process. I will be demonstrating KSE/M3 at BSDcon.
The Netgraph ATM package has been split into a number of smaller packages: bsnmp is a general-purpose SNMP daemon with support for loadable modules. Two modules come with it: one implementing the standard network-interface and IP related parts of MIB-2 and one for interfacing other modules to the NetGraph sub-system. ngatmbase contains the drivers for the ATM hardware, the ng_atm netgraph type and a few test tools. This package allows one to use ATM PVCs. It should be possible, for example, to do PPP over ATM with this package. Both bsnmp and ngatmbase are available in version 1.0 under the link above. Two other - modules will be released in february: ngatmsig containing the + modules will be released in February: ngatmsig containing the UNI-4.0 signalling stack as netgraph nodes and ngatmip containing CLIP and LANE-2.0.
A significant amount of progress was made in December and January, particularly in the area of utility conformance. Several utilities were updated to conform to SUSv3, they include: at(1), mailx(1), pwd(1), split(1), and uudecode(1). Several patches have been submitted to increase conformance in other utilities, they include: fold(1), patch(1), m4(1), nice(1), pr(1), renice(1), wc(1), and xargs(1). These are in the process of being reviewed and committed. Two new utilities have been written, specifically pathchk(1) and tabs(1). These are also being reviewed and will be committed shortly.
A patch which implements most of the requirements of scanf(3) is being reviewed and is expected to be committed shortly. This will allow us to MFC a number of new functions and headers. Additionally, work has started on wide string and complex number support.
For 4.5-RELEASE, port ja-man-doc-4.5.tgz is in sync with base system except for OpenSSH pages (OpenSSH 2.3 based instead of 2.9) and perl5 pages (jpman project do not maintain). Section 3 updating has 55% finished.
OKAZAKI Tetsurou has incorporated changes on base system's groff into port japanese/groff. MORI Kouji has fixed two bugs of port japanese/man.
The KAME project is currently focusing on the scoped addressing architecture, the advanced API implementation, NATPT and the mobile ipv6 implementation. Though these stuffs are not stable enough to be merge into the FreeBSD tree, you can get and try them from the above URL.
The FreeBSD in Bulgarian project aims to bring a more comfortable working environment to Bulgarian users of the FreeBSD OS. This includes, but is not limited to, font, keymap and locale support, translation of the FreeBSD documentation into Bulgarian, local user groups and various forms of on-line help channels and discussion forums to help Bulgarians adopt and use FreeBSD.
A guide for using FreeBSD with Bulgarian settings has been put up on the project's website. The CVS repository will be made public shortly, linked to on the URL's above.
An independent project for making FreeBSD easier to use by
Bulgarians has appeared,
The past two months have been an exciting time in the FreeBSD Java Project with the signing of a license between the FreeBSD Foundation and Sun allowing us access to updated JDK source code and the Java Compatibility Kit (JCK). This license will also allow the project to release a binary version of both the JDK and JRE once JCK testing is complete. Work on this testing is under way with the project hopeful of being able to make a binary release in the not too distant future.
In lieu of the binary release which was hoped for with FreeBSD 4.5 the project will release an updated source patchset this weekend. This patchset will feature further work on the FreeBSD "native" threads subsystem from Bill Huey. Also, thanks to hard work by Joe Kelsey and Fuyuhiko Maruyama, the patchset will for the first time feature a working Java browser plugin!
Extending camcontrol's page definition file format to include both modepage and logpage definitions; adding support to camcontrol to query and reset log page parameters. Consideration is being made to possibly include support for diagnostic and vital product data pages, but that is outside the current project scope. New page definition file format includes capability to conditionally include page definitions based on SCSI INQUIRY results allowing vendor-specific pages to be described also. Approximately 90% complete.
OpenPAM, a new library intended to replace Linux-PAM in FreeBSD, has been written and is undergoing integration testing. It is available for download from the URL listed above.
In addition to this, a couple of new modules have been written (pam_lastlog(8), pam_login_access(8)), and the pam_unix(8) module has been extended to perform most of the tasks normally performed by login(1), which is now fully PAMified.
The PAM FDP article has been put on hold until OpenPAM replaces Linux-PAM in CVS, to avoid wasting effort on soon-to-be obsolete documentation.
Substantial progress has been made towards a working MAC implementation. The focus over the last two months has been moving from a hard-coded series of MAC policies to a more flexible implementation. A pluggable policy framework has been created (and is still under development), supporting Biba, MLS, TE, a "BSD Extended" model, and a sample mac_none module. Some modules must be compiled in or loaded prior to boot; others may be introduced at run-time. Support for networking has improved, with improved handling of IP fragmentation in IPv4, support for various pseudo-interfaces such as if_tun and if_tap, improved integration into userland, NFS-related fixes, moving the VFS enforcement out of individual filesystems, support for a 'multilevel' mount flag, support for explicit labeling in procfs and devfs, addition of an 'extattrctl lsattr' argument to list EAs on a filesystem, support for label ranges in the Biba and MAC policies, and much more.
Targets for the next two months include more universal enforcement of VFS-related calls, improved support for alternative ABIs, improved flexibility of in-kernel subject and object labels, support for IPv6 and IPsec, and improved support for NFS serving.
Development continues in the FreeBSD Perforce repository, which may be accessed using cvsup.
Now that the patch has been mailed to the freebsd-arch@freebsd.org mailing list, and that there were no objections, the commit will happen soon. Poul is currently testing it in his own tree. After it has been committed, it will be time to modify the filesystems in the tree to use VFS_NMOUNT instead of VFS_MOUNT. Mount(8) will also need some modifications. Some new manpages -- nmount(2) and kernel_vmount(9) -- are being created in the meantime.
Alfred Perlstein commited file descriptor locking code - which was definetly a good push towards trying to lock down +
Alfred Perlstein committed file descriptor locking code + which was definitely a good push towards trying to lock down some important pieces of global data. Peter Wemm has made progress on pmap cleanups for x86 SMP TLB shootdowns. Matt Dillon and John Baldwin have made progress on getting patches done for moving accesses to ucred's out from under Giant's protection. John Baldwin has also made some commits in order to get the alpha port's SMP working. Matt Dillon has plans for hunting down fileops locking issues in order to continue his previous Giant pushdown work.
This report covers FreeBSD development activities from February, 2002 through April, 2002. It's been a busy few months -- BSDCon in San Francisco, the FreeBSD Developer Summit, a first development preview of 5.0-CURRENT, not to mention lots of progress on the 5.0 feature set (SMPng, sparc64, GEOM, ... the list goes on).
In the next two months, the USENIX ATC occurs (highly recommended event for both developers and users), and a number of new software components will hit the tree, including UFS2 and the TrustedBSD MAC framework. We'll also complete the elections for the FreeBSD Core Team, and should have the next Core Team online by the time the next report rolls around. Stay tuned for more!
Robert Watson
Packages are built from the FreeBSD Ports Collection on a cluster of i386 and alpha machines using scripts available in /usr/ports/Tools/portbuild/. Over the past few months I have been cleaning up and extending these scripts to improve efficiency and allow for greater flexibility in how package builds are performed. Major improvements so far have been: cleaning up and modularizing the scripts to avoid code duplication and reduce the need for ongoing maintenance; optimizing the build process and making it much more robust against client machine failure; and allowing package builds to be restarted if they are interrupted. The i386 package cluster is currently running FreeBSD 5.0-CURRENT, and it has proven to be a useful testing ground for exposing kernel bugs, especially those which only manifest under system load.
Future plans include the ability to perform incremental package rebuilds which only build packages that have changed since the last run. This will allow packages to be made available on the FTP site within an hour or two of the CVS commit to the ports collection. We also hope to set up a sparc64 package cluster in the near future, but this is contingent on suitable hardware.
FreeBSD's new kernel memory allocator has been commited to +
FreeBSD's new kernel memory allocator has been committed to 5.0. UMA is a slabs derived allocator that supports memory - reclaiming, object caching, type stable storage, and per cpu + reclaiming, object caching, type stable storage, and per CPU free lists for optimal SMP performance. It has both a malloc(9) interface and a zone style interface for specific object types. uma(9) will be available shortly.
Read-only support for UDF filesystems was checked into the 5-CURRENT branch in April. Backporting for 4-STABLE is being conducted by Jeroen. The next phase is to write a newfs_udf, then move on to adding write support to the filesystem. I'm still looking for a volunteer to handle read and write support for write-once media (e.g. CD-R).
I have released a new zero copy sockets snapshot, the first since November, 2000. The code has been ported up to the latest -current, and the jumbo code now has mutex protection. Also, zero copy send and receive can be selectively turned on and off via sysctl to make it easier to compare performance with and without zero copy. Reviews and comments are welcome.
I'm slowly making progress. The second engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20020506.tar.gz
This release includes support for H4 UART transport layer, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices.
I'm currently working on RFCOMM protocol implementation (Serial port emulation over Bluetooth link). My next goal is to port Service Discovery Protocol (SDP) implementation from BlueZ (http://bluez.sf.net). I'm also thinking about adding USB device support (as soon as i find/buy hardware).
Issues: 1) Bluetooth hardware; I have couple PC-CARDs that i use for development and testing purposes, but i'd love to have more. 2) Time; My regular day job kicked in, so i will be spending more time doing stuff i'm getting paid for.
Since the last status report, two developers working on utility conformance were given commit access to the FreeBSD CVS repository to help expedite development. As a result, the following utilities have been brought up to conformance, they include: csplit(1), env(1), expr(1), fold(1), join(1), m4(1), mesg(1), paste(1), patch(1), pr(1), uuencode(1), uuexpand(1), and xargs(1). The printf(1) utility was brought up to conformance with the 1992 edition of POSIX.2, with further development planned.
-On the header front, much progress has been made. Specically, - infrastructure to control visiblity of components of a header, based +
On the header front, much progress has been made. Specifically, + infrastructure to control visibility of components of a header, based on the standard requested by an application, has been added to <sys/cdefs.h>. Some work has been completed on renovating the way types are defined. This has lead to the creation of <sys/_types.h>. Further improvements such as the merger of <machine/ansi.h> and <machine/types.h> are planned. Additionally, the headers: <strings.h>, <string.h>, and <sys/un.h> have been made to conform to POSIX.1-2001.
On the API front, scanf(3) has received support for 5 new length modifiers (hh, j, ll, t, and z). A patch to implement two additional conversion specifiers (j and z) has been developed for printf(9) and is expected to be committed soon.
In other news, the project's web site has been moved to the main FreeBSD site. It is now available at the URL at the top of this status report. Please update your bookmarks.
Version 1.1 for FreeBSD-current is now available. It includes the SNMP-daemon package bsnmp, the driver package ngatmbase, the UNI4.0 signaling package ngatmsig and the network emulation package ngatmnet. NgAtm allows both to build applications running directly on top of ATM and to use ATM-Forum LAN emulation to use IP over ATM. Currently we are working on a simple switch module, that implements the network side signaling and ILMI as well as simple routing and call admission control.
The GNOME project has seen quite a few changes lately. For one, the author of this update has recently been given "The Bit." Joe Marcus Clarke now has CVS access, and is working primarily on the GNOME project. Joe has been closing a good deal of GNOME PRs, as well as patching some of the existing GNOME 1.4 components.
The GNOME 2 porting effort continues on. We have completed porting of the GNOME 2.0 API, and are 75% complete on porting the full GNOME 2.0 desktop. When complete, GNOME 1.4 and GNOME 2.0 will be co-resident in the ports tree. Both APIs can be installed concurrently in the same PREFIX, but the respective desktops will remain mutually independent. Maxim Sobolev is working on adapting bsd.gnome.mk to handle both versions of the desktop in an elegant fashion.
Not to be left out, the existing GNOME 1.4 components have received numerous updates to keep them in sync with the stable distfiles - on gnome.org. We have seen many "1.0" milsestone releases including + on gnome.org. We have seen many "1.0" milestone releases including the most recent AbiWord 1.0.0. In the next few weeks, we will be making sure all the GNOME 1.4 components build correct packages on bento so that GNOME 1.4 will be on the 4.6-RELEASE CD.
FreeBSD/KGI started last year after the port of GGI to VGL. KGI (Kernel Graphic Interface) is a kernel infrastructure providing user applications with access to hardware graphic resources (dma, - irqs, mmio). KGI is already available under Linux as a seperate + irqs, mmio). KGI is already available under Linux as a separate project. The FreeBSD/KGI project aims at integrating KGI in the FreeBSD kernel. Mostly a port for now, but optimized for FreeBSD in the future. Currently FreeBSD/KGI is under development and the code is only available for reading, compiling but not running. More interesting are design hints found at the project URL.
We now have a loadable mfsroot floppy. It contains just the diskeditor (which is really a disk partitioner) which has been - enhanced and is probably in his final form. It's been geared - towards making the newfs(1) and mount(1) step seperate dialogs, so - it reduced its complexity. A basic fstab class has been + enhanced and is probably in its final form. It's been geared + towards making the newfs(1) and mount(1) steps separate dialogs, so + it reduceed its complexity. A basic fstab class has been implemented to manipulate /etc/fstab and mountpoint. This might find a use outside libh, by the way. Libh package format is still incomplete and somehow buggy, so it's my next target.
There is a API documentation effort underway with the help of doxygen(1), so there's now more documentation for people that want to get started with libh.
All this lead me to prepare the release of another alpha preview of libh that will shortly be available in the ports collection (0.2.2). Also, a new committer (okumoto) has joined the project (as well as I) and he is currently working on cleaning up the build system. It's been a few months without news, so this probably seemed a bit long, but don't worry, we still need your help to really get this going!
There are several new topics, including: Source Code Tour is now separated into kernel part and userland part, yet another snapshots from RELENG_4_x branch (currently 4.5-RELEASE-p4), add several packages including XFree86 4.x to installation CD-ROM, new - cdboot-only ISO image, fix breackage of duplex.iso, etc. See also + cdboot-only ISO image, fix breakage of duplex.iso, etc. See also the project webpage for more detail. Also, I have a plan to add FreeBSD/alpha distribution to this project -- stay tuned.
KAME Project has been extended until March 2004, and we decided the project roadmap for these two years. The first one year is for implementation, and the remaining year is for feedback of our results into other BSD projects (please refer to the above URL for further detail). Great change is lack of NAT-PT support due to a lack of human resource, although KAME snap still contains it as it is.
SUZUKI Shinsuke (suz@kame.net) has begun working for KAME and FreeBSD merge task in cooperation with Umemoto-san (ume@FreeBSD.org). Some of KAME stuff (critical bug fix, newest ports for pim6sd and racoon, etc) has been merged into 4-stable in this April.
Over the past couple of months, progress has pretty much stopped until very recently. The past few changes to the audit code were update the usage of zones to UMA zones, cleanup some old cruft, and start toying with the idea of having an audit write thread implemented as an ithd. The next step is to decide two realistic approaches to the where the records will be dumped -- whether that is to a local disk or fed up to userland and then dealt with. After that, the goal will be to expand the number of events that are being audited, while also working in some performance testing procedures. I will be posting to trustedbsd-audit about the recent changes shortly.
Over the last three months, there has been a lot of activity in the TrustedBSD MAC tree. An initial commit of the SEBSD code (NSA FLASK and SELinux implementation) was made; many MAC policies previously linked directly to the kernel via kernel options were moved to kernel modules; the flexibility of the framework was improved relating to the life cycle of object labels; additional labeling and access control hooks were introduced; new policies were introduced to demonstrate the flexibility of the framework (including a cleanup of inter-process authorization, additional VFS hooks, improved support for multilabel filesystems, network booting, IPv6, IPsec, support for "peer" labels on stream sockets). Current modules include Biba integrity policy, MLS confidentiality policy, Type Enforcement, "BSD Extended" (permitting firewall-like rulesets for filesystem protection), "ifoff" (limit interface communication by policy), mac_seeotheruids (limit visibility of processes/etc of other users), "babyaudit" (a simple audit implementation), and SEBSD (FLASK/SELinux port).
Over the next month, a final move to completely dynamic labeling will be made, permitting policies to introduce new state relating to process credentials, vnodes, sockets, mounts, interfaces, and mbufs at run-time, allowing a broad range of flexible label-driven policies to be developed. In addition, application APIs will be re-designed and re-implemented so as to better support a fully dynamic policy framework. We plan to make an initial prototype patchset available for review in June, with the intent of committing that patchset in mid-June.
Updated prototype code may be retrieved from the TrustedBSD CVS trees on cvsup10.FreeBSD.org.
The painful parts are now completed, with all authentication- related utilities converted to PAM (except for those cases where it doesn't make sense, like Kerberos- or OPIE-specific commands). OpenPAM is complete (except for a few missing man pages) and seems to work well.
For more details, see the activity reports linked to above.
OpenSSH has been upgraded to 3.1, and the kinks seem to have been worked out by now. OpenSSH will now use PAM for both ssh1 and ssh2 authentication.
The KSE project had floundered due to lack of development time for awhile, but has been picked up recently by Jonathan Mini. Currently, the main focus is to prepare the "milestone 3" code for inclusion into -CURRENT.
The project is still working towards "milestone 4" (allowing threads from the same process to run on multiple CPUs), which should be significantly easier now due to work done by the SMPng project over the past several months.
Help could be used in several areas of the project, especially with porting the libc_r (pthreads) library to KSE's threading model.
NEWCARD support tried to merge CardBus functions with PCI functions, but that failed to properly route interrupts. A branch for the merge was created and will be merged into the main line at a later date. Too many other things going on in my life to make much progress.
Work on the host access point support for the Prism2 and Prism2.5 based wireless cards has been integrated into the kernel. This work is largely based on Thomas Skibo's initial implementation.
Continued bug fixing and hardening for this last few months.
Future work will include making target mode work correctly and fast.
The LSI-Logic chipset's MPT Fusion driver is also being evaluated.
The FreeBSD MTRR code has been made more robust against unexpected values sometimes found in the Athlon's Memory Type Range Registers. Problems with these values had prevented XFree 4.2 running on some motherboards. Experimentation indicates that these undocumented values may control the mapping of BIOS/ROMs or have something to do with SMM. If anyone can provide details of what these values mean, can they please let me know, so the MTRR code can be completed.
IPMI Tools for FreeBSD is a collection of C and Python applications and modules for exploring the information available via the Intelligent Platform Management Interface (IPMI), as implemented on server motherboards by Intel and HP. IPMI is an open standard with patent protection for adopters which defines standard interfaces to on-board management hardware. The management hardware consists of a CPU, sensors such as temperature probes and fan speeds, and repositories such as the System Event Log and Field-Replaceable Unit (FRU) inventory, and other system information.
A basic set of tools was recently made available which uses the KCS and SMIC system interfaces to retrieve the System Event Log, FRU repository, and system sensors. Additional features are currently under research. Suggestions for additional features and programs are greatly appreciated.
The PowerPC port is moving ahead. It can now mount a root file system and exec init, but fails when trying to map init's text segment in. I'm hoping to have it starting my fake "Hello, world!" init soon, after which I plan to try and get some libc bits in place so that I can build /bin and /sbin and try to get to actual single-user.
4.5-RELEASE Japanese manapge package, ja-man-doc-4.5.tgz, once +
4.5-RELEASE Japanese manpage package, ja-man-doc-4.5.tgz, once published with OpenSSH 2.3 (as reported by previous status report) on January 31, is replaced with new package with OpenSSH 2.9 based manpages on March 3. Since then, we have been updating Japanese manpages for 4.6-RELEASE. For new translation and massive update, we have been making a lot of effort.
Continuing section 3 updating has 73% finished.
The GEOM code has gotten so far that it beats our current code - in some areas while stil lacking in others. Work continues on + in some areas while still lacking in others. Work continues on a generalized interface for "magic data" (boot blocks, disklabels MBR's etc) manipulation from userland.
With GEOM enabled in the kernel any FreeBSD platform will now recognize PC style MBR's, i386 disklabels, alpha disklabels, PC98 extended MBRs and SUN/Solaris style disklabels.
Since the last progress report, the initialization code was much cleaned (thanks to NetBSD's acort32 port) and partial DDB support as been added. I'm now struggling to put the pmap module into a working state. The latest patch set only includes the initialization changes. I did some tries to get what I had so far working on my iPAQ without much successes (downloading a kernel over a serial link is way too painful). If anyone has had success in getting any iPAQ to work as a USB storage device under *BSD please contact me.
I've been mentoring someone on locking up the protocol control blocks in the networking stack. She has already finished TCP and UDP and I'm currently reviewing the patch with her and going over some networking lock order issues. Locking up raw protocol interface control blocks follows next.
Support for stf(4), faith(4), and loopback interfaces has been committed to current. The stf and faith support has been MFC'd. In current the API has changed to move unit allocation into the generic cloning code reducing the amount of support code required in each driver. Code improvements to increase our API - compatability with NetBSD will be commited soon along with cloning + compatibility with NetBSD will be committed soon along with cloning support for discard interfaces and ppp(4) interfaces.
Thanks to
IA64 has had a busy few months. Aside from gcc, we are now fully self hosting on IA64. Doug Rabson has performed his magic and implemented the execution of 32 bit i386 application binaries although more work remains to be done to make ld-elf.so.1 happy with the different underlying page size. We have been using the i386 perforce binary to do actual development work and submit from the ia64 systems themselves. Marcel Moolenaar has been working on SMP and machine-check support. We have been running SMP kernels amazingly reliably on our development boxes for quite some time now. syscons is now functional. We have produced a self-booting run-root-on-cdrom ISO image (idea taken from the sparc64 folks) that has been used to manually self install an IA64 system from a blank disk. Aside from a few minor loose ends we now have complete 'make world' functionality. sysinstall works on ia64. We plan on producing a semi-respectable boot/install cdrom image shortly.
As of Thur May 9th, 2002 FreeBSD 5-CURRENT is now using a GCC 3.1 prerelease snapshot as the system C compiler. At this time of cutting over, the compiler is working well on i386, Alpha, Sparc64, and IA-64 for building world. There is a known problem with our atomic ops on Alpha that prevents a GCC 3.1 built kernel from booting.
Currently the C++ support libraries (libstdc++, et.al.) does not build and thus prevents the system C++ compiler from being used.
The release engineering team released FreeBSD
The next releases of FreeBSD will be 4.6-RELEASE (scheduled for 1 June 2002) and 5.0-DP2 (scheduled for 25 June 2002). Information on the release schedules and more can be found on the team's new area on the FreeBSD Web site (see the URL above).
Finally, the team has gained two new members: Brian Somers and Bruce A. Mah.
libradius now supports RADIUS vendor attribute extensions and user-ppp is now capable of doing MS-CHAP authentication via a RADIUS server. A new net/freeradius port has been created for support of MS-CHAP in a RADIUS server.
MS-CHAPv2 support will be added soon.
The work is sponsored by Monzoon.
Mike Makonnen has done quite a bit of excellent work on porting the scripts from FreeBSD into the NetBSD framework. The next step seems to be to try to reduce the amount of diffs between our implementation and the original set from NetBSD.
The SMPng project has been picking up steam in the last few months thankfully. In February, Seigo Tanimura-san committed the first round of process group and session locking. Alfred Perlstein also added locking to most of the pipe implementation. In March, Alfred fixed several problems with the locking for select() and pushed down Giant some in several system calls. Andrew Reiter added locking for kernel module metadata, and Jeff Roberson wrote a new SMP-friendly slab allocator to replace both the zone allocator and the in-kernel malloc(). The use of the critical section API was cleaned up to not be abused as replacements for disabling and enabling interrupts. Also, Matt Dillon optimized the MD portion of the critical section code on the i386 architecture. Several other subsystems were also locked in April as well. See the SMPng website and todo list for more details.
Some of the current works in progress include locking for the kernel linker by Andrew Reiter and light-weight interrupt threads for the i386 by Bosko Milekic. Seigo Tanimura-san, Alfred Perlstein, and Jeffrey Hsu are also working on locking down various pieces of the networking stack. Alan Cox has started working on fixing the existing locking in the VM subsystem and moving bits of it out from under Giant. John Baldwin has written an implementation of turnstiles as well as adaptive mutexes in the jhb_lock Perforce branch. The adaptive mutexes appear to be stable on i386, alpha, and sparc64, but the turnstile code still contains several tricky lock order reversals. John also plans to commit the p_canfoo() API change to use td_ucred in the very near future and then finish the task of making ktrace(4) use a worker thread.
The patch for the new mount API has now been committed to the tree. Several filesystems also have been converted to this new mount API, namely procfs, linprocfs, fdescfs and devfs. I'm working on converting more filesystems to nmount, and actually already have UFS done. It has not been committed yet to avoid conflicting with the UFS2 work, but it should hit the tree soon. Manpages are still missing at the moment because I had to modify the API slightly. I hope to have them done soon now.
The second FreeBSD Developer Summit, held following the BSD Conference in San Francisco in February, was a great success. Around 40 developers attended in person, another five by phone, and many others by webcast. During a marathon-esque eight hour session, a variety of development topics were discussed, including adding - inheritence to the KOBJ system, ports to new architectures, + inheritance to the KOBJ system, ports to new architectures, adaptations of the toolchain for new architectures, the GEOM extensible storage device framework, upcoming changes to the network stack, TrustedBSD features, KSE, SMPng, and the release engineering schedule. This event was sponsored by DARPA and NAI Labs, with webcasting provided by Joe Karthauser, bandwidth provided by Yahoo!. Planning for future such events is now underway; a summary/transcript of discussion may be found at the URL above.
May and June were remarkably busy months for the FreeBSD Project-- FreeBSD developers met in Monterey, CA in June for FreeBSD Developer Summit III to discuss strategy for the FreeBSD 5.0 release later this year, for the USENIX Annual Technical conference and for the FreeBSD BoF. Substantial technical progress was made on FreeBSD 5.0, and FreeBSD 4.6-RELEASE was cut on the RELENG_4 branch in June.
The remainder of the summer will continue to be busy. Final components and features for 5.0-RELEASE will go into the tree, and the development direction will change from new features to stability, performance, and production-readiness. With additional 5.0 development previews late in the summer, we hope to broaden the tester base for the -CURRENT branch, and start to get early adopters digging out any potential problems in their test environments. I encourage both FreeBSD Developers and FreeBSD Users to give 5.0-DP2 a spin (on a machine without critical data!) and let us know how it goes. The more testing that happens before the release, the less fixing we have to do afterwards!
Robert Watson
The current cache for the TCP metrics is embedded directly into the routing table route objects. This is highly inefficient as every route has an empty 56 Byte large metrics structure in it. TCP is the only consumer (except the MTU and Expiry field) of the structure. A full view of the Internet routes (110k routes) has more than 6 Mbyte of unused overhead due to it. The hit rate today is at only approx. 10% in webserver applications. The TCP hostcache will move this entire metrics structure from the routing table to the TCP stack. Every entry is a host entry so a simple hash table is sufficient to keep the entries. Its implementation is much like the TCP Syncache.
The hostcache is going through testing on our servers and will be ready for committing in September. The results of the TCP metrics measurement will be used to tune the cache.
The current Patricia Trie routing table in BSD UNIX is not very efficient and wastes an enormous amount of space for every node (more than 256 bytes) (A full Internet view of 110k routes takes 33 MByte of KVM). Another problem are pointers from and to everywhere in the routing table. This makes replacing the table very hard and - also significantly highers the table maintainance burden (for example - for some kinds of updates the entire PCB has be searched lineary). + also significantly increases the table maintenance burden (for example + for some kinds of updates the entire PCB has to be searched linearly). Also this is a heavy burden for SMP locking. The rewrite focuses on - untangeling the pointer mess, making the routing table replaceable + untangling the pointer mess, making the routing table replaceable and providing a more IP optimized table (5 MByte for 110k routes). - Other new options include policy routing and some structual alignments - in the network stack for clarity, cleaness and flexibilty.
+ Other new options include policy routing and some structural alignments + in the network stack for clarity, simplicity and flexibility.The rewritten IP routing table will be ready for committing in October.
These students will analyse the tcpdumps of five major Swiss newspaper websites which give a representative overview of the user structure in Switzerland. The nice thing about Switzerland is that is has a very good mix of Modem/ISDN, leased line, Cable, ADSL and 3G/GSM/GPRS users. Every Internet access technology is - represented. The goal is to analyse the behaviour of all TCP - sessions to the monitored sites. Parameters to be analysed include + represented. The goal is to analyze the behavior of all TCP + sessions to the monitored sites. Parameters to be analyzed include TCP session RTT, RTT variance, in/outbound BDP, MSS changes, flow - control behaviour, packet loss, packet loss, packet retransmit and + control behavior, packet loss, packet retransmit and timing of HTTP traffic to find optimal TCP parameter caching method.
If you have any other metrics you think is useful please contact me so I can put that into the job description for the Students. The study will be made in September and October.
The current natd is pretty powerful in translating different kinds of traffic but not very powerful in configuration. This project rewrites natd and parts of libalias to give it a configuration set as powerful and expressive as the ones in ipf (ipnat) and pf. In addition it'll use kqueue and will support aliasing to multiple IP addresses.
The rewritten natd will be ready for committing in early September.
IA64 has been progressing slowly. We have access to a prototype 4-way Itaninum2 system from Intel and have managed to get it up and running to the point of being able to access disk and network with SMP enabled. We have a big problem with ACPI2.0 and PCI routing table entries behind pci-pci bridges with no short-term solution in sight. Various WIP items have been committed to CVS, namely more complete support for executing 32bit i386 binaries as well as Marcel Moolenaar's prototype EFI GPT tools.
Max has been busy cleaning up the user interface dark side, and has - come up with a plan to improve the build system (using an automated - Makefile dependency generator); the UI design and the TCL glue magic - (using Swig). A develepment page has been created on usw4, publishing - a lot of information about the current project status, a Changelog, - screenshots, documentation, etc. A new listbox widget has been - implemented, making diskeditor look nicer and more useable. The package - system backend is being inspected and redesigned to conform to a standard - that is itself being re-thought. Indeed, the old sysinstall2.txt text has - been SGML-ized and enhanced and now provides a good (altough rough) overview - of libh package system. This allowed the document to be enhanced with diagrams - of how different procedures work. We are therefore getting closer to a - real pkgAPI specification document. The package management tools have been - sligthly enhanced and should be a bit more useable, and we started commiting - regression test suites in the tree, mostly to test and maintain pkg API - conformance.
- -So work continues on libh. I plan to take a look at the rhtvision port - to see if it would be better to use it for the tvision backend. I'll keep - on working on the package system to make it really trustworthy, while Max - is continuing his great work on the UI subsystem. I hope to make a new libh - alpha release soon. Note that from now on, libh progress will be published - on the development page.
- -Max has been busy cleaning up the user interface dark side, and has + come up with a plan to improve the build system (using an automated + Makefile dependency generator); the UI design and the TCL glue magic + (using Swig). A development page has been created on usw4, publishing + a lot of information about the current project status, a Changelog, + screenshots, documentation, etc. A new listbox widget has been + implemented, making diskeditor look nicer and more usable. The package + system backend is being inspected and redesigned to conform to a standard + that is itself being re-thought. Indeed, the old sysinstall2.txt text has + been SGML-ized and enhanced and now provides a good (although rough) overview + of libh package system. This allowed the document to be enhanced with diagrams + of how different procedures work. We are therefore getting closer to a + real pkgAPI specification document. The package management tools have been + slightly enhanced and should be a bit more usable, and we started committing + regression test suites in the tree, mostly to test and maintain pkg API + conformance.
+ +So work continues on libh. I plan to take a look at the rhtvision port + to see if it would be better to use it for the tvision backend. I'll keep + on working on the package system to make it really trustworthy, while Max + is continuing his great work on the UI subsystem. I hope to make a new libh + alpha release soon. Note that from now on, libh progress will be published + on the development page.
+ +A major power bug was fixed in oldcard. This caused many problems for people using PCI interrupts having their machines hang on boot. This fix has made it into 4.6.1.
Cardbus power is now used on all cardbus bridges that support it. This means that we now support 3.3V cards on all cardbus bridges. Before, we only supported them on some of the bridges because every bridge uses different 3.3V power control when programmed through the ExCA registers. Now that we're going through the CardBus bridge's power control register, 3.3V cards work. In fact, for CardBus bridges, the so called X.XV and Y.YV cards will work in those bridges that support them. However, X.XV and Y.YV haven't been defined yet, and no bridges support them (but the bridge interface define it). Obviously this latter part is untested.
CL-PD6722 support has been augmented slightly. Now it is possible to instruct the driver which type of 3.3V card detection strategy to use. There are three choices: none, do it like the CL-PD6710 does it and do it like the CL-PD6722 does it.
Preliminary support for the CL-PD6729 on a PCI card using PCI interrupts has been committed. However, it fails for at least one of the cards like this the author has.
Client drivers can now ask for the manufacturer and model number of the card without parsing the CIS directly.
Except for fixing bugs and updating pccard.conf entries, no additional work is planned on the OLDCARD system.
A devd daemon, to replace pccardd and usbd, has been designed. A few minor bugs have been fixed in NEWCARD. NEWCARD is now the default in -current. There is an experimental pci/cardbus bus code merge available as a branch which will be merged into current as soon as it is stable.
Status: The ed driver, for non-ne2000 clones, is broken and won't probe. The ata driver won't attach. The sio driver hangs on the first character. The wi driver is known to work well. Cardbus cards are generally known to work well, except for some de based cards, -which unfortuntely includes the popular Xircom cards. Many systems +which unfortunately includes the popular Xircom cards. Many systems fail to work because acpi fails to route interrupts correctly for non-root pci bridges.
Things are going well with the FreeBSD GNOME Project. We have just finished porting the GNOME 2.0 Final development platform and desktop to FreeBSD! We hope to be able to make GNOME 2.0 the default for 5.0-DP2 and 4.7-RELEASE. In the meantime, we're working to port more GNOME 2.0 applications.
In order to allow GNOME 1.4.1 applications to work with GNOME 2.0, we are revamping the GNOME porting infrastructure. GNOME 1.4.1 based ports are being converted to use the new GNOMENG porting structure. The specifics of this new system will be written up in the GNOME porting guide found on the FreeBSD GNOME project homepage.
The BSD Java Porting Team has been making slow but steady progress on a number of fronts in the last few months. Unfortunately most of this has occurred behind the scenes, meaning this is a good opportunity to bring the community up to date.
I'm afraid KAME Project does not work actively with regard to FreeBSD in these two month, since we are too busy with the demonstration of our IPv6 implementation at Networld+Interop 2002 Tokyo. (Thanks to a great effort, the demonstration was quite successful)
We are aware of netinet6-related bug reports regarding socket handling, fine-grain locking, ip6fw etc. Regret to say, we could not answer them right now due to the above situation, however we'll discus these issues internally and determine what to do.
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
Over the past few months the FreeBSD Release Engineering Team oversaw a release process that culminated in the release of FreeBSD 4.6 for the i386 and Alpha architectures on June 15. The RE team is currently working concurrently on FreeBSD 4.6.1 and 5.0 DP2. 4.6.1 is a minor point release with an updated SSH and BIND, fixes for some of the reported ata(4) problems, and assorted security enhancements that will be detailed in the release notes. The release engineering activities for 4.6.1 are taking place on the RELENG_4_6 branch in CVS, while the work on 5.0 DP2 is taking place in Perforce so as not to disturb ongoing -CURRENT development. We are still committed to FreeBSD 5.0 on or around November 15, 2002. For more information about upcoming release schedules, please see our website above. The RE team would like to thank Sentex Communications for providing the release builders with access to a fast i386 build machine. Compaq also donated a couple of fast Alpha build machines to the project.
The main goal of this project is to modify the IPSEC protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPSEC protocols.
Basic functionality is operational for IPv4 protocols. IPv6 support is coded but not yet tested. Hardware assisted cryptographic operations are working with good performance improvements. Operation with software-based cryptographic calculations appears to be at least as good as the existing implementation. Numerous opportunities for performance improvements have been identified.
This work is currently being done in the -stable tree. A port to the -current tree is about to start.
Since the last status report, the following utilities have been brought up to conformance (at least to some degree) with POSIX.1-2001, they include: asa(1), cd(1), compress(1), ctags(1), ls(1), newgrp(1), nice(1), od(1), pathchk(1), renice(1), tabs(1), tr(1), uniq(1), wc(1), and who(1). In addition, development is taking place on bringing the BSD SCCS suite up to date with newer standards.
On the API front, printf(9) has been given support for the `j' and 'n' flags, waitpid(2) now supports the WCONTINUED option, and an implementation of fstatvfs() and statvfs() has been committed. An implementation of utmpx is in progress, which has an aim to address some of the major problems with the current utmp. Several headers have been brought up to conformance with POSIX.1-2001, they include: <netinet/in.h>, <pwd.h>, <sys/statvfs.h>, and <sys/wait.h>.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPSEC), and OpenSSL (through the /dev/crypto device).
The software has been available as a patch against the -stable tree for about six months. The core crypto support is tested, including device drivers for the Hifn 7951, and Broadcom 5805, 5820, and 5821 parts. Recent work has concentrated on fixing device driver bugs, fixing support for Hifn 7811 parts, adding support for public key operations, and adding flow-control between the crypto layer and device drivers. Future work includes porting this facility to the -current tree.
The project took a major step at the beginning of July when Milestone-III was committed. Milestone-III allows a simple test program (available at /usr/src/tools/KSE/ksetest/) to run multiple threads, using kernel support. It does not yet allow the ability to allow these threads to run on different CPUs simultaneously. Milestone IV will be to allow this, however Milestone-III should allow Dan to start (with any interested parties) to start prototyping the userland part of the system. Milestone-III is only currently usable on x86, and does not include some of the requirements for full thread-control, suspension etc. that will be required later.
- Before M-IV is started some small tweeking is likely + Before M-IV is started some small tweaking is likely in the central sources on M-III as we discover issues as we try to get the userland jumpstarted. These will have no effect on non-KSE processes, (i.e. all of them :-) and should not be an issue for other developers.
A tex/fig->html guru is needed to help maintain the - KSE web page (not mentionned above as it is broken). + KSE web page (not mentioned above as it is broken).
The SMPng project has continued to make steady progress in the past two months. Jeff Roberson completed the switch over to UMA for the general kernel malloc() and free() pushing down Giant appropriately so that callers of malloc() and free() are no longer required to hold Giant. Alan Cox continues to clean up the locking in the VM system pushing down Giant in several of the VM related system calls. Jeffrey Hsu committed locking for TCP/IP protocol control blocks in the network stack. John Baldwin committed the changes to the p_canfoo() API to use thread credentials for subject threads and added appropriate locking for the targer process credentials. Support for adaptive mutexes on SMP systems as well as the new IA32 PAUSE instruction were also committed in May. The kernel tracing facility KTRACE also received an overhaul such that the majority of its work was pushed out into a worker thread allowing trace points to no longer require Giant. Andrew Reiter has also been pushing down Giant in several system calls.
Bosko continues to work on light-weight interrupt threads for i386. Most of the bugs in the turnstile code have been found and fixed; however, the turnstile and preemption patches have temporarily been put on hold so that more emphasis can be placed on fixing bugs and making -current more stable in preparation for 5.0 release in November. Alan Cox and Andrew Reiter are continuing the work mentioned above. Jeff Roberson is also working on fixing the current vnode locking in VFS. Peter Wemm has also started to tackle TLB issues on SMP in the i386 pmap again as well.
After an outstanding job serving the project as Security Officer for over a year, Kris stepped down in January in order to focus more of his time pursuing his PhD. I offered to attempt to fill the vacant role.
This is the first report by the SO Team. Notable events since the beginning of 2002 follow.
28 FreeBSD Security Advisories have been issued, 16 of which were regarding the base system. Of those sixteen, 8 affected only FreeBSD.
FreeBSD Security Notices were introduced, and four have been issued so far. The Security Notices cover issues that are not regarded as critical enough to warrant a Security Advisory. So far only Ports Collection issues (i.e. vulnerabilities in optional 3rd party packages) have been reported in Security Notices. The first four Security Notices covered 53 individual issues.
Issues reported to the SO team are now being tracked using a RequestTracker ticket database.
The SO team has undergone membership changes, as well as some changes in internal organization. The membership and organization has also been made publicly visible on the FreeBSD Security Officer web page.
For 4.6-RELEASE, we announced the package ja-man-doc-4.6.tgz which is in sync with 4.6-RELEASE base system manual pages except for perl5 pages (jpman project do not maintain them). Continuing section 3 updating has 88% finished.
Progression is slow, but the effort is maintained. Most of fb over KGI has been written in parallel with a KGI display driver based on fb. DDC/DDC2 is being discussed for Plug & Play monitor support. KGI aims at providing - a generic OS independant interface which would take advantage of FreeBSD I2C (iic(4)) + a generic OS independent interface which would take advantage of FreeBSD I2C (iic(4)) infrastructure.
UFS2 is an extension to the well-known UFS filesystem which using a new inode format adds support for "64bit everywhere" and later for extended attribute support, in addition to the current UFS features: soft-updates and snapshots.
The basic UFS2 code has been committed and work on the extended attribute interface and vnode operations will continue.
The GEOM code has gotten so far that it beats our current code - in some areas while stil lacking in others. The goal is for + in some areas while still lacking in others. The goal is for GEOM to be the default in 5.0-RELEASE.
Currently work on a cryptographic module which should be able to protect a diskpartition from practically any sort of attack is progressing.
The port of openoffice 1.0 has been finished. Most showstopper issues with rtld, libc and our toolchain have been fixed. There is one remaining deadlock in the web-browser code of OO.org. If anybody like to help us with fixing this bug (may be another libc_r bug as it looks like) - just mail me ! Unfortunalty gcc2 support got broken again with the import - of gcc2.95.4 in STABLE. Exceptions support seems to be broken again, we get + just mail me! Unfortunately gcc2 support got broken again with the import + of gcc2.95.4 in STABLE. Exceptions support seems to be broken again; we get internal compiler errors with c++ exceptions code. You'll have to use gcc31 again.
Since our package cluster is outdated and can not build OO.org packages anytime soon, I did my own little package cluster and can now offer packages for 4.6R for 16 different languages. They can be found on the project homepage.
Porting of OpenOffice1.0.1 is on it's way. A beta port and a package have been made available on the project homepage.
The lightweight interrupt scheduling code makes scheduling an interrupt on i386 without having to grab the sched_lock possible, and also avoids a full-blown context switch.
Currently, the code in the p4 branch works, although needs a little bit of cleanup and, most importantly, requires a merge to post-KSE III. Now that stuff seems to have stabilized a bit, I'm waiting to get a little time (and nerve) to do the merge. Also, looking forward for some KSE interface that will allow for "KSE borrowing," which would make this cleaner with regards to KSE and lightweight interrupts. This is a 5.0 feature.
A lot of remaining PR's and Bugs have been closed. All relevant rpc - concerning patches have been comitted. Thank goes to Alfred and Ian Dowese. + concerning patches have been committed. Thanks go to Alfred and Ian Dowese.
Jean-Luc Richier <Jean-Luc.Richier@imag.fr> has made a patch available which adds IPv6 support to all remaining rpc servers. See ftp://ftp.imag.fr/pub/ipv6/NFS/NFS_IPV6_FreeBSD5.0.gz and ftp://ftp.imag.fr/pub/ipv6/NFS/0README_NFS_IPV6_FreeBSD5.0 We will check his code and add it to CURRENT ASAP.
A first commit part from TIRPC99 has been done. I'm working now on porting the remaining parts so when FreeBSD 5.0 gets released, it will be TIRPC99 based. This will happen together with the NetBSD project, as they use the same codebase as we do.
mb_alloc is getting some updates and a couple of optimisations. +
mb_alloc is getting some updates and a couple of optimizations. A new allocator interface routine should already be committed by the time this report is "published:" m_getcl() allocates an mbuf and a cluster in one shot. This is the result of months (literally) of requests from Alfred and, recently, Luigi - who, coincidentally, is the author of the same [upcoming] routine in -STABLE.
Other than that, mb_alloc is being shown how to perform multi-mbuf or cluster allocations without dropping the cache lock in between (m_getcl() and m_getm() will use this). Finally, work is - being done to optimise ext_buf ref. count allocations and to provide + being done to optimize ext_buf ref. count allocations and to provide support for jumbo (> 9K) clusters.
We are making excellent progress. There is a fully functioning implementation imported to -current now. We need as many people as possible to rc_ng equal to YES in /etc/rc.conf.
The next step is to set the default to YES, which we plan to do before DP 2.
In summer 2002 the native FreeBSD firewall has been completely rewritten in a form that uses BPF-like instructions to perform packet matching in a more effective way. The external user interface is completely backward compatible, though you can make use of some newer match patterns (e.g. to handle sparse sets of IP addresses) which can dramatically simplify the writing of ruleset (and speed up their processing). The new firewall, called ipfw2, is much faster and easier to extend than the old one. It has been already included in FreeBSD-CURRENT, and patches for FreeBSD-STABLE are available from the author.
I spent busy days in last two months, many new topics are emerged from the project. We now support FreeBSD/alpha 5-current distribution by cross-compiling on the x86 PC. Anonymous ftp area is now exported to the yet another web server. Our release branch snapshots are relocated to daemon.jp.FreeBSD.org because of our CPU/network bandwidth problem.
I'm seriously considering to solve the lack of CPU and network resources for the project's future evolution. Maybe the bandwidth - problem can be resolved (several bandwidth offering are received!), - but there is no answer about CPU problem (I have a plan to upgrade - our PCs from P3-500Mhz to P4 or something better than previous). - If you have interested to donate PCs to the project, please email me + problem can be resolved (several bandwidth offers have been received!), + but there is no answer about CPU problems (I have a plan to upgrade + our PCs from P3-500MHz to P4 or better). + If you have interested in donating PCs to the project, please email me for more detail.
Regression tests for many bugs fixed in text manipulation utilities have been added, as well as tests for various non-standard versions of functionality that FreeBSD users should expect. A library of m4 macros for creating the tests themselves has been added.
The final version of SCCS distributed by CSRG has been integrated into the projects CVS repository, and worked on extensively to the point where essential functionality works on FreeBSD (and other operating systems). Some standards-related functionality has been implemented
The zero copy sockets code was committed to FreeBSD-current on June 25th, 2002. I'm not planning on doing any more patches, although I will leave the web page up as it contains useful information.
Many thanks to the folks who have tested and reviewed the code over the years.
Jennifer Yang's patch was committed June 10 for the BSD Summit. After a few bugs which were reported initially and fixed that same week, networking in -current has been stable, including the parts that were not locked up, like IPv6. Work is on-going to lock up the rest of the stack.
Not much to report. Another engineering snapshot is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20020709.tar.gz. If anyone has Bluetooth hardware and spare time please join in and help me with testing.
This snapshot includes basic support for USB devices and manual pages. The HCI layer now has support for multiple control hooks. All HCI transport drivers (H4, BT3C and UBT) has been changed to provide consistent interface to the rest of the world. Some userspace utilities have been changed as well.
Still no support for RFCOMM (Serial port emulation over Bluetooth link) and SDP (Service Discovery Protocol). Several design flaws have been discovered and it might take some time to resolve these issues.
The TrustedBSD Project has been busy in May and June, developing new features, presenting on the technology at the FreeBSD Developer Summit, and improving the readiness of the MAC branch for integration into the main FreeBSD tree. The migration to dynamic labeling in the TrustedBSD MAC framework is complete, with all policies now making use of dynamic labels in the kernel. This permits policies to associate arbitrary additional security data with a variety of kernel objects at run-time. Implement mac_test, a sanity checking module. Pass labels as well as objects to each policy entry point to reduce knowledge of label storage in the policies. Implement mac_partition, a simple jail-like policy. Adapt the MAC framework for process locking.
Improve support for sockets: provide a peerlabel maintained for stream sockets (unix domain, tcp), entry points for accept, bind, connect, listen. Improve support for IPv4 and IPv6 by labeling IP fragment reassembly queues, and providing entry points to instrument fragment matching, update, reassembly, etc. Locally disable KAME if_loop mbuf contiguity hack because it drops labels on mbufs: we need to make sure the label is propagated. Label pipes and provide access control for them. Improve vnode labeling: now handle labeling for devfs, pseudofs, procfs. Fix interactions between MAC and ACLs relating to the new VAPPEND flag.
SELinux policy tools now ported to SEBSD. SEBSD now labels subjects and file system objects. Provide ugidfw, a tool for managing rules for the mac_bsdextended policy.
Massive diff reduction. KSEIII merged. Main tree integration will begin shortly.
Updated prototype code may be retrieved from the TrustedBSD CVS trees on cvsup10.FreeBSD.org.
Throughout July and August, the FreeBSD Project has been working on pulling together the last few major pieces of new functionality for FreeBSD 5.0-RELEASE. At this point, the release appears to be on track for late November or early December. Work on fine-grained locking continues, especially in the VFS, as with improved support for threading through the KSE work; features such as GEOM, UFS2, and TrustedBSD MAC are maturing, and the new ia64 and sparc64 hardware ports are approaching production quality. In the next two months, we have a lot to look forward to: additional 5.0 developer preview snapshots, additional locking and threading improvements, and many cleanups on the new supported architectures. Firewire support has been imported into the main tree, and substantial cleanup of the ACPI/legacy PCI code is also in the works. Also, expect the import of new IPsec hardware acceleration support in the near future.
When new developer previews are posted, please give them a try! While we know that 5.0-RELEASE will be for "early adopters", the more testing we get out of the way now, the less we have to tidy up later. The new features are extremely exciting, and understanding when and how to deploy them properly will be important. In the next two months, among other things, the release engineering team will post updated release schedules, as well as guidance for FreeBSD consumers as to how to decide what releases of FreeBSD will be right for them. Keep an eye out for this, and provide us with feedback.
Also, for those of you in Europe -- we look forward to seeing you at BSDCon Europe in a couple of months!
Scott Long, Robert Watson
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
Cloning support for ppp(4) and disc(4) interfaces has been committed. A man page for disc has been created and the disc devices now appear as disc# instead of ds#. Some work is still needed on pppd to make it understand cloning though it should work as long as the devices are created beforehand.
On the API front, management of mandatory interfaces (i.e. lo0) is handled by the generic cloning code so if_clone_destroy has the same API as NetBSD again and <if>_modevent doesn't need to create the necessary devices manually.
At this point, all pseudo interfaces have been converted to the cloning API or already did their own cloning (sl(4) for example uses it's own mechanism). Some devices such as tun(4) and tap/vmware should probably be converted to use the cloning API instead of their current ad-hoc, devfs based cloning system. This would be a good junior kernel hacker task. Also, the handbook and FAQ could use some general cloning documentation prior to 5.0 release.
We have been updating RELENG_4 targeting for 4.7-RELEASE. When port ja-man-1.1j_5 was broken around the end of July, Kumano-san and Mori-san tried to update the port to be based on a newer FreeBSD base system's man commands. But, we decided only to fix the port ja-man-1.1j_5 to be buildable, as the new one was not complete at that time.
The GEOM code has gotten so far that it beats our current code - in some areas while stil lacking in others. The goal is for + in some areas while still lacking in others. The goal is for GEOM to be the default in 5.0-RELEASE.
Currently work on a cryptographic module which should be able to protect a diskpartition from practically any sort of attack is progressing.
The UFS2 filesystem approaches feature completion: Extended attribute functionality have been added, including a new compound modification API and basic testing has been passed.
We've got currently almost 50% of the new handbook translated (all the installation part is translated). Most of the articles are translated too.
The web site in on the way, see the Web Server. We need now to integrate it on the US CVS tree.
One of the big job now, is to translate the latest FAQ and the very big project will be the manual pages
I'm very pleased to announce that another engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20020909.tar.gz
This release features several major changes and includes support for H4 UART and H2 USB transport layers, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices. Also there are several man pages.
Service Discovery Protocol (SDP) is now supported. This release includes SDP daemon, configuration tool and user space library (ported from BlueZ-sdp-0.7).
RFCOMM is now supported. This release includes rfcommd daemon that provides RFCOMM service via pseudo ttys. Not very useful for legacy application, but it is possible to run PPP over Bluetooth now. This was ported from old BlueZ-rfcommd-1.1 (no longer supported by BlueZ) and still has some bugs in it.
Next step is to fix current RFCOMM support and work on new in-kernel RFCOMM and BNEP (Bluetooth Network Encapsulation Protocol) implementation. Also user space need more work (better tools, libraries, documentation etc.).
Version 1.2 has been released recently. It should compile and work an any recent FreeBSD-current. Support to manipulate SUNI registers has been added to the ATM drivers (to switch between SONET and SDH modes, for example). The ngatmsig package now includes a small and - simple call control module that mayh be used to build a simple ATM + simple call control module that may be used to build a simple ATM switch. The netgraph stuff has been patched to use the official netgraph locking.
On the API front, fmtmsg(3) was implemented, glob(3) was given support for new flags, ulimit(3) was implemented, and wide character/string support was significantly improved with the addition of 30 new functions (see the project status board for details). Work is progressing on adding the C99 restrict type-qualifier to functions throughout the system. This allows the compiler to make additional optimizations based on the knowledge that a restrict-qualified argument is the only reference to a given object (ie. it doesn't overlap with another argument).
Several headers have been brought up to conformance with POSIX.1-2001, they include: <fmtmsg.h>, <poll.h>, <sys/mman.h>, and <ulimit.h>. The header <cpio.h> was implemented. The headers <machine/ansi.h> and <machine/types.h> were merged into a single header to help simplify the way variable types are created.
The sh(1) built-in, command(1), was reimplemented to conform with POSIX. Additionally, several utilities which were previously brought - up to conformance were merged into the 4-STABLE banch.
+ up to conformance were merged into the 4-STABLE branch.The GNOME 2 desktop port has reach version 2.0.2rc1 with an expected 2.0.2 release before 4.7-RELEASE. Mozilla 1.1 has been ported, and is resident in the tree with Mozilla 1.0.1. The GNOMENG porting effort is going well. A good deal of ports have been moved to the new infrastructure with the help of Edwin Groothuis. We are now working on smoothing out some of the rough edges, then, once all the work is done, make GNOMENG the default.
A long-standing annoyance in Nautilus has also been recently - corrected. The desktop is no longer clutered with volume icons, and + corrected. The desktop is no longer cluttered with volume icons, and removable media (such as CDs) should now be handled correctly.
The ATAPI/CAM module allows ATAPI devices (CD-ROM, CD-RW, DVD drives, floppy drives such as Iomega Zip, tape drives) to be accessed through the SCSI subsystem (CAM). ATAPI/CAM has been integrated in -CURRENT. The code should be fairly functional (it has been used by many testers as patches against -STABLE and -CURRENT over the past eight months), but there are pending issues on SMP machines. Testers most welcome.
A MFC of this feature will probably happen after the end of the 4.7 code freeze.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPSEC), and OpenSSL (through the /dev/crypto device).
OpenSSL 0.9.7 beta 3 was imported and patched with fixes from OpenBSD's source tree. This permits any user-level application that use -lcrypto to - automatically get hardware crypto acceleration. Otherwse the core crypto + automatically get hardware crypto acceleration. Otherwise the core crypto support is stable and has been in production use on -stable machines for several months.
Import of this work into the -current tree has started. A publicly available patch against 4.7 will be released once 4.7 ships. Integration of this work into the -stable source tree is planned for 4.8.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
Recent work focused on increasing performance. Support is still limited to IPv4 protocols, with IPv6 support coded but not yet tested.
Import of this work into the -current tree has started. A publicly available patch against 4.7 will be released once 4.7 ships.
Work is in progress to MFC a number of bug fixes related to vm_map corruption into -stable. This work is probably too involved to make it into the 4.7 release but is expected to - be comitted just after the freeze is lifted. The corruption + be committed just after the freeze is lifted. The corruption in question typically occurs in large-memory systems under heavy loads and typically panics or KPFs (kernel-page-fault's) the machine in a vm_map related function.
The existing SCSI target code has been rewritten. The kernel driver is much simpler, deferring all functionality to usermode and simply passing CCBs to and from the SIM. The supplied usermode emulates a disk (RBC) with IO going to a backing file. It replaces /sys/cam/scsi/scsi_target* and /usr/share/examples/scsi_target.
The code is definitely alpha quality and has known problems on -current although it appears to work ok on -stable. See the included README for how to install and test. Feedback is welcome!
Yet another implementation of Lottery Scheduling devised by Carl Waldspurger et. al. is being developed against FreeBSD -STABLE branch. It is being developed as part of a graduation project in Computer Science at Universidade de Brasília in Brazil. Therefore, other implementations have not yet been verified to avoid plagiarization but will be checked in a later stage of this project searching for better implementation ideas. Currently, part of the necessary scheduling kernel structure has been mapped and work has progressed despite the general lack of kernel documentation. Further outcomes of this project will be a simple documentation of the kernel scheduler structure of -STABLE branch, a port of the Lottery Scheduler to -CURRENT branch and additional implementations of other scheduling disciplines from Carl Waldspurger et. al. Members of the FreeBSD community have been and will continue to be instrumental in both testing and providing feedback for ideas implemented here.
The FreeBSD Brazilian Portuguese Documentation Project is merging with a translation group formed by members of the FUG-BR FreeBSD Brazilian user group. The Brazilian Project decided to become an official group under FUG-BR after receiving continued excellent contributions from them. They have managed to complete the translation of the FreeBSD FAQ which is currently undergoing both proofing and SGML"fication" stages. Work is progressing fast: the Handbook has been half translated and articles are under way. The previous Brazilian Project is proud to become part of such a dedicate group. The contacts above represent the current official contacts for the new translation group. We hope to have at least part of this work ready for the FreeBSD 4.7 Release.
David Xu and I have been working on cleaning up some of the work done in KSE-III and Jonathon and Dan have been working on the userland - interface. The userland libray will be committed soon in a + interface. The userland library will be committed soon in a prototypical state and a working test program using that interface will hopefully accompany it. I have just committed a rework of the run states for kernel threads that simplifies or solves some problems that were being seen recently.
Hopefully in the next few weeks we will be able to run threads on separate processors. The basics of Signal support are presently evolving. Archie Cobbs will also be assisting with some of this work. I have a mail alias for all the developers at kse@elischer.org. It is managed by hand at the moment.
The Release Engineering (RE) Team completed and released FreeBSD 4.6.2. This ``point release'' fixes several important bugs in the ATA subsystem, as well as addressing a number of security issues in the base system that surfaced shortly after FreeBSD 4.6 was released. The release documentation distributed with FreeBSD 4.6.2 contains more details. (Note: Some earlier documents and reports referred to this release as version 4.6.1.) The next release in the 4.X series will be FreeBSD 4.7, which has a scheduled release date of 1 October 2002.
Concurrently, work is continuing on the 5.0-DP2 developer preview snapshot, an important milestone along the release path of FreeBSD 5.0, which is scheduled for release on 20 November. As 5.0 draws closer, we are focusing more on getting the system stabilized, as opposed to adding new functionality. To help us with this effort, developers should discuss with us any new features planned for -CURRENT, beginning 1 October.
The project runs as it should be. New security-branch snapshots are available for both 4.5 and 4.6(.2). I've update buildboxes OS to the latest 5-current/4-stable without any errors. Also current problem, less CPU power for the future, is not solved yet -- but situation is not so bad, I hope I'll show a good news in the next report.
The Donations team started rolling in the last couple of months. Offers of equipment are coming in, and we are allocating them to FreeBSD committers as quickly as possible. We now have a "Committer Want List" available in our section of the Web site. Several small items, such as network cards, have been routed to people who are willing to write the code to support them. We have a few larger donations (i.e., actual servers) ready to go to developers, once shipping information is straightened out.
Work on RAIDFrame stalled for quite a bit, then it picked up in early summer, then it stalled, and now it's going again. A significant amount of work has been done to make the locking SMPng-friendly and to cut down on kernel stack abuse. I'm happy to say that it's starting to work reliably when used with file- backed 'md' disks. Even more exciting is that it's finally starting to work on real disks, too. A lot of cleanup is still needed, and a few gross hacks still exist, but it might actually be ready for the FreeBSD 5.0 release. Patches for FreeBSD 5-current and 4-stable are available from the website. The 4-stable patches are a year old but still apply and perform well.
The primary libh development box, where the CVS repo and development webpage was living, is dead. The server has crashed after a system upgrade and has never came back to life. We had to pull the drives out of it to make proper backups. We will setup another box in place of this one and hope for the best. So right now, the port is broken because the CVS is unaccessible, as the development web page. We're working on it, please bear with us.
On a brighter note, Max started implementing the changes he proposed to the build system and the TCL API; LibH is switching to SWIG for its TCL bindings, which should simplify the system a lot, and shorten build times. The Hui subsystem is therefore being completely re-written. On my side, I made a few tests in building and running LibH under rhtvision, and it didn't fulfill the promises I thought it would, so I just put aside that idea. Work on libh stalled during July because I completely lost network access for the whole month. So right now, LibH is in a bit of a mess, but we have high hopes of settling everything down to a new release pretty soon, which will make full use of the new SWIG bindings.
The Security Team continues to be very busy. The security-officer mailing list traffic for the months of June, July, and August consisted of 1,230 messages (over 13 messages a day). This is well over 50% of the freebsd-hackers traffic volume in the same period!
Since June (the time of our last report), 9 new Security Advisories were published, and one Security Notice was published covering 25 Ports Collection issues.
FreeBSD 4.6.2-RELEASE was released on August 15th. This marked the first time a point release was created from the security branch. The process went smoothly from the Security Team perspective, despite a schedule slippage due to newly discovered bugs, and a snafu which resulted in 4.6.1-RELEASE being skipped.
In September, the FreeBSD Security Officer published a new PGP key (ID 0xCA6CDFB2, found on the FTP site and in the Handbook). This aligned the set of those who possess the corresponding private key with the membership of the security-officer alias published on the FreeBSD Security web site. It also worked around an issue with the deprecated PGP key being found corrupted on some public key servers.
It's been a busy few months, with a variety of development, documentation, and public relations activities. The MAC Framework, our pluggable kernel access control mechanism for FreeBSD, has matured substantially, and large parts of it were merged to the main FreeBSD tree over July and August.
A variety of entry point changes were made, including: component - names are now passed to VFS namespace VOPs; agressive caching + names are now passed to VFS namespace VOPs; aggressive caching of MAC labels in vnodes; mmap memory access downgrades on subject relabel; check for access()/eaccess(); checks for vnode read, write, ioctl, pool, permitting revocation post-open() by aware policies; labeling and access control checks for pipe IPC objects, clean up of socket/visibility checks; checks for socket bind, connect, listen, ....; many locking improvements and assertions, especially for vnodes, processes; framework now supports partial label updates on subjects and objects; credential management in 'struct file' improved so that active_cred and file_cred are more carefully distinguished and passed to MAC framework explicitly; accounting system uses cached credentials for write operations now; socreate() can use cached credential to label sockets fixing deferred nfs socket connections and reconnections with TCP; kse interactions with proc1 fixed; IO_NOMACCHECK flag to vn_rdwr() for internal use to avoid redundant or incorrect MAC checks on aio vnode operations; mac_syscall() policy function demux; su no longer changes MAC labels by default; mac_get_pid() to support ps and getpmac -p pid; mmap revocation defaults to "fail stop"; MAC_DEBUG wraps atomic label counters; UFS2 extended attributes supported; initial port of LOMAC to the MAC framework; update all policies for all these changes; merge of KSE III; merge of nmount(); upgrade of ugidfw to speak user and group names; libugidfw; many namespace and naming consistency improvements; module dependencies on MAC framework; large scale merging of MAC functionality to the main FreeBSD tree. KDE interfaces to common management activities.
Wrote and taught full-day MAC framework tutorial at STOS - BSD and Darwin Security Symposium; first draft of MAC fraemwork + BSD and Darwin Security Symposium; first draft of MAC framework architecture and API guide. This is now in the Developer's Handbook.
Next couple of months will bring continued maturity improvements, labeling and protection of more objects; VFS performance - improvements; better support for UFS2 EAs and seperate EA + improvements; better support for UFS2 EAs and separate EA entries for each policy; improved support for LOMAC; MLS compartments; IPsec security association labeling; improved SEBSD FLASK/TE port; and much more.
Another busy pair of months at the FreeBSD Project have brought substantial maturity and feature completeness to the fledgeling 5.0-CURRENT branch. And just in time too, because by the time you read the next status report, we hope that you'll have FreeBSD 5.0 running on your desktop! Over the past two months, we've seen an upgrade of sparc64 to Tier 1 (Fully Supported) status, integration of a high quality storage encryption module, the commit of hardware-accelerated IPsec support, the addition of a general-purpose "Device Daemon" to process hardware attach/detach events to replace earlier single-purpose and bus-specific daemons, the commit of RAIDFrame, and the improved maturity of the TrustedBSD work. We've also seen another successful release of the 4.x branch, 4.7-RELEASE, which will continue to be the production supported platform as 5.X is brought in for landing.
-Over the next two months, the FreeBSD Project will be focussed +
Over the next two months, the FreeBSD Project will be focused almost entirely on making 5.0 a success: improving system stability and performance, as well as increasing the pool of applications that build and run on 5.0. The Release Engineering team will have announced the 5.0 code freeze, and released DP2 by the time you read this. Following DP2 will be a series of Release Candidates (RC's), and then the release itself. If you're interested in getting involved in the testing process, please lend a hand -- a spare box and a copy of the DP and RC ISOs burnt onto CD will make a difference. The normal caveats associated with pre-release versions of operating systems apply! You may also be interested in reading the Early Adopter's guide produced by the Release Engineering team to help determine when a transition from the 4.x branch to the 5.x branch will be appropriate for you and your organization.
Thanks,
Robert Watson, Scott Long
I'm very pleased to announce that another engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20021104.tar.gz
This release features minor bug fixes and new OpenOBEX library port. The snapshot includes support for H4 UART and H2 USB transport layers, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices. Also there are several man pages.
Service Discovery Protocol (SDP) port has been updated to version 0.8. (ported from BlueZ-sdp-0.8). Most of the RFCOMM issues have been resolved and now rfcommd works with Windows (3COM, Xircom and Widcomm) and Linux stacks.
New supported USB device - EPoX BT-DG02 dongle. Also I have received successful report about Mitsumi USB dongle and C413S Bluetooth enabled cell phone (L2CAP and SDP works, waiting on RFCOMM report).
I'm currently working on OBEX server (Push and File Transfer profiles) which will be based on OpenOBEX library (included in the snapshot).
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
October 10, 2002 marked the one year anniversary of our project. During that time we have made significant advances in FreeBSD's standards conformance. FreeBSD 5.0-RELEASE will be the showcase for most of our hard work. We hope that our tireless effort has had a positive effect on FreeBSD and software vendors that maintain or are considering porting their software to FreeBSD.
On the API front, _Exit(3) (an alias for _exit(2)) was added, sysconf(3) was update for POSIX.1-2001, and some of the glob(3) additions were MFC'd. The insque(), lsearch(), and remque() family of functions were reimplemented and moved to libc from libcompat. Several wide character functions were implemented, including all printf() and scanf() variants. Finally, support for wide character format types (%C, %S, %lc, %ls) were added to printf(3).
Work on utility conformance continued as getconf(1)'s compliance was updated, c99(1) (a new version of c89(1)) was implemented, and cd(1) and command(1) changes were MFC'd.
Almost 20 headers were brought up to conformance with applicable standards. Not much work remains to fix conformance issues in the remaining standard headers. Work in this area, as well as others, has slowed down in preparation for 5.0-RELEASE.
DEVD has been integrated into FreeBSD current. It was integrated in an incomplete state. However, it is useful in the state that it is in for doing simple things like running camcontrol rescan when a SCSI pcmcia card is inserted, or running /etc/pccard_ether with an ethernet card is inserted. The more sophisticated regular expression matching is not yet complete. Devd only does actions on device arrival and departure, but does not yet do anything with unknown devices. In addition to listening for device events, there is some desire to have /dev/devctl also allow for some direct control of the device tree.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
This work was committed to -current. To configure it for use specify options FAST_IPSEC in your system configuration file. At present support is limited to IPv4.
GBDE has been committed to -current.
The "Geom Based Disk Encryption" module provides a mechanism for very strong encryption of a GEOM "disk". The algorithm has passed informal review by a couple of seasoned crypto heavy-weights. Any GEOM device can be protected with GBDE, entire physical disks, - MBR slices, BSD paritions etc etc. Booting from an encrypted - partition is not possible however.
+ MBR slices, BSD partitions etc etc. Booting from an encrypted + partition is not possible, however.The focus of GBDE is to protect a "cold" disk media. (FreeBSD is not equipped well for protecting key material on a running system from being compromised.) For a cold media, the only feasible attack on a GBDE protected media is guessing the pass-phrase.
Summary of the GBDE multilevel protection scheme: Up to four separate pass-phrases can unlock their own separate copies of the 2048 bit masterkey. The master-keys are protected using AES/256/CBC keyed with a SHA-2 hash derived from the pass-phrase. A salted MD5 hash over the sectoroffset "cherry-picks" which masterkey bytes participate in the MD5 hash which generates the "kkey" for each particular sector. The kkey AES/128/CBC encrypts the PRNG produced single-use key which AES/128/CBC encrypts the actual sector data.
GBDE has features for master-key destruction and pass-phrase invalidation.
See gbde(4) and gbde(8) for more details.
This software was developed for the FreeBSD Project by Poul-Henning Kamp and NAI Labs, the Security Research Division of Network Associates, Inc. under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS research program.
The GEOM code is now the default on most (if not all ?) architectures and the few remaining issues in libdisk/sysinstall is being hashed out.
Although we are far from finished developing GEOM, its current feature set is a significant step forward for FreeBSD, providing not only immediate relief for new architectures (sparc64, ia64 etc) but also because it is designed as SMPng code from the start.
This software was developed for the FreeBSD Project by Poul-Henning Kamp and NAI Labs, the Security Research Division of Network Associates, Inc. under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS research program.
These last two months have seen quite a lot of GNOME activity.
GNOME has started releasing development snapshots of the upcoming
GNOME 2.2 desktop. FreeBSD porting has begun outside of the
main ports tree in the
Evolution 1.2 is also close at hand. Ximian has posted its first release candidate, 1.1.90, which has been ported to FreeBSD, and is available from the MarcusCom CVS repo listed above. As soon as Ximian officially releases Evolution 1.2, it will be placed in the FreeBSD ports tree.
The Mozilla ports have received numerous updates. We are now tracking all three released Mozilla versions. The mozilla-vendor port is tracking the 1.0.x branch, mozilla is tracking 1.1.x, and mozilla-devel is tracking 1.2.x. The mozilla-devel port now has support for anti-aliased fonts as well as a GTK+-2 interface
Finally, the GNOME team would like to welcome its newest team member, Adam Weinberger. Adam has been submitting patches for both GNOME ports as well as documentation. Currently, he has been active in the GNOME 2.2 porting effort. We are happy to have him.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPsec), and OpenSSL (through the /dev/crypto device).
This work was committed to the -current tree. To configure it for use - specifiy device crypto in your system configuration file or you can load the + specify device crypto in your system configuration file or you can load the crypto module. The /dev/crypto device support is brought in with device cryptodev or by loading the cryptodev module. Two crypto device drivers exist: ubsec for Broadcom-based PCI hardware and hifn for Hifn-based PCI hardware.
Integration of this work into the -stable source tree should be completed by the time this report is published.
Since the last status report the BSD Java Porting Team has continued to make steady progress. The most exciting news we have is courtesy of our newest team member, Alexey Zelkin of FreeBSD committer fame.
For 4.7-RELEASE, we privately published package ja-man-doc-4.7.tgz which consists of man[1256789] entries 10 days after the 4.7-RELEASE release date. Man3 update god no progress, as updating other sections busied us. We decided to suspend man3 update officially, as we need to spend most of our time to catch up with the forthcoming 5.0-RELEASE.
The KDE/FreeBSD team has been working on two major goals during the last two months, Maintenance of the KDE 3.0.x ports and Preparing the upcoming KDE 3.1 Release.
Maintenance KDE 3.0 conducted by Alan Eldrige: September started with the Removal of the KDE 2.x Ports from the FreeBSD-Repository. Later Packages of KDE 3.0.4 were released and the FreeBSD Ports were updated.
Preparing for KDE 3.1 conducted by Will Andrews: A lot of effort was spent on Improving the Fruitsalad-Build-System. We are now able to create packages directly from the KDE CVS.
The KSE code has now all the basic kernel functionality to start being used by the userland. There are still things - to be done for testing and familiarisation.
+ to be done for testing and familiarization.General system utilities have not yet been changed. e.g. ps and top etc. need to know about threads.
There is quite a lot of code in the kernel that still assumes that there is one thread in a process. Signals are not yet handled in the final manner (though they are delivered to a random thread in the process :-/ ).
The system calls and datastructures are now however in place. The test program successfully starts several threads that can be scheduled on different processors, and closes them down again. The userland is probably going to be able to do simple scheduling of pthread threads using KSE by the time that this report is published.
I still need someone to take over the "official" web page - since jason left. LaTex sure isn't my thing.
+ since jason left. LaTeX sure isn't my thing.Not much since the last status report, except that we now have the repo and development web page back online, thanks to the services of John De Boskey who freely provided the necessary hardware and bandwidth to host the project. We have also ported LibH to GCC 3.x, so that it can compile on -CURRENT correctly. This, however, broke tvision, which doesn't compile under GCC 3.x, so we moved to rhtvision but this caused linking problems so we're stuck with no console front end, for now.
Work on a Hui rewrite and SWIG bindings stalled. Alex was able to come up with a simple patch to make the ports system use LibH's pkg_create script to build libh packages, so we're getting closer to a real pkg_create(1) drop-in replacement. I rewrote the milestone list to show a bit more relevant and encouraging tasks that will be dealt with in order to really push LibH forward.
A mailing list was created, freebsd-mips, and a Perforce branch was created in //depot/projects/mips. Changes which will be necessary to allow multiple MIPS (and PowerPC) metaports to exist under one architecture port were made, and are being pushed back into the main FreeBSD tree. Some preliminary header work has been done, and porting the ARCBIOS interfaces to the kernel has begun. The toolchain in tree was updated and modified in places to support a FreeBSD/MIPS (Big Endian) target, in the Perforce branch. Some early boot code has proven the GDB MIPS simulator to work, for at least R3000 code, though whether R3000 will be supported has been under discussion. Some initial architectural decisions were also made, to steer current work.
Work on newcard continues. A number of bugs have been fixed in the last few months. You are now able to load and unload drivers (including the bridge) to test changes to pccard and/or cardbus bus code. It is now possible to load a driver that has a pccard attachment and have a previously inserted card probe and attach. This is also true for CardBus. A number of issues remain to be solved before 5.0. However, with the integration of devd into the tree nearly all of old functionality of OLDCARD is now present in NEWCARD (the biggest remaining parts are power control for the sockets, as well as pccardc dumpcis).
The PowerPC port has been running diskless on NewWorld G3/G4 machines for a while now. A GEOM module to support Apple Partition Maps is being written. There should be an installable ISO image available in the near future.
RAIDFrame was imported into FreeBSD-current in late October, a major milestone after 18 months. It is still very experimental and not suitable for production environments. The website contains a lengthy TODO list which I hope to start attending to soon. Still, I encourage everyone to try it out and report bugs back to me.
The Release Engineering (RE) team completed and released FreeBSD 4.7 on 10 October 2002. This release features updates for a number of contributed software programs in the base system, as well as all of the security and bug fixes from FreeBSD 4.6.2. The next release in the 4.X series will be FreeBSD 4.8, which has a scheduled release date of 1 February 2003.
Before that time, however, will be the release of FreeBSD 5.0. Thus far, we have not been able to release the 5.0-DP2 developer snapshot due to various stability issues. Thanks to much effort from many of our fellow developers, we believe that most of these have been resolved. The RE team wishes to emphasize that FreeBSD 5.0 will involve new code and features that have not seen widespread testing, and that more conservative users may wish to continue to track the 4.X series for the near-term future. To provide more information on these issues, we have added an Early Adopter's Guide to the release documentation for 5.0.
Brian Somers has resigned from the RE team due to increased time pressures. We thank him for all of his help with FreeBSD 4.5, 4.6, 4.6.2, and 4.7, and we hope to continue working with him as a fellow developer.
Scott Long has graciously offered to help improve the communication between the RE team and the rest of the developer community. We greatly appreciate his assistance.
Recent 5-current release procedure troubles prevent the project from releasing a new snapshots. But 5-current FreeBSD/i386 release is back again in late Oct/2002! I have a plan to build daily FreeBSD/sparc64 snapshots for 5-current. Stay tuned...
A lot has happened recently for the sparc64 port. Sysinstall and make release work and can be used to build installable snapshots. The gdb5.3 port now works, and, thanks to Thomas Moestl, kernel crash dumps are supported which can be analyzed by gdb. These 2 items are the last things considered necessary by the Core team for FreeBSD/sparc64 to be a Tier 1 architecture, which means that 5.0-RELEASE for sparc64 will be officially supported by the release engineering team and by the security officer team.
Recently Jake Burkholder has been working on alternate installation methods other than bootable iso, including a mini-root filesystem which can be written to the swap partition of an existing machine. Thomas Moestl has been putting some finishing touches on the release process, ensuring that the release documentation can be built properly, and that the port readme files can be generated by the release process.
An experimental iso built with make release is now available on the freebsd ftp site and mirrors in /pub/FreeBSD/development/sparc64/5.0-20021031-SNAP. It is expected that by the middle of November new 5.0-SNAP releases will be available every few days for download and for ftp install, cpu power and bandwidth permitting.
Most progress on TrustedBSD over the last two months related to improving the maturity of the ACL and MAC implementations, and merging new aspects of those features into the primary FreeBSD CVS Repository for inclusion in FreeBSD 5.0. This included fixes to run better on sparc64, improved tuning of what system objects are mediated, locking fixes and optimizations especially relating to the vnode and pipe implementations, improved support for MAC labeling on symlinks, support for asynchronous process label changes as required in some locking situations, remove use of "temporary labels" and prefer use of object type specific labels reducing redundant and/or confusing label management code in policies, improve avoidance of memory allocation in M_NOWAIT scenarios for socket allocation in the syncache, mediation of link operations, race condition fixes for devfs involving label creation, improve handling of VM events such as mmaping, improve mediation of socket send/receive events (as distinguished from socket transmit/deliver events), support for manipulating EAs on symlinks using new system calls, support for MNT_ACLS and MNT_MULTILABEL flags at mount time, as well as FS_ACLS and FS_MULTILABEL superblock flags to key useful defaults using tunefs, correction of a memory leak in the UFS ACL code, enable UFS ACL support by default in GENERIC, mediation points for file creation, deletion, and rename, support for a mac_execve() execution interface in the style of SELinux's execve_secure() permitting a label transition request as part of the exec operation for policies that support it, more consistent handling of NFS lookups, support for labeling of multicast encapsulated packets, ATM packet labeling, FDDI packet labeling, STF packet labeling, revised label interface that avoids userland parsing of per-policy elements, reducing us to a single instance of parsing and printing for each policy (and further abstracting policy implementation details from the library code).
Also, change to single-level sockets for Biba and MLS policies, support for partial label updates for Biba and MLS, addition of mac.9 man page, revised user API system calls, implementation of mac_get_pid(), and various other related bits, creation of mac.conf(5) to specify label defaults, checks for various system operations including swapon(), settime(), and sysctl(), reboot(), acct(), introduction of command line utilities for maintaining file and process labels, support for user labels tied to login class, su support for label changes, ifconfig support for interface labels, ps support for process labels, ls support for file labels, ftpd support for login labels, development of the Biba and MLS notions of privilege, and a move to C99 sparse structure initialization, restoring full type checking for policy entry points.
Universally Unique Identifiers (UUIDs) are 128 bit values that may - be generated independently on seperate nodes (hosts), which, result in + be generated independently on separate nodes (hosts), which result in globally unique strings. UUIDs are also known as Globally Unique Identifiers (GUIDs). The UUID support for FreeBSD (libc) conforms to the DCE 1.1 RPC specification.
-UUID suport has been added to FreeBSD -CURRENT, and will be available +
UUID support has been added to FreeBSD -CURRENT, and will be available in version 5.0. It is being extensively used in GPT partition handling for IA-64 platform. For now, a simple manual page has been provided, which outlines information about the provided uuid routines. Many documentation additions and enhancements to uuidgen(1) are in the pipeline.
The goal of this project is to improve the wireless networking support in the system. The initial work will incorporate the 802.11 link layer done by Atsushi Onoe for NetBSD. This core support code implements the basic 802.11 protocols required for Station and AP operation in BSS, IBSS, and Ad Hoc modes of operation. Wireless device drivers will then be revised to use this common code instead of their private implementations.
Following this initial stage the wireless networking support will be extended to support functionality needed for workgroup, enterprise, and metropolitan (e.g. mesh) networking environments. This will include full power management support, the 802.1D spanning tree protocol for running multiple AP's in a bridged configuration, QoS support, and enhanced security protocols (LEAP, AES, EAP). Support for new hardware devices is also planned.
At long last, FreeBSD 5.0 is here. Along with putting the final polish on the tree, FreeBSD developers somehow found the time to work on other things too. IA64 took some major steps towards working on the Itanium2 platform, an effort was started to convert all drivers to use busdma and ban vtophys(), hardware crypto support and DEVD hit the tree, NewReno was fixed and effort began on locking down the network layer of the kernel. Also high performance, modular scheduler started taking shape and will be a welcome addition to the kernel soon.
Looking forward, the focus will be on stabilizing and improving the performance of 5.0. The RELENG_5 (aka 5-STABLE) branch will be created once we've reached our goals in this area, so hopefully we will get there quickly. Meanwhile, preparations for the next release from the 4.x series, 4.8, will begin soon. Of course, the best way to get 5.x to stabilize os to install and run it!
Thanks,
Scott Long, Robert Watson
I'm very pleased to announce that all kernel modules and few userland tools made it to the FreeBSD source tree. Many thanks to Julian Elischer.
Unfortunately no big changes since the last report. Some minor problems have been discovered and patches are available on request. I will prepare all the patches and submit them to Julian for review.
OBEX server and client (based on OpenOBEX library) is almost complete. I'm currently doing interoperability testing. If anyone has hardware and time please contact me. The HCI security daemon has been implemented and tested with Sony Ericsson T68i cell phone and Windows stack. It is now possible to setup secure Bluetooth connections.
A few people have complained about RFCOMM daemon. These individuals want to use GPRS and Bluetooth enabled cell phone to access Internet. If you have this problem please contact me for possible workaround. My next goal is to get robust RFCOMM implementation to address all these issues.
Largely bug-fixing and userland application tweaks; new interfaces were added to manipulate ACLs on extended attributes; bugs were fixed in ls relating to ACL flagging. Patches to teach cp, mv, gzip, bzip, and other apps about ACL preservation are in testing and review. tunefs flags were added to ease configuration of ACLs, especially on UFS2 file systems.
Possible changes to make use of Linux/Solaris umask semantics are under consideration: right now we implement verbatim POSIX.1e/IRIX merging of the umask, ACL mask, and requested creation mode during file, device, fifo, and directory creation. Solaris and the most recent Linux patches ignore the umask in the context of a default ACL; this requires some rearrangement of umask handling in our VFS, although the results would be quite useful. We're exploring how to do this in a low impact way.
Framework changes:
Instrument KLD system calls (module and kld load, unload, stat) Instrument NFSd system call. Instrument swapoff(2). Instrument per-architecture privileged parts of sysarch(). Make use of condition variables to allow callers to wait for the framework to "unbusy" when loading/unloading policies, rather than returning EBUSY. Store mount pointer in devfs_mount structure for use by policies. Improve handling of labels in loopback interface "re-align" packet copy case. Provide full paths on devfs object creations to help policies label them properly (not merged). Experimentation with moving MAC labels into m_tags (not merged). NFS server now uses real ucreds, not hacked up ucreds, meaning we can start laying the groundwork for enforcement on NFS operations. (not merged)
Policy changes
LOMAC: mac_lomac replaces lomac (LOMAC now uses the MAC Framework), SEBSD: Improved support for devfs labeling based on SELinux genfs. Handling of hard link checks. Support export of process transition information for login and others using sysctl. Login now prompts for roles. Allow policy reload. TTY labeling. Locking adaptation from Linux. Many, many policy adaptations and fixes. We can now boot in enforcing mode! mac_bsdextended: fix a bug in which VAPPEND wasn't mapped to VWRITE, so opens with the O_APPEND bug failed improperly.
Userland changes
setfmac(8) now supports a setfsmac(8) execution mode, which accepts initial labeling specification files. Supports an SELinux compatibility mode so it can accept SELinux label specfiles using the SEBSD module. sendmail(8) now sets user labels as part of the context switch for mail delivery.
Documentation changes
Man page updates for MAC command line tools, modules, admin hints, etc. Updates to the FreeBSD Developer's Handbook chapter on MAC policies and entry points. MAC section in FreeBSD Handbook.
This project has been coming along pretty well. The amd(4) and xl(4) drivers have now been converted to use the busdma API, sparc64 got the bus_dmamap_load_mbuf() and bus_dmamap_load_uio() functions, and the gem(4) and hme(4) drivers have been updated to use bus_dmamap_load_mbuf() instead of bus_dmamap_load().
A lot more still needs to be done, as shown on the project's page. A fair number of conversions are on their way though, and we can expect a fair number of drivers to be converted soon, thanks to all the developers who are working on this project.
The POSIX Utility Conformance in FreeBSD list (link above) has been updated to reflect current reality. Not much work remains to complete base utility conformance.
On the API front, grantpt(), posix_openpt(), unlockpt(), wordexp(), and wordfree() were implemented. The header <wordexp.h> was added.
There are currently about 40 unassigned tasks on our project's status board ranging from documentation, utilities, to kernel hacking. We would encourage any developers looking for something to work on to check out the status board and see if anything interests them.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPsec), and OpenSSL (through the /dev/crypto device).
This work will be part of the 5.0 release and has been committed to the -stable source tree for inclusion in the 4.8 release.
Recent work has focused on improving performance. System statistics are now maintained and an optional profiling facility was added for analyzing performance. Using this facility the overhead for using the crypto API has been significantly reduced.
The ubsec (Broadcom) driver was changed to significantly improve performance under load. In addition several memory leaks were fixed in the driver and the public key support was enabled for use.
Upcoming work will focus on load-balancing requests across multiple crypto devices and integrating OpenSSL 0.9.7 which will automatically enable application use of crypto hardware.
Devd has been integrated into FreeBSD 5.0-RELEASE. The integrated code supports a range of configuration options. The config files are fully parsed now and their actions are performed.
-Future work in this area are likely to be limited to imporving +
Future work in this area is likely to be limited to improving the devctl interface. /dev/devctl likely will be a cloneable device in future versions. Individual device control via devctl is also planned.
The Donations project expedited several dozen donations during 2002, and was able to place most of what was offered. We still are in dire need of SMP and Sparc systems. You can see information on our needs and donations that have been handled by the team on the donations web page.
We are relying increasingly upon the developer wantlist to place items offered to the Project, and using the commit statistics to help place items. As such, active committers who ask for what they want beforehand have a decent chance of getting it. Less active committers, and committers who do not ask for what they want, will be lower in our priorities but will not be excluded.
We are in the process of streamlining the tax deduction process for donations, and hope to have news on that shortly. We are also always working to accelerate and reduce our internal processes, to get the most equipment in the hands of the most people as quickly as possible.
I especially want to thank David O'Brien and Tom Rhodes for stepping up and making the team far more successful. Also, the FreeBSD Foundation has been quite helpful in handling tax-deductible contributions.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
This work will be part of the 5.0 release. Performance has been improved due to work on the crypto subsystem.
The goal of the project is to use a small amount of space in the FFS superblock to store a volume label of the user's choice. A GEOM module will then expose the volume labels into a namespace in devfs. The idea is to make it easier to manage filesystems across disk swaps and movement from system to system.
At this point, everything pretty much works. I've submitted parts of the patch to respective subsystem maintainers for review. There are some issues with namespace collision that I haven't addressed yet, but the basic functionality is there
Most of the articles are translated too. Marc is still translating the handbook, 60% is currently translated. Stéphane has began the integration of our French localization web site in the US CVS Tree. Sébastien is still maintaining the Release Notes.
We launched a new site, www.FreeBSD-fr.info, consisting in a French - Dameon News like site. Netasq have donated our new server; we will + Daemon News like site. Netasq have donated our new server; we will install it in a new hosting provider in the few next weeks. One of the - big job now, project now, is the translation of the FAQ, and the big - project will be the manual pages
+ big job now is the translation of the FAQ, and the big + project will be the manual pages.Since the ports tree has been frozen for most of this reporting period,
there have not been too many GNOME updates going into the official CVS
tree. However, development has not stopped. GNOME 2.2 is nearing
completion, and quite a few FreeBSD users have stepped up to test the
GNOME 2.1 port sources from the
The upcoming FreeBSD 5.0-RELEASE will be the first release to have the GNOME 2.0 desktop as the default GNOME desktop choice. During the previously mentioned ports freeze, all the GNOME 2 ports were fixed up so that they build and package on both i386 and Alpha platforms. Alas, the one port that will not make the cut for Alpha is Mozilla. There are still problems with the xpcom code, but work is ongoing to get a working Alpha port.
Finally, the FreeBSD Mono (an OpenSource C# runtime) port has also received some new life. Mono has been updated to 0.17 (the latest released version), and Juli Mallett has ported gtk-sharp (GTK+ bindings for C#).
The ia64 port is up and running on the new Itanium2 based hp machines thanks to a lot of hard work by Marcel Moolenaar. So far we are running on the hp rx2600 as these were the machines graciously donated by Hewlett-Packard and Intel. We had a prototype Intel Tiger4 system for a while, but we had to return the machine and we do not know if it currently runs. Most of the changes necessary to run these are sitting in the perforce tree and are not in the -current or RELENG_5 cvs tree. As a result, the cvs derived builds (-current and the 5.0-RC series - and presumably 5.0-RELEASE) are only useable on obsolete Itanium1 + and presumably 5.0-RELEASE) are only usable on obsolete Itanium1 systems.
Lots of other stability and functionality fixes have been made over the last few months, including initial libc_r support. The OS appears to be stable enough for sustained workloads - it is building packages now, for example. We still do not have gdb support, even for reading core files.
We have been updating our Japanese translated manual pages to RELENG_5 based. All existing entries have been updated, but 15 exceptions are not, most of which require massive update. We will also need to add translations which did not exist on RELENG_4.
KGI (Kernel Graphic Interface) is a kernel infrastructure providing user applications with means to access hardware graphic resources (dma, - irqs, mmio). KGI is already available under Linux as a seperate + irqs, mmio). KGI is already available under Linux as a separate standalone project. The KGI/FreeBSD project aims at integrating KGI in the FreeBSD kernel.
KGI/FreeBSD has been recently donated 2 PCI graphic cards (Matrox Millenium II and a coming Mach64) and other have been proposed. Please see the FreeBSD web pages for details. Thanks to donation@ for organizing and promoting donations. Thanks to the donators for their contribution to KGI/FreeBSD.
KGI/FreeBSD progressed fine the last months. Most of the VM issues for mapping HW resources in user space have been addressed and a first attempt of coding was made. This prototyping raised some API compatibility problems with the current Linux implementation and was discussed heavily on the kgi devel lists. Ask if you're interested in such issues, I'll be pleased to share them.
Most of coding is now done. Let's start debugging!
Work is ongoing to continue to lock up the network stack. Recently, the focus has been on the IP stack. The plan there involves a series of inter-related pieces to lock up the ifaddr ref count, the inet list, the ifaddr uses, the ARP code, the routing tree, and the routing entries. We are over 3/5 of the way done down this path.
In addition to TCP and UDP, the other networking protocols such as raw IP, IPv6, AppleTalk, and XNS need to be locked up. Around 1/4 these remaining protocols have been locked and - will be commited after the IP stack is locked.
+ will be committed after the IP stack is locked.The protocol independent socket layer needs to be locked and operating correctly with the protocol dependent locks. This part is mostly done save for much needed testing and code cleanup.
Finally, a pass will be need to be made to lock up the devices drivers and various statistics counters.
This effort fixes some outstanding problems in our TCP stack with regard to congestion control. The first item is to fix our NewReno implementation. Following that, the next urgent correction is to fix a problem involving window updates and dupack counts. When that stabilizes, we will then change the recovery code to make use of SACK information. Eventually, this project will update the BSD stack to add Limited Transmit and other new internet standards and standards-track improvements.
The 3 FreeBSD package clusters (i386, alpha, sparc64) have been unified to run from the same master machine, instead of using 3 separate masters. This has freed up some machine resources to use as additional client machine, as well as simplifying administrative overheads. Build logs for all 3 architectures can now be found on the http://bento.freebsd.org webpage. The sparc64 package cluster now has 3 build machines (an u5 and two u10s), and an ia64 cluster is about to be created.
Package builds now keep track of how many sequential times a port has failed to build (html summaries are available on the bento website). This allows tracking of ports which have suddenly become broken (e.g. due to a bad upgrade, or due to changes in the FreeBSD source tree), and in the future will be used to send out notifications to port maintainers when their port fails to build 5 times in a row. This feature is currently experimental, and further code changes will be needed to stabilize it.
The goal of this project is to improve the wireless networking support in the system. By the time of this report the 802.11 link layer code should be committed. A version of the wi driver that uses this code should be committed shortly. Conversion of other drivers is planned as are drivers for new devices.
Support for 802.1x/EAP is the next planned milestone (both as a supplicant and authenticator).
November and December were especially busy for the release egineering +
November and December were especially busy for the release engineering team. Scott Long joined the team to help with secretary and communications tasks while Brian Somers bowed out to focus on other projects.
FreeBSD 5.0-DP2 was released in November after much delay and anticipation, and marked the final milestone needed for 5.0 to become a reality. Shortly after that, we imposed a code freeze on the HEAD branch of CVS and released 5.0-RC1. Creation of the RELENG_5_0 branch came next, followed by the release of 5.0-RC2 from this branch. At this point, enough critical problems still existed that we scheduled an RC3 release for the new year, and pushed the final 5.0-RELEASE date to mid-January. By the time this is published, FreeBSD 5.0-RELEASE should be a reality.
For the time being, there will not be a RELENG_5 (aka 5-STABLE) branch. FreeBSD 4.x releases will continue, with 4.8 being scheduled for March 2003. Release in the 4.x series will be lead by Murray Stokely, and releases in the 5.x series will be lead by Scott Long. Once HEAD has reached acceptable performance and stability goals, the RELENG_5 branch will be created and HEAD will move towards 6.0 development. We hope to reach this with the 5.1 release this spring.
A new scheduler will be available as an optional component along side the current scheduler in the 5.1 release. It has been designed to work well with KSE and SMP. Some ideas have been borrowed from solaris and linux along with many novel approaches. It has O(1) performance with regard to the number of processes in the system. It also has cpu affinity which should provide a speed boost for many applications.
The scheduler has a few loose ends and lots of tuning before it is production quality although it is quite stable. Please see the post to arch and subsequent discussion for more details.
Another busy two months have passed in the FreeBSD project. With 5.0 released, attention is focusing on making it faster via more fine-grained locking, adding more high-end features like large memory (PAE) support for i386, and further progress on many other projects. FreeBSD 5.1 is expected to ship in late May or early June, with 5.2 following at the end of summer. A roadmap for the push to 5-STABLE is available at http://www.freebsd.org/doc/en/articles/5-roadmap. Although the 5.x series isn't expected to fully stabilize until the 5.2 release, 5.1 promises to be an exciting release and a significant improvement over 5.0 in terms of speed and stability.
Not to be forgotten, FreeBSD 4.8, the latest in the 4-STABLE series, is nearing release. Lots of last minute work is going into to it to deliver features like XFree86 4.3.0, Intel HyperThreading(tm) support, and of course many more bug fixes. Don't forget to support the FreeBSD vendors and developers by buying a copy of the CD set when it comes out!.
Thanks,
Scott Long, Robert Watson
Large portions of headers have been filled in, all have been stubbed out. Minimal functions and data elements have been stubbed out or filled in. Machinery added to support some requisite tunables for building real kernels. GCC fixed to generate correct local label prefixes making it possible to link real kernels. Work begun on providing enough to create and boot real kernels, on real hardware. Decision to only support MIPS-III and above made.
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
I'm very pleased to announce that another release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20030305.tar.gz
This release features new in-kernel RFCOMM implementation that provides SOCK_STREAM sockets interface. This makes old user-space RFCOMM daemon obsolete. People should not use old user-space RFCOMM daemon any longer. The release features new RFCOMM PPP daemon that supports DUN and LAN profiles. Note: PPP patch (support for chat scripts in -direct mode) is required for DUN support. Look for it in the mailing list archive or contact me directly. People with Bluetooth enabled cell phones can now use them to access Internet.
The Bluetooth sockets layer has been cleaned up. People should not see any WITNESS complaints with new code. Locking issues have been revisited and code in much better shape now, although it probably is not 100% SMP ready just yet. The code should work on SMP system anyway because sockets layer is still under Giant.
The simple OBEX server and client (based on OpenOBEX library) is complete. OBEX File Push and OBEX File Transfer profiles work and have been tested with Sony Ericsson T68i cell phone and Bluetooth 3COM stack on Windows2K. It is now possible to send pictures, address book and calendar entries from the cell phone via Bluetooth. Minor bug in OpenOBEX library has been fixed and OPEX Put-Empty command now works.
Due to changes in API userland tools must be in sync with the kernel. People should install new include files, recompile and reinstall all userland tools as part of upgrade. I'm sorry about that.
The FreeBSD 4.8 Release Process is well underway. The RELENG_4 branch has been under code freeze since February 15, and the first release candidates were made available in early March. A testing guide has been put together and is available from http://www.FreeBSD.org/releases/4.8R/qa.html.
Developers should coordinate with re@FreeBSD.org about any changes they would like to include in this release, and users are encouraged to try out the release candidates and help find as many bugs as possible now, before the final release is made.
FreeBSD 4.8 represents the newest production release from the stable '4.X' branch. It does not include all of the features that were made available in the "new technology" 5.0 release in January.
The doceng@ team is a new body to handle some of the meta-project issues associated with the FreeBSD Documentation Project. The main responsibilities of this team are to grant approval of new doc committers, to manage the doc release process, to ensure the documentation toolchains are functional, to maintain the doc project primer, and to maintain the sanctity of the doc/ and www/ trees. The current members of this team are Nik Clayton, Ruslan Ermilov, Jun Kuriyama, Bruce A. Mah, and Murray Stokely.
The later months have been very busy on KGI. Most of the framework has been debugged for typical usage (fb, no accel). I got KII (the input interface) connected to syscons through atkbd. Opening /dev/graphic works and framebuffer resource access is permitted. Finally, the KGIM (KGI module) framework has a better building tree for board / monitor drivers and board drivers are now loading with resource allocation.
Most important on the TODO list: 5.0-RELEASE move (I currently work with a May-2002 5.0-current). Most of debug is now done. Let's validate!
Note that KGI project homepage has changed since the last report.
We have released Japanese translation of 5.0-RELEASE online manual pages on February 2nd. Most of entries which did not exist on RELENG_4 were not yet translated. I hope we can finish such entries soon.
We have the first disk device driver (aac) out from under Giant now, and in certain scenarios it gives improvements up to 20%. - The device drive API was pruned to reflect that NO_GEOM + The device driver API was pruned to reflect that NO_GEOM compatibility is unnecessary, this resulted in approx 1000 lines less source code, the majority of which were removed from the device drivers. The new API for cdevsw is a lot simpler and hopefully less likely to confuse people. The ability to automatically allocate a device major number has been introduced and is already used by a handful of drivers. Checks introduced with this facility has shown that the uniqueness of manually allocated major numbers had already broken down.
Work continues on the statistics collection API and on a unified API for manual configuration of GEOM nodes.
Support for PAE is mostly complete, and has been checked into the jake_pae branch. The approach that is being taken to add support for PAE is to allow the pmap module to view the page table directory as 4 pages instead of 1, and to avoid using the 3rd level structure, the page directory pointer table, as much as possible. Due to its small size, 32 bytes, the PDPT cannot be uniformly recursively mapped, and as such does not provide a regular multi level structure like the page tables used by the alpha or x86-64 architectures. What remains to be done for PAE support is to develop an API for manipulating page table entries which will allow idempotent 64 bit loads and stores to be used where necessary.
Experimental support for >4G ram using PAE has been developed and checked into the jake_pae_test branch in Perforce. This involved adding a physical address type separate from virtual addresses, for use by the vm system and bus code which needs to use physical addresses directly. Initial testing has shown good results with device drivers that can dma to 64 bit physical addresses.
Funding for this project is being provided by DARPA and Network Associate Laboratories, and hardware support by FreeBSD Systems.
In the period from September 2002 through February 2003, the FreeBSD Security Team email aliases saw 1297 messages, a much smaller volume than over the summer (remember the Apache and OpenSSL worms? 4.6.1 oops I mean 4.6.2-RELEASE?).
Also during this period: 95 items were added to the SO issue-tracking database; 39 of these involved the FreeBSD base system while the rest involved ports. 9 new Security Advisories were published, 2 of which covered issues unique to FreeBSD.
In January, the SO published a new PGP key (ID 0xCA6CDFB2, found on the FTP site and in the Handbook). This aligned the set of those who possess the corresponding private key with the membership of the security-officer alias published on the FreeBSD Security web site. It also worked around an issue with the deprecated PGP key being found corrupted on some public key servers.
In February, Mike Tancsa of Sentex donated two machines to the Security Officer. These have been a great help already in testing the security branches, preparing patches, and generating updated binaries. Thank you very much, Mike!
FreeBSD 4.8-RELEASE will continue in the tradition of 5.0-RELEASE, and include GNOME 2 as the default GNOME desktop. This means that 4.8 will ship with GNOME 2.2.
Following on the heels of the recent GNOME 2.2 release, GNOME 2.3 snapshots are gearing up. The development schedule is available from http://www.gnome.org/start/2.3/. Ports will be made available the same way they were for the 2.1 development releases. Stay tuned to freebsd-gnome@ for more details.
We are currently in another ports freeze in preparation for 4.8-RELEASE. Following the freeze, a new bsd.gnome.mk will be committed that effectively removes the USE_GNOMENG macro. This new version will add support for GNOME 2 as well as setup backward compatibility for ports that have not yet been converted to the new GNOME infrastructure. People interested in testing this new Mk file, can check out the ``ports'' module following the instructions at http://www.marcuscom.com:8080/cgi-bin/cvsweb.cgi.
Work on PowerPC is progressing steadily. The system can now boot multi-user from the net and disk. ATA-DMA is being integrated with the ATAng code, and support for older G3 machines is being added.
January and February were quiet months that saw with them the addition of some C99 math functions and macros, which include: fpclassify(), isfinite(), isgreater(), isgreaterequal(), isinf(), isless(), islessequal(), islessgreater(), isnan(), isnormal(), and signbit(). Additional C99 math library support is in the works.
Most of the file system buffer cache has been reviewed and protected. The vnode interlock was extended to cover some buffer flag fields so - that a seperate interlock was not required. The global buffer queue + that a separate interlock was not required. The global buffer queue data structures were locked and counters were converted to atomic ops. The BUF_*LOCK functions grew an interlock argument so that buffers could be safely removed from the vnode clean and dirty lists. The lockmgr lock is now required for all access to buf fields. This was not strictly followed before because splbio provided the needed protection.
There are a few areas of code that need to be protected and cleaned up before giant can be pushed down. Most notably the background write code is currently unsafe without giant. Also, many of the VM bits that the buffer cache relies on are not safe. This work has been done with the expectation that the VM and VFS subsystems will be giant free soon.
The ULE scheduler has been committed to the 5.0-CURRENT branch. Early adopters and experimenters are welcome to try it and submit bug - reports. It has shown noticable performance improvements over the old + reports. It has shown noticeable performance improvements over the old scheduler under some workloads. There are currently problems with nice fairness but otherwise the interactive performance is very good. More work to improve the load balancing algorithm is required as well. This should be ready for use by the general FreeBSD user base in the next month or so.
Some improvements have been made to the clustered read ahead code. They allow for many more outstanding IO requests when an application does sequential access. This has a larger impact on RAID systems than on single disk systems. The maximum number of file system blocks that we will read ahead is tunable via the 'vfs.read_max' sysctl. This optimization has shown a 20% improvement in simple tests.
Locking of the non-obj parts of newbus is nearing completion. A single lock is used for the device tree. Minimal changes to subr_bus have so far been necessary to make this work, however some lock order issues remain. After this work, it will no longer be necessary to hold Giant to call device_* routines safely. kobj work is being done by others and - will likely require more extensive design work to make smp + will likely require more extensive design work to make SMP friendly.
The objective of this effort is to improve the performance, stability, and correctness of the BSD networking stack by adding support for new standards and standards track proposals while maintaining compliance with existing specifications. The upcoming 4.8 and 5.1 releases will be the first ones using the new NewReno logic. Recently, we implemented the Limited Transmit algorithm (RFC 3042) which benefits - connections with small congestions windows, as happens, for example, + connections with small congestion windows, as happens, for example, on many short web connections. We also recently added support for larger sized starting congestion windows as described in RFC 3390. This helps short TCP connections as well as those with large round-trip delays, such as those over satellite links.
The list of subsystems locked up include IP, UDP, TCP, ifaddr reference counting, syncache, the ifnet list, routing radix trees, and ARP. These have already been committed into the tree. In addition, SMP locking for raw IP, divert socket processing, and Unix domain sockets have also recently been completed and tested. Work is currently being done in some of the subsystems required to make parallel networking processing SMP-safe.
The FreeBSD Project made substantial progress in the month of August, 2001, both on continuing the development of the RELENG_4 line (4.x-STABLE and 4.x-RELEASE), and on 5.0-CURRENT, the main development branch. During this month, the decision was made to push the release of 5.0-CURRENT back so that KSE (support for fine-grained user threads) could be completed in time for the release, rather than postponing that support for 6.0. As such, the lifespan of the RELENG_4 line will be extended, with new features continuing to be backported to that branch. 4.4-RELEASE went into final beta during this month, and will also be available shortly.
This month's edition of the status report has been written with the assistance of Nik Clayton and Chris Costello.
For next month, the submission procedures remain the same: reports should be between one and two paragraphs long, sent by e-mail, and in a format approximately that of this month's submissions (Project, Contact, URL, and text). Reminders will be mailed to the hackers@FreeBSD.org and developers@FreeBSD.org mailing lists at least a week before the deadline; complete submission instructions may be found in those reminders.
-- Robert Watson
2 Gigabit support was integrated on 8/31/2001 (QLogic 2300/2312 cards). Because of the author's shrinking time commitment for FreeBSD, the previously planned "next step" which would have been more complete new CAM Transport integration is now probably just the addition of an FC-IP adjunct (as this can benefit many platforms simultaneously).
A major update to error handling was done on 8/28/2001 which should correct most of the EOM detection problems that have been around for a while. There are several things to fix. The principle thing to fix next is the establishment of a loader(8) mediated device quirks method.
No change since last status. Some discussion amongst all of us occurred, but lack of time and commitment to FreeBSD has meant - little has actually been committed to the tree. SMPNG work will + little has actually been committed to the tree. SMPng work will be left to those who seem to have a notion about what needs to be done.
No new status to report. This driver will be worked on again soon and cleaned up to work better.
Work in adding supporting infrastructure to the kernel for KSE threading support has reached "milestone 2".
Milestone 2 is where the kernel source consistently refers to its resources in terms of per-thread and per-process resources, in the way that it will need to when there are > 1 threads per process, but the LOGICAL changes to such things as the scheduler, and fork and exit, have not yet been made to allow more than one thread to be created. (nor have new threading syscalls been added yet). This is an important milestone as it represents the last point where the kernel has only "mechanical" changes. To go - further we must start adding new algorythms and functions.
+ further we must start adding new algorithms and functions. -The kernel for milestone 2 is reliable and has no noticable - performance degradations when compared to a matchung -current +
The kernel for milestone 2 is reliable and has no noticeable + performance degradations when compared to a matching -current kernel. (the differences are less than the margin of error, so that sometimes the new kernel actually fractionally beats the unaltered kernel).
We hope that by the time this is published, the KSE patches will have been committed. The Major effect for most developers will be only that the device driver interface requires a 'thread' pointer instead of a Proc pointer in the open, close and ioctl entrypoints.
I'm sure there will be small teething problems but we are not expecting great problems at the commit.
The position of Core Secretary was filled by Alan Clegg <abc@freebsd.org> The first core-secretary report should be available the second week in September and will cover the issues discussed by core during August 2001.
Development is continuing; pam_unix has gained the ability to change passwords, login(1) has had PAM made compulsory (and is going to have more PAM-capable features handed over to PAM).
The ATM stack has been tested with a number of FreeBSD machines and a Marconi ATM switch and seems to be quite stable running CLIP. Multi port support for the native ATM API has been implemented but needs some testing.
PRFW is a set of hooks for the FreeBSD kernel. It allows users to insert code into system calls, for such purposes as creating extended security features. Last week, PRFW reached 0.1.0, with many bugfixes and cleaning. I urge anyone who is interested to please visit the site, join the mailing list. Also take a peek at lsm.immunix.org, the Linux hooks. It will be a good contrast.
Work is still progressing to make all of the perl scripts run using perl's 'strict' mode, and to migrate all FreeBSD specific options into the configuration file (CVSROOT/cfg.pm). I'll be looking for help soon to write a guide on how to make use of these scripts for use in your own repository. Anyone interested in helping should contact me at the above email address.
The software has been committed to -current and seems functional. Outstanding issues include dealing with IPV6CP events (linkup & linkdown scripts) and allocating site-local and global addresses (currently, ``iface add'' is the only way to actually use the link).
Status is unchanged since last month. Patches have been submitted to get ppp working under HURD, and mostly under Linux. There are GPL copyright problems that need to be addressed. Many conflicts are expected after the commit of IPv6 support in ppp.
Making pppoed function in a production environment. All known problems have been fixed and committed.
I looked at bringing PPPoA into the base system, but could not because of an overly restrictive distribution license on the Alcatel Speedtouch modem firmware. It has been committed as a port instead and is running live at a FreeBSD Services client site.
The OLDCARD improvements have been completed, except for a few edge cases for older laptops with CL-PD6729/30 chips and some pci bios issues. Some minor work will continue, but after 4.4R is released, only a few remaining bugs will be fixed before the author moves on to greener fields of NEWCARD development.
Targeting 4.4-RELEASE, one team has been translating newly MFC'ed section [125678] manpages. The other team has been updating section 3 since May and one third (1/3) is finished. The port ja-groff is updated to be groff-1.17.2 based, and now it has the same functionality as base system does. The port ja-man is updated to have the search capability under an architecture subdirectory, as base system does. The doc/ja_JP.eucJP/man hierarchy update (adding architecture subdirectories) is planned after 4.4-RELEASE.
Basic footbridge support is now functional and the kernel is now able to probe the pci bus. Access primitives for the bus are still missing so I can't attach any drivers yet.
The syncache implementation is completed, and currently under testing and review. The code should be committed to -current in the near future, and a patchset for -stable made available.
State information for TCP connections is primarily kept in the TCP/IP control blocks in the kernel. Not all of the TCP states make use of the entire structure, and significant memory savings can be had by using a cut-down version of the state in some cases. The first phase of this project will address connections that are in the TIME_WAIT state by moving them into a smaller structure.
This project has completed the initial research and rough design phases, with actual code development starting immediately.
For 5.0, the goal is for the network stack to run without the Giant lock. Initial development in this area may focus on partitioning the code and data structures into distinct areas of responsibilities. A first pass of locking may involve using a several smaller mini-giant code locks in order to reduce the problem to a manageable size.
Progress for this month includes the creation of a perforce repository to officially track the locking changes, and the initial submission of locks for the &ifnet list. Some code cleanup has also been done to the main tree in order to better support future locking additions.
Currently, all network devices (fxp0, lo0, etc) exist in their own namespace, and are accessed through a socket interface. This project creates device nodes in /dev for network devices, and allows control and access in that fashion.
This is experimental work, and suggestions for APIs and functionality are strongly encouraged and welcomed. In is not clear whether it will be possible (or desirable) to provide the exact same set of operations that can be done through the socket interface.
Benefits of approach include the fact that a kqueue filter can be attached to a network device for monitoring purposes. Initial - code exists to send a kq event whever the network link status + code exists to send a kq event whenever the network link status changes. Other benefits may include better access control by using filesystem ACLs to control access to the device.
RELNOTESng, the DocBook-ified set of release documentation files, has been merged to the RELENG_4 branch. 4.4-RELEASE will be the first release of FreeBSD with the new-style release notes, hardware list, etc. Some of these documents are being translated by the Japanese and Russian translation teams.
Snapshots of RELNOTESng for CURRENT and 4-STABLE in HTML, text, and PDF are available at the above URL and are updated irregularly but frequently. Dima Dorfman <dd@FreeBSD.org> and Nik Clayton <nik@FreeBSD.org> have been working to have automatically-generated snapshots on the main FreeBSD web site.
On my TODO list: 1) Resynchronize the FreeBSD installation document with the installation chapter in the Handbook. 2) Update the hardware lists (with particular emphasis on PCCARD and USB devices). 3) Update the infrastructure to allow the architecture-dependent parts of RELNOTESng to scale to more hardware platforms.
Sparc64 development is still continuing rapidly and we're making some excellent progress. Of note, some problems with the way the pmap module implements copy-on-write mappings have been fixed and fork() now works as expected, support for signals has - been added, and the port has been updated for kse in the perforce + been added, and the port has been updated for KSE in the perforce repository. Thomas Moestl has begun work on pci bus support, and a basic nexus bus for sparc64 has been written. The driver for the Sun `Psycho' and `Sabre' UPA-to-PCI bridges and associated code has been ported from NetBSD (the Sabre is the on-chip version found in the UltraSparc IIi and IIe). PCI configuration, I/O and memory space accesses do already work, as well as interrupt assignment and delivery for devices attached directly to the bridge, and the first PCI device drivers can attach and seem to work mostly. Interrupt routing and busdma support still need much work.
The Handbook has been the main focus of activity this month. Due to go to the printers on the 15th a vast amount of new content has been submitted and committed. This includes a complete rewrite of the "Installing FreeBSD", which massively expands the amount of information available to people new to FreeBSD. It even includes screenshots.
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/install.html
Comments, and contributions are, of course, welcome.
FreeBSD's IP Multicast Routing support was recently updated in several ways. One big change is that it's now able to be loaded as a KLD instead of statically compiled into the kernel; this is especially useful for experimentation or updating of an existing system. It also now coexists nicely with the kernel IP encapsulation infrastructure, so that multicast tunnels can better coexist with MobileIP, certain IPSec tunnels and generic IPv4-in-IPv4 tunnels.
The allocator appears to be stable. Mbtypes statistics have been re-activated thanks, in part, to Jiangyi Liu <jyliu@163.net> although the diff has not yet been committed (I'm just in the process of cleaning it up a little and final testing). More work to come: cleanups, follow TODO from the original commit, and perhaps an eventual generalization of the allocator for various network-related allocations (in a more distant future).
After two months of little progress, RAIDframe work is gearing up again. The port to -stable has some known bugs but is fairly stable. The port to -current was recently completed and patches will be released soon. RAIDframe is a multi-platform RAID subsystem designed at CMU. This is a port of the NetBSD version by Greg Oster.
The aac driver has been given a lot of attention lately and is now nearly feature complete. Changes include crashdump support, correct handling of controller initiated commands, and more complete management interface support. The Linux RAID management tool available from Dell and HP now fully works; a FreeBSD native version of the tool is also in the works. These changes have been checked into -current, and will appear in -stable once 4.4 has been released.
We are making some progress, we are now down to 2170 open PR's down from an all time high of 3270 just 3 months ago. The aim is still to get rid of all the dead-wood in the PR database so only relevant PRs in the database. A big thanks from me to the people who have made this happen!
Support for cloning vlan devices via ifconfig has been committed to -current and will be MFC'd after further testing. - Additionaly, Maksim Yevmenkin submitted code to allow cloning of + Additionally, Maksim Yevmenkin submitted code to allow cloning of tap and vmnet devices on devfs systems. Code for faith and stf should be committed shortly.
Current status is that the ia64 kernel builds and runs in a simulator environment up to single user mode and has been tested lightly in that environment. My current focus is on completing the ia64 loader so that I can start to get kernels working on the real hardware. The loader is coming along well and I expect to be able to load kernels (but not necessary execute them) soon.
I have access to the libh CVS repo again and am testing a new, OBJDIR capable build structure at the moment. Done that, I'm going to continue testing the package library and implement the missing functionality. Currently, import of libh into the base system is under discussion (arch mailinglist). Now that 5.0-RELEASE has been shifted, I want 5.0 ship with a libh installer and package system. We can really need people who are good in C++, are able to understand what the current implementation does and also feel that working on libh is fun and thus are willing to help.
Getting GNOME Fifth-Toe metaport ready for 4.4-RELEASE was the main focus of activity this month. In the process many components were updated, many bugs were tracked down and solved, which allowed to make this 97-component meta-package building and working properly.
-Next month the project will be focused on organising work of +
Next month the project will be focused on organizing work of the FreeBSD GNOME Team as well as on attempts to increase amount of people participating in the team (anybody who is willing to participate is welcome to drop a note to gnome@FreeBSD with a short explanation of how he/she could help).
NVIDIA Corporation releases Linux drivers by using a combination of binary object files and source (under a constrictive license). The FreeBSD NVIDIA driver project aimed to completely replace the source component of the driver using code - targetting FreeBSD 4.3 and released under the BSD license. The + targeting FreeBSD 4.3 and released under the BSD license. The binary module provided is supposedly the same module used on Windows, BeOS, and OS/2, so it should be portable between different i80x86 based OS's.
The project is currently on indefinite hold. Our contact at NVIDIA seemed enthusiastic about the project, and was fairly quick about returning email, but when we discovered issues that prevented porting without changes to the binary component or - error codes we needed decyphered, Nick (the contact) said he'd + error codes we needed deciphered, Nick (the contact) said he'd look into it and never got back. The first major problem was the - ioctl interface, the nvidia driver passes a pointer and depends + ioctl interface, the NVIDIA driver passes a pointer and depends on the kernel side to copyout the right amount, where FreeBSD expect the parameters to be correct and the copyout is performed by the subsystem. This was worked around using Dave Rufinos "ioctl tunnel" idea. After that, we found that X refused to load and traced it down to an ioctl defined in the binary component erroring. We cannot tell what that ioctl is, were told that we - could not sign an nda for source to that component, and have been + could not sign an NDA for source to that component, and have been waiting a month for Nick to "look into it". Therefore progress is impossible (without breaking the license) and we believe that the flaws make the driver unportable to any *nix other than Linux.
The FreeBSD release engineering process for FreeBSD 4.4 started to ramp up around August 1st when the "code slush" took affect. During this time all commits to the RELENG_4 branch were reviewed by re@FreeBSD.org (over 250 code snippets had to be reviewed). After the first release candidate on August 15th, all submissions were scrutinized under a more strict potential risk vs benefit curve. The best way to help get involved with the release engineering process is to simply follow the low volume freebsd-qa mailing list, help out with the neverending supply of PRs related to our installation tools (sysinstall), or to work on a possible next-generation replacement for our installation technology, such as the libh or OpenPackages projects.
Many companies donated equipment, network access, or paychecks to finance these activities. Including Compaq, Yahoo!, Wind River Systems, and many more.
In mid March, 2001, Tim Newsham of Guardent identified an attack possible against the initial sequence number generation scheme of FreeBSD (and other OSes.) In order to guard against this threat, a randomized sequence number generation scheme was ported over from OpenBSD and included in 4.3-release. Unfortunately, non-monotonic generation was found to cause major problems with applications which initiate continuous, rapid connections to a single host.
In order to restore proper operation under such circumstances while still providing strong resistance against sequence number prediction, FreeBSD 4.4 uses the algorithm specified in RFC 1948. This algorithm hashes together host and port information with a piece of secret data to generate a unique sequence number space for each connection. As a result, outgoing initial sequence numbers are again monotonic, but also unguessable by an attacker.
The port of LOMAC to FreeBSD is progressing well, and already has a very high level of stability (no known outstanding bugs!). Aspects which have already been implemented include a stacking filesystem overlay with fully-functional access controls (for files and directories) based on path names, access controls for sending signals, and file-backed-memory revocation for processes.
Updates to things from last month:
New stuff since last month:
Most of the work this month has focused on development of the native JDK 1.3.1 patchset. The 3rd patchset is out and has been accompanied with the creation of a FreeBSD "port". This has allowed early adopters much easier access to the code and naturally resulted in a number of bugs being found. Development work has mostly focused on fixing these problems and the project is now set to release fourth patchset over the weekend, which - should see the JDK in a reasonable useable state. One of the big + should see the JDK in a reasonably usable state. One of the big challenges left is producing a working HotSpot JVM, which looks like it will require some heavy hacking.
We also welcome OpenBSD's Heikki Korpela to the porting team :)
As part of some ongoing development activity, the floppy driver (fdc(4)) enjoyed some overhaul in the past which is part of an ongoing process. Automatic density selection will come next, something i meant to implement for years now. As part of that, the entire density selection stuff has been rewritten. 2.88 MB floppies are on the wishlist as well, but I need a working 2.88 drive before attempting to implement that.
sppp(4) should be merged with the ISDN4BSD offspring variant. This will merge some features and bugfixes from the i4b branch (like VJ compression), and eventually end up in a single sppp(4) in the tree. While being at that, incorporating many changes and bugfixes from NetBSD is considered as well.
The KAME project (http://www.kame.net/) has merged its IPv6 and IPsec implementation as of July 2001 to FreeBSD CURRENT and STABLE, in cooperation with some contributors of the project. The latest code includes a number of bug fixes, has been fully tested in FreeBSD STABLE, and will appear in FreeBSD 4.4 RELEASE. Thus, the new RELEASE version will be quite stable in terms of IPv6 and IPsec.
The project has assigned a talented guy to be responsible for merge from KAME to FreeBSD, so future merge efforts will be smoother.
The TrustedBSD project continues to move ahead, with progress made in the ACL, Capability, and MAC implementations. In addition, support from DARPA is permitting new work to improve the extended attribute code, improve security abstractions, and work on security documentation. Due to the push-back of the FreeBSD 5.0 release, it should now be possible to include a complete MAC implementation in that release. Specific status reports appear for components where substantial progress is being made.
Capabilities support is currently being comitted to the base +
Capabilities support is currently being committed to the base FreeBSD tree--userland libraries are now fully committed, and kernel infrastructure is being integrated.
Planning for BSDCon Europe is going well. We're still accepting proposals for talks but the schedule is starting to fill up so we may not be for much longer.
An update of the site that includes accommodation information, a preliminary schedule, a list of speakers and an online payment page will be launched on Wednesday 19 September.
The fee will be £150 for individuals and £250 for corporations. The individual pricing is valid only until the end of September, the price will rise to £200 for October and late registrations in November will be £250.
The updated website will include a list of sponsorship options, we're still looking for more sponsorship.
This bi-monthly report covers development activities on the FreeBSD Project for December 2001 and January 2002. A variety of accomplishments have been made over the last couple of months, including strong progress relating to the KSE project, which brings Scheduler Activations to the FreeBSD kernel, as well as less visible infrastructure projects such as improvements to the mount interface, PAM integration work, and translation efforts. Shortly following the deadline for this status report, the BSD Conference and FreeBSD Developer Summit were held, and will be covered in the next bi-monthly report at the end of March. Plans are already under way for the USENIX Annual Technical Conference in Monterey, CA, later this year, and all and sundry are encouraged to attend to get further insight in FreeBSD development.
Robert Watson
I've been working to integrate recent improvements in the NetBSD usb stack to FreeBSD -current. Both NetBSD and OpenBSD currently share the same source, as FreeBSD did too at once point before it diverged. The goal is to get back to that state, but there are many improvements on both sides that need to be merged before this is complete.
I'm currently looking for someone to help maintain usb in -stable. Please let me know if you're interested.
Patches for cp(1), ls(1), and mv(1) to bring in POSIX.1e-compliant Access Control List support have been updated to patch against builds of -CURRENT. Other system utilities are currently being evaluated for ACL support including install(1) (patch available) and mtree(8). Work is in progress to verify the native getfacl(1), setfacl(1), and other utilities build and work correctly on other ACL-enabled systems (e.g. Linux w/ACL patches) and to help verify POSIX-compliance of the continuing TrustedBSD work along with other systems. Finally, experimental Perl and PHP modules are available allowing limited access to native ACLs for languages other than C.
The project is making progress. The goal is to design and implement Host Controller Interface (HCI) and Link Layer Control and Adaptation Protocol (L2CAP) layers using Netgraph framework. More distant goal is to write support for Service Discovery Protocol (SDP) and RFCOMM protocol (Serial port emulation over Bluetooth link) . All information was obtained from Bluetooth Specification Book v1.1.
Project status: In progress. 1) Design: mostly complete, there are some minor issues to be resolved. 2) Implementation: Kernel - HCI and L2CAP Netgraph nodes have been implemented; 3) User space (API, library, utilities) - in progress. 4) Testing: In progress. I do not have real Bluetooth hardware at this point, so i wrote some tools that allow me to test the code. Some of them will be used as foundation for future user space utilities.
Issues: 1) Bluetooth hardware; I do not have real Bluetooth hardware, so if people can donate hardware/specs it would be great. I promise to write all required drivers and make them available. I also promise to return hardware/specs on first request. 2) Project name; I would like to see the name that reflects the following: it is a Bluetooth stack, implementation is for FreeBSD and implementation is based on Netgraph framework
This project is now finally underway, thanks to DARPA and NAI getting a sponsorship lined up. The infrastructure code and data structures are currently taking form inside a userland simulation harness. Basic MBR and BSD methods have been written and device attach/taste/dettach algorithms been implemented and validated.
I've update OS of buildboxes to the latest FreeBSD 5-current and 4-stable. Everything goes fine. From January 2002, I've started a webzine, SNAPSHOTS Notes (only Japanese version is available). SNAPSHOTs Notes pickups tips and information especially for the people living with FreeBSD 5-current/4-stable. Article or idea for SNAPSHOTs notes are always welcome (you don't need to write in Japanese :-).
Robert Watson created the TrustedBSD audit perforce tree, which is a branch from the TrustedBSD base tree, in order to start pushing development efforts towards using a revision control system. Andrew Reiter started to merge in some framework related code for generation of audit records, enqueueing writes, and handling data writing. There is a great deal of work to be done with updates and discussion on the trustedbsd-discuss@TrustedBSD.org mailing list.
The KSE project (an attempt to support scalable thread in FreeBSD using kernel support), has reached What I call "milestone 3". At this milestone it is possible to run a multithreaded - program on a single CPU but with full concurrancy of threads on + program on a single CPU but with full concurrency of threads on that CPU. In other words the kernel supports the fact that one thread can block by allowing another thread to run in its place. A test program that demonstrates this is available at the above website.
Milestone 4 will be to allow threads from the same program to - run on multiple CPUS but may require more input from the SMPNG + run on multiple CPUs but may require more input from the SMPng project. I am at the moment (Feb 6) getting ready to commit a first set of changes for milestone 3, that have no real effect but serve to drastically reduce the complexity of the remaining diff so that others can read it more easily. After changes to libkvm to support this diff have been added it should be possible to run 'ps' and look at multiple threads in a treaded process. I will be demonstrating KSE/M3 at BSDcon.
The Netgraph ATM package has been split into a number of smaller packages: bsnmp is a general-purpose SNMP daemon with support for loadable modules. Two modules come with it: one implementing the standard network-interface and IP related parts of MIB-2 and one for interfacing other modules to the NetGraph sub-system. ngatmbase contains the drivers for the ATM hardware, the ng_atm netgraph type and a few test tools. This package allows one to use ATM PVCs. It should be possible, for example, to do PPP over ATM with this package. Both bsnmp and ngatmbase are available in version 1.0 under the link above. Two other - modules will be released in february: ngatmsig containing the + modules will be released in February: ngatmsig containing the UNI-4.0 signalling stack as netgraph nodes and ngatmip containing CLIP and LANE-2.0.
A significant amount of progress was made in December and January, particularly in the area of utility conformance. Several utilities were updated to conform to SUSv3, they include: at(1), mailx(1), pwd(1), split(1), and uudecode(1). Several patches have been submitted to increase conformance in other utilities, they include: fold(1), patch(1), m4(1), nice(1), pr(1), renice(1), wc(1), and xargs(1). These are in the process of being reviewed and committed. Two new utilities have been written, specifically pathchk(1) and tabs(1). These are also being reviewed and will be committed shortly.
A patch which implements most of the requirements of scanf(3) is being reviewed and is expected to be committed shortly. This will allow us to MFC a number of new functions and headers. Additionally, work has started on wide string and complex number support.
For 4.5-RELEASE, port ja-man-doc-4.5.tgz is in sync with base system except for OpenSSH pages (OpenSSH 2.3 based instead of 2.9) and perl5 pages (jpman project do not maintain). Section 3 updating has 55% finished.
OKAZAKI Tetsurou has incorporated changes on base system's groff into port japanese/groff. MORI Kouji has fixed two bugs of port japanese/man.
The KAME project is currently focusing on the scoped addressing architecture, the advanced API implementation, NATPT and the mobile ipv6 implementation. Though these stuffs are not stable enough to be merge into the FreeBSD tree, you can get and try them from the above URL.
The FreeBSD in Bulgarian project aims to bring a more comfortable working environment to Bulgarian users of the FreeBSD OS. This includes, but is not limited to, font, keymap and locale support, translation of the FreeBSD documentation into Bulgarian, local user groups and various forms of on-line help channels and discussion forums to help Bulgarians adopt and use FreeBSD.
A guide for using FreeBSD with Bulgarian settings has been put up on the project's website. The CVS repository will be made public shortly, linked to on the URL's above.
An independent project for making FreeBSD easier to use by
Bulgarians has appeared,
The past two months have been an exciting time in the FreeBSD Java Project with the signing of a license between the FreeBSD Foundation and Sun allowing us access to updated JDK source code and the Java Compatibility Kit (JCK). This license will also allow the project to release a binary version of both the JDK and JRE once JCK testing is complete. Work on this testing is under way with the project hopeful of being able to make a binary release in the not too distant future.
In lieu of the binary release which was hoped for with FreeBSD 4.5 the project will release an updated source patchset this weekend. This patchset will feature further work on the FreeBSD "native" threads subsystem from Bill Huey. Also, thanks to hard work by Joe Kelsey and Fuyuhiko Maruyama, the patchset will for the first time feature a working Java browser plugin!
Extending camcontrol's page definition file format to include both modepage and logpage definitions; adding support to camcontrol to query and reset log page parameters. Consideration is being made to possibly include support for diagnostic and vital product data pages, but that is outside the current project scope. New page definition file format includes capability to conditionally include page definitions based on SCSI INQUIRY results allowing vendor-specific pages to be described also. Approximately 90% complete.
OpenPAM, a new library intended to replace Linux-PAM in FreeBSD, has been written and is undergoing integration testing. It is available for download from the URL listed above.
In addition to this, a couple of new modules have been written (pam_lastlog(8), pam_login_access(8)), and the pam_unix(8) module has been extended to perform most of the tasks normally performed by login(1), which is now fully PAMified.
The PAM FDP article has been put on hold until OpenPAM replaces Linux-PAM in CVS, to avoid wasting effort on soon-to-be obsolete documentation.
Substantial progress has been made towards a working MAC implementation. The focus over the last two months has been moving from a hard-coded series of MAC policies to a more flexible implementation. A pluggable policy framework has been created (and is still under development), supporting Biba, MLS, TE, a "BSD Extended" model, and a sample mac_none module. Some modules must be compiled in or loaded prior to boot; others may be introduced at run-time. Support for networking has improved, with improved handling of IP fragmentation in IPv4, support for various pseudo-interfaces such as if_tun and if_tap, improved integration into userland, NFS-related fixes, moving the VFS enforcement out of individual filesystems, support for a 'multilevel' mount flag, support for explicit labeling in procfs and devfs, addition of an 'extattrctl lsattr' argument to list EAs on a filesystem, support for label ranges in the Biba and MAC policies, and much more.
Targets for the next two months include more universal enforcement of VFS-related calls, improved support for alternative ABIs, improved flexibility of in-kernel subject and object labels, support for IPv6 and IPsec, and improved support for NFS serving.
Development continues in the FreeBSD Perforce repository, which may be accessed using cvsup.
Now that the patch has been mailed to the freebsd-arch@freebsd.org mailing list, and that there were no objections, the commit will happen soon. Poul is currently testing it in his own tree. After it has been committed, it will be time to modify the filesystems in the tree to use VFS_NMOUNT instead of VFS_MOUNT. Mount(8) will also need some modifications. Some new manpages -- nmount(2) and kernel_vmount(9) -- are being created in the meantime.
Alfred Perlstein commited file descriptor locking code - which was definetly a good push towards trying to lock down +
Alfred Perlstein committed file descriptor locking code + which was definitely a good push towards trying to lock down some important pieces of global data. Peter Wemm has made progress on pmap cleanups for x86 SMP TLB shootdowns. Matt Dillon and John Baldwin have made progress on getting patches done for moving accesses to ucred's out from under Giant's protection. John Baldwin has also made some commits in order to get the alpha port's SMP working. Matt Dillon has plans for hunting down fileops locking issues in order to continue his previous Giant pushdown work.
This report covers FreeBSD development activities from February, 2002 through April, 2002. It's been a busy few months -- BSDCon in San Francisco, the FreeBSD Developer Summit, a first development preview of 5.0-CURRENT, not to mention lots of progress on the 5.0 feature set (SMPng, sparc64, GEOM, ... the list goes on).
In the next two months, the USENIX ATC occurs (highly recommended event for both developers and users), and a number of new software components will hit the tree, including UFS2 and the TrustedBSD MAC framework. We'll also complete the elections for the FreeBSD Core Team, and should have the next Core Team online by the time the next report rolls around. Stay tuned for more!
Robert Watson
Packages are built from the FreeBSD Ports Collection on a cluster of i386 and alpha machines using scripts available in /usr/ports/Tools/portbuild/. Over the past few months I have been cleaning up and extending these scripts to improve efficiency and allow for greater flexibility in how package builds are performed. Major improvements so far have been: cleaning up and modularizing the scripts to avoid code duplication and reduce the need for ongoing maintenance; optimizing the build process and making it much more robust against client machine failure; and allowing package builds to be restarted if they are interrupted. The i386 package cluster is currently running FreeBSD 5.0-CURRENT, and it has proven to be a useful testing ground for exposing kernel bugs, especially those which only manifest under system load.
Future plans include the ability to perform incremental package rebuilds which only build packages that have changed since the last run. This will allow packages to be made available on the FTP site within an hour or two of the CVS commit to the ports collection. We also hope to set up a sparc64 package cluster in the near future, but this is contingent on suitable hardware.
FreeBSD's new kernel memory allocator has been commited to +
FreeBSD's new kernel memory allocator has been committed to 5.0. UMA is a slabs derived allocator that supports memory - reclaiming, object caching, type stable storage, and per cpu + reclaiming, object caching, type stable storage, and per CPU free lists for optimal SMP performance. It has both a malloc(9) interface and a zone style interface for specific object types. uma(9) will be available shortly.
Read-only support for UDF filesystems was checked into the 5-CURRENT branch in April. Backporting for 4-STABLE is being conducted by Jeroen. The next phase is to write a newfs_udf, then move on to adding write support to the filesystem. I'm still looking for a volunteer to handle read and write support for write-once media (e.g. CD-R).
I have released a new zero copy sockets snapshot, the first since November, 2000. The code has been ported up to the latest -current, and the jumbo code now has mutex protection. Also, zero copy send and receive can be selectively turned on and off via sysctl to make it easier to compare performance with and without zero copy. Reviews and comments are welcome.
I'm slowly making progress. The second engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20020506.tar.gz
This release includes support for H4 UART transport layer, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices.
I'm currently working on RFCOMM protocol implementation (Serial port emulation over Bluetooth link). My next goal is to port Service Discovery Protocol (SDP) implementation from BlueZ (http://bluez.sf.net). I'm also thinking about adding USB device support (as soon as i find/buy hardware).
Issues: 1) Bluetooth hardware; I have couple PC-CARDs that i use for development and testing purposes, but i'd love to have more. 2) Time; My regular day job kicked in, so i will be spending more time doing stuff i'm getting paid for.
Since the last status report, two developers working on utility conformance were given commit access to the FreeBSD CVS repository to help expedite development. As a result, the following utilities have been brought up to conformance, they include: csplit(1), env(1), expr(1), fold(1), join(1), m4(1), mesg(1), paste(1), patch(1), pr(1), uuencode(1), uuexpand(1), and xargs(1). The printf(1) utility was brought up to conformance with the 1992 edition of POSIX.2, with further development planned.
-On the header front, much progress has been made. Specically, - infrastructure to control visiblity of components of a header, based +
On the header front, much progress has been made. Specifically, + infrastructure to control visibility of components of a header, based on the standard requested by an application, has been added to <sys/cdefs.h>. Some work has been completed on renovating the way types are defined. This has lead to the creation of <sys/_types.h>. Further improvements such as the merger of <machine/ansi.h> and <machine/types.h> are planned. Additionally, the headers: <strings.h>, <string.h>, and <sys/un.h> have been made to conform to POSIX.1-2001.
On the API front, scanf(3) has received support for 5 new length modifiers (hh, j, ll, t, and z). A patch to implement two additional conversion specifiers (j and z) has been developed for printf(9) and is expected to be committed soon.
In other news, the project's web site has been moved to the main FreeBSD site. It is now available at the URL at the top of this status report. Please update your bookmarks.
Version 1.1 for FreeBSD-current is now available. It includes the SNMP-daemon package bsnmp, the driver package ngatmbase, the UNI4.0 signaling package ngatmsig and the network emulation package ngatmnet. NgAtm allows both to build applications running directly on top of ATM and to use ATM-Forum LAN emulation to use IP over ATM. Currently we are working on a simple switch module, that implements the network side signaling and ILMI as well as simple routing and call admission control.
The GNOME project has seen quite a few changes lately. For one, the author of this update has recently been given "The Bit." Joe Marcus Clarke now has CVS access, and is working primarily on the GNOME project. Joe has been closing a good deal of GNOME PRs, as well as patching some of the existing GNOME 1.4 components.
The GNOME 2 porting effort continues on. We have completed porting of the GNOME 2.0 API, and are 75% complete on porting the full GNOME 2.0 desktop. When complete, GNOME 1.4 and GNOME 2.0 will be co-resident in the ports tree. Both APIs can be installed concurrently in the same PREFIX, but the respective desktops will remain mutually independent. Maxim Sobolev is working on adapting bsd.gnome.mk to handle both versions of the desktop in an elegant fashion.
Not to be left out, the existing GNOME 1.4 components have received numerous updates to keep them in sync with the stable distfiles - on gnome.org. We have seen many "1.0" milsestone releases including + on gnome.org. We have seen many "1.0" milestone releases including the most recent AbiWord 1.0.0. In the next few weeks, we will be making sure all the GNOME 1.4 components build correct packages on bento so that GNOME 1.4 will be on the 4.6-RELEASE CD.
FreeBSD/KGI started last year after the port of GGI to VGL. KGI (Kernel Graphic Interface) is a kernel infrastructure providing user applications with access to hardware graphic resources (dma, - irqs, mmio). KGI is already available under Linux as a seperate + irqs, mmio). KGI is already available under Linux as a separate project. The FreeBSD/KGI project aims at integrating KGI in the FreeBSD kernel. Mostly a port for now, but optimized for FreeBSD in the future. Currently FreeBSD/KGI is under development and the code is only available for reading, compiling but not running. More interesting are design hints found at the project URL.
We now have a loadable mfsroot floppy. It contains just the diskeditor (which is really a disk partitioner) which has been - enhanced and is probably in his final form. It's been geared - towards making the newfs(1) and mount(1) step seperate dialogs, so - it reduced its complexity. A basic fstab class has been + enhanced and is probably in its final form. It's been geared + towards making the newfs(1) and mount(1) steps separate dialogs, so + it reduceed its complexity. A basic fstab class has been implemented to manipulate /etc/fstab and mountpoint. This might find a use outside libh, by the way. Libh package format is still incomplete and somehow buggy, so it's my next target.
There is a API documentation effort underway with the help of doxygen(1), so there's now more documentation for people that want to get started with libh.
All this lead me to prepare the release of another alpha preview of libh that will shortly be available in the ports collection (0.2.2). Also, a new committer (okumoto) has joined the project (as well as I) and he is currently working on cleaning up the build system. It's been a few months without news, so this probably seemed a bit long, but don't worry, we still need your help to really get this going!
There are several new topics, including: Source Code Tour is now separated into kernel part and userland part, yet another snapshots from RELENG_4_x branch (currently 4.5-RELEASE-p4), add several packages including XFree86 4.x to installation CD-ROM, new - cdboot-only ISO image, fix breackage of duplex.iso, etc. See also + cdboot-only ISO image, fix breakage of duplex.iso, etc. See also the project webpage for more detail. Also, I have a plan to add FreeBSD/alpha distribution to this project -- stay tuned.
KAME Project has been extended until March 2004, and we decided the project roadmap for these two years. The first one year is for implementation, and the remaining year is for feedback of our results into other BSD projects (please refer to the above URL for further detail). Great change is lack of NAT-PT support due to a lack of human resource, although KAME snap still contains it as it is.
SUZUKI Shinsuke (suz@kame.net) has begun working for KAME and FreeBSD merge task in cooperation with Umemoto-san (ume@FreeBSD.org). Some of KAME stuff (critical bug fix, newest ports for pim6sd and racoon, etc) has been merged into 4-stable in this April.
Over the past couple of months, progress has pretty much stopped until very recently. The past few changes to the audit code were update the usage of zones to UMA zones, cleanup some old cruft, and start toying with the idea of having an audit write thread implemented as an ithd. The next step is to decide two realistic approaches to the where the records will be dumped -- whether that is to a local disk or fed up to userland and then dealt with. After that, the goal will be to expand the number of events that are being audited, while also working in some performance testing procedures. I will be posting to trustedbsd-audit about the recent changes shortly.
Over the last three months, there has been a lot of activity in the TrustedBSD MAC tree. An initial commit of the SEBSD code (NSA FLASK and SELinux implementation) was made; many MAC policies previously linked directly to the kernel via kernel options were moved to kernel modules; the flexibility of the framework was improved relating to the life cycle of object labels; additional labeling and access control hooks were introduced; new policies were introduced to demonstrate the flexibility of the framework (including a cleanup of inter-process authorization, additional VFS hooks, improved support for multilabel filesystems, network booting, IPv6, IPsec, support for "peer" labels on stream sockets). Current modules include Biba integrity policy, MLS confidentiality policy, Type Enforcement, "BSD Extended" (permitting firewall-like rulesets for filesystem protection), "ifoff" (limit interface communication by policy), mac_seeotheruids (limit visibility of processes/etc of other users), "babyaudit" (a simple audit implementation), and SEBSD (FLASK/SELinux port).
Over the next month, a final move to completely dynamic labeling will be made, permitting policies to introduce new state relating to process credentials, vnodes, sockets, mounts, interfaces, and mbufs at run-time, allowing a broad range of flexible label-driven policies to be developed. In addition, application APIs will be re-designed and re-implemented so as to better support a fully dynamic policy framework. We plan to make an initial prototype patchset available for review in June, with the intent of committing that patchset in mid-June.
Updated prototype code may be retrieved from the TrustedBSD CVS trees on cvsup10.FreeBSD.org.
The painful parts are now completed, with all authentication- related utilities converted to PAM (except for those cases where it doesn't make sense, like Kerberos- or OPIE-specific commands). OpenPAM is complete (except for a few missing man pages) and seems to work well.
For more details, see the activity reports linked to above.
OpenSSH has been upgraded to 3.1, and the kinks seem to have been worked out by now. OpenSSH will now use PAM for both ssh1 and ssh2 authentication.
The KSE project had floundered due to lack of development time for awhile, but has been picked up recently by Jonathan Mini. Currently, the main focus is to prepare the "milestone 3" code for inclusion into -CURRENT.
The project is still working towards "milestone 4" (allowing threads from the same process to run on multiple CPUs), which should be significantly easier now due to work done by the SMPng project over the past several months.
Help could be used in several areas of the project, especially with porting the libc_r (pthreads) library to KSE's threading model.
NEWCARD support tried to merge CardBus functions with PCI functions, but that failed to properly route interrupts. A branch for the merge was created and will be merged into the main line at a later date. Too many other things going on in my life to make much progress.
Work on the host access point support for the Prism2 and Prism2.5 based wireless cards has been integrated into the kernel. This work is largely based on Thomas Skibo's initial implementation.
Continued bug fixing and hardening for this last few months.
Future work will include making target mode work correctly and fast.
The LSI-Logic chipset's MPT Fusion driver is also being evaluated.
The FreeBSD MTRR code has been made more robust against unexpected values sometimes found in the Athlon's Memory Type Range Registers. Problems with these values had prevented XFree 4.2 running on some motherboards. Experimentation indicates that these undocumented values may control the mapping of BIOS/ROMs or have something to do with SMM. If anyone can provide details of what these values mean, can they please let me know, so the MTRR code can be completed.
IPMI Tools for FreeBSD is a collection of C and Python applications and modules for exploring the information available via the Intelligent Platform Management Interface (IPMI), as implemented on server motherboards by Intel and HP. IPMI is an open standard with patent protection for adopters which defines standard interfaces to on-board management hardware. The management hardware consists of a CPU, sensors such as temperature probes and fan speeds, and repositories such as the System Event Log and Field-Replaceable Unit (FRU) inventory, and other system information.
A basic set of tools was recently made available which uses the KCS and SMIC system interfaces to retrieve the System Event Log, FRU repository, and system sensors. Additional features are currently under research. Suggestions for additional features and programs are greatly appreciated.
The PowerPC port is moving ahead. It can now mount a root file system and exec init, but fails when trying to map init's text segment in. I'm hoping to have it starting my fake "Hello, world!" init soon, after which I plan to try and get some libc bits in place so that I can build /bin and /sbin and try to get to actual single-user.
4.5-RELEASE Japanese manapge package, ja-man-doc-4.5.tgz, once +
4.5-RELEASE Japanese manpage package, ja-man-doc-4.5.tgz, once published with OpenSSH 2.3 (as reported by previous status report) on January 31, is replaced with new package with OpenSSH 2.9 based manpages on March 3. Since then, we have been updating Japanese manpages for 4.6-RELEASE. For new translation and massive update, we have been making a lot of effort.
Continuing section 3 updating has 73% finished.
The GEOM code has gotten so far that it beats our current code - in some areas while stil lacking in others. Work continues on + in some areas while still lacking in others. Work continues on a generalized interface for "magic data" (boot blocks, disklabels MBR's etc) manipulation from userland.
With GEOM enabled in the kernel any FreeBSD platform will now recognize PC style MBR's, i386 disklabels, alpha disklabels, PC98 extended MBRs and SUN/Solaris style disklabels.
Since the last progress report, the initialization code was much cleaned (thanks to NetBSD's acort32 port) and partial DDB support as been added. I'm now struggling to put the pmap module into a working state. The latest patch set only includes the initialization changes. I did some tries to get what I had so far working on my iPAQ without much successes (downloading a kernel over a serial link is way too painful). If anyone has had success in getting any iPAQ to work as a USB storage device under *BSD please contact me.
I've been mentoring someone on locking up the protocol control blocks in the networking stack. She has already finished TCP and UDP and I'm currently reviewing the patch with her and going over some networking lock order issues. Locking up raw protocol interface control blocks follows next.
Support for stf(4), faith(4), and loopback interfaces has been committed to current. The stf and faith support has been MFC'd. In current the API has changed to move unit allocation into the generic cloning code reducing the amount of support code required in each driver. Code improvements to increase our API - compatability with NetBSD will be commited soon along with cloning + compatibility with NetBSD will be committed soon along with cloning support for discard interfaces and ppp(4) interfaces.
Thanks to
IA64 has had a busy few months. Aside from gcc, we are now fully self hosting on IA64. Doug Rabson has performed his magic and implemented the execution of 32 bit i386 application binaries although more work remains to be done to make ld-elf.so.1 happy with the different underlying page size. We have been using the i386 perforce binary to do actual development work and submit from the ia64 systems themselves. Marcel Moolenaar has been working on SMP and machine-check support. We have been running SMP kernels amazingly reliably on our development boxes for quite some time now. syscons is now functional. We have produced a self-booting run-root-on-cdrom ISO image (idea taken from the sparc64 folks) that has been used to manually self install an IA64 system from a blank disk. Aside from a few minor loose ends we now have complete 'make world' functionality. sysinstall works on ia64. We plan on producing a semi-respectable boot/install cdrom image shortly.
As of Thur May 9th, 2002 FreeBSD 5-CURRENT is now using a GCC 3.1 prerelease snapshot as the system C compiler. At this time of cutting over, the compiler is working well on i386, Alpha, Sparc64, and IA-64 for building world. There is a known problem with our atomic ops on Alpha that prevents a GCC 3.1 built kernel from booting.
Currently the C++ support libraries (libstdc++, et.al.) does not build and thus prevents the system C++ compiler from being used.
The release engineering team released FreeBSD
The next releases of FreeBSD will be 4.6-RELEASE (scheduled for 1 June 2002) and 5.0-DP2 (scheduled for 25 June 2002). Information on the release schedules and more can be found on the team's new area on the FreeBSD Web site (see the URL above).
Finally, the team has gained two new members: Brian Somers and Bruce A. Mah.
libradius now supports RADIUS vendor attribute extensions and user-ppp is now capable of doing MS-CHAP authentication via a RADIUS server. A new net/freeradius port has been created for support of MS-CHAP in a RADIUS server.
MS-CHAPv2 support will be added soon.
The work is sponsored by Monzoon.
Mike Makonnen has done quite a bit of excellent work on porting the scripts from FreeBSD into the NetBSD framework. The next step seems to be to try to reduce the amount of diffs between our implementation and the original set from NetBSD.
The SMPng project has been picking up steam in the last few months thankfully. In February, Seigo Tanimura-san committed the first round of process group and session locking. Alfred Perlstein also added locking to most of the pipe implementation. In March, Alfred fixed several problems with the locking for select() and pushed down Giant some in several system calls. Andrew Reiter added locking for kernel module metadata, and Jeff Roberson wrote a new SMP-friendly slab allocator to replace both the zone allocator and the in-kernel malloc(). The use of the critical section API was cleaned up to not be abused as replacements for disabling and enabling interrupts. Also, Matt Dillon optimized the MD portion of the critical section code on the i386 architecture. Several other subsystems were also locked in April as well. See the SMPng website and todo list for more details.
Some of the current works in progress include locking for the kernel linker by Andrew Reiter and light-weight interrupt threads for the i386 by Bosko Milekic. Seigo Tanimura-san, Alfred Perlstein, and Jeffrey Hsu are also working on locking down various pieces of the networking stack. Alan Cox has started working on fixing the existing locking in the VM subsystem and moving bits of it out from under Giant. John Baldwin has written an implementation of turnstiles as well as adaptive mutexes in the jhb_lock Perforce branch. The adaptive mutexes appear to be stable on i386, alpha, and sparc64, but the turnstile code still contains several tricky lock order reversals. John also plans to commit the p_canfoo() API change to use td_ucred in the very near future and then finish the task of making ktrace(4) use a worker thread.
The patch for the new mount API has now been committed to the tree. Several filesystems also have been converted to this new mount API, namely procfs, linprocfs, fdescfs and devfs. I'm working on converting more filesystems to nmount, and actually already have UFS done. It has not been committed yet to avoid conflicting with the UFS2 work, but it should hit the tree soon. Manpages are still missing at the moment because I had to modify the API slightly. I hope to have them done soon now.
The second FreeBSD Developer Summit, held following the BSD Conference in San Francisco in February, was a great success. Around 40 developers attended in person, another five by phone, and many others by webcast. During a marathon-esque eight hour session, a variety of development topics were discussed, including adding - inheritence to the KOBJ system, ports to new architectures, + inheritance to the KOBJ system, ports to new architectures, adaptations of the toolchain for new architectures, the GEOM extensible storage device framework, upcoming changes to the network stack, TrustedBSD features, KSE, SMPng, and the release engineering schedule. This event was sponsored by DARPA and NAI Labs, with webcasting provided by Joe Karthauser, bandwidth provided by Yahoo!. Planning for future such events is now underway; a summary/transcript of discussion may be found at the URL above.
Another busy two months have passed in the FreeBSD project. With 5.0 released, attention is focusing on making it faster via more fine-grained locking, adding more high-end features like large memory (PAE) support for i386, and further progress on many other projects. FreeBSD 5.1 is expected to ship in late May or early June, with 5.2 following at the end of summer. A roadmap for the push to 5-STABLE is available at http://www.freebsd.org/doc/en/articles/5-roadmap. Although the 5.x series isn't expected to fully stabilize until the 5.2 release, 5.1 promises to be an exciting release and a significant improvement over 5.0 in terms of speed and stability.
Not to be forgotten, FreeBSD 4.8, the latest in the 4-STABLE series, is nearing release. Lots of last minute work is going into to it to deliver features like XFree86 4.3.0, Intel HyperThreading(tm) support, and of course many more bug fixes. Don't forget to support the FreeBSD vendors and developers by buying a copy of the CD set when it comes out!.
Thanks,
Scott Long, Robert Watson
Large portions of headers have been filled in, all have been stubbed out. Minimal functions and data elements have been stubbed out or filled in. Machinery added to support some requisite tunables for building real kernels. GCC fixed to generate correct local label prefixes making it possible to link real kernels. Work begun on providing enough to create and boot real kernels, on real hardware. Decision to only support MIPS-III and above made.
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
I'm very pleased to announce that another release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20030305.tar.gz
This release features new in-kernel RFCOMM implementation that provides SOCK_STREAM sockets interface. This makes old user-space RFCOMM daemon obsolete. People should not use old user-space RFCOMM daemon any longer. The release features new RFCOMM PPP daemon that supports DUN and LAN profiles. Note: PPP patch (support for chat scripts in -direct mode) is required for DUN support. Look for it in the mailing list archive or contact me directly. People with Bluetooth enabled cell phones can now use them to access Internet.
The Bluetooth sockets layer has been cleaned up. People should not see any WITNESS complaints with new code. Locking issues have been revisited and code in much better shape now, although it probably is not 100% SMP ready just yet. The code should work on SMP system anyway because sockets layer is still under Giant.
The simple OBEX server and client (based on OpenOBEX library) is complete. OBEX File Push and OBEX File Transfer profiles work and have been tested with Sony Ericsson T68i cell phone and Bluetooth 3COM stack on Windows2K. It is now possible to send pictures, address book and calendar entries from the cell phone via Bluetooth. Minor bug in OpenOBEX library has been fixed and OPEX Put-Empty command now works.
Due to changes in API userland tools must be in sync with the kernel. People should install new include files, recompile and reinstall all userland tools as part of upgrade. I'm sorry about that.
The FreeBSD 4.8 Release Process is well underway. The RELENG_4 branch has been under code freeze since February 15, and the first release candidates were made available in early March. A testing guide has been put together and is available from http://www.FreeBSD.org/releases/4.8R/qa.html.
Developers should coordinate with re@FreeBSD.org about any changes they would like to include in this release, and users are encouraged to try out the release candidates and help find as many bugs as possible now, before the final release is made.
FreeBSD 4.8 represents the newest production release from the stable '4.X' branch. It does not include all of the features that were made available in the "new technology" 5.0 release in January.
The doceng@ team is a new body to handle some of the meta-project issues associated with the FreeBSD Documentation Project. The main responsibilities of this team are to grant approval of new doc committers, to manage the doc release process, to ensure the documentation toolchains are functional, to maintain the doc project primer, and to maintain the sanctity of the doc/ and www/ trees. The current members of this team are Nik Clayton, Ruslan Ermilov, Jun Kuriyama, Bruce A. Mah, and Murray Stokely.
The later months have been very busy on KGI. Most of the framework has been debugged for typical usage (fb, no accel). I got KII (the input interface) connected to syscons through atkbd. Opening /dev/graphic works and framebuffer resource access is permitted. Finally, the KGIM (KGI module) framework has a better building tree for board / monitor drivers and board drivers are now loading with resource allocation.
Most important on the TODO list: 5.0-RELEASE move (I currently work with a May-2002 5.0-current). Most of debug is now done. Let's validate!
Note that KGI project homepage has changed since the last report.
We have released Japanese translation of 5.0-RELEASE online manual pages on February 2nd. Most of entries which did not exist on RELENG_4 were not yet translated. I hope we can finish such entries soon.
We have the first disk device driver (aac) out from under Giant now, and in certain scenarios it gives improvements up to 20%. - The device drive API was pruned to reflect that NO_GEOM + The device driver API was pruned to reflect that NO_GEOM compatibility is unnecessary, this resulted in approx 1000 lines less source code, the majority of which were removed from the device drivers. The new API for cdevsw is a lot simpler and hopefully less likely to confuse people. The ability to automatically allocate a device major number has been introduced and is already used by a handful of drivers. Checks introduced with this facility has shown that the uniqueness of manually allocated major numbers had already broken down.
Work continues on the statistics collection API and on a unified API for manual configuration of GEOM nodes.
Support for PAE is mostly complete, and has been checked into the jake_pae branch. The approach that is being taken to add support for PAE is to allow the pmap module to view the page table directory as 4 pages instead of 1, and to avoid using the 3rd level structure, the page directory pointer table, as much as possible. Due to its small size, 32 bytes, the PDPT cannot be uniformly recursively mapped, and as such does not provide a regular multi level structure like the page tables used by the alpha or x86-64 architectures. What remains to be done for PAE support is to develop an API for manipulating page table entries which will allow idempotent 64 bit loads and stores to be used where necessary.
Experimental support for >4G ram using PAE has been developed and checked into the jake_pae_test branch in Perforce. This involved adding a physical address type separate from virtual addresses, for use by the vm system and bus code which needs to use physical addresses directly. Initial testing has shown good results with device drivers that can dma to 64 bit physical addresses.
Funding for this project is being provided by DARPA and Network Associate Laboratories, and hardware support by FreeBSD Systems.
In the period from September 2002 through February 2003, the FreeBSD Security Team email aliases saw 1297 messages, a much smaller volume than over the summer (remember the Apache and OpenSSL worms? 4.6.1 oops I mean 4.6.2-RELEASE?).
Also during this period: 95 items were added to the SO issue-tracking database; 39 of these involved the FreeBSD base system while the rest involved ports. 9 new Security Advisories were published, 2 of which covered issues unique to FreeBSD.
In January, the SO published a new PGP key (ID 0xCA6CDFB2, found on the FTP site and in the Handbook). This aligned the set of those who possess the corresponding private key with the membership of the security-officer alias published on the FreeBSD Security web site. It also worked around an issue with the deprecated PGP key being found corrupted on some public key servers.
In February, Mike Tancsa of Sentex donated two machines to the Security Officer. These have been a great help already in testing the security branches, preparing patches, and generating updated binaries. Thank you very much, Mike!
FreeBSD 4.8-RELEASE will continue in the tradition of 5.0-RELEASE, and include GNOME 2 as the default GNOME desktop. This means that 4.8 will ship with GNOME 2.2.
Following on the heels of the recent GNOME 2.2 release, GNOME 2.3 snapshots are gearing up. The development schedule is available from http://www.gnome.org/start/2.3/. Ports will be made available the same way they were for the 2.1 development releases. Stay tuned to freebsd-gnome@ for more details.
We are currently in another ports freeze in preparation for 4.8-RELEASE. Following the freeze, a new bsd.gnome.mk will be committed that effectively removes the USE_GNOMENG macro. This new version will add support for GNOME 2 as well as setup backward compatibility for ports that have not yet been converted to the new GNOME infrastructure. People interested in testing this new Mk file, can check out the ``ports'' module following the instructions at http://www.marcuscom.com:8080/cgi-bin/cvsweb.cgi.
Work on PowerPC is progressing steadily. The system can now boot multi-user from the net and disk. ATA-DMA is being integrated with the ATAng code, and support for older G3 machines is being added.
January and February were quiet months that saw with them the addition of some C99 math functions and macros, which include: fpclassify(), isfinite(), isgreater(), isgreaterequal(), isinf(), isless(), islessequal(), islessgreater(), isnan(), isnormal(), and signbit(). Additional C99 math library support is in the works.
Most of the file system buffer cache has been reviewed and protected. The vnode interlock was extended to cover some buffer flag fields so - that a seperate interlock was not required. The global buffer queue + that a separate interlock was not required. The global buffer queue data structures were locked and counters were converted to atomic ops. The BUF_*LOCK functions grew an interlock argument so that buffers could be safely removed from the vnode clean and dirty lists. The lockmgr lock is now required for all access to buf fields. This was not strictly followed before because splbio provided the needed protection.
There are a few areas of code that need to be protected and cleaned up before giant can be pushed down. Most notably the background write code is currently unsafe without giant. Also, many of the VM bits that the buffer cache relies on are not safe. This work has been done with the expectation that the VM and VFS subsystems will be giant free soon.
The ULE scheduler has been committed to the 5.0-CURRENT branch. Early adopters and experimenters are welcome to try it and submit bug - reports. It has shown noticable performance improvements over the old + reports. It has shown noticeable performance improvements over the old scheduler under some workloads. There are currently problems with nice fairness but otherwise the interactive performance is very good. More work to improve the load balancing algorithm is required as well. This should be ready for use by the general FreeBSD user base in the next month or so.
Some improvements have been made to the clustered read ahead code. They allow for many more outstanding IO requests when an application does sequential access. This has a larger impact on RAID systems than on single disk systems. The maximum number of file system blocks that we will read ahead is tunable via the 'vfs.read_max' sysctl. This optimization has shown a 20% improvement in simple tests.
Locking of the non-obj parts of newbus is nearing completion. A single lock is used for the device tree. Minimal changes to subr_bus have so far been necessary to make this work, however some lock order issues remain. After this work, it will no longer be necessary to hold Giant to call device_* routines safely. kobj work is being done by others and - will likely require more extensive design work to make smp + will likely require more extensive design work to make SMP friendly.
The objective of this effort is to improve the performance, stability, and correctness of the BSD networking stack by adding support for new standards and standards track proposals while maintaining compliance with existing specifications. The upcoming 4.8 and 5.1 releases will be the first ones using the new NewReno logic. Recently, we implemented the Limited Transmit algorithm (RFC 3042) which benefits - connections with small congestions windows, as happens, for example, + connections with small congestion windows, as happens, for example, on many short web connections. We also recently added support for larger sized starting congestion windows as described in RFC 3390. This helps short TCP connections as well as those with large round-trip delays, such as those over satellite links.
The list of subsystems locked up include IP, UDP, TCP, ifaddr reference counting, syncache, the ifnet list, routing radix trees, and ARP. These have already been committed into the tree. In addition, SMP locking for raw IP, divert socket processing, and Unix domain sockets have also recently been completed and tested. Work is currently being done in some of the subsystems required to make parallel networking processing SMP-safe.
Last month's status report was apparently a great success: I received countless e-mails with comments, questions, and suggestions. I've tried to incorporate any suggestions and address any problems from these e-mails in this month's report, which captures a far more extensive snapshot of FreeBSD activity in the last month. Unlike last month's report, it does a better job of reflecting non-development activity, such as on-going conference planning, documentation, and so on. This is a trend I hope to see improve in future months as well.
On the topic of conferences, in the future I'd like to report more on publication activities relating to FreeBSD, including online journals with articles relating to FreeBSD, paper journals, conference papers, and so on. Likewise, I would be interested in including references to Call for Papers relating to FreeBSD. I'll take this opportunity to plug both registration and paper submission for BSDCon Europe in November, which has status included in this report, and for the general BSD Conference being hosted by USENIX in February. Your attendance and submissions make these conferences "happen", and promote FreeBSD as a platform for new research, feature development, and application products. Work of extremely high calibre is performed on FreeBSD, and we need to get the word out.
Next month, we're maintaining much the same submission requirements: reports should be one or two paragraphs long, sent by e-mail, and approximate the layout of the entries this month (Project, Contact, URL, and text). I'll send out reminders again over the week before the deadline, with more specific instructions. An area where I'd like to explore improvement lies in the coordination of related status reports for larger projects, such as new architectural work or platform ports. This might even have the effect of encouraging communication within these projects :-). I'd like to continue to focus on pulling in a broader range of groups and their activities, including the Security Officer, Release Engineer, and Core Team.
-- Robert Watson < rwatson@FreeBSD.org >
ACPI (Advanced Configuration and Power Interface) is an industry standard which obsoletes APM, Intel MPS, PnPBIOS, and other Intel PC firmware interface standards. It is also used on the IA64 platform. More information on ACPI is available at
http://developer.intel.com/technology/iapc/acpiThe FreeBSD ACPI subsystem project is based heavily on the Intel ACPI Component Architecture. This status report outlines the current state of the project; future updates will focus on changes as they occur.
The Intel ACPI interpreter is fully integrated, although bugs are still coming out of the woodwork occasionally.
Work is ongoing in the following areas:
The ARM port is currently going pretty well. The kernel is compiling and is able to boot to the point where it panics trying to initialize the network subsystem. The current reference platform is the Netwinder but this may change as many people expressed interest in a more broadly available platform. Things that need to be done before it can get further includes adding footbridge, timer and interrupt supports. The pmap module is not completed yet either.
Now that BIND 8.2.4 is finally imported the time has come to look at getting BIND 9 imported into CURRENT. The current idea is to have it imported alongside BIND 8 so that people can play with either one until all import problems have been taken care of and people have tested it a bit.
Although gaining a new name, the project has been at a standstill due to both resource availability during the move between BSDi and Wind River, and other commitments of the developers. The project should obtain an official mailing list, as well as return to an active state after the dust settles.
The conference will take place at the Thistle Hotel, Brighton, UK from 9-11 November 2001.
The aim of the conference is to provide a focal point for European users and developers of all the BSD derived operating systems. The format will be similar to other conferences, with 2 days of technical sessions over the Saturday and Sunday.
-We'll be finalising the schedule towards the end of the month +
We'll be finalizing the schedule towards the end of the month and anybody who is interested in doing a talk should contact us - asap. There are no restrictions on the use of talks, if it's been + ASAP. There are no restrictions on the use of talks; if it's been done before we may still be interested in having it presented to an European audience, and we make no claims to the talks so speakers are free to present the talks again at other conferences.
We're also still looking for sponsors.
We had 80 pre-registrations in the first week so we're expecting a good turnout.
The new CAM transport code is starting to get supported in more HBAs and to get refined so that it does the intended - per-protocol support. No progress on doing any SMPNG work for CAM + per-protocol support. No progress on doing any SMPng work for CAM has been made yet. This is a fairly high priority.
Thanks to various outstanding individual efforts, we are now down to just below 2300 open bug-reports. This means that we have fought our way back to the level we had around march 2000.
Work continues (in large part sponsored by WRS) on updating the Handbook ready for the second print edition. There has been a flurry of activity in this area recently, and the ToDo list can be seen at
http://www.freebsd.org/docproj/handbook.html
Dima and others are doing a stellar job of keeping up with the steady flow of incoming PRs relating to the documentation project.
The Developers' Handbook,
http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/index.html
is a year old; it contains a wealth of useful content for developers developing on, or for, FreeBSD. As ever, more contributions are always required, not only for the developers' handbook, but for all of the FreeBSD documentation set.
The basic design hasn't changed and this project mainly is in the phase of continued hardening and test case development. The next major feature will be to fully integrate into the new CAM TRAN code and to fully support on the fly device addition and removal. The only HBA supported is QLogic at this time. Future support for the QLogic line is planned to have 2300 (2Gb) and IP support before October.
Hardware watchpoints are now available for kernel debugging on the IA32 (i386) architecture. One can now set hardware watchpoints using the new ddb command 'hwatch', which is analogous to the existing 'watch' command. Alternatively, if greater flexibility is required, direct access to the debug registers is available using the ddb 'set' command which allows complete control over the processor hardware debug facilities. Hardware watchpoints are very useful in tracking down those elusive memory overwrite bugs in the kernel. Hardware watchpoints can even be used to set a code breakpoint in ROM, which is commonly found in embedded systems.
Support for configuring IEEE 802.11 wireless devices via ifconfig has been committed to -current and -stable. It contains most of the functionality needed to configure an wireless device. Some missing features are being worked on including integrated support for DHCP so a single entry in /etc/rc.conf can be used to fully configure a wireless device on a DHCP lan and setting the CTS/RTS threshold. Currently the an(4) and wi(4) drivers are supported in -current and -stable with the awi(4) device supported in -current. Further work is needed to support Frequency Hopping devices such as ray(4).
jailNG is a from-scratch rewrite of the popular jail(8) - service, focussing on improved management functions, as well as + service, focusing on improved management functions, as well as more fine-grained configurability. An initial prototype has been written, based on explicitly named and configured jails, and work is proceeding on userland integration. Currently, it's not clear if the timeline for this will be 5.0-RELEASE, or 5.1-RELEASE.
The main development in the FreeBSD Java Project over the last month was the release of an initial "Developers Only" patchset for the JDK 1.3.1. Since that release progress had been made - towards a much more useable alpha quality patchset which is + towards a much more usable alpha quality patchset which is likely to be turned into a port, as per the current JDK 1.2.2 patchset. This new patchset will feature a number of bugfixes, which essentially get the JDK to a working state for early adopters, and an initial implementation of "native threads" based on FreeBSD's userland pthreads. Unfortunately this implementation - isn't fully functional, but is included in the hope of more - getting more eyesballs on the code (particularly experience + isn't fully functional, but is included in the hope of + getting more eyeballs on the code (particularly experienced pthread programmers). We'd also like to welcome Fuyuhiko Maruyama-san as a new committer, the usual punishment for too many good patches.
We have been working to provide Japanese version of FreeBSD online manuals, since 1996. Currently, RELENG_4 manuals are based. Translated versions are placed on doc/ja_JP.eucJP/man and provided to users using ports/japanese/man-doc. Also, we discuss about related commands (e.g. ports/japanese/man and ports/japanese/groff).
The first FreeBSD kernel summit meeting was held June 29-30, 2001 in Boston, MA at the Usenix 2001 Annual Technical Conference. Links to a variety of files are posted on the web site.
Note: I (jhb) am still working on writing up a general summary of the meeting. When that is completed it will be posted here and mailed to the -hackers mailing list.
I'm working on multithreading the kernel. So far I have over 400KB of diffs relative to todays -current (I'm keeping my tree updated with changes as they occur rather than get hit with a big - updte at the end).
+ update at the end).I have split the proc structure and am changing most of the kernel to pass around a thread identifier instead of a proc structure.
The following interfaces have been changed so far:
I have still a lot of work to go with a lot of "dumb editing" (s/struct proc \*p/struct thread \*td/) usually I change a few items and then fix everything that breaks when I try compile it. I'd like to check it in on a branch so others can help the editing but haven't worked out the best way to do it yet.
-I have implemented changes to the scheduler so that kse's are +
I have implemented changes to the scheduler so that KSE's are scheduled instead of processes, and threads sleep, letting the - kse pick up a new thread. but it's not anywhere ready yet (heck + KSE pick up a new thread. but it's not anywhere ready yet (heck it doesn't compile yet :-)
Note that I have not yet updated the document listed above.. everywhere it mentions "ksec" or "KSE-context", the code uses the word "thread". I will update it soon as Jason has sent me the source.
The FreeBSD Monthly Development Status Report aims to keep users and developers up-to-date on the latest goings-on in the FreeBSD project by providing summaries of each project and its status. At the time of this writing, the July 2001 status report is being prepared and is very near release. The FreeBSD Web site now has a Status Reports section, which, when the July 2001 report is released, will be updated to include a link to an HTML-ified version.
The NetBSD rc.d port aims to improve the FreeBSD startup process by porting Luke Mewburn's rc.d work from NetBSD to FreeBSD. This will score FreeBSD startup and shutdown dependencies without losing the traditional and much loved - monolothic configuration filesystem.
+ monolithic configuration filesystem.Luke Mewburn's USENIX paper and slides on the system as implemented in NetBSD are available here:
http://groups.yahoo.com/group/FreeBSD-rc/message/3
Interested parties are urged to study this material before joining the discussion list.
The intention at this stage is to decide on an approach that will ensure that the differences between the NetBSD rc.d system and the system as ported to FreeBSD will be kept to a minimum. This will probably involve discussions with Luke around those areas of the system that are identified as areas for potential improvement.
The goal of this project is the implementation of ATM signalling and other ATM protocols by means of the netgraph(4) - framework. This should provide an easily extendable architecture + framework. This should provide an easily extensible architecture for using ATM on FreeBSD. Currently the full UNI4.0 stack (except for the LIJ capability) has been implemented, including ILMI and a first version of the ATM Forum API for UNI. An implementation of Classical IP over ATM is also available. Drivers have been implemented for the Fore PCA200E and Fore HE-155 cards.
Network device cloning support has been imported from NetBSD. This allows virtual devices to be allocated on demand rather then - being staticly allocated at compile time. Our implementation + being statically allocated at compile time. Our implementation differs slightly from that of NetBSD's in that we allow both the creation of specific devices (i.e. gif0) and arbitrary devices instead of just allowing specific devices. Currently, the only device in the tree which has been converted is the gif(4) device which has been converted in both -current and -stable. Work is ongoing to convert all other virtual network devices with work in progress on faith, stf, and vlan interfaces. In general this conversion is accompanied by appropriate modifications to make these devices fully modular.
NGPT is an effort led by IBM engineers to implement MxN threads (also known as many user threads to one kernel thread mapping) on Linux. I have ported it to FreeBSD to use rfork(2).
The port is right here:
Funded by: Monzoon Networking, LLC
This month has been a month of conventration and consolidation. Much of the changes from current have been migrating into stable. I've improved power support, suspend/resume interactions, interrupt handling, and ability to work after windows/NEWCARD has run. Interrupt routing continues to be a locking issue for a complete MFC. Current patches are available at the above website. I'm racing to get this done before 4.4 is released.
Information on Intel ORP - a BSD licensed Java VM is right here:
http://www.intel.com/research/mrl/orp/
A FreeBSD patch has been tested to work with NGPT and submitted to the ORP project. The patch is available here:
http://www.sharma-home.net/~adsharma/projects/orp/orp-freebsd-1.0.5.patch.txt.gz
There are some issues to be ironed out to make it work with FreeBSD's default (user level) pthread implementation.
OpenPackages intends to create a software packaging system that will allow third-party programs to be installed, without operating system dependent changes, on as many platforms as are feasible. OpenPackages was originally based on code from the BSD ports systems, and has been improved and extended by developers of many heritages.
The OpenPackages Project is pleased to release the Milestone 2 codebase. This release contains a working package building system and a single test package. OP currently is known to build on certain instances of the following operating systems: FreeBSD, HP/UX, IRIX, Linux (Debian, Red Hat, Suse, Mandrake, TurboLinux, Caldera, etc.), NetBSD, OpenBSD, Solaris
(First report)
Large cleanup and extension of FreeBSD PAM modules. All - modules are to be documented, consistant in style (style(9) used) + modules are to be documented, consistent in style (style(9) used) and as complete as possible WRT functionality. Mostly done.
We now have the rudiments of device support. We have a nexus driver for OpenFirmware machines, along with support for the Apple UniNorth PCI/AGP host bridge. I'm currently trying to get the USB hardware working so that I can get closer to having a - console driver independant of OpenFirmware, then I'll be trying + console driver independent of OpenFirmware, then I'll be trying to get the system to get to single-user mode using NFS.
Work has begun, but nothing has yet been committed. The NCP addresses used by ppp have been abstracted and initial support has been added to the filter set for ipv6 addresses. NCP negotiation hasn't yet been started.
Patches have been submitted to get ppp working under HURD, and mostly under Linux. There are GPL copyright problems that need to be addressed.
Making pppoed function in a production environment. Most of the work is complete and committed. Additional work includes adding a -l option where ``-l label'' is shorthand for ``-e exec ppp -direct label'' and discovering why rogue child processes are being left around.
PRFW is a set of hooks which I have integrated into the FreeBSD kernel. This allows modules to easily intercept system calls with less overhead. It also supports per-pid restrictions, which means, one process may not be able to use X function in Y manner, but another process may.
Progress: I was working on this in 4.3-RELEASE, but now I'm merging it into current. I will be submitting a patch to the mailing lists in about a week.
This driver is currently not working well under -current and is undergoing some work at this time. No major design or feature changes are planned. There was some notion of adding TapeAlert support, but HP supports that as a binary product via a user library and it was felt that it'd be more politically prudent to leave it alone.
In the 'smpng' p4 branch there is code to make the ast() function loop to close the race when an AST is triggered while we are handling previously triggered AST's.
In the 'jhb_preemption' p4 branch work is being done to make the kernel fully preemptive. It is reportedly stable on UP x86, but SMP x86 locks up, UP alpha has problems during shutdown and can recurse indefinitely until it exhausts its stack.
We are using a perforce repository for live development work, - which can track multiple seperate long-lived works-in-progress + which can track multiple separate long-lived works-in-progress and collaborate between multiple developers at the same time on the same change set.
FreeBSD-current is being imported into p4 hourly, for easy tracking of the moving -current tree.
I haven't written up a good primer yet, but we're able to open this up to the general developer community. NEWCARD work looks like it will be done here too. Perforce is ideal for tracking this sort of long-lived project without having to resort to passing patches around.
KSE work is now being checked into a kse p4 branch - thanks Julian!
KSE work is focusing on getting the main API changes into the base tree well before 5.0.
mb_alloc is a specialized allocator for mbufs and mbuf clusters. It offers various important advantages over the old - mbuf allocator, particularily for MP machines. Additionally, it + mbuf allocator, particularly for MP machines. Additionally, it is designed with the possibility of important future - enchancements in mind.
+ enhancements in mind.The mb_alloc code has been committed to -CURRENT a month ago and appears to be holding up well. Prior to committing it, preliminary performance measurements were done merely to ensure that it is not significantly worse than the old allocator, even with Giant still in place. Results were promising [http://people.freebsd.org/~bmilekic/code/mb_alloc/results.html] - also see jlemon's results (link at the bottom of accompanying text). Since the commit, Matt Jacob has provided useful feedback and bugfixes. Work is now being done to re-enable mbtypes statistics and make appropriate changes to netstat(1) and systat(1).
The sparc64 port has been committed to the FreeBSD repository. As such further development will occur in cvs, rather than as a separately maintained patch set. Significant progress has been made since the last status report, including; support for kernel debugging with ddb, much more complete pmap support, support for context switching and process creation, and filling out of important machine dependent data structures. Thomas Moestl has shown a strong interest in working on the port and is in the process of implementing support for saving and restoring a process's floating point context. I look forward to working with him and any other developers that happen to fall out of the wood works.
The sparc64 loader is functional enough to boot an ELF binary from an UFS filesystem using the existent openfirmware library, which has been revised to work flawlessly on 32-bit and 64-bit architectures. Support for netbooting and modules will be implemented next, followed by a better openfirmware mapping strategy.
This project brings a SYN cache implementation to FreeBSD, in order to make it more robust to DoS attacks. A SYN cookie - approach was considered, but ultimately rejected becuase it does + approach was considered, but ultimately rejected because it does not conform to the TCP protocol. The SYN cache will work with T/TCP, IPV6 and IPSEC, and the size of each cache element is currently is less than 1/5th the size of a normal TCP control block.
It's been a busy month, with a number of relevant news items. Not least important is that NAI Labs was awarded a $1.2M contract from the US Defense Advanced Research Projects Agency (DARPA) to work on a variety of components relevant to the TrustedBSD Project, including support for pluggable security models, and supporting features such as improving the extended attributes implementation, simple crypto support for swap and filesystems, documentation, and much more.
On the features side, progress continues on Mandatory Access Control, object labeling, and improving the consistency of kernel access control mechanisms--in particular, with regard to inter-process authorization and credential management. Work has begun on porting LOMAC, NAI Labs' Low-Watermark Mandatory Access Control scheme, from Linux to FreeBSD, and it has been re-licensed under a BSD license. We hope to have an initial port complete in time for 5.0-RELEASE later this year.
Throughout July and August, the FreeBSD Project has been working on pulling together the last few major pieces of new functionality for FreeBSD 5.0-RELEASE. At this point, the release appears to be on track for late November or early December. Work on fine-grained locking continues, especially in the VFS, as with improved support for threading through the KSE work; features such as GEOM, UFS2, and TrustedBSD MAC are maturing, and the new ia64 and sparc64 hardware ports are approaching production quality. In the next two months, we have a lot to look forward to: additional 5.0 developer preview snapshots, additional locking and threading improvements, and many cleanups on the new supported architectures. Firewire support has been imported into the main tree, and substantial cleanup of the ACPI/legacy PCI code is also in the works. Also, expect the import of new IPsec hardware acceleration support in the near future.
When new developer previews are posted, please give them a try! While we know that 5.0-RELEASE will be for "early adopters", the more testing we get out of the way now, the less we have to tidy up later. The new features are extremely exciting, and understanding when and how to deploy them properly will be important. In the next two months, among other things, the release engineering team will post updated release schedules, as well as guidance for FreeBSD consumers as to how to decide what releases of FreeBSD will be right for them. Keep an eye out for this, and provide us with feedback.
Also, for those of you in Europe -- we look forward to seeing you at BSDCon Europe in a couple of months!
Scott Long, Robert Watson
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
Cloning support for ppp(4) and disc(4) interfaces has been committed. A man page for disc has been created and the disc devices now appear as disc# instead of ds#. Some work is still needed on pppd to make it understand cloning though it should work as long as the devices are created beforehand.
On the API front, management of mandatory interfaces (i.e. lo0) is handled by the generic cloning code so if_clone_destroy has the same API as NetBSD again and <if>_modevent doesn't need to create the necessary devices manually.
At this point, all pseudo interfaces have been converted to the cloning API or already did their own cloning (sl(4) for example uses it's own mechanism). Some devices such as tun(4) and tap/vmware should probably be converted to use the cloning API instead of their current ad-hoc, devfs based cloning system. This would be a good junior kernel hacker task. Also, the handbook and FAQ could use some general cloning documentation prior to 5.0 release.
We have been updating RELENG_4 targeting for 4.7-RELEASE. When port ja-man-1.1j_5 was broken around the end of July, Kumano-san and Mori-san tried to update the port to be based on a newer FreeBSD base system's man commands. But, we decided only to fix the port ja-man-1.1j_5 to be buildable, as the new one was not complete at that time.
The GEOM code has gotten so far that it beats our current code - in some areas while stil lacking in others. The goal is for + in some areas while still lacking in others. The goal is for GEOM to be the default in 5.0-RELEASE.
Currently work on a cryptographic module which should be able to protect a diskpartition from practically any sort of attack is progressing.
The UFS2 filesystem approaches feature completion: Extended attribute functionality have been added, including a new compound modification API and basic testing has been passed.
We've got currently almost 50% of the new handbook translated (all the installation part is translated). Most of the articles are translated too.
The web site in on the way, see the Web Server. We need now to integrate it on the US CVS tree.
One of the big job now, is to translate the latest FAQ and the very big project will be the manual pages
I'm very pleased to announce that another engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20020909.tar.gz
This release features several major changes and includes support for H4 UART and H2 USB transport layers, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices. Also there are several man pages.
Service Discovery Protocol (SDP) is now supported. This release includes SDP daemon, configuration tool and user space library (ported from BlueZ-sdp-0.7).
RFCOMM is now supported. This release includes rfcommd daemon that provides RFCOMM service via pseudo ttys. Not very useful for legacy application, but it is possible to run PPP over Bluetooth now. This was ported from old BlueZ-rfcommd-1.1 (no longer supported by BlueZ) and still has some bugs in it.
Next step is to fix current RFCOMM support and work on new in-kernel RFCOMM and BNEP (Bluetooth Network Encapsulation Protocol) implementation. Also user space need more work (better tools, libraries, documentation etc.).
Version 1.2 has been released recently. It should compile and work an any recent FreeBSD-current. Support to manipulate SUNI registers has been added to the ATM drivers (to switch between SONET and SDH modes, for example). The ngatmsig package now includes a small and - simple call control module that mayh be used to build a simple ATM + simple call control module that may be used to build a simple ATM switch. The netgraph stuff has been patched to use the official netgraph locking.
On the API front, fmtmsg(3) was implemented, glob(3) was given support for new flags, ulimit(3) was implemented, and wide character/string support was significantly improved with the addition of 30 new functions (see the project status board for details). Work is progressing on adding the C99 restrict type-qualifier to functions throughout the system. This allows the compiler to make additional optimizations based on the knowledge that a restrict-qualified argument is the only reference to a given object (ie. it doesn't overlap with another argument).
Several headers have been brought up to conformance with POSIX.1-2001, they include: <fmtmsg.h>, <poll.h>, <sys/mman.h>, and <ulimit.h>. The header <cpio.h> was implemented. The headers <machine/ansi.h> and <machine/types.h> were merged into a single header to help simplify the way variable types are created.
The sh(1) built-in, command(1), was reimplemented to conform with POSIX. Additionally, several utilities which were previously brought - up to conformance were merged into the 4-STABLE banch.
+ up to conformance were merged into the 4-STABLE branch.The GNOME 2 desktop port has reach version 2.0.2rc1 with an expected 2.0.2 release before 4.7-RELEASE. Mozilla 1.1 has been ported, and is resident in the tree with Mozilla 1.0.1. The GNOMENG porting effort is going well. A good deal of ports have been moved to the new infrastructure with the help of Edwin Groothuis. We are now working on smoothing out some of the rough edges, then, once all the work is done, make GNOMENG the default.
A long-standing annoyance in Nautilus has also been recently - corrected. The desktop is no longer clutered with volume icons, and + corrected. The desktop is no longer cluttered with volume icons, and removable media (such as CDs) should now be handled correctly.
The ATAPI/CAM module allows ATAPI devices (CD-ROM, CD-RW, DVD drives, floppy drives such as Iomega Zip, tape drives) to be accessed through the SCSI subsystem (CAM). ATAPI/CAM has been integrated in -CURRENT. The code should be fairly functional (it has been used by many testers as patches against -STABLE and -CURRENT over the past eight months), but there are pending issues on SMP machines. Testers most welcome.
A MFC of this feature will probably happen after the end of the 4.7 code freeze.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPSEC), and OpenSSL (through the /dev/crypto device).
OpenSSL 0.9.7 beta 3 was imported and patched with fixes from OpenBSD's source tree. This permits any user-level application that use -lcrypto to - automatically get hardware crypto acceleration. Otherwse the core crypto + automatically get hardware crypto acceleration. Otherwise the core crypto support is stable and has been in production use on -stable machines for several months.
Import of this work into the -current tree has started. A publicly available patch against 4.7 will be released once 4.7 ships. Integration of this work into the -stable source tree is planned for 4.8.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
Recent work focused on increasing performance. Support is still limited to IPv4 protocols, with IPv6 support coded but not yet tested.
Import of this work into the -current tree has started. A publicly available patch against 4.7 will be released once 4.7 ships.
Work is in progress to MFC a number of bug fixes related to vm_map corruption into -stable. This work is probably too involved to make it into the 4.7 release but is expected to - be comitted just after the freeze is lifted. The corruption + be committed just after the freeze is lifted. The corruption in question typically occurs in large-memory systems under heavy loads and typically panics or KPFs (kernel-page-fault's) the machine in a vm_map related function.
The existing SCSI target code has been rewritten. The kernel driver is much simpler, deferring all functionality to usermode and simply passing CCBs to and from the SIM. The supplied usermode emulates a disk (RBC) with IO going to a backing file. It replaces /sys/cam/scsi/scsi_target* and /usr/share/examples/scsi_target.
The code is definitely alpha quality and has known problems on -current although it appears to work ok on -stable. See the included README for how to install and test. Feedback is welcome!
Yet another implementation of Lottery Scheduling devised by Carl Waldspurger et. al. is being developed against FreeBSD -STABLE branch. It is being developed as part of a graduation project in Computer Science at Universidade de Brasília in Brazil. Therefore, other implementations have not yet been verified to avoid plagiarization but will be checked in a later stage of this project searching for better implementation ideas. Currently, part of the necessary scheduling kernel structure has been mapped and work has progressed despite the general lack of kernel documentation. Further outcomes of this project will be a simple documentation of the kernel scheduler structure of -STABLE branch, a port of the Lottery Scheduler to -CURRENT branch and additional implementations of other scheduling disciplines from Carl Waldspurger et. al. Members of the FreeBSD community have been and will continue to be instrumental in both testing and providing feedback for ideas implemented here.
The FreeBSD Brazilian Portuguese Documentation Project is merging with a translation group formed by members of the FUG-BR FreeBSD Brazilian user group. The Brazilian Project decided to become an official group under FUG-BR after receiving continued excellent contributions from them. They have managed to complete the translation of the FreeBSD FAQ which is currently undergoing both proofing and SGML"fication" stages. Work is progressing fast: the Handbook has been half translated and articles are under way. The previous Brazilian Project is proud to become part of such a dedicate group. The contacts above represent the current official contacts for the new translation group. We hope to have at least part of this work ready for the FreeBSD 4.7 Release.
David Xu and I have been working on cleaning up some of the work done in KSE-III and Jonathon and Dan have been working on the userland - interface. The userland libray will be committed soon in a + interface. The userland library will be committed soon in a prototypical state and a working test program using that interface will hopefully accompany it. I have just committed a rework of the run states for kernel threads that simplifies or solves some problems that were being seen recently.
Hopefully in the next few weeks we will be able to run threads on separate processors. The basics of Signal support are presently evolving. Archie Cobbs will also be assisting with some of this work. I have a mail alias for all the developers at kse@elischer.org. It is managed by hand at the moment.
The Release Engineering (RE) Team completed and released FreeBSD 4.6.2. This ``point release'' fixes several important bugs in the ATA subsystem, as well as addressing a number of security issues in the base system that surfaced shortly after FreeBSD 4.6 was released. The release documentation distributed with FreeBSD 4.6.2 contains more details. (Note: Some earlier documents and reports referred to this release as version 4.6.1.) The next release in the 4.X series will be FreeBSD 4.7, which has a scheduled release date of 1 October 2002.
Concurrently, work is continuing on the 5.0-DP2 developer preview snapshot, an important milestone along the release path of FreeBSD 5.0, which is scheduled for release on 20 November. As 5.0 draws closer, we are focusing more on getting the system stabilized, as opposed to adding new functionality. To help us with this effort, developers should discuss with us any new features planned for -CURRENT, beginning 1 October.
The project runs as it should be. New security-branch snapshots are available for both 4.5 and 4.6(.2). I've update buildboxes OS to the latest 5-current/4-stable without any errors. Also current problem, less CPU power for the future, is not solved yet -- but situation is not so bad, I hope I'll show a good news in the next report.
The Donations team started rolling in the last couple of months. Offers of equipment are coming in, and we are allocating them to FreeBSD committers as quickly as possible. We now have a "Committer Want List" available in our section of the Web site. Several small items, such as network cards, have been routed to people who are willing to write the code to support them. We have a few larger donations (i.e., actual servers) ready to go to developers, once shipping information is straightened out.
Work on RAIDFrame stalled for quite a bit, then it picked up in early summer, then it stalled, and now it's going again. A significant amount of work has been done to make the locking SMPng-friendly and to cut down on kernel stack abuse. I'm happy to say that it's starting to work reliably when used with file- backed 'md' disks. Even more exciting is that it's finally starting to work on real disks, too. A lot of cleanup is still needed, and a few gross hacks still exist, but it might actually be ready for the FreeBSD 5.0 release. Patches for FreeBSD 5-current and 4-stable are available from the website. The 4-stable patches are a year old but still apply and perform well.
The primary libh development box, where the CVS repo and development webpage was living, is dead. The server has crashed after a system upgrade and has never came back to life. We had to pull the drives out of it to make proper backups. We will setup another box in place of this one and hope for the best. So right now, the port is broken because the CVS is unaccessible, as the development web page. We're working on it, please bear with us.
On a brighter note, Max started implementing the changes he proposed to the build system and the TCL API; LibH is switching to SWIG for its TCL bindings, which should simplify the system a lot, and shorten build times. The Hui subsystem is therefore being completely re-written. On my side, I made a few tests in building and running LibH under rhtvision, and it didn't fulfill the promises I thought it would, so I just put aside that idea. Work on libh stalled during July because I completely lost network access for the whole month. So right now, LibH is in a bit of a mess, but we have high hopes of settling everything down to a new release pretty soon, which will make full use of the new SWIG bindings.
The Security Team continues to be very busy. The security-officer mailing list traffic for the months of June, July, and August consisted of 1,230 messages (over 13 messages a day). This is well over 50% of the freebsd-hackers traffic volume in the same period!
Since June (the time of our last report), 9 new Security Advisories were published, and one Security Notice was published covering 25 Ports Collection issues.
FreeBSD 4.6.2-RELEASE was released on August 15th. This marked the first time a point release was created from the security branch. The process went smoothly from the Security Team perspective, despite a schedule slippage due to newly discovered bugs, and a snafu which resulted in 4.6.1-RELEASE being skipped.
In September, the FreeBSD Security Officer published a new PGP key (ID 0xCA6CDFB2, found on the FTP site and in the Handbook). This aligned the set of those who possess the corresponding private key with the membership of the security-officer alias published on the FreeBSD Security web site. It also worked around an issue with the deprecated PGP key being found corrupted on some public key servers.
It's been a busy few months, with a variety of development, documentation, and public relations activities. The MAC Framework, our pluggable kernel access control mechanism for FreeBSD, has matured substantially, and large parts of it were merged to the main FreeBSD tree over July and August.
A variety of entry point changes were made, including: component - names are now passed to VFS namespace VOPs; agressive caching + names are now passed to VFS namespace VOPs; aggressive caching of MAC labels in vnodes; mmap memory access downgrades on subject relabel; check for access()/eaccess(); checks for vnode read, write, ioctl, pool, permitting revocation post-open() by aware policies; labeling and access control checks for pipe IPC objects, clean up of socket/visibility checks; checks for socket bind, connect, listen, ....; many locking improvements and assertions, especially for vnodes, processes; framework now supports partial label updates on subjects and objects; credential management in 'struct file' improved so that active_cred and file_cred are more carefully distinguished and passed to MAC framework explicitly; accounting system uses cached credentials for write operations now; socreate() can use cached credential to label sockets fixing deferred nfs socket connections and reconnections with TCP; kse interactions with proc1 fixed; IO_NOMACCHECK flag to vn_rdwr() for internal use to avoid redundant or incorrect MAC checks on aio vnode operations; mac_syscall() policy function demux; su no longer changes MAC labels by default; mac_get_pid() to support ps and getpmac -p pid; mmap revocation defaults to "fail stop"; MAC_DEBUG wraps atomic label counters; UFS2 extended attributes supported; initial port of LOMAC to the MAC framework; update all policies for all these changes; merge of KSE III; merge of nmount(); upgrade of ugidfw to speak user and group names; libugidfw; many namespace and naming consistency improvements; module dependencies on MAC framework; large scale merging of MAC functionality to the main FreeBSD tree. KDE interfaces to common management activities.
Wrote and taught full-day MAC framework tutorial at STOS - BSD and Darwin Security Symposium; first draft of MAC fraemwork + BSD and Darwin Security Symposium; first draft of MAC framework architecture and API guide. This is now in the Developer's Handbook.
Next couple of months will bring continued maturity improvements, labeling and protection of more objects; VFS performance - improvements; better support for UFS2 EAs and seperate EA + improvements; better support for UFS2 EAs and separate EA entries for each policy; improved support for LOMAC; MLS compartments; IPsec security association labeling; improved SEBSD FLASK/TE port; and much more.
One of the benefits of the FreeBSD development model is a focus on centralized design and implementation, in which the operating system is maintained in a central repository, and discussed on centrally maintained lists. This allows for a high level of coordination between authors of various components of the system, and allows policies to be enforced over the entire system, covering issues ranging from architecture to style. However, as the FreeBSD developer community has grown, and the rate of both mailing list traffic and tree modifications has increased, making it difficult even for the most dedicated developer to remain on top of all the work going on in the tree.
The FreeBSD Monthly Development Status Report attempts to address this problem by providing a vehicle that allows developers to make the broader community aware of their on-going work on FreeBSD, both in and out of the central source repository. This is the first issue, and as such is an experiment. For each project and sub-project, a one paragraph summary is included, indicating progress since the last summary (in this case, simply recent progress, as there have been no prior summaries).
This status report may be reproduced in whole or in part, as long as the source is clearly identified and appropriate credit given.
Assuming there is some positive feedback on this idea, and that future submissions get made such that there is content for future issues, the goal is to release a development status report once a month. As such, the next deadline will be July 31, 2001, with a scheduled publication date in the first week of August. This will put the status report on a schedule in line with the calendar, as well as providing a little over a month until the next deadline, which will include a number of pertinent events, including the Annual USENIX Technical Conference in Boston, MA. Submissions should be e-mailed to:
robert+freebsd.monthly@cyrus.watson.org
Many submitters will want to wait until the last week of July so as to provide the most up-to-date status report; however, submissions will be accepted at any time prior to that date.
-- Robert Watson < rwatson@FreeBSD.org >
The FreeBSD Binary Updater Project aims to provide a secure mechanism for the distribution of binary updates for FreeBSD. This project is complementary to the Open Packages and libh efforts and there should be very little overlap with those projects. The system uses a client / server mechanism that allows clients to install any known "profile" or release of FreeBSD over the network. Where a specific profile might contain a specific set of FreeBSD software to install, additional packages, and configuration actions that make it more ideal for a specific environment (ie FreeBSD 4.3 Secure Web Server Profile)
The system can currently be used to install a FreeBSD system or perform the most simple of upgrades but many features are absent. In particular, the client is in its infancy and much work remains to be done. We need additional developers so please get in touch with us at updater@osd.bsdi.com if you are interested in spending some cycles on this.
Poul-Henning Kamp kicked off a drive to get our GNATS PR database cleaned up so the wheat can be sorted from the chaff. Progress is good, but there is still a lot of work to do. Give a hand if you can. Remember: every unhandled PR is a pissed off contributor or user.
I'm in the process of rewriting the CVSROOT/scripts to make them more clean and configurable. A lot of other projects also use these and so it makes sense to make them as easy to use in other environments as possible.
Status: work in progress. There is now a configuration file, but not all the scripts use it yet.
Work is progressing on implementing true cloning devices in DEVFS. Brian Somers and Poul-Henning Kamp are working to make if_tun the first truly cloning driver in the system. Next will be the pty driver and the bpf driver.
From July 1st DEVFS will be standard in -current.
Added the digi driver. Initial work was done by John Prince <johnp@knight-trosoft.com>, but all the modular stuff was done by me and initial work on supporting Xe and Xi cards (ala dgb) was done by me. I'm now awaiting an Xe card being sent from joerg@ (almost a donation) so that I can get that side of things working properly.
Ben Smithurst has written a "diskcheckd" daemon which will read all sectors on the disks over a configured period. With recent increases in disksizes it is by no means a given that disk read errors will be discovered before they are fatal. This daemon will hopefully result in the drive firmware being able to relocate bad sectors before they become unreadable. This code is now committed to 5.0-CURRENT.
In the last month (May-June), the new fxp driver was brought into -stable. This new driver uses the common MII code, so support for new PHYs is easy to add. Support for the new Intel 82562 chips was added. The driver was updated to add VLAN support and a workaround for a bug affecting Intel 815-based boards.
The FreeBSD Java Project has continued its "behind the scenes" work over the last month. Progress was made both technically, with the help of Bill Huey (of Wind River), on a port of JDK 1.3.1 and legally, with Nate Williams continuing negotiations with Sun on a mutually acceptable license to release a binary Java 2 SDK under. The JDK 1.2.2 port has also seen some development, with a new patchset likely to be released soon which includes JPDA and NetBSD support (the latter courtesy of Scott Bartram).
The Kernel Graphics Interface project has worked for several years to provide a framework for graphic drivers under Linux receiving input from other groups like the UDI project. Currently the KGI core implementation is quite settled, as is the driver coding model as a whole. Work is being done to newbussify KGI and produce a kld, as part of a future redesign of the graphics subsystem in FreeBSD. KGI will be an alternative for graphic card producers that don't accept the XFree86 model of userland graphic adapters and will also provide accelerated support for any other graphic alternative.
The libh project is a next generation sysinstall. It is written in C++ using QT for its graphical frontend and tvision for its console support. The menus are scriptable via an embedded tcl interpreter. It has been growing functionality quite a bit lately, including a new disklabel editor. Current work is on installation scripts for CDROM, FTP, ... installs as well as a fully functional standalone disk-partition and label editor. The GUI API was extended a little and many bugs were fixed. There seems to be some interest in i18n work.
Maxime Henrion is working on implementing a new and more extensible mount(2) systemcall, mainly to overcome the 32 bits for mountoptions limit, secondary goal to make it possible to mount filesystems from inside the kernel.
In the last two months, the OLDCARD pccard implemenation was +
In the last two months, the OLDCARD pccard implementation was rototilled to within an inch of its life. Many new pci cardbus bridges were added. Power handling was improved. PCI Card cardbus bridges are nearly supported and should be committed in early June to the tree. This will likely be the last major work done on OLDCARD. After pci cards are supported, work will shift to improving NEWCARD.
The PowerPC port is proceeding well. All seems to be working in pmap.c after a number of problems encountered where FreeBSD passes a vm_page_t to a NetBSD-derived function that expects a vm_offset_t. Then after debugging the atomic operations code, I'm - now at the point where VM appears to be initialised and it's now + now at the point where VM appears to be initialized and it's now hanging while in sys/kern/kern_malloc.c:kmeminit(). Progress continues. =)
Developing full MPPE support for Andre Opperman @ Monzoon in Switzerland. Work is now complete and will eventually be brought into -current, but no dates are yet known.
Pseudofs is a framework for pseudo-filesystems, like procfs and linprocfs. The goal of pseudofs is twofold:
Pseudofs has reached the point where it is sufficiently functional and stable that linprocfs has been almost fully reimplemented on top of it; the only bit that's missing is the proc/<pid>/mem file.
The primary to-do item for pseudofs right now is to add support for writeable files (which are required for procfs, and are quite a bit less trivial to handle than read-only files). In addition, pseudofs needs either generic support for raw (non-sbuf'ed, possibly mmap'able) files, or failing that, special-case code to handle proc/<pid>/mem.
RELNOTESng is the name I've given to the rewrite of the *.TXT files that typically accompany a FreeBSD release. The information from these files (which include, among other things, the release notes and the supported hardware list) have been reorganized and converted to SGML. This helps us produce the documentation in - various formats, as well as facilitating the maintainence of + various formats, as well as facilitating the maintenance of documentation for multiple architectures. This work was recently committed to -CURRENT, and I intend to MFC it to 4-STABLE before 4.4-RELEASE.
The SMPng project aims to provide multithreaded support for the FreeBSD kernel. Currently the kernel still runs almost exclusively under the Giant kernel lock. Recently, progress has been made in locking the process group and session structures as well as file descriptors by Seigo Tanimura-san. Alfred Perlstein has also added in a giant lock around the entire virtual memory (VM) subsystem which will eventually be split up into several smaller locks. The locking of the VM subsystem has proved tricky, and some of the current effort is focused on finding and fixing a few remaining bugs in on the alpha architecture.
mb_alloc is a new specialized allocator for mbufs and mbuf clusters. Presently, it offers various important advantages over - the old (status quo) mbuf allocator, particularily for MP + the old (status quo) mbuf allocator, particularly for MP machines. Additionally, it is designed with the possibility of - future enchancements in mind.
+ future enhancements in mind.Presently in initial review & testing stages, most of the code is already written.
Work has (re)started on a port of FreeBSD to the UltraSPARC architecture, specifically targeting PCI based workstations. Jake Burkholder will be porting the kernel, and Ade Lovett has expressed an interest in working on userland. Recent work on the project includes:
At this point the kernel can be net-booted and prints the FreeBSD copyright before calling code that is not yet implemented. I am currently working on a design for the pmap module and plan to begin implementation in the next few days.
The TrustedBSD Project seeks to improve the security of the FreeBSD operating system by adding new security features, many derived from common trusted operating system requirements. This includes Access Control Lists (ACLs), Fine-grained Event Logging (Audit), Fine-grained Privileges (Capabilities), Mandatory Access Control (MAC), and other architecture features, including file system extended attributes, and improved object labeling.
-Individual feature status reports are documented seperately +
Individual feature status reports are documented separately below; in general, basic features (such as EAs, ACLs, and kernel support for Capabilities) will be initially available in 5.0-RELEASE, conditional on specific kernel options. A - performance-enhanced version of EAs is currently being targetted + performance-enhanced version of EAs is currently being targeted at 6.0-RELEASE, along with an integrated capability-aware userland, and MAC support.
Patches are now available to add ACL support to cp(1) and mv(1) along with preliminary support for install(1). Ilmar's i18n patches for getfacl(1) and setfacl(1) need to be updated for the last set of changes and committed. Some other functional improvements are also in the pipeline.
The kernel part of the capability implementation is mostly finished; all uses of suser() and suser_xxx() and nearly all comparisons of uid's with 0 have been converted to use the newly introduced cap_check() call. Some details still need clarification. More documentation for this needs to be done.
POSIX.2c-compatible getfcap and setfcap programs have been written. Experimental capability support in su(1), login(1), install(1) and bsd.prog.mk is being tested.
Support for capabilities, ACL's, capabilities and MAC labels in tar(1) is being developed; only the capability part is tested right now. Generic support for extended attributes is planned, this will require extensions to the current EA interface, which are written and will probably be committed to -CURRENT in a few weeks. A port of these features to pax(1) is planned.
An initial prototype of a Mandatory Access Control implementation was completed earlier this year, supporting Multi-Level Security, Biba Integrity protection, and a more general jail-based access control model. Based on that implementation, I'm now in the process of improving the FreeBSD security abstractions to simplify both the implementation and integration of MAC support, as well as increase the number of kernel objects protected by both discretionary and mandatory protection schemes. Generic object labeling introduces a structure not dissimilar in properties to the kernel ucred structure, only it is intended to be associated with kernel objects, rather than kernel subjects, permitting the creation of generic security protection routines for objects. This would allow the easy extension of procfs and devfs to support ACLs and MAC, for example. A prototype is underway, with compiling and running code and simple protections now associated with sysctl's.
May and June were remarkably busy months for the FreeBSD Project-- FreeBSD developers met in Monterey, CA in June for FreeBSD Developer Summit III to discuss strategy for the FreeBSD 5.0 release later this year, for the USENIX Annual Technical conference and for the FreeBSD BoF. Substantial technical progress was made on FreeBSD 5.0, and FreeBSD 4.6-RELEASE was cut on the RELENG_4 branch in June.
The remainder of the summer will continue to be busy. Final components and features for 5.0-RELEASE will go into the tree, and the development direction will change from new features to stability, performance, and production-readiness. With additional 5.0 development previews late in the summer, we hope to broaden the tester base for the -CURRENT branch, and start to get early adopters digging out any potential problems in their test environments. I encourage both FreeBSD Developers and FreeBSD Users to give 5.0-DP2 a spin (on a machine without critical data!) and let us know how it goes. The more testing that happens before the release, the less fixing we have to do afterwards!
Robert Watson
The current cache for the TCP metrics is embedded directly into the routing table route objects. This is highly inefficient as every route has an empty 56 Byte large metrics structure in it. TCP is the only consumer (except the MTU and Expiry field) of the structure. A full view of the Internet routes (110k routes) has more than 6 Mbyte of unused overhead due to it. The hit rate today is at only approx. 10% in webserver applications. The TCP hostcache will move this entire metrics structure from the routing table to the TCP stack. Every entry is a host entry so a simple hash table is sufficient to keep the entries. Its implementation is much like the TCP Syncache.
The hostcache is going through testing on our servers and will be ready for committing in September. The results of the TCP metrics measurement will be used to tune the cache.
The current Patricia Trie routing table in BSD UNIX is not very efficient and wastes an enormous amount of space for every node (more than 256 bytes) (A full Internet view of 110k routes takes 33 MByte of KVM). Another problem are pointers from and to everywhere in the routing table. This makes replacing the table very hard and - also significantly highers the table maintainance burden (for example - for some kinds of updates the entire PCB has be searched lineary). + also significantly increases the table maintenance burden (for example + for some kinds of updates the entire PCB has to be searched linearly). Also this is a heavy burden for SMP locking. The rewrite focuses on - untangeling the pointer mess, making the routing table replaceable + untangling the pointer mess, making the routing table replaceable and providing a more IP optimized table (5 MByte for 110k routes). - Other new options include policy routing and some structual alignments - in the network stack for clarity, cleaness and flexibilty.
+ Other new options include policy routing and some structural alignments + in the network stack for clarity, simplicity and flexibility.The rewritten IP routing table will be ready for committing in October.
These students will analyse the tcpdumps of five major Swiss newspaper websites which give a representative overview of the user structure in Switzerland. The nice thing about Switzerland is that is has a very good mix of Modem/ISDN, leased line, Cable, ADSL and 3G/GSM/GPRS users. Every Internet access technology is - represented. The goal is to analyse the behaviour of all TCP - sessions to the monitored sites. Parameters to be analysed include + represented. The goal is to analyze the behavior of all TCP + sessions to the monitored sites. Parameters to be analyzed include TCP session RTT, RTT variance, in/outbound BDP, MSS changes, flow - control behaviour, packet loss, packet loss, packet retransmit and + control behavior, packet loss, packet retransmit and timing of HTTP traffic to find optimal TCP parameter caching method.
If you have any other metrics you think is useful please contact me so I can put that into the job description for the Students. The study will be made in September and October.
The current natd is pretty powerful in translating different kinds of traffic but not very powerful in configuration. This project rewrites natd and parts of libalias to give it a configuration set as powerful and expressive as the ones in ipf (ipnat) and pf. In addition it'll use kqueue and will support aliasing to multiple IP addresses.
The rewritten natd will be ready for committing in early September.
IA64 has been progressing slowly. We have access to a prototype 4-way Itaninum2 system from Intel and have managed to get it up and running to the point of being able to access disk and network with SMP enabled. We have a big problem with ACPI2.0 and PCI routing table entries behind pci-pci bridges with no short-term solution in sight. Various WIP items have been committed to CVS, namely more complete support for executing 32bit i386 binaries as well as Marcel Moolenaar's prototype EFI GPT tools.
Max has been busy cleaning up the user interface dark side, and has - come up with a plan to improve the build system (using an automated - Makefile dependency generator); the UI design and the TCL glue magic - (using Swig). A develepment page has been created on usw4, publishing - a lot of information about the current project status, a Changelog, - screenshots, documentation, etc. A new listbox widget has been - implemented, making diskeditor look nicer and more useable. The package - system backend is being inspected and redesigned to conform to a standard - that is itself being re-thought. Indeed, the old sysinstall2.txt text has - been SGML-ized and enhanced and now provides a good (altough rough) overview - of libh package system. This allowed the document to be enhanced with diagrams - of how different procedures work. We are therefore getting closer to a - real pkgAPI specification document. The package management tools have been - sligthly enhanced and should be a bit more useable, and we started commiting - regression test suites in the tree, mostly to test and maintain pkg API - conformance.
- -So work continues on libh. I plan to take a look at the rhtvision port - to see if it would be better to use it for the tvision backend. I'll keep - on working on the package system to make it really trustworthy, while Max - is continuing his great work on the UI subsystem. I hope to make a new libh - alpha release soon. Note that from now on, libh progress will be published - on the development page.
- -Max has been busy cleaning up the user interface dark side, and has + come up with a plan to improve the build system (using an automated + Makefile dependency generator); the UI design and the TCL glue magic + (using Swig). A development page has been created on usw4, publishing + a lot of information about the current project status, a Changelog, + screenshots, documentation, etc. A new listbox widget has been + implemented, making diskeditor look nicer and more usable. The package + system backend is being inspected and redesigned to conform to a standard + that is itself being re-thought. Indeed, the old sysinstall2.txt text has + been SGML-ized and enhanced and now provides a good (although rough) overview + of libh package system. This allowed the document to be enhanced with diagrams + of how different procedures work. We are therefore getting closer to a + real pkgAPI specification document. The package management tools have been + slightly enhanced and should be a bit more usable, and we started committing + regression test suites in the tree, mostly to test and maintain pkg API + conformance.
+ +So work continues on libh. I plan to take a look at the rhtvision port + to see if it would be better to use it for the tvision backend. I'll keep + on working on the package system to make it really trustworthy, while Max + is continuing his great work on the UI subsystem. I hope to make a new libh + alpha release soon. Note that from now on, libh progress will be published + on the development page.
+ +A major power bug was fixed in oldcard. This caused many problems for people using PCI interrupts having their machines hang on boot. This fix has made it into 4.6.1.
Cardbus power is now used on all cardbus bridges that support it. This means that we now support 3.3V cards on all cardbus bridges. Before, we only supported them on some of the bridges because every bridge uses different 3.3V power control when programmed through the ExCA registers. Now that we're going through the CardBus bridge's power control register, 3.3V cards work. In fact, for CardBus bridges, the so called X.XV and Y.YV cards will work in those bridges that support them. However, X.XV and Y.YV haven't been defined yet, and no bridges support them (but the bridge interface define it). Obviously this latter part is untested.
CL-PD6722 support has been augmented slightly. Now it is possible to instruct the driver which type of 3.3V card detection strategy to use. There are three choices: none, do it like the CL-PD6710 does it and do it like the CL-PD6722 does it.
Preliminary support for the CL-PD6729 on a PCI card using PCI interrupts has been committed. However, it fails for at least one of the cards like this the author has.
Client drivers can now ask for the manufacturer and model number of the card without parsing the CIS directly.
Except for fixing bugs and updating pccard.conf entries, no additional work is planned on the OLDCARD system.
A devd daemon, to replace pccardd and usbd, has been designed. A few minor bugs have been fixed in NEWCARD. NEWCARD is now the default in -current. There is an experimental pci/cardbus bus code merge available as a branch which will be merged into current as soon as it is stable.
Status: The ed driver, for non-ne2000 clones, is broken and won't probe. The ata driver won't attach. The sio driver hangs on the first character. The wi driver is known to work well. Cardbus cards are generally known to work well, except for some de based cards, -which unfortuntely includes the popular Xircom cards. Many systems +which unfortunately includes the popular Xircom cards. Many systems fail to work because acpi fails to route interrupts correctly for non-root pci bridges.
Things are going well with the FreeBSD GNOME Project. We have just finished porting the GNOME 2.0 Final development platform and desktop to FreeBSD! We hope to be able to make GNOME 2.0 the default for 5.0-DP2 and 4.7-RELEASE. In the meantime, we're working to port more GNOME 2.0 applications.
In order to allow GNOME 1.4.1 applications to work with GNOME 2.0, we are revamping the GNOME porting infrastructure. GNOME 1.4.1 based ports are being converted to use the new GNOMENG porting structure. The specifics of this new system will be written up in the GNOME porting guide found on the FreeBSD GNOME project homepage.
The BSD Java Porting Team has been making slow but steady progress on a number of fronts in the last few months. Unfortunately most of this has occurred behind the scenes, meaning this is a good opportunity to bring the community up to date.
I'm afraid KAME Project does not work actively with regard to FreeBSD in these two month, since we are too busy with the demonstration of our IPv6 implementation at Networld+Interop 2002 Tokyo. (Thanks to a great effort, the demonstration was quite successful)
We are aware of netinet6-related bug reports regarding socket handling, fine-grain locking, ip6fw etc. Regret to say, we could not answer them right now due to the above situation, however we'll discus these issues internally and determine what to do.
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
Over the past few months the FreeBSD Release Engineering Team oversaw a release process that culminated in the release of FreeBSD 4.6 for the i386 and Alpha architectures on June 15. The RE team is currently working concurrently on FreeBSD 4.6.1 and 5.0 DP2. 4.6.1 is a minor point release with an updated SSH and BIND, fixes for some of the reported ata(4) problems, and assorted security enhancements that will be detailed in the release notes. The release engineering activities for 4.6.1 are taking place on the RELENG_4_6 branch in CVS, while the work on 5.0 DP2 is taking place in Perforce so as not to disturb ongoing -CURRENT development. We are still committed to FreeBSD 5.0 on or around November 15, 2002. For more information about upcoming release schedules, please see our website above. The RE team would like to thank Sentex Communications for providing the release builders with access to a fast i386 build machine. Compaq also donated a couple of fast Alpha build machines to the project.
The main goal of this project is to modify the IPSEC protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPSEC protocols.
Basic functionality is operational for IPv4 protocols. IPv6 support is coded but not yet tested. Hardware assisted cryptographic operations are working with good performance improvements. Operation with software-based cryptographic calculations appears to be at least as good as the existing implementation. Numerous opportunities for performance improvements have been identified.
This work is currently being done in the -stable tree. A port to the -current tree is about to start.
Since the last status report, the following utilities have been brought up to conformance (at least to some degree) with POSIX.1-2001, they include: asa(1), cd(1), compress(1), ctags(1), ls(1), newgrp(1), nice(1), od(1), pathchk(1), renice(1), tabs(1), tr(1), uniq(1), wc(1), and who(1). In addition, development is taking place on bringing the BSD SCCS suite up to date with newer standards.
On the API front, printf(9) has been given support for the `j' and 'n' flags, waitpid(2) now supports the WCONTINUED option, and an implementation of fstatvfs() and statvfs() has been committed. An implementation of utmpx is in progress, which has an aim to address some of the major problems with the current utmp. Several headers have been brought up to conformance with POSIX.1-2001, they include: <netinet/in.h>, <pwd.h>, <sys/statvfs.h>, and <sys/wait.h>.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPSEC), and OpenSSL (through the /dev/crypto device).
The software has been available as a patch against the -stable tree for about six months. The core crypto support is tested, including device drivers for the Hifn 7951, and Broadcom 5805, 5820, and 5821 parts. Recent work has concentrated on fixing device driver bugs, fixing support for Hifn 7811 parts, adding support for public key operations, and adding flow-control between the crypto layer and device drivers. Future work includes porting this facility to the -current tree.
The project took a major step at the beginning of July when Milestone-III was committed. Milestone-III allows a simple test program (available at /usr/src/tools/KSE/ksetest/) to run multiple threads, using kernel support. It does not yet allow the ability to allow these threads to run on different CPUs simultaneously. Milestone IV will be to allow this, however Milestone-III should allow Dan to start (with any interested parties) to start prototyping the userland part of the system. Milestone-III is only currently usable on x86, and does not include some of the requirements for full thread-control, suspension etc. that will be required later.
- Before M-IV is started some small tweeking is likely + Before M-IV is started some small tweaking is likely in the central sources on M-III as we discover issues as we try to get the userland jumpstarted. These will have no effect on non-KSE processes, (i.e. all of them :-) and should not be an issue for other developers.
A tex/fig->html guru is needed to help maintain the - KSE web page (not mentionned above as it is broken). + KSE web page (not mentioned above as it is broken).
The SMPng project has continued to make steady progress in the past two months. Jeff Roberson completed the switch over to UMA for the general kernel malloc() and free() pushing down Giant appropriately so that callers of malloc() and free() are no longer required to hold Giant. Alan Cox continues to clean up the locking in the VM system pushing down Giant in several of the VM related system calls. Jeffrey Hsu committed locking for TCP/IP protocol control blocks in the network stack. John Baldwin committed the changes to the p_canfoo() API to use thread credentials for subject threads and added appropriate locking for the targer process credentials. Support for adaptive mutexes on SMP systems as well as the new IA32 PAUSE instruction were also committed in May. The kernel tracing facility KTRACE also received an overhaul such that the majority of its work was pushed out into a worker thread allowing trace points to no longer require Giant. Andrew Reiter has also been pushing down Giant in several system calls.
Bosko continues to work on light-weight interrupt threads for i386. Most of the bugs in the turnstile code have been found and fixed; however, the turnstile and preemption patches have temporarily been put on hold so that more emphasis can be placed on fixing bugs and making -current more stable in preparation for 5.0 release in November. Alan Cox and Andrew Reiter are continuing the work mentioned above. Jeff Roberson is also working on fixing the current vnode locking in VFS. Peter Wemm has also started to tackle TLB issues on SMP in the i386 pmap again as well.
After an outstanding job serving the project as Security Officer for over a year, Kris stepped down in January in order to focus more of his time pursuing his PhD. I offered to attempt to fill the vacant role.
This is the first report by the SO Team. Notable events since the beginning of 2002 follow.
28 FreeBSD Security Advisories have been issued, 16 of which were regarding the base system. Of those sixteen, 8 affected only FreeBSD.
FreeBSD Security Notices were introduced, and four have been issued so far. The Security Notices cover issues that are not regarded as critical enough to warrant a Security Advisory. So far only Ports Collection issues (i.e. vulnerabilities in optional 3rd party packages) have been reported in Security Notices. The first four Security Notices covered 53 individual issues.
Issues reported to the SO team are now being tracked using a RequestTracker ticket database.
The SO team has undergone membership changes, as well as some changes in internal organization. The membership and organization has also been made publicly visible on the FreeBSD Security Officer web page.
For 4.6-RELEASE, we announced the package ja-man-doc-4.6.tgz which is in sync with 4.6-RELEASE base system manual pages except for perl5 pages (jpman project do not maintain them). Continuing section 3 updating has 88% finished.
Progression is slow, but the effort is maintained. Most of fb over KGI has been written in parallel with a KGI display driver based on fb. DDC/DDC2 is being discussed for Plug & Play monitor support. KGI aims at providing - a generic OS independant interface which would take advantage of FreeBSD I2C (iic(4)) + a generic OS independent interface which would take advantage of FreeBSD I2C (iic(4)) infrastructure.
UFS2 is an extension to the well-known UFS filesystem which using a new inode format adds support for "64bit everywhere" and later for extended attribute support, in addition to the current UFS features: soft-updates and snapshots.
The basic UFS2 code has been committed and work on the extended attribute interface and vnode operations will continue.
The GEOM code has gotten so far that it beats our current code - in some areas while stil lacking in others. The goal is for + in some areas while still lacking in others. The goal is for GEOM to be the default in 5.0-RELEASE.
Currently work on a cryptographic module which should be able to protect a diskpartition from practically any sort of attack is progressing.
The port of openoffice 1.0 has been finished. Most showstopper issues with rtld, libc and our toolchain have been fixed. There is one remaining deadlock in the web-browser code of OO.org. If anybody like to help us with fixing this bug (may be another libc_r bug as it looks like) - just mail me ! Unfortunalty gcc2 support got broken again with the import - of gcc2.95.4 in STABLE. Exceptions support seems to be broken again, we get + just mail me! Unfortunately gcc2 support got broken again with the import + of gcc2.95.4 in STABLE. Exceptions support seems to be broken again; we get internal compiler errors with c++ exceptions code. You'll have to use gcc31 again.
Since our package cluster is outdated and can not build OO.org packages anytime soon, I did my own little package cluster and can now offer packages for 4.6R for 16 different languages. They can be found on the project homepage.
Porting of OpenOffice1.0.1 is on it's way. A beta port and a package have been made available on the project homepage.
The lightweight interrupt scheduling code makes scheduling an interrupt on i386 without having to grab the sched_lock possible, and also avoids a full-blown context switch.
Currently, the code in the p4 branch works, although needs a little bit of cleanup and, most importantly, requires a merge to post-KSE III. Now that stuff seems to have stabilized a bit, I'm waiting to get a little time (and nerve) to do the merge. Also, looking forward for some KSE interface that will allow for "KSE borrowing," which would make this cleaner with regards to KSE and lightweight interrupts. This is a 5.0 feature.
A lot of remaining PR's and Bugs have been closed. All relevant rpc - concerning patches have been comitted. Thank goes to Alfred and Ian Dowese. + concerning patches have been committed. Thanks go to Alfred and Ian Dowese.
Jean-Luc Richier <Jean-Luc.Richier@imag.fr> has made a patch available which adds IPv6 support to all remaining rpc servers. See ftp://ftp.imag.fr/pub/ipv6/NFS/NFS_IPV6_FreeBSD5.0.gz and ftp://ftp.imag.fr/pub/ipv6/NFS/0README_NFS_IPV6_FreeBSD5.0 We will check his code and add it to CURRENT ASAP.
A first commit part from TIRPC99 has been done. I'm working now on porting the remaining parts so when FreeBSD 5.0 gets released, it will be TIRPC99 based. This will happen together with the NetBSD project, as they use the same codebase as we do.
mb_alloc is getting some updates and a couple of optimisations. +
mb_alloc is getting some updates and a couple of optimizations. A new allocator interface routine should already be committed by the time this report is "published:" m_getcl() allocates an mbuf and a cluster in one shot. This is the result of months (literally) of requests from Alfred and, recently, Luigi - who, coincidentally, is the author of the same [upcoming] routine in -STABLE.
Other than that, mb_alloc is being shown how to perform multi-mbuf or cluster allocations without dropping the cache lock in between (m_getcl() and m_getm() will use this). Finally, work is - being done to optimise ext_buf ref. count allocations and to provide + being done to optimize ext_buf ref. count allocations and to provide support for jumbo (> 9K) clusters.
We are making excellent progress. There is a fully functioning implementation imported to -current now. We need as many people as possible to rc_ng equal to YES in /etc/rc.conf.
The next step is to set the default to YES, which we plan to do before DP 2.
In summer 2002 the native FreeBSD firewall has been completely rewritten in a form that uses BPF-like instructions to perform packet matching in a more effective way. The external user interface is completely backward compatible, though you can make use of some newer match patterns (e.g. to handle sparse sets of IP addresses) which can dramatically simplify the writing of ruleset (and speed up their processing). The new firewall, called ipfw2, is much faster and easier to extend than the old one. It has been already included in FreeBSD-CURRENT, and patches for FreeBSD-STABLE are available from the author.
I spent busy days in last two months, many new topics are emerged from the project. We now support FreeBSD/alpha 5-current distribution by cross-compiling on the x86 PC. Anonymous ftp area is now exported to the yet another web server. Our release branch snapshots are relocated to daemon.jp.FreeBSD.org because of our CPU/network bandwidth problem.
I'm seriously considering to solve the lack of CPU and network resources for the project's future evolution. Maybe the bandwidth - problem can be resolved (several bandwidth offering are received!), - but there is no answer about CPU problem (I have a plan to upgrade - our PCs from P3-500Mhz to P4 or something better than previous). - If you have interested to donate PCs to the project, please email me + problem can be resolved (several bandwidth offers have been received!), + but there is no answer about CPU problems (I have a plan to upgrade + our PCs from P3-500MHz to P4 or better). + If you have interested in donating PCs to the project, please email me for more detail.
Regression tests for many bugs fixed in text manipulation utilities have been added, as well as tests for various non-standard versions of functionality that FreeBSD users should expect. A library of m4 macros for creating the tests themselves has been added.
The final version of SCCS distributed by CSRG has been integrated into the projects CVS repository, and worked on extensively to the point where essential functionality works on FreeBSD (and other operating systems). Some standards-related functionality has been implemented
The zero copy sockets code was committed to FreeBSD-current on June 25th, 2002. I'm not planning on doing any more patches, although I will leave the web page up as it contains useful information.
Many thanks to the folks who have tested and reviewed the code over the years.
Jennifer Yang's patch was committed June 10 for the BSD Summit. After a few bugs which were reported initially and fixed that same week, networking in -current has been stable, including the parts that were not locked up, like IPv6. Work is on-going to lock up the rest of the stack.
Not much to report. Another engineering snapshot is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20020709.tar.gz. If anyone has Bluetooth hardware and spare time please join in and help me with testing.
This snapshot includes basic support for USB devices and manual pages. The HCI layer now has support for multiple control hooks. All HCI transport drivers (H4, BT3C and UBT) has been changed to provide consistent interface to the rest of the world. Some userspace utilities have been changed as well.
Still no support for RFCOMM (Serial port emulation over Bluetooth link) and SDP (Service Discovery Protocol). Several design flaws have been discovered and it might take some time to resolve these issues.
The TrustedBSD Project has been busy in May and June, developing new features, presenting on the technology at the FreeBSD Developer Summit, and improving the readiness of the MAC branch for integration into the main FreeBSD tree. The migration to dynamic labeling in the TrustedBSD MAC framework is complete, with all policies now making use of dynamic labels in the kernel. This permits policies to associate arbitrary additional security data with a variety of kernel objects at run-time. Implement mac_test, a sanity checking module. Pass labels as well as objects to each policy entry point to reduce knowledge of label storage in the policies. Implement mac_partition, a simple jail-like policy. Adapt the MAC framework for process locking.
Improve support for sockets: provide a peerlabel maintained for stream sockets (unix domain, tcp), entry points for accept, bind, connect, listen. Improve support for IPv4 and IPv6 by labeling IP fragment reassembly queues, and providing entry points to instrument fragment matching, update, reassembly, etc. Locally disable KAME if_loop mbuf contiguity hack because it drops labels on mbufs: we need to make sure the label is propagated. Label pipes and provide access control for them. Improve vnode labeling: now handle labeling for devfs, pseudofs, procfs. Fix interactions between MAC and ACLs relating to the new VAPPEND flag.
SELinux policy tools now ported to SEBSD. SEBSD now labels subjects and file system objects. Provide ugidfw, a tool for managing rules for the mac_bsdextended policy.
Massive diff reduction. KSEIII merged. Main tree integration will begin shortly.
Updated prototype code may be retrieved from the TrustedBSD CVS trees on cvsup10.FreeBSD.org.
At long last, FreeBSD 5.0 is here. Along with putting the final polish on the tree, FreeBSD developers somehow found the time to work on other things too. IA64 took some major steps towards working on the Itanium2 platform, an effort was started to convert all drivers to use busdma and ban vtophys(), hardware crypto support and DEVD hit the tree, NewReno was fixed and effort began on locking down the network layer of the kernel. Also high performance, modular scheduler started taking shape and will be a welcome addition to the kernel soon.
Looking forward, the focus will be on stabilizing and improving the performance of 5.0. The RELENG_5 (aka 5-STABLE) branch will be created once we've reached our goals in this area, so hopefully we will get there quickly. Meanwhile, preparations for the next release from the 4.x series, 4.8, will begin soon. Of course, the best way to get 5.x to stabilize os to install and run it!
Thanks,
Scott Long, Robert Watson
I'm very pleased to announce that all kernel modules and few userland tools made it to the FreeBSD source tree. Many thanks to Julian Elischer.
Unfortunately no big changes since the last report. Some minor problems have been discovered and patches are available on request. I will prepare all the patches and submit them to Julian for review.
OBEX server and client (based on OpenOBEX library) is almost complete. I'm currently doing interoperability testing. If anyone has hardware and time please contact me. The HCI security daemon has been implemented and tested with Sony Ericsson T68i cell phone and Windows stack. It is now possible to setup secure Bluetooth connections.
A few people have complained about RFCOMM daemon. These individuals want to use GPRS and Bluetooth enabled cell phone to access Internet. If you have this problem please contact me for possible workaround. My next goal is to get robust RFCOMM implementation to address all these issues.
Largely bug-fixing and userland application tweaks; new interfaces were added to manipulate ACLs on extended attributes; bugs were fixed in ls relating to ACL flagging. Patches to teach cp, mv, gzip, bzip, and other apps about ACL preservation are in testing and review. tunefs flags were added to ease configuration of ACLs, especially on UFS2 file systems.
Possible changes to make use of Linux/Solaris umask semantics are under consideration: right now we implement verbatim POSIX.1e/IRIX merging of the umask, ACL mask, and requested creation mode during file, device, fifo, and directory creation. Solaris and the most recent Linux patches ignore the umask in the context of a default ACL; this requires some rearrangement of umask handling in our VFS, although the results would be quite useful. We're exploring how to do this in a low impact way.
Framework changes:
Instrument KLD system calls (module and kld load, unload, stat) Instrument NFSd system call. Instrument swapoff(2). Instrument per-architecture privileged parts of sysarch(). Make use of condition variables to allow callers to wait for the framework to "unbusy" when loading/unloading policies, rather than returning EBUSY. Store mount pointer in devfs_mount structure for use by policies. Improve handling of labels in loopback interface "re-align" packet copy case. Provide full paths on devfs object creations to help policies label them properly (not merged). Experimentation with moving MAC labels into m_tags (not merged). NFS server now uses real ucreds, not hacked up ucreds, meaning we can start laying the groundwork for enforcement on NFS operations. (not merged)
Policy changes
LOMAC: mac_lomac replaces lomac (LOMAC now uses the MAC Framework), SEBSD: Improved support for devfs labeling based on SELinux genfs. Handling of hard link checks. Support export of process transition information for login and others using sysctl. Login now prompts for roles. Allow policy reload. TTY labeling. Locking adaptation from Linux. Many, many policy adaptations and fixes. We can now boot in enforcing mode! mac_bsdextended: fix a bug in which VAPPEND wasn't mapped to VWRITE, so opens with the O_APPEND bug failed improperly.
Userland changes
setfmac(8) now supports a setfsmac(8) execution mode, which accepts initial labeling specification files. Supports an SELinux compatibility mode so it can accept SELinux label specfiles using the SEBSD module. sendmail(8) now sets user labels as part of the context switch for mail delivery.
Documentation changes
Man page updates for MAC command line tools, modules, admin hints, etc. Updates to the FreeBSD Developer's Handbook chapter on MAC policies and entry points. MAC section in FreeBSD Handbook.
This project has been coming along pretty well. The amd(4) and xl(4) drivers have now been converted to use the busdma API, sparc64 got the bus_dmamap_load_mbuf() and bus_dmamap_load_uio() functions, and the gem(4) and hme(4) drivers have been updated to use bus_dmamap_load_mbuf() instead of bus_dmamap_load().
A lot more still needs to be done, as shown on the project's page. A fair number of conversions are on their way though, and we can expect a fair number of drivers to be converted soon, thanks to all the developers who are working on this project.
The POSIX Utility Conformance in FreeBSD list (link above) has been updated to reflect current reality. Not much work remains to complete base utility conformance.
On the API front, grantpt(), posix_openpt(), unlockpt(), wordexp(), and wordfree() were implemented. The header <wordexp.h> was added.
There are currently about 40 unassigned tasks on our project's status board ranging from documentation, utilities, to kernel hacking. We would encourage any developers looking for something to work on to check out the status board and see if anything interests them.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPsec), and OpenSSL (through the /dev/crypto device).
This work will be part of the 5.0 release and has been committed to the -stable source tree for inclusion in the 4.8 release.
Recent work has focused on improving performance. System statistics are now maintained and an optional profiling facility was added for analyzing performance. Using this facility the overhead for using the crypto API has been significantly reduced.
The ubsec (Broadcom) driver was changed to significantly improve performance under load. In addition several memory leaks were fixed in the driver and the public key support was enabled for use.
Upcoming work will focus on load-balancing requests across multiple crypto devices and integrating OpenSSL 0.9.7 which will automatically enable application use of crypto hardware.
Devd has been integrated into FreeBSD 5.0-RELEASE. The integrated code supports a range of configuration options. The config files are fully parsed now and their actions are performed.
-Future work in this area are likely to be limited to imporving +
Future work in this area is likely to be limited to improving the devctl interface. /dev/devctl likely will be a cloneable device in future versions. Individual device control via devctl is also planned.
The Donations project expedited several dozen donations during 2002, and was able to place most of what was offered. We still are in dire need of SMP and Sparc systems. You can see information on our needs and donations that have been handled by the team on the donations web page.
We are relying increasingly upon the developer wantlist to place items offered to the Project, and using the commit statistics to help place items. As such, active committers who ask for what they want beforehand have a decent chance of getting it. Less active committers, and committers who do not ask for what they want, will be lower in our priorities but will not be excluded.
We are in the process of streamlining the tax deduction process for donations, and hope to have news on that shortly. We are also always working to accelerate and reduce our internal processes, to get the most equipment in the hands of the most people as quickly as possible.
I especially want to thank David O'Brien and Tom Rhodes for stepping up and making the team far more successful. Also, the FreeBSD Foundation has been quite helpful in handling tax-deductible contributions.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
This work will be part of the 5.0 release. Performance has been improved due to work on the crypto subsystem.
The goal of the project is to use a small amount of space in the FFS superblock to store a volume label of the user's choice. A GEOM module will then expose the volume labels into a namespace in devfs. The idea is to make it easier to manage filesystems across disk swaps and movement from system to system.
At this point, everything pretty much works. I've submitted parts of the patch to respective subsystem maintainers for review. There are some issues with namespace collision that I haven't addressed yet, but the basic functionality is there
Most of the articles are translated too. Marc is still translating the handbook, 60% is currently translated. Stéphane has began the integration of our French localization web site in the US CVS Tree. Sébastien is still maintaining the Release Notes.
We launched a new site, www.FreeBSD-fr.info, consisting in a French - Dameon News like site. Netasq have donated our new server; we will + Daemon News like site. Netasq have donated our new server; we will install it in a new hosting provider in the few next weeks. One of the - big job now, project now, is the translation of the FAQ, and the big - project will be the manual pages
+ big job now is the translation of the FAQ, and the big + project will be the manual pages.Since the ports tree has been frozen for most of this reporting period,
there have not been too many GNOME updates going into the official CVS
tree. However, development has not stopped. GNOME 2.2 is nearing
completion, and quite a few FreeBSD users have stepped up to test the
GNOME 2.1 port sources from the
The upcoming FreeBSD 5.0-RELEASE will be the first release to have the GNOME 2.0 desktop as the default GNOME desktop choice. During the previously mentioned ports freeze, all the GNOME 2 ports were fixed up so that they build and package on both i386 and Alpha platforms. Alas, the one port that will not make the cut for Alpha is Mozilla. There are still problems with the xpcom code, but work is ongoing to get a working Alpha port.
Finally, the FreeBSD Mono (an OpenSource C# runtime) port has also received some new life. Mono has been updated to 0.17 (the latest released version), and Juli Mallett has ported gtk-sharp (GTK+ bindings for C#).
The ia64 port is up and running on the new Itanium2 based hp machines thanks to a lot of hard work by Marcel Moolenaar. So far we are running on the hp rx2600 as these were the machines graciously donated by Hewlett-Packard and Intel. We had a prototype Intel Tiger4 system for a while, but we had to return the machine and we do not know if it currently runs. Most of the changes necessary to run these are sitting in the perforce tree and are not in the -current or RELENG_5 cvs tree. As a result, the cvs derived builds (-current and the 5.0-RC series - and presumably 5.0-RELEASE) are only useable on obsolete Itanium1 + and presumably 5.0-RELEASE) are only usable on obsolete Itanium1 systems.
Lots of other stability and functionality fixes have been made over the last few months, including initial libc_r support. The OS appears to be stable enough for sustained workloads - it is building packages now, for example. We still do not have gdb support, even for reading core files.
We have been updating our Japanese translated manual pages to RELENG_5 based. All existing entries have been updated, but 15 exceptions are not, most of which require massive update. We will also need to add translations which did not exist on RELENG_4.
KGI (Kernel Graphic Interface) is a kernel infrastructure providing user applications with means to access hardware graphic resources (dma, - irqs, mmio). KGI is already available under Linux as a seperate + irqs, mmio). KGI is already available under Linux as a separate standalone project. The KGI/FreeBSD project aims at integrating KGI in the FreeBSD kernel.
KGI/FreeBSD has been recently donated 2 PCI graphic cards (Matrox Millenium II and a coming Mach64) and other have been proposed. Please see the FreeBSD web pages for details. Thanks to donation@ for organizing and promoting donations. Thanks to the donators for their contribution to KGI/FreeBSD.
KGI/FreeBSD progressed fine the last months. Most of the VM issues for mapping HW resources in user space have been addressed and a first attempt of coding was made. This prototyping raised some API compatibility problems with the current Linux implementation and was discussed heavily on the kgi devel lists. Ask if you're interested in such issues, I'll be pleased to share them.
Most of coding is now done. Let's start debugging!
Work is ongoing to continue to lock up the network stack. Recently, the focus has been on the IP stack. The plan there involves a series of inter-related pieces to lock up the ifaddr ref count, the inet list, the ifaddr uses, the ARP code, the routing tree, and the routing entries. We are over 3/5 of the way done down this path.
In addition to TCP and UDP, the other networking protocols such as raw IP, IPv6, AppleTalk, and XNS need to be locked up. Around 1/4 these remaining protocols have been locked and - will be commited after the IP stack is locked.
+ will be committed after the IP stack is locked.The protocol independent socket layer needs to be locked and operating correctly with the protocol dependent locks. This part is mostly done save for much needed testing and code cleanup.
Finally, a pass will be need to be made to lock up the devices drivers and various statistics counters.
This effort fixes some outstanding problems in our TCP stack with regard to congestion control. The first item is to fix our NewReno implementation. Following that, the next urgent correction is to fix a problem involving window updates and dupack counts. When that stabilizes, we will then change the recovery code to make use of SACK information. Eventually, this project will update the BSD stack to add Limited Transmit and other new internet standards and standards-track improvements.
The 3 FreeBSD package clusters (i386, alpha, sparc64) have been unified to run from the same master machine, instead of using 3 separate masters. This has freed up some machine resources to use as additional client machine, as well as simplifying administrative overheads. Build logs for all 3 architectures can now be found on the http://bento.freebsd.org webpage. The sparc64 package cluster now has 3 build machines (an u5 and two u10s), and an ia64 cluster is about to be created.
Package builds now keep track of how many sequential times a port has failed to build (html summaries are available on the bento website). This allows tracking of ports which have suddenly become broken (e.g. due to a bad upgrade, or due to changes in the FreeBSD source tree), and in the future will be used to send out notifications to port maintainers when their port fails to build 5 times in a row. This feature is currently experimental, and further code changes will be needed to stabilize it.
The goal of this project is to improve the wireless networking support in the system. By the time of this report the 802.11 link layer code should be committed. A version of the wi driver that uses this code should be committed shortly. Conversion of other drivers is planned as are drivers for new devices.
Support for 802.1x/EAP is the next planned milestone (both as a supplicant and authenticator).
November and December were especially busy for the release egineering +
November and December were especially busy for the release engineering team. Scott Long joined the team to help with secretary and communications tasks while Brian Somers bowed out to focus on other projects.
FreeBSD 5.0-DP2 was released in November after much delay and anticipation, and marked the final milestone needed for 5.0 to become a reality. Shortly after that, we imposed a code freeze on the HEAD branch of CVS and released 5.0-RC1. Creation of the RELENG_5_0 branch came next, followed by the release of 5.0-RC2 from this branch. At this point, enough critical problems still existed that we scheduled an RC3 release for the new year, and pushed the final 5.0-RELEASE date to mid-January. By the time this is published, FreeBSD 5.0-RELEASE should be a reality.
For the time being, there will not be a RELENG_5 (aka 5-STABLE) branch. FreeBSD 4.x releases will continue, with 4.8 being scheduled for March 2003. Release in the 4.x series will be lead by Murray Stokely, and releases in the 5.x series will be lead by Scott Long. Once HEAD has reached acceptable performance and stability goals, the RELENG_5 branch will be created and HEAD will move towards 6.0 development. We hope to reach this with the 5.1 release this spring.
A new scheduler will be available as an optional component along side the current scheduler in the 5.1 release. It has been designed to work well with KSE and SMP. Some ideas have been borrowed from solaris and linux along with many novel approaches. It has O(1) performance with regard to the number of processes in the system. It also has cpu affinity which should provide a speed boost for many applications.
The scheduler has a few loose ends and lots of tuning before it is production quality although it is quite stable. Please see the post to arch and subsequent discussion for more details.
This months report covers activity during the second half of October, and the month of November. During these months, substantial work was performed to improve system performance and stability, in particular addressing concerns regarding regressions in network performance for the TCP protocol, and via the introduction of polled network device driver support. Work continues on long-term architectural projects for 5.0, including KSEs, NEWCARD, and TrustedBSD, as well as the cleaning up of long-standing problems in FreeBSD, such as PAM integration. Administrative changes are also documented, including work to redefine and formalize the release engineering process, and the approval of a new portmgr group which will administer the ports collection.
FreeBSD users and developers are strongly encouraged to attend the USENIX BSD Conference in February of next year; it is expected that this will be a useful forum both for learning about FreeBSD and on-going work, as well as providing an opportunity for developers to work more closely and act as a vehicle for discussion and round-the-clock hacking. More information is available at the USENIX web site.
Robert Watson
A number of serious TCP bugs effecting throughput snuck into the system over the last few releases and have finally been fixed. TCP performance should be greatly improved for a number of cases, including TCP/NFS.
The wx driver is desupported and removed from -current. No further support for wx in -stable is planned. Newer and better drivers are now in the tree.
Ongoing bug fixes. Work is underway, to be integrated shortly, that makes the cross platform endian support easier and will prepare the FreeBSD version for eventual sparc64 and PowerPC usage.
Currently, we are exploring a variety of strategies to learn about the implementation and performance issues in order to have a solid design. One of our main goals will be to use a standardized interface to the system, whether it be POSIX.1e, or another of the other standards, because as they say "Standards are great because you have so many to choose from." Hopefully within the next month or so, we will populate the perforce TrustedBSD tree with an agreed upon framework that is ready for serious final work.
On the code side, a number of libpam bugs have been fixed; a new PAM module, pam_self(8) , has been written; and preparations have been made for - transitioning from + the transition from /etc/pam.conf to /etc/pam.d .
On the documentation side, new manual pages have been written for pam_ssh(8) , pam_get_item(3) and pam_set_item(3) , and work has started on a longer article about PAM which is expected to be finished by the end of the year.
A lot of work still remains to be done to integrate PAM more tightly with the FreeBSD base system—particularly the passwd(1) , chpass(1) etc. utilities—and ports collection.
Presently re-style(9)ing mbuf code with the help of Bruce (bde). The next larger step is approaching: to better performance, as initially planned, not have reference counters for clusters allocated separately via malloc(9). Rather, use some of the [unused] space at the end of each cluster as a counter; since this space is totally unused and since ref. counter <--> mbuf cluster is a one-to-one relationship, this is most convenient.
Release engineering activities for FreeBSD 4.5 have begun. An overview of the entire process has been added to the FreeBSD web site, along with a specific schedule for 4.5. The code freeze is scheduled to start on December 20. The team responsible for responding to MFC requests sent to re@FreeBSD.org for this release is: Murray Stokely, Robert Watson, and John Baldwin. Some of our many goals for this release include closing more installation-related problem reports, being more conservative with our approval of changes during the code freeze, and continuing to document the entire process. For suggestions or questions about FreeBSD 4.5 release activities, please subscribe to the public freebsd-qa@FreeBSD.org mailing list.
Work is (slowly) progressing on converting the web site to use pages marked up in a simple XML schema, and then generating HTML and other output formats using XSLT style sheets. The work so far can be tested by doing "cvs checkout -r XML_XSL_XP www" and then "cd www/en; make index.html". Take a look at index.page in the same directory to see the source XML. The CVS logs for index.page contain detailed instructions explaining how index.page was generated from its earlier form.
The FreeBSD in Bulgarian project aims to bring a more comfortable working environment to Bulgarian users of the FreeBSD OS. This includes, but is not limited to, font, keymap and locale support, translation of the FreeBSD documentation into Bulgarian, local user groups and various forms of on-line help channels and discussion forums to help Bulgarians adopt and use FreeBSD.
Bulgarian locale support has been committed to FreeBSD 5.0-CURRENT (and later merged into 4.x-STABLE on December 10th). A local CVS repository for the translation of the FreeBSD documentation into Bulgarian has been created.
There is now some code ready for the new mount API, which has to be reviewed and tested. If it is adopted, we will probably start converting all the filesystems, as well as other code in the kernel, to make them use it. If you want to play with it, the patch is available at the above URL.
Support for VLAN cloning has been merged from current and will - ship with 4.5-RELEASE. Additionaly, new rc.conf support for + ship with 4.5-RELEASE. Additionally, new rc.conf support for cloning interfaces at boot has been MFD'd. Work is ongoing to MFC stf and faith cloning as well as adding cloning for ppp devices and enhancing VLAN modularity.
This work uses a mixed interrupt-polling architecture to handle network device drivers, giving the system substantial improvements in terms of stability and robustness to overloads, as well as the ability to control the sharing of CPU between network-related kernel processing and other user/kernel tasks. Last not least, you might even see a moderate (up to 20-30%, machine dependent) performance improvement.
I've been working on making the Hardware Notes less i386-centric. This will be especially important for -CURRENT as the ia64 and sparc ports reach maturity; most of this work should be completed in time to be MFC-ed for FreeBSD 4.5-RELEASE. I encourage any interested parties to review the release documentation and send me comments or patches.
The port of the driver is around 90% feature complete. AGP support and "Registry" support via sysctl need to be finished/implemented. The NVIDIA guys are working on a build of the X11 libs and extensions for FreeBSD; once this is done hardware accelerated direct rendering should work. The previous version this driver is no longer available. I'm planning on making a snapshot of my code once I chase out a few more bugs.
Please note that development is taking place under -CURRENT right now; a port to -STABLE will be available at some later time.
jp.FreeBSD.org daily SNAPSHOTs project is yet another snapshots server that provides latest 4-stable and 5-current distribution. You also find installable ISO image, live filesystem, HTMLed source code with search engine, and more; please check project webpage for more details.
Modest gains have been made on the UDF filesystem since the last report. Reading of files from DVD-ROM now works (and is fast, according to some reports), and there is preliminary support for reading from CD-RW media. The CD-RW support has only been tested against CD's created with Adaptec/ Roxio DirectCD, and much, much more testing is needed. Once this support is solid, I plan to check it into the tree and start work on making the filesystem writable.
Not much to report. A number of minor bugs in OLDCARD have been corrected. A larger number of machines now work. Additional work on ToPIC support has been committed, but continued lack of a suitable ToPIC machine has left the author unable to do much work. A few stubborn machines still need to be supported (the author has an example of one such machine, so there is hope for it being fixed. Some pci related issues remain for both OLDCARD and NEWCARD.
NEWCARD work is ramping up, while OLDCARD work is ramping down. A number of things remain to be done for NEWCARD, including suspend/ resume support, generic device arrival/removal daemon and hopefully automatic loading of drivers. A number of current pccard drivers still need to be converted to NEWBUS. Several Chipset issues remain, as does the merging of isa pccard bridge code with the pccbb code.
This project is now finally underway, thanks to DARPA and NAI getting a sponsorship lined up. The infrastructure code and data structures are currently taking form inside a userland simulation harness.
Targeting 4.5-RELEASE, we continued to revising doc/ja_JP.eucJP/man/man[1256789] to catch up with RELENG_4. Section 3 updating has 45% finished.
A FreeBSD -CURRENT snapshot with LOMAC is currently being prepared, with aid of Perforce on the "green_lomac" branch. Very soon there should be a working demonstration installation CD of FreeBSD with LOMAC, including the ability to enable LOMAC in rc.conf with sysinstall, being a legitimate "out-of-the-box" FreeBSD experience. Actual release build is pending debugging issues with program start-up (especially xdm).
Work is underways to support failing mirror disks better and handle hotswapping in a new replacement disk and have it rebuild automagically.
Support for the Promise TX4 is now working in my lab, seems they did the PCI-PCI bridging in the not so obvious way.
Plans are in the works to backport the -current ATA driver to -stable with hotswap and the works. Now that -current is delayed I'm working on ways to give me time to get this done, since I've had lots of requests lately and we really can't let down our customers :).
SMART support is being worked on, but no timelines yet.
Although not strictly ATA, Promise has equipped me with a couple SuperTrak sx6000 RAID controllers, they take 6 ATA disks and does RAID0-5 in hardware. I have done a driver (its an I2O - device) for both -current and -stable and it works butifully with + device) for both -current and -stable and it works beautifully with hotswap the works. It will enter the tree when it is more mature, and I have an agreement with Promise on how we handle userland control util etc. BTW it seems it can also be used as a normal 6 channel PCI ATA controller, a bit on the expensive side maybe...
Extending camcontrol's page definition file format to include both modepage and logpage definitions; adding support to camcontrol to query and reset log page parameters. Consideration is being made to possibly include support for diagnostic and vital product data pages, but that is outside the current project scope. New page definition file format includes capability to conditionally include page definitions based on SCSI INQUIRY results allowing vendor-specific pages to be described also. Approximately 80% complete.
Work on the FreeBSD C99 & POSIX Conformance Project is progressing nicely. Since the last status report, two new headers have been added [<stdint.h> and <inttypes.h>], several new functions implemented [atoll(3), imaxabs(3), imaxdiv(3), llabs(3), lldiv(3), strerror_r(3), strtoimax(3), and strtoumax(3)], and changes to assert(3) and printf(3) were made to support C99. More printf(3) changes are in the works to support the remaining C99 and POSIX requirements. Additionally, research was done into our POSIX Utility conformance and a list of tasks was derived from that research.
Several other interesting events occurred during November and the beginning of December. The project mailing list was moved to the FreeBSD.org domain, and is now available at standards@FreeBSD.org. On December 6, 2001, the IEEE Standards Board approved the Austin Group Specification as IEEE Std 1003.1-2001, thus making the work we're doing ever more important.
This group is for discussion about the startup scripts in FreeBSD, primarily the scripts in /etc/rc*. Primary focus will be on improvements and importation of NetBSD's excellent work on this topic.
<-- from Gordon Tetlow's ranting -->Due to personal commitments by the folks working on this project we have been unable to spend much time porting the rc.d infrastructure into the FreeBSD boot framework.
Currently, the system will boot (with a little fudging) just before network utilization. There are patches floating around for this (see the -arch list from September).
I have been working behind the scenes on design rather than programming for this last month. I have been working however in the p4 tree to make the system run with the thread structure NOT - a part of the proc structure (a prerequisite fo threading)
+ a part of the proc structure (a prerequisite for threading)After a discussion with the Core Team about our status regarding the ports collection, we heard from them that they'd decided to recognize us as the final authority for approving ports committers. We've spent the last few weeks working on our ports build cluster (see the link) and trying to find ways to improve it for the ports development community. We've also handled a few minor issues in the ports collection.
The TrustedBSD Project continued focussing development efforts +
The TrustedBSD Project continued focusing development efforts on fine-grained Capabilities and Mandatory Access Control this month. Kernel support for capabilities is essentially complete, and efforts are underway to adapt userland applications to use Capabilities. The login process has been updated to allow users to run with additional privilege based on /etc/capabilities. The MAC implementation work has also been active, with improved support for the labeling of IPC objects, including better integration into the network stack. Both development trees have been updated to work with recent KSE-related developments, as well as exist more happily in a fine-grained SMP kernel. Initial - audit-related work appears in a seperate entry.
+ audit-related work appears in a separate entry.Development of TrustedBSD source code was moved to the FreeBSD Perforce repository, permitting better source code management. As such, the TrustedBSD development trees will now be available via cvsup.
October ended up being a bit busier than November for SMPng. During October, Peter Wemm finally finished the ambitious task of unwinding all the macros in NFS and splitting it up into two halves: client and server. Andrew Reiter also submitted some code to add locks to taskqueues, and the folks working on the TTY subsystem designed the locking strategy they will be using. Per-thread ucred references were also added for user traps and syscalls. Once the necessary locking on the process ucred references is committed, this will allow kernel code to access the credentials of the current thread without needing locks while also ensuring that a thread has constant credentials for the lifetime of a syscall. November only saw a few small bug fixes unfortunately, but December is already shaping up to be a very active month, so next month's report should be a bit more interesting.
In non-coding news, the website for the SMPng project has moved from its old location to the new location above. Also, I have completed a paper I am presenting for BSDCon regarding the SMPng project. The paper will be available in the conference proceedings and will be available online after the conference as well.
Another busy pair of months at the FreeBSD Project have brought substantial maturity and feature completeness to the fledgeling 5.0-CURRENT branch. And just in time too, because by the time you read the next status report, we hope that you'll have FreeBSD 5.0 running on your desktop! Over the past two months, we've seen an upgrade of sparc64 to Tier 1 (Fully Supported) status, integration of a high quality storage encryption module, the commit of hardware-accelerated IPsec support, the addition of a general-purpose "Device Daemon" to process hardware attach/detach events to replace earlier single-purpose and bus-specific daemons, the commit of RAIDFrame, and the improved maturity of the TrustedBSD work. We've also seen another successful release of the 4.x branch, 4.7-RELEASE, which will continue to be the production supported platform as 5.X is brought in for landing.
-Over the next two months, the FreeBSD Project will be focussed +
Over the next two months, the FreeBSD Project will be focused almost entirely on making 5.0 a success: improving system stability and performance, as well as increasing the pool of applications that build and run on 5.0. The Release Engineering team will have announced the 5.0 code freeze, and released DP2 by the time you read this. Following DP2 will be a series of Release Candidates (RC's), and then the release itself. If you're interested in getting involved in the testing process, please lend a hand -- a spare box and a copy of the DP and RC ISOs burnt onto CD will make a difference. The normal caveats associated with pre-release versions of operating systems apply! You may also be interested in reading the Early Adopter's guide produced by the Release Engineering team to help determine when a transition from the 4.x branch to the 5.x branch will be appropriate for you and your organization.
Thanks,
Robert Watson, Scott Long
I'm very pleased to announce that another engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20021104.tar.gz
This release features minor bug fixes and new OpenOBEX library port. The snapshot includes support for H4 UART and H2 USB transport layers, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices. Also there are several man pages.
Service Discovery Protocol (SDP) port has been updated to version 0.8. (ported from BlueZ-sdp-0.8). Most of the RFCOMM issues have been resolved and now rfcommd works with Windows (3COM, Xircom and Widcomm) and Linux stacks.
New supported USB device - EPoX BT-DG02 dongle. Also I have received successful report about Mitsumi USB dongle and C413S Bluetooth enabled cell phone (L2CAP and SDP works, waiting on RFCOMM report).
I'm currently working on OBEX server (Push and File Transfer profiles) which will be based on OpenOBEX library (included in the snapshot).
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
October 10, 2002 marked the one year anniversary of our project. During that time we have made significant advances in FreeBSD's standards conformance. FreeBSD 5.0-RELEASE will be the showcase for most of our hard work. We hope that our tireless effort has had a positive effect on FreeBSD and software vendors that maintain or are considering porting their software to FreeBSD.
On the API front, _Exit(3) (an alias for _exit(2)) was added, sysconf(3) was update for POSIX.1-2001, and some of the glob(3) additions were MFC'd. The insque(), lsearch(), and remque() family of functions were reimplemented and moved to libc from libcompat. Several wide character functions were implemented, including all printf() and scanf() variants. Finally, support for wide character format types (%C, %S, %lc, %ls) were added to printf(3).
Work on utility conformance continued as getconf(1)'s compliance was updated, c99(1) (a new version of c89(1)) was implemented, and cd(1) and command(1) changes were MFC'd.
Almost 20 headers were brought up to conformance with applicable standards. Not much work remains to fix conformance issues in the remaining standard headers. Work in this area, as well as others, has slowed down in preparation for 5.0-RELEASE.
DEVD has been integrated into FreeBSD current. It was integrated in an incomplete state. However, it is useful in the state that it is in for doing simple things like running camcontrol rescan when a SCSI pcmcia card is inserted, or running /etc/pccard_ether with an ethernet card is inserted. The more sophisticated regular expression matching is not yet complete. Devd only does actions on device arrival and departure, but does not yet do anything with unknown devices. In addition to listening for device events, there is some desire to have /dev/devctl also allow for some direct control of the device tree.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
This work was committed to -current. To configure it for use specify options FAST_IPSEC in your system configuration file. At present support is limited to IPv4.
GBDE has been committed to -current.
The "Geom Based Disk Encryption" module provides a mechanism for very strong encryption of a GEOM "disk". The algorithm has passed informal review by a couple of seasoned crypto heavy-weights. Any GEOM device can be protected with GBDE, entire physical disks, - MBR slices, BSD paritions etc etc. Booting from an encrypted - partition is not possible however.
+ MBR slices, BSD partitions etc etc. Booting from an encrypted + partition is not possible, however.The focus of GBDE is to protect a "cold" disk media. (FreeBSD is not equipped well for protecting key material on a running system from being compromised.) For a cold media, the only feasible attack on a GBDE protected media is guessing the pass-phrase.
Summary of the GBDE multilevel protection scheme: Up to four separate pass-phrases can unlock their own separate copies of the 2048 bit masterkey. The master-keys are protected using AES/256/CBC keyed with a SHA-2 hash derived from the pass-phrase. A salted MD5 hash over the sectoroffset "cherry-picks" which masterkey bytes participate in the MD5 hash which generates the "kkey" for each particular sector. The kkey AES/128/CBC encrypts the PRNG produced single-use key which AES/128/CBC encrypts the actual sector data.
GBDE has features for master-key destruction and pass-phrase invalidation.
See gbde(4) and gbde(8) for more details.
This software was developed for the FreeBSD Project by Poul-Henning Kamp and NAI Labs, the Security Research Division of Network Associates, Inc. under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS research program.
The GEOM code is now the default on most (if not all ?) architectures and the few remaining issues in libdisk/sysinstall is being hashed out.
Although we are far from finished developing GEOM, its current feature set is a significant step forward for FreeBSD, providing not only immediate relief for new architectures (sparc64, ia64 etc) but also because it is designed as SMPng code from the start.
This software was developed for the FreeBSD Project by Poul-Henning Kamp and NAI Labs, the Security Research Division of Network Associates, Inc. under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS research program.
These last two months have seen quite a lot of GNOME activity.
GNOME has started releasing development snapshots of the upcoming
GNOME 2.2 desktop. FreeBSD porting has begun outside of the
main ports tree in the
Evolution 1.2 is also close at hand. Ximian has posted its first release candidate, 1.1.90, which has been ported to FreeBSD, and is available from the MarcusCom CVS repo listed above. As soon as Ximian officially releases Evolution 1.2, it will be placed in the FreeBSD ports tree.
The Mozilla ports have received numerous updates. We are now tracking all three released Mozilla versions. The mozilla-vendor port is tracking the 1.0.x branch, mozilla is tracking 1.1.x, and mozilla-devel is tracking 1.2.x. The mozilla-devel port now has support for anti-aliased fonts as well as a GTK+-2 interface
Finally, the GNOME team would like to welcome its newest team member, Adam Weinberger. Adam has been submitting patches for both GNOME ports as well as documentation. Currently, he has been active in the GNOME 2.2 porting effort. We are happy to have him.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPsec), and OpenSSL (through the /dev/crypto device).
This work was committed to the -current tree. To configure it for use - specifiy device crypto in your system configuration file or you can load the + specify device crypto in your system configuration file or you can load the crypto module. The /dev/crypto device support is brought in with device cryptodev or by loading the cryptodev module. Two crypto device drivers exist: ubsec for Broadcom-based PCI hardware and hifn for Hifn-based PCI hardware.
Integration of this work into the -stable source tree should be completed by the time this report is published.
Since the last status report the BSD Java Porting Team has continued to make steady progress. The most exciting news we have is courtesy of our newest team member, Alexey Zelkin of FreeBSD committer fame.
For 4.7-RELEASE, we privately published package ja-man-doc-4.7.tgz which consists of man[1256789] entries 10 days after the 4.7-RELEASE release date. Man3 update god no progress, as updating other sections busied us. We decided to suspend man3 update officially, as we need to spend most of our time to catch up with the forthcoming 5.0-RELEASE.
The KDE/FreeBSD team has been working on two major goals during the last two months, Maintenance of the KDE 3.0.x ports and Preparing the upcoming KDE 3.1 Release.
Maintenance KDE 3.0 conducted by Alan Eldrige: September started with the Removal of the KDE 2.x Ports from the FreeBSD-Repository. Later Packages of KDE 3.0.4 were released and the FreeBSD Ports were updated.
Preparing for KDE 3.1 conducted by Will Andrews: A lot of effort was spent on Improving the Fruitsalad-Build-System. We are now able to create packages directly from the KDE CVS.
The KSE code has now all the basic kernel functionality to start being used by the userland. There are still things - to be done for testing and familiarisation.
+ to be done for testing and familiarization.General system utilities have not yet been changed. e.g. ps and top etc. need to know about threads.
There is quite a lot of code in the kernel that still assumes that there is one thread in a process. Signals are not yet handled in the final manner (though they are delivered to a random thread in the process :-/ ).
The system calls and datastructures are now however in place. The test program successfully starts several threads that can be scheduled on different processors, and closes them down again. The userland is probably going to be able to do simple scheduling of pthread threads using KSE by the time that this report is published.
I still need someone to take over the "official" web page - since jason left. LaTex sure isn't my thing.
+ since jason left. LaTeX sure isn't my thing.Not much since the last status report, except that we now have the repo and development web page back online, thanks to the services of John De Boskey who freely provided the necessary hardware and bandwidth to host the project. We have also ported LibH to GCC 3.x, so that it can compile on -CURRENT correctly. This, however, broke tvision, which doesn't compile under GCC 3.x, so we moved to rhtvision but this caused linking problems so we're stuck with no console front end, for now.
Work on a Hui rewrite and SWIG bindings stalled. Alex was able to come up with a simple patch to make the ports system use LibH's pkg_create script to build libh packages, so we're getting closer to a real pkg_create(1) drop-in replacement. I rewrote the milestone list to show a bit more relevant and encouraging tasks that will be dealt with in order to really push LibH forward.
A mailing list was created, freebsd-mips, and a Perforce branch was created in //depot/projects/mips. Changes which will be necessary to allow multiple MIPS (and PowerPC) metaports to exist under one architecture port were made, and are being pushed back into the main FreeBSD tree. Some preliminary header work has been done, and porting the ARCBIOS interfaces to the kernel has begun. The toolchain in tree was updated and modified in places to support a FreeBSD/MIPS (Big Endian) target, in the Perforce branch. Some early boot code has proven the GDB MIPS simulator to work, for at least R3000 code, though whether R3000 will be supported has been under discussion. Some initial architectural decisions were also made, to steer current work.
Work on newcard continues. A number of bugs have been fixed in the last few months. You are now able to load and unload drivers (including the bridge) to test changes to pccard and/or cardbus bus code. It is now possible to load a driver that has a pccard attachment and have a previously inserted card probe and attach. This is also true for CardBus. A number of issues remain to be solved before 5.0. However, with the integration of devd into the tree nearly all of old functionality of OLDCARD is now present in NEWCARD (the biggest remaining parts are power control for the sockets, as well as pccardc dumpcis).
The PowerPC port has been running diskless on NewWorld G3/G4 machines for a while now. A GEOM module to support Apple Partition Maps is being written. There should be an installable ISO image available in the near future.
RAIDFrame was imported into FreeBSD-current in late October, a major milestone after 18 months. It is still very experimental and not suitable for production environments. The website contains a lengthy TODO list which I hope to start attending to soon. Still, I encourage everyone to try it out and report bugs back to me.
The Release Engineering (RE) team completed and released FreeBSD 4.7 on 10 October 2002. This release features updates for a number of contributed software programs in the base system, as well as all of the security and bug fixes from FreeBSD 4.6.2. The next release in the 4.X series will be FreeBSD 4.8, which has a scheduled release date of 1 February 2003.
Before that time, however, will be the release of FreeBSD 5.0. Thus far, we have not been able to release the 5.0-DP2 developer snapshot due to various stability issues. Thanks to much effort from many of our fellow developers, we believe that most of these have been resolved. The RE team wishes to emphasize that FreeBSD 5.0 will involve new code and features that have not seen widespread testing, and that more conservative users may wish to continue to track the 4.X series for the near-term future. To provide more information on these issues, we have added an Early Adopter's Guide to the release documentation for 5.0.
Brian Somers has resigned from the RE team due to increased time pressures. We thank him for all of his help with FreeBSD 4.5, 4.6, 4.6.2, and 4.7, and we hope to continue working with him as a fellow developer.
Scott Long has graciously offered to help improve the communication between the RE team and the rest of the developer community. We greatly appreciate his assistance.
Recent 5-current release procedure troubles prevent the project from releasing a new snapshots. But 5-current FreeBSD/i386 release is back again in late Oct/2002! I have a plan to build daily FreeBSD/sparc64 snapshots for 5-current. Stay tuned...
A lot has happened recently for the sparc64 port. Sysinstall and make release work and can be used to build installable snapshots. The gdb5.3 port now works, and, thanks to Thomas Moestl, kernel crash dumps are supported which can be analyzed by gdb. These 2 items are the last things considered necessary by the Core team for FreeBSD/sparc64 to be a Tier 1 architecture, which means that 5.0-RELEASE for sparc64 will be officially supported by the release engineering team and by the security officer team.
Recently Jake Burkholder has been working on alternate installation methods other than bootable iso, including a mini-root filesystem which can be written to the swap partition of an existing machine. Thomas Moestl has been putting some finishing touches on the release process, ensuring that the release documentation can be built properly, and that the port readme files can be generated by the release process.
An experimental iso built with make release is now available on the freebsd ftp site and mirrors in /pub/FreeBSD/development/sparc64/5.0-20021031-SNAP. It is expected that by the middle of November new 5.0-SNAP releases will be available every few days for download and for ftp install, cpu power and bandwidth permitting.
Most progress on TrustedBSD over the last two months related to improving the maturity of the ACL and MAC implementations, and merging new aspects of those features into the primary FreeBSD CVS Repository for inclusion in FreeBSD 5.0. This included fixes to run better on sparc64, improved tuning of what system objects are mediated, locking fixes and optimizations especially relating to the vnode and pipe implementations, improved support for MAC labeling on symlinks, support for asynchronous process label changes as required in some locking situations, remove use of "temporary labels" and prefer use of object type specific labels reducing redundant and/or confusing label management code in policies, improve avoidance of memory allocation in M_NOWAIT scenarios for socket allocation in the syncache, mediation of link operations, race condition fixes for devfs involving label creation, improve handling of VM events such as mmaping, improve mediation of socket send/receive events (as distinguished from socket transmit/deliver events), support for manipulating EAs on symlinks using new system calls, support for MNT_ACLS and MNT_MULTILABEL flags at mount time, as well as FS_ACLS and FS_MULTILABEL superblock flags to key useful defaults using tunefs, correction of a memory leak in the UFS ACL code, enable UFS ACL support by default in GENERIC, mediation points for file creation, deletion, and rename, support for a mac_execve() execution interface in the style of SELinux's execve_secure() permitting a label transition request as part of the exec operation for policies that support it, more consistent handling of NFS lookups, support for labeling of multicast encapsulated packets, ATM packet labeling, FDDI packet labeling, STF packet labeling, revised label interface that avoids userland parsing of per-policy elements, reducing us to a single instance of parsing and printing for each policy (and further abstracting policy implementation details from the library code).
Also, change to single-level sockets for Biba and MLS policies, support for partial label updates for Biba and MLS, addition of mac.9 man page, revised user API system calls, implementation of mac_get_pid(), and various other related bits, creation of mac.conf(5) to specify label defaults, checks for various system operations including swapon(), settime(), and sysctl(), reboot(), acct(), introduction of command line utilities for maintaining file and process labels, support for user labels tied to login class, su support for label changes, ifconfig support for interface labels, ps support for process labels, ls support for file labels, ftpd support for login labels, development of the Biba and MLS notions of privilege, and a move to C99 sparse structure initialization, restoring full type checking for policy entry points.
Universally Unique Identifiers (UUIDs) are 128 bit values that may - be generated independently on seperate nodes (hosts), which, result in + be generated independently on separate nodes (hosts), which result in globally unique strings. UUIDs are also known as Globally Unique Identifiers (GUIDs). The UUID support for FreeBSD (libc) conforms to the DCE 1.1 RPC specification.
-UUID suport has been added to FreeBSD -CURRENT, and will be available +
UUID support has been added to FreeBSD -CURRENT, and will be available in version 5.0. It is being extensively used in GPT partition handling for IA-64 platform. For now, a simple manual page has been provided, which outlines information about the provided uuid routines. Many documentation additions and enhancements to uuidgen(1) are in the pipeline.
The goal of this project is to improve the wireless networking support in the system. The initial work will incorporate the 802.11 link layer done by Atsushi Onoe for NetBSD. This core support code implements the basic 802.11 protocols required for Station and AP operation in BSS, IBSS, and Ad Hoc modes of operation. Wireless device drivers will then be revised to use this common code instead of their private implementations.
Following this initial stage the wireless networking support will be extended to support functionality needed for workgroup, enterprise, and metropolitan (e.g. mesh) networking environments. This will include full power management support, the 802.1D spanning tree protocol for running multiple AP's in a bridged configuration, QoS support, and enhanced security protocols (LEAP, AES, EAP). Support for new hardware devices is also planned.
In the month of September, the FreeBSD Project continued its investment in long-term projects, including continuing work on a fine-grained SMP implementation, support for Kernel Schedulable Entities (KSE) supporting highly efficient threading, and broadening support for modern hardware platforms, including Intel's new IA64 architecture, UltraSparc, and PowerPC. Additional focus was placed on the release process, including work on the release notes infrastructure, support for DVD releases, and work on a binary updating tool.
Due to the delay in getting the September report out the door, the November status report will also cover October. During the month of November, we look forward to BSDCon Europe, the first such event outside the continental United States. The USENIX conference paper submission deadlines are also in November, and FreeBSD users and developers are encouraged to submit to the general and FREENIX tracks. Please see www.usenix.org for more information.
PRFW provides hooks in the FreeBSD kernel, allowing users to insert their own checks in system calls and various kernel functions. PRFW is nearing 0.5, which will incorporate numerous structural changes such as, much faster per-process hooks, kernel function hooks, plus, a new way of adding hooks which would enable users to reference hooks by a string.
The build process is now creating four different versions of the libs, which include support for TVision, Qt, both or none. I created some first packages from existing ports and installed those libh packages on my system only using libh's tools, including registering all the files in the package database, recording their checksums etc. Patches to the disk editor have been submitted, which include functionality to write the changes in the fdisk part and initial support for a disk label editor. We'll soon have a new committer.
FreeBSD 4.4-RELEASE was the first release of FreeBSD with its new-style release documentation. Both English and Japanese versions of these documents were created. Regularly-built snapshots of -CURRENT and 4-STABLE release documentation are now available on the Web site, but they require a little HTML infrastructure to make them viewer-friendly. I intend to continue updating my snapshot site at the URL above, at least for a little while.
-Call for help: The hardware compatability lists need to be +
Call for help: The hardware compatibility lists need to be updated in the areas of the Alpha architecture, USB devices, and PCCARD devices. I'm looking for volunteers to help; interested parties should contact me at the email address above. DocBook experience is not required; familiarity with the hardware above would be very helpful.
Bug fixing and move to -STABLE of 2Gb support.
Quite a lot of cleanup of this driver. Bug fixes and some performance enhancements. However, this driver is likely to be removed shortly and replaced by one from Intel itself.
As you know, in march 2001 the version 2.3 of TIRPC has been - comitted together with many userland changes. Alfred Perlstein + committed together with many userland changes. Alfred Perlstein and Ian Dowse have helped me a lot with the porting effort and if I had problems with understanding the code.
Most bugs are now fixed, some remaining areas to fix are secure RPC (keyserv) and unix domain support. I've patches for these area available. Ian Dowse fixed a lot of outstanding bugs in the rpcbind binary itself. Thank you Ian !
The plan is now to migrate slowly towards TIRPC 2.8, which is threadsafe for the server- and clientside. One first patch I've made available on my URL. TIRPC 2.8 is licensed under the "Sun Standards License Version 1.0" and we have to add some license lines and the license itself to all modified files.
A example is timed_clnt_create.diff which can be found on the homepage.
The project has gained a mailing list, freebsd-binup@FreeBSD.org - and the source tree has been moved into the projects/ directory in the FreeBSD CVS repository. Current work is focusing on extending the FreeBSD package framework, and the client library should be rewritten and completed by the end of the year.
TODO: make the projects/ hierarchy into a cvsup distribution and add it to cvs-all. Then update distrib.self.
Status is unchanged since last month. Patches have been submitted to get ppp working under HURD, and mostly under Linux. There are GPL copyright problems that need to be addressed. Many conflicts are expected after the commit of IPv6 support in ppp.
The software has been committed to -current and seems functional. Outstanding issues include dealing with IPV6CP events (linkup & linkdown scripts) and allocating site-local and global addresses (currently, ``iface add'' is the only way to actually use the link). A bug exists in -stable (running the not-yet-MFC'd ppp code) whereby routing entries are disappearing after a time (around 12 or 24 hours). No further details are yet available.
A two disc set has been mastered and sent for pressing. There are a few surprises with this release - details will be given in the official announcement (at BSDConEurope).
ATM-Forum LAN-emulation version 2.0 without support for QoS has been implemented and tested. The ILMI daemon has been modularized into a general mini-SNMP daemon, an ILMI module and a not yet finished IPOA (IP over ATM) module.
We have finished updating section [125678] manpages to 4.4-RELEASE based, 1 week after 4.4-RELEASE is announced. To finish this update, OKAZAKI Tetsurou has imported Ex/Rv macro support on ja-groff-1.17.2_1. SUZUKI Koichi did most Ex/Rv changes on Japanese manpages. He also find some issues of these - macro usage on some original manapges and filed a PR. For + macro usage on some original manpages and filed a PR. For post-4.4-RELEASE, now we target 4.5-RELEASE. Section 3 update is also in progress.
We've made some good progress now, and the new nmount(2) syscall is nearly finished. There is still some work to do to have a working kernel_mount() and to convert all filesystems to use this new API for their VFS_MOUNT() functions.
I am pleased to announce that as of 1 AM Friday October 19th, the sparc64 port boots to single user mode. A few binaries from the base system have been built and verified to work properly. Much of this work is still in review for commit, but will be integrated into the cvs tree as soon as possible. EBus support has been ported from NetBSD, and ISA support has been written. The PCI host bridge code has stabilized, and busdma seems to work correctly now. The sio driver has had EBus support added, and the ATA driver has been modified so that it works on big-endian systems and uses the busdma API. With these changes, a root file system can now be successfully mounted from ATA disks on sparc64, even in DMA mode. The gem driver, which supports Sun GEM and ERI and Apple GMAC and GMAC2 ethernet adaptor, has been ported from - NetBSD but has not yet had sufficent testing.
+ NetBSD but has not yet had sufficient testing.No new status to report, the code is still waiting to be committed. It is likely that this code will be expanded to include syn cookies as a further fallback mechanism.
Development on this project has been slowed, pending the commit of the syncache code, as this builds on part of that work.
Not much progress has been made this month, with other projects occupying most of my time. However, reviewing all the code and data structures had a side benefit; a hash table for inet addresses has been added. This will significantly speed up interface address lookups in the case where there are a larger number of interface aliases.
Currently, a single device may act as a console at any time, which requires the user to choose the console device at boot time. With the upcoming network console support, it is desirable to allow multiple console devices which behave identically, and to alter consoles while the kernel is running.
The code is completed, and needs some final polishing to clean up the rough edges. Console output can be sent to both syscons and sio, (as well as the network) and when in ddb, input can be taken from any input source. A small control program allows adding and removing consoles on the fly.
This project's goal is to add low level network functionality to FreeBSD. The initial target is to make a network console available for remote debugging with ddb or gdb. A secondary target is to utilize the code to perform network crash dumps. The design assumes that the network card and driver are working, but does not rely on other parts of the kernel.
Initial development has been fairly rapid, and a minimal TCP/IP stack has been written. It is currently possible to telnet to a machine which is at the ddb> prompt and interact with the debugger.
Network devices now support aliases in the form of /dev/netN, where N is the interface index. Devices may be wired down to a specific index number by entries in /boot/device.hints of either:
hint.net.<ifindex>.dev="devname" hint.net.<ifindex>.ether="ethernet address"
Additionally, ifconfig has been updated so that it will accept the alias name when configuring a device.
The gx driver has finally been committed to the tree. The driver provides support for the Intel PRO/1000 cards, both fiber and copper variants. The driver supports VLAN tagging and TCP/IP checksum offload.
In the last month, not a lot has happenned other than settling +
In the last month, not a lot has happened other than settling in of the big August commit. Largely due to me having a sudden increased workload at work, and a need for increased time to be spent elsewhere. However some design work has proceeded. The API has firmed up somewhat and several people have been reading through what has been done already in order to be able to help in the next phase.
Milestone 3 will be to have the ability to generate and remove multiple threads/KSEs per process. Milestone 3 will NOT require that doing so will be safe. (especially in SMP systems), i.e. - locking issues will not be fully addressed, so while some testign + locking issues will not be fully addressed, so while some testing will be possible, it will not be possible to actually run in this mode with any load.
This will require allocators and destructors for the new - structures. Creation of the syscalls. Generation of an acurate + structures. Creation of the syscalls. Generation of an accurate written API for the userland crew. Writing of the upcall launch code. Production of a userland test program (not a full thread scheduler). Resolution of some of the more glaring incompatibilities (e.g. the scheduler) in a backwards compatible manner. (i.e. if there are no multi threaded processes on a system it should behave the same as now (and be as reliable)).
-Criterea for knowing when we have reached Milestione 3 is the +
Criteria for knowing when we have reached Milestone 3 is the ability for a simple process on an unloaded system to perform a series of blocking syscalls reliably. e.g. open 2 sockets, and - send data on one, after having done a read on another,and then + send data on one, after having done a read on another, and then 'respond' in like manner..
There have been a few major successes in the PowerPC port this month. Mark Peek has succeeded in getting the FreeBSD/PowerPC kernel cross compiled on FreeBSD and booting under the PSIM simulator (now in /usr/ports/emulators/psim-freebsd). I have succeeded in getting the FreeBSD loader to load and execute kernels using the OpenFirmware found on Apple Macintosh hardware. Mark is now working on completing some of the startup and pmap code, while I am taking advantage of the simulator to work on some interrupt and device issues.
The project has moved forward on JDK 1.3.1 development this month, with the release of two more patchsets. The team is reasonably confident that the latest patchset is a stable release of the core JDK 1.3.1 tools and classes, when the default "green" threads subsystem is used. This is mostly thanks to hard work by - Fuyuhiko Maruyama to stabilise and fix the code. Bill Huey has + Fuyuhiko Maruyama to stabilize and fix the code. Bill Huey has also been progressing with his work on the "native" threads subsystem, although this hasn't yet reached the stability of "green" threads. Another (arguably the) major highlight of the latest patchset was the integration of NetBSD support by Scott Bartram and Alistair Crooks (the latter of NetBSD packages fame). Hopefully OpenBSD support will follow, making it truly a united BSD Java Project.
This group is for discussion about the startup scripts in FreeBSD, primarily the scripts in /etc/rc*. Primary focus will be on improvements and importation of NetBSD's excellent work on this topic.
Alright folks, I finally got off my butt last night and put together a roadmap for the migration to the new rc.d init scripts that were imported from NetBSD a long time ago and just sat in the tree.
M1 (Patch included)
Setup infrastructure
Make rcorder compile
Hook rc.subr into the distribution (and mergemaster)
Hook rcorder into the world
Add toggle in rc.conf to switch between rc_ng and current boot
scripts
M2
Get FreeBSD to boot with the new boot scripts
Rewrite the /etc/rc.d scripts to work with FreeBSD
M3
Add some FreeBSD specific support into rc.subr
M4
Add true dependency checking to the infrastructure so that
starting nfsd will start mountd and rpcbind
add support into rc.subr
Add dependencies into rc.d scripts
I'd like a couple of people to take a look at this and then I'll submit a pr for it if there aren't too many objections. I'm expecting M2 to run into quite a bikeshed, but hey, I got my nice shiny asbestos back from the cleaners.
The FreeBSD C99/POSIX Conformance Project aims to implement all requirements of the C99 Standard and the latest 1003.1-200x POSIX draft (currently Draft 7). In cases where aspects of the standard cannot be followed, those aspects will be documented in the c99(7) or posix(7) manuals. It is also an aim of this project to implement regression tests to ensure correctness whenever possible.
Patches that implement the <stdint.h> and <inttypes.h> headers, and modifications to printf(3) have been developed and will be committed shortly. They will allow us to use some of the new types C99 introduces, such as intmax_t and the printf(3) conversion specifier "%j".
Some progress has been made on the proc locking this month. Also, a new LOCK_DEBUG macro was defined to allow some locking infrastructure to be more efficient. Kernels now only include the filenames of files calling mutex, sx, or semaphore lock operations if the filenames are needed. Also, mutex operations are no longer inlined if any debugging options are turned on. The ucred API was also overhauled to be more locking friendly. A group has also started investigating the tty subsystem to design and possibly implement a locking strategy.