diff --git a/en/smp/index.sgml b/en/smp/index.sgml index f5cf1eb48c..855420f657 100644 --- a/en/smp/index.sgml +++ b/en/smp/index.sgml @@ -1,1338 +1,1347 @@ - + %includes; Done"> In progress"> Stalled"> Not Started"> Resolved"> Unresolved"> + ]> &header;

Contents

Project Goal

The FreeBSD SMP project, often referred to as SMPng (SMP next generation), is focused on implementing fine-grained SMP support for the FreeBSD 5.0 kernel (scheduled for November 2002). Due to FreeBSD's history, this is much like trying to fit a square peg into a round hole, and as such, the intermediate results aren't pretty in many ways. We are specifically not attempting to rewrite the kernel from scratch, nor are we on a crusade to fix all the architectural nits currently present in the kernel. In fact, we expect to leave a trail of architectural nits that will still be evident in many ways when FreeBSD 5.0 is released. This is a pragmatic project rather than a theoretical one; we need to have the kernel working and stable in under a year, so time restraints require that we be realistic about what to do when.

Project Plan

This web page contains information related to the effort to improve SMP support in FreeBSD. In general, this project uses what it can from the BSD/OS 5.0 development kernel, and re-implements what cannot be directly used due to divergence in the code bases.

As with any free software project, a detailed schedule is not possible. We expect to have significant performance and stability issues that need to be worked through over the first several months of the project, though every effort will be made to keep -current running as well as possible.

Resources and Links

Status

Following is an incomplete list of general tasks.

- + + + + + + + +
Task Responsible Last updated Status
Convert the giant lock from spinning to blocking, add the scheduler lock, add per-CPU idle processes. &dillon; 25 June 2000 &status.done;
Port the BSD/OS locking primitives (i386). &jake; 3 July 2000 &status.done;
Implement heavy-weight interrupt threads (i386). &grog; 3 August 2000 &status.done;
Rewrite the low level interrupt code (i386 UP). &grog; 3 August 2000 &status.done;
Demonstrated reasonable stability (self-hosted buildworld) (i386 UP). -smp developers 12 August 2000 &status.done;
Port the BSD/OS locking primitives (alpha). 𝔡 24 August 2000 &status.done;
Stub out (disable) spl()s. &grog; 30 August 2000 &status.done;
Port the BSD/OS ktr code. &grog;, &jhb; 30 August 2000 &status.done;
Rewrite the low level interrupt code (i386 SMP). &jhb; 1 September 2000 &status.done;
Demonstrated reasonable stability (self-hosted buildworld) (i386 SMP). -smp developers 6 September 2000 &status.done;
Demonstrated reasonable stability (self-hosted buildworld) (alpha). -smp developers 6 September 2000 &status.done;
Make malloc and friends thread-safe. &jasone; 10 September 2000 &status.done;
Implement msleep(), make tsleep() an msleep() wrapper. &jake; 11 September 2000 &status.done;
Make fxp driver thread-safe. &cp; 17 September 2000 &status.done;
Make mbuf's thread-safe. &bmilekic; 29 September 2000 &status.done;
Lock manager re-work. &jasone; 3 October 2000 &status.done;
Implement heavy-weight interrupt threads (alpha). &jhb;, 𝔡 5 October 2000 &status.done;
Rewrite the low level interrupt code (alpha). 𝔡, &jhb; 5 October 2000 &status.done;
Process accounting. Tor Egge, &jhb; 5 October 2000 &status.done;
Make ethernet drivers thread-safe. Bill Paul 15 October 2000 &status.done;
Make the mutex headers mostly machine-independent. &jhb; 20 October 2000 &status.done;
Rename SMP_DEBUG to MUTEX_DEBUG. &jhb; 20 October 2000 &status.done;
Give each soft interrupt its own thread. &cp; 25 October 2000 &status.done;
Make sf_bufs (sendfile(2)) thread-safe. &bmilekic; 5 November 2000 &status.done;
Make the witness code work correctly. &jhb; 18 November 2000 &status.done;
Split the ktr-specific code out of db_interface.c. &jhb; 15 December 2000 &status.done;
Convert the sio driver to using a spin mutex. &jhb; 18 December 2000 &status.done;
Implement condition variables. &jake;, &jasone; 15 January 2001 &status.done;
Add a flag to mtx_init() (MTX_RECURSE) that denotes whether a mutex is allowed to recurse. &bmilekic; 19 January 2001 &status.done;
Make the zone allocator thread-safe. Dag-Erling Smorgrav 21 January 2001 &status.done;
Convert simplelocks to mutexes. &jasone; 24 January 2001 &status.done;
Make kernel preemptive with respect to interrupts. &jake; 31 January 2001 &status.done;
Cleanup of mutex API. &bmilekic; 8 February 2001 &status.done;
Remove COM_LOCK. &markm; 11 February 2001 &status.done;
Merge various scheduling classes into one run queue. Modify scheduler to support preemptable kernel. &jake; 11 February 2001 &status.done;
Make priority propagation work correctly. &jake; 11 February 2001 &status.done;
Make most of the interrupt thread code MI and shared between hardware and software interrupts. &jhb; 18 February 2001 &status.done;
Add protection to struct jail and jail-related functionality. &rwatson; 20 February 2001 &status.done;
Implement sx (shared/exclusive) locks. &jasone; 5 March 2001 &status.done;
Generalize/improve witness to handle more complex locking primitives (mtx, sx). &jhb; 28 March 2001 &status.done;
Convert the allproc and proctree locks from lockmgr locks to sx locks. &jhb; 28 March 2001 &status.done;
Make mbuf system use condition variables instead of msleep()/wakeup(). &bmilekic; 2 April 2001 &status.done;
Remove <sys/mutex.h> includes from other kernel headers such as <vm/vm_zone.h>, <sys/resourcevar.h>, <sys/ucred.h>, and <sys/mbuf.h>. &markm; 15 May 2001 &status.done;
Cleanup the various mp_machdep.c's, unify various SMP API's such as IPI delivery, etc. &jhb; 15 May 2001 &status.done;
Make most of the forward_* and forwarded_* functions MI. &jhb; 15 May 2001 &status.done;
Complete the MD support for SMP on the Alpha platform. &gallatin;, 𝔡, &jhb; 15 May 2001 &status.done;
Convert select() to use condition variables. &tanimura; 15 May 2001 &status.done;
Add a "giant" lock around the VM subsystem. &alfred; 13 June 2001 &status.done;
Introduce a modified slab allocator for the mbuf subsystem. &bmilekic; 21 June 2001 &status.done;
Add a witness_assert() function to handle lock assertions. &jhb; 27 June 2001 &status.done;
Extend sx locks to support try lock operations. &jhb; 27 June 2001 &status.done;
Document KTR. &jhb; 28 June 2001 &status.done;
Make fork_return, fork_exit, ast, and userret MI. &jhb; 29 June 2001 &status.done;
Make sched_lock's savecrit a per-process property saved and restored in mi_switch and initialized in fork_exit. &jhb; 30 June 2001 &status.done;
Make ast() loop. &jhb; 10 August 2001 &status.done;
Add upgrade/downgrade sx lock operations. Alexander Kabaev, &jasone; 13 August 2001 &status.done;
Implement semaphores. &jasone; 14 August 2001 &status.done;
Add support for upgrade/downgrades in witness. &jhb; 23 August 2001 &status.done;
Remove the MP safe syscall flag from the syscall table and add explicit mtx_lock/unlock's of Giant to all syscalls. &dillon; 7 September 2001 &status.done;
Make most of cpu_wait() and cpu_exit() MI. &peter; 9 September 2001 &status.done;
Split NFS into client and server. &peter; 18 Oct 2001 &status.done;
Lock taskqueues. Andrew Reiter, &jhb; 25 October 2001 &status.done;
Add a per-thread ucred reference. &jhb; 25 October 2001 &status.done;
Push the grabbing of Giant into FreeBSD ABI system calls. &dillon; 26 October 2001 &status.done;
Make most of the per-CPU stuff MI. &jhb; 11 December 2001 &status.done;
Make critical section saved state per-thread instead of per-lock so that interlocking spin locks work properly. &jhb; 17 December 2001 &status.done;
Replace the APIC-specific imen_mtx with a MI-named icu_lock to protect interrupt controllers and associated data within the kernel for both i386 and alpha. &jhb; 20 December 2001 &status.done;
Use the per-thread critical section nesting level in the mutex and interrupt thread code to automatically determine when to not preempt. This makes the MTX_NOSWITCH, SWI_SWITCH, and SWI_NOSWITCH flags obslete as the kernel will be able to figure out the proper behavior on its own. &jhb; 5 January 2002 &status.done;
Lock struct filedesc and struct file. &tanimura;, &alfred; 12 January 2002 &status.done;
Lock struct pgrp, struct session, and struct sigio. &tanimura; 23 February 2002 &status.done;
Lock pipe implementation, but not sigio/fown, VM interactions &alfred; 27 February 2002 &status.done;
Lock struct proc. &jhb; 20 February 2001 &status.wip;
Make KTRACE write into tracefiles asynchronously. &jhb; 6 September 2001 &status.wip;
Make the kernel fully preemptive. &jhb; 7 September 2001 &status.wip;
Convert vm_map locks to sx locks. &green; 18 October 2001 &status.wip;
Lock down the tty subsystem. Dick Garner, Jeremy Scofield 16 November 2001 &status.wip;
Fix clock locking to be the same on all platforms. &jhb; 16 November 2001 &status.wip;
Make sleep mutexes spin if the current lock holder is executing on another CPU. &jhb; 16 November 2001 &status.wip;
Add support for the IA32 pause instruction to spin loops in locks. &jhb; 16 November 2001 &status.wip;
Optimize UP support by changing spin locks to only perform critical section enter and exits. &jhb; 20 December 2001 &status.wip;
Implement lazy interrupt thread switching (context stealing) on i386. &bmilekic; 4 January 2002 &status.wip;
Fix synchronization of TLB flushes and invlpg() on x86 SMP. &peter; 4 January 2002 &status.wip;
Lock pipe implementation: sigio/fown-related evil &alfred; 27 February 2002 &status.wip;
Broadly transition to td_ucred from p_ucred once KSE dependencies are in place. &jhb; 27 February 2002 &status.wip;
Update suser() and p_can*() APIs to accept threads instead of processes. &jhb; 27 February 2002 &status.wip;
Lock down kernel linker and module structures &arr; 27 February 2002 &status.wip;
Make use of process locking and process reference counting to protect debugging interfaces (and procfs). &jhb; 27 February 2002 &status.wip;
Make use of process locking to protect process monitoring sysctls, including those employed by 'ps' and related tools. &jhb; 27 February 2002 &status.wip;
Create mechanism in cdevsw structure to protect thread-unsafe drivers. &jhb; 15 May 2001 &status.stalled;
Make printf() safe to call in almost any situation to avoid deadlocks. &cp; 15 May 2001 &status.stalled;
Implement trap handler for cmpxchgl on 80386 to support generic userland atomic operations. &jake; 15 May 2001 &status.stalled;
Add locking to NFS.   15 May 2001 &status.new;
Use per-CPU buffers for ktr to reduce synchronization.   5 September 2000 &status.new;
Remove priority argument from tsleep(), msleep(), cv_*wait*().   12 January 2001 &status.new;
Reimplement kqueue using condition variables. &jlemon; 15 March 2001 &status.new;
Conditionalize atomic ops in the SMP code that are used for debugging statistics. &peter; 15 March 2001 &status.new;
Add a new witness check for exiting processes to verify that an exiting process holds no locks. &jhb; 13 June 2001 &status.new;
Make cpu_core MI.   13 June 2001 &status.new;
Lock struct pargs.   20 June 2001 &status.new;
Specify priorities for condition variables, semaphores, and sx locks.   7 September 2001 &status.new;
Fix SIGXPU and other #if 0'd things in mi_switch().   7 September 2001 &status.new;
Axe schedpu() in favor of event driven priority updates as much as possible.   7 September 2001 &status.new;
Fix PHOLD() so that it blocks to guarantee PS_INMEM.   7 September 2001 &status.new;
Make grow_stack() MI. Possibly even a macro or inline.   7 September 2001 &status.new;
Fix *hold (e.g. crhold) to return reference to object.   7 September 2001 &status.new;
Fix various procfs_machdep.c to use PHOLD, not sched_lock.   7 September 2001 &status.new;
Convert eventhandlers to sx locks.   7 September 2001 &status.new;
Add witness checking for lockmgr locks.   7 September 2001 &status.new;
Add ICU spin locks on ia64 and sparc64.   4 January 2002 &status.new;
Lock sysctl hierarchy and access methods.   25 February 2002 &status.new;
Fast-path push-down of Giant for VOP_READ() and VOP_WRITE().   25 February 2002 &status.new;
Lock contention measurement tool to measure heat of various locks, including Giant, and permit more directed performance and locking strategy optimization.   25 February 2002 &status.new;
Push the grabbing of Giant into Linux i386 ABI system calls.   25 February 2002 &status.new;
Push the grabbing of Giant into Linux AXP ABI system calls.   25 February 2002 &status.new;
Push the grabbing of Giant into SVR4 i386 ABI system calls.   25 February 2002 &status.new;
Push the grabbing of Giant into OSF/1 AXP ABI system calls.   25 February 2002 &status.new;
Push the grabbing of Giant into IBCS i386 ABI system calls.   25 February 2002 &status.new;
Lock pipe implementation: VM optimizations Lock pipe implementation: VM optimizations.   27 February 2002 &status.new;
Lock down newbus infrastructure to support driver + fine-graining. &imp; 27 February 2002 &status.new;

This table lists the todo subtasks for multithreading the network stack.

Task Responsible Last updated Status
Protect network interface queues. &jlemon; 24 November 2000 &status.done;
Lock struct ifnet.   19 January 2001 &status.new;

Known Issues

Issue Last updated Status
Idle processor time is not charged to the idle processes. 20 September 2000 &status.resolved;
microuptime creeps backwards. 4 October 2000 &status.resolved;
microuptime() went backwards 4 October 2000 &status.resolved;
Process accounting is not accurate (the more CPUs, the closer to correct it is). 5 October 2000 &status.resolved;
M_DEVBUF is probably the wrong memory pool for interrupt stuff and we should think about creating a new malloc pool for that stuff. 9 February 2001 &status.resolved;
PC card eject panics due to a race condition in the interrupt thread code. 15 March 2001 &status.resolved;
SMP x86 boxes are seeing NCPU * 100 clk interrupts and NCPU * 128 rtc interrupts. 15 May 2001 &status.resolved;
Witness will infinitely recurse when it acquires Giant after sleeping with a sleepable lock. 27 June 2001 &status.resolved;
Serial gdb does not work if boot_ddb and boot_gdb options are specified. 5 September 2000 &status.unresolved;
Serial gdb does not work at 115200 baud. 5 September 2000 &status.unresolved;
Profiling is broken. 20 February 2001 &status.unresolved;

News

The remainder of this page is structured as a reverse-chronological log.

13 January 2002 15 May 2001 22 March 2001 5 March 2001 24 January 2001 12 January 2001 11 October 2000 8 September 2000 6 September 2000 5 September 2000 1 September 2000 30 August 2000 12 August 2000 3 August 2000 6 July 2000 5 July 2000 3 July 2000 26 June 2000 25 June 2000 19 June 2000 &footer;