FreeBSD C99 & &posix; Conformance Project

diff --git a/en/projects/c99/index.sgml b/en/projects/c99/index.sgml index 2db3aadbee..9c688eedb6 100644 --- a/en/projects/c99/index.sgml +++ b/en/projects/c99/index.sgml @@ -1,2005 +1,2005 @@ - + %includes; %developers; "> "> "> "> "> assert(3)"> printf(3)"> scanf(3)"> strtod(3)"> printf(9)"> freebsd-standards-subscribe@FreeBSD.org"> freebsd-standards@FreeBSD.org"> ]> FreeBSD C99 & &posix; Conformance Project FreeBSD C99 & POSIX Conformance Project

The FreeBSD C99 & &posix; Conformance Project aims to implement all requirements of the ISO 9899:1999 (C99) and IEEE 1003.1-2001 (POSIX) standards. In cases where aspects of these standards cannot be followed, those aspects will be documented in the c99(7) or posix(7) manuals. It is also an aim of this project to implement regression tests to ensure correctness whenever possible.

All of the initial development takes place in the 5-CURRENT branch and most changes are merged into the 4-STABLE branch. Since this is a rather large project and is being done on a volunteer basis, there is no set schedule.

The project is always looking for additional contributors. If you wish to participate, the best way to find out what is happening is to subscribe to the freebsd-standards mailing list. If you are working on a specific task relating to the project, you should contact mike@FreeBSD.org with a description of the task so that your task can be listed below.

ISO/IEC 9899:1999, also known as C99, is available for purchase in electronic form from the American National Standards Institute, or available from the International Organization for Standardization store in electronic and paper form.
IEEE Std 1003.1-2001 - The free (registration required), online version of POSIX.1-2001 and SUSv3 standard.
Are you Ready For C99? - This article discusses some of the new features in C99.
Most discussion relating to this project takes place on the &a.standards; mailing list. To subscribe to the list, send a message to &a.standards.subscribe;.
Jens Schweikhardt maintains a list with the current status of POSIX Utility Conformance in FreeBSD.

Task	Responsible	Last Updated	Status
Task	Responsible	Last Updated	5.x	4.x
Documentation
Create stdint(7) manual to document <stdint.h>.	&a.mike;	15 September 2002	&status.done;	&status.n-a;
Document flockfile(3), ftrylockfile(3), funlockfile(3).	&a.tjr;	22 January 2004	&status.done;	&status.n-a;
Document new _SC options in sysconf(3) (PR: 20528).	&a.trhodes;	13 January 2005	&status.done;	&status.blank;
Document standards conformity level of the API.		26 March 2002	&status.wip;	&status.blank;
Document thread safety and async-cancel safety.		26 March 2002	&status.wip;	&status.blank;
Create c99(7) and posix(7) manuals.		11 December 2001	&status.new;	&status.blank;
Document in posix(7) that the batch job management utilities are available in the port net/generic-nqs.		11 December 2001	&status.new;	&status.blank;

Task	Responsible	Last Updated	Status
Task	Responsible	Last Updated	5.0	4.x
Functions and System Calls
Add __func__ reporting to &man.assert.3;.	&a.asmodai;	27 October 2001	&status.done;	&status.n-a;
Implement imaxabs(), imaxdiv(), llabs(), lldiv().	&a.mike;	14 November 2001	&status.done;	&status.n-a;
Implement atoll(), strtoimax(), strtoumax().	&a.fenner;	27 November 2001	&status.done;	&status.n-a;
Implement new length modifiers (hh, j, t, z) and new flag (') in &man.printf.3;.	&a.wollman;, &a.fenner;	30 November 2001	&status.done;	&status.n-a;
Implement strerror_r().	&a.wes;	20 December 2001	&status.done;	&status.done;
Implement tgamma() based on Lite1's gamma().	&a.bde;	27 March 2002	&status.done;	&status.n-a;
Implement new length modifiers (hh, j, ll, t, z) in &man.scanf.3;.	&a.fenner;	20 April 2002	&status.done;	&status.n-a;
Add WCONTINUED to waitpid(2).	&a.mike;	1 June 2002	&status.done;	&status.n-a;
Implement new conversion specifiers (%j and %n) in &man.printf.9;.	&a.dd; &a.des;	3 June 2002	&status.done;	&status.n-a;
Implement fstatvfs() and statvfs() (trivial).	&a.wollman;	14 June 2002	&status.done;	&status.n-a;
Implement fmtmsg().	&a.mike;	8 August 2002	&status.done;	&status.n-a;
Implement ulimit().	&a.mkm;	8 August 2002	&status.done;	&status.n-a;
Implement C99's _Exit().	&a.wollman;	11 September 2002	&status.done;	&status.n-a;
Add required options for POSIX.1-2001 to glob(). (partial MFC)	&a.mikeh;	20 September 2002	&status.done;	&status.done;
Update sysconf(3) for POSIX.1-2001.	&a.wollman;	18 September 2002	&status.done;	&status.n-a;
Restore insque(), lsearch(), remque() from libcompat.	&a.robert;	16 November 2002	&status.done;	&status.n-a;
Implement fpclassify().	&a.das;, &a.mike;	8 February 2003	&status.done;	&status.n-a;
Implement isfinite(), isinf(), isnan(), isnormal().	&a.das;	17 February 2003	&status.done;	&status.n-a;
Implement isgreater(), isgreaterequal(), isless(), islessequal(), islessgreater().	&a.das;	17 February 2003	&status.done;	&status.n-a;
Implement signbit().	&a.mike;	17 February 2003	&status.done;	&status.n-a;
Implement sockatmark().	&a.fenner;	23 December 2002	&status.done;	&status.new;
Implement wordexp() and wordfree().	&a.tjr;	31 December 2002	&status.done;	&status.new;
Implement grantpt(), posix_openpt(), ptsname(), unlockpt().	&a.ryany;	4 February 2003	&status.done;	&status.new;
Add floating-point support for the (') modifier in &man.printf.3;	&a.das;	22 January 2004	&status.done;	&status.blank;
Add new conversion specifiers (%a and %A) to &man.printf.3;.	&a.das;	22 January 2004	&status.done;	&status.n-a;
Transition to vendor (Netlib) source for strtod.c.	&a.das;	22 January 2004	&status.done;	&status.n-a;
Import Netlib's gdtoa (strtof, strtold, etc.).	&a.das;	22 January 2004	&status.done;	&status.n-a;
Implement C99 macros fpclassify(), isfinite(), isinf(), isnan(), isnormal(), signbit().	&a.mike;, &a.das;	22 January 2004	&status.done;	&status.n-a;
Implement C99 macros isgreater(), isgreaterequal(), isless(), islessequal(), islessgreater(), isunordered().	&a.mike;, &a.das;	22 January 2004	&status.done;	&status.n-a;
Implement new hex floating-point format types (%a/%A) in &man.scanf.3;.	&a.das;	22 January 2004	&status.done;	&status.n-a;
Implement getpwnam_r(), getpwuid_r().	&a.nectar;	7 June 2004	&status.done;	&status.n-a;
Implement posix_madvise().	&a.bms;	7 June 2004	&status.done;	&status.n-a;
Implement new functions that appear in <fenv.h>.	&a.das;	7 June 2004	&status.done;	&status.n-a;
Implement ftw() and nftw() in terms of fts().	&a.das;	8 July 2004	&status.done;	&status.blank;
Implement C99 rounding functions lrint(), lrintf(), lrintl(), llrint(), llrintf(), llrintl().	&a.das;	13 January 2005	&status.done;	&status.blank;
Implement getgrgid_r(), getgrnam_r().	&a.wes;	19 August 2005	&status.done;	&status.blank;
Implement new functions that appear in <complex.h>.	&a.markm;	6 February 2002	&status.wip;	&status.blank;
Implement waitid(2).	&a.mike;	27 May 2002	&status.wip;	&status.blank;
Implement the utmpx family of functions.	&a.robert;	28 June 2002	&status.wip;	&status.blank;
Add restrict type-qualifier to functions that require it.	&a.robert;	16 August 2002	&status.wip;	&status.blank;
Implement iconv(), iconv_close(), iconv_open().	&a.tjr;	8 October 2002	&status.wip;	&status.blank;
Update fmtcheck(3) to recognize new printf(3) flags.		16 October 2002	&status.wip;	&status.n-a;
Make non thread-safe functions thread-safe.		22 October 2001	&status.new;	&status.blank;
Make functions async-cancel safe.		22 October 2001	&status.new;	&status.blank;
Obtain a64l(), l64a(), and l64a_r() (extention) from NetBSD.		9 September 2002	&status.new;	&status.blank;
Add SIG_HOLD support to signal(3)/sigaction(2).		13 October 2002	&status.new;	&status.blank;
Implement RTS functions sigqueue(), sigtimedwait(), sigwaitinfo().		13 October 2002	&status.new;	&status.blank;
Implement posix_mem_offset(), posix_typed_mem_get_info(), posix_typed_mem_open(), posix_memalign().		7 June 2004	&status.new;	&status.blank;
Implement long double versions of libm routines.		13 January 2005	&status.new;	&status.blank;
Implement exp2(), fma(), log2(), nan(), nexttoward(), and remquo() in libm.		13 January 2005	&status.new;	&status.blank;

Task	Responsible	Last Updated	Status
Task	Responsible	Last Updated	5.0	4.x
Headers
Fix prototypes and hide non-conforming portions of header <arpa/inet.h>.	&a.mike;	19 February 2002	&status.done;	&status.n-a;
Implement the <stdint.h> and <inttypes.h> headers.	&a.mike;	1 March 2002	&status.done;	&status.n-a;
Add exposure control primitives to <sys/cdefs.h> (originally developed by &a.wollman;).	&a.mike;	1 April 2002	&status.done;	&status.n-a;
Implement <sys/_types> to store shared MI types.	&a.mike;	1 April 2002	&status.done;	&status.n-a;
Implement header <strings.h>, and work out compatibility issues with header <string.h>.	&a.mike;	4 April 2002	&status.done;	&status.n-a;
Hide non-conforming portions, add missing types to <sys/un.h>.	&a.mike;	19 April 2002	&status.done;	&status.n-a;
Remove pollution and add missing macros to <netinet/in.h> and <netinet6/in6.h>.	&a.mike;	23 May 2002	&status.done;	&status.n-a;
Hide non-conforming portions of <sys/wait.h>.	&a.mike;	3 June 2002	&status.done;	&status.n-a;
Hide non-conforming portions of <pwd.h>.	&a.mike;	9 June 2002	&status.done;	&status.n-a;
Create <sys/statvfs.h> based on POSIX spec.	&a.wollman;	14 June 2002	&status.done;	&status.n-a;
Add nfds_t and resolve other issues in <poll.h> (actually <sys/poll.h>).	&a.mike;	8 July 2002	&status.done;	&status.n-a;
Implement the XSI header, <cpio.h>.	&a.mike;	1 August 2002	&status.done;	&status.n-a;
Implement <fmtmsg.h>.	&a.mike;	8 August 2002	&status.done;	&status.n-a;
Implement <ulimit.h>.	&a.mkm;	8 August 2002	&status.done;	&status.n-a;
Merge <machine/ansi.h> and <machine/types> into a new common header <machine/_types.h>.	&a.bde;, &a.mike;	23 August 2002	&status.done;	&status.n-a;
Fix namespaces, add missing types in <sys/mman.h>.	&a.mike;	29 August 2002	&status.done;	&status.n-a;
Fix namespace issues in <ctype.h>.	&a.mike;	9 September 2002	&status.done;	&status.n-a;
Fix namespace issues in <dlfcn.h>.	&a.wollman;, &a.mike;	11 September 2002	&status.done;	&status.n-a;
Fix namespace issues in <fcntl.h>.	&a.mike;	18 September 2002	&status.done;	&status.n-a;
Fix namespace issues in <fnmatch.h>.	&a.mike;	18 September 2002	&status.done;	&status.n-a;
Fix namespace issues in <grp.h>.	&a.mike;	18 September 2002	&status.done;	&status.n-a;
Fix namespace issues in <langinfo.h>.	&a.mike;	18 September 2002	&status.done;	&status.n-a;
Add va_copy() macro to <stdarg.h> (actually <machine/stdarg.h>).	&a.mike;	20 September 2002	&status.done;	&status.n-a;
Fix namespaces issues in <stdlib.h>.	&a.wollman;	21 September 2002	&status.done;	&status.n-a;
Add new pathname, system variable, and POSIX option constants to <unistd.h>.	&a.wollman;	21 September 2002	&status.done;	&status.n-a;
Merge two conflicting math.h headers into a single header.	&a.bde;	2 October 2002	&status.done;	&status.n-a;
Fix namespace issues in <net/if.h>.	&a.mike;	2 October 2002	&status.done;	&status.n-a;
Fix namespace issues in <netinet/tcp.h>.	&a.mike;	2 October 2002	&status.done;	&status.n-a;
Add size_t, fix regoff_t in <regex.h>.	&a.mike;	2 October 2002	&status.done;	&status.n-a;
Fix namespace issues in <sched.h> (actually <posix4/sched.h>).	&a.mike;	5 October 2002	&status.done;	&status.n-a;
Fix namespace issues in <semaphore.h> (actually <posix4/semaphore.h>).	&a.mike;	5 October 2002	&status.done;	&status.n-a;
Fix namespace issues in <setjmp.h> (actually <machine/setjmp.h>).	&a.mike;	5 October 2002	&status.done;	&status.n-a;
Fix namespace issues in <stdio.h>.	&a.wollman;	7 October 2002	&status.done;	&status.n-a;
Add missing members to struct lconv in <locale.h>.	&a.tjr;	9 October 2002	&status.done;	&status.n-a;
Fix namespace issues, add missing types in <sys/socket.h>.	&a.mike;	13 October 2002	&status.done;	&status.n-a;
Fix namespace issues, add missing types in <sys/uio.h>.	&a.mike;	13 October 2002	&status.done;	&status.n-a;
Fix namespace issues in <signal.h> (also <sys/signal.h> and <machine/signal.h>).	&a.mike;	13 October 2002	&status.done;	&status.n-a;
Eliminate <sys/_posix.h>.	&a.mike;	16 October 2002	&status.done;	&status.n-a;
Implement the header <wordexp.h>.	&a.tjr;	31 December 2002	&status.done;	&status.new;
Add missing constants FLT_EVAL_METHOD, DECIMAL_DIG to <float.h> (actually <machine/float.h>).	&a.mike;, &a.das;	22 January 2004	&status.done;	&status.n-a;
Implement the header <fenv.h>.	&a.das;	7 June 2004	&status.done;	&status.n-a;
Implement <ftw.h>.	&a.das;	8 July 2004	&status.done;	&status.blank;
Implement <tgmath.h>.	&a.stefanf;	13 January 2005	&status.done;	&status.blank;
Implement new types and prototypes in header <complex.h>.	&a.markm;	6 February 2002	&status.wip;	&status.blank;
Hide non-conforming portions, add missing types to <sys/types.h>.	&a.mike;	1 April 2002	&status.wip;	&status.blank;
Reduce namespace pollution and add missing typedefs in <unistd.h>	&a.mike;	25 March 2002	&status.wip;	&status.blank;
Implement <utmpx.h> to eventually replace <utmp.h>.	&a.robert;	28 June 2002	&status.wip;	&status.blank;
Add missing errnos to <errno.h> (actually <sys/errno.h>).	&a.mike;	11 September 2002	&status.wip;	&status.blank;
Implement the header <iconv.h>.	&a.tjr;	7 October 2002	&status.wip;	&status.blank;
Hide non-conforming portions, add missing types to <sys/stat.h>.	&a.mike;	27 February 2003	&status.wip;	&status.blank;
Fix sizes of members in glob_t, add restrict type-qualifiers in <glob.h>.		20 September 2002	&status.new;	&status.blank;
Decide whether to remove const type-qualifiers in <libgen.h>.		20 September 2002	&status.new;	&status.blank;
Add missing constants in <limits.h>.		20 September 2002	&status.new;	&status.blank;
Fix sizes of members in datum, add missing types, fix namespace issues in <ndbm.h>.		2 October 2002	&status.new;	&status.blank;
Fix size of si_addrlen in struct addrinfo, add missing types, fix namespace issues in <netdb.h>.		2 October 2002	&status.new;	&status.blank;
Evaluate missing features of <pthread.h>.		2 October 2002	&status.new;	&status.blank;
Add missing members to struct sigevent, fix type of ss_sp in struct sigstack, add SIGEV_THREAD constant, add ucontext_t and mcontext_t in <sys/signal.h>.		13 October 2002	&status.new;	&status.blank;
Fix types for members of struct ipc_perm in <sys/ipc.h>.		16 October 2002	&status.new;	&status.blank;

Task	Responsible	Last Updated	Status
Task	Responsible	Last Updated	5.0	4.x
Miscellaneous
Obtain a copy of Proc. ACM SIGPLAN '90 (required for &man.printf.3; %'f).	&a.fenner;	9 November 2001	&status.done;	&status.n-a;
Upgrade system compiler to GCC 3.1.	&a.obrien;	15 May 2002	&status.done;	&status.n-a;
Add regression test for each function.		17 November 2001	&status.new;	&status.blank;
Merge sys/posix4/* into regular kernel hierachy, and remove +	Merge sys/posix4/* into regular kernel hierarchy, and remove sys/posix4.		21 September 2002	&status.new;	&status.blank;
Open an Aardvark on how to deal with no newline on the last line of a file in diff(1).		25 November 2002	&status.new;	&status.blank;
Globally rename the `lint' macro to something in the implementation namespace so it can safely modify the implementation (reference).		25 November 2002	&status.new;	&status.blank;
Globally rename the `LOCORE' macro to something in the implementation namespace so it can safely modify the implementation (reference).		25 November 2002	&status.new;	&status.blank;

- +

Task	Responsible	Last Updated	Status
Task	Responsible	Last Updated	5.0	4.x
Utility Conformance
Add -o option and base64 decoding to the uudecode utility.	&a.jmallett;	26 March 2002	&status.done;	&status.done;
Add -m option and base64 encoding to the uuencode utility.	&a.jmallett;	26 March 2002	&status.done;	&status.done;
Require atleast one file argument in paste(1). (PR: 36074)	Require at least one file argument in paste(1). (PR: 36074)	&a.tjr;	1 April 2002	&status.done;	&status.done;
Add the -i option to the patch utility.	&a.jmallett;	9 April 2002	&status.done;	&status.done;
Add -r and -t options to the at utility.	&a.joe.halpin;	16 April 2002	&status.done;	&status.n-a;
Add the -f and -p options to the pr utility. (PR: 36243)	&a.tjr;	16 April 2002	&status.done;	&status.done;
Add the -s option to the m4 utility. (PR: 36075)	&a.tjr;	24 April 2002	&status.done;	&status.done;
Make printf(1) utility conform to POSIX.2 (1992). (PR: 35616)	&a.tjr;	2 May 2002	&status.done;	&status.done;
Correct output stream and exit status of the mesg(1) utility.	&a.tjr;	6 May 2002	&status.done;	&status.n-a;
Implement the asa utility. (PR: 36130)	&a.tjr;	15 May 2002	&status.done;	&status.n-a;
Add the tabs utility. (PR: 36126)	&a.tjr;	20 May 2002	&status.done;	&status.n-a;
Restore sccs utility from CVS attic.	&a.jmallett;	22 May 2002	&status.done;	&status.n-a;
Make expr(1) conform to POSIX.1-2001.	&a.wollman;	22 May 2002	&status.done;	&status.n-a;
Add the newgrp utility. (PR: 36190)	&a.tjr;	28 May 2002	&status.done;	&status.n-a;
Fix ctags(1)'s -t option; fix ordering with -x.	&a.tjr;	2 June 2002	&status.done;	&status.n-a;
Add -L option to the pwd utility.	&a.tjr;	20 June 2002	&status.done;	&status.done;
Fix exit status of the env(1) utility.	&a.tjr;	20 June 2002	&status.done;	&status.done;
Add the -b and -s option to the fold utility. (PR: 36245)	&a.tjr;	20 June 2002	&status.done;	&status.done;
Add -I and -L options to the xargs utility.	&a.jmallett;	20 June 2002	&status.done;	&status.done;
Implement the csplit utility. (PR: 36191)	&a.tjr;	20 June 2002	&status.done;	&status.n-a;
Fix exit status of the nice(1) utility.	&a.tjr;	20 June 2002	&status.done;	&status.done;
Add -n option to the nice utility.	&a.pavalos;	20 June 2002	&status.done;	&status.done;
Add -n option and resolve other issues in the renice utility. (PR: 36950)	&a.pavalos;, &a.maxim;	20 June 2002	&status.done;	&status.done;
Add -t option to unexpand(1) utility. (PR: 35621)	&a.tjr;	20 June 2002	&status.done;	&status.done;
Make the uniq(1) utility accept `-' as a stdin specifier.	&a.tjr;	28 June 2002	&status.done;	&status.done;
Improve join(1)'s POSIX.1-2001 conformance. (PR: 36072)	&a.tjr;	28 June 2002	&status.done;	&status.done;
Add -C option to the tr utility.	&a.tjr;	28 June 2002	&status.done;	&status.n-a;
Implement the -m, -p, and -x options in the ls utility.	&a.mkm;	8 July 2002	&status.done;	&status.done;
Reimplement the who utility. (PR: 36128)	&a.tjr;	14 July 2002	&status.done;	&status.done;
Fix exit status, stdout/stderr confusion, and add `-' stdin support to the compress(1) utility.	&a.tjr;	17 July 2002	&status.done;	&status.done;
Add the -s, -A, -j, -N, and -t options to the od utility. (PR: 36783)	&a.tjr;	24 July 2002	&status.done;	&status.done;
Add -a option to the split utility.	&a.tjr;	30 July 2002	&status.done;	&status.done;
Add -m option to the wc utility.	&a.tjr;	29 August 2002	&status.done;	&status.done;
Add -L and -P options to the cd utility (shell built-in).	&a.tjr;	7 October 2002	&status.done;	&status.done;
Reimplement command(1) (shell built-in).	&a.tjr;	7 October 2002	&status.done;	&status.done;
Add the c99 utility (similar to c89). (PR: 36087)	&a.tjr;	8 October 2002	&status.done;	&status.n-a;
Update getconf(1) for POSIX.1-2001.	&a.wollman;	12 November 2002	&status.done;	&status.done;
Add the pathchk utility.	&a.tjr;	12 November 2002	&status.done;	&status.done;
Document the -n option in the echo utility as being implementation-defined.	&a.schweikh;	8 May 2003	&status.done;	&status.done;
Add missing options to the mailx utility.	&a.mikeh;	22 December 2001	&status.wip;	&status.blank;
Add missing components to the jobs utility.	&a.mkm;	5 March 2002	&status.wip;	&status.blank;
Work out conflicting -t option in the df utility.	&a.tjr;	22 April 2002	&status.wip;	&status.blank;
Add unimplemented features to the sh utility.	&a.tjr;	20 May 2002	&status.wip;	&status.blank;
Implement the -m, -w, and -t options in the lp utility.	&a.gad;	23 May 2002	&status.wip;	&status.blank;
Implement/import the SCCS development utilities: admin, delta, get, prs, rmdel, sact, unget, val	&a.jmallett;	23 May 2002	&status.wip;	&status.blank;
Investigate adopting some of the POSIX options in the ps utility.	&a.jmallett;	6 June 2002	&status.wip;	&status.blank;
Add missing options and resolve conflicting options in the nm utility.	&a.robert;	2 August 2002	&status.wip;	&status.blank;
Implement the iconv utility.	&a.tjr;	9 October 2002	&status.wip;	&status.blank;
Implement the locale and localedef utilities.	&a.phantom;	16 October 2002	&status.wip;	&status.blank;
Add -C and -T options to the ar utility.		30 November 2001	&status.new;	&status.blank;
Investigate importing the cflow utility from the ports collection.		30 November 2001	&status.new;	&status.blank;
Add -p, -v, and -V options to the command utility (shell built-in).		30 November 2001	&status.new;	&status.blank;
Bring the cp utility up to conformance.		30 November 2001	&status.new;	&status.blank;
Implement the cxref utility.		30 November 2001	&status.new;	&status.blank;
Add the -d, -h, and -M options and fix conflicting -i option in the file utility.		30 November 2001	&status.new;	&status.blank;
Bring the make utility up to conformance.		30 November 2001	&status.new;	&status.blank;
Add -p option to the more utility.		30 November 2001	&status.new;	&status.blank;
Add the qsub utility.		30 November 2001	&status.new;	&status.blank;
Make printf(1) utility conform to SUSv3.		23 April 2002	&status.new;	&status.blank;
Implement the fuser utility. (PR: 36076)		4 February 2003	&status.new;	&status.blank;

Task	Responsible	Last Updated	Status
Task	Responsible	Last Updated	5.0	4.x
Wide Character/String Support
Implement basic support for wide character I/O: getwc(), fgetwc(), getwchar(), putwc(), fputwc(), putwchar(), ungetwc(), fwide().	&a.tjr;	16 August 2002	&status.done;	&status.n-a;
Add stubs for reentrant wc handling functions btowc(), mbrlen(), mbrtowc(), mbsrtowcs(), wcrtomb(), wcsrtombs(), wctob(), wctype().	&a.tjr;	23 August 2002	&status.done;	&status.n-a;
Add stubs for reentrant wc handling functions iswalnum(), iswalpha(), iswcntrl(), iswctype(), iswdigit(), iswgraph(), iswlower(), iswprint(), iswpunct(), iswspace(), iswupper(), iswxdigit().	&a.tjr;	23 August 2002	&status.done;	&status.n-a;
Implement wcwidth().	&a.keichii;	23 August 2002	&status.done;	&status.n-a;
Implement wcswidth().	&a.ache;	23 August 2002	&status.done;	&status.n-a;
Implement wcstok().	&a.tjr;	9 September 2002	&status.done;	&status.n-a;
Implement wcstod(), wcstol(), wcstoul().	&a.tjr;	15 September 2002	&status.done;	&status.n-a;
Implement wcsftime().	&a.tjr;	15 September 2002	&status.done;	&status.n-a;
Implement fwprintf(), swprintf(), vfwprintf(), vswprintf(), vwprintf(), wprintf().	&a.tjr;	23 September 2002	&status.done;	&status.n-a;
Implement fwscanf(), swscanf(), vfwscanf(), vswscanf(), vwscanf(), wscanf().	&a.tjr;	23 September 2002	&status.done;	&status.n-a;
Implement wcstoimax(), wcstoll(), wcstoumax(), wcstoull().	&a.tjr;	23 September 2002	&status.done;	&status.n-a;
Implement wide character format types (%C, %S, %lc, %ls) in &man.printf.3;.	&a.tjr;	16 October 2002	&status.done;	&status.n-a;
Implement wcscoll(), wcsxfrm().	&a.tjr;	7 June 2004	&status.done;	&status.n-a;
Implement wide character format types (%C, %S, %lc, %ls) in &man.scanf.3;.	&a.tjr;	7 June 2004	&status.done;	&status.n-a;

Legend
&status.new;	Not yet started
&status.blocked;	Blocked awaiting completion of another task
&status.wip;	Work in progress
&status.done;	Task completed
&status.n-a;	Not planned

The remainder of this page is structured as a reverse-chronological log.

31 December 2002

Jens Schweikhardt has updated his POSIX Utility Conformance in FreeBSD list. Not much works remains to complete utility conformance.

13 December 2002

A new status report on the state of the project, covering September through October, has been produced.

9 October 2002

A new status report on the state of the project, covering July through August, has been produced.

16 August 2002

A new status report on the state of the project, covering May through June, has been produced.

1 August 2002

The Open Group has generously donated several copies of the Authorized Guide to Single &unix; Specification, Version 3, books and CD-ROMs to the project. We are greatly appreciative of their gift.

22 May 2002

A new status report on the state of the project, covering February through April, has been produced.

3 March 2002

A new status report on the state of the project, covering December and January, has been produced.

30 January 2002

The Open Group has announced the completion of SUSv3 and POSIX.1-2001 and the availability of a free, online version of the specification.

14 December 2001

A new status report on the current state of the project has been produced.

6 December 2001

The IEEE Standards Board has approved the Austin Group Specification as IEEE Std 1003.1.

28 November 2001

The project mailing list has been moved from freebsd-standards@bostonradio.org to &a.standards;.

19 October 2001

A status report on the current state of the project has been produced.

10 October 2001

This web page was created.

&footer; diff --git a/en/projects/dingo/index.sgml b/en/projects/dingo/index.sgml index 93acf4e07c..b416b38482 100644 --- a/en/projects/dingo/index.sgml +++ b/en/projects/dingo/index.sgml @@ -1,262 +1,262 @@ - + %includes; N/A"> Done"> Done in CURRENT"> Prototyped"> Merged to HEAD; RELENG_5 candidate"> New task"> Unknown"> %developers; ]> &header;

Project Goal
Project Tasks

Project Goal

The Dingo project is a collection of work that needs to be done to clean up and advance the network stack. The overriding goal is to remove duplicated functionality while also adding new features that will make FreeBSD easier to use both for the network engineer and experimenter and for the first time user.

The Dingo project is closely related to the Netperf Project which is removing all the Giant locks from the various network stacks.

Project Tasks

- + - +

Task	Responsible	Last updated	Status	Notes
ARP rewrite, layer 2 separation from FIB code		20041128	&status.new;	luigi started but didn't finish. Mail Message Problem Report
Add IP_SENDIF. Get SO_BINDTODEVICE for next to free. fenner likes this		20041128	&status.new;
Update TCP-MD5 support in FreeBSD.		20041128	&status.new;
Refactor PF_ROUTE as a tag-length-value based ABI		20041128	&status.new;
Rework code in FreeBSD's ip_icmp.c such that ICMP responses for forwarding can be throttled also. Call badport_bandlim() before icmp_error()?		20041128	&status.new;
Take M_PROMISC from NetBSD.		20041128	&status.new;
Refactor wi(4) to implement IFF_ALLMULTI in PPROMISC using M_PROMISC		20041128	&status.new;	Refactor wi(4) to implement IFF_ALLMULTI in PPROMISC using M_PROMISC and if_ethersubr or 802.11 layer handling to throw out the promisc stuff which isn't needed. Or on any hardware without IFF_ALLMULTI
Make sure mlaier's multiple inaddr on same subnet change is A-OK		20041212	&status.done;	A bug with handling netmasked aliases was discovered, and a patch has been committed.
Import CARP from OpenBSD	mlaier	20041203	&status.prototyped;	Patches are available for testing. May need coordination with the ARP rewrite.
Get pff(4) house in order wrt M_PROMISC		20041128	&status.new;
Make an(4) radiotap-aware. 80% DONE! Depends on task below		20041128	&status.new;
Make an(4) net80211-aware (bringin NetBSD diffs from onoe).	Make an(4) net80211-aware (bring in NetBSD diffs from onoe).		20041128	&status.new;
Bringin busdma vr(4) from netbsd	Bring in busdma vr(4) from netbsd		20041128	&status.new;
Add radiotap support to acx(4).		20041128	&status.new;	In touch with darron about the firmware redistribution issue
Import Benno's software AAL5 layer for native ATM DSL, bug him about it		20041128	&status.new;
Port howl's autoipd and nifd to FreeBSD --> bsd-licensed full zeroconf...	Andrea Campi andrea+freebsd at webcom dot it	20041212	&status.prototyped;
KAME tunnel code needs to be brought in line with the tunnel mib.		20041128	&status.new;	(As in, if_type should be IFT_TUNNEL). This can probably be done for IFT_GIF but need review from ume/kame people.
Take sam's new ifconfig from p4		20041208	&status.current;	Done in HEAD, not going to be MFCed.
We need to add features to gre(4). Use dev.* sysctls.		20041128	&status.new;
Write a plugin for Sam's ifconfig		20041128	&status.new;	RFC1701, RFC2784 Checksum+Offset Sequence number RFC1701 Key (for multipoint tunnel support) Other Path MTU Discovery
Investigate policy/classful routing as an option for FreeBSD.		20041128	&status.new;
Nuke struct arpcom		20041203	&status.new;	partially related to ARP rewrite
Generic mechanism to attach per-subsystem data to interfaces		20041203	&status.new;	needed to clean up ng_fec and remove ac_netgraph from struct arpcom.
Remove struct ifnet from softc.	brooks	20050617	&status.current;	Done in HEAD, not going to be MFCed.
Protocol Test Harness.	George V. Neville-Neil gnn at freebsd dot org	20041212	&status.new;	My plan is to write a Python program and a set of C language extensions for the easy creation of packet level tests. The input language will be controlled by a DTD the input language will be a set of packets and other structures specified via XML.

&footer; diff --git a/en/projects/netperf/index.sgml b/en/projects/netperf/index.sgml index 6f923f46f5..073d7ec3a6 100644 --- a/en/projects/netperf/index.sgml +++ b/en/projects/netperf/index.sgml @@ -1,315 +1,315 @@ - + %includes; N/A"> Done"> Prototyped"> Merged to HEAD; RELENG_5 candidate"> New task"> Unknown"> %developers; ]> &header;

Project Goal
Project Strategies
Project Tasks
Netperf Cluster
Links

Project Goal

The netperf project is working to enhance the performance of the FreeBSD network stack. This work grew out of the SMPng Project, which moved the FreeBSD kernel from a "Giant Lock" to more fine-grained locking and multi-threading. SMPng offered both performance improvement and degradation for the network stack, improving parallelism and preemption, but substantially increasing per-packet processing costs. The netperf project is primarily focussed on further improving parallelism in network processing while reducing the SMP synchronization overhead. This in turn will lead to higher processing throughput and lower processing latency.

Project Strategies

Robert Watson

The two primary focuses of this work are to increase parallelism while decreasing overhead. Several activities are being performed that will work toward these goals:

Complete locking work to make sure all components of the stack are able to run without the Giant lock. While most of the network stack, especially mainstream protocols, runs without Giant, some components require Giant to be placed back over the stack if compiled into the kernel, reducing parallelism.
Optimize locking strategies to find better balances between locking granularity and locking overhead. In the first cut at locking for the kernel, the goal was to adopt a medium-grained locking approach based on data locking. This approach identifies critical data structures, and inserts new locks and locking operations to protect those data structures. Depending on the data model of the code being protected, this may lead to the introduction of a substantial number of locks offering unnecessary granularity, where the overhead of locking overwhelms the benefits of available parallelism and preemption. By selectively reducing granularity, it is possible to improve performance by decreasing locking overhead.
Amortize the cost of locking by processing queues of packets or events. While the cost of individual synchronization operations may be high, it is possible to amortize the cost of synchronization operations by grouping processing of similar data (packets, events) under the same protection. This approach focuses on identifying places where similar locking occurs frequently in succession, and introducing queueing or coalescing of lock operations across the body of the work. For example, when a series of packets is inserted into an outgoing interface queue, a basic locking approach would lock the queue for each insert operation, unlock it, and hand off to the interface driver to begin the send, repeating this sequence as required. With a coalesced approach, the caller would pass off a queue of packets in order to reduce the locking overhead, as well as eliminate unnecessary synchronization due to the queue being thread-local. This approach can be applied at several levels in the stack, and is particularly applicable at lower levels of the stack where streams of packets require almost identical processing.
Introduce new synchronization strategies with reduced overhead relative to traditional strategies. Most traditional strategies employ a combination of interrupt disabling and atomic operations to achieve mutual exclusion and non-preemption guarantees. However, these operations are expensive on modern CPUs, leading to the desire for cheaper primitives with weaker semantics. For example, the application of uni-processor primitives where synchronization is required only on a single processor, and optimizations to critical section primitives to avoid the need for interrupt disabling.
Modify synchronization strategies to take advantage of additional, non-locking, synchronization primitives. This approach might take the form of making increased use of per-CPU or per-thread data structures, which require little or no synchronization. For example, through the use of critical sections, it is possible to synchronize access to per-CPU caches and queues. Through the use of per-thread queues, data can be handed off between stack layers without the use of synchronization.
Increase the opportunities for parallelism through increased threading in the network stack. The current network stack model offers the opportunity for substantial parallelism, with outbound processing typically taking place in the context of the sending - thread in kernel, crypto occuring in crypto worker threads, and + thread in kernel, crypto occurring in crypto worker threads, and receive processing taking place in a combination of the receiving ithread and dispatched netisr thread. While handoffs between threads introduces overhead (synchronization, context switching), there is the opportunity to increase parallelism in some workloads through introducing additional worker threads. Identifying work that may be relocated to new threads must be done carefully to balance overhead, and latency concerns, but can pay off by increasing effective CPU utilization and hence throughput. For example, introducing additional netisr threads capable of running on more than one CPU at a time can increase input parallelism, subject to maintaining desirable packet ordering.

Project Tasks

Task	Responsible	Last updated	Status	Notes
Prefer file descriptor reference counts to socket reference counts for system calls.	&a.rwatson;	20041124	&status.done;	Sockets and file descriptors both have reference counts in order to prevent these objects from being free'd while in use. However, if a file descriptor is used to reach the socket, the reference counts are somewhat interchangeable, as either will prevent undesired garbage collection. For socket system calls, overhead can be reduced by relying on the file descriptor reference count, thus avoiding the synchronized operations necessary to modify the socket reference count, an approach also taken in the VFS code. This change has been made for most socket system calls, and has been committed to HEAD (6.x). It has also been merged to RELENG_5 for inclusion in 5.4.
Mbuf queue library	&a.rwatson;	20041124	&status.prototyped;	In order to facilitate passing off queues of packets between network stack components, create an mbuf queue primitive, struct mbufqueue. The initial implementation is complete, and the primitive is now being applied in several sample cases to determine whether it offers the desired semantics and benefits. The implementation can be found in the rwatson_dispatch Perforce branch. Additional work must also be done to explore the performance impact of "queues" vs arrays of mbuf pointers, which are likely to behave better from a caching perspective.
Employ queued dispatch in interface send API	&a.rwatson;	20041106	&status.prototyped;	An experimental if_start_mbufqueue() interface to struct ifnet has been added, which passes an mbuf queue to the device driver for processing, avoiding redundant synchronization against the interface queue, even in the event that additional queueing is required. This has not yet been benchmarked. A subset change to dispatch a single mbuf to a driver has also been prototyped, and - bencharked at a several percentage point improvement in packet send + benchmarked at a several percentage point improvement in packet send rates from user space.
Employ queued dispatch in the interface receive API	&a.rwatson;	20041106	&status.new;	Similar to if_start_mbufqueue, allow input of a queue of mbufs from the device driver into the lowest protocol layers, such as ether_input_mbufqueue.
Employ queued dispatch across netisr dispatch API	&a.rwatson;	20041124	&status.prototyped;	Pull all of the mbufs in the netisr ifqueue out of the ifqueue into a thread-local mbuf queue to avoid repeated lock operations to access the queue. Also use lock-free operations to test for queue contents being present. This has been prototyped in the rwatson_netperf branch.
Modify UMA allocator to use critical sections not mutexes for per-CPU caches.	&a.rwatson;	20050429	&status.done;	The mutexes protecting per-CPU caches require atomic operations on SMP systems; as they are per-CPU objects, the cost of synchronizing access to the caches can be reduced by combining CPU pinning and/or critical sections instead. This change has now been committed and will appear in 6.0-RELEASE; it results in a several percentage performance in UDP send from user space, and there have been reports of 20%+ improvements in allocation intensive code within the kernel. In micro-benchmarks, the cost of allocation on SMP is dramatically reduced.
Modify malloc(9) allocator to use per-CPU statistics with critical sections to protect malloc_type statistics rather than global statistics with a mutex.	&a.rwatson;	20050529	&status.done;	Previously, malloc(9) used a single statistics structure protected by a mutex to hold global malloc statistics for each malloc type. This change moves to per-CPU statistics structures, which are coalesced when reporting memory allocation statistics to the user, and protects them using critical sections. This reduces cache line contention for common allocation types by avoiding shared lines, and also reduces synchronization costs by using critical sections to synchronize access instead of a mutex. While malloc(9) is less frequently used in the network stack than uma(9), it is used for socket address data, so is on performance critical paths for datagram operations. This has been committed and will appear in 6.0-RELEASE.
Optimize critical section performance	&a.jhb;	20050404	&status.done;	Critical sections prevent preemption of a thread on a CPU, as well as preventing migration of that thread to another CPU, and maybe used for synchronizing access to per-CPU data structures, as well as preventing recursion in interrupt processing. Currently, critical sections disable interrupts on the CPU. In previous versions of FreeBSD (4.x and before), optimizations were present that allowed for software interrupt disabling, which lowers the cost of critical sections in the common case by avoiding expensive microcode operations on the CPU. By restoring this model, or a variation on it, critical sections can be made substantially cheaper to enter. In particular, this change will lower the cost of critical sections on UP such that it is approximately the same cost as a mutex, meaning that optimizations on SMP to use critical sections instead of mutexes will not harm UP performance. This change has now been committed, and will appear in 6.0-RELEASE.

Netperf Cluster

Through the generous donations and investment of Sentex Data Communications, FreeBSD Systems, IronPort Systems, and the FreeBSD Foundation, a network performance testbed has been created in Ontario, Canada for use by FreeBSD developers working in the area of network performance. A similar cluster, made possible through the generous donation of Verio, is being prepared for use in more general SMP performance work in Virginia, US. Each cluster consists of several SMP systems inter-connected with giga-bit ethernet such that relatively arbitrary topologies can be constructed in order to test host-host, IP forwarding, and bridging performance scenarios. Systems are network booted, have serial console, and remote power, in order to maximize availability and minimize configuration overhead. These systems are available on a check-out basis for experimentation and performance measurement to FreeBSD developers working on the Netperf project, and in related areas.

More detailed information on the netperf cluster can be found by following this link.

Contents

Project Goal

Project Tasks

Contents

Project Goal

Project Strategies

Project Tasks

Netperf Cluster

Links