- formatting is ugly
hints are easy to find on the web, e.g.
http://stackoverflow.com/questions/1713048/
see also matthew@ Fri, 18 Jul 2014 19:25:12 -0700
loc * exist * algo ** size * imp ***
+- In -man -Thtml, .nf does not preserve indentation.
+ It should either convert blanks to
+ or use
rather than
(like .Bd -literal does). + Reported by afresh1@ 12 Apr 2016 14:35:45 -0700 + - .Bf at the beginning of a paragraph inserts a bogus 1ex horizontal space, see for example random(3). Introduced in http://mdocml.bsd.lv/cgi-bin/cvsweb/mdoc_html.c.diff?r1=1.91&r2=1.92 reported by deraadt@ Mon, 28 Sep 2015 20:14:13 -0600 (MDT) loc ** exist ** algo ** size * imp * - jsg on icb, Nov 3, 2014: try to guess Xr in man(7) for hyperlinking - The tables used to render the three-part page headers actually force the width of the to the max-width given for . Not yet sure how to fix that... Observed by an Anonymous Coward on undeadly.org: http://undeadly.org/cgi?action=article&sid=20140925064244&pid=1 loc * exist * algo ** size * imp *** - consider whether can be used for Ar Dv Er Ev Fa Va. from bentley@ Wed, 13 Aug 2014 09:17:55 -0600 - generatetags in HTML idea from florian@ Tue, 7 Apr 2015 00:26:28 +0000 may be possible to implement with .Lk img://something.png alt_text - check https://github.com/trentm/mdocml ************************************************************************ * formatting issues: gratuitous differences ************************************************************************ - .Fn reopens a new scope after punctuation in mandoc, but closes its scope for good in groff. Do we want to change mandoc or groff? Steffen Nurpmeso Sat, 08 Nov 2014 13:34:59 +0100 loc * exist ** algo ** size * imp ** - In .Bl -enum -width 0n, groff continues one the same line after the number, mandoc breaks the line. mail to kristaps@ Mon, 20 Jul 2009 02:21:39 +0200 loc * exist ** algo ** size * imp ** - .Pp between two .It in .Bl -column should produce one, not two blank lines, see e.g. login.conf(5). reported by jmc@ Sun, 17 Apr 2011 14:04:58 +0059 reported again by sthen@ Wed, 18 Jan 2012 02:09:39 +0000 (UTC) loc * exist *** algo ** size * imp ** - If the *first* line after .It is .Pp, break the line right after the tag, do not pad with space characters before breaking. See the description of the a, c, and i commands in sed(1). loc * exist ** algo ** size * imp ** - If the first line after .It is .D1, do not assert a blank line in between, see for example tmux(1). reported by nicm@ 13 Jan 2011 00:18:57 +0000 loc * exist ** algo ** size * imp ** - Trailing punctuation after .It should trigger EOS spacing. reported by Nicolas Joly Sat, 17 Nov 2012 11:49:54 +0100 Probably, this should be fixed somewhere in termp_it_pre(), not sure. loc * exist ** algo ** size * imp ** - .Nx 1.0a should be "NetBSD 1.0A", not "NetBSD 1.0a", see OpenBSD ccdconfig(8). loc * exist * algo * size * imp ** - In .Bl -tag, if a tag exceeds the right margin and must be continued on the next line, it must be indented by -width, not width+1; see "rule block|pass" in OpenBSD ifconfig(8). loc * exist *** algo ** size * imp ** - When the -width string contains macros, the macros must be rendered before measuring the width, for example .Bl -tag -width ".Dv message" in magic(5), located in src/usr.bin/file, is the same as -width 7n, not -width 11n. The same applies to .Bl -column column widths; reported again by Nicolas Joly Thu, 1 Mar 2012 13:41:26 +0100 via wiz@ 5 Mar reported again by Franco Fichtner Fri, 27 Sep 2013 21:02:28 +0200 loc *** exist *** algo *** size ** imp *** An easy partial fix would be to just skip the first word if it starts with a dot, including any following white space, when measuring. loc * exist * algo * size * imp *** - The \& zero-width character counts as output. That is, when it is alone on a line between two .Pp, we want three blank lines, not two as in mandoc. loc ** exist ** algo ** size * imp ** - Header lines of excessive length: Port OpenBSD man_term.c rev. 1.25 to mdoc_term.c and document it in mdoc(7) and man(7) COMPATIBILITY found while talking to Chris Bennett loc * exist * algo * size * imp * - Sequences of multiple man(7) paragraphs (.PP, .IP) interspersed with .ps and .nf/.fi produce execessive blank lines, see libJudy and graphics/dcmtk. The parser reorg may help with this. - trailing whitespace must be ignored even when followed by a font escape, see for example makes \fBdig \fR operate in batch mode in dig(1). loc ** exist ** algo ** size * imp ** ************************************************************************ * portability ************************************************************************ - systems having UTF-8 but not en_US.UTF-8 call locale(1) from ./configure, select a UTF-8-locale, and use that for test-wchar.c and term_ascii.c to Markus Waldeck Sat, 18 Jul 2015 01:55:37 +0200 loc * exist * algo * size * imp * ************************************************************************ * warning issues ************************************************************************ - provide a way in mandoc(1) to warn about broken .Xr links; probably cannot be on by default in -Tlint because it needs to access the manpath and mandoc.db(3) after parsing. asked for by jmc@ Fri, 4 Dec 2015 22:39:40 +0000 - Report errors in -O suboption parsing. loc * exist * algo * size * imp ** - warn when .Sh or .Ss contain other macros Steffen Nurpmeso, savannah.gnu.org/bugs/index.php?45034 loc * exist * algo * size * imp ** - check that MANDOCERR_BADTAB is thrown in the right cases, i.e. when finding a literal tab character in fill mode, and possibly change the wording of the warning message to refer to fill mode, not literal mode See the mail from Werner LEMBERG on the groff list, Fri, 14 Feb 2014 18:54:42 +0100 (CET) loc * exist ** algo ** size * imp ** - warn about attempts to call non-callable macros Steffen Nurpmeso Tue, 11 Nov 2014 22:55:16 +0100 Note that formatting is inconsistent in groff. .Fn Po prints "Po()", .Ar Sh prints "file ..." and no "Sh". Relatively hard because the relevant code is scattered all over mdoc_macro.c and all subtly different. loc ** exist ** algo ** size ** imp ** - warn about "new sentence, new line" loc ** exist ** algo *** size * imp ** - mandoc_special does not really check the escape sequence, but just the overall format loc ** exist ** algo *** size ** imp ** - integrate mdoclint into mandoc ("end-of-line whitespace" thread) from jmc@ Mon, 13 Jul 2009 17:12:09 +0100 from kristaps@ Mon, 13 Jul 2009 18:34:53 +0200 from jmc@ Mon, 13 Jul 2009 17:45:37 +0059 from kristaps@ Mon, 13 Jul 2009 19:02:03 +0200 (mostly done, check what remains) - -Tlint parser errors and warnings to stdout to tech@mdocml, naddy@ Wed, 28 Sep 2011 11:21:46 +0200 wait! kristaps@ Sun, 02 Oct 2011 17:12:52 +0200 -- for system errors, use errno/strerror/warn/err - ************************************************************************ * documentation issues ************************************************************************ - mention hyphenation rules: breaking at letter-letter in text mode (not macro args) proper hyphenation is unimplemented - talk about spacing around delimiters to jmc@, kristaps@ Sat, 23 Apr 2011 17:41:27 +0200 - mark macros as: page structure domain, manual domain, general text domain is this useful? - mention /usr/share/misc/mdoc.template in mdoc(7)? - Is all the content from http://www.std.com/obi/BSD/doc/usd/28.tbl/tbl covered in tbl(7)? ************************************************************************ * performance issues ************************************************************************ - Why are we using MAP_SHARED, not MAP_PRIVATE for mmap(2)? How does SQLITE_CONFIG_PAGECACHE actually work? Document it! from kristaps@ Sat, 09 Aug 2014 13:51:36 +0200 Several areas can be cleaned up to make mandoc even faster. These are - improve hashing mechanism for macros (quite important: performance) - improve hashing mechanism for characters (not as important) - the PDF file is HUGE: this can be reduced by using relative offsets - instead of re-initialising the roff predefined-strings set before each parse, create a read-only version the first time and copy it loc * exist ** algo ** size * imp ** ************************************************************************ * structural issues ************************************************************************ - -- Use libz directly instead of forking gunzip(1). - Suggested by bapt at FreeBSD among others. - We use the input line number at several places to distinguish same-line from different-line input. That plainly doesn't work with user-defined macros, leading to random breakage. - Find better ways to prevent endless loops in roff(7) macro and string expansion. - Finish cleanup of date handling. Decide which formats should be recognized where. Update both mdoc(7) and man(7) documentation. Triggered by Tim van der Molen Tue, 22 Feb 2011 20:30:45 +0100 - struct mparse refactoring Steffen Nurpmeso Thu, 04 Sep 2014 12:50:00 +0200 - Consider creating some views that will make the database more readable from the sqlite3 shell. Consider using them to abstract from the database structure, too. suggested by espie@ Sat, 19 Apr 2014 14:52:57 +0200 ************************************************************************ * CGI issues ************************************************************************ - Enable HTTP compression by detecting gzip encoding and filtering output through libz. - Sandbox (see OpenSSH). - Enable caching support via HTTP 304 and If-Modified-Since. - Allow for cgi.h to be overridden by CGI environment variables. Otherwise, binary distributions will inherit the compile-time behaviour, which is not optimal. - Have Mac OSX systems automatically disable -static compilation of the CGI: -static isn't supported. ************************************************************************ * to improve in the groff_mdoc(7) macros ************************************************************************ - use uname(1) to set doc-default-operating-system at install time tobimensch Mon, 1 Dec 2014 00:25:07 +0100 Index: vendor/mdocml/dist/cgi.c =================================================================== --- vendor/mdocml/dist/cgi.c (revision 303220) +++ vendor/mdocml/dist/cgi.c (revision 303221) @@ -1,1129 +1,1186 @@ -/* $Id: cgi.c,v 1.116 2016/01/04 12:36:26 schwarze Exp $ */ +/* $Id: cgi.c,v 1.135 2016/07/11 22:48:37 schwarze Exp $ */ /* * Copyright (c) 2011, 2012 Kristaps Dzonsons
#include #include +#include #include #include #include #include #include #include #include #include #include "mandoc_aux.h" #include "mandoc.h" #include "roff.h" #include "mdoc.h" #include "man.h" #include "main.h" #include "manconf.h" #include "mansearch.h" #include "cgi.h" /* * A query as passed to the search function. */ struct query { char *manpath; /* desired manual directory */ char *arch; /* architecture */ char *sec; /* manual section */ char *query; /* unparsed query expression */ int equal; /* match whole names, not substrings */ }; struct req { struct query q; char **p; /* array of available manpaths */ size_t psz; /* number of available manpaths */ + int isquery; /* QUERY_STRING used, not PATH_INFO */ }; -static void catman(const struct req *, const char *); -static void format(const struct req *, const char *); +enum focus { + FOCUS_NONE = 0, + FOCUS_QUERY +}; + static void html_print(const char *); static void html_putchar(char); static int http_decode(char *); -static void http_parse(struct req *, const char *); -static void pathgen(struct req *); +static void parse_manpath_conf(struct req *); +static void parse_path_info(struct req *req, const char *path); +static void parse_query_string(struct req *, const char *); static void pg_error_badrequest(const char *); static void pg_error_internal(void); static void pg_index(const struct req *); static void pg_noresult(const struct req *, const char *); static void pg_search(const struct req *); static void pg_searchres(const struct req *, struct manpage *, size_t); static void pg_show(struct req *, const char *); static void resp_begin_html(int, const char *); static void resp_begin_http(int, const char *); +static void resp_catman(const struct req *, const char *); static void resp_copy(const char *); static void resp_end_html(void); -static void resp_searchform(const struct req *); +static void resp_format(const struct req *, const char *); +static void resp_searchform(const struct req *, enum focus); static void resp_show(const struct req *, const char *); static void set_query_attr(char **, char **); static int validate_filename(const char *); static int validate_manpath(const struct req *, const char *); static int validate_urifrag(const char *); -static const char *scriptname; /* CGI script name */ +static const char *scriptname = SCRIPT_NAME; static const int sec_prios[] = {1, 4, 5, 8, 6, 3, 7, 2, 9}; static const char *const sec_numbers[] = { "0", "1", "2", "3", "3p", "4", "5", "6", "7", "8", "9" }; static const char *const sec_names[] = { "All Sections", "1 - General Commands", "2 - System Calls", "3 - Library Functions", "3p - Perl Library", "4 - Device Drivers", "5 - File Formats", "6 - Games", "7 - Miscellaneous Information", "8 - System Manager\'s Manual", "9 - Kernel Developer\'s Manual" }; static const int sec_MAX = sizeof(sec_names) / sizeof(char *); static const char *const arch_names[] = { "amd64", "alpha", "armish", "armv7", - "aviion", "hppa", "hppa64", "i386", - "ia64", "landisk", "loongson", "luna88k", - "macppc", "mips64", "octeon", "sgi", - "socppc", "solbourne", "sparc", "sparc64", - "vax", "zaurus", + "hppa", "hppa64", "i386", "landisk", + "loongson", "luna88k", "macppc", "mips64", + "octeon", "sgi", "socppc", "sparc", + "sparc64", "zaurus", "amiga", "arc", "arm32", "atari", - "beagle", "cats", "hp300", "mac68k", - "mvme68k", "mvme88k", "mvmeppc", "palm", - "pc532", "pegasos", "pmax", "powerpc", - "sun3", "wgrisc", "x68k" + "aviion", "beagle", "cats", "hp300", + "ia64", "mac68k", "mvme68k", "mvme88k", + "mvmeppc", "palm", "pc532", "pegasos", + "pmax", "powerpc", "solbourne", "sun3", + "vax", "wgrisc", "x68k" }; static const int arch_MAX = sizeof(arch_names) / sizeof(char *); /* * Print a character, escaping HTML along the way. * This will pass non-ASCII straight to output: be warned! */ static void html_putchar(char c) { switch (c) { case ('"'): printf(""e;"); break; case ('&'): printf("&"); break; case ('>'): printf(">"); break; case ('<'): printf("<"); break; default: putchar((unsigned char)c); break; } } /* * Call through to html_putchar(). * Accepts NULL strings. */ static void html_print(const char *p) { if (NULL == p) return; while ('\0' != *p) html_putchar(*p++); } /* * Transfer the responsibility for the allocated string *val * to the query structure. */ static void set_query_attr(char **attr, char **val) { free(*attr); if (**val == '\0') { *attr = NULL; free(*val); } else *attr = *val; *val = NULL; } /* * Parse the QUERY_STRING for key-value pairs * and store the values into the query structure. */ static void -http_parse(struct req *req, const char *qs) +parse_query_string(struct req *req, const char *qs) { char *key, *val; size_t keysz, valsz; + req->isquery = 1; req->q.manpath = NULL; req->q.arch = NULL; req->q.sec = NULL; req->q.query = NULL; req->q.equal = 1; key = val = NULL; while (*qs != '\0') { /* Parse one key. */ keysz = strcspn(qs, "=;&"); key = mandoc_strndup(qs, keysz); qs += keysz; if (*qs != '=') goto next; /* Parse one value. */ valsz = strcspn(++qs, ";&"); val = mandoc_strndup(qs, valsz); qs += valsz; /* Decode and catch encoding errors. */ if ( ! (http_decode(key) && http_decode(val))) goto next; /* Handle key-value pairs. */ if ( ! strcmp(key, "query")) set_query_attr(&req->q.query, &val); else if ( ! strcmp(key, "apropos")) req->q.equal = !strcmp(val, "0"); else if ( ! strcmp(key, "manpath")) { #ifdef COMPAT_OLDURI if ( ! strncmp(val, "OpenBSD ", 8)) { val[7] = '-'; if ('C' == val[8]) val[8] = 'c'; } #endif set_query_attr(&req->q.manpath, &val); } else if ( ! (strcmp(key, "sec") #ifdef COMPAT_OLDURI && strcmp(key, "sektion") #endif )) { if ( ! strcmp(val, "0")) *val = '\0'; set_query_attr(&req->q.sec, &val); } else if ( ! strcmp(key, "arch")) { if ( ! strcmp(val, "default")) *val = '\0'; set_query_attr(&req->q.arch, &val); } /* * The key must be freed in any case. * The val may have been handed over to the query * structure, in which case it is now NULL. */ next: free(key); key = NULL; free(val); val = NULL; if (*qs != '\0') qs++; } } /* * HTTP-decode a string. The standard explanation is that this turns * "%4e+foo" into "n foo" in the regular way. This is done in-place * over the allocated string. */ static int http_decode(char *p) { char hex[3]; char *q; int c; hex[2] = '\0'; q = p; for ( ; '\0' != *p; p++, q++) { if ('%' == *p) { if ('\0' == (hex[0] = *(p + 1))) return 0; if ('\0' == (hex[1] = *(p + 2))) return 0; if (1 != sscanf(hex, "%x", &c)) return 0; if ('\0' == c) return 0; *q = (char)c; p += 2; } else *q = '+' == *p ? ' ' : *p; } *q = '\0'; return 1; } static void resp_begin_http(int code, const char *msg) { if (200 != code) printf("Status: %d %s\r\n", code, msg); printf("Content-Type: text/html; charset=utf-8\r\n" "Cache-Control: no-cache\r\n" "Pragma: no-cache\r\n" "\r\n"); fflush(stdout); } static void resp_copy(const char *filename) { char buf[4096]; ssize_t sz; int fd; if ((fd = open(filename, O_RDONLY)) != -1) { fflush(stdout); while ((sz = read(fd, buf, sizeof(buf))) > 0) write(STDOUT_FILENO, buf, sz); } } static void resp_begin_html(int code, const char *msg) { resp_begin_http(code, msg); printf("\n" - "\n" - "\n" - "\n" - "\n" - " %s \n" - "\n" - "\n" + "\n" + "\n" + "\n" + "\n" + "%s \n" + "\n" + "\n" "\n", CSS_DIR, CUSTOMIZE_TITLE); resp_copy(MAN_DIR "/header.html"); } static void resp_end_html(void) { resp_copy(MAN_DIR "/footer.html"); - puts("\n" - ""); + puts("\n" + ""); } static void -resp_searchform(const struct req *req) +resp_searchform(const struct req *req, enum focus focus) { int i; puts(""); - printf("\n" - "\n" + ""); puts(""); } static int validate_urifrag(const char *frag) { while ('\0' != *frag) { if ( ! (isalnum((unsigned char)*frag) || '-' == *frag || '.' == *frag || '/' == *frag || '_' == *frag)) return 0; frag++; } return 1; } static int validate_manpath(const struct req *req, const char* manpath) { size_t i; - if ( ! strcmp(manpath, "mandoc")) - return 1; - for (i = 0; i < req->psz; i++) if ( ! strcmp(manpath, req->p[i])) return 1; return 0; } static int validate_filename(const char *file) { if ('.' == file[0] && '/' == file[1]) file += 2; return ! (strstr(file, "../") || strstr(file, "/..") || (strncmp(file, "man", 3) && strncmp(file, "cat", 3))); } static void pg_index(const struct req *req) { resp_begin_html(200, NULL); - resp_searchform(req); - printf("\n" + resp_searchform(req, FOCUS_QUERY); + printf("
\n" "This web interface is documented in the\n" - "man.cgi\n" + "man.cgi(8)\n" "manual, and the\n" - "apropos\n" + "apropos(1)\n" "manual explains the query syntax.\n" - "
\n", - scriptname, scriptname); + "\n", + scriptname, *scriptname == '\0' ? "" : "/", + scriptname, *scriptname == '\0' ? "" : "/"); resp_end_html(); } static void pg_noresult(const struct req *req, const char *msg) { resp_begin_html(200, NULL); - resp_searchform(req); - puts(""); + resp_searchform(req, FOCUS_QUERY); + puts("
"); puts(msg); - puts("
"); + puts(""); resp_end_html(); } static void pg_error_badrequest(const char *msg) { resp_begin_html(400, "Bad Request"); - puts("Bad Request
\n" - "\n"); + puts("
Bad Request
\n" + "\n"); puts(msg); printf("Try again from the\n" - "main page.\n" - "
", scriptname); + "main page.\n" + "", scriptname); resp_end_html(); } static void pg_error_internal(void) { resp_begin_html(500, "Internal Server Error"); - puts("Internal Server Error
"); + puts("Internal Server Error
"); resp_end_html(); } static void pg_searchres(const struct req *req, struct manpage *r, size_t sz) { char *arch, *archend; - size_t i, iuse, isec; + const char *sec; + size_t i, iuse; int archprio, archpriouse; int prio, priouse; - char sec; for (i = 0; i < sz; i++) { if (validate_filename(r[i].file)) continue; - fprintf(stderr, "invalid filename %s in %s database\n", + warnx("invalid filename %s in %s database", r[i].file, req->q.manpath); pg_error_internal(); return; } - if (1 == sz) { + if (req->isquery && sz == 1) { /* * If we have just one result, then jump there now * without any delay. */ printf("Status: 303 See Other\r\n"); - printf("Location: http://%s%s/%s/%s", - HTTP_HOST, scriptname, req->q.manpath, r[0].file); + printf("Location: http://%s/%s%s%s/%s", + HTTP_HOST, scriptname, + *scriptname == '\0' ? "" : "/", + req->q.manpath, r[0].file); printf("\r\n" "Content-Type: text/html; charset=utf-8\r\n" "\r\n"); return; } resp_begin_html(200, NULL); - resp_searchform(req); - puts(""); - puts(""); + } + /* * In man(1) mode, show one of the pages * even if more than one is found. */ - if (req->q.equal) { - puts(""); + resp_searchform(req, + req->q.equal || sz == 1 ? FOCUS_NONE : FOCUS_QUERY); - for (i = 0; i < sz; i++) { - printf("
\n" + "\n" - " "); - } + if (sz > 1) { + puts("\n" - "q.manpath, r[i].file); - printf("\">"); - html_print(r[i].names); - printf("\n" - " \n" - ""); - html_print(r[i].output); - puts(" \n" - ""); + puts(""); + for (i = 0; i < sz; i++) { + printf(""); - puts("
\n" - "\n" + " "); + } + puts("\n" + "q.manpath, r[i].file); + printf("\">"); + html_print(r[i].names); + printf("\n" + " \n" + ""); + html_print(r[i].output); + puts(" \n" + "
"); + if (req->q.equal || sz == 1) { + puts("
"); iuse = 0; - priouse = 10; + priouse = 20; archpriouse = 3; for (i = 0; i < sz; i++) { - isec = strcspn(r[i].file, "123456789"); - sec = r[i].file[isec]; - if ('\0' == sec) + sec = r[i].file; + sec += strcspn(sec, "123456789"); + if (sec[0] == '\0') continue; - prio = sec_prios[sec - '1']; - if (NULL == req->q.arch) { + prio = sec_prios[sec[0] - '1']; + if (sec[1] != '/') + prio += 10; + if (req->q.arch == NULL) { archprio = - (NULL == (arch = strchr( - r[i].file + isec, '/'))) ? 3 : - (NULL == (archend = strchr( - arch + 1, '/'))) ? 0 : + ((arch = strchr(sec + 1, '/')) + == NULL) ? 3 : + ((archend = strchr(arch + 1, '/')) + == NULL) ? 0 : strncmp(arch, "amd64/", archend - arch) ? 2 : 1; if (archprio < archpriouse) { archpriouse = archprio; priouse = prio; iuse = i; continue; } if (archprio > archpriouse) continue; } if (prio >= priouse) continue; priouse = prio; iuse = i; } resp_show(req, r[iuse].file); } resp_end_html(); } static void -catman(const struct req *req, const char *file) +resp_catman(const struct req *req, const char *file) { FILE *f; char *p; size_t sz; ssize_t len; int i; int italic, bold; if ((f = fopen(file, "r")) == NULL) { - puts("You specified an invalid manual file.
"); + puts("You specified an invalid manual file.
"); return; } - puts("\n" - ""); fclose(f); } static void -format(const struct req *req, const char *file) +resp_format(const struct req *req, const char *file) { struct manoutput conf; struct mparse *mp; struct roff_man *man; void *vp; int fd; int usepath; if (-1 == (fd = open(file, O_RDONLY, 0))) { - puts(""); + puts("\n" + "\n" + ""); + puts(""); p = NULL; sz = 0; while ((len = getline(&p, &sz, f)) != -1) { bold = italic = 0; for (i = 0; i < len - 1; i++) { /* * This means that the catpage is out of state. * Ignore it and keep going (although the * catpage is bogus). */ if ('\b' == p[i] || '\n' == p[i]) continue; /* * Print a regular character. * Close out any bold/italic scopes. * If we're in back-space mode, make sure we'll * have something to enter when we backspace. */ if ('\b' != p[i + 1]) { if (italic) - printf(""); + printf(""); if (bold) - printf(""); + printf(""); italic = bold = 0; html_putchar(p[i]); continue; } else if (i + 2 >= len) continue; /* Italic mode. */ if ('_' == p[i]) { if (bold) - printf(""); + printf(""); if ( ! italic) - printf(""); + printf(""); bold = 0; italic = 1; i += 2; html_putchar(p[i]); continue; } /* * Handle funny behaviour troff-isms. * These grok'd from the original man2html.c. */ if (('+' == p[i] && 'o' == p[i + 2]) || ('o' == p[i] && '+' == p[i + 2]) || ('|' == p[i] && '=' == p[i + 2]) || ('=' == p[i] && '|' == p[i + 2]) || ('*' == p[i] && '=' == p[i + 2]) || ('=' == p[i] && '*' == p[i + 2]) || ('*' == p[i] && '|' == p[i + 2]) || ('|' == p[i] && '*' == p[i + 2])) { if (italic) - printf(""); + printf(""); if (bold) - printf(""); + printf(""); italic = bold = 0; putchar('*'); i += 2; continue; } else if (('|' == p[i] && '-' == p[i + 2]) || ('-' == p[i] && '|' == p[i + 1]) || ('+' == p[i] && '-' == p[i + 1]) || ('-' == p[i] && '+' == p[i + 1]) || ('+' == p[i] && '|' == p[i + 1]) || ('|' == p[i] && '+' == p[i + 1])) { if (italic) - printf(""); + printf(""); if (bold) - printf(""); + printf(""); italic = bold = 0; putchar('+'); i += 2; continue; } /* Bold mode. */ if (italic) - printf(""); + printf(""); if ( ! bold) - printf(""); + printf(""); bold = 1; italic = 0; i += 2; html_putchar(p[i]); } /* * Clean up the last character. * We can get to a newline; don't print that. */ if (italic) - printf(""); + printf(""); if (bold) - printf(""); + printf(""); if (i == len - 1 && p[i] != '\n') html_putchar(p[i]); putchar('\n'); } free(p); - puts("\n" - "You specified an invalid manual file.
"); + puts("You specified an invalid manual file.
"); return; } mchars_alloc(); mp = mparse_alloc(MPARSE_SO, MANDOCLEVEL_BADARG, NULL, req->q.manpath); mparse_readfd(mp, fd, file); close(fd); memset(&conf, 0, sizeof(conf)); conf.fragment = 1; usepath = strcmp(req->q.manpath, req->p[0]); - mandoc_asprintf(&conf.man, "%s?query=%%N&sec=%%S%s%s%s%s", - scriptname, - req->q.arch ? "&arch=" : "", - req->q.arch ? req->q.arch : "", - usepath ? "&manpath=" : "", - usepath ? req->q.manpath : ""); + mandoc_asprintf(&conf.man, "/%s%s%%N.%%S", + usepath ? req->q.manpath : "", usepath ? "/" : ""); mparse_result(mp, &man, NULL); if (man == NULL) { - fprintf(stderr, "fatal mandoc error: %s/%s\n", - req->q.manpath, file); + warnx("fatal mandoc error: %s/%s", req->q.manpath, file); pg_error_internal(); mparse_free(mp); mchars_free(); return; } vp = html_alloc(&conf); if (man->macroset == MACROSET_MDOC) { mdoc_validate(man); html_mdoc(vp, man); } else { man_validate(man); html_man(vp, man); } html_free(vp); mparse_free(mp); mchars_free(); free(conf.man); } static void resp_show(const struct req *req, const char *file) { if ('.' == file[0] && '/' == file[1]) file += 2; if ('c' == *file) - catman(req, file); + resp_catman(req, file); else - format(req, file); + resp_format(req, file); } static void pg_show(struct req *req, const char *fullpath) { char *manpath; const char *file; if ((file = strchr(fullpath, '/')) == NULL) { pg_error_badrequest( "You did not specify a page to show."); return; } manpath = mandoc_strndup(fullpath, file - fullpath); file++; if ( ! validate_manpath(req, manpath)) { pg_error_badrequest( "You specified an invalid manpath."); free(manpath); return; } /* * Begin by chdir()ing into the manpath. * This way we can pick up the database files, which are * relative to the manpath root. */ if (chdir(manpath) == -1) { - fprintf(stderr, "chdir %s: %s\n", - manpath, strerror(errno)); + warn("chdir %s", manpath); pg_error_internal(); free(manpath); return; } + free(manpath); - if (strcmp(manpath, "mandoc")) { - free(req->q.manpath); - req->q.manpath = manpath; - } else - free(manpath); - if ( ! validate_filename(file)) { pg_error_badrequest( "You specified an invalid manual file."); return; } resp_begin_html(200, NULL); - resp_searchform(req); + resp_searchform(req, FOCUS_NONE); resp_show(req, file); resp_end_html(); } static void pg_search(const struct req *req) { struct mansearch search; struct manpaths paths; struct manpage *res; char **argv; char *query, *rp, *wp; size_t ressz; int argc; /* * Begin by chdir()ing into the root of the manpath. * This way we can pick up the database files, which are * relative to the manpath root. */ - if (-1 == (chdir(req->q.manpath))) { - fprintf(stderr, "chdir %s: %s\n", - req->q.manpath, strerror(errno)); + if (chdir(req->q.manpath) == -1) { + warn("chdir %s", req->q.manpath); pg_error_internal(); return; } search.arch = req->q.arch; search.sec = req->q.sec; search.outkey = "Nd"; search.argmode = req->q.equal ? ARG_NAME : ARG_EXPR; search.firstmatch = 1; paths.sz = 1; paths.paths = mandoc_malloc(sizeof(char *)); paths.paths[0] = mandoc_strdup("."); /* * Break apart at spaces with backslash-escaping. */ argc = 0; argv = NULL; rp = query = mandoc_strdup(req->q.query); for (;;) { while (isspace((unsigned char)*rp)) rp++; if (*rp == '\0') break; argv = mandoc_reallocarray(argv, argc + 1, sizeof(char *)); argv[argc++] = wp = rp; for (;;) { if (isspace((unsigned char)*rp)) { *wp = '\0'; rp++; break; } if (rp[0] == '\\' && rp[1] != '\0') rp++; if (wp != rp) *wp = *rp; if (*rp == '\0') break; wp++; rp++; } } if (0 == mansearch(&search, &paths, argc, argv, &res, &ressz)) pg_noresult(req, "You entered an invalid query."); else if (0 == ressz) pg_noresult(req, "No results found."); else pg_searchres(req, res, ressz); free(query); mansearch_free(res, ressz); free(paths.paths[0]); free(paths.paths); } int main(void) { struct req req; struct itimerval itimer; const char *path; const char *querystring; int i; /* Poor man's ReDoS mitigation. */ itimer.it_value.tv_sec = 2; itimer.it_value.tv_usec = 0; itimer.it_interval.tv_sec = 2; itimer.it_interval.tv_usec = 0; if (setitimer(ITIMER_VIRTUAL, &itimer, NULL) == -1) { - fprintf(stderr, "setitimer: %s\n", strerror(errno)); + warn("setitimer"); pg_error_internal(); return EXIT_FAILURE; } - /* Scan our run-time environment. */ - - if (NULL == (scriptname = getenv("SCRIPT_NAME"))) - scriptname = ""; - - if ( ! validate_urifrag(scriptname)) { - fprintf(stderr, "unsafe SCRIPT_NAME \"%s\"\n", - scriptname); - pg_error_internal(); - return EXIT_FAILURE; - } - /* * First we change directory into the MAN_DIR so that * subsequent scanning for manpath directories is rooted * relative to the same position. */ - if (-1 == chdir(MAN_DIR)) { - fprintf(stderr, "MAN_DIR: %s: %s\n", - MAN_DIR, strerror(errno)); + if (chdir(MAN_DIR) == -1) { + warn("MAN_DIR: %s", MAN_DIR); pg_error_internal(); return EXIT_FAILURE; } memset(&req, 0, sizeof(struct req)); - pathgen(&req); + req.q.equal = 1; + parse_manpath_conf(&req); - /* Next parse out the query string. */ + /* Parse the path info and the query string. */ - if (NULL != (querystring = getenv("QUERY_STRING"))) - http_parse(&req, querystring); + if ((path = getenv("PATH_INFO")) == NULL) + path = ""; + else if (*path == '/') + path++; + if (*path != '\0') { + parse_path_info(&req, path); + if (req.q.manpath == NULL || access(path, F_OK) == -1) + path = ""; + } else if ((querystring = getenv("QUERY_STRING")) != NULL) + parse_query_string(&req, querystring); + + /* Validate parsed data and add defaults. */ + if (req.q.manpath == NULL) req.q.manpath = mandoc_strdup(req.p[0]); else if ( ! validate_manpath(&req, req.q.manpath)) { pg_error_badrequest( "You specified an invalid manpath."); return EXIT_FAILURE; } if ( ! (NULL == req.q.arch || validate_urifrag(req.q.arch))) { pg_error_badrequest( "You specified an invalid architecture."); return EXIT_FAILURE; } /* Dispatch to the three different pages. */ - path = getenv("PATH_INFO"); - if (NULL == path) - path = ""; - else if ('/' == *path) - path++; - if ('\0' != *path) pg_show(&req, path); else if (NULL != req.q.query) pg_search(&req); else pg_index(&req); free(req.q.manpath); free(req.q.arch); free(req.q.sec); free(req.q.query); for (i = 0; i < (int)req.psz; i++) free(req.p[i]); free(req.p); return EXIT_SUCCESS; } /* + * If PATH_INFO is not a file name, translate it to a query. + */ +static void +parse_path_info(struct req *req, const char *path) +{ + char *dir[4]; + int i; + + req->isquery = 0; + req->q.equal = 1; + req->q.manpath = mandoc_strdup(path); + req->q.arch = NULL; + + /* Mandatory manual page name. */ + if ((req->q.query = strrchr(req->q.manpath, '/')) == NULL) { + req->q.query = req->q.manpath; + req->q.manpath = NULL; + } else + *req->q.query++ = '\0'; + + /* Optional trailing section. */ + if ((req->q.sec = strrchr(req->q.query, '.')) != NULL) { + if(isdigit((unsigned char)req->q.sec[1])) { + *req->q.sec++ = '\0'; + req->q.sec = mandoc_strdup(req->q.sec); + } else + req->q.sec = NULL; + } + + /* Handle the case of name[.section] only. */ + if (req->q.manpath == NULL) + return; + req->q.query = mandoc_strdup(req->q.query); + + /* Split directory components. */ + dir[i = 0] = req->q.manpath; + while ((dir[i + 1] = strchr(dir[i], '/')) != NULL) { + if (++i == 3) { + pg_error_badrequest( + "You specified too many directory components."); + exit(EXIT_FAILURE); + } + *dir[i]++ = '\0'; + } + + /* Optional manpath. */ + if ((i = validate_manpath(req, req->q.manpath)) == 0) + req->q.manpath = NULL; + else if (dir[1] == NULL) + return; + + /* Optional section. */ + if (strncmp(dir[i], "man", 3) == 0) { + free(req->q.sec); + req->q.sec = mandoc_strdup(dir[i++] + 3); + } + if (dir[i] == NULL) { + if (req->q.manpath == NULL) + free(dir[0]); + return; + } + if (dir[i + 1] != NULL) { + pg_error_badrequest( + "You specified an invalid directory component."); + exit(EXIT_FAILURE); + } + + /* Optional architecture. */ + if (i) { + req->q.arch = mandoc_strdup(dir[i]); + if (req->q.manpath == NULL) + free(dir[0]); + } else + req->q.arch = dir[0]; +} + +/* * Scan for indexable paths. */ static void -pathgen(struct req *req) +parse_manpath_conf(struct req *req) { FILE *fp; char *dp; size_t dpsz; ssize_t len; - if (NULL == (fp = fopen("manpath.conf", "r"))) { - fprintf(stderr, "%s/manpath.conf: %s\n", - MAN_DIR, strerror(errno)); + if ((fp = fopen("manpath.conf", "r")) == NULL) { + warn("%s/manpath.conf", MAN_DIR); pg_error_internal(); exit(EXIT_FAILURE); } dp = NULL; dpsz = 0; while ((len = getline(&dp, &dpsz, fp)) != -1) { if (dp[len - 1] == '\n') dp[--len] = '\0'; req->p = mandoc_realloc(req->p, (req->psz + 1) * sizeof(char *)); if ( ! validate_urifrag(dp)) { - fprintf(stderr, "%s/manpath.conf contains " - "unsafe path \"%s\"\n", MAN_DIR, dp); + warnx("%s/manpath.conf contains " + "unsafe path \"%s\"", MAN_DIR, dp); pg_error_internal(); exit(EXIT_FAILURE); } - if (NULL != strchr(dp, '/')) { - fprintf(stderr, "%s/manpath.conf contains " - "path with slash \"%s\"\n", MAN_DIR, dp); + if (strchr(dp, '/') != NULL) { + warnx("%s/manpath.conf contains " + "path with slash \"%s\"", MAN_DIR, dp); pg_error_internal(); exit(EXIT_FAILURE); } req->p[req->psz++] = dp; dp = NULL; dpsz = 0; } free(dp); - if ( req->p == NULL ) { - fprintf(stderr, "%s/manpath.conf is empty\n", MAN_DIR); + if (req->p == NULL) { + warnx("%s/manpath.conf is empty", MAN_DIR); pg_error_internal(); exit(EXIT_FAILURE); } } Index: vendor/mdocml/dist/cgi.h.example =================================================================== --- vendor/mdocml/dist/cgi.h.example (revision 303220) +++ vendor/mdocml/dist/cgi.h.example (revision 303221) @@ -1,7 +1,8 @@ /* Example compile-time configuration file for man.cgi(8). */ #define HTTP_HOST "mdocml.bsd.lv" -#define MAN_DIR "/var/www/man" +#define SCRIPT_NAME "cgi-bin/man.cgi" +#define MAN_DIR "/man" #define CSS_DIR "" #define CUSTOMIZE_TITLE "Manual pages with mandoc" #define COMPAT_OLDURI Yes Index: vendor/mdocml/dist/configure =================================================================== --- vendor/mdocml/dist/configure (revision 303220) +++ vendor/mdocml/dist/configure (revision 303221) @@ -1,464 +1,470 @@ #!/bin/sh # -# Copyright (c) 2014, 2015 Ingo Schwarze" [ ${HAVE_VASPRINTF} -eq 0 ] && echo "#include " [ ${HAVE_GETLINE} -eq 0 ] && echo "#include " echo echo "#define MAN_CONF_FILE \"/etc/${MANM_MANCONF}\"" echo "#define MANPATH_DEFAULT \"${MANPATH_DEFAULT}\"" [ -n "${OSNAME}" ] && echo "#define OSNAME \"${OSNAME}\"" [ -n "${HOMEBREWDIR}" ] && echo "#define HOMEBREWDIR \"${HOMEBREWDIR}\"" cat << __HEREDOC__ #define HAVE_DIRENT_NAMLEN ${HAVE_DIRENT_NAMLEN} #define HAVE_ERR ${HAVE_ERR} #define HAVE_FTS ${HAVE_FTS} #define HAVE_GETLINE ${HAVE_GETLINE} #define HAVE_GETSUBOPT ${HAVE_GETSUBOPT} #define HAVE_ISBLANK ${HAVE_ISBLANK} #define HAVE_MKDTEMP ${HAVE_MKDTEMP} #define HAVE_MMAP ${HAVE_MMAP} #define HAVE_PLEDGE ${HAVE_PLEDGE} #define HAVE_PROGNAME ${HAVE_PROGNAME} #define HAVE_REALLOCARRAY ${HAVE_REALLOCARRAY} #define HAVE_REWB_BSD ${HAVE_REWB_BSD} #define HAVE_REWB_SYSV ${HAVE_REWB_SYSV} +#define HAVE_SANDBOX_INIT ${HAVE_SANDBOX_INIT} #define HAVE_STRCASESTR ${HAVE_STRCASESTR} #define HAVE_STRINGLIST ${HAVE_STRINGLIST} #define HAVE_STRLCAT ${HAVE_STRLCAT} #define HAVE_STRLCPY ${HAVE_STRLCPY} #define HAVE_STRPTIME ${HAVE_STRPTIME} #define HAVE_STRSEP ${HAVE_STRSEP} #define HAVE_STRTONUM ${HAVE_STRTONUM} #define HAVE_VASPRINTF ${HAVE_VASPRINTF} #define HAVE_WCHAR ${HAVE_WCHAR} #define HAVE_SQLITE3 ${HAVE_SQLITE3} #define HAVE_SQLITE3_ERRSTR ${HAVE_SQLITE3_ERRSTR} #define HAVE_OHASH ${HAVE_OHASH} #define HAVE_MANPATH ${HAVE_MANPATH} #define BINM_APROPOS "${BINM_APROPOS}" #define BINM_MAKEWHATIS "${BINM_MAKEWHATIS}" #define BINM_MAN "${BINM_MAN}" #define BINM_SOELIM "${BINM_SOELIM}" #define BINM_WHATIS "${BINM_WHATIS}" __HEREDOC__ if [ ${HAVE_ERR} -eq 0 ]; then echo "extern void err(int, const char *, ...);" echo "extern void errx(int, const char *, ...);" echo "extern void warn(const char *, ...);" echo "extern void warnx(const char *, ...);" fi [ ${HAVE_GETLINE} -eq 0 ] && \ echo "extern ssize_t getline(char **, size_t *, FILE *);" [ ${HAVE_GETSUBOPT} -eq 0 ] && \ echo "extern int getsubopt(char **, char * const *, char **);" [ ${HAVE_ISBLANK} -eq 0 ] && \ echo "extern int isblank(int);" [ ${HAVE_MKDTEMP} -eq 0 ] && \ echo "extern char *mkdtemp(char *);" if [ ${HAVE_PROGNAME} -eq 0 ]; then echo "extern const char *getprogname(void);" echo "extern void setprogname(const char *);" fi [ ${HAVE_REALLOCARRAY} -eq 0 ] && \ echo "extern void *reallocarray(void *, size_t, size_t);" [ ${BUILD_DB} -gt 0 -a ${HAVE_SQLITE3_ERRSTR} -eq 0 ] && echo "extern const char *sqlite3_errstr(int);" [ ${HAVE_STRCASESTR} -eq 0 ] && \ echo "extern char *strcasestr(const char *, const char *);" [ ${HAVE_STRLCAT} -eq 0 ] && \ echo "extern size_t strlcat(char *, const char *, size_t);" [ ${HAVE_STRLCPY} -eq 0 ] && \ echo "extern size_t strlcpy(char *, const char *, size_t);" [ ${HAVE_STRSEP} -eq 0 ] && \ echo "extern char *strsep(char **, const char *);" [ ${HAVE_STRTONUM} -eq 0 ] && \ echo "extern long long strtonum(const char *, long long, long long, const char **);" [ ${HAVE_VASPRINTF} -eq 0 ] && \ echo "extern int vasprintf(char **, const char *, va_list);" echo echo "#endif /* MANDOC_CONFIG_H */" echo "config.h: written" 1>&2 echo "config.h: written" 1>&3 # --- tests for Makefile.local ----------------------------------------- exec > Makefile.local [ -z "${BINDIR}" ] && BINDIR="${PREFIX}/bin" [ -z "${SBINDIR}" ] && SBINDIR="${PREFIX}/sbin" [ -z "${INCLUDEDIR}" ] && INCLUDEDIR="${PREFIX}/include/mandoc" [ -z "${LIBDIR}" ] && LIBDIR="${PREFIX}/lib/mandoc" [ -z "${MANDIR}" ] && MANDIR="${PREFIX}/man" [ -z "${HTDOCDIR}" ] && HTDOCDIR="${WWWPREFIX}/htdocs" [ -z "${CGIBINDIR}" ] && CGIBINDIR="${WWWPREFIX}/cgi-bin" [ -z "${INSTALL_PROGRAM}" ] && INSTALL_PROGRAM="${INSTALL} -m 0555" [ -z "${INSTALL_LIB}" ] && INSTALL_LIB="${INSTALL} -m 0444" [ -z "${INSTALL_MAN}" ] && INSTALL_MAN="${INSTALL} -m 0444" [ -z "${INSTALL_DATA}" ] && INSTALL_DATA="${INSTALL} -m 0444" if [ ${BUILD_DB} -eq 0 -a ${BUILD_CGI} -gt 0 ]; then echo "BUILD_CGI=0 (no BUILD_DB)" 1>&2 echo "BUILD_CGI=0 (no BUILD_DB)" 1>&3 BUILD_CGI=0 fi BUILD_TARGETS="base-build" [ ${BUILD_CGI} -gt 0 ] && BUILD_TARGETS="${BUILD_TARGETS} cgi-build" INSTALL_TARGETS="base-install" [ ${BUILD_DB} -gt 0 ] && INSTALL_TARGETS="${INSTALL_TARGETS} db-install" [ ${BUILD_CGI} -gt 0 ] && INSTALL_TARGETS="${INSTALL_TARGETS} cgi-install" cat << __HEREDOC__ BUILD_TARGETS = ${BUILD_TARGETS} INSTALL_TARGETS = ${INSTALL_TARGETS} CC = ${CC} CFLAGS = ${CFLAGS} -DBLIB = ${DBLIB} +LDADD = ${LDADD} +LDFLAGS = ${LDFLAGS} STATIC = ${STATIC} PREFIX = ${PREFIX} BINDIR = ${BINDIR} SBINDIR = ${SBINDIR} INCLUDEDIR = ${INCLUDEDIR} LIBDIR = ${LIBDIR} MANDIR = ${MANDIR} WWWPREFIX = ${WWWPREFIX} HTDOCDIR = ${HTDOCDIR} CGIBINDIR = ${CGIBINDIR} BINM_APROPOS = ${BINM_APROPOS} BINM_MAKEWHATIS = ${BINM_MAKEWHATIS} BINM_MAN = ${BINM_MAN} BINM_SOELIM = ${BINM_SOELIM} BINM_WHATIS = ${BINM_WHATIS} MANM_MAN = ${MANM_MAN} MANM_MANCONF = ${MANM_MANCONF} MANM_MDOC = ${MANM_MDOC} MANM_ROFF = ${MANM_ROFF} MANM_EQN = ${MANM_EQN} MANM_TBL = ${MANM_TBL} INSTALL = ${INSTALL} INSTALL_PROGRAM = ${INSTALL_PROGRAM} INSTALL_LIB = ${INSTALL_LIB} INSTALL_MAN = ${INSTALL_MAN} INSTALL_DATA = ${INSTALL_DATA} __HEREDOC__ [ ${BUILD_DB} -gt 0 ] && \ echo "MAIN_OBJS = \$(BASE_OBJS) \$(DB_OBJS)" echo "Makefile.local: written" 1>&2 echo "Makefile.local: written" 1>&3 exit 0 Index: vendor/mdocml/dist/configure.local.example =================================================================== --- vendor/mdocml/dist/configure.local.example (revision 303220) +++ vendor/mdocml/dist/configure.local.example (revision 303221) @@ -1,245 +1,281 @@ -# $Id: configure.local.example,v 1.10 2015/11/07 13:14:21 schwarze Exp $ +# $Id: configure.local.example,v 1.13 2016/07/14 11:09:06 schwarze Exp $ # -# Copyright (c) 2014, 2015 Ingo Schwarze # providing setlocale(3) and providing wcwidth(3) and # putwchar(3) with a wchar_t storing UCS-4 values. Theoretically, # the latter should be tested with the __STDC_ISO_10646__ feature # macro. In practice, many headers do not provide that # macro even though they treat wchar_t as UCS-4. So the automatic # test only checks that wchar_t is wide enough, that is, at least # four bytes. # The following line forces multi-byte support. # If your C library does not treat wchar_t as UCS-4, the UTF-8 output # mode will print garbage. HAVE_WCHAR=1 # The following line disables multi-byte support. # The output modes -Tutf8 and -Tlocale will be the same as -Tascii. HAVE_WCHAR=0 # When man(1) or apropos(1) is called without -m and -M options, # MANPATH is not set in the environment, man.conf(5) is not available # and manpath(1) not used, manuals are searched for in the following # directory trees by default. MANPATH_DEFAULT="/usr/share/man:/usr/X11R6/man:/usr/local/man" # In manual pages written in the mdoc(7) language, the operating system # version is displayed in the page footer line. If an operating system # is specified as an argument to the .Os macro, that is always used. # If the .Os macro has no argument and an operation system is specified # with the mandoc(1) -Ios= command line option, that is used. # Otherwise, the uname(3) library function is called at runtime to find # the name of the operating system. # If you do not want uname(3) to be called but instead want a fixed # string to be used, use the following line: -OSNAME="OpenBSD 5.6" +OSNAME="OpenBSD 5.9" # The following installation directories are used. # It is possible to set only one or a few of these variables, # there is no need to copy the whole block. # Even if you set PREFIX to something else, the other variables # pick it up without copying them all over. PREFIX="/usr/local" BINDIR="${PREFIX}/bin" SBINDIR="${PREFIX}/sbin" INCLUDEDIR="${PREFIX}/include/mandoc" LIBDIR="${PREFIX}/lib/mandoc" MANDIR="${PREFIX}/man" # The man(1) utility needs to know where the manuals reside. # We know of two ways to tell it: via manpath(1) or man.conf(5). # The latter is used by OpenBSD and NetBSD, the former by most # other systems. # Force usage of manpath(1). # If it is not installed or not operational, # man(1), makewhatis(8), and apropos(1) will not work properly. HAVE_MANPATH=1 # Force usage of man.conf(5). # If it does not exist or contains no valid configuration, # man(1), makewhatis(8), and apropos(1) will not work properly. HAVE_MANPATH=0 # Some distributions may want to avoid naming conflicts # with the configuration files of other man(1) implementations. # This changes the name of the installed section 5 manual page as well. MANM_MANCONF="mandoc.conf" # default is "man.conf" # Some distributions may want to avoid naming conflicts among manuals. # If you want to change the names of installed section 7 manual pages, # the following alternative names are suggested. # The suffix ".7" will automatically be appended. # It is possible to set only one or a few of these variables, # there is no need to copy the whole block. MANM_MAN="mandoc_man" # default is "man" MANM_MDOC="mandoc_mdoc" # default is "mdoc" MANM_ROFF="mandoc_roff" # default is "roff" MANM_EQN="mandoc_eqn" # default is "eqn" MANM_TBL="mandoc_tbl" # default is "tbl" # Some distributions may want to avoid naming conflicts # with other man(1) and soelim(1) utilities. # If you want to change the names of binary programs, # the following alternative names are suggested. # Using different names is possible as well. # This changes the names of the installed section 1 manual pages as well. BINM_MAN=mman # default is "man" BINM_SOELIM=msoelim # default is "soelim" +# Before falling back to the bundled version of the ohash(3) hashing +# library, autoconfiguration tries the following linker flag to +# link against your system version. If you do have ohash(3) on +# your system but it needs different linker flags, set the following +# variable to specify the required linker flags. + +LD_OHASH="-lutil" + +# Some platforms may need additional linker flags to link against libmandoc +# that are not autodetected. +# For example, Solaris 9 and 10 need -lrt for nanosleep(2). + +LDADD="-lrt" + +# Some systems may want to set additional linker flags for all the +# binaries, not only for those using libmandoc, for example for +# hardening options. + +LDFLAGS="-Wl,-z,relro" + # It is possible to change the utility program used for installation # and the modes files are installed with. The defaults are: INSTALL="install" INSTALL_PROGRAM="${INSTALL} -m 0555" INSTALL_LIB="${INSTALL} -m 0444" INSTALL_MAN="${INSTALL} -m 0444" INSTALL_DATA="${INSTALL} -m 0444" # --- user settings related to database support ------------------------ # By default, building makewhatis(8) and apropos(1) is enabled. # To disable it, for example to avoid the dependency on SQLite3, # use the following line. It that case, the remaining settings # in this section are irrelevant. BUILD_DB=0 -# Two libraries are needed: SQLite3 and ohash(3). -# Autoconfiguration tries the following linker flags to find them. -# If none of these work, add a working DBLIB line to configure.local, -# disabling autodetection for library directories. +# Autoconfiguration tries the following linker flags to find the +# SQLite3 library installed on your system. If none of these work, +# set the following variable to specify the required linker flags. -DBLIB="-lsqlite3" -DBLIB="-lsqlite3 -lutil" -DBLIB="-L/usr/local/lib -lsqlite3" +LD_SQLITE3="-lsqlite3" +LD_SQLITE3="-L/usr/local/lib -lsqlite3" # When library autodetection decides to use -L/usr/local/lib, # -I/usr/local/include is automatically added to CFLAGS. -# If you manually set DBLIB to something including -L/usr/local/lib, +# If you manually set LD_SQLITE3 to something including -L/usr/local/lib, # chances are you will also need the following line: CFLAGS="${CFLAGS} -I/usr/local/include" # Some distributions may want to avoid naming conflicts # with another implementation of apropos(1) and makewhatis(8). # If you want to change the names of the binary programs, # the following alternative names are suggested. # Using other names is possible as well. # This changes the names of the installed section 1 and section 8 # manual pages as well. # It is possible to set only one or two of these variables, # there is no need to copy the whole block. BINM_APROPOS=mapropos # default is "apropos" BINM_WHATIS=mwhatis # default is "whatis" BINM_MAKEWHATIS=mandocdb # default is "makewhatis" # When using the "homebrew" package manager on Mac OS X, the actual # manuals are located in a so-called "cellar" and only symlinked # into the manual trees. To allow mandoc to follow such symlinks, # you have to specify the physical location of the cellar as returned # by realpath(3), for example: PREFIX="/usr/local" HOMEBREWDIR="${PREFIX}/Cellar" # --- user settings related man.cgi ------------------------------------ # By default, building man.cgi(8) is disabled. To enable it, copy # cgi.h.example to cgi.h, edit it, and use the following line. # Obviously, this requires that BUILD_DB is enabled, too. BUILD_CGI=1 # The remaining settings in this section are only relevant if BUILD_CGI # is enabled. Otherwise, they have no effect either way. # By default, man.cgi(8) is linked statically. # Some systems do not support static linking, for example Mac OS X. # In that case, use the following line: STATIC= # Some systems, for example Linux, require -pthread for static linking: STATIC="-static -pthread" # Some directories. # This works just like PREFIX, see above. WWWPREFIX="/var/www" HTDOCDIR="${WWWPREFIX}/htdocs" CGIBINDIR="${WWWPREFIX}/cgi-bin" # --- settings that rarely need to be touched -------------------------- # Do not set these variables unless you really need to. # You can manually override the compiler to be used. # But that's rarely useful because ./configure asks your make(1) # which compiler to use, and that answer will hardly be wrong. CC=cc +# IBM AIX may need: + +CC=xlc + # The default compiler flags are: CFLAGS="-g -W -Wall -Wstrict-prototypes -Wno-unused-parameter -Wwrite-strings" +# IBM AIX xlc does not support -W; in that case, please use: + +CFLAGS="-g" + # In rare cases, it may be required to skip individual automatic tests. # Each of the following variables can be set to 0 (test will not be run # and will be regarded as failed) or 1 (test will not be run and will # be regarded as successful). HAVE_DIRENT_NAMLEN=0 -HAVE_FGETLN=0 +HAVE_ERR=0 HAVE_FTS=0 +HAVE_GETLINE=0 HAVE_GETSUBOPT=0 +HAVE_ISBLANK=0 +HAVE_MKDTEMP=0 HAVE_MMAP=0 +HAVE_PLEDGE=0 +HAVE_PROGNAME=0 HAVE_REALLOCARRAY=0 +HAVE_REWB_BSD=0 +HAVE_REWB_SYSV=0 HAVE_STRCASESTR=0 +HAVE_STRINGLIST=0 HAVE_STRLCAT=0 HAVE_STRLCPY=0 HAVE_STRPTIME=0 HAVE_STRSEP=0 HAVE_STRTONUM=0 +HAVE_VASPRINTF=0 +HAVE_WCHAR=0 HAVE_SQLITE3=0 HAVE_SQLITE3_ERRSTR=0 HAVE_OHASH=0 Index: vendor/mdocml/dist/demandoc.c =================================================================== --- vendor/mdocml/dist/demandoc.c (revision 303220) +++ vendor/mdocml/dist/demandoc.c (revision 303221) @@ -1,264 +1,263 @@ -/* $Id: demandoc.c,v 1.26 2016/01/08 02:53:13 schwarze Exp $ */ +/* $Id: demandoc.c,v 1.27 2016/07/09 15:24:19 schwarze Exp $ */ /* * Copyright (c) 2011 Kristaps Dzonsons #include #include -#include #include #include #include #include #include "roff.h" #include "man.h" #include "mdoc.h" #include "mandoc.h" static void pline(int, int *, int *, int); static void pman(const struct roff_node *, int *, int *, int); static void pmandoc(struct mparse *, int, const char *, int); static void pmdoc(const struct roff_node *, int *, int *, int); static void pstring(const char *, int, int *, int); static void usage(void); static const char *progname; int main(int argc, char *argv[]) { struct mparse *mp; int ch, fd, i, list; extern int optind; if (argc < 1) progname = "demandoc"; else if ((progname = strrchr(argv[0], '/')) == NULL) progname = argv[0]; else ++progname; mp = NULL; list = 0; while (-1 != (ch = getopt(argc, argv, "ikm:pw"))) switch (ch) { case ('i'): /* FALLTHROUGH */ case ('k'): /* FALLTHROUGH */ case ('m'): /* FALLTHROUGH */ case ('p'): break; case ('w'): list = 1; break; default: usage(); return (int)MANDOCLEVEL_BADARG; } argc -= optind; argv += optind; mchars_alloc(); mp = mparse_alloc(MPARSE_SO, MANDOCLEVEL_BADARG, NULL, NULL); assert(mp); if (argc < 1) pmandoc(mp, STDIN_FILENO, " ", list); for (i = 0; i < argc; i++) { mparse_reset(mp); if ((fd = mparse_open(mp, argv[i])) == -1) { perror(argv[i]); continue; } pmandoc(mp, fd, argv[i], list); } mparse_free(mp); mchars_free(); return (int)MANDOCLEVEL_OK; } static void usage(void) { fprintf(stderr, "usage: %s [-w] [files...]\n", progname); } static void pmandoc(struct mparse *mp, int fd, const char *fn, int list) { struct roff_man *man; int line, col; mparse_readfd(mp, fd, fn); close(fd); mparse_result(mp, &man, NULL); line = 1; col = 0; if (man == NULL) return; if (man->macroset == MACROSET_MDOC) { mdoc_validate(man); pmdoc(man->first->child, &line, &col, list); } else { man_validate(man); pman(man->first->child, &line, &col, list); } if ( ! list) putchar('\n'); } /* * Strip the escapes out of a string, emitting the results. */ static void pstring(const char *p, int col, int *colp, int list) { enum mandoc_esc esc; const char *start, *end; int emit; /* * Print as many column spaces til we achieve parity with the * input document. */ again: if (list && '\0' != *p) { while (isspace((unsigned char)*p)) p++; while ('\'' == *p || '(' == *p || '"' == *p) p++; emit = isalpha((unsigned char)p[0]) && isalpha((unsigned char)p[1]); for (start = p; '\0' != *p; p++) if ('\\' == *p) { p++; esc = mandoc_escape(&p, NULL, NULL); if (ESCAPE_ERROR == esc) return; emit = 0; } else if (isspace((unsigned char)*p)) break; end = p - 1; while (end > start) if ('.' == *end || ',' == *end || '\'' == *end || '"' == *end || ')' == *end || '!' == *end || '?' == *end || ':' == *end || ';' == *end) end--; else break; if (emit && end - start >= 1) { for ( ; start <= end; start++) if (ASCII_HYPH == *start) putchar('-'); else putchar((unsigned char)*start); putchar('\n'); } if (isspace((unsigned char)*p)) goto again; return; } while (*colp < col) { putchar(' '); (*colp)++; } /* * Print the input word, skipping any special characters. */ while ('\0' != *p) if ('\\' == *p) { p++; esc = mandoc_escape(&p, NULL, NULL); if (ESCAPE_ERROR == esc) break; } else { putchar((unsigned char )*p++); (*colp)++; } } static void pline(int line, int *linep, int *col, int list) { if (list) return; /* * Print out as many lines as needed to reach parity with the * original input. */ while (*linep < line) { putchar('\n'); (*linep)++; } *col = 0; } static void pmdoc(const struct roff_node *p, int *line, int *col, int list) { for ( ; p; p = p->next) { if (MDOC_LINE & p->flags) pline(p->line, line, col, list); if (ROFFT_TEXT == p->type) pstring(p->string, p->pos, col, list); if (p->child) pmdoc(p->child, line, col, list); } } static void pman(const struct roff_node *p, int *line, int *col, int list) { for ( ; p; p = p->next) { if (MAN_LINE & p->flags) pline(p->line, line, col, list); if (ROFFT_TEXT == p->type) pstring(p->string, p->pos, col, list); if (p->child) pman(p->child, line, col, list); } } Index: vendor/mdocml/dist/libmandoc.h =================================================================== --- vendor/mdocml/dist/libmandoc.h (revision 303220) +++ vendor/mdocml/dist/libmandoc.h (revision 303221) @@ -1,85 +1,84 @@ -/* $Id: libmandoc.h,v 1.62 2015/11/07 14:01:16 schwarze Exp $ */ +/* $Id: libmandoc.h,v 1.63 2016/07/07 19:19:01 schwarze Exp $ */ /* * Copyright (c) 2009, 2010, 2011, 2012 Kristaps Dzonsons #include /* MACHINE */ #include #include #include #if HAVE_ERR #include #endif #include #include #include +#if HAVE_SANDBOX_INIT +#include +#endif #include #include #include #include #include +#include #include #include "mandoc_aux.h" #include "mandoc.h" #include "roff.h" #include "mdoc.h" #include "man.h" #include "tag.h" #include "main.h" #include "manconf.h" #include "mansearch.h" #if !defined(__GNUC__) || (__GNUC__ < 2) # if !defined(lint) # define __attribute__(x) # endif #endif /* !defined(__GNUC__) || (__GNUC__ < 2) */ enum outmode { OUTMODE_DEF = 0, OUTMODE_FLN, OUTMODE_LST, OUTMODE_ALL, OUTMODE_INT, OUTMODE_ONE }; enum outt { OUTT_ASCII = 0, /* -Tascii */ OUTT_LOCALE, /* -Tlocale */ OUTT_UTF8, /* -Tutf8 */ OUTT_TREE, /* -Ttree */ OUTT_MAN, /* -Tman */ OUTT_HTML, /* -Thtml */ OUTT_LINT, /* -Tlint */ OUTT_PS, /* -Tps */ OUTT_PDF /* -Tpdf */ }; struct curparse { struct mparse *mp; enum mandoclevel wlevel; /* ignore messages below this */ int wstop; /* stop after a file with a warning */ enum outt outtype; /* which output to use */ void *outdata; /* data for output */ struct manoutput *outopts; /* output options */ }; static int fs_lookup(const struct manpaths *, size_t ipath, const char *, const char *, const char *, struct manpage **, size_t *); static void fs_search(const struct mansearch *, const struct manpaths *, int, char**, struct manpage **, size_t *); static int koptions(int *, char *); #if HAVE_SQLITE3 int mandocdb(int, char**); #endif static int moptions(int *, char *); static void mmsg(enum mandocerr, enum mandoclevel, const char *, int, int, const char *); static void parse(struct curparse *, int, const char *); static void passthrough(const char *, int, int); static pid_t spawn_pager(struct tag_files *); static int toptions(struct curparse *, char *); static void usage(enum argmode) __attribute__((noreturn)); static int woptions(struct curparse *, char *); static const int sec_prios[] = {1, 4, 5, 8, 6, 3, 7, 2, 9}; static char help_arg[] = "help"; static char *help_argv[] = {help_arg, NULL}; static enum mandoclevel rc; int main(int argc, char *argv[]) { struct manconf conf; struct curparse curp; struct mansearch search; struct tag_files *tag_files; const char *progname; char *auxpaths; char *defos; unsigned char *uc; struct manpage *res, *resp; char *conf_file, *defpaths; - size_t isec, i, sz; + const char *sec; + size_t i, sz; int prio, best_prio; - char sec; enum outmode outmode; int fd; int show_usage; int options; int use_pager; int status, signum; int c; pid_t pager_pid, tc_pgid, man_pgid, pid; #if HAVE_PROGNAME progname = getprogname(); #else if (argc < 1) progname = mandoc_strdup("mandoc"); else if ((progname = strrchr(argv[0], '/')) == NULL) progname = argv[0]; else ++progname; setprogname(progname); #endif #if HAVE_SQLITE3 if (strncmp(progname, "mandocdb", 8) == 0 || strcmp(progname, BINM_MAKEWHATIS) == 0) return mandocdb(argc, argv); #endif #if HAVE_PLEDGE if (pledge("stdio rpath tmppath tty proc exec flock", NULL) == -1) err((int)MANDOCLEVEL_SYSERR, "pledge"); #endif +#if HAVE_SANDBOX_INIT + if (sandbox_init(kSBXProfileNoInternet, SANDBOX_NAMED, NULL) == -1) + errx((int)MANDOCLEVEL_SYSERR, "sandbox_init"); +#endif + /* Search options. */ memset(&conf, 0, sizeof(conf)); conf_file = defpaths = NULL; auxpaths = NULL; memset(&search, 0, sizeof(struct mansearch)); search.outkey = "Nd"; if (strcmp(progname, BINM_MAN) == 0) search.argmode = ARG_NAME; else if (strcmp(progname, BINM_APROPOS) == 0) search.argmode = ARG_EXPR; else if (strcmp(progname, BINM_WHATIS) == 0) search.argmode = ARG_WORD; else if (strncmp(progname, "help", 4) == 0) search.argmode = ARG_NAME; else search.argmode = ARG_FILE; /* Parser and formatter options. */ memset(&curp, 0, sizeof(struct curparse)); curp.outtype = OUTT_LOCALE; curp.wlevel = MANDOCLEVEL_BADARG; curp.outopts = &conf.output; options = MPARSE_SO | MPARSE_UTF8 | MPARSE_LATIN1; defos = NULL; use_pager = 1; tag_files = NULL; show_usage = 0; outmode = OUTMODE_DEF; while (-1 != (c = getopt(argc, argv, "aC:cfhI:iK:klM:m:O:S:s:T:VW:w"))) { switch (c) { case 'a': outmode = OUTMODE_ALL; break; case 'C': conf_file = optarg; break; case 'c': use_pager = 0; break; case 'f': search.argmode = ARG_WORD; break; case 'h': conf.output.synopsisonly = 1; use_pager = 0; outmode = OUTMODE_ALL; break; case 'I': if (strncmp(optarg, "os=", 3)) { warnx("-I %s: Bad argument", optarg); return (int)MANDOCLEVEL_BADARG; } if (defos) { warnx("-I %s: Duplicate argument", optarg); return (int)MANDOCLEVEL_BADARG; } defos = mandoc_strdup(optarg + 3); break; case 'i': outmode = OUTMODE_INT; break; case 'K': if ( ! koptions(&options, optarg)) return (int)MANDOCLEVEL_BADARG; break; case 'k': search.argmode = ARG_EXPR; break; case 'l': search.argmode = ARG_FILE; outmode = OUTMODE_ALL; break; case 'M': defpaths = optarg; break; case 'm': auxpaths = optarg; break; case 'O': search.outkey = optarg; while (optarg != NULL) manconf_output(&conf.output, strsep(&optarg, ",")); break; case 'S': search.arch = optarg; break; case 's': search.sec = optarg; break; case 'T': if ( ! toptions(&curp, optarg)) return (int)MANDOCLEVEL_BADARG; break; case 'W': if ( ! woptions(&curp, optarg)) return (int)MANDOCLEVEL_BADARG; break; case 'w': outmode = OUTMODE_FLN; break; default: show_usage = 1; break; } } if (show_usage) usage(search.argmode); /* Postprocess options. */ if (outmode == OUTMODE_DEF) { switch (search.argmode) { case ARG_FILE: outmode = OUTMODE_ALL; use_pager = 0; break; case ARG_NAME: outmode = OUTMODE_ONE; break; default: outmode = OUTMODE_LST; break; } } if (outmode == OUTMODE_FLN || outmode == OUTMODE_LST || !isatty(STDOUT_FILENO)) use_pager = 0; #if HAVE_PLEDGE if (!use_pager) if (pledge("stdio rpath flock", NULL) == -1) err((int)MANDOCLEVEL_SYSERR, "pledge"); #endif /* Parse arguments. */ if (argc > 0) { argc -= optind; argv += optind; } resp = NULL; /* * Quirks for help(1) * and for a man(1) section argument without -s. */ if (search.argmode == ARG_NAME) { if (*progname == 'h') { if (argc == 0) { argv = help_argv; argc = 1; } } else if (argc > 1 && ((uc = (unsigned char *)argv[0]) != NULL) && ((isdigit(uc[0]) && (uc[1] == '\0' || (isalpha(uc[1]) && uc[2] == '\0'))) || (uc[0] == 'n' && uc[1] == '\0'))) { search.sec = (char *)uc; argv++; argc--; } if (search.arch == NULL) search.arch = getenv("MACHINE"); #ifdef MACHINE if (search.arch == NULL) search.arch = MACHINE; #endif } rc = MANDOCLEVEL_OK; /* man(1), whatis(1), apropos(1) */ if (search.argmode != ARG_FILE) { if (argc == 0) usage(search.argmode); if (search.argmode == ARG_NAME && outmode == OUTMODE_ONE) search.firstmatch = 1; /* Access the mandoc database. */ manconf_parse(&conf, conf_file, defpaths, auxpaths); #if HAVE_SQLITE3 mansearch_setup(1); if ( ! mansearch(&search, &conf.manpath, argc, argv, &res, &sz)) usage(search.argmode); #else if (search.argmode != ARG_NAME) { fputs("mandoc: database support not compiled in\n", stderr); return (int)MANDOCLEVEL_BADARG; } sz = 0; #endif if (sz == 0) { if (search.argmode == ARG_NAME) fs_search(&search, &conf.manpath, argc, argv, &res, &sz); else warnx("nothing appropriate"); } if (sz == 0) { rc = MANDOCLEVEL_BADARG; goto out; } /* * For standard man(1) and -a output mode, * prepare for copying filename pointers * into the program parameter array. */ if (outmode == OUTMODE_ONE) { argc = 1; - best_prio = 10; + best_prio = 20; } else if (outmode == OUTMODE_ALL) argc = (int)sz; /* Iterate all matching manuals. */ resp = res; for (i = 0; i < sz; i++) { if (outmode == OUTMODE_FLN) puts(res[i].file); else if (outmode == OUTMODE_LST) printf("%s - %s\n", res[i].names, res[i].output == NULL ? "" : res[i].output); else if (outmode == OUTMODE_ONE) { /* Search for the best section. */ - isec = strcspn(res[i].file, "123456789"); - sec = res[i].file[isec]; - if ('\0' == sec) + sec = res[i].file; + sec += strcspn(sec, "123456789"); + if (sec[0] == '\0') continue; - prio = sec_prios[sec - '1']; + prio = sec_prios[sec[0] - '1']; + if (sec[1] != '/') + prio += 10; if (prio >= best_prio) continue; best_prio = prio; resp = res + i; } } /* * For man(1), -a and -i output mode, fall through * to the main mandoc(1) code iterating files * and running the parsers on each of them. */ if (outmode == OUTMODE_FLN || outmode == OUTMODE_LST) goto out; } /* mandoc(1) */ #if HAVE_PLEDGE if (use_pager) { if (pledge("stdio rpath tmppath tty proc exec", NULL) == -1) err((int)MANDOCLEVEL_SYSERR, "pledge"); } else { if (pledge("stdio rpath", NULL) == -1) err((int)MANDOCLEVEL_SYSERR, "pledge"); } #endif if (search.argmode == ARG_FILE && ! moptions(&options, auxpaths)) return (int)MANDOCLEVEL_BADARG; mchars_alloc(); curp.mp = mparse_alloc(options, curp.wlevel, mmsg, defos); /* * Conditionally start up the lookaside buffer before parsing. */ if (OUTT_MAN == curp.outtype) mparse_keep(curp.mp); if (argc < 1) { if (use_pager) tag_files = tag_init(); parse(&curp, STDIN_FILENO, " "); } while (argc > 0) { fd = mparse_open(curp.mp, resp != NULL ? resp->file : *argv); if (fd != -1) { if (use_pager) { tag_files = tag_init(); use_pager = 0; } if (resp == NULL) parse(&curp, fd, *argv); else if (resp->form & FORM_SRC) { /* For .so only; ignore failure. */ chdir(conf.manpath.paths[resp->ipath]); parse(&curp, fd, resp->file); } else passthrough(resp->file, fd, conf.output.synopsisonly); if (argc > 1 && curp.outtype <= OUTT_UTF8) - ascii_sepline(curp.outdata); + terminal_sepline(curp.outdata); } else if (rc < MANDOCLEVEL_ERROR) rc = MANDOCLEVEL_ERROR; if (MANDOCLEVEL_OK != rc && curp.wstop) break; if (resp != NULL) resp++; else argv++; if (--argc) mparse_reset(curp.mp); } if (curp.outdata != NULL) { switch (curp.outtype) { case OUTT_HTML: html_free(curp.outdata); break; case OUTT_UTF8: case OUTT_LOCALE: case OUTT_ASCII: ascii_free(curp.outdata); break; case OUTT_PDF: case OUTT_PS: pspdf_free(curp.outdata); break; default: break; } } mparse_free(curp.mp); mchars_free(); out: if (search.argmode != ARG_FILE) { manconf_free(&conf); #if HAVE_SQLITE3 mansearch_free(res, sz); mansearch_setup(0); #endif } free(defos); /* * When using a pager, finish writing both temporary files, * fork it, wait for the user to close it, and clean up. */ if (tag_files != NULL) { fclose(stdout); tag_write(); man_pgid = getpgid(0); tag_files->tcpgid = man_pgid == getpid() ? getpgid(getppid()) : man_pgid; pager_pid = 0; signum = SIGSTOP; for (;;) { /* Stop here until moved to the foreground. */ tc_pgid = tcgetpgrp(STDIN_FILENO); if (tc_pgid != man_pgid) { if (tc_pgid == pager_pid) { (void)tcsetpgrp(STDIN_FILENO, man_pgid); if (signum == SIGTTIN) continue; } else tag_files->tcpgid = tc_pgid; kill(0, signum); continue; } /* Once in the foreground, activate the pager. */ if (pager_pid) { (void)tcsetpgrp(STDIN_FILENO, pager_pid); kill(pager_pid, SIGCONT); } else pager_pid = spawn_pager(tag_files); /* Wait for the pager to stop or exit. */ while ((pid = waitpid(pager_pid, &status, WUNTRACED)) == -1 && errno == EINTR) continue; if (pid == -1) { warn("wait"); rc = MANDOCLEVEL_SYSERR; break; } if (!WIFSTOPPED(status)) break; signum = WSTOPSIG(status); } tag_unlink(); } return (int)rc; } static void usage(enum argmode argmode) { switch (argmode) { case ARG_FILE: fputs("usage: mandoc [-acfhkl] [-I os=name] " "[-K encoding] [-mformat] [-O option]\n" "\t [-T output] [-W level] [file ...]\n", stderr); break; case ARG_NAME: fputs("usage: man [-acfhklw] [-C file] [-I os=name] " "[-K encoding] [-M path] [-m path]\n" "\t [-O option=value] [-S subsection] [-s section] " "[-T output] [-W level]\n" "\t [section] name ...\n", stderr); break; case ARG_WORD: fputs("usage: whatis [-acfhklw] [-C file] " "[-M path] [-m path] [-O outkey] [-S arch]\n" "\t [-s section] name ...\n", stderr); break; case ARG_EXPR: fputs("usage: apropos [-acfhklw] [-C file] " "[-M path] [-m path] [-O outkey] [-S arch]\n" "\t [-s section] expression ...\n", stderr); break; } exit((int)MANDOCLEVEL_BADARG); } static int fs_lookup(const struct manpaths *paths, size_t ipath, const char *sec, const char *arch, const char *name, struct manpage **res, size_t *ressz) { glob_t globinfo; struct manpage *page; char *file; int form, globres; form = FORM_SRC; mandoc_asprintf(&file, "%s/man%s/%s.%s", paths->paths[ipath], sec, name, sec); if (access(file, R_OK) != -1) goto found; free(file); mandoc_asprintf(&file, "%s/cat%s/%s.0", paths->paths[ipath], sec, name); if (access(file, R_OK) != -1) { form = FORM_CAT; goto found; } free(file); if (arch != NULL) { mandoc_asprintf(&file, "%s/man%s/%s/%s.%s", paths->paths[ipath], sec, arch, name, sec); if (access(file, R_OK) != -1) goto found; free(file); } mandoc_asprintf(&file, "%s/man%s/%s.[01-9]*", paths->paths[ipath], sec, name); globres = glob(file, 0, NULL, &globinfo); if (globres != 0 && globres != GLOB_NOMATCH) warn("%s: glob", file); free(file); if (globres == 0) file = mandoc_strdup(*globinfo.gl_pathv); globfree(&globinfo); if (globres != 0) return 0; found: #if HAVE_SQLITE3 - warnx("outdated mandoc.db lacks %s(%s) entry, run makewhatis %s", - name, sec, paths->paths[ipath]); + warnx("outdated mandoc.db lacks %s(%s) entry, run %s %s", + name, sec, BINM_MAKEWHATIS, paths->paths[ipath]); #endif *res = mandoc_reallocarray(*res, ++*ressz, sizeof(struct manpage)); page = *res + (*ressz - 1); page->file = file; page->names = NULL; page->output = NULL; page->ipath = ipath; page->bits = NAME_FILE & NAME_MASK; page->sec = (*sec >= '1' && *sec <= '9') ? *sec - '1' + 1 : 10; page->form = form; return 1; } static void fs_search(const struct mansearch *cfg, const struct manpaths *paths, int argc, char **argv, struct manpage **res, size_t *ressz) { const char *const sections[] = - {"1", "8", "6", "2", "3", "3p", "5", "7", "4", "9"}; + {"1", "8", "6", "2", "3", "5", "7", "4", "9", "3p"}; const size_t nsec = sizeof(sections)/sizeof(sections[0]); size_t ipath, isec, lastsz; assert(cfg->argmode == ARG_NAME); *res = NULL; *ressz = lastsz = 0; while (argc) { for (ipath = 0; ipath < paths->sz; ipath++) { if (cfg->sec != NULL) { if (fs_lookup(paths, ipath, cfg->sec, cfg->arch, *argv, res, ressz) && cfg->firstmatch) return; } else for (isec = 0; isec < nsec; isec++) if (fs_lookup(paths, ipath, sections[isec], cfg->arch, *argv, res, ressz) && cfg->firstmatch) return; } if (*ressz == lastsz) warnx("No entry for %s in the manual.", *argv); lastsz = *ressz; argv++; argc--; } } static void parse(struct curparse *curp, int fd, const char *file) { enum mandoclevel rctmp; struct roff_man *man; /* Begin by parsing the file itself. */ assert(file); assert(fd >= 0); rctmp = mparse_readfd(curp->mp, fd, file); if (fd != STDIN_FILENO) close(fd); if (rc < rctmp) rc = rctmp; /* * With -Wstop and warnings or errors of at least the requested * level, do not produce output. */ if (rctmp != MANDOCLEVEL_OK && curp->wstop) return; /* If unset, allocate output dev now (if applicable). */ if (curp->outdata == NULL) { switch (curp->outtype) { case OUTT_HTML: curp->outdata = html_alloc(curp->outopts); break; case OUTT_UTF8: curp->outdata = utf8_alloc(curp->outopts); break; case OUTT_LOCALE: curp->outdata = locale_alloc(curp->outopts); break; case OUTT_ASCII: curp->outdata = ascii_alloc(curp->outopts); break; case OUTT_PDF: curp->outdata = pdf_alloc(curp->outopts); break; case OUTT_PS: curp->outdata = ps_alloc(curp->outopts); break; default: break; } } mparse_result(curp->mp, &man, NULL); /* Execute the out device, if it exists. */ if (man == NULL) return; if (man->macroset == MACROSET_MDOC) { mdoc_validate(man); switch (curp->outtype) { case OUTT_HTML: html_mdoc(curp->outdata, man); break; case OUTT_TREE: tree_mdoc(curp->outdata, man); break; case OUTT_MAN: man_mdoc(curp->outdata, man); break; case OUTT_PDF: case OUTT_ASCII: case OUTT_UTF8: case OUTT_LOCALE: case OUTT_PS: terminal_mdoc(curp->outdata, man); break; default: break; } } if (man->macroset == MACROSET_MAN) { man_validate(man); switch (curp->outtype) { case OUTT_HTML: html_man(curp->outdata, man); break; case OUTT_TREE: tree_man(curp->outdata, man); break; case OUTT_MAN: man_man(curp->outdata, man); break; case OUTT_PDF: case OUTT_ASCII: case OUTT_UTF8: case OUTT_LOCALE: case OUTT_PS: terminal_man(curp->outdata, man); break; default: break; } } } static void passthrough(const char *file, int fd, int synopsis_only) { const char synb[] = "S\bSY\bYN\bNO\bOP\bPS\bSI\bIS\bS"; const char synr[] = "SYNOPSIS"; FILE *stream; const char *syscall; char *line, *cp; size_t linesz; int print; line = NULL; linesz = 0; if ((stream = fdopen(fd, "r")) == NULL) { close(fd); syscall = "fdopen"; goto fail; } print = 0; while (getline(&line, &linesz, stream) != -1) { cp = line; if (synopsis_only) { if (print) { if ( ! isspace((unsigned char)*cp)) goto done; while (isspace((unsigned char)*cp)) cp++; } else { if (strcmp(cp, synb) == 0 || strcmp(cp, synr) == 0) print = 1; continue; } } if (fputs(cp, stdout)) { fclose(stream); syscall = "fputs"; goto fail; } } if (ferror(stream)) { fclose(stream); syscall = "getline"; goto fail; } done: free(line); fclose(stream); return; fail: free(line); warn("%s: SYSERR: %s", file, syscall); if (rc < MANDOCLEVEL_SYSERR) rc = MANDOCLEVEL_SYSERR; } static int koptions(int *options, char *arg) { if ( ! strcmp(arg, "utf-8")) { *options |= MPARSE_UTF8; *options &= ~MPARSE_LATIN1; } else if ( ! strcmp(arg, "iso-8859-1")) { *options |= MPARSE_LATIN1; *options &= ~MPARSE_UTF8; } else if ( ! strcmp(arg, "us-ascii")) { *options &= ~(MPARSE_UTF8 | MPARSE_LATIN1); } else { warnx("-K %s: Bad argument", arg); return 0; } return 1; } static int moptions(int *options, char *arg) { if (arg == NULL) /* nothing to do */; else if (0 == strcmp(arg, "doc")) *options |= MPARSE_MDOC; else if (0 == strcmp(arg, "andoc")) /* nothing to do */; else if (0 == strcmp(arg, "an")) *options |= MPARSE_MAN; else { warnx("-m %s: Bad argument", arg); return 0; } return 1; } static int toptions(struct curparse *curp, char *arg) { if (0 == strcmp(arg, "ascii")) curp->outtype = OUTT_ASCII; else if (0 == strcmp(arg, "lint")) { curp->outtype = OUTT_LINT; curp->wlevel = MANDOCLEVEL_WARNING; } else if (0 == strcmp(arg, "tree")) curp->outtype = OUTT_TREE; else if (0 == strcmp(arg, "man")) curp->outtype = OUTT_MAN; else if (0 == strcmp(arg, "html")) curp->outtype = OUTT_HTML; else if (0 == strcmp(arg, "utf8")) curp->outtype = OUTT_UTF8; else if (0 == strcmp(arg, "locale")) curp->outtype = OUTT_LOCALE; else if (0 == strcmp(arg, "xhtml")) curp->outtype = OUTT_HTML; else if (0 == strcmp(arg, "ps")) curp->outtype = OUTT_PS; else if (0 == strcmp(arg, "pdf")) curp->outtype = OUTT_PDF; else { warnx("-T %s: Bad argument", arg); return 0; } return 1; } static int woptions(struct curparse *curp, char *arg) { char *v, *o; const char *toks[7]; toks[0] = "stop"; toks[1] = "all"; toks[2] = "warning"; toks[3] = "error"; toks[4] = "unsupp"; toks[5] = "fatal"; toks[6] = NULL; while (*arg) { o = arg; switch (getsubopt(&arg, UNCONST(toks), &v)) { case 0: curp->wstop = 1; break; case 1: case 2: curp->wlevel = MANDOCLEVEL_WARNING; break; case 3: curp->wlevel = MANDOCLEVEL_ERROR; break; case 4: curp->wlevel = MANDOCLEVEL_UNSUPP; break; case 5: curp->wlevel = MANDOCLEVEL_BADARG; break; default: warnx("-W %s: Bad argument", o); return 0; } } return 1; } static void mmsg(enum mandocerr t, enum mandoclevel lvl, const char *file, int line, int col, const char *msg) { const char *mparse_msg; fprintf(stderr, "%s: %s:", getprogname(), file); if (line) fprintf(stderr, "%d:%d:", line, col + 1); fprintf(stderr, " %s", mparse_strlevel(lvl)); if (NULL != (mparse_msg = mparse_strerror(t))) fprintf(stderr, ": %s", mparse_msg); if (msg) fprintf(stderr, ": %s", msg); fputc('\n', stderr); } static pid_t spawn_pager(struct tag_files *tag_files) { + const struct timespec timeout = { 0, 100000000 }; /* 0.1s */ #define MAX_PAGER_ARGS 16 char *argv[MAX_PAGER_ARGS]; const char *pager; char *cp; size_t cmdlen; int argc; pid_t pager_pid; pager = getenv("MANPAGER"); if (pager == NULL || *pager == '\0') pager = getenv("PAGER"); if (pager == NULL || *pager == '\0') pager = "more -s"; cp = mandoc_strdup(pager); /* * Parse the pager command into words. * Intentionally do not do anything fancy here. */ argc = 0; while (argc + 4 < MAX_PAGER_ARGS) { argv[argc++] = cp; cp = strchr(cp, ' '); if (cp == NULL) break; *cp++ = '\0'; while (*cp == ' ') cp++; if (*cp == '\0') break; } - /* For more(1) and less(1), use the tag file. */ + /* For less(1), use the tag file. */ if ((cmdlen = strlen(argv[0])) >= 4) { cp = argv[0] + cmdlen - 4; - if (strcmp(cp, "less") == 0 || strcmp(cp, "more") == 0) { + if (strcmp(cp, "less") == 0) { argv[argc++] = mandoc_strdup("-T"); argv[argc++] = tag_files->tfn; } } argv[argc++] = tag_files->ofn; argv[argc] = NULL; switch (pager_pid = fork()) { case -1: err((int)MANDOCLEVEL_SYSERR, "fork"); case 0: - /* Set pgrp in both parent and child to avoid racing exec. */ - (void)setpgid(0, 0); break; default: (void)setpgid(pager_pid, 0); (void)tcsetpgrp(STDIN_FILENO, pager_pid); #if HAVE_PLEDGE if (pledge("stdio rpath tmppath tty proc", NULL) == -1) err((int)MANDOCLEVEL_SYSERR, "pledge"); #endif tag_files->pager_pid = pager_pid; return pager_pid; } /* The child process becomes the pager. */ if (dup2(tag_files->ofd, STDOUT_FILENO) == -1) err((int)MANDOCLEVEL_SYSERR, "pager stdout"); close(tag_files->ofd); close(tag_files->tfd); + + /* Do not start the pager before controlling the terminal. */ + + while (tcgetpgrp(STDIN_FILENO) != getpid()) + nanosleep(&timeout, NULL); + execvp(argv[0], argv); err((int)MANDOCLEVEL_SYSERR, "exec %s", argv[0]); } Index: vendor/mdocml/dist/main.h =================================================================== --- vendor/mdocml/dist/main.h (revision 303220) +++ vendor/mdocml/dist/main.h (revision 303221) @@ -1,53 +1,53 @@ -/* $Id: main.h,v 1.24 2015/11/07 14:01:16 schwarze Exp $ */ +/* $Id: main.h,v 1.25 2016/07/08 22:29:05 schwarze Exp $ */ /* * Copyright (c) 2009, 2010, 2011 Kristaps Dzonsons element. -.It Ev HTTP_HOST +.It Dv HTTP_HOST The FQDN of the (possibly virtual) host the HTTP server is running on. This is used for .Ic Location: headers in HTTP 303 responses. -.It Ev MAN_DIR -A path to the +.It Dv MAN_DIR +A file system path to the .Nm -data directory to be used instead of -.Pa /var/www/man , -relative to the web server +data directory relative to the web server .Xr chroot 2 -directory, to be specified without a trailing slash. -This is prepended to the manpath when opening +directory, to be specified with a leading slash and without a trailing slash. +It needs to have at least one component; the root directory cannot be used +for this purpose. +The files +.Pa manpath.conf , +.Pa header.html , +and +.Pa footer.html +are looked up in this directory. +It is also prepended to the manpath when opening .Xr mandoc.db 5 and manual page files. +.It Dv SCRIPT_NAME +The initial component of URIs, to be specified without leading +and trailing slashes. +It can be empty. .El .Pp After editing .Pa cgi.h , run .Pp .Dl make man.cgi .Pp -and copy the files to the proper locations. -Reading the -.Cm installcgi -target in the -.Pa Makefile -can help with that, but do not run it without carefully checking it -because the directory layouts of web servers vary greatly. +and copy the resulting binary to the proper location, +for example using the command: +.Pp +.Dl make installcgi +.Pp +In addition to that, make sure the default manpath contains the files +.Pa man1/apropos.1 +and +.Pa man8/man.cgi.8 , +or the documentation links at the bottom of the index page will not work. .Ss URI interface .Nm uniform resource identifiers are not needed for interactive use, but can be useful for deep linking. They consist of: .Bl -enum .It The .Cm http:// protocol specifier. .It -The host name and a following slash. +The host name. .It -The path to the program, normally -.Pa cgi-bin/man.cgi/ . +The +.Dv SCRIPT_NAME , +preceded by a slash unless empty. .It To show a single page, a slash, the manpath, another slash, and the name of the requested file, for example .Pa /OpenBSD-current/man1/mandoc.1 . +This can be abbreviated according to the following syntax: +.Sm off +.Op / Ar manpath +.Op / Cm man Ar sec +.Op / Ar arch +.Pf / Ar name Op \&. Ar sec +.Sm on .It For searches, a query string starting with a question mark and consisting of .Ar key Ns = Ns Ar value pairs, separated by ampersands, for example .Pa ?manpath=OpenBSD-current&query=mandoc . Supported keys are .Cm manpath , .Cm query , .Cm sec , .Cm arch , corresponding to .Xr apropos 1 .Fl M , .Ar expression , .Fl s , .Fl S , respectively, and .Cm apropos , which is a boolean parameter to select or deselect the .Xr apropos 1 query mode. For backward compatibility with the traditional .Nm , .Cm sektion is supported as an alias for .Cm sec . .El .Ss Restricted character set For security reasons, in particular to prevent cross site scripting attacks, some strings used by .Nm can only contain the following characters: .Pp .Bl -dash -compact -offset indent .It lower case and upper case ASCII letters .It the ten decimal digits .It the dash .Pq Sq - .It the dot .Pq Sq \&. .It the slash .Pq Sq / .It the underscore .Pq Sq _ .El .Pp -In particular, this applies to the -.Ev SCRIPT_NAME , -to all manpaths, and to all architecture names. +In particular, this applies to all manpaths and architecture names. .Sh ENVIRONMENT The web server may pass the following CGI variables to .Nm : .Bl -tag -width Ds +.It Ev SCRIPT_NAME +The initial part of the URI passed from the client to the server, +starting after the server's host name and ending before +.Ev PATH_INFO . +This is ignored by +.Nm . +When constructing URIs for links and redirections, the +.Dv SCRIPT_NAME +preprocessor constant is used instead. .It Ev PATH_INFO The final part of the URI path passed from the client to the server, starting after the .Ev SCRIPT_NAME and ending before the .Ev QUERY_STRING . It is used by the .Cm show page to acquire the manpath and filename it needs. .It Ev QUERY_STRING The HTTP query string passed from the client to the server. It is the final part of the URI, after the question mark. It is used by the .Cm search page to acquire the named parameters it needs. -.It Ev SCRIPT_NAME -The path to the -.Nm -binary relative to the server root, usually -.Pa /cgi-bin/man.cgi . -This is used for generating URIs to be embedded -in generated HTML code and HTTP headers. -If this contains any character not contained in the -.Sx Restricted character set , -.Nm -reports an internal server error and exits without doing anything. .El .Sh FILES .Bl -tag -width Ds .It Pa /var/www Default web server .Xr chroot 2 directory. All the following paths are specified relative to this directory. .It Pa /cgi-bin/man.cgi -The path to the +The usual file system path to the .Nm -program relative to the server root. -Can be overridden by -.Ev SCRIPT_NAME . +program inside the web server +.Xr chroot 2 +directory. +A different name can be chosen, but in any case, it needs to be configured in +.Xr httpd.conf 5 . .It Pa /htdocs -The path to the server document root relative to the server root. +The file system path to the server document root directory +relative to the server +.Xr chroot 2 +directory. This is part of the web server configuration and not specific to .Nm . .It Pa /htdocs/mandoc.css A style sheet for .Xr mandoc 1 HTML styling, referenced from each generated HTML page. .It Pa /man Default .Nm data directory containing all the manual trees. Can be overridden by -.Ev MAN_DIR . -.It Pa /man/mandoc/man1/apropos.1 , /man/mandoc/man8/man.cgi.8 -Manual pages documenting -.Nm -itself, linked from the index page. +.Dv MAN_DIR . .It Pa /man/manpath.conf The list of available manpaths, one per line. If any of the lines in this file contains a slash .Pq Sq / or any character not contained in the .Sx Restricted character set , .Nm reports an internal server error and exits without doing anything. .It Pa /man/header.html An optional file containing static HTML code to be inserted right after opening the <BODY> element. .It Pa /man/footer.html An optional file containing static HTML code to be inserted right before closing the <BODY> element. .It Pa /man/OpenBSD-current/man1/mandoc.1 An example .Xr mdoc 7 source file located below the .Dq OpenBSD-current manpath. .El .Sh COMPATIBILITY The .Nm CGI program is call-compatible with queries from the traditional .Pa man.cgi script by Wolfram Schneider. However, the output may not be quite the same. .Sh SEE ALSO .Xr apropos 1 , .Xr mandoc.db 5 , .Xr makewhatis 8 , .Xr slowcgi 8 .Sh HISTORY A version of .Nm based on .Xr mandoc 1 first appeared in mdocml-1.12.1 (March 2012). The current SQLite3-based version first appeared in .Ox 5.6 . .Sh AUTHORS .An -nosplit The .Nm program was written by .An Kristaps Dzonsons Aq Mt kristaps@bsd.lv and ported to the SQLite3-based .Xr mandoc.db 5 backend by .An Ingo Schwarze Aq Mt schwarze@openbsd.org . Index: vendor/mdocml/dist/mandoc.3 =================================================================== --- vendor/mdocml/dist/mandoc.3 (revision 303220) +++ vendor/mdocml/dist/mandoc.3 (revision 303221) @@ -1,686 +1,677 @@ -.\" $Id: mandoc.3,v 1.36 2016/01/08 17:48:09 schwarze Exp $ +.\" $Id: mandoc.3,v 1.37 2016/07/07 19:19:01 schwarze Exp $ .\" .\" Copyright (c) 2009, 2010, 2011 Kristaps Dzonsons <kristaps@bsd.lv> .\" Copyright (c) 2010-2016 Ingo Schwarze <schwarze@openbsd.org> .\" .\" Permission to use, copy, modify, and distribute this software for any .\" purpose with or without fee is hereby granted, provided that the above .\" copyright notice and this permission notice appear in all copies. .\" .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. .\" -.Dd $Mdocdate: January 8 2016 $ +.Dd $Mdocdate: July 7 2016 $ .Dt MANDOC 3 .Os .Sh NAME .Nm mandoc , -.Nm man_deroff , -.Nm man_meta , +.Nm deroff , +.Nm mandocmsg , .Nm man_mparse , -.Nm man_node , -.Nm mdoc_deroff , -.Nm mdoc_meta , -.Nm mdoc_node , +.Nm man_validate , +.Nm mdoc_validate , .Nm mparse_alloc , .Nm mparse_free , .Nm mparse_getkeep , .Nm mparse_keep , .Nm mparse_open , .Nm mparse_readfd , .Nm mparse_reset , .Nm mparse_result , .Nm mparse_strerror , .Nm mparse_strlevel .Nd mandoc macro compiler library .Sh SYNOPSIS .In sys/types.h .In mandoc.h .Pp .Fd "#define ASCII_NBRSP" .Fd "#define ASCII_HYPH" .Fd "#define ASCII_BREAK" .Ft struct mparse * .Fo mparse_alloc .Fa "int options" .Fa "enum mandoclevel wlevel" .Fa "mandocmsg mmsg" .Fa "char *defos" .Fc .Ft void .Fo (*mandocmsg) .Fa "enum mandocerr errtype" .Fa "enum mandoclevel level" .Fa "const char *file" .Fa "int line" .Fa "int col" .Fa "const char *msg" .Fc .Ft void .Fo mparse_free .Fa "struct mparse *parse" .Fc .Ft const char * .Fo mparse_getkeep .Fa "const struct mparse *parse" .Fc .Ft void .Fo mparse_keep .Fa "struct mparse *parse" .Fc .Ft int .Fo mparse_open .Fa "struct mparse *parse" .Fa "const char *fname" .Fc .Ft "enum mandoclevel" .Fo mparse_readfd .Fa "struct mparse *parse" .Fa "int fd" .Fa "const char *fname" .Fc .Ft void .Fo mparse_reset .Fa "struct mparse *parse" .Fc .Ft void .Fo mparse_result .Fa "struct mparse *parse" -.Fa "struct mdoc **mdoc" -.Fa "struct man **man" +.Fa "struct roff_man **man" .Fa "char **sodest" .Fc .Ft "const char *" .Fo mparse_strerror .Fa "enum mandocerr" .Fc .Ft "const char *" .Fo mparse_strlevel .Fa "enum mandoclevel" .Fc -.In sys/types.h -.In mandoc.h -.In mdoc.h +.In roff.h .Ft void -.Fo mdoc_deroff +.Fo deroff .Fa "char **dest" -.Fa "const struct mdoc_node *node" +.Fa "const struct roff_node *node" .Fc -.Ft "const struct mdoc_meta *" -.Fo mdoc_meta -.Fa "const struct mdoc *mdoc" -.Fc -.Ft "const struct mdoc_node *" -.Fo mdoc_node -.Fa "const struct mdoc *mdoc" -.Fc +.In sys/types.h +.In mandoc.h +.In mdoc.h .Vt extern const char * const * mdoc_argnames; .Vt extern const char * const * mdoc_macronames; +.Ft void +.Fo mdoc_validate +.Fa "struct roff_man *mdoc" +.Fc .In sys/types.h .In mandoc.h .In man.h -.Ft void -.Fo man_deroff -.Fa "char **dest" -.Fa "const struct man_node *node" -.Fc -.Ft "const struct man_meta *" -.Fo man_meta -.Fa "const struct man *man" -.Fc +.Vt extern const char * const * man_macronames; .Ft "const struct mparse *" .Fo man_mparse -.Fa "const struct man *man" +.Fa "const struct roff_man *man" .Fc -.Ft "const struct man_node *" -.Fo man_node -.Fa "const struct man *man" +.Ft void +.Fo man_validate +.Fa "struct roff_man *man" .Fc -.Vt extern const char * const * man_macronames; .Sh DESCRIPTION The .Nm mandoc library parses a .Ux manual into an abstract syntax tree (AST). .Ux manuals are composed of .Xr mdoc 7 or .Xr man 7 , and may be mixed with .Xr roff 7 , .Xr tbl 7 , and .Xr eqn 7 invocations. .Pp The following describes a general parse sequence: .Bl -enum .It initiate a parsing sequence with .Xr mchars_alloc 3 and .Fn mparse_alloc ; .It open a file with .Xr open 2 or .Fn mparse_open ; .It parse it with .Fn mparse_readfd ; .It close it with .Xr close 2 ; .It retrieve the syntax tree with .Fn mparse_result ; .It -iterate over parse nodes with -.Fn mdoc_node +depending on whether the +.Fa macroset +member of the returned +.Vt struct roff_man +is +.Dv MACROSET_MDOC or -.Fn man_node ; +.Dv MACROSET_MAN , +validate it with +.Fn mdoc_validate +or +.Fn man_validate , +respectively; .It +iterate over parse nodes with starting from the +.Fa first +member of the returned +.Vt struct roff_man ; +.It free all allocated memory with .Fn mparse_free and .Xr mchars_free 3 , or invoke .Fn mparse_reset -and parse new files. +and go back to step 2 to parse new files. .El .Sh REFERENCE This section documents the functions, types, and variables available via .In mandoc.h , with the exception of those documented in .Xr mandoc_escape 3 and .Xr mchars_alloc 3 . .Ss Types .Bl -ohang .It Vt "enum mandocerr" An error or warning message during parsing. .It Vt "enum mandoclevel" A classification of an .Vt "enum mandocerr" as regards system operation. +See the DIAGNOSTICS section in +.Xr mandoc 1 +regarding the meanings of the levels. .It Vt "struct mparse" An opaque pointer to a running parse sequence. Created with .Fn mparse_alloc and freed with .Fn mparse_free . This may be used across parsed input if .Fn mparse_reset is called between parses. .It Vt "mandocmsg" A prototype for a function to handle error and warning messages emitted by the parser. .El .Ss Functions .Bl -ohang -.It Fn man_deroff +.It Fn deroff Obtain a text-only representation of a -.Vt struct man_node , +.Vt struct roff_node , including text contained in its child nodes. -To be used on children of the pointer returned from -.Fn man_node . +To be used on children of the +.Fa first +member of +.Vt struct roff_man . When it is no longer needed, the pointer returned from -.Fn man_deroff +.Fn deroff can be passed to .Xr free 3 . -.It Fn man_meta -Obtain the meta-data of a successful -.Xr man 7 -parse. -This may only be used on a pointer returned by -.Fn mparse_result . -Declared in -.In man.h , -implemented in -.Pa man.c . .It Fn man_mparse Get the parser used for the current output. Declared in .In man.h , implemented in .Pa man.c . -.It Fn man_node -Obtain the root node of a successful -.Xr man 7 -parse. -This may only be used on a pointer returned by +.It Fn man_validate +Validate the +.Dv MACROSET_MAN +parse tree obtained with .Fn mparse_result . Declared in .In man.h , implemented in .Pa man.c . -.It Fn mdoc_deroff -Obtain a text-only representation of a -.Vt struct mdoc_node , -including text contained in its child nodes. -To be used on children of the pointer returned from -.Fn mdoc_node . -When it is no longer needed, the pointer returned from -.Fn mdoc_deroff -can be passed to -.Xr free 3 . -.It Fn mdoc_meta -Obtain the meta-data of a successful -.Xr mdoc -parse. -This may only be used on a pointer returned by +.It Fn mdoc_validate +Validate the +.Dv MACROSET_MDOC +parse tree obtained with .Fn mparse_result . Declared in .In mdoc.h , implemented in .Pa mdoc.c . -.It Fn mdoc_node -Obtain the root node of a successful -.Xr mdoc -parse. -This may only be used on a pointer returned by -.Fn mparse_result . -Declared in -.In mdoc.h , -implemented in -.Pa mdoc.c . .It Fn mparse_alloc Allocate a parser. The arguments have the following effect: .Bl -tag -offset 5n -width inttype .It Ar options When the .Dv MPARSE_MDOC or .Dv MPARSE_MAN bit is set, only that parser is used. Otherwise, the document type is automatically detected. .Pp When the .Dv MPARSE_SO bit is set, .Xr roff 7 .Ic \&so file inclusion requests are always honoured. Otherwise, if the request is the only content in an input file, only the file name is remembered, to be returned in the .Fa sodest argument of .Fn mparse_result . .Pp When the .Dv MPARSE_QUICK bit is set, parsing is aborted after the NAME section. This is for example useful in .Xr makewhatis 8 .Fl Q to quickly build minimal databases. .It Ar wlevel Can be set to .Dv MANDOCLEVEL_BADARG , .Dv MANDOCLEVEL_ERROR , or .Dv MANDOCLEVEL_WARNING . Messages below the selected level will be suppressed. .It Ar mmsg A callback function to handle errors and warnings. See .Pa main.c for an example. +If printing of error messages is not desired, +.Dv NULL +may be passed. .It Ar defos A default string for the .Xr mdoc 7 .Sq \&Os macro, overriding the .Dv OSNAME preprocessor definition and the results of .Xr uname 3 . +Passing +.Dv NULL +sets no default. .El .Pp The same parser may be used for multiple files so long as .Fn mparse_reset is called between parses. .Fn mparse_free must be called to free the memory allocated by this function. Declared in .In mandoc.h , implemented in .Pa read.c . .It Fn mparse_free Free all memory allocated by .Fn mparse_alloc . Declared in .In mandoc.h , implemented in .Pa read.c . .It Fn mparse_getkeep Acquire the keep buffer. Must follow a call of .Fn mparse_keep . Declared in .In mandoc.h , implemented in .Pa read.c . .It Fn mparse_keep Instruct the parser to retain a copy of its parsed input. This can be acquired with subsequent .Fn mparse_getkeep calls. Declared in .In mandoc.h , implemented in .Pa read.c . .It Fn mparse_open Open the file for reading. If that fails and .Fa fname does not already end in .Ql .gz , try again after appending .Ql .gz . Save the information whether the file is zipped or not. Return a file descriptor open for reading or -1 on failure. It can be passed to .Fn mparse_readfd or used directly. Declared in .In mandoc.h , implemented in .Pa read.c . .It Fn mparse_readfd Parse a file descriptor opened with .Xr open 2 or .Fn mparse_open . Pass the associated filename in .Va fname . This function may be called multiple times with different parameters; however, .Xr close 2 and .Fn mparse_reset should be invoked between parses. Declared in .In mandoc.h , implemented in .Pa read.c . .It Fn mparse_reset Reset a parser so that .Fn mparse_readfd may be used again. Declared in .In mandoc.h , implemented in .Pa read.c . .It Fn mparse_result Obtain the result of a parse. -One of the three pointers will be filled in. +One of the two pointers will be filled in. Declared in .In mandoc.h , implemented in .Pa read.c . .It Fn mparse_strerror Return a statically-allocated string representation of an error code. Declared in .In mandoc.h , implemented in .Pa read.c . .It Fn mparse_strlevel Return a statically-allocated string representation of a level code. Declared in .In mandoc.h , implemented in .Pa read.c . .El .Ss Variables .Bl -ohang .It Va man_macronames -The string representation of a man macro as indexed by +The string representation of a +.Xr man 7 +macro as indexed by .Vt "enum mant" . .It Va mdoc_argnames -The string representation of a mdoc macro argument as indexed by +The string representation of an +.Xr mdoc 7 +macro argument as indexed by .Vt "enum mdocargt" . .It Va mdoc_macronames -The string representation of a mdoc macro as indexed by +The string representation of an +.Xr mdoc 7 +macro as indexed by .Vt "enum mdoct" . .El .Sh IMPLEMENTATION NOTES This section consists of structural documentation for .Xr mdoc 7 and .Xr man 7 syntax trees and strings. .Ss Man and Mdoc Strings Strings may be extracted from mdoc and man meta-data, or from text nodes (MDOC_TEXT and MAN_TEXT, respectively). These strings have special non-printing formatting cues embedded in the text itself, as well as .Xr roff 7 escapes preserved from input. Implementing systems will need to handle both situations to produce human-readable text. In general, strings may be assumed to consist of 7-bit ASCII characters. .Pp The following non-printing characters may be embedded in text strings: .Bl -tag -width Ds .It Dv ASCII_NBRSP A non-breaking space character. .It Dv ASCII_HYPH A soft hyphen. .It Dv ASCII_BREAK A breakable zero-width space. .El .Pp Escape characters are also passed verbatim into text strings. An escape character is a sequence of characters beginning with the backslash .Pq Sq \e . To construct human-readable text, these should be intercepted with .Xr mandoc_escape 3 and converted with one the functions described in .Xr mchars_alloc 3 . .Ss Man Abstract Syntax Tree This AST is governed by the ontological rules dictated in .Xr man 7 and derives its terminology accordingly. .Pp The AST is composed of -.Vt struct man_node +.Vt struct roff_node nodes with element, root and text types as declared by the .Va type field. Each node also provides its parse point (the .Va line , -.Va sec , +.Va pos , and -.Va pos +.Va sec fields), its position in the tree (the .Va parent , .Va child , .Va next and .Va prev fields) and some type-specific data. .Pp The tree itself is arranged according to the following normal form, where capitalised non-terminals represent nodes. .Pp .Bl -tag -width "ELEMENTXX" -compact .It ROOT \(<- mnode+ .It mnode \(<- ELEMENT | TEXT | BLOCK .It BLOCK \(<- HEAD BODY .It HEAD \(<- mnode* .It BODY \(<- mnode* .It ELEMENT \(<- ELEMENT | TEXT* .It TEXT \(<- [[:ascii:]]* .El .Pp The only elements capable of nesting other elements are those with next-line scope as documented in .Xr man 7 . .Ss Mdoc Abstract Syntax Tree This AST is governed by the ontological rules dictated in .Xr mdoc 7 and derives its terminology accordingly. .Qq In-line elements described in .Xr mdoc 7 are described simply as .Qq elements . .Pp The AST is composed of -.Vt struct mdoc_node +.Vt struct roff_node nodes with block, head, body, element, root and text types as declared by the .Va type field. Each node also provides its parse point (the .Va line , -.Va sec , +.Va pos , and -.Va pos +.Va sec fields), its position in the tree (the .Va parent , .Va child , .Va last , .Va next and .Va prev fields) and some type-specific data, in particular, for nodes generated from macros, the generating macro in the .Va tok field. .Pp The tree itself is arranged according to the following normal form, where capitalised non-terminals represent nodes. .Pp .Bl -tag -width "ELEMENTXX" -compact .It ROOT \(<- mnode+ .It mnode \(<- BLOCK | ELEMENT | TEXT .It BLOCK \(<- HEAD [TEXT] (BODY [TEXT])+ [TAIL [TEXT]] .It ELEMENT \(<- TEXT* .It HEAD \(<- mnode* .It BODY \(<- mnode* [ENDBODY mnode*] .It TAIL \(<- mnode* .It TEXT \(<- [[:ascii:]]* .El .Pp Of note are the TEXT nodes following the HEAD, BODY and TAIL nodes of the BLOCK production: these refer to punctuation marks. Furthermore, although a TEXT node will generally have a non-zero-length string, in the specific case of .Sq \&.Bd \-literal , an empty line will produce a zero-length string. Multiple body parts are only found in invocations of .Sq \&Bl \-column , where a new body introduces a new phrase. .Pp The .Xr mdoc 7 syntax tree accommodates for broken block structures as well. The ENDBODY node is available to end the formatting associated with a given block before the physical end of that block. It has a non-null .Va end field, is of the BODY .Va type , has the same .Va tok as the BLOCK it is ending, and has a .Va pending field pointing to that BLOCK's BODY node. It is an indirect child of that BODY node and has no children of its own. .Pp An ENDBODY node is generated when a block ends while one of its child blocks is still open, like in the following example: .Bd -literal -offset indent \&.Ao ao \&.Bo bo ac \&.Ac bc \&.Bc end .Ed .Pp This example results in the following block structure: .Bd -literal -offset indent BLOCK Ao HEAD Ao BODY Ao TEXT ao BLOCK Bo, pending -> Ao HEAD Bo BODY Bo TEXT bo TEXT ac ENDBODY Ao, pending -> Ao TEXT bc TEXT end .Ed .Pp Here, the formatting of the .Sq \&Ao block extends from TEXT ao to TEXT ac, while the formatting of the .Sq \&Bo block extends from TEXT bo to TEXT bc. It renders as follows in .Fl T Ns Cm ascii mode: .Pp .Dl <ao [bo ac> bc] end .Pp Support for badly-nested blocks is only provided for backward compatibility with some older .Xr mdoc 7 implementations. Using badly-nested blocks is .Em strongly discouraged ; for example, the .Fl T Ns Cm html and .Fl T Ns Cm xhtml front-ends to .Xr mandoc 1 are unable to render them in any meaningful way. Furthermore, behaviour when encountering badly-nested blocks is not consistent across troff implementations, especially when using multiple levels of badly-nested blocks. .Sh SEE ALSO .Xr mandoc 1 , +.Xr man.cgi 3 , .Xr mandoc_escape 3 , +.Xr mandoc_headers 3 , .Xr mandoc_malloc 3 , +.Xr mansearch 3 , .Xr mchars_alloc 3 , +.Xr tbl 3 , .Xr eqn 7 , .Xr man 7 , .Xr mandoc_char 7 , .Xr mdoc 7 , .Xr roff 7 , .Xr tbl 7 .Sh AUTHORS +.An -nosplit The .Nm library was written by -.An Kristaps Dzonsons Aq Mt kristaps@bsd.lv . +.An Kristaps Dzonsons Aq Mt kristaps@bsd.lv +and is maintained by +.An Ingo Schwarze Aq Mt schwarze@openbsd.org . Index: vendor/mdocml/dist/mandoc.css =================================================================== --- vendor/mdocml/dist/mandoc.css (revision 303220) +++ vendor/mdocml/dist/mandoc.css (revision 303221) @@ -1,158 +1,159 @@ -/* $Id: mandoc.css,v 1.1 2015/11/05 17:47:51 schwarze Exp $ */ +/* $Id: mandoc.css,v 1.2 2016/04/13 10:19:23 schwarze Exp $ */ /* * This is an example style-sheet provided for mandoc(1) and the -Thtml * or -Txhtml output mode. * * It mimics the appearance of the traditional cvsweb output. * * See mdoc(7) and man(7) for macro explanations. */ html { max-width: 880px; margin-left: 1em; } body { font-size: smaller; font-family: Helvetica,Arial,sans-serif; } body > div { padding-left: 2em; padding-top: 1em; } body > div.mandoc, body > div#mancgi { padding-left: 0em; padding-top: 0em; } body > div.results { font-size: smaller; } #mancgi fieldset { text-align: center; border: thin solid silver; border-radius: 1em; font-size: small; } #mancgi input[name=expr] { width: 25%; } .results td.title { vertical-align: top; padding-right: 1em; } -h1 { margin-bottom: 1ex; font-size: 110%; margin-left: -4ex; } /* Section header (Sh, SH). */ +h1 { margin-bottom: 1ex; font-size: 110% } +div.section > h1 { margin-left: -4ex; } /* Section header (Sh, SH). */ h2 { margin-bottom: 1ex; font-size: 105%; margin-left: -2ex; } /* Sub-section header (Ss, SS). */ table { width: 100%; margin-top: 0ex; margin-bottom: 0ex; } /* All tables. */ td { vertical-align: top; } /* All table cells. */ p { } /* Paragraph: Pp, Lp. */ blockquote { margin-left: 5ex; margin-top: 0ex; margin-bottom: 0ex; } /* D1. */ div.section { margin-bottom: 2ex; margin-left: 5ex; } /* Sections (Sh, SH). */ div.subsection { } /* Sub-sections (Ss, SS). */ table.synopsis { } /* SYNOPSIS section table. */ div.spacer { margin: 1em 0; } /* Preamble structure. */ table.foot { font-size: smaller; margin-top: 1em; border-top: 1px dotted #dddddd; } /* Document footer. */ td.foot-date { width: 50%; } /* Document footer: date. */ td.foot-os { width: 50%; } /* Document footer: OS/source. */ table.head { font-size: smaller; margin-bottom: 1em; border-bottom: 1px dotted #dddddd; } /* Document header. */ td.head-ltitle { width: 10%; } /* Document header: left-title. */ td.head-vol { width: 80%; } /* Document header: volume. */ td.head-rtitle { width: 10%; } /* Document header: right-title. */ /* General font modes. */ i { } /* Italic: BI, IB, I, (implicit). */ .emph { font-style: italic; font-weight: normal; } /* Emphasis: Em, Bl -emphasis. */ b { } /* Bold: SB, BI, IB, BR, RB, B, (implicit). */ .symb { font-style: normal; font-weight: bold; } /* Symbolic: Sy, Ms, Bf -symbolic. */ small { } /* Small: SB, SM. */ .lit { font-style: normal; font-weight: normal; font-family: monospace; } /* Literal: Dl, Li, Ql, Bf -literal, Bl -literal, Bl -unfilled. */ /* Block modes. */ .display { } /* Top of all Bd, D1, Dl. */ .list { } /* Top of all Bl. */ /* Context-specific modes. */ i.addr { font-weight: normal; } /* Address (Ad). */ i.arg { font-weight: normal; } /* Command argument (Ar). */ span.author { } /* Author name (An). */ b.cmd { font-style: normal; } /* Command (Cm). */ b.config { font-style: normal; } /* Config statement (Cd). */ span.define { } /* Defines (Dv). */ span.desc { } /* Nd. After em-dash. */ b.diag { font-style: normal; } /* Diagnostic (Bl -diag). */ span.env { } /* Environment variables (Ev). */ span.errno { } /* Error string (Er). */ i.farg { font-weight: normal; } /* Function argument (Fa, Fn). */ i.file { font-weight: normal; } /* File (Pa). */ b.flag { font-style: normal; } /* Flag (Fl, Cm). */ b.fname { font-style: normal; } /* Function name (Fa, Fn, Rv). */ i.ftype { font-weight: normal; } /* Function types (Ft, Fn). */ b.includes { font-style: normal; } /* Header includes (In). */ span.lib { } /* Library (Lb). */ i.link-sec { font-weight: normal; } /* Section links (Sx). */ b.macro { font-style: normal; } /* Macro-ish thing (Fd). */ b.name { font-style: normal; } /* Name of utility (Nm). */ span.opt { } /* Options (Op, Oo/Oc). */ span.ref { } /* Citations (Rs). */ span.ref-auth { } /* Reference author (%A). */ i.ref-book { font-weight: normal; } /* Reference book (%B). */ span.ref-city { } /* Reference city (%C). */ span.ref-date { } /* Reference date (%D). */ i.ref-issue { font-weight: normal; } /* Reference issuer/publisher (%I). */ i.ref-jrnl { font-weight: normal; } /* Reference journal (%J). */ span.ref-num { } /* Reference number (%N). */ span.ref-opt { } /* Reference optionals (%O). */ span.ref-page { } /* Reference page (%P). */ span.ref-corp { } /* Reference corporate/foreign author (%Q). */ span.ref-rep { } /* Reference report (%R). */ span.ref-title { text-decoration: underline; } /* Reference title (%T). */ span.ref-vol { } /* Reference volume (%V). */ span.type { font-style: italic; font-weight: normal; } /* Variable types (Vt). */ span.unix { } /* Unices (Ux, Ox, Nx, Fx, Bx, Bsx, Dx). */ b.utility { font-style: normal; } /* Name of utility (Ex). */ b.var { font-style: normal; } /* Variables (Rv). */ a.link-ext { } /* Off-site link (Lk). */ a.link-includes { } /* Include-file link (In). */ a.link-mail { } /* Mailto links (Mt). */ a.link-man { } /* Manual links (Xr). */ a.link-ref { } /* Reference section links (%Q). */ a.link-sec { } /* Section links (Sx). */ /* Formatting for lists. See mdoc(7). */ dl.list-diag { } dt.list-diag { } dd.list-diag { } dl.list-hang { } dt.list-hang { } dd.list-hang { } dl.list-inset { } dt.list-inset { } dd.list-inset { } dl.list-ohang { } dt.list-ohang { } dd.list-ohang { margin-left: 0ex; } dl.list-tag { } dt.list-tag { } dd.list-tag { } table.list-col { } tr.list-col { } td.list-col { } ul.list-bul { list-style-type: disc; padding-left: 1em; } li.list-bul { } ul.list-dash { list-style-type: none; padding-left: 0em; } li.list-dash:before { content: "\2014 "; } ul.list-hyph { list-style-type: none; padding-left: 0em; } li.list-hyph:before { content: "\2013 "; } ul.list-item { list-style-type: none; padding-left: 0em; } li.list-item { } ol.list-enum { padding-left: 2em; } li.list-enum { } /* Equation modes. See eqn(7). */ span.eqn { } /* Table modes. See tbl(7). */ table.tbl { } Index: vendor/mdocml/dist/mandoc.db.5 =================================================================== --- vendor/mdocml/dist/mandoc.db.5 (revision 303220) +++ vendor/mdocml/dist/mandoc.db.5 (revision 303221) @@ -1,157 +1,156 @@ -.\" $Id: mandoc.db.5,v 1.3 2014/12/30 21:34:57 schwarze Exp $ +.\" $Id: mandoc.db.5,v 1.4 2016/07/07 14:35:48 schwarze Exp $ .\" .\" Copyright (c) 2014 Ingo Schwarze <schwarze@openbsd.org> .\" .\" Permission to use, copy, modify, and distribute this software for any .\" purpose with or without fee is hereby granted, provided that the above .\" copyright notice and this permission notice appear in all copies. .\" .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. .\" -.Dd $Mdocdate: December 30 2014 $ +.Dd $Mdocdate: July 7 2016 $ .Dt MANDOC.DB 5 .Os .Sh NAME .Nm mandoc.db .Nd manual page database .Sh DESCRIPTION The .Nm SQLite3 file format is used to store information about installed manual pages to facilitate semantic searching for manuals. Each manual page tree contains its own .Nm file; see .Sx FILES for examples. .Pp Such database files are generated by .Xr makewhatis 8 and used by .Xr apropos 1 and .Xr whatis 1 . .Pp One line in the following tables describes: .Bl -tag -width Ds .It Sy mpages One physical manual page file, no matter how many times and under which names it may appear in the file system. .It Sy mlinks One entry in the file system, no matter which content it points to. .It Sy names One manual page name, no matter whether it appears in a page header, in a NAME or SYNOPSIS section, or as a file name. .It Sy keys One chunk of text from some macro invocation. .El .Pp Each record in the latter three tables uses its .Va pageid column to point to a record in the .Sy mpages table. .Pp The other columns are as follows; unless stated otherwise, they are of type .Vt TEXT . .Bl -tag -width mpages.desc .It Sy mpages.desc The description line .Pq Sq \&Nd of the page. .It Sy mpages.form An .Vt INTEGER bit field. If bit .Dv FORM_GZ is set, the page is compressed and requires .Xr gunzip 1 for display. If bit .Dv FORM_SRC is set, the page is unformatted, that is in .Xr mdoc 7 or .Xr man 7 format, and requires .Xr mandoc 1 for display. If bit .Dv FORM_SRC is not set, the page is formatted, i.e. a .Sq cat page. .It Sy mlinks.sec The manual section as found in the subdirectory name. .It Sy mlinks.arch The manual architecture as found in the subdirectory name, or .Qq any . .It Sy mlinks.name The manual name as found in the file name. .It Sy names.bits An .Vt INTEGER bit mask telling whether the name came from a header line, from the NAME or SYNOPSIS section, or from a file name. Bits are defined in .In mansearch.h . .It Sy names.name The name itself. .It Sy keys.bits An .Vt INTEGER bit mask telling which semantic contexts the key was found in; defined in .In mansearch.h , documented in .Xr apropos 1 . .It Sy keys.key The string found in those contexts. .El .Sh FILES .Bl -tag -width /usr/share/man/mandoc.db -compact .It Pa /usr/share/man/mandoc.db The manual page database for the base system. .It Pa /usr/X11R6/man/mandoc.db The same for the .Xr X 7 Window System. .It Pa /usr/local/man/mandoc.db The same for .Xr packages 7 . .El .Sh SEE ALSO .Xr apropos 1 , .Xr man 1 , .Xr sqlite3 1 , .Xr whatis 1 , -.Xr mansearch 3 , .Xr makewhatis 8 .Sh HISTORY A manual page database .Pa /usr/lib/whatis first appeared in .Bx 2 . The present format first appeared in .Ox 5.6 . .Sh AUTHORS .An -nosplit The original version of .Xr makewhatis 8 was written by .An Bill Joy in 1979. An SQLite3 version was first implemented by .An Kristaps Dzonsons Aq Mt kristaps@bsd.lv in 2012. The present database format was designed by .An Ingo Schwarze Aq Mt schwarze@openbsd.org in 2014. Index: vendor/mdocml/dist/mandoc_headers.3 =================================================================== --- vendor/mdocml/dist/mandoc_headers.3 (revision 303220) +++ vendor/mdocml/dist/mandoc_headers.3 (revision 303221) @@ -1,517 +1,525 @@ .Dd December 1, 2014 .Dt MANDOC_HEADERS 3 .Os .Sh NAME .Nm mandoc_headers .Nd ordering of mandoc include files .Sh DESCRIPTION To support a cleaner coding style, the mandoc header files do not contain any include directives and do not guard against multiple inclusion. The application developer has to make sure that the headers are included in a proper order, and that no header is included more than once. .Pp The headers and functions form three major groups: .Sx Parser interface , .Sx Parser internals , and .Sx Formatter interface . .Pp Various rules are given below prohibiting the inclusion of certain combinations of headers into the same file. The intention is to keep the following functional components separate from each other: .Pp .Bl -dash -offset indent -compact .It .Xr mdoc 7 parser .It .Xr man 7 parser .It .Xr roff 7 parser .It .Xr tbl 7 parser .It .Xr eqn 7 parser .It terminal formatters .It HTML formatters .It search tools .El .Pp Note that mere usage of an opaque struct type does .Em not require inclusion of the header where that type is defined. .Ss Parser interface Each of the following headers can be included without including any other mandoc header. These headers should be included before any other mandoc headers. .Bl -tag -width Ds .It Qq Pa mandoc_aux.h Requires .In sys/types.h for .Vt size_t . Provides the utility functions documented in .Xr mandoc_malloc 3 . +.It Qq Pa mandoc_ohash.h +Includes +.In ohash.h +and provides +.Fn mandoc_ohash_init . .It Qq Pa mandoc.h Requires .In sys/types.h for .Vt size_t . .Pp Provides .Vt enum mandoc_esc , .Vt enum mandocerr , .Vt enum mandoclevel , .Vt enum tbl_cellt , .Vt enum tbl_datt , .Vt enum tbl_spant , .Vt enum eqn_boxt , .Vt enum eqn_fontt , .Vt enum eqn_pilet , .Vt enum eqn_post , .Vt struct tbl_opts , -.Vt struct tbl_head , .Vt struct tbl_cell , .Vt struct tbl_row , .Vt struct tbl_dat , .Vt struct tbl_span , .Vt struct eqn_box , .Vt struct eqn , the function prototype typedef .Fn mandocmsg , the function .Xr mandoc_escape 3 , the functions described in .Xr mchars_alloc 3 , and the functions .Fn mparse_* described in .Xr mandoc 3 . .Pp Uses the opaque type .Vt struct mparse from .Pa read.c for function prototypes. -Uses the types -.Vt struct mdoc +Uses the type +.Vt struct roff_man from -.Pa libmdoc.h -and -.Vt struct man -from -.Pa libman.h -as opaque types for function prototypes. +.Pa roff.h +as an opaque type for function prototypes. .It Qq Pa roff.h Provides .Vt enum mdoc_endbody , +.Vt enum roff_macroset , +.Vt enum roff_next , .Vt enum roff_sec , .Vt enum roff_type , +.Vt struct roff_man , .Vt struct roff_meta , -and -.Vt struct roff_node . +.Vt struct roff_node , +and the function +.Fn deroff . .Pp Uses pointers to the types .Vt struct mdoc_arg and .Vt union mdoc_data from -.Qq Pa mdoc.h +.Pa mdoc.h as opaque struct members. .El .Pp The following two require .Qq Pa roff.h but no other mandoc headers. Afterwards, any other mandoc headers can be included as needed. .Bl -tag -width Ds .It Qq Pa mdoc.h Requires .In sys/types.h for -.Vt size_t -and -.Qq Pa roff.h -for -.Vt enum roff_type . +.Vt size_t . .Pp Provides .Vt enum mdocargt , -.Vt enum mdoc_disp , -.Vt enum mdoc_list , .Vt enum mdoc_auth , +.Vt enum mdoc_disp , .Vt enum mdoc_font , +.Vt enum mdoc_list , .Vt struct mdoc_argv , .Vt struct mdoc_arg , -.Vt struct mdoc_bd , -.Vt struct mdoc_bl , .Vt struct mdoc_an , +.Vt struct mdoc_bd , .Vt struct mdoc_bf , +.Vt struct mdoc_bl , .Vt struct mdoc_rs , +.Vt union mdoc_data , and the functions .Fn mdoc_* described in .Xr mandoc 3 . .Pp Uses the type -.Vt struct mdoc +.Vt struct roff_man from -.Pa libmdoc.h +.Pa roff.h as an opaque type for function prototypes. -Uses pointers to the types -.Vt struct tbl_span -and -.Vt struct eqn -as opaque struct members. .Pp When this header is included, the same file should not include .Pa libman.h or .Pa libroff.h . .It Qq Pa man.h -Requires -.Qq Pa roff.h -for -.Vt enum roff_type . -.Pp Provides the functions .Fn man_* described in .Xr mandoc 3 . .Pp Uses the opaque type .Vt struct mparse from .Pa read.c for function prototypes. Uses the type -.Vt struct man +.Vt struct roff_man from -.Pa libman.h +.Pa roff.h as an opaque type for function prototypes. -Uses pointers to the types -.Vt struct tbl_span -and -.Vt struct eqn -as opaque struct members. .Pp When this header is included, the same file should not include .Pa libmdoc.h or .Pa libroff.h . .El .Ss Parser internals The following headers require inclusion of a parser interface header before they can be included. All parser interface headers should precede all parser internal headers. When any parser internal headers are included, the same file should not include any formatter headers. .Bl -tag -width Ds .It Qq Pa libmandoc.h Requires .In sys/types.h for -.Vt size_t , +.Vt size_t +and .Qq Pa mandoc.h for -.Vt enum mandocerr , -and -.Qq Pa roff.h -for -.Vt struct roff_meta -and -.Vt struct roff_node . +.Vt enum mandocerr . .Pp Provides .Vt enum rofferr , .Vt struct buf , utility functions needed by multiple parsers, and the top-level functions to call the parsers. .Pp Uses the opaque types .Vt struct mparse from .Pa read.c and .Vt struct roff from .Pa roff.c for function prototypes. Uses the types .Vt struct tbl_span and .Vt struct eqn from +.Pa mandoc.h +and +.Vt struct roff_man +from +.Pa roff.h +as opaque types for function prototypes. +.It Qq Pa roff_int.h +Requires +.Qq Pa roff.h +for +.Vt enum roff_type . +.Pp +Provides functions named +.Fn roff_* +to handle roff nodes and the two special functions +.Fn man_breakscope +and +.Fn mdoc_argv_free +because the latter two are needed by +.Qq Pa roff.c . +.Pp +Uses the types +.Vt struct eqn +and +.Vt struct tbl_span +from .Pa mandoc.h , -.Vt struct mdoc +.Vt struct roff_man +and +.Vt struct roff_node from -.Pa libmdoc.h , +.Pa roff.h , and -.Vt struct man +.Vt struct mdoc_arg from -.Pa libman.h +.Pa mdoc.h as opaque types for function prototypes. .It Qq Pa libmdoc.h Requires .Qq Pa mdoc.h for .Vt enum mdoc_* and .Vt struct mdoc_* . .Pp Provides -.Vt enum mdoc_next , .Vt enum margserr , .Vt enum mdelim , -.Vt struct mdoc , .Vt struct mdoc_macro , and many functions internal to the .Xr mdoc 7 parser. .Pp -Uses the opaque types +Uses the opaque type .Vt struct mparse from -.Pa read.c +.Pa read.c . +Uses the types +.Vt struct roff_man and -.Vt struct roff +.Vt struct roff_node from -.Pa roff.c . +.Pa roff.h +as opaque types for function prototypes. .Pp When this header is included, the same file should not include .Pa man.h , .Pa libman.h , or .Pa libroff.h . .It Qq Pa libman.h -Requires -.Qq Pa roff.h -for -.Vt struct roff_meta -and -.Vt struct roff_node . -.Pp Provides -.Vt enum man_next , -.Vt struct man , -.Vt struct man_macro , -and many functions internal to the +.Vt struct man_macro +and some functions internal to the .Xr man 7 parser. .Pp -Uses the opaque types -.Vt struct mparse -from -.Pa read.c +Uses the types +.Vt struct roff_man and -.Vt struct roff +.Vt struct roff_node from -.Pa roff.c . +.Pa roff.h +as opaque types for function prototypes. .Pp When this header is included, the same file should not include .Pa mdoc.h , .Pa libmdoc.h , or .Pa libroff.h . .It Qq Pa libroff.h Requires .In sys/types.h for .Vt size_t , .Qq Pa mandoc.h for .Vt struct tbl_* and .Vt struct eqn , and .Qq Pa libmandoc.h for .Vt enum rofferr . .Pp Provides .Vt enum tbl_part , .Vt struct tbl_node , .Vt struct eqn_def , .Vt struct eqn_node , and many functions internal to the .Xr tbl 7 and .Xr eqn 7 parsers. .Pp Uses the opaque type .Vt struct mparse from .Pa read.c . .Pp When this header is included, the same file should not include .Pa man.h , .Pa mdoc.h , .Pa libman.h , or .Pa libmdoc.h . .El .Ss Formatter interface These headers should be included after any parser interface headers. No parser internal headers should be included by the same file. .Bl -tag -width Ds .It Qq Pa out.h Requires .In sys/types.h for .Vt size_t . .Pp Provides .Vt enum roffscale , .Vt struct roffcol , .Vt struct roffsu , .Vt struct rofftbl , .Fn a2roffsu , and .Fn tblcalc . .Pp Uses .Vt struct tbl_span from .Pa mandoc.h as an opaque type for function prototypes. .Pp When this header is included, the same file should not include .Pa mansearch.h . .It Qq Pa term.h Requires .In sys/types.h for .Vt size_t and .Qq Pa out.h for .Vt struct roffsu and .Vt struct rofftbl . .Pp Provides .Vt enum termenc , .Vt enum termfont , .Vt enum termtype , .Vt struct termp_tbl , .Vt struct termp , and many terminal formatting functions. .Pp Uses the opaque type .Vt struct termp_ps from .Pa term_ps.c . Uses .Vt struct tbl_span and .Vt struct eqn from .Pa mandoc.h and .Vt struct roff_meta from -.Qq Pa roff.h +.Pa roff.h as opaque types for function prototypes. .Pp When this header is included, the same file should not include .Pa html.h or .Pa mansearch.h . .It Qq Pa html.h Requires .In sys/types.h for .Vt size_t , .In stdio.h for .Dv BUFSIZ , and .Qq Pa out.h for .Vt struct roffsu and .Vt struct rofftbl . .Pp Provides .Vt enum htmltag , .Vt enum htmlattr , .Vt enum htmlfont , .Vt struct tag , .Vt struct tagq , .Vt struct htmlpair , .Vt struct html , and many HTML formatting functions. .Pp When this header is included, the same file should not include .Pa term.h or .Pa mansearch.h . +.It Qq Pa tag.h +Requires +.In sys/types.h +for +.Vt size_t . +.Pp +Provides an interface to generate +.Xr ctags 1 +files for the +.Ic :t +functionality mentioned in +.Xr man 1 . .It Qq Pa main.h Provides the top level steering functions for all formatters. .Pp -Uses the types -.Vt struct mdoc +Uses the type +.Vt struct roff_man from -.Pa libmdoc.h -and -.Vt struct man -from -.Pa libman.h -as opaque types for function prototypes. +.Pa roff.h +as an opaque type for function prototypes. .It Qq Pa manconf.h Requires .In sys/types.h for .Vt size_t . .Pp Provides .Vt struct manconf , .Vt struct manpaths , .Vt struct manoutput , and the functions .Fn manconf_parse , .Fn manconf_output , and .Fn manconf_free . .It Qq Pa mansearch.h Requires .In sys/types.h for .Vt size_t and .In stdint.h for .Vt uint64_t . .Pp Provides .Vt enum argmode , .Vt struct manpage , .Vt struct mansearch , and the functions .Fn mansearch_setup , .Fn mansearch , and .Fn mansearch_free . .Pp Uses .Vt struct manpaths from .Pa manconf.h as an opaque type for function prototypes. .Pp When this header is included, the same file should not include .Pa out.h , .Pa term.h , or .Pa html.h . .El Index: vendor/mdocml/dist/mandoc_malloc.3 =================================================================== --- vendor/mdocml/dist/mandoc_malloc.3 (revision 303220) +++ vendor/mdocml/dist/mandoc_malloc.3 (revision 303221) @@ -1,197 +1,191 @@ -.\" $Id: mandoc_malloc.3,v 1.1 2014/08/05 05:48:56 schwarze Exp $ +.\" $Id: mandoc_malloc.3,v 1.2 2016/07/07 19:19:01 schwarze Exp $ .\" .\" Copyright (c) 2014 Ingo Schwarze <schwarze@openbsd.org> .\" .\" Permission to use, copy, modify, and distribute this software for any .\" purpose with or without fee is hereby granted, provided that the above .\" copyright notice and this permission notice appear in all copies. .\" .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. .\" -.Dd $Mdocdate: August 5 2014 $ +.Dd $Mdocdate: July 7 2016 $ .Dt MANDOC_MALLOC 3 .Os .Sh NAME .Nm mandoc_malloc , .Nm mandoc_realloc , .Nm mandoc_reallocarray , .Nm mandoc_calloc , .Nm mandoc_strdup , .Nm mandoc_strndup , .Nm mandoc_asprintf .Nd memory allocation function wrappers used in the mandoc library -.Sh LIBRARY -.Lb libmandoc .Sh SYNOPSIS .In sys/types.h .In mandoc_aux.h .Ft "void *" .Fo mandoc_malloc .Fa "size_t size" .Fc .Ft "void *" .Fo mandoc_realloc .Fa "void *ptr" .Fa "size_t size" .Fc .Ft "void *" .Fo mandoc_reallocarray .Fa "void *ptr" .Fa "size_t nmemb" .Fa "size_t size" .Fc .Ft "void *" .Fo mandoc_calloc .Fa "size_t nmemb" .Fa "size_t size" .Fc .Ft "char *" .Fo mandoc_strdup .Fa "const char *s" .Fc .Ft "char *" .Fo mandoc_strndup .Fa "const char *s" .Fa "size_t maxlen" .Fc .Ft int .Fo mandoc_asprintf .Fa "char **ret" .Fa "const char *format" .Fa "..." .Fc .Sh DESCRIPTION -These functions call the -.Lb libc -functions of the same names, passing through their return values when -successful. +These functions call the libc functions of the same names, passing +through their return values when successful. In case of failure, they do not return, but instead call -.Xr perror 3 -and -.Xr exit 3 . -They can be used both internally by any code in the -.Lb libmandoc +.Xr err 3 . +They can be used both internally by any code in the mandoc libraries and externally by programs using that library, for example .Xr mandoc 1 , +.Xr man 1 , .Xr apropos 1 , +.Xr makewhatis 8 , and -.Xr makewhatis 8 . +.Xr man.cgi 8 . .Pp The function .Fn mandoc_malloc allocates one new object, leaving the memory uninitialized. The functions .Fn mandoc_realloc and .Fn mandoc_reallocarray change the size of an existing object or array, possibly moving it. When shrinking the size, existing data is truncated; when growing, the additional memory is not initialized. The function .Fn mandoc_calloc allocates a new array, initializing it to zero. .Pp The argument .Fa size is the size of each object. The argument .Fa nmemb is the new number of objects in the array. The argument .Fa ptr is a pointer to the existing object or array to be resized; if it is .Dv NULL , a new object or array is allocated. .Pp The functions .Fn mandoc_strdup and .Fn mandoc_strndup copy a string into newly allocated memory. For .Fn mandoc_strdup , the string pointed to by .Fa s needs to be NUL-terminated. For .Fn mandoc_strndup , at most .Fa maxlen bytes are copied. The function .Fn mandoc_asprintf writes output formatted according to .Fa format into newly allocated memory and returns a pointer to the result in .Fa ret . For all three string functions, the result is always NUL-terminated. .Pp When the objects and strings are no longer needed, the pointers returned by these functions can be passed to .Xr free 3 . .Sh RETURN VALUES The function .Fn mandoc_asprintf always returns the number of characters written, excluding the final NUL byte. It never returns -1. .Pp The other functions always return a valid pointer; they never return .Dv NULL . .Sh FILES These functions are implemented in .Pa mandoc_aux.c . .Sh SEE ALSO .Xr asprintf 3 , -.Xr exit 3 , +.Xr err 3 , .Xr malloc 3 , -.Xr perror 3 , .Xr strdup 3 .Sh STANDARDS The functions .Fn malloc , .Fn realloc , and .Fn calloc are required by .St -ansiC . The functions .Fn strdup and .Fn strndup are required by .St -p1003.1-2008 . The function .Fn asprintf is a widespread extension that first appeared in the GNU C library. .Pp The function .Fn reallocarray is an extension that first appeared in .Ox 5.6 . If it is not provided by the operating system, the mandoc build system uses a bundled portable implementation. .Sh HISTORY The functions .Fn mandoc_malloc , .Fn mandoc_realloc , .Fn mandoc_calloc , and .Fn mandoc_strdup have been available since mandoc 1.9.12, .Fn mandoc_strndup since 1.11.5, and .Fn mandoc_asprintf and .Fn mandoc_reallocarray since 1.12.4 and 1.13.0. .Sh AUTHORS .An Kristaps Dzonsons Aq Mt kristaps@bsd.lv .An Ingo Schwarze Aq Mt schwarze@openbsd.org Index: vendor/mdocml/dist/mandocdb.c =================================================================== --- vendor/mdocml/dist/mandocdb.c (revision 303220) +++ vendor/mdocml/dist/mandocdb.c (revision 303221) @@ -1,2531 +1,2540 @@ -/* $Id: mandocdb.c,v 1.215 2016/01/08 17:48:09 schwarze Exp $ */ +/* $Id: mandocdb.c,v 1.218 2016/07/12 05:18:38 kristaps Exp $ */ /* * Copyright (c) 2011, 2012 Kristaps Dzonsons <kristaps@bsd.lv> * Copyright (c) 2011-2016 Ingo Schwarze <schwarze@openbsd.org> * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHORS DISCLAIM ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include "config.h" #include <sys/types.h> #include <sys/stat.h> #include <sys/wait.h> #include <assert.h> #include <ctype.h> #if HAVE_ERR #include <err.h> #endif #include <errno.h> #include <fcntl.h> #if HAVE_FTS #include <fts.h> #else #include "compat_fts.h" #endif -#include <getopt.h> #include <limits.h> +#if HAVE_SANDBOX_INIT +#include <sandbox.h> +#endif #include <stddef.h> #include <stdio.h> #include <stdint.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include <sqlite3.h> #include "mandoc_aux.h" #include "mandoc_ohash.h" #include "mandoc.h" #include "roff.h" #include "mdoc.h" #include "man.h" #include "manconf.h" #include "mansearch.h" extern int mansearch_keymax; extern const char *const mansearch_keynames[]; #define SQL_EXEC(_v) \ if (SQLITE_OK != sqlite3_exec(db, (_v), NULL, NULL, NULL)) \ say("", "%s: %s", (_v), sqlite3_errmsg(db)) #define SQL_BIND_TEXT(_s, _i, _v) \ if (SQLITE_OK != sqlite3_bind_text \ ((_s), (_i)++, (_v), -1, SQLITE_STATIC)) \ say(mlink->file, "%s", sqlite3_errmsg(db)) #define SQL_BIND_INT(_s, _i, _v) \ if (SQLITE_OK != sqlite3_bind_int \ ((_s), (_i)++, (_v))) \ say(mlink->file, "%s", sqlite3_errmsg(db)) #define SQL_BIND_INT64(_s, _i, _v) \ if (SQLITE_OK != sqlite3_bind_int64 \ ((_s), (_i)++, (_v))) \ say(mlink->file, "%s", sqlite3_errmsg(db)) #define SQL_STEP(_s) \ if (SQLITE_DONE != sqlite3_step((_s))) \ say(mlink->file, "%s", sqlite3_errmsg(db)) enum op { OP_DEFAULT = 0, /* new dbs from dir list or default config */ OP_CONFFILE, /* new databases from custom config file */ OP_UPDATE, /* delete/add entries in existing database */ OP_DELETE, /* delete entries from existing database */ OP_TEST /* change no databases, report potential problems */ }; struct str { const struct mpage *mpage; /* if set, the owning parse */ uint64_t mask; /* bitmask in sequence */ char key[]; /* rendered text */ }; struct inodev { ino_t st_ino; dev_t st_dev; }; struct mpage { struct inodev inodev; /* used for hashing routine */ int64_t pageid; /* pageid in mpages SQL table */ char *sec; /* section from file content */ char *arch; /* architecture from file content */ char *title; /* title from file content */ char *desc; /* description from file content */ struct mlink *mlinks; /* singly linked list */ int form; /* format from file content */ int name_head_done; }; struct mlink { char file[PATH_MAX]; /* filename rel. to manpath */ char *dsec; /* section from directory */ char *arch; /* architecture from directory */ char *name; /* name from file name (not empty) */ char *fsec; /* section from file name suffix */ struct mlink *next; /* singly linked list */ struct mpage *mpage; /* parent */ int dform; /* format from directory */ int fform; /* format from file name suffix */ int gzip; /* filename has a .gz suffix */ }; enum stmt { STMT_DELETE_PAGE = 0, /* delete mpage */ STMT_INSERT_PAGE, /* insert mpage */ STMT_INSERT_LINK, /* insert mlink */ STMT_INSERT_NAME, /* insert name */ STMT_SELECT_NAME, /* retrieve existing name flags */ STMT_INSERT_KEY, /* insert parsed key */ STMT__MAX }; typedef int (*mdoc_fp)(struct mpage *, const struct roff_meta *, const struct roff_node *); struct mdoc_handler { mdoc_fp fp; /* optional handler */ uint64_t mask; /* set unless handler returns 0 */ }; static void dbclose(int); static void dbadd(struct mpage *); static void dbadd_mlink(const struct mlink *mlink); static void dbadd_mlink_name(const struct mlink *mlink); static int dbopen(int); static void dbprune(void); static void filescan(const char *); static void mlink_add(struct mlink *, const struct stat *); static void mlink_check(struct mpage *, struct mlink *); static void mlink_free(struct mlink *); static void mlinks_undupe(struct mpage *); static void mpages_free(void); static void mpages_merge(struct mparse *); static void names_check(void); static void parse_cat(struct mpage *, int); static void parse_man(struct mpage *, const struct roff_meta *, const struct roff_node *); static void parse_mdoc(struct mpage *, const struct roff_meta *, const struct roff_node *); static int parse_mdoc_head(struct mpage *, const struct roff_meta *, const struct roff_node *); static int parse_mdoc_Fd(struct mpage *, const struct roff_meta *, const struct roff_node *); static void parse_mdoc_fname(struct mpage *, const struct roff_node *); static int parse_mdoc_Fn(struct mpage *, const struct roff_meta *, const struct roff_node *); static int parse_mdoc_Fo(struct mpage *, const struct roff_meta *, const struct roff_node *); static int parse_mdoc_Nd(struct mpage *, const struct roff_meta *, const struct roff_node *); static int parse_mdoc_Nm(struct mpage *, const struct roff_meta *, const struct roff_node *); static int parse_mdoc_Sh(struct mpage *, const struct roff_meta *, const struct roff_node *); static int parse_mdoc_Va(struct mpage *, const struct roff_meta *, const struct roff_node *); static int parse_mdoc_Xr(struct mpage *, const struct roff_meta *, const struct roff_node *); static void putkey(const struct mpage *, char *, uint64_t); static void putkeys(const struct mpage *, char *, size_t, uint64_t); static void putmdockey(const struct mpage *, const struct roff_node *, uint64_t); static int render_string(char **, size_t *); static void say(const char *, const char *, ...); static int set_basedir(const char *, int); static int treescan(void); static size_t utf8(unsigned int, char [7]); static char tempfilename[32]; static int nodb; /* no database changes */ static int mparse_options; /* abort the parse early */ static int use_all; /* use all found files */ static int debug; /* print what we're doing */ static int warnings; /* warn about crap */ static int write_utf8; /* write UTF-8 output; else ASCII */ static int exitcode; /* to be returned by main */ static enum op op; /* operational mode */ static char basedir[PATH_MAX]; /* current base directory */ static struct ohash mpages; /* table of distinct manual pages */ static struct ohash mlinks; /* table of directory entries */ static struct ohash names; /* table of all names */ static struct ohash strings; /* table of all strings */ static sqlite3 *db = NULL; /* current database */ static sqlite3_stmt *stmts[STMT__MAX]; /* current statements */ static uint64_t name_mask; static const struct mdoc_handler mdocs[MDOC_MAX] = { { NULL, 0 }, /* Ap */ { NULL, 0 }, /* Dd */ { NULL, 0 }, /* Dt */ { NULL, 0 }, /* Os */ { parse_mdoc_Sh, TYPE_Sh }, /* Sh */ { parse_mdoc_head, TYPE_Ss }, /* Ss */ { NULL, 0 }, /* Pp */ { NULL, 0 }, /* D1 */ { NULL, 0 }, /* Dl */ { NULL, 0 }, /* Bd */ { NULL, 0 }, /* Ed */ { NULL, 0 }, /* Bl */ { NULL, 0 }, /* El */ { NULL, 0 }, /* It */ { NULL, 0 }, /* Ad */ { NULL, TYPE_An }, /* An */ { NULL, TYPE_Ar }, /* Ar */ { NULL, TYPE_Cd }, /* Cd */ { NULL, TYPE_Cm }, /* Cm */ { NULL, TYPE_Dv }, /* Dv */ { NULL, TYPE_Er }, /* Er */ { NULL, TYPE_Ev }, /* Ev */ { NULL, 0 }, /* Ex */ { NULL, TYPE_Fa }, /* Fa */ { parse_mdoc_Fd, 0 }, /* Fd */ { NULL, TYPE_Fl }, /* Fl */ { parse_mdoc_Fn, 0 }, /* Fn */ { NULL, TYPE_Ft }, /* Ft */ { NULL, TYPE_Ic }, /* Ic */ { NULL, TYPE_In }, /* In */ { NULL, TYPE_Li }, /* Li */ { parse_mdoc_Nd, 0 }, /* Nd */ { parse_mdoc_Nm, 0 }, /* Nm */ { NULL, 0 }, /* Op */ { NULL, 0 }, /* Ot */ { NULL, TYPE_Pa }, /* Pa */ { NULL, 0 }, /* Rv */ { NULL, TYPE_St }, /* St */ { parse_mdoc_Va, TYPE_Va }, /* Va */ { parse_mdoc_Va, TYPE_Vt }, /* Vt */ { parse_mdoc_Xr, 0 }, /* Xr */ { NULL, 0 }, /* %A */ { NULL, 0 }, /* %B */ { NULL, 0 }, /* %D */ { NULL, 0 }, /* %I */ { NULL, 0 }, /* %J */ { NULL, 0 }, /* %N */ { NULL, 0 }, /* %O */ { NULL, 0 }, /* %P */ { NULL, 0 }, /* %R */ { NULL, 0 }, /* %T */ { NULL, 0 }, /* %V */ { NULL, 0 }, /* Ac */ { NULL, 0 }, /* Ao */ { NULL, 0 }, /* Aq */ { NULL, TYPE_At }, /* At */ { NULL, 0 }, /* Bc */ { NULL, 0 }, /* Bf */ { NULL, 0 }, /* Bo */ { NULL, 0 }, /* Bq */ { NULL, TYPE_Bsx }, /* Bsx */ { NULL, TYPE_Bx }, /* Bx */ { NULL, 0 }, /* Db */ { NULL, 0 }, /* Dc */ { NULL, 0 }, /* Do */ { NULL, 0 }, /* Dq */ { NULL, 0 }, /* Ec */ { NULL, 0 }, /* Ef */ { NULL, TYPE_Em }, /* Em */ { NULL, 0 }, /* Eo */ { NULL, TYPE_Fx }, /* Fx */ { NULL, TYPE_Ms }, /* Ms */ { NULL, 0 }, /* No */ { NULL, 0 }, /* Ns */ { NULL, TYPE_Nx }, /* Nx */ { NULL, TYPE_Ox }, /* Ox */ { NULL, 0 }, /* Pc */ { NULL, 0 }, /* Pf */ { NULL, 0 }, /* Po */ { NULL, 0 }, /* Pq */ { NULL, 0 }, /* Qc */ { NULL, 0 }, /* Ql */ { NULL, 0 }, /* Qo */ { NULL, 0 }, /* Qq */ { NULL, 0 }, /* Re */ { NULL, 0 }, /* Rs */ { NULL, 0 }, /* Sc */ { NULL, 0 }, /* So */ { NULL, 0 }, /* Sq */ { NULL, 0 }, /* Sm */ { NULL, 0 }, /* Sx */ { NULL, TYPE_Sy }, /* Sy */ { NULL, TYPE_Tn }, /* Tn */ { NULL, 0 }, /* Ux */ { NULL, 0 }, /* Xc */ { NULL, 0 }, /* Xo */ { parse_mdoc_Fo, 0 }, /* Fo */ { NULL, 0 }, /* Fc */ { NULL, 0 }, /* Oo */ { NULL, 0 }, /* Oc */ { NULL, 0 }, /* Bk */ { NULL, 0 }, /* Ek */ { NULL, 0 }, /* Bt */ { NULL, 0 }, /* Hf */ { NULL, 0 }, /* Fr */ { NULL, 0 }, /* Ud */ { NULL, TYPE_Lb }, /* Lb */ { NULL, 0 }, /* Lp */ { NULL, TYPE_Lk }, /* Lk */ { NULL, TYPE_Mt }, /* Mt */ { NULL, 0 }, /* Brq */ { NULL, 0 }, /* Bro */ { NULL, 0 }, /* Brc */ { NULL, 0 }, /* %C */ { NULL, 0 }, /* Es */ { NULL, 0 }, /* En */ { NULL, TYPE_Dx }, /* Dx */ { NULL, 0 }, /* %Q */ { NULL, 0 }, /* br */ { NULL, 0 }, /* sp */ { NULL, 0 }, /* %U */ { NULL, 0 }, /* Ta */ { NULL, 0 }, /* ll */ }; int mandocdb(int argc, char *argv[]) { struct manconf conf; struct mparse *mp; const char *path_arg, *progname; size_t j, sz; int ch, i; #if HAVE_PLEDGE if (pledge("stdio rpath wpath cpath fattr flock proc exec", NULL) == -1) { warn("pledge"); return (int)MANDOCLEVEL_SYSERR; } #endif +#if HAVE_SANDBOX_INIT + if (sandbox_init(kSBXProfileNoInternet, SANDBOX_NAMED, NULL) == -1) { + warnx("sandbox_init"); + return (int)MANDOCLEVEL_SYSERR; + } +#endif + memset(&conf, 0, sizeof(conf)); memset(stmts, 0, STMT__MAX * sizeof(sqlite3_stmt *)); /* * We accept a few different invocations. * The CHECKOP macro makes sure that invocation styles don't * clobber each other. */ #define CHECKOP(_op, _ch) do \ if (OP_DEFAULT != (_op)) { \ warnx("-%c: Conflicting option", (_ch)); \ goto usage; \ } while (/*CONSTCOND*/0) path_arg = NULL; op = OP_DEFAULT; while (-1 != (ch = getopt(argc, argv, "aC:Dd:npQT:tu:v"))) switch (ch) { case 'a': use_all = 1; break; case 'C': CHECKOP(op, ch); path_arg = optarg; op = OP_CONFFILE; break; case 'D': debug++; break; case 'd': CHECKOP(op, ch); path_arg = optarg; op = OP_UPDATE; break; case 'n': nodb = 1; break; case 'p': warnings = 1; break; case 'Q': mparse_options |= MPARSE_QUICK; break; case 'T': if (strcmp(optarg, "utf8")) { warnx("-T%s: Unsupported output format", optarg); goto usage; } write_utf8 = 1; break; case 't': CHECKOP(op, ch); dup2(STDOUT_FILENO, STDERR_FILENO); op = OP_TEST; nodb = warnings = 1; break; case 'u': CHECKOP(op, ch); path_arg = optarg; op = OP_DELETE; break; case 'v': /* Compatibility with espie@'s makewhatis. */ break; default: goto usage; } argc -= optind; argv += optind; #if HAVE_PLEDGE if (nodb) { if (pledge("stdio rpath", NULL) == -1) { warn("pledge"); return (int)MANDOCLEVEL_SYSERR; } } #endif if (OP_CONFFILE == op && argc > 0) { warnx("-C: Too many arguments"); goto usage; } exitcode = (int)MANDOCLEVEL_OK; mchars_alloc(); mp = mparse_alloc(mparse_options, MANDOCLEVEL_BADARG, NULL, NULL); mandoc_ohash_init(&mpages, 6, offsetof(struct mpage, inodev)); mandoc_ohash_init(&mlinks, 6, offsetof(struct mlink, file)); if (OP_UPDATE == op || OP_DELETE == op || OP_TEST == op) { /* * Most of these deal with a specific directory. * Jump into that directory first. */ if (OP_TEST != op && 0 == set_basedir(path_arg, 1)) goto out; if (dbopen(1)) { /* * The existing database is usable. Process * all files specified on the command-line. */ #if HAVE_PLEDGE if (!nodb) { if (pledge("stdio rpath wpath cpath fattr flock", NULL) == -1) { warn("pledge"); exitcode = (int)MANDOCLEVEL_SYSERR; goto out; } } #endif use_all = 1; for (i = 0; i < argc; i++) filescan(argv[i]); if (OP_TEST != op) dbprune(); } else { /* * Database missing or corrupt. * Recreate from scratch. */ exitcode = (int)MANDOCLEVEL_OK; op = OP_DEFAULT; if (0 == treescan()) goto out; if (0 == dbopen(0)) goto out; } if (OP_DELETE != op) mpages_merge(mp); dbclose(OP_DEFAULT == op ? 0 : 1); } else { /* * If we have arguments, use them as our manpaths. * If we don't, grok from manpath(1) or however else * manconf_parse() wants to do it. */ if (argc > 0) { conf.manpath.paths = mandoc_reallocarray(NULL, argc, sizeof(char *)); conf.manpath.sz = (size_t)argc; for (i = 0; i < argc; i++) conf.manpath.paths[i] = mandoc_strdup(argv[i]); } else manconf_parse(&conf, path_arg, NULL, NULL); if (conf.manpath.sz == 0) { exitcode = (int)MANDOCLEVEL_BADARG; say("", "Empty manpath"); } /* * First scan the tree rooted at a base directory, then * build a new database and finally move it into place. * Ignore zero-length directories and strip trailing * slashes. */ for (j = 0; j < conf.manpath.sz; j++) { sz = strlen(conf.manpath.paths[j]); if (sz && conf.manpath.paths[j][sz - 1] == '/') conf.manpath.paths[j][--sz] = '\0'; if (0 == sz) continue; if (j) { mandoc_ohash_init(&mpages, 6, offsetof(struct mpage, inodev)); mandoc_ohash_init(&mlinks, 6, offsetof(struct mlink, file)); } if ( ! set_basedir(conf.manpath.paths[j], argc > 0)) continue; if (0 == treescan()) continue; if (0 == dbopen(0)) continue; mpages_merge(mp); if (warnings && !nodb && ! (MPARSE_QUICK & mparse_options)) names_check(); dbclose(0); if (j + 1 < conf.manpath.sz) { mpages_free(); ohash_delete(&mpages); ohash_delete(&mlinks); } } } out: manconf_free(&conf); mparse_free(mp); mchars_free(); mpages_free(); ohash_delete(&mpages); ohash_delete(&mlinks); return exitcode; usage: progname = getprogname(); fprintf(stderr, "usage: %s [-aDnpQ] [-C file] [-Tutf8]\n" " %s [-aDnpQ] [-Tutf8] dir ...\n" " %s [-DnpQ] [-Tutf8] -d dir [file ...]\n" " %s [-Dnp] -u dir [file ...]\n" " %s [-Q] -t file ...\n", progname, progname, progname, progname, progname); return (int)MANDOCLEVEL_BADARG; } /* * Scan a directory tree rooted at "basedir" for manpages. * We use fts(), scanning directory parts along the way for clues to our * section and architecture. * * If use_all has been specified, grok all files. * If not, sanitise paths to the following: * * [./]man*[/<arch>]/<name>.<section> * or * [./]cat<section>[/<arch>]/<name>.0 * - * TODO: accomodate for multi-language directories. + * TODO: accommodate for multi-language directories. */ static int treescan(void) { char buf[PATH_MAX]; FTS *f; FTSENT *ff; struct mlink *mlink; int dform, gzip; char *dsec, *arch, *fsec, *cp; const char *path; const char *argv[2]; argv[0] = "."; argv[1] = (char *)NULL; f = fts_open((char * const *)argv, FTS_PHYSICAL | FTS_NOCHDIR, NULL); if (f == NULL) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "&fts_open"); return 0; } dsec = arch = NULL; dform = FORM_NONE; while ((ff = fts_read(f)) != NULL) { path = ff->fts_path + 2; switch (ff->fts_info) { /* * Symbolic links require various sanity checks, * then get handled just like regular files. */ case FTS_SL: if (realpath(path, buf) == NULL) { if (warnings) say(path, "&realpath"); continue; } if (strstr(buf, basedir) != buf #ifdef HOMEBREWDIR && strstr(buf, HOMEBREWDIR) != buf #endif ) { if (warnings) say("", "%s: outside base directory", buf); continue; } /* Use logical inode to avoid mpages dupe. */ if (stat(path, ff->fts_statp) == -1) { if (warnings) say(path, "&stat"); continue; } /* FALLTHROUGH */ /* * If we're a regular file, add an mlink by using the * stored directory data and handling the filename. */ case FTS_F: if ( ! strcmp(path, MANDOC_DB)) continue; if ( ! use_all && ff->fts_level < 2) { if (warnings) say(path, "Extraneous file"); continue; } gzip = 0; fsec = NULL; while (fsec == NULL) { fsec = strrchr(ff->fts_name, '.'); if (fsec == NULL || strcmp(fsec+1, "gz")) break; gzip = 1; *fsec = '\0'; fsec = NULL; } if (fsec == NULL) { if ( ! use_all) { if (warnings) say(path, "No filename suffix"); continue; } } else if ( ! strcmp(++fsec, "html")) { if (warnings) say(path, "Skip html"); continue; } else if ( ! strcmp(fsec, "ps")) { if (warnings) say(path, "Skip ps"); continue; } else if ( ! strcmp(fsec, "pdf")) { if (warnings) say(path, "Skip pdf"); continue; } else if ( ! use_all && ((dform == FORM_SRC && strncmp(fsec, dsec, strlen(dsec))) || (dform == FORM_CAT && strcmp(fsec, "0")))) { if (warnings) say(path, "Wrong filename suffix"); continue; } else fsec[-1] = '\0'; mlink = mandoc_calloc(1, sizeof(struct mlink)); if (strlcpy(mlink->file, path, sizeof(mlink->file)) >= sizeof(mlink->file)) { say(path, "Filename too long"); free(mlink); continue; } mlink->dform = dform; mlink->dsec = dsec; mlink->arch = arch; mlink->name = ff->fts_name; mlink->fsec = fsec; mlink->gzip = gzip; mlink_add(mlink, ff->fts_statp); continue; case FTS_D: case FTS_DP: break; default: if (warnings) say(path, "Not a regular file"); continue; } switch (ff->fts_level) { case 0: /* Ignore the root directory. */ break; case 1: /* * This might contain manX/ or catX/. * Try to infer this from the name. * If we're not in use_all, enforce it. */ cp = ff->fts_name; if (ff->fts_info == FTS_DP) { dform = FORM_NONE; dsec = NULL; break; } if ( ! strncmp(cp, "man", 3)) { dform = FORM_SRC; dsec = cp + 3; } else if ( ! strncmp(cp, "cat", 3)) { dform = FORM_CAT; dsec = cp + 3; } else { dform = FORM_NONE; dsec = NULL; } if (dsec != NULL || use_all) break; if (warnings) say(path, "Unknown directory part"); fts_set(f, ff, FTS_SKIP); break; case 2: /* * Possibly our architecture. * If we're descending, keep tabs on it. */ if (ff->fts_info != FTS_DP && dsec != NULL) arch = ff->fts_name; else arch = NULL; break; default: if (ff->fts_info == FTS_DP || use_all) break; if (warnings) say(path, "Extraneous directory part"); fts_set(f, ff, FTS_SKIP); break; } } fts_close(f); return 1; } /* * Add a file to the mlinks table. * Do not verify that it's a "valid" looking manpage (we'll do that * later). * * Try to infer the manual section, architecture, and page name from the * path, assuming it looks like * * [./]man*[/<arch>]/<name>.<section> * or * [./]cat<section>[/<arch>]/<name>.0 * * See treescan() for the fts(3) version of this. */ static void filescan(const char *file) { char buf[PATH_MAX]; struct stat st; struct mlink *mlink; char *p, *start; assert(use_all); if (0 == strncmp(file, "./", 2)) file += 2; /* * We have to do lstat(2) before realpath(3) loses * the information whether this is a symbolic link. * We need to know that because for symbolic links, * we want to use the orginal file name, while for * regular files, we want to use the real path. */ if (-1 == lstat(file, &st)) { exitcode = (int)MANDOCLEVEL_BADARG; say(file, "&lstat"); return; } else if (0 == ((S_IFREG | S_IFLNK) & st.st_mode)) { exitcode = (int)MANDOCLEVEL_BADARG; say(file, "Not a regular file"); return; } /* * We have to resolve the file name to the real path * in any case for the base directory check. */ if (NULL == realpath(file, buf)) { exitcode = (int)MANDOCLEVEL_BADARG; say(file, "&realpath"); return; } if (OP_TEST == op) start = buf; else if (strstr(buf, basedir) == buf) start = buf + strlen(basedir); #ifdef HOMEBREWDIR else if (strstr(buf, HOMEBREWDIR) == buf) start = buf; #endif else { exitcode = (int)MANDOCLEVEL_BADARG; say("", "%s: outside base directory", buf); return; } /* * Now we are sure the file is inside our tree. * If it is a symbolic link, ignore the real path * and use the original name. * This implies passing stuff like "cat1/../man1/foo.1" * on the command line won't work. So don't do that. * Note the stat(2) can still fail if the link target * doesn't exist. */ if (S_IFLNK & st.st_mode) { if (-1 == stat(buf, &st)) { exitcode = (int)MANDOCLEVEL_BADARG; say(file, "&stat"); return; } if (strlcpy(buf, file, sizeof(buf)) >= sizeof(buf)) { say(file, "Filename too long"); return; } start = buf; if (OP_TEST != op && strstr(buf, basedir) == buf) start += strlen(basedir); } mlink = mandoc_calloc(1, sizeof(struct mlink)); mlink->dform = FORM_NONE; if (strlcpy(mlink->file, start, sizeof(mlink->file)) >= sizeof(mlink->file)) { say(start, "Filename too long"); free(mlink); return; } /* * First try to guess our directory structure. * If we find a separator, try to look for man* or cat*. * If we find one of these and what's underneath is a directory, * assume it's an architecture. */ if (NULL != (p = strchr(start, '/'))) { *p++ = '\0'; if (0 == strncmp(start, "man", 3)) { mlink->dform = FORM_SRC; mlink->dsec = start + 3; } else if (0 == strncmp(start, "cat", 3)) { mlink->dform = FORM_CAT; mlink->dsec = start + 3; } start = p; if (NULL != mlink->dsec && NULL != (p = strchr(start, '/'))) { *p++ = '\0'; mlink->arch = start; start = p; } } /* * Now check the file suffix. * Suffix of `.0' indicates a catpage, `.1-9' is a manpage. */ p = strrchr(start, '\0'); while (p-- > start && '/' != *p && '.' != *p) /* Loop. */ ; if ('.' == *p) { *p++ = '\0'; mlink->fsec = p; } /* * Now try to parse the name. * Use the filename portion of the path. */ mlink->name = start; if (NULL != (p = strrchr(start, '/'))) { mlink->name = p + 1; *p = '\0'; } mlink_add(mlink, &st); } static void mlink_add(struct mlink *mlink, const struct stat *st) { struct inodev inodev; struct mpage *mpage; unsigned int slot; assert(NULL != mlink->file); mlink->dsec = mandoc_strdup(mlink->dsec ? mlink->dsec : ""); mlink->arch = mandoc_strdup(mlink->arch ? mlink->arch : ""); mlink->name = mandoc_strdup(mlink->name ? mlink->name : ""); mlink->fsec = mandoc_strdup(mlink->fsec ? mlink->fsec : ""); if ('0' == *mlink->fsec) { free(mlink->fsec); mlink->fsec = mandoc_strdup(mlink->dsec); mlink->fform = FORM_CAT; } else if ('1' <= *mlink->fsec && '9' >= *mlink->fsec) mlink->fform = FORM_SRC; else mlink->fform = FORM_NONE; slot = ohash_qlookup(&mlinks, mlink->file); assert(NULL == ohash_find(&mlinks, slot)); ohash_insert(&mlinks, slot, mlink); memset(&inodev, 0, sizeof(inodev)); /* Clear padding. */ inodev.st_ino = st->st_ino; inodev.st_dev = st->st_dev; slot = ohash_lookup_memory(&mpages, (char *)&inodev, sizeof(struct inodev), inodev.st_ino); mpage = ohash_find(&mpages, slot); if (NULL == mpage) { mpage = mandoc_calloc(1, sizeof(struct mpage)); mpage->inodev.st_ino = inodev.st_ino; mpage->inodev.st_dev = inodev.st_dev; ohash_insert(&mpages, slot, mpage); } else mlink->next = mpage->mlinks; mpage->mlinks = mlink; mlink->mpage = mpage; } static void mlink_free(struct mlink *mlink) { free(mlink->dsec); free(mlink->arch); free(mlink->name); free(mlink->fsec); free(mlink); } static void mpages_free(void) { struct mpage *mpage; struct mlink *mlink; unsigned int slot; mpage = ohash_first(&mpages, &slot); while (NULL != mpage) { while (NULL != (mlink = mpage->mlinks)) { mpage->mlinks = mlink->next; mlink_free(mlink); } free(mpage->sec); free(mpage->arch); free(mpage->title); free(mpage->desc); free(mpage); mpage = ohash_next(&mpages, &slot); } } /* * For each mlink to the mpage, check whether the path looks like * it is formatted, and if it does, check whether a source manual * exists by the same name, ignoring the suffix. * If both conditions hold, drop the mlink. */ static void mlinks_undupe(struct mpage *mpage) { char buf[PATH_MAX]; struct mlink **prev; struct mlink *mlink; char *bufp; mpage->form = FORM_CAT; prev = &mpage->mlinks; while (NULL != (mlink = *prev)) { if (FORM_CAT != mlink->dform) { mpage->form = FORM_NONE; goto nextlink; } (void)strlcpy(buf, mlink->file, sizeof(buf)); bufp = strstr(buf, "cat"); assert(NULL != bufp); memcpy(bufp, "man", 3); if (NULL != (bufp = strrchr(buf, '.'))) *++bufp = '\0'; (void)strlcat(buf, mlink->dsec, sizeof(buf)); if (NULL == ohash_find(&mlinks, ohash_qlookup(&mlinks, buf))) goto nextlink; if (warnings) say(mlink->file, "Man source exists: %s", buf); if (use_all) goto nextlink; *prev = mlink->next; mlink_free(mlink); continue; nextlink: prev = &(*prev)->next; } } static void mlink_check(struct mpage *mpage, struct mlink *mlink) { struct str *str; unsigned int slot; /* * Check whether the manual section given in a file * agrees with the directory where the file is located. * Some manuals have suffixes like (3p) on their * section number either inside the file or in the * directory name, some are linked into more than one * section, like encrypt(1) = makekey(8). */ if (FORM_SRC == mpage->form && strcasecmp(mpage->sec, mlink->dsec)) say(mlink->file, "Section \"%s\" manual in %s directory", mpage->sec, mlink->dsec); /* * Manual page directories exist for each kernel * architecture as returned by machine(1). * However, many manuals only depend on the * application architecture as returned by arch(1). * For example, some (2/ARM) manuals are shared * across the "armish" and "zaurus" kernel * architectures. * A few manuals are even shared across completely * different architectures, for example fdformat(1) * on amd64, i386, sparc, and sparc64. */ if (strcasecmp(mpage->arch, mlink->arch)) say(mlink->file, "Architecture \"%s\" manual in " "\"%s\" directory", mpage->arch, mlink->arch); /* * XXX * parse_cat() doesn't set NAME_TITLE yet. */ if (FORM_CAT == mpage->form) return; /* * Check whether this mlink * appears as a name in the NAME section. */ slot = ohash_qlookup(&names, mlink->name); str = ohash_find(&names, slot); assert(NULL != str); if ( ! (NAME_TITLE & str->mask)) say(mlink->file, "Name missing in NAME section"); } /* * Run through the files in the global vector "mpages" * and add them to the database specified in "basedir". * * This handles the parsing scheme itself, using the cues of directory * and filename to determine whether the file is parsable or not. */ static void mpages_merge(struct mparse *mp) { char any[] = "any"; struct mpage *mpage, *mpage_dest; struct mlink *mlink, *mlink_dest; struct roff_man *man; char *sodest; char *cp; int fd; unsigned int pslot; if ( ! nodb) SQL_EXEC("BEGIN TRANSACTION"); mpage = ohash_first(&mpages, &pslot); while (mpage != NULL) { mlinks_undupe(mpage); if ((mlink = mpage->mlinks) == NULL) { mpage = ohash_next(&mpages, &pslot); continue; } name_mask = NAME_MASK; mandoc_ohash_init(&names, 4, offsetof(struct str, key)); mandoc_ohash_init(&strings, 6, offsetof(struct str, key)); mparse_reset(mp); man = NULL; sodest = NULL; if ((fd = mparse_open(mp, mlink->file)) == -1) { say(mlink->file, "&open"); goto nextpage; } /* * Interpret the file as mdoc(7) or man(7) source * code, unless it is known to be formatted. */ if (mlink->dform != FORM_CAT || mlink->fform != FORM_CAT) { mparse_readfd(mp, fd, mlink->file); close(fd); mparse_result(mp, &man, &sodest); } if (sodest != NULL) { mlink_dest = ohash_find(&mlinks, ohash_qlookup(&mlinks, sodest)); if (mlink_dest == NULL) { mandoc_asprintf(&cp, "%s.gz", sodest); mlink_dest = ohash_find(&mlinks, ohash_qlookup(&mlinks, cp)); free(cp); } if (mlink_dest != NULL) { /* The .so target exists. */ mpage_dest = mlink_dest->mpage; while (1) { mlink->mpage = mpage_dest; /* * If the target was already * processed, add the links * to the database now. * Otherwise, this will * happen when we come * to the target. */ if (mpage_dest->pageid) dbadd_mlink_name(mlink); if (mlink->next == NULL) break; mlink = mlink->next; } /* Move all links to the target. */ mlink->next = mlink_dest->next; mlink_dest->next = mpage->mlinks; mpage->mlinks = NULL; } goto nextpage; } else if (man != NULL && man->macroset == MACROSET_MDOC) { mdoc_validate(man); mpage->form = FORM_SRC; mpage->sec = man->meta.msec; mpage->sec = mandoc_strdup( mpage->sec == NULL ? "" : mpage->sec); mpage->arch = man->meta.arch; mpage->arch = mandoc_strdup( mpage->arch == NULL ? "" : mpage->arch); mpage->title = mandoc_strdup(man->meta.title); } else if (man != NULL && man->macroset == MACROSET_MAN) { man_validate(man); mpage->form = FORM_SRC; mpage->sec = mandoc_strdup(man->meta.msec); mpage->arch = mandoc_strdup(mlink->arch); mpage->title = mandoc_strdup(man->meta.title); } else { mpage->form = FORM_CAT; mpage->sec = mandoc_strdup(mlink->dsec); mpage->arch = mandoc_strdup(mlink->arch); mpage->title = mandoc_strdup(mlink->name); } putkey(mpage, mpage->sec, TYPE_sec); if (*mpage->arch != '\0') putkey(mpage, mpage->arch, TYPE_arch); for ( ; mlink != NULL; mlink = mlink->next) { if ('\0' != *mlink->dsec) putkey(mpage, mlink->dsec, TYPE_sec); if ('\0' != *mlink->fsec) putkey(mpage, mlink->fsec, TYPE_sec); putkey(mpage, '\0' == *mlink->arch ? any : mlink->arch, TYPE_arch); putkey(mpage, mlink->name, NAME_FILE); } assert(mpage->desc == NULL); if (man != NULL && man->macroset == MACROSET_MDOC) parse_mdoc(mpage, &man->meta, man->first); else if (man != NULL) parse_man(mpage, &man->meta, man->first); else parse_cat(mpage, fd); if (mpage->desc == NULL) mpage->desc = mandoc_strdup(mpage->mlinks->name); if (warnings && !use_all) for (mlink = mpage->mlinks; mlink; mlink = mlink->next) mlink_check(mpage, mlink); dbadd(mpage); mlink = mpage->mlinks; nextpage: ohash_delete(&strings); ohash_delete(&names); mpage = ohash_next(&mpages, &pslot); } if (0 == nodb) SQL_EXEC("END TRANSACTION"); } static void names_check(void) { sqlite3_stmt *stmt; const char *name, *sec, *arch, *key; sqlite3_prepare_v2(db, "SELECT name, sec, arch, key FROM (" "SELECT name AS key, pageid FROM names " "WHERE bits & ? AND NOT EXISTS (" "SELECT pageid FROM mlinks " "WHERE mlinks.pageid == names.pageid " "AND mlinks.name == names.name" ")" ") JOIN (" "SELECT sec, arch, name, pageid FROM mlinks " "GROUP BY pageid" ") USING (pageid);", -1, &stmt, NULL); if (sqlite3_bind_int64(stmt, 1, NAME_TITLE) != SQLITE_OK) say("", "%s", sqlite3_errmsg(db)); while (sqlite3_step(stmt) == SQLITE_ROW) { name = (const char *)sqlite3_column_text(stmt, 0); sec = (const char *)sqlite3_column_text(stmt, 1); arch = (const char *)sqlite3_column_text(stmt, 2); key = (const char *)sqlite3_column_text(stmt, 3); say("", "%s(%s%s%s) lacks mlink \"%s\"", name, sec, '\0' == *arch ? "" : "/", '\0' == *arch ? "" : arch, key); } sqlite3_finalize(stmt); } static void parse_cat(struct mpage *mpage, int fd) { FILE *stream; char *line, *p, *title; size_t linesz, plen, titlesz; ssize_t len; int offs; stream = (-1 == fd) ? fopen(mpage->mlinks->file, "r") : fdopen(fd, "r"); if (NULL == stream) { if (-1 != fd) close(fd); if (warnings) say(mpage->mlinks->file, "&fopen"); return; } line = NULL; linesz = 0; /* Skip to first blank line. */ while (getline(&line, &linesz, stream) != -1) if (*line == '\n') break; /* * Assume the first line that is not indented * is the first section header. Skip to it. */ while (getline(&line, &linesz, stream) != -1) if (*line != '\n' && *line != ' ') break; /* * Read up until the next section into a buffer. * Strip the leading and trailing newline from each read line, * appending a trailing space. * Ignore empty (whitespace-only) lines. */ titlesz = 0; title = NULL; while ((len = getline(&line, &linesz, stream)) != -1) { if (*line != ' ') break; offs = 0; while (isspace((unsigned char)line[offs])) offs++; if (line[offs] == '\0') continue; title = mandoc_realloc(title, titlesz + len - offs); memcpy(title + titlesz, line + offs, len - offs); titlesz += len - offs; title[titlesz - 1] = ' '; } free(line); /* * If no page content can be found, or the input line * is already the next section header, or there is no * trailing newline, reuse the page title as the page * description. */ if (NULL == title || '\0' == *title) { if (warnings) say(mpage->mlinks->file, "Cannot find NAME section"); fclose(stream); free(title); return; } title[titlesz - 1] = '\0'; /* * Skip to the first dash. * Use the remaining line as the description (no more than 70 * bytes). */ if (NULL != (p = strstr(title, "- "))) { for (p += 2; ' ' == *p || '\b' == *p; p++) /* Skip to next word. */ ; } else { if (warnings) say(mpage->mlinks->file, "No dash in title line"); p = title; } plen = strlen(p); /* Strip backspace-encoding from line. */ while (NULL != (line = memchr(p, '\b', plen))) { len = line - p; if (0 == len) { memmove(line, line + 1, plen--); continue; } memmove(line - 1, line + 1, plen - len); plen -= 2; } mpage->desc = mandoc_strdup(p); fclose(stream); free(title); } /* * Put a type/word pair into the word database for this particular file. */ static void putkey(const struct mpage *mpage, char *value, uint64_t type) { char *cp; assert(NULL != value); if (TYPE_arch == type) for (cp = value; *cp; cp++) if (isupper((unsigned char)*cp)) *cp = _tolower((unsigned char)*cp); putkeys(mpage, value, strlen(value), type); } /* * Grok all nodes at or below a certain mdoc node into putkey(). */ static void putmdockey(const struct mpage *mpage, const struct roff_node *n, uint64_t m) { for ( ; NULL != n; n = n->next) { if (NULL != n->child) putmdockey(mpage, n->child, m); if (n->type == ROFFT_TEXT) putkey(mpage, n->string, m); } } static void parse_man(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { const struct roff_node *head, *body; char *start, *title; char byte; size_t sz; if (n == NULL) return; /* * We're only searching for one thing: the first text child in * the BODY of a NAME section. Since we don't keep track of * sections in -man, run some hoops to find out whether we're in * the correct section or not. */ if (n->type == ROFFT_BODY && n->tok == MAN_SH) { body = n; if ((head = body->parent->head) != NULL && (head = head->child) != NULL && head->next == NULL && head->type == ROFFT_TEXT && strcmp(head->string, "NAME") == 0 && body->child != NULL) { /* * Suck the entire NAME section into memory. * Yes, we might run away. * But too many manuals have big, spread-out * NAME sections over many lines. */ title = NULL; deroff(&title, body); if (NULL == title) return; /* * Go through a special heuristic dance here. * Conventionally, one or more manual names are * comma-specified prior to a whitespace, then a * dash, then a description. Try to puzzle out * the name parts here. */ start = title; for ( ;; ) { sz = strcspn(start, " ,"); if ('\0' == start[sz]) break; byte = start[sz]; start[sz] = '\0'; /* * Assume a stray trailing comma in the * name list if a name begins with a dash. */ if ('-' == start[0] || ('\\' == start[0] && '-' == start[1])) break; putkey(mpage, start, NAME_TITLE); if ( ! (mpage->name_head_done || strcasecmp(start, meta->title))) { putkey(mpage, start, NAME_HEAD); mpage->name_head_done = 1; } if (' ' == byte) { start += sz + 1; break; } assert(',' == byte); start += sz + 1; while (' ' == *start) start++; } if (start == title) { putkey(mpage, start, NAME_TITLE); if ( ! (mpage->name_head_done || strcasecmp(start, meta->title))) { putkey(mpage, start, NAME_HEAD); mpage->name_head_done = 1; } free(title); return; } while (isspace((unsigned char)*start)) start++; if (0 == strncmp(start, "-", 1)) start += 1; else if (0 == strncmp(start, "\\-\\-", 4)) start += 4; else if (0 == strncmp(start, "\\-", 2)) start += 2; else if (0 == strncmp(start, "\\(en", 4)) start += 4; else if (0 == strncmp(start, "\\(em", 4)) start += 4; while (' ' == *start) start++; mpage->desc = mandoc_strdup(start); free(title); return; } } for (n = n->child; n; n = n->next) { if (NULL != mpage->desc) break; parse_man(mpage, meta, n); } } static void parse_mdoc(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { assert(NULL != n); for (n = n->child; NULL != n; n = n->next) { switch (n->type) { case ROFFT_ELEM: case ROFFT_BLOCK: case ROFFT_HEAD: case ROFFT_BODY: case ROFFT_TAIL: if (NULL != mdocs[n->tok].fp) if (0 == (*mdocs[n->tok].fp)(mpage, meta, n)) break; if (mdocs[n->tok].mask) putmdockey(mpage, n->child, mdocs[n->tok].mask); break; default: assert(n->type != ROFFT_ROOT); continue; } if (NULL != n->child) parse_mdoc(mpage, meta, n); } } static int parse_mdoc_Fd(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { char *start, *end; size_t sz; if (SEC_SYNOPSIS != n->sec || NULL == (n = n->child) || n->type != ROFFT_TEXT) return 0; /* * Only consider those `Fd' macro fields that begin with an * "inclusion" token (versus, e.g., #define). */ if (strcmp("#include", n->string)) return 0; if ((n = n->next) == NULL || n->type != ROFFT_TEXT) return 0; /* * Strip away the enclosing angle brackets and make sure we're * not zero-length. */ start = n->string; if ('<' == *start || '"' == *start) start++; if (0 == (sz = strlen(start))) return 0; end = &start[(int)sz - 1]; if ('>' == *end || '"' == *end) end--; if (end > start) putkeys(mpage, start, end - start + 1, TYPE_In); return 0; } static void parse_mdoc_fname(struct mpage *mpage, const struct roff_node *n) { char *cp; size_t sz; if (n->type != ROFFT_TEXT) return; /* Skip function pointer punctuation. */ cp = n->string; while (*cp == '(' || *cp == '*') cp++; sz = strcspn(cp, "()"); putkeys(mpage, cp, sz, TYPE_Fn); if (n->sec == SEC_SYNOPSIS) putkeys(mpage, cp, sz, NAME_SYN); } static int parse_mdoc_Fn(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { if (n->child == NULL) return 0; parse_mdoc_fname(mpage, n->child); for (n = n->child->next; n != NULL; n = n->next) if (n->type == ROFFT_TEXT) putkey(mpage, n->string, TYPE_Fa); return 0; } static int parse_mdoc_Fo(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { if (n->type != ROFFT_HEAD) return 1; if (n->child != NULL) parse_mdoc_fname(mpage, n->child); return 0; } static int parse_mdoc_Va(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { char *cp; if (n->type != ROFFT_ELEM && n->type != ROFFT_BODY) return 0; if (n->child != NULL && n->child->next == NULL && n->child->type == ROFFT_TEXT) return 1; cp = NULL; deroff(&cp, n); if (cp != NULL) { putkey(mpage, cp, TYPE_Vt | (n->tok == MDOC_Va || n->type == ROFFT_BODY ? TYPE_Va : 0)); free(cp); } return 0; } static int parse_mdoc_Xr(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { char *cp; if (NULL == (n = n->child)) return 0; if (NULL == n->next) { putkey(mpage, n->string, TYPE_Xr); return 0; } mandoc_asprintf(&cp, "%s(%s)", n->string, n->next->string); putkey(mpage, cp, TYPE_Xr); free(cp); return 0; } static int parse_mdoc_Nd(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { if (n->type == ROFFT_BODY) deroff(&mpage->desc, n); return 0; } static int parse_mdoc_Nm(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { if (SEC_NAME == n->sec) putmdockey(mpage, n->child, NAME_TITLE); else if (n->sec == SEC_SYNOPSIS && n->type == ROFFT_HEAD) { if (n->child == NULL) putkey(mpage, meta->name, NAME_SYN); else putmdockey(mpage, n->child, NAME_SYN); } if ( ! (mpage->name_head_done || n->child == NULL || n->child->string == NULL || strcasecmp(n->child->string, meta->title))) { putkey(mpage, n->child->string, ROFFT_HEAD); mpage->name_head_done = 1; } return 0; } static int parse_mdoc_Sh(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { return n->sec == SEC_CUSTOM && n->type == ROFFT_HEAD; } static int parse_mdoc_head(struct mpage *mpage, const struct roff_meta *meta, const struct roff_node *n) { return n->type == ROFFT_HEAD; } /* * Add a string to the hash table for the current manual. * Each string has a bitmask telling which macros it belongs to. * When we finish the manual, we'll dump the table. */ static void putkeys(const struct mpage *mpage, char *cp, size_t sz, uint64_t v) { struct ohash *htab; struct str *s; const char *end; unsigned int slot; int i, mustfree; if (0 == sz) return; mustfree = render_string(&cp, &sz); if (TYPE_Nm & v) { htab = &names; v &= name_mask; if (v & NAME_FIRST) name_mask &= ~NAME_FIRST; if (debug > 1) say(mpage->mlinks->file, "Adding name %*s, bits=%d", sz, cp, v); } else { htab = &strings; if (debug > 1) for (i = 0; i < mansearch_keymax; i++) if ((uint64_t)1 << i & v) say(mpage->mlinks->file, "Adding key %s=%*s", mansearch_keynames[i], sz, cp); } end = cp + sz; slot = ohash_qlookupi(htab, cp, &end); s = ohash_find(htab, slot); if (NULL != s && mpage == s->mpage) { s->mask |= v; return; } else if (NULL == s) { s = mandoc_calloc(1, sizeof(struct str) + sz + 1); memcpy(s->key, cp, sz); ohash_insert(htab, slot, s); } s->mpage = mpage; s->mask = v; if (mustfree) free(cp); } /* * Take a Unicode codepoint and produce its UTF-8 encoding. * This isn't the best way to do this, but it works. * The magic numbers are from the UTF-8 packaging. * They're not as scary as they seem: read the UTF-8 spec for details. */ static size_t utf8(unsigned int cp, char out[7]) { size_t rc; rc = 0; if (cp <= 0x0000007F) { rc = 1; out[0] = (char)cp; } else if (cp <= 0x000007FF) { rc = 2; out[0] = (cp >> 6 & 31) | 192; out[1] = (cp & 63) | 128; } else if (cp <= 0x0000FFFF) { rc = 3; out[0] = (cp >> 12 & 15) | 224; out[1] = (cp >> 6 & 63) | 128; out[2] = (cp & 63) | 128; } else if (cp <= 0x001FFFFF) { rc = 4; out[0] = (cp >> 18 & 7) | 240; out[1] = (cp >> 12 & 63) | 128; out[2] = (cp >> 6 & 63) | 128; out[3] = (cp & 63) | 128; } else if (cp <= 0x03FFFFFF) { rc = 5; out[0] = (cp >> 24 & 3) | 248; out[1] = (cp >> 18 & 63) | 128; out[2] = (cp >> 12 & 63) | 128; out[3] = (cp >> 6 & 63) | 128; out[4] = (cp & 63) | 128; } else if (cp <= 0x7FFFFFFF) { rc = 6; out[0] = (cp >> 30 & 1) | 252; out[1] = (cp >> 24 & 63) | 128; out[2] = (cp >> 18 & 63) | 128; out[3] = (cp >> 12 & 63) | 128; out[4] = (cp >> 6 & 63) | 128; out[5] = (cp & 63) | 128; } else return 0; out[rc] = '\0'; return rc; } /* * If the string contains escape sequences, * replace it with an allocated rendering and return 1, * such that the caller can free it after use. * Otherwise, do nothing and return 0. */ static int render_string(char **public, size_t *psz) { const char *src, *scp, *addcp, *seq; char *dst; size_t ssz, dsz, addsz; char utfbuf[7], res[6]; int seqlen, unicode; res[0] = '\\'; res[1] = '\t'; res[2] = ASCII_NBRSP; res[3] = ASCII_HYPH; res[4] = ASCII_BREAK; res[5] = '\0'; src = scp = *public; ssz = *psz; dst = NULL; dsz = 0; while (scp < src + *psz) { /* Leave normal characters unchanged. */ if (strchr(res, *scp) == NULL) { if (dst != NULL) dst[dsz++] = *scp; scp++; continue; } /* * Found something that requires replacing, * make sure we have a destination buffer. */ if (dst == NULL) { dst = mandoc_malloc(ssz + 1); dsz = scp - src; memcpy(dst, src, dsz); } /* Handle single-char special characters. */ switch (*scp) { case '\\': break; case '\t': case ASCII_NBRSP: dst[dsz++] = ' '; scp++; continue; case ASCII_HYPH: dst[dsz++] = '-'; /* FALLTHROUGH */ case ASCII_BREAK: scp++; continue; default: abort(); } /* * Found an escape sequence. * Read past the slash, then parse it. * Ignore everything except characters. */ scp++; if (mandoc_escape(&scp, &seq, &seqlen) != ESCAPE_SPECIAL) continue; /* * Render the special character * as either UTF-8 or ASCII. */ if (write_utf8) { unicode = mchars_spec2cp(seq, seqlen); if (unicode <= 0) continue; addsz = utf8(unicode, utfbuf); if (addsz == 0) continue; addcp = utfbuf; } else { addcp = mchars_spec2str(seq, seqlen, &addsz); if (addcp == NULL) continue; if (*addcp == ASCII_NBRSP) { addcp = " "; addsz = 1; } } /* Copy the rendered glyph into the stream. */ ssz += addsz; dst = mandoc_realloc(dst, ssz + 1); memcpy(dst + dsz, addcp, addsz); dsz += addsz; } if (dst != NULL) { *public = dst; *psz = dsz; } /* Trim trailing whitespace and NUL-terminate. */ while (*psz > 0 && (*public)[*psz - 1] == ' ') --*psz; if (dst != NULL) { (*public)[*psz] = '\0'; return 1; } else return 0; } static void dbadd_mlink(const struct mlink *mlink) { size_t i; i = 1; SQL_BIND_TEXT(stmts[STMT_INSERT_LINK], i, mlink->dsec); SQL_BIND_TEXT(stmts[STMT_INSERT_LINK], i, mlink->arch); SQL_BIND_TEXT(stmts[STMT_INSERT_LINK], i, mlink->name); SQL_BIND_INT64(stmts[STMT_INSERT_LINK], i, mlink->mpage->pageid); SQL_STEP(stmts[STMT_INSERT_LINK]); sqlite3_reset(stmts[STMT_INSERT_LINK]); } static void dbadd_mlink_name(const struct mlink *mlink) { uint64_t bits; size_t i; dbadd_mlink(mlink); i = 1; SQL_BIND_INT64(stmts[STMT_SELECT_NAME], i, mlink->mpage->pageid); bits = NAME_FILE & NAME_MASK; if (sqlite3_step(stmts[STMT_SELECT_NAME]) == SQLITE_ROW) { bits |= sqlite3_column_int64(stmts[STMT_SELECT_NAME], 0); sqlite3_reset(stmts[STMT_SELECT_NAME]); } i = 1; SQL_BIND_INT64(stmts[STMT_INSERT_NAME], i, bits); SQL_BIND_TEXT(stmts[STMT_INSERT_NAME], i, mlink->name); SQL_BIND_INT64(stmts[STMT_INSERT_NAME], i, mlink->mpage->pageid); SQL_STEP(stmts[STMT_INSERT_NAME]); sqlite3_reset(stmts[STMT_INSERT_NAME]); } /* * Flush the current page's terms (and their bits) into the database. * Wrap the entire set of additions in a transaction to make sqlite be a * little faster. * Also, handle escape sequences at the last possible moment. */ static void dbadd(struct mpage *mpage) { struct mlink *mlink; struct str *key; char *cp; size_t i; unsigned int slot; int mustfree; mlink = mpage->mlinks; if (nodb) { for (key = ohash_first(&names, &slot); NULL != key; key = ohash_next(&names, &slot)) free(key); for (key = ohash_first(&strings, &slot); NULL != key; key = ohash_next(&strings, &slot)) free(key); if (0 == debug) return; while (NULL != mlink) { fputs(mlink->name, stdout); if (NULL == mlink->next || strcmp(mlink->dsec, mlink->next->dsec) || strcmp(mlink->fsec, mlink->next->fsec) || strcmp(mlink->arch, mlink->next->arch)) { putchar('('); if ('\0' == *mlink->dsec) fputs(mlink->fsec, stdout); else fputs(mlink->dsec, stdout); if ('\0' != *mlink->arch) printf("/%s", mlink->arch); putchar(')'); } mlink = mlink->next; if (NULL != mlink) fputs(", ", stdout); } printf(" - %s\n", mpage->desc); return; } if (debug) say(mlink->file, "Adding to database"); cp = mpage->desc; i = strlen(cp); mustfree = render_string(&cp, &i); i = 1; SQL_BIND_TEXT(stmts[STMT_INSERT_PAGE], i, cp); SQL_BIND_INT(stmts[STMT_INSERT_PAGE], i, mpage->form); SQL_STEP(stmts[STMT_INSERT_PAGE]); mpage->pageid = sqlite3_last_insert_rowid(db); sqlite3_reset(stmts[STMT_INSERT_PAGE]); if (mustfree) free(cp); while (NULL != mlink) { dbadd_mlink(mlink); mlink = mlink->next; } mlink = mpage->mlinks; for (key = ohash_first(&names, &slot); NULL != key; key = ohash_next(&names, &slot)) { assert(key->mpage == mpage); i = 1; SQL_BIND_INT64(stmts[STMT_INSERT_NAME], i, key->mask); SQL_BIND_TEXT(stmts[STMT_INSERT_NAME], i, key->key); SQL_BIND_INT64(stmts[STMT_INSERT_NAME], i, mpage->pageid); SQL_STEP(stmts[STMT_INSERT_NAME]); sqlite3_reset(stmts[STMT_INSERT_NAME]); free(key); } for (key = ohash_first(&strings, &slot); NULL != key; key = ohash_next(&strings, &slot)) { assert(key->mpage == mpage); i = 1; SQL_BIND_INT64(stmts[STMT_INSERT_KEY], i, key->mask); SQL_BIND_TEXT(stmts[STMT_INSERT_KEY], i, key->key); SQL_BIND_INT64(stmts[STMT_INSERT_KEY], i, mpage->pageid); SQL_STEP(stmts[STMT_INSERT_KEY]); sqlite3_reset(stmts[STMT_INSERT_KEY]); free(key); } } static void dbprune(void) { struct mpage *mpage; struct mlink *mlink; size_t i; unsigned int slot; if (0 == nodb) SQL_EXEC("BEGIN TRANSACTION"); for (mpage = ohash_first(&mpages, &slot); NULL != mpage; mpage = ohash_next(&mpages, &slot)) { mlink = mpage->mlinks; if (debug) say(mlink->file, "Deleting from database"); if (nodb) continue; for ( ; NULL != mlink; mlink = mlink->next) { i = 1; SQL_BIND_TEXT(stmts[STMT_DELETE_PAGE], i, mlink->dsec); SQL_BIND_TEXT(stmts[STMT_DELETE_PAGE], i, mlink->arch); SQL_BIND_TEXT(stmts[STMT_DELETE_PAGE], i, mlink->name); SQL_STEP(stmts[STMT_DELETE_PAGE]); sqlite3_reset(stmts[STMT_DELETE_PAGE]); } } if (0 == nodb) SQL_EXEC("END TRANSACTION"); } /* * Close an existing database and its prepared statements. * If "real" is not set, rename the temporary file into the real one. */ static void dbclose(int real) { size_t i; int status; pid_t child; if (nodb) return; for (i = 0; i < STMT__MAX; i++) { sqlite3_finalize(stmts[i]); stmts[i] = NULL; } sqlite3_close(db); db = NULL; if (real) return; if ('\0' == *tempfilename) { if (-1 == rename(MANDOC_DB "~", MANDOC_DB)) { exitcode = (int)MANDOCLEVEL_SYSERR; say(MANDOC_DB, "&rename"); } return; } switch (child = fork()) { case -1: exitcode = (int)MANDOCLEVEL_SYSERR; say("", "&fork cmp"); return; case 0: execlp("cmp", "cmp", "-s", tempfilename, MANDOC_DB, (char *)NULL); say("", "&exec cmp"); exit(0); default: break; } if (-1 == waitpid(child, &status, 0)) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "&wait cmp"); } else if (WIFSIGNALED(status)) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "cmp died from signal %d", WTERMSIG(status)); } else if (WEXITSTATUS(status)) { exitcode = (int)MANDOCLEVEL_SYSERR; say(MANDOC_DB, "Data changed, but cannot replace database"); } *strrchr(tempfilename, '/') = '\0'; switch (child = fork()) { case -1: exitcode = (int)MANDOCLEVEL_SYSERR; say("", "&fork rm"); return; case 0: execlp("rm", "rm", "-rf", tempfilename, (char *)NULL); say("", "&exec rm"); exit((int)MANDOCLEVEL_SYSERR); default: break; } if (-1 == waitpid(child, &status, 0)) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "&wait rm"); } else if (WIFSIGNALED(status) || WEXITSTATUS(status)) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "%s: Cannot remove temporary directory", tempfilename); } } /* * This is straightforward stuff. * Open a database connection to a "temporary" database, then open a set * of prepared statements we'll use over and over again. * If "real" is set, we use the existing database; if not, we truncate a * temporary one. * Must be matched by dbclose(). */ static int dbopen(int real) { const char *sql; int rc, ofl; if (nodb) return 1; *tempfilename = '\0'; ofl = SQLITE_OPEN_READWRITE; if (real) { rc = sqlite3_open_v2(MANDOC_DB, &db, ofl, NULL); if (SQLITE_OK != rc) { exitcode = (int)MANDOCLEVEL_SYSERR; if (SQLITE_CANTOPEN != rc) say(MANDOC_DB, "%s", sqlite3_errstr(rc)); return 0; } goto prepare_statements; } ofl |= SQLITE_OPEN_CREATE | SQLITE_OPEN_EXCLUSIVE; remove(MANDOC_DB "~"); rc = sqlite3_open_v2(MANDOC_DB "~", &db, ofl, NULL); if (SQLITE_OK == rc) goto create_tables; if (MPARSE_QUICK & mparse_options) { exitcode = (int)MANDOCLEVEL_SYSERR; say(MANDOC_DB "~", "%s", sqlite3_errstr(rc)); return 0; } (void)strlcpy(tempfilename, "/tmp/mandocdb.XXXXXX", sizeof(tempfilename)); if (NULL == mkdtemp(tempfilename)) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "&%s", tempfilename); return 0; } (void)strlcat(tempfilename, "/" MANDOC_DB, sizeof(tempfilename)); rc = sqlite3_open_v2(tempfilename, &db, ofl, NULL); if (SQLITE_OK != rc) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "%s: %s", tempfilename, sqlite3_errstr(rc)); return 0; } create_tables: sql = "CREATE TABLE \"mpages\" (\n" " \"desc\" TEXT NOT NULL,\n" " \"form\" INTEGER NOT NULL,\n" " \"pageid\" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL\n" ");\n" "\n" "CREATE TABLE \"mlinks\" (\n" " \"sec\" TEXT NOT NULL,\n" " \"arch\" TEXT NOT NULL,\n" " \"name\" TEXT NOT NULL,\n" " \"pageid\" INTEGER NOT NULL REFERENCES mpages(pageid) " "ON DELETE CASCADE\n" ");\n" "CREATE INDEX mlinks_pageid_idx ON mlinks (pageid);\n" "\n" "CREATE TABLE \"names\" (\n" " \"bits\" INTEGER NOT NULL,\n" " \"name\" TEXT NOT NULL,\n" " \"pageid\" INTEGER NOT NULL REFERENCES mpages(pageid) " "ON DELETE CASCADE,\n" " UNIQUE (\"name\", \"pageid\") ON CONFLICT REPLACE\n" ");\n" "\n" "CREATE TABLE \"keys\" (\n" " \"bits\" INTEGER NOT NULL,\n" " \"key\" TEXT NOT NULL,\n" " \"pageid\" INTEGER NOT NULL REFERENCES mpages(pageid) " "ON DELETE CASCADE\n" ");\n" "CREATE INDEX keys_pageid_idx ON keys (pageid);\n"; if (SQLITE_OK != sqlite3_exec(db, sql, NULL, NULL, NULL)) { exitcode = (int)MANDOCLEVEL_SYSERR; say(MANDOC_DB, "%s", sqlite3_errmsg(db)); sqlite3_close(db); return 0; } prepare_statements: if (SQLITE_OK != sqlite3_exec(db, "PRAGMA foreign_keys = ON", NULL, NULL, NULL)) { exitcode = (int)MANDOCLEVEL_SYSERR; say(MANDOC_DB, "PRAGMA foreign_keys: %s", sqlite3_errmsg(db)); sqlite3_close(db); return 0; } sql = "DELETE FROM mpages WHERE pageid IN " "(SELECT pageid FROM mlinks WHERE " "sec=? AND arch=? AND name=?)"; sqlite3_prepare_v2(db, sql, -1, &stmts[STMT_DELETE_PAGE], NULL); sql = "INSERT INTO mpages " "(desc,form) VALUES (?,?)"; sqlite3_prepare_v2(db, sql, -1, &stmts[STMT_INSERT_PAGE], NULL); sql = "INSERT INTO mlinks " "(sec,arch,name,pageid) VALUES (?,?,?,?)"; sqlite3_prepare_v2(db, sql, -1, &stmts[STMT_INSERT_LINK], NULL); sql = "SELECT bits FROM names where pageid = ?"; sqlite3_prepare_v2(db, sql, -1, &stmts[STMT_SELECT_NAME], NULL); sql = "INSERT INTO names " "(bits,name,pageid) VALUES (?,?,?)"; sqlite3_prepare_v2(db, sql, -1, &stmts[STMT_INSERT_NAME], NULL); sql = "INSERT INTO keys " "(bits,key,pageid) VALUES (?,?,?)"; sqlite3_prepare_v2(db, sql, -1, &stmts[STMT_INSERT_KEY], NULL); #ifndef __APPLE__ /* * When opening a new database, we can turn off * synchronous mode for much better performance. */ if (real && SQLITE_OK != sqlite3_exec(db, "PRAGMA synchronous = OFF", NULL, NULL, NULL)) { exitcode = (int)MANDOCLEVEL_SYSERR; say(MANDOC_DB, "PRAGMA synchronous: %s", sqlite3_errmsg(db)); sqlite3_close(db); return 0; } #endif return 1; } static int set_basedir(const char *targetdir, int report_baddir) { static char startdir[PATH_MAX]; static int getcwd_status; /* 1 = ok, 2 = failure */ static int chdir_status; /* 1 = changed directory */ char *cp; /* * Remember the original working directory, if possible. * This will be needed if the second or a later directory * on the command line is given as a relative path. * Do not error out if the current directory is not * searchable: Maybe it won't be needed after all. */ if (0 == getcwd_status) { if (NULL == getcwd(startdir, sizeof(startdir))) { getcwd_status = 2; (void)strlcpy(startdir, strerror(errno), sizeof(startdir)); } else getcwd_status = 1; } /* * We are leaving the old base directory. * Do not use it any longer, not even for messages. */ *basedir = '\0'; /* * If and only if the directory was changed earlier and * the next directory to process is given as a relative path, * first go back, or bail out if that is impossible. */ if (chdir_status && '/' != *targetdir) { if (2 == getcwd_status) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "getcwd: %s", startdir); return 0; } if (-1 == chdir(startdir)) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "&chdir %s", startdir); return 0; } } /* * Always resolve basedir to the canonicalized absolute * pathname and append a trailing slash, such that * we can reliably check whether files are inside. */ if (NULL == realpath(targetdir, basedir)) { if (report_baddir || errno != ENOENT) { exitcode = (int)MANDOCLEVEL_BADARG; say("", "&%s: realpath", targetdir); } return 0; } else if (-1 == chdir(basedir)) { if (report_baddir || errno != ENOENT) { exitcode = (int)MANDOCLEVEL_BADARG; say("", "&chdir"); } return 0; } chdir_status = 1; cp = strchr(basedir, '\0'); if ('/' != cp[-1]) { if (cp - basedir >= PATH_MAX - 1) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "Filename too long"); return 0; } *cp++ = '/'; *cp = '\0'; } return 1; } static void say(const char *file, const char *format, ...) { va_list ap; int use_errno; if ('\0' != *basedir) fprintf(stderr, "%s", basedir); if ('\0' != *basedir && '\0' != *file) fputc('/', stderr); if ('\0' != *file) fprintf(stderr, "%s", file); use_errno = 1; if (NULL != format) { switch (*format) { case '&': format++; break; case '\0': format = NULL; break; default: use_errno = 0; break; } } if (NULL != format) { if ('\0' != *basedir || '\0' != *file) fputs(": ", stderr); va_start(ap, format); vfprintf(stderr, format, ap); va_end(ap); } if (use_errno) { if ('\0' != *basedir || '\0' != *file || NULL != format) fputs(": ", stderr); perror(NULL); } else fputc('\n', stderr); } Index: vendor/mdocml/dist/manpage.c =================================================================== --- vendor/mdocml/dist/manpage.c (revision 303220) +++ vendor/mdocml/dist/manpage.c (revision 303221) @@ -1,196 +1,195 @@ -/* $Id: manpage.c,v 1.13 2015/11/07 17:58:55 schwarze Exp $ */ +/* $Id: manpage.c,v 1.14 2016/07/09 15:24:19 schwarze Exp $ */ /* * Copyright (c) 2012 Kristaps Dzonsons <kristaps@bsd.lv> * Copyright (c) 2013 Ingo Schwarze <schwarze@openbsd.org> * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include "config.h" #include <sys/types.h> #include <assert.h> -#include <getopt.h> #include <limits.h> #include <stdint.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include "manconf.h" #include "mansearch.h" static void show(const char *, const char *); int main(int argc, char *argv[]) { int ch, term; size_t i, sz, linesz; ssize_t len; struct mansearch search; struct manpage *res; char *conf_file, *defpaths, *auxpaths, *line; char buf[PATH_MAX]; const char *cmd; struct manconf conf; char *progname; extern char *optarg; extern int optind; term = isatty(STDIN_FILENO) && isatty(STDOUT_FILENO); progname = strrchr(argv[0], '/'); if (progname == NULL) progname = argv[0]; else ++progname; auxpaths = defpaths = conf_file = NULL; memset(&conf, 0, sizeof(conf)); memset(&search, 0, sizeof(struct mansearch)); while (-1 != (ch = getopt(argc, argv, "C:M:m:S:s:"))) switch (ch) { case ('C'): conf_file = optarg; break; case ('M'): defpaths = optarg; break; case ('m'): auxpaths = optarg; break; case ('S'): search.arch = optarg; break; case ('s'): search.sec = optarg; break; default: goto usage; } argc -= optind; argv += optind; if (0 == argc) goto usage; search.outkey = "Nd"; search.argmode = ARG_EXPR; manconf_parse(&conf, conf_file, defpaths, auxpaths); ch = mansearch(&search, &conf.manpath, argc, argv, &res, &sz); manconf_free(&conf); if (0 == ch) goto usage; if (0 == sz) { free(res); return EXIT_FAILURE; } else if (1 == sz && term) { i = 1; goto show; } else if (NULL == res) return EXIT_FAILURE; for (i = 0; i < sz; i++) { printf("%6zu %s: %s\n", i + 1, res[i].names, res[i].output); free(res[i].names); free(res[i].output); } if (0 == term) { for (i = 0; i < sz; i++) free(res[i].file); free(res); return EXIT_SUCCESS; } i = 1; printf("Enter a choice [1]: "); fflush(stdout); line = NULL; linesz = 0; if ((len = getline(&line, &linesz, stdin)) != -1) { if ('\n' == line[--len] && len > 0) { line[len] = '\0'; if ((i = atoi(line)) < 1 || i > sz) i = 0; } } free(line); if (0 == i) { for (i = 0; i < sz; i++) free(res[i].file); free(res); return EXIT_SUCCESS; } show: cmd = res[i - 1].form ? "mandoc" : "cat"; strlcpy(buf, res[i - 1].file, PATH_MAX); for (i = 0; i < sz; i++) free(res[i].file); free(res); show(cmd, buf); /* NOTREACHED */ usage: fprintf(stderr, "usage: %s [-C conf] " "[-M paths] " "[-m paths] " "[-S arch] " "[-s section] " "expr ...\n", progname); return EXIT_FAILURE; } static void show(const char *cmd, const char *file) { int fds[2]; pid_t pid; if (-1 == pipe(fds)) { perror(NULL); exit(EXIT_FAILURE); } if (-1 == (pid = fork())) { perror(NULL); exit(EXIT_FAILURE); } else if (pid > 0) { dup2(fds[0], STDIN_FILENO); close(fds[1]); cmd = NULL != getenv("MANPAGER") ? getenv("MANPAGER") : (NULL != getenv("PAGER") ? getenv("PAGER") : "more"); execlp(cmd, cmd, (char *)NULL); perror(cmd); exit(EXIT_FAILURE); } dup2(fds[1], STDOUT_FILENO); close(fds[0]); execlp(cmd, cmd, file, (char *)NULL); perror(cmd); exit(EXIT_FAILURE); } Index: vendor/mdocml/dist/manpath.c =================================================================== --- vendor/mdocml/dist/manpath.c (revision 303220) +++ vendor/mdocml/dist/manpath.c (revision 303221) @@ -1,336 +1,335 @@ -/* $Id: manpath.c,v 1.29 2015/11/07 17:58:55 schwarze Exp $ */ +/* $Id: manpath.c,v 1.30 2016/05/28 13:44:13 schwarze Exp $ */ /* * Copyright (c) 2011, 2014, 2015 Ingo Schwarze <schwarze@openbsd.org> * Copyright (c) 2011 Kristaps Dzonsons <kristaps@bsd.lv> * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHORS DISCLAIM ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include "config.h" #include <sys/types.h> #include <sys/stat.h> #include <ctype.h> #if HAVE_ERR #include <err.h> #endif #include <limits.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include "mandoc_aux.h" #include "manconf.h" #if !HAVE_MANPATH static void manconf_file(struct manconf *, const char *); #endif static void manpath_add(struct manpaths *, const char *, int); static void manpath_parseline(struct manpaths *, char *, int); void manconf_parse(struct manconf *conf, const char *file, char *defp, char *auxp) { #if HAVE_MANPATH char cmd[(PATH_MAX * 3) + 20]; FILE *stream; char *buf; size_t sz, bsz; strlcpy(cmd, "manpath", sizeof(cmd)); if (file) { strlcat(cmd, " -C ", sizeof(cmd)); strlcat(cmd, file, sizeof(cmd)); } if (auxp) { strlcat(cmd, " -m ", sizeof(cmd)); strlcat(cmd, auxp, sizeof(cmd)); } if (defp) { strlcat(cmd, " -M ", sizeof(cmd)); strlcat(cmd, defp, sizeof(cmd)); } /* Open manpath(1). Ignore errors. */ stream = popen(cmd, "r"); if (NULL == stream) return; buf = NULL; bsz = 0; /* Read in as much output as we can. */ do { buf = mandoc_realloc(buf, bsz + 1024); sz = fread(buf + bsz, 1, 1024, stream); bsz += sz; } while (sz > 0); if ( ! ferror(stream) && feof(stream) && bsz && '\n' == buf[bsz - 1]) { buf[bsz - 1] = '\0'; manpath_parseline(&conf->manpath, buf, 1); } free(buf); pclose(stream); #else char *insert; /* Always prepend -m. */ manpath_parseline(&conf->manpath, auxp, 1); /* If -M is given, it overrides everything else. */ if (NULL != defp) { manpath_parseline(&conf->manpath, defp, 1); return; } /* MANPATH and man.conf(5) cooperate. */ defp = getenv("MANPATH"); if (NULL == file) file = MAN_CONF_FILE; /* No MANPATH; use man.conf(5) only. */ if (NULL == defp || '\0' == defp[0]) { manconf_file(conf, file); return; } /* Prepend man.conf(5) to MANPATH. */ if (':' == defp[0]) { manconf_file(conf, file); manpath_parseline(&conf->manpath, defp, 0); return; } /* Append man.conf(5) to MANPATH. */ if (':' == defp[strlen(defp) - 1]) { manpath_parseline(&conf->manpath, defp, 0); manconf_file(conf, file); return; } /* Insert man.conf(5) into MANPATH. */ insert = strstr(defp, "::"); if (NULL != insert) { *insert++ = '\0'; manpath_parseline(&conf->manpath, defp, 0); manconf_file(conf, file); manpath_parseline(&conf->manpath, insert + 1, 0); return; } /* MANPATH overrides man.conf(5) completely. */ manpath_parseline(&conf->manpath, defp, 0); #endif } /* * Parse a FULL pathname from a colon-separated list of arrays. */ static void manpath_parseline(struct manpaths *dirs, char *path, int complain) { char *dir; if (NULL == path) return; for (dir = strtok(path, ":"); dir; dir = strtok(NULL, ":")) manpath_add(dirs, dir, complain); } /* * Add a directory to the array, ignoring bad directories. * Grow the array one-by-one for simplicity's sake. */ static void manpath_add(struct manpaths *dirs, const char *dir, int complain) { char buf[PATH_MAX]; struct stat sb; char *cp; size_t i; if (NULL == (cp = realpath(dir, buf))) { if (complain) warn("manpath: %s", dir); return; } for (i = 0; i < dirs->sz; i++) if (0 == strcmp(dirs->paths[i], dir)) return; if (stat(cp, &sb) == -1) { if (complain) warn("manpath: %s", dir); return; } dirs->paths = mandoc_reallocarray(dirs->paths, dirs->sz + 1, sizeof(char *)); dirs->paths[dirs->sz++] = mandoc_strdup(cp); } void manconf_free(struct manconf *conf) { size_t i; for (i = 0; i < conf->manpath.sz; i++) free(conf->manpath.paths[i]); free(conf->manpath.paths); free(conf->output.includes); free(conf->output.man); free(conf->output.paper); free(conf->output.style); } #if !HAVE_MANPATH static void manconf_file(struct manconf *conf, const char *file) { const char *const toks[] = { "manpath", "output", "_whatdb" }; char manpath_default[] = MANPATH_DEFAULT; FILE *stream; char *line, *cp, *ep; size_t linesz, tok, toklen; ssize_t linelen; if ((stream = fopen(file, "r")) == NULL) goto out; line = NULL; linesz = 0; while ((linelen = getline(&line, &linesz, stream)) != -1) { cp = line; - ep = cp + linelen; - if (ep[-1] != '\n') - break; - *--ep = '\0'; + ep = cp + linelen - 1; + while (ep > cp && isspace((unsigned char)*ep)) + *ep-- = '\0'; while (isspace((unsigned char)*cp)) cp++; - if (*cp == '#') + if (cp == ep || *cp == '#') continue; for (tok = 0; tok < sizeof(toks)/sizeof(toks[0]); tok++) { toklen = strlen(toks[tok]); if (cp + toklen < ep && isspace((unsigned char)cp[toklen]) && strncmp(cp, toks[tok], toklen) == 0) { cp += toklen; while (isspace((unsigned char)*cp)) cp++; break; } } switch (tok) { case 2: /* _whatdb */ while (ep > cp && ep[-1] != '/') ep--; if (ep == cp) continue; *ep = '\0'; /* FALLTHROUGH */ case 0: /* manpath */ manpath_add(&conf->manpath, cp, 0); *manpath_default = '\0'; break; case 1: /* output */ manconf_output(&conf->output, cp); break; default: break; } } free(line); fclose(stream); out: if (*manpath_default != '\0') manpath_parseline(&conf->manpath, manpath_default, 0); } #endif void manconf_output(struct manoutput *conf, const char *cp) { const char *const toks[] = { "includes", "man", "paper", "style", "indent", "width", "fragment", "mdoc" }; size_t len, tok; for (tok = 0; tok < sizeof(toks)/sizeof(toks[0]); tok++) { len = strlen(toks[tok]); if ( ! strncmp(cp, toks[tok], len) && strchr(" = ", cp[len]) != NULL) { cp += len; if (*cp == '=') cp++; while (isspace((unsigned char)*cp)) cp++; break; } } if (tok < 6 && *cp == '\0') return; switch (tok) { case 0: if (conf->includes == NULL) conf->includes = mandoc_strdup(cp); break; case 1: if (conf->man == NULL) conf->man = mandoc_strdup(cp); break; case 2: if (conf->paper == NULL) conf->paper = mandoc_strdup(cp); break; case 3: if (conf->style == NULL) conf->style = mandoc_strdup(cp); break; case 4: if (conf->indent == 0) conf->indent = strtonum(cp, 0, 1000, NULL); break; case 5: if (conf->width == 0) conf->width = strtonum(cp, 58, 1000, NULL); break; case 6: conf->fragment = 1; break; case 7: conf->mdoc = 1; break; default: break; } } Index: vendor/mdocml/dist/mansearch.c =================================================================== --- vendor/mdocml/dist/mansearch.c (revision 303220) +++ vendor/mdocml/dist/mansearch.c (revision 303221) @@ -1,853 +1,852 @@ -/* $Id: mansearch.c,v 1.64 2016/01/08 15:02:54 schwarze Exp $ */ +/* $Id: mansearch.c,v 1.65 2016/07/09 15:24:19 schwarze Exp $ */ /* * Copyright (c) 2012 Kristaps Dzonsons <kristaps@bsd.lv> * Copyright (c) 2013, 2014, 2015 Ingo Schwarze <schwarze@openbsd.org> * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHORS DISCLAIM ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include "config.h" #include <sys/mman.h> #include <sys/types.h> #include <assert.h> #if HAVE_ERR #include <err.h> #endif #include <errno.h> #include <fcntl.h> -#include <getopt.h> #include <glob.h> #include <limits.h> #include <regex.h> #include <stdio.h> #include <stdint.h> #include <stddef.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include <sqlite3.h> #ifndef SQLITE_DETERMINISTIC #define SQLITE_DETERMINISTIC 0 #endif #include "mandoc.h" #include "mandoc_aux.h" #include "mandoc_ohash.h" #include "manconf.h" #include "mansearch.h" extern int mansearch_keymax; extern const char *const mansearch_keynames[]; #define SQL_BIND_TEXT(_db, _s, _i, _v) \ do { if (SQLITE_OK != sqlite3_bind_text \ ((_s), (_i)++, (_v), -1, SQLITE_STATIC)) \ errx((int)MANDOCLEVEL_SYSERR, "%s", sqlite3_errmsg((_db))); \ } while (0) #define SQL_BIND_INT64(_db, _s, _i, _v) \ do { if (SQLITE_OK != sqlite3_bind_int64 \ ((_s), (_i)++, (_v))) \ errx((int)MANDOCLEVEL_SYSERR, "%s", sqlite3_errmsg((_db))); \ } while (0) #define SQL_BIND_BLOB(_db, _s, _i, _v) \ do { if (SQLITE_OK != sqlite3_bind_blob \ ((_s), (_i)++, (&_v), sizeof(_v), SQLITE_STATIC)) \ errx((int)MANDOCLEVEL_SYSERR, "%s", sqlite3_errmsg((_db))); \ } while (0) struct expr { regex_t regexp; /* compiled regexp, if applicable */ const char *substr; /* to search for, if applicable */ struct expr *next; /* next in sequence */ uint64_t bits; /* type-mask */ int equal; /* equality, not subsring match */ int open; /* opening parentheses before */ int and; /* logical AND before */ int close; /* closing parentheses after */ }; struct match { uint64_t pageid; /* identifier in database */ uint64_t bits; /* name type mask */ char *desc; /* manual page description */ int form; /* bit field: formatted, zipped? */ }; static void buildnames(const struct mansearch *, struct manpage *, sqlite3 *, sqlite3_stmt *, uint64_t, const char *, int form); static char *buildoutput(sqlite3 *, sqlite3_stmt *, uint64_t, uint64_t); static struct expr *exprcomp(const struct mansearch *, int, char *[]); static void exprfree(struct expr *); static struct expr *exprterm(const struct mansearch *, char *, int); static int manpage_compare(const void *, const void *); static void sql_append(char **sql, size_t *sz, const char *newstr, int count); static void sql_match(sqlite3_context *context, int argc, sqlite3_value **argv); static void sql_regexp(sqlite3_context *context, int argc, sqlite3_value **argv); static char *sql_statement(const struct expr *); int mansearch_setup(int start) { static void *pagecache; int c; #define PC_PAGESIZE 1280 #define PC_NUMPAGES 256 if (start) { if (NULL != pagecache) { warnx("pagecache already enabled"); return (int)MANDOCLEVEL_BADARG; } pagecache = mmap(NULL, PC_PAGESIZE * PC_NUMPAGES, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANON, -1, 0); if (MAP_FAILED == pagecache) { warn("mmap"); pagecache = NULL; return (int)MANDOCLEVEL_SYSERR; } c = sqlite3_config(SQLITE_CONFIG_PAGECACHE, pagecache, PC_PAGESIZE, PC_NUMPAGES); if (SQLITE_OK == c) return (int)MANDOCLEVEL_OK; warnx("pagecache: %s", sqlite3_errstr(c)); } else if (NULL == pagecache) { warnx("pagecache missing"); return (int)MANDOCLEVEL_BADARG; } if (-1 == munmap(pagecache, PC_PAGESIZE * PC_NUMPAGES)) { warn("munmap"); pagecache = NULL; return (int)MANDOCLEVEL_SYSERR; } pagecache = NULL; return (int)MANDOCLEVEL_OK; } int mansearch(const struct mansearch *search, const struct manpaths *paths, int argc, char *argv[], struct manpage **res, size_t *sz) { int64_t pageid; uint64_t outbit, iterbit; char buf[PATH_MAX]; char *sql; struct manpage *mpage; struct expr *e, *ep; sqlite3 *db; sqlite3_stmt *s, *s2; struct match *mp; struct ohash htab; unsigned int idx; size_t i, j, cur, maxres; int c, chdir_status, getcwd_status, indexbit; if (argc == 0 || (e = exprcomp(search, argc, argv)) == NULL) { *sz = 0; return 0; } cur = maxres = 0; *res = NULL; if (NULL != search->outkey) { outbit = TYPE_Nd; for (indexbit = 0, iterbit = 1; indexbit < mansearch_keymax; indexbit++, iterbit <<= 1) { if (0 == strcasecmp(search->outkey, mansearch_keynames[indexbit])) { outbit = iterbit; break; } } } else outbit = 0; /* * Remember the original working directory, if possible. * This will be needed if the second or a later directory * is given as a relative path. * Do not error out if the current directory is not * searchable: Maybe it won't be needed after all. */ if (getcwd(buf, PATH_MAX) == NULL) { getcwd_status = 0; (void)strlcpy(buf, strerror(errno), sizeof(buf)); } else getcwd_status = 1; sql = sql_statement(e); /* * Loop over the directories (containing databases) for us to * search. * Don't let missing/bad databases/directories phase us. * In each, try to open the resident database and, if it opens, * scan it for our match expression. */ chdir_status = 0; for (i = 0; i < paths->sz; i++) { if (chdir_status && paths->paths[i][0] != '/') { if ( ! getcwd_status) { warnx("%s: getcwd: %s", paths->paths[i], buf); continue; } else if (chdir(buf) == -1) { warn("%s", buf); continue; } } if (chdir(paths->paths[i]) == -1) { warn("%s", paths->paths[i]); continue; } chdir_status = 1; c = sqlite3_open_v2(MANDOC_DB, &db, SQLITE_OPEN_READONLY, NULL); if (SQLITE_OK != c) { warn("%s/%s", paths->paths[i], MANDOC_DB); sqlite3_close(db); continue; } /* * Define the SQL functions for substring * and regular expression matching. */ c = sqlite3_create_function(db, "match", 2, SQLITE_UTF8 | SQLITE_DETERMINISTIC, NULL, sql_match, NULL, NULL); assert(SQLITE_OK == c); c = sqlite3_create_function(db, "regexp", 2, SQLITE_UTF8 | SQLITE_DETERMINISTIC, NULL, sql_regexp, NULL, NULL); assert(SQLITE_OK == c); j = 1; c = sqlite3_prepare_v2(db, sql, -1, &s, NULL); if (SQLITE_OK != c) errx((int)MANDOCLEVEL_SYSERR, "%s", sqlite3_errmsg(db)); for (ep = e; NULL != ep; ep = ep->next) { if (NULL == ep->substr) { SQL_BIND_BLOB(db, s, j, ep->regexp); } else SQL_BIND_TEXT(db, s, j, ep->substr); if (0 == ((TYPE_Nd | TYPE_Nm) & ep->bits)) SQL_BIND_INT64(db, s, j, ep->bits); } mandoc_ohash_init(&htab, 4, offsetof(struct match, pageid)); /* * Hash each entry on its [unique] document identifier. * This is a uint64_t. * Instead of using a hash function, simply convert the * uint64_t to a uint32_t, the hash value's type. * This gives good performance and preserves the * distribution of buckets in the table. */ while (SQLITE_ROW == (c = sqlite3_step(s))) { pageid = sqlite3_column_int64(s, 2); idx = ohash_lookup_memory(&htab, (char *)&pageid, sizeof(uint64_t), (uint32_t)pageid); if (NULL != ohash_find(&htab, idx)) continue; mp = mandoc_calloc(1, sizeof(struct match)); mp->pageid = pageid; mp->form = sqlite3_column_int(s, 1); mp->bits = sqlite3_column_int64(s, 3); if (TYPE_Nd == outbit) mp->desc = mandoc_strdup((const char *) sqlite3_column_text(s, 0)); ohash_insert(&htab, idx, mp); } if (SQLITE_DONE != c) warnx("%s", sqlite3_errmsg(db)); sqlite3_finalize(s); c = sqlite3_prepare_v2(db, "SELECT sec, arch, name, pageid FROM mlinks " "WHERE pageid=? ORDER BY sec, arch, name", -1, &s, NULL); if (SQLITE_OK != c) errx((int)MANDOCLEVEL_SYSERR, "%s", sqlite3_errmsg(db)); c = sqlite3_prepare_v2(db, "SELECT bits, key, pageid FROM keys " "WHERE pageid=? AND bits & ?", -1, &s2, NULL); if (SQLITE_OK != c) errx((int)MANDOCLEVEL_SYSERR, "%s", sqlite3_errmsg(db)); for (mp = ohash_first(&htab, &idx); NULL != mp; mp = ohash_next(&htab, &idx)) { if (cur + 1 > maxres) { maxres += 1024; *res = mandoc_reallocarray(*res, maxres, sizeof(struct manpage)); } mpage = *res + cur; mpage->ipath = i; mpage->bits = mp->bits; mpage->sec = 10; mpage->form = mp->form; buildnames(search, mpage, db, s, mp->pageid, paths->paths[i], mp->form); if (mpage->names != NULL) { mpage->output = TYPE_Nd & outbit ? mp->desc : outbit ? buildoutput(db, s2, mp->pageid, outbit) : NULL; cur++; } free(mp); } sqlite3_finalize(s); sqlite3_finalize(s2); sqlite3_close(db); ohash_delete(&htab); /* * In man(1) mode, prefer matches in earlier trees * over matches in later trees. */ if (cur && search->firstmatch) break; } qsort(*res, cur, sizeof(struct manpage), manpage_compare); if (chdir_status && getcwd_status && chdir(buf) == -1) warn("%s", buf); exprfree(e); free(sql); *sz = cur; return 1; } void mansearch_free(struct manpage *res, size_t sz) { size_t i; for (i = 0; i < sz; i++) { free(res[i].file); free(res[i].names); free(res[i].output); } free(res); } static int manpage_compare(const void *vp1, const void *vp2) { const struct manpage *mp1, *mp2; int diff; mp1 = vp1; mp2 = vp2; return (diff = mp2->bits - mp1->bits) ? diff : (diff = mp1->sec - mp2->sec) ? diff : strcasecmp(mp1->names, mp2->names); } static void buildnames(const struct mansearch *search, struct manpage *mpage, sqlite3 *db, sqlite3_stmt *s, uint64_t pageid, const char *path, int form) { glob_t globinfo; char *firstname, *newnames, *prevsec, *prevarch; const char *oldnames, *sep1, *name, *sec, *sep2, *arch, *fsec; size_t i; int c, globres; mpage->file = NULL; mpage->names = NULL; firstname = prevsec = prevarch = NULL; i = 1; SQL_BIND_INT64(db, s, i, pageid); while (SQLITE_ROW == (c = sqlite3_step(s))) { /* Decide whether we already have some names. */ if (NULL == mpage->names) { oldnames = ""; sep1 = ""; } else { oldnames = mpage->names; sep1 = ", "; } /* Fetch the next name, rejecting sec/arch mismatches. */ sec = (const char *)sqlite3_column_text(s, 0); if (search->sec != NULL && strcasecmp(sec, search->sec)) continue; arch = (const char *)sqlite3_column_text(s, 1); if (search->arch != NULL && *arch != '\0' && strcasecmp(arch, search->arch)) continue; name = (const char *)sqlite3_column_text(s, 2); /* Remember the first section found. */ if (9 < mpage->sec && '1' <= *sec && '9' >= *sec) mpage->sec = (*sec - '1') + 1; /* If the section changed, append the old one. */ if (NULL != prevsec && (strcmp(sec, prevsec) || strcmp(arch, prevarch))) { sep2 = '\0' == *prevarch ? "" : "/"; mandoc_asprintf(&newnames, "%s(%s%s%s)", oldnames, prevsec, sep2, prevarch); free(mpage->names); oldnames = mpage->names = newnames; free(prevsec); free(prevarch); prevsec = prevarch = NULL; } /* Save the new section, to append it later. */ if (NULL == prevsec) { prevsec = mandoc_strdup(sec); prevarch = mandoc_strdup(arch); } /* Append the new name. */ mandoc_asprintf(&newnames, "%s%s%s", oldnames, sep1, name); free(mpage->names); mpage->names = newnames; /* Also save the first file name encountered. */ if (mpage->file != NULL) continue; if (form & FORM_SRC) { sep1 = "man"; fsec = sec; } else { sep1 = "cat"; fsec = "0"; } sep2 = *arch == '\0' ? "" : "/"; mandoc_asprintf(&mpage->file, "%s/%s%s%s%s/%s.%s", path, sep1, sec, sep2, arch, name, fsec); if (access(mpage->file, R_OK) != -1) continue; /* Handle unusual file name extensions. */ if (firstname == NULL) firstname = mpage->file; else free(mpage->file); mandoc_asprintf(&mpage->file, "%s/%s%s%s%s/%s.*", path, sep1, sec, sep2, arch, name); globres = glob(mpage->file, 0, NULL, &globinfo); free(mpage->file); mpage->file = globres ? NULL : mandoc_strdup(*globinfo.gl_pathv); globfree(&globinfo); } if (c != SQLITE_DONE) warnx("%s", sqlite3_errmsg(db)); sqlite3_reset(s); /* If none of the files is usable, use the first name. */ if (mpage->file == NULL) mpage->file = firstname; else if (mpage->file != firstname) free(firstname); /* Append one final section to the names. */ if (prevsec != NULL) { sep2 = *prevarch == '\0' ? "" : "/"; mandoc_asprintf(&newnames, "%s(%s%s%s)", mpage->names, prevsec, sep2, prevarch); free(mpage->names); mpage->names = newnames; free(prevsec); free(prevarch); } } static char * buildoutput(sqlite3 *db, sqlite3_stmt *s, uint64_t pageid, uint64_t outbit) { char *output, *newoutput; const char *oldoutput, *sep1, *data; size_t i; int c; output = NULL; i = 1; SQL_BIND_INT64(db, s, i, pageid); SQL_BIND_INT64(db, s, i, outbit); while (SQLITE_ROW == (c = sqlite3_step(s))) { if (NULL == output) { oldoutput = ""; sep1 = ""; } else { oldoutput = output; sep1 = " # "; } data = (const char *)sqlite3_column_text(s, 1); mandoc_asprintf(&newoutput, "%s%s%s", oldoutput, sep1, data); free(output); output = newoutput; } if (SQLITE_DONE != c) warnx("%s", sqlite3_errmsg(db)); sqlite3_reset(s); return output; } /* * Implement substring match as an application-defined SQL function. * Using the SQL LIKE or GLOB operators instead would be a bad idea * because that would require escaping metacharacters in the string * being searched for. */ static void sql_match(sqlite3_context *context, int argc, sqlite3_value **argv) { assert(2 == argc); sqlite3_result_int(context, NULL != strcasestr( (const char *)sqlite3_value_text(argv[1]), (const char *)sqlite3_value_text(argv[0]))); } /* * Implement regular expression match * as an application-defined SQL function. */ static void sql_regexp(sqlite3_context *context, int argc, sqlite3_value **argv) { assert(2 == argc); sqlite3_result_int(context, !regexec( (regex_t *)sqlite3_value_blob(argv[0]), (const char *)sqlite3_value_text(argv[1]), 0, NULL, 0)); } static void sql_append(char **sql, size_t *sz, const char *newstr, int count) { size_t newsz; newsz = 1 < count ? (size_t)count : strlen(newstr); *sql = mandoc_realloc(*sql, *sz + newsz + 1); if (1 < count) memset(*sql + *sz, *newstr, (size_t)count); else memcpy(*sql + *sz, newstr, newsz); *sz += newsz; (*sql)[*sz] = '\0'; } /* * Prepare the search SQL statement. */ static char * sql_statement(const struct expr *e) { char *sql; size_t sz; int needop; sql = mandoc_strdup(e->equal ? "SELECT desc, form, pageid, bits " "FROM mpages NATURAL JOIN names WHERE " : "SELECT desc, form, pageid, 0 FROM mpages WHERE "); sz = strlen(sql); for (needop = 0; NULL != e; e = e->next) { if (e->and) sql_append(&sql, &sz, " AND ", 1); else if (needop) sql_append(&sql, &sz, " OR ", 1); if (e->open) sql_append(&sql, &sz, "(", e->open); sql_append(&sql, &sz, TYPE_Nd & e->bits ? (NULL == e->substr ? "desc REGEXP ?" : "desc MATCH ?") : TYPE_Nm == e->bits ? (NULL == e->substr ? "pageid IN (SELECT pageid FROM names " "WHERE name REGEXP ?)" : e->equal ? "name = ? " : "pageid IN (SELECT pageid FROM names " "WHERE name MATCH ?)") : (NULL == e->substr ? "pageid IN (SELECT pageid FROM keys " "WHERE key REGEXP ? AND bits & ?)" : "pageid IN (SELECT pageid FROM keys " "WHERE key MATCH ? AND bits & ?)"), 1); if (e->close) sql_append(&sql, &sz, ")", e->close); needop = 1; } return sql; } /* * Compile a set of string tokens into an expression. * Tokens in "argv" are assumed to be individual expression atoms (e.g., * "(", "foo=bar", etc.). */ static struct expr * exprcomp(const struct mansearch *search, int argc, char *argv[]) { uint64_t mask; int i, toopen, logic, igncase, toclose; struct expr *first, *prev, *cur, *next; first = cur = NULL; logic = igncase = toopen = toclose = 0; for (i = 0; i < argc; i++) { if (0 == strcmp("(", argv[i])) { if (igncase) goto fail; toopen++; toclose++; continue; } else if (0 == strcmp(")", argv[i])) { if (toopen || logic || igncase || NULL == cur) goto fail; cur->close++; if (0 > --toclose) goto fail; continue; } else if (0 == strcmp("-a", argv[i])) { if (toopen || logic || igncase || NULL == cur) goto fail; logic = 1; continue; } else if (0 == strcmp("-o", argv[i])) { if (toopen || logic || igncase || NULL == cur) goto fail; logic = 2; continue; } else if (0 == strcmp("-i", argv[i])) { if (igncase) goto fail; igncase = 1; continue; } next = exprterm(search, argv[i], !igncase); if (NULL == next) goto fail; if (NULL == first) first = next; else cur->next = next; prev = cur = next; /* * Searching for descriptions must be split out * because they are stored in the mpages table, * not in the keys table. */ for (mask = TYPE_Nm; mask <= TYPE_Nd; mask <<= 1) { if (mask & cur->bits && ~mask & cur->bits) { next = mandoc_calloc(1, sizeof(struct expr)); memcpy(next, cur, sizeof(struct expr)); prev->open = 1; cur->bits = mask; cur->next = next; cur = next; cur->bits &= ~mask; } } prev->and = (1 == logic); prev->open += toopen; if (cur != prev) cur->close = 1; toopen = logic = igncase = 0; } if ( ! (toopen || logic || igncase || toclose)) return first; fail: if (NULL != first) exprfree(first); return NULL; } static struct expr * exprterm(const struct mansearch *search, char *buf, int cs) { char errbuf[BUFSIZ]; struct expr *e; char *key, *val; uint64_t iterbit; int i, irc; if ('\0' == *buf) return NULL; e = mandoc_calloc(1, sizeof(struct expr)); if (search->argmode == ARG_NAME) { e->bits = TYPE_Nm; e->substr = buf; e->equal = 1; return e; } /* * Separate macro keys from search string. * If needed, request regular expression handling * by setting e->substr to NULL. */ if (search->argmode == ARG_WORD) { e->bits = TYPE_Nm; e->substr = NULL; #if HAVE_REWB_BSD mandoc_asprintf(&val, "[[:<:]]%s[[:>:]]", buf); #elif HAVE_REWB_SYSV mandoc_asprintf(&val, "\\<%s\\>", buf); #else mandoc_asprintf(&val, "(^|[^a-zA-Z01-9_])%s([^a-zA-Z01-9_]|$)", buf); #endif cs = 0; } else if ((val = strpbrk(buf, "=~")) == NULL) { e->bits = TYPE_Nm | TYPE_Nd; e->substr = buf; } else { if (val == buf) e->bits = TYPE_Nm | TYPE_Nd; if ('=' == *val) e->substr = val + 1; *val++ = '\0'; if (NULL != strstr(buf, "arch")) cs = 0; } /* Compile regular expressions. */ if (NULL == e->substr) { irc = regcomp(&e->regexp, val, REG_EXTENDED | REG_NOSUB | (cs ? 0 : REG_ICASE)); if (search->argmode == ARG_WORD) free(val); if (irc) { regerror(irc, &e->regexp, errbuf, sizeof(errbuf)); warnx("regcomp: %s", errbuf); free(e); return NULL; } } if (e->bits) return e; /* * Parse out all possible fields. * If the field doesn't resolve, bail. */ while (NULL != (key = strsep(&buf, ","))) { if ('\0' == *key) continue; for (i = 0, iterbit = 1; i < mansearch_keymax; i++, iterbit <<= 1) { if (0 == strcasecmp(key, mansearch_keynames[i])) { e->bits |= iterbit; break; } } if (i == mansearch_keymax) { if (strcasecmp(key, "any")) { free(e); return NULL; } e->bits |= ~0ULL; } } return e; } static void exprfree(struct expr *p) { struct expr *pp; while (NULL != p) { pp = p->next; free(p); p = pp; } } Index: vendor/mdocml/dist/mchars_alloc.3 =================================================================== --- vendor/mdocml/dist/mchars_alloc.3 (revision 303220) +++ vendor/mdocml/dist/mchars_alloc.3 (revision 303221) @@ -1,226 +1,227 @@ -.\" $Id: mchars_alloc.3,v 1.3 2015/10/13 22:59:54 schwarze Exp $ +.\" $Id: mchars_alloc.3,v 1.4 2016/07/07 19:19:01 schwarze Exp $ .\" .\" Copyright (c) 2014 Ingo Schwarze <schwarze@openbsd.org> .\" .\" Permission to use, copy, modify, and distribute this software for any .\" purpose with or without fee is hereby granted, provided that the above .\" copyright notice and this permission notice appear in all copies. .\" .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. .\" -.Dd $Mdocdate: October 13 2015 $ +.Dd $Mdocdate: July 7 2016 $ .Dt MCHARS_ALLOC 3 .Os .Sh NAME .Nm mchars_alloc , .Nm mchars_free , .Nm mchars_num2char , .Nm mchars_num2uc , .Nm mchars_spec2cp , -.Nm mchars_spec2str +.Nm mchars_spec2str , +.Nm mchars_uc2str .Nd character table for mandoc .Sh SYNOPSIS .In sys/types.h .In mandoc.h .Ft void .Fn mchars_alloc void .Ft void .Fn mchars_free void .Ft char .Fo mchars_num2char .Fa "const char *decimal" .Fa "size_t sz" .Fc .Ft int .Fo mchars_num2uc .Fa "const char *hexadecimal" .Fa "size_t sz" .Fc .Ft int .Fo mchars_spec2cp .Fa "const char *name" .Fa "size_t sz" .Fc .Ft "const char *" .Fo mchars_spec2str .Fa "const char *name" .Fa "size_t sz" .Fa "size_t *rsz" .Fc .Ft "const char *" .Fn mchars_uc2str "int codepoint" .Sh DESCRIPTION These functions translate Unicode character numbers and .Xr roff 7 character names into glyphs. See .Xr mandoc_char 7 for a list of .Xr roff 7 special characters. These functions are intended for external use by programs formatting .Xr mdoc 7 and .Xr man 7 pages for output, for example the .Xr mandoc 1 output formatter modules and .Xr makewhatis 8 . The .Fa decimal , .Fa hexadecimal , .Fa name , and .Fa size input arguments are usually obtained from the .Xr mandoc_escape 3 parser function. .Pp The function .Fn mchars_num2char converts a .Fa decimal string representation of a character number consisting of .Fa sz digits into a printable ASCII character. If the input string is non-numeric or does not represent a printable ASCII character, the NUL character .Pq Sq \e0 is returned. For example, the .Xr mandoc 1 .Fl Tascii , .Fl Tutf8 , and .Fl Thtml output modules use this function to render .Xr roff 7 .Ic \eN escape sequences. .Pp The function .Fn mchars_num2uc converts a .Fa hexadecimal string representation of a Unicode codepoint consisting of .Fa sz digits into an integer representation. If the input string is non-numeric or represents an ASCII character, the NUL character .Pq Sq \e0 is returned. For example, the .Xr mandoc 1 .Fl Tutf8 and .Fl Thtml output modules use this function to render .Xr roff 7 .Ic \e[u Ns Ar XXXX Ns Ic \&] and .Ic \eC\(aqu Ns Ar XXXX Ns Ic \(aq escape sequences. .Pp The function .Fn mchars_alloc initializes a static .Vt "struct ohash" object for subsequent use by the following two lookup functions. When no longer needed, this object can be destroyed with .Fn mchars_free . .Pp The function .Fn mchars_spec2cp looks up a .Xr roff 7 special character .Fa name consisting of .Fa sz characters and returns the corresponding Unicode codepoint. If the .Ar name is not recognized, \-1 is returned. For example, the .Xr mandoc 1 .Fl Tutf8 and .Fl Thtml output modules use this function to render .Xr roff 7 .Ic \e[ Ns Ar name Ns Ic \&] and .Ic \eC\(aq Ns Ar name Ns Ic \(aq escape sequences. .Pp The function .Fn mchars_spec2str looks up a .Xr roff 7 special character .Fa name consisting of .Fa sz characters and returns an ASCII string representation. The length of the representation is returned in .Fa rsz . In many cases, the meaning of such ASCII representations is not quite obvious, so using .Xr roff 7 special characters in documents intended for ASCII rendering is usually a bad idea. If the .Ar name is not recognized, .Dv NULL is returned. For example, .Xr makewhatis 8 and the .Xr mandoc 1 .Fl Tascii output module use this function to render .Xr roff 7 .Ic \e[ Ns Ar name Ns Ic \&] and .Ic \eC\(aq Ns Ar name Ns Ic \(aq escape sequences. .Pp The function .Fn mchars_uc2str performs a reverse lookup of the Unicode .Fa codepoint and returns an ASCII string representation, or the string .Qq <?> if none is available. .Sh FILES These funtions are implemented in the file .Pa chars.c . .Sh SEE ALSO .Xr mandoc 1 , .Xr mandoc_escape 3 , .Xr ohash_init 3 , .Xr mandoc_char 7 , .Xr roff 7 .Sh HISTORY These functions and their predecessors have been available since the following mandoc versions: .Bl -column "mchars_num2char()" "1.11.3" "chars_num2char()" "1.10.10" .It Sy function Ta since Ta Sy predecessor Ta since .It Fn mchars_alloc Ta 1.11.3 Ta Fn ascii2htab Ta 1.5.3 .It Fn mchars_free Ta 1.11.2 Ta Fn asciifree Ta 1.6.0 .It Fn mchars_num2char Ta 1.11.2 Ta Fn chars_num2char Ta 1.10.10 .It Fn mchars_num2uc Ta 1.11.3 Ta \(em Ta \(em .It Fn mchars_spec2cp Ta 1.11.2 Ta Fn chars_spec2cp Ta 1.10.5 .It Fn mchars_spec2str Ta 1.11.2 Ta Fn a2ascii Ta 1.5.3 .It Fn mchars_uc2str Ta 1.13.2 Ta \(em Ta \(em .El .Sh AUTHORS .An Kristaps Dzonsons Aq Mt kristaps@bsd.lv .An Ingo Schwarze Aq Mt schwarze@openbsd.org Index: vendor/mdocml/dist/read.c =================================================================== --- vendor/mdocml/dist/read.c (revision 303220) +++ vendor/mdocml/dist/read.c (revision 303221) @@ -1,948 +1,948 @@ -/* $Id: read.c,v 1.148 2016/01/08 02:53:13 schwarze Exp $ */ +/* $Id: read.c,v 1.149 2016/07/10 13:34:30 schwarze Exp $ */ /* * Copyright (c) 2008, 2009, 2010, 2011 Kristaps Dzonsons <kristaps@bsd.lv> * Copyright (c) 2010-2016 Ingo Schwarze <schwarze@openbsd.org> * Copyright (c) 2010, 2012 Joerg Sonnenberger <joerg@netbsd.org> * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHORS DISCLAIM ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include "config.h" #include <sys/types.h> #if HAVE_MMAP #include <sys/mman.h> #include <sys/stat.h> #endif #include <assert.h> #include <ctype.h> #if HAVE_ERR #include <err.h> #endif #include <errno.h> #include <fcntl.h> #include <stdarg.h> #include <stdint.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include <zlib.h> #include "mandoc_aux.h" #include "mandoc.h" #include "roff.h" #include "mdoc.h" #include "man.h" #include "libmandoc.h" #include "roff_int.h" #define REPARSE_LIMIT 1000 struct mparse { struct roff_man *man; /* man parser */ struct roff *roff; /* roff parser (!NULL) */ char *sodest; /* filename pointed to by .so */ const char *file; /* filename of current input file */ struct buf *primary; /* buffer currently being parsed */ struct buf *secondary; /* preprocessed copy of input */ const char *defos; /* default operating system */ mandocmsg mmsg; /* warning/error message handler */ enum mandoclevel file_status; /* status of current parse */ enum mandoclevel wlevel; /* ignore messages below this */ int options; /* parser options */ int gzip; /* current input file is gzipped */ int filenc; /* encoding of the current file */ int reparse_count; /* finite interp. stack */ int line; /* line number in the file */ }; static void choose_parser(struct mparse *); static void resize_buf(struct buf *, size_t); static void mparse_buf_r(struct mparse *, struct buf, size_t, int); static int read_whole_file(struct mparse *, const char *, int, struct buf *, int *); static void mparse_end(struct mparse *); static void mparse_parse_buffer(struct mparse *, struct buf, const char *); static const enum mandocerr mandoclimits[MANDOCLEVEL_MAX] = { MANDOCERR_OK, MANDOCERR_WARNING, MANDOCERR_WARNING, MANDOCERR_ERROR, MANDOCERR_UNSUPP, MANDOCERR_MAX, MANDOCERR_MAX }; static const char * const mandocerrs[MANDOCERR_MAX] = { "ok", "generic warning", /* related to the prologue */ "missing manual title, using UNTITLED", "missing manual title, using \"\"", "lower case character in document title", "missing manual section, using \"\"", "unknown manual section", "missing date, using today's date", "cannot parse date, using it verbatim", "missing Os macro, using \"\"", "duplicate prologue macro", "late prologue macro", "skipping late title macro", "prologue macros out of order", /* related to document structure */ ".so is fragile, better use ln(1)", "no document body", "content before first section header", "first section is not \"NAME\"", "NAME section without name", "NAME section without description", "description not at the end of NAME", "bad NAME section content", "missing description line, using \"\"", "sections out of conventional order", "duplicate section title", "unexpected section", "unusual Xr order", "unusual Xr punctuation", "AUTHORS section without An macro", /* related to macros and nesting */ "obsolete macro", "macro neither callable nor escaped", "skipping paragraph macro", "moving paragraph macro out of list", "skipping no-space macro", "blocks badly nested", "nested displays are not portable", "moving content out of list", "fill mode already enabled, skipping", "fill mode already disabled, skipping", "line scope broken", /* related to missing macro arguments */ "skipping empty request", "conditional request controls empty scope", "skipping empty macro", "empty block", "empty argument, using 0n", "missing display type, using -ragged", "list type is not the first argument", "missing -width in -tag list, using 8n", "missing utility name, using \"\"", "missing function name, using \"\"", "empty head in list item", "empty list item", "missing font type, using \\fR", "unknown font type, using \\fR", "nothing follows prefix", "empty reference block", "missing -std argument, adding it", "missing option string, using \"\"", "missing resource identifier, using \"\"", "missing eqn box, using \"\"", /* related to bad macro arguments */ "unterminated quoted argument", "duplicate argument", "skipping duplicate argument", "skipping duplicate display type", "skipping duplicate list type", "skipping -width argument", "wrong number of cells", "unknown AT&T UNIX version", "comma in function argument", "parenthesis in function name", "invalid content in Rs block", "invalid Boolean argument", "unknown font, skipping request", "odd number of characters in request", /* related to plain text */ "blank line in fill mode, using .sp", "tab in filled text", "whitespace at end of input line", "bad comment style", "invalid escape sequence", "undefined string, using \"\"", /* related to tables */ "tbl line starts with span", "tbl column starts with span", "skipping vertical bar in tbl layout", "generic error", /* related to tables */ "non-alphabetic character in tbl options", "skipping unknown tbl option", "missing tbl option argument", "wrong tbl option argument size", "empty tbl layout", "invalid character in tbl layout", "unmatched parenthesis in tbl layout", "tbl without any data cells", "ignoring data in spanned tbl cell", "ignoring extra tbl data cells", "data block open at end of tbl", /* related to document structure and macros */ NULL, "input stack limit exceeded, infinite loop?", "skipping bad character", "skipping unknown macro", "skipping insecure request", "skipping item outside list", "skipping column outside column list", "skipping end of block that is not open", "fewer RS blocks open, skipping", "inserting missing end of block", "appending missing end of block", /* related to request and macro arguments */ "escaped character not allowed in a name", "NOT IMPLEMENTED: Bd -file", "skipping display without arguments", "missing list type, using -item", "missing manual name, using \"\"", "uname(3) system call failed, using UNKNOWN", "unknown standard specifier", "skipping request without numeric argument", "NOT IMPLEMENTED: .so with absolute path or \"..\"", ".so request failed", "skipping all arguments", "skipping excess arguments", "divide by zero", "unsupported feature", "input too large", "unsupported control character", "unsupported roff request", "eqn delim option in tbl", "unsupported tbl layout modifier", "ignoring macro in table", }; static const char * const mandoclevels[MANDOCLEVEL_MAX] = { "SUCCESS", "RESERVED", "WARNING", "ERROR", "UNSUPP", "BADARG", "SYSERR" }; static void resize_buf(struct buf *buf, size_t initial) { buf->sz = buf->sz > initial/2 ? 2 * buf->sz : initial; buf->buf = mandoc_realloc(buf->buf, buf->sz); } static void choose_parser(struct mparse *curp) { char *cp, *ep; int format; /* * If neither command line arguments -mdoc or -man select * a parser nor the roff parser found a .Dd or .TH macro * yet, look ahead in the main input buffer. */ if ((format = roff_getformat(curp->roff)) == 0) { cp = curp->primary->buf; ep = cp + curp->primary->sz; while (cp < ep) { if (*cp == '.' || *cp == '\'') { cp++; if (cp[0] == 'D' && cp[1] == 'd') { format = MPARSE_MDOC; break; } if (cp[0] == 'T' && cp[1] == 'H') { format = MPARSE_MAN; break; } } cp = memchr(cp, '\n', ep - cp); if (cp == NULL) break; cp++; } } if (curp->man == NULL) { curp->man = roff_man_alloc(curp->roff, curp, curp->defos, curp->options & MPARSE_QUICK ? 1 : 0); curp->man->macroset = MACROSET_MAN; curp->man->first->tok = TOKEN_NONE; } if (format == MPARSE_MDOC) { mdoc_hash_init(); curp->man->macroset = MACROSET_MDOC; curp->man->first->tok = TOKEN_NONE; } else { man_hash_init(); curp->man->macroset = MACROSET_MAN; curp->man->first->tok = TOKEN_NONE; } } /* * Main parse routine for a buffer. * It assumes encoding and line numbering are already set up. * It can recurse directly (for invocations of user-defined * macros, inline equations, and input line traps) * and indirectly (for .so file inclusion). */ static void mparse_buf_r(struct mparse *curp, struct buf blk, size_t i, int start) { const struct tbl_span *span; struct buf ln; const char *save_file; char *cp; size_t pos; /* byte number in the ln buffer */ enum rofferr rr; int of; int lnn; /* line number in the real file */ int fd; unsigned char c; memset(&ln, 0, sizeof(ln)); lnn = curp->line; pos = 0; while (i < blk.sz) { if (0 == pos && '\0' == blk.buf[i]) break; if (start) { curp->line = lnn; curp->reparse_count = 0; if (lnn < 3 && curp->filenc & MPARSE_UTF8 && curp->filenc & MPARSE_LATIN1) curp->filenc = preconv_cue(&blk, i); } while (i < blk.sz && (start || blk.buf[i] != '\0')) { /* * When finding an unescaped newline character, * leave the character loop to process the line. * Skip a preceding carriage return, if any. */ if ('\r' == blk.buf[i] && i + 1 < blk.sz && '\n' == blk.buf[i + 1]) ++i; if ('\n' == blk.buf[i]) { ++i; ++lnn; break; } /* * Make sure we have space for the worst * case of 11 bytes: "\\[u10ffff]\0" */ if (pos + 11 > ln.sz) resize_buf(&ln, 256); /* * Encode 8-bit input. */ c = blk.buf[i]; if (c & 0x80) { if ( ! (curp->filenc && preconv_encode( &blk, &i, &ln, &pos, &curp->filenc))) { mandoc_vmsg(MANDOCERR_CHAR_BAD, curp, curp->line, pos, "0x%x", c); ln.buf[pos++] = '?'; i++; } continue; } /* * Exclude control characters. */ if (c == 0x7f || (c < 0x20 && c != 0x09)) { mandoc_vmsg(c == 0x00 || c == 0x04 || c > 0x0a ? MANDOCERR_CHAR_BAD : MANDOCERR_CHAR_UNSUPP, curp, curp->line, pos, "0x%x", c); i++; if (c != '\r') ln.buf[pos++] = '?'; continue; } /* Trailing backslash = a plain char. */ if (blk.buf[i] != '\\' || i + 1 == blk.sz) { ln.buf[pos++] = blk.buf[i++]; continue; } /* * Found escape and at least one other character. * When it's a newline character, skip it. * When there is a carriage return in between, * skip that one as well. */ if ('\r' == blk.buf[i + 1] && i + 2 < blk.sz && '\n' == blk.buf[i + 2]) ++i; if ('\n' == blk.buf[i + 1]) { i += 2; ++lnn; continue; } if ('"' == blk.buf[i + 1] || '#' == blk.buf[i + 1]) { i += 2; /* Comment, skip to end of line */ for (; i < blk.sz; ++i) { if ('\n' == blk.buf[i]) { ++i; ++lnn; break; } } /* Backout trailing whitespaces */ for (; pos > 0; --pos) { if (ln.buf[pos - 1] != ' ') break; if (pos > 2 && ln.buf[pos - 2] == '\\') break; } break; } /* Catch escaped bogus characters. */ c = (unsigned char) blk.buf[i+1]; if ( ! (isascii(c) && (isgraph(c) || isblank(c)))) { mandoc_vmsg(MANDOCERR_CHAR_BAD, curp, curp->line, pos, "0x%x", c); i += 2; ln.buf[pos++] = '?'; continue; } /* Some other escape sequence, copy & cont. */ ln.buf[pos++] = blk.buf[i++]; ln.buf[pos++] = blk.buf[i++]; } if (pos >= ln.sz) resize_buf(&ln, 256); ln.buf[pos] = '\0'; /* * A significant amount of complexity is contained by * the roff preprocessor. It's line-oriented but can be * expressed on one line, so we need at times to * readjust our starting point and re-run it. The roff * preprocessor can also readjust the buffers with new * data, so we pass them in wholesale. */ of = 0; /* * Maintain a lookaside buffer of all parsed lines. We * only do this if mparse_keep() has been invoked (the * buffer may be accessed with mparse_getkeep()). */ if (curp->secondary) { curp->secondary->buf = mandoc_realloc( curp->secondary->buf, curp->secondary->sz + pos + 2); memcpy(curp->secondary->buf + curp->secondary->sz, ln.buf, pos); curp->secondary->sz += pos; curp->secondary->buf [curp->secondary->sz] = '\n'; curp->secondary->sz++; curp->secondary->buf [curp->secondary->sz] = '\0'; } rerun: rr = roff_parseln(curp->roff, curp->line, &ln, &of); switch (rr) { case ROFF_REPARSE: if (REPARSE_LIMIT >= ++curp->reparse_count) mparse_buf_r(curp, ln, of, 0); else mandoc_msg(MANDOCERR_ROFFLOOP, curp, curp->line, pos, NULL); pos = 0; continue; case ROFF_APPEND: pos = strlen(ln.buf); continue; case ROFF_RERUN: goto rerun; case ROFF_IGN: pos = 0; continue; case ROFF_SO: if ( ! (curp->options & MPARSE_SO) && (i >= blk.sz || blk.buf[i] == '\0')) { curp->sodest = mandoc_strdup(ln.buf + of); free(ln.buf); return; } /* * We remove `so' clauses from our lookaside * buffer because we're going to descend into * the file recursively. */ if (curp->secondary) curp->secondary->sz -= pos + 1; save_file = curp->file; if ((fd = mparse_open(curp, ln.buf + of)) != -1) { mparse_readfd(curp, fd, ln.buf + of); close(fd); curp->file = save_file; } else { curp->file = save_file; mandoc_vmsg(MANDOCERR_SO_FAIL, curp, curp->line, pos, ".so %s", ln.buf + of); ln.sz = mandoc_asprintf(&cp, ".sp\nSee the file %s.\n.sp", ln.buf + of); free(ln.buf); ln.buf = cp; of = 0; mparse_buf_r(curp, ln, of, 0); } pos = 0; continue; default: break; } /* * If input parsers have not been allocated, do so now. * We keep these instanced between parsers, but set them * locally per parse routine since we can use different * parsers with each one. */ if (curp->man == NULL || curp->man->macroset == MACROSET_NONE) choose_parser(curp); /* * Lastly, push down into the parsers themselves. * If libroff returns ROFF_TBL, then add it to the * currently open parse. Since we only get here if * there does exist data (see tbl_data.c), we're * guaranteed that something's been allocated. * Do the same for ROFF_EQN. */ if (rr == ROFF_TBL) while ((span = roff_span(curp->roff)) != NULL) roff_addtbl(curp->man, span); else if (rr == ROFF_EQN) roff_addeqn(curp->man, roff_eqn(curp->roff)); else if ((curp->man->macroset == MACROSET_MDOC ? mdoc_parseln(curp->man, curp->line, ln.buf, of) : man_parseln(curp->man, curp->line, ln.buf, of)) == 2) break; /* Temporary buffers typically are not full. */ if (0 == start && '\0' == blk.buf[i]) break; /* Start the next input line. */ pos = 0; } free(ln.buf); } static int read_whole_file(struct mparse *curp, const char *file, int fd, struct buf *fb, int *with_mmap) { gzFile gz; size_t off; ssize_t ssz; #if HAVE_MMAP struct stat st; if (fstat(fd, &st) == -1) err((int)MANDOCLEVEL_SYSERR, "%s", file); /* * If we're a regular file, try just reading in the whole entry * via mmap(). This is faster than reading it into blocks, and * since each file is only a few bytes to begin with, I'm not * concerned that this is going to tank any machines. */ if (curp->gzip == 0 && S_ISREG(st.st_mode)) { if (st.st_size > 0x7fffffff) { mandoc_msg(MANDOCERR_TOOLARGE, curp, 0, 0, NULL); return 0; } *with_mmap = 1; fb->sz = (size_t)st.st_size; fb->buf = mmap(NULL, fb->sz, PROT_READ, MAP_SHARED, fd, 0); if (fb->buf != MAP_FAILED) return 1; } #endif if (curp->gzip) { if ((gz = gzdopen(fd, "rb")) == NULL) err((int)MANDOCLEVEL_SYSERR, "%s", file); } else gz = NULL; /* * If this isn't a regular file (like, say, stdin), then we must * go the old way and just read things in bit by bit. */ *with_mmap = 0; off = 0; fb->sz = 0; fb->buf = NULL; for (;;) { if (off == fb->sz) { if (fb->sz == (1U << 31)) { mandoc_msg(MANDOCERR_TOOLARGE, curp, 0, 0, NULL); break; } resize_buf(fb, 65536); } ssz = curp->gzip ? gzread(gz, fb->buf + (int)off, fb->sz - off) : read(fd, fb->buf + (int)off, fb->sz - off); if (ssz == 0) { fb->sz = off; return 1; } if (ssz == -1) err((int)MANDOCLEVEL_SYSERR, "%s", file); off += (size_t)ssz; } free(fb->buf); fb->buf = NULL; return 0; } static void mparse_end(struct mparse *curp) { if (curp->man == NULL && curp->sodest == NULL) curp->man = roff_man_alloc(curp->roff, curp, curp->defos, curp->options & MPARSE_QUICK ? 1 : 0); if (curp->man->macroset == MACROSET_NONE) curp->man->macroset = MACROSET_MAN; if (curp->man->macroset == MACROSET_MDOC) mdoc_endparse(curp->man); else man_endparse(curp->man); roff_endparse(curp->roff); } static void mparse_parse_buffer(struct mparse *curp, struct buf blk, const char *file) { struct buf *svprimary; const char *svfile; size_t offset; static int recursion_depth; if (64 < recursion_depth) { mandoc_msg(MANDOCERR_ROFFLOOP, curp, curp->line, 0, NULL); return; } /* Line number is per-file. */ svfile = curp->file; curp->file = file; svprimary = curp->primary; curp->primary = &blk; curp->line = 1; recursion_depth++; /* Skip an UTF-8 byte order mark. */ if (curp->filenc & MPARSE_UTF8 && blk.sz > 2 && (unsigned char)blk.buf[0] == 0xef && (unsigned char)blk.buf[1] == 0xbb && (unsigned char)blk.buf[2] == 0xbf) { offset = 3; curp->filenc &= ~MPARSE_LATIN1; } else offset = 0; mparse_buf_r(curp, blk, offset, 1); if (--recursion_depth == 0) mparse_end(curp); curp->primary = svprimary; curp->file = svfile; } enum mandoclevel mparse_readmem(struct mparse *curp, void *buf, size_t len, const char *file) { struct buf blk; blk.buf = buf; blk.sz = len; mparse_parse_buffer(curp, blk, file); return curp->file_status; } /* * Read the whole file into memory and call the parsers. * Called recursively when an .so request is encountered. */ enum mandoclevel mparse_readfd(struct mparse *curp, int fd, const char *file) { struct buf blk; int with_mmap; int save_filenc; if (read_whole_file(curp, file, fd, &blk, &with_mmap)) { save_filenc = curp->filenc; curp->filenc = curp->options & (MPARSE_UTF8 | MPARSE_LATIN1); mparse_parse_buffer(curp, blk, file); curp->filenc = save_filenc; #if HAVE_MMAP if (with_mmap) munmap(blk.buf, blk.sz); else #endif free(blk.buf); } return curp->file_status; } int mparse_open(struct mparse *curp, const char *file) { char *cp; int fd; curp->file = file; cp = strrchr(file, '.'); curp->gzip = (cp != NULL && ! strcmp(cp + 1, "gz")); /* First try to use the filename as it is. */ if ((fd = open(file, O_RDONLY)) != -1) return fd; /* * If that doesn't work and the filename doesn't * already end in .gz, try appending .gz. */ if ( ! curp->gzip) { mandoc_asprintf(&cp, "%s.gz", file); - fd = open(file, O_RDONLY); + fd = open(cp, O_RDONLY); free(cp); if (fd != -1) { curp->gzip = 1; return fd; } } /* Neither worked, give up. */ mandoc_msg(MANDOCERR_FILE, curp, 0, 0, strerror(errno)); return -1; } struct mparse * mparse_alloc(int options, enum mandoclevel wlevel, mandocmsg mmsg, const char *defos) { struct mparse *curp; curp = mandoc_calloc(1, sizeof(struct mparse)); curp->options = options; curp->wlevel = wlevel; curp->mmsg = mmsg; curp->defos = defos; curp->roff = roff_alloc(curp, options); curp->man = roff_man_alloc( curp->roff, curp, curp->defos, curp->options & MPARSE_QUICK ? 1 : 0); if (curp->options & MPARSE_MDOC) { mdoc_hash_init(); curp->man->macroset = MACROSET_MDOC; } else if (curp->options & MPARSE_MAN) { man_hash_init(); curp->man->macroset = MACROSET_MAN; } curp->man->first->tok = TOKEN_NONE; return curp; } void mparse_reset(struct mparse *curp) { roff_reset(curp->roff); if (curp->man != NULL) roff_man_reset(curp->man); if (curp->secondary) curp->secondary->sz = 0; curp->file_status = MANDOCLEVEL_OK; free(curp->sodest); curp->sodest = NULL; } void mparse_free(struct mparse *curp) { roff_man_free(curp->man); if (curp->roff) roff_free(curp->roff); if (curp->secondary) free(curp->secondary->buf); free(curp->secondary); free(curp->sodest); free(curp); } void mparse_result(struct mparse *curp, struct roff_man **man, char **sodest) { if (sodest && NULL != (*sodest = curp->sodest)) { *man = NULL; return; } if (man) *man = curp->man; } void mandoc_vmsg(enum mandocerr t, struct mparse *m, int ln, int pos, const char *fmt, ...) { char buf[256]; va_list ap; va_start(ap, fmt); (void)vsnprintf(buf, sizeof(buf), fmt, ap); va_end(ap); mandoc_msg(t, m, ln, pos, buf); } void mandoc_msg(enum mandocerr er, struct mparse *m, int ln, int col, const char *msg) { enum mandoclevel level; level = MANDOCLEVEL_UNSUPP; while (er < mandoclimits[level]) level--; if (level < m->wlevel && er != MANDOCERR_FILE) return; if (m->mmsg) (*m->mmsg)(er, level, m->file, ln, col, msg); if (m->file_status < level) m->file_status = level; } const char * mparse_strerror(enum mandocerr er) { return mandocerrs[er]; } const char * mparse_strlevel(enum mandoclevel lvl) { return mandoclevels[lvl]; } void mparse_keep(struct mparse *p) { assert(NULL == p->secondary); p->secondary = mandoc_calloc(1, sizeof(struct buf)); } const char * mparse_getkeep(const struct mparse *p) { assert(p->secondary); return p->secondary->sz ? p->secondary->buf : NULL; } Index: vendor/mdocml/dist/tag.c =================================================================== --- vendor/mdocml/dist/tag.c (revision 303220) +++ vendor/mdocml/dist/tag.c (revision 303221) @@ -1,192 +1,204 @@ -/* $Id: tag.c,v 1.11 2015/11/20 21:59:54 schwarze Exp $ */ +/* $Id: tag.c,v 1.12 2016/07/08 20:42:15 schwarze Exp $ */ /* * Copyright (c) 2015 Ingo Schwarze <schwarze@openbsd.org> * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include "config.h" #include <sys/types.h> #include <signal.h> #include <stddef.h> #include <stdint.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include "mandoc_aux.h" #include "mandoc_ohash.h" #include "tag.h" struct tag_entry { size_t line; int prio; char s[]; }; static void tag_signal(int); static struct ohash tag_data; static struct tag_files tag_files; /* * Prepare for using a pager. * Not all pagers are capable of using a tag file, * but for simplicity, create it anyway. */ struct tag_files * tag_init(void) { struct sigaction sa; int ofd; ofd = -1; tag_files.tfd = -1; tag_files.tcpgid = -1; + /* Clean up when dying from a signal. */ + + memset(&sa, 0, sizeof(sa)); + sigfillset(&sa.sa_mask); + sa.sa_handler = tag_signal; + sigaction(SIGHUP, &sa, NULL); + sigaction(SIGINT, &sa, NULL); + sigaction(SIGTERM, &sa, NULL); + + /* + * POSIX requires that a process calling tcsetpgrp(3) + * from the background gets a SIGTTOU signal. + * In that case, do not stop. + */ + + sa.sa_handler = SIG_IGN; + sigaction(SIGTTOU, &sa, NULL); + /* Save the original standard output for use by the pager. */ if ((tag_files.ofd = dup(STDOUT_FILENO)) == -1) goto fail; /* Create both temporary output files. */ (void)strlcpy(tag_files.ofn, "/tmp/man.XXXXXXXXXX", sizeof(tag_files.ofn)); (void)strlcpy(tag_files.tfn, "/tmp/man.XXXXXXXXXX", sizeof(tag_files.tfn)); - memset(&sa, 0, sizeof(sa)); - sigfillset(&sa.sa_mask); - sa.sa_handler = tag_signal; - sigaction(SIGHUP, &sa, NULL); - sigaction(SIGINT, &sa, NULL); - sigaction(SIGTERM, &sa, NULL); if ((ofd = mkstemp(tag_files.ofn)) == -1) goto fail; if ((tag_files.tfd = mkstemp(tag_files.tfn)) == -1) goto fail; if (dup2(ofd, STDOUT_FILENO) == -1) goto fail; close(ofd); /* * Set up the ohash table to collect output line numbers * where various marked-up terms are documented. */ mandoc_ohash_init(&tag_data, 4, offsetof(struct tag_entry, s)); return &tag_files; fail: tag_unlink(); if (ofd != -1) close(ofd); if (tag_files.ofd != -1) close(tag_files.ofd); if (tag_files.tfd != -1) close(tag_files.tfd); *tag_files.ofn = '\0'; *tag_files.tfn = '\0'; tag_files.ofd = -1; tag_files.tfd = -1; return NULL; } /* * Set the line number where a term is defined, * unless it is already defined at a higher priority. */ void tag_put(const char *s, int prio, size_t line) { struct tag_entry *entry; size_t len; unsigned int slot; if (tag_files.tfd <= 0 || strchr(s, ' ') != NULL) return; slot = ohash_qlookup(&tag_data, s); entry = ohash_find(&tag_data, slot); if (entry == NULL) { len = strlen(s) + 1; entry = mandoc_malloc(sizeof(*entry) + len); memcpy(entry->s, s, len); ohash_insert(&tag_data, slot, entry); } else if (entry->prio <= prio) return; entry->line = line; entry->prio = prio; } /* * Write out the tags file using the previously collected * information and clear the ohash table while going along. */ void tag_write(void) { FILE *stream; struct tag_entry *entry; unsigned int slot; if (tag_files.tfd <= 0) return; stream = fdopen(tag_files.tfd, "w"); entry = ohash_first(&tag_data, &slot); while (entry != NULL) { if (stream != NULL) fprintf(stream, "%s %s %zu\n", entry->s, tag_files.ofn, entry->line); free(entry); entry = ohash_next(&tag_data, &slot); } ohash_delete(&tag_data); if (stream != NULL) fclose(stream); } void tag_unlink(void) { pid_t tc_pgid; if (tag_files.tcpgid != -1) { tc_pgid = tcgetpgrp(STDIN_FILENO); if (tc_pgid == tag_files.pager_pid || tc_pgid == getpgid(0) || getpgid(tc_pgid) == -1) (void)tcsetpgrp(STDIN_FILENO, tag_files.tcpgid); } if (*tag_files.ofn != '\0') unlink(tag_files.ofn); if (*tag_files.tfn != '\0') unlink(tag_files.tfn); } static void tag_signal(int signum) { struct sigaction sa; tag_unlink(); memset(&sa, 0, sizeof(sa)); sigemptyset(&sa.sa_mask); sa.sa_handler = SIG_DFL; sigaction(signum, &sa, NULL); kill(getpid(), signum); /* NOTREACHED */ _exit(1); } Index: vendor/mdocml/dist/term.c =================================================================== --- vendor/mdocml/dist/term.c (revision 303220) +++ vendor/mdocml/dist/term.c (revision 303221) @@ -1,829 +1,829 @@ -/* $Id: term.c,v 1.256 2016/01/07 21:03:54 schwarze Exp $ */ +/* $Id: term.c,v 1.257 2016/04/12 15:30:00 schwarze Exp $ */ /* * Copyright (c) 2008, 2009, 2010, 2011 Kristaps Dzonsons <kristaps@bsd.lv> * Copyright (c) 2010-2015 Ingo Schwarze <schwarze@openbsd.org> * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHORS DISCLAIM ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include "config.h" #include <sys/types.h> #include <assert.h> #include <ctype.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include "mandoc.h" #include "mandoc_aux.h" #include "out.h" #include "term.h" #include "main.h" static size_t cond_width(const struct termp *, int, int *); static void adjbuf(struct termp *p, size_t); static void bufferc(struct termp *, char); static void encode(struct termp *, const char *, size_t); static void encode1(struct termp *, int); void term_free(struct termp *p) { free(p->buf); free(p->fontq); free(p); } void term_begin(struct termp *p, term_margin head, term_margin foot, const struct roff_meta *arg) { p->headf = head; p->footf = foot; p->argf = arg; (*p->begin)(p); } void term_end(struct termp *p) { (*p->end)(p); } /* * Flush a chunk of text. By default, break the output line each time * the right margin is reached, and continue output on the next line * at the same offset as the chunk itself. By default, also break the * output line at the end of the chunk. * The following flags may be specified: * * - TERMP_NOBREAK: Do not break the output line at the right margin, * but only at the max right margin. Also, do not break the output * line at the end of the chunk, such that the next call can pad to * the next column. However, if less than p->trailspace blanks, * which can be 0, 1, or 2, remain to the right margin, the line * will be broken. * - TERMP_BRTRSP: Consider trailing whitespace significant * when deciding whether the chunk fits or not. * - TERMP_BRIND: If the chunk does not fit and the output line has * to be broken, start the next line at the right margin instead * of at the offset. Used together with TERMP_NOBREAK for the tags * in various kinds of tagged lists. * - TERMP_DANGLE: Do not break the output line at the right margin, * append the next chunk after it even if this one is too long. * To be used together with TERMP_NOBREAK. * - TERMP_HANG: Like TERMP_DANGLE, and also suppress padding before * the next chunk if this column is not full. */ void term_flushln(struct termp *p) { size_t i; /* current input position in p->buf */ int ntab; /* number of tabs to prepend */ size_t vis; /* current visual position on output */ size_t vbl; /* number of blanks to prepend to output */ size_t vend; /* end of word visual position on output */ size_t bp; /* visual right border position */ size_t dv; /* temporary for visual pos calculations */ size_t j; /* temporary loop index for p->buf */ size_t jhy; /* last hyph before overflow w/r/t j */ size_t maxvis; /* output position of visible boundary */ /* * First, establish the maximum columns of "visible" content. * This is usually the difference between the right-margin and * an indentation, but can be, for tagged lists or columns, a * small set of values. * * The following unsigned-signed subtractions look strange, * but they are actually correct. If the int p->overstep * is negative, it gets sign extended. Subtracting that * very large size_t effectively adds a small number to dv. */ dv = p->rmargin > p->offset ? p->rmargin - p->offset : 0; maxvis = (int)dv > p->overstep ? dv - (size_t)p->overstep : 0; if (p->flags & TERMP_NOBREAK) { dv = p->maxrmargin > p->offset ? p->maxrmargin - p->offset : 0; bp = (int)dv > p->overstep ? dv - (size_t)p->overstep : 0; } else bp = maxvis; /* * Calculate the required amount of padding. */ vbl = p->offset + p->overstep > p->viscol ? p->offset + p->overstep - p->viscol : 0; vis = vend = 0; i = 0; while (i < p->col) { /* * Handle literal tab characters: collapse all * subsequent tabs into a single huge set of spaces. */ ntab = 0; while (i < p->col && '\t' == p->buf[i]) { vend = (vis / p->tabwidth + 1) * p->tabwidth; vbl += vend - vis; vis = vend; ntab++; i++; } /* * Count up visible word characters. Control sequences * (starting with the CSI) aren't counted. A space * generates a non-printing word, which is valid (the * space is printed according to regular spacing rules). */ for (j = i, jhy = 0; j < p->col; j++) { if (' ' == p->buf[j] || '\t' == p->buf[j]) break; - /* Back over the the last printed character. */ + /* Back over the last printed character. */ if (8 == p->buf[j]) { assert(j); vend -= (*p->width)(p, p->buf[j - 1]); continue; } /* Regular word. */ /* Break at the hyphen point if we overrun. */ if (vend > vis && vend < bp && (ASCII_HYPH == p->buf[j] || ASCII_BREAK == p->buf[j])) jhy = j; /* * Hyphenation now decided, put back a real * hyphen such that we get the correct width. */ if (ASCII_HYPH == p->buf[j]) p->buf[j] = '-'; vend += (*p->width)(p, p->buf[j]); } /* * Find out whether we would exceed the right margin. * If so, break to the next line. */ if (vend > bp && 0 == jhy && vis > 0) { vend -= vis; (*p->endline)(p); p->viscol = 0; if (TERMP_BRIND & p->flags) { vbl = p->rmargin; vend += p->rmargin; vend -= p->offset; } else vbl = p->offset; /* use pending tabs on the new line */ if (0 < ntab) vbl += ntab * p->tabwidth; /* * Remove the p->overstep width. * Again, if p->overstep is negative, * sign extension does the right thing. */ bp += (size_t)p->overstep; p->overstep = 0; } /* Write out the [remaining] word. */ for ( ; i < p->col; i++) { if (vend > bp && jhy > 0 && i > jhy) break; if ('\t' == p->buf[i]) break; if (' ' == p->buf[i]) { j = i; while (i < p->col && ' ' == p->buf[i]) i++; dv = (i - j) * (*p->width)(p, ' '); vbl += dv; vend += dv; break; } if (ASCII_NBRSP == p->buf[i]) { vbl += (*p->width)(p, ' '); continue; } if (ASCII_BREAK == p->buf[i]) continue; /* * Now we definitely know there will be * printable characters to output, * so write preceding white space now. */ if (vbl) { (*p->advance)(p, vbl); p->viscol += vbl; vbl = 0; } (*p->letter)(p, p->buf[i]); if (8 == p->buf[i]) p->viscol -= (*p->width)(p, p->buf[i-1]); else p->viscol += (*p->width)(p, p->buf[i]); } vis = vend; } /* * If there was trailing white space, it was not printed; * so reset the cursor position accordingly. */ if (vis > vbl) vis -= vbl; else vis = 0; p->col = 0; p->overstep = 0; p->flags &= ~(TERMP_BACKAFTER | TERMP_BACKBEFORE); if ( ! (TERMP_NOBREAK & p->flags)) { p->viscol = 0; (*p->endline)(p); return; } if (TERMP_HANG & p->flags) { p->overstep += (int)(p->offset + vis - p->rmargin + p->trailspace * (*p->width)(p, ' ')); /* * If we have overstepped the margin, temporarily move * it to the right and flag the rest of the line to be * shorter. * If there is a request to keep the columns together, * allow negative overstep when the column is not full. */ if (p->trailspace && p->overstep < 0) p->overstep = 0; return; } else if (TERMP_DANGLE & p->flags) return; /* Trailing whitespace is significant in some columns. */ if (vis && vbl && (TERMP_BRTRSP & p->flags)) vis += vbl; /* If the column was overrun, break the line. */ if (maxvis < vis + p->trailspace * (*p->width)(p, ' ')) { (*p->endline)(p); p->viscol = 0; } } /* * A newline only breaks an existing line; it won't assert vertical * space. All data in the output buffer is flushed prior to the newline * assertion. */ void term_newln(struct termp *p) { p->flags |= TERMP_NOSPACE; if (p->col || p->viscol) term_flushln(p); } /* * Asserts a vertical space (a full, empty line-break between lines). * Note that if used twice, this will cause two blank spaces and so on. * All data in the output buffer is flushed prior to the newline * assertion. */ void term_vspace(struct termp *p) { term_newln(p); p->viscol = 0; if (0 < p->skipvsp) p->skipvsp--; else (*p->endline)(p); } /* Swap current and previous font; for \fP and .ft P */ void term_fontlast(struct termp *p) { enum termfont f; f = p->fontl; p->fontl = p->fontq[p->fonti]; p->fontq[p->fonti] = f; } /* Set font, save current, discard previous; for \f, .ft, .B etc. */ void term_fontrepl(struct termp *p, enum termfont f) { p->fontl = p->fontq[p->fonti]; p->fontq[p->fonti] = f; } /* Set font, save previous. */ void term_fontpush(struct termp *p, enum termfont f) { p->fontl = p->fontq[p->fonti]; if (++p->fonti == p->fontsz) { p->fontsz += 8; p->fontq = mandoc_reallocarray(p->fontq, p->fontsz, sizeof(*p->fontq)); } p->fontq[p->fonti] = f; } /* Flush to make the saved pointer current again. */ void term_fontpopq(struct termp *p, int i) { assert(i >= 0); if (p->fonti > i) p->fonti = i; } /* Pop one font off the stack. */ void term_fontpop(struct termp *p) { assert(p->fonti); p->fonti--; } /* * Handle pwords, partial words, which may be either a single word or a * phrase that cannot be broken down (such as a literal string). This * handles word styling. */ void term_word(struct termp *p, const char *word) { const char nbrsp[2] = { ASCII_NBRSP, 0 }; const char *seq, *cp; int sz, uc; size_t ssz; enum mandoc_esc esc; if ( ! (TERMP_NOSPACE & p->flags)) { if ( ! (TERMP_KEEP & p->flags)) { bufferc(p, ' '); if (TERMP_SENTENCE & p->flags) bufferc(p, ' '); } else bufferc(p, ASCII_NBRSP); } if (TERMP_PREKEEP & p->flags) p->flags |= TERMP_KEEP; if ( ! (p->flags & TERMP_NONOSPACE)) p->flags &= ~TERMP_NOSPACE; else p->flags |= TERMP_NOSPACE; p->flags &= ~(TERMP_SENTENCE | TERMP_NONEWLINE); p->skipvsp = 0; while ('\0' != *word) { if ('\\' != *word) { if (TERMP_NBRWORD & p->flags) { if (' ' == *word) { encode(p, nbrsp, 1); word++; continue; } ssz = strcspn(word, "\\ "); } else ssz = strcspn(word, "\\"); encode(p, word, ssz); word += (int)ssz; continue; } word++; esc = mandoc_escape(&word, &seq, &sz); if (ESCAPE_ERROR == esc) continue; switch (esc) { case ESCAPE_UNICODE: uc = mchars_num2uc(seq + 1, sz - 1); break; case ESCAPE_NUMBERED: uc = mchars_num2char(seq, sz); if (uc < 0) continue; break; case ESCAPE_SPECIAL: if (p->enc == TERMENC_ASCII) { cp = mchars_spec2str(seq, sz, &ssz); if (cp != NULL) encode(p, cp, ssz); } else { uc = mchars_spec2cp(seq, sz); if (uc > 0) encode1(p, uc); } continue; case ESCAPE_FONTBOLD: term_fontrepl(p, TERMFONT_BOLD); continue; case ESCAPE_FONTITALIC: term_fontrepl(p, TERMFONT_UNDER); continue; case ESCAPE_FONTBI: term_fontrepl(p, TERMFONT_BI); continue; case ESCAPE_FONT: case ESCAPE_FONTROMAN: term_fontrepl(p, TERMFONT_NONE); continue; case ESCAPE_FONTPREV: term_fontlast(p); continue; case ESCAPE_NOSPACE: if (p->flags & TERMP_BACKAFTER) p->flags &= ~TERMP_BACKAFTER; else if (*word == '\0') p->flags |= (TERMP_NOSPACE | TERMP_NONEWLINE); continue; case ESCAPE_SKIPCHAR: p->flags |= TERMP_BACKAFTER; continue; case ESCAPE_OVERSTRIKE: cp = seq + sz; while (seq < cp) { if (*seq == '\\') { mandoc_escape(&seq, NULL, NULL); continue; } encode1(p, *seq++); if (seq < cp) { if (p->flags & TERMP_BACKBEFORE) p->flags |= TERMP_BACKAFTER; else p->flags |= TERMP_BACKBEFORE; } } /* Trim trailing backspace/blank pair. */ if (p->col > 2 && p->buf[p->col - 1] == ' ') p->col -= 2; continue; default: continue; } /* * Common handling for Unicode and numbered * character escape sequences. */ if (p->enc == TERMENC_ASCII) { cp = ascii_uc2str(uc); encode(p, cp, strlen(cp)); } else { if ((uc < 0x20 && uc != 0x09) || (uc > 0x7E && uc < 0xA0)) uc = 0xFFFD; encode1(p, uc); } } p->flags &= ~TERMP_NBRWORD; } static void adjbuf(struct termp *p, size_t sz) { if (0 == p->maxcols) p->maxcols = 1024; while (sz >= p->maxcols) p->maxcols <<= 2; p->buf = mandoc_reallocarray(p->buf, p->maxcols, sizeof(int)); } static void bufferc(struct termp *p, char c) { if (p->col + 1 >= p->maxcols) adjbuf(p, p->col + 1); p->buf[p->col++] = c; } /* * See encode(). * Do this for a single (probably unicode) value. * Does not check for non-decorated glyphs. */ static void encode1(struct termp *p, int c) { enum termfont f; if (p->col + 7 >= p->maxcols) adjbuf(p, p->col + 7); f = (c == ASCII_HYPH || c > 127 || isgraph(c)) ? p->fontq[p->fonti] : TERMFONT_NONE; if (p->flags & TERMP_BACKBEFORE) { if (p->buf[p->col - 1] == ' ') p->col--; else p->buf[p->col++] = 8; p->flags &= ~TERMP_BACKBEFORE; } if (TERMFONT_UNDER == f || TERMFONT_BI == f) { p->buf[p->col++] = '_'; p->buf[p->col++] = 8; } if (TERMFONT_BOLD == f || TERMFONT_BI == f) { if (ASCII_HYPH == c) p->buf[p->col++] = '-'; else p->buf[p->col++] = c; p->buf[p->col++] = 8; } p->buf[p->col++] = c; if (p->flags & TERMP_BACKAFTER) { p->flags |= TERMP_BACKBEFORE; p->flags &= ~TERMP_BACKAFTER; } } static void encode(struct termp *p, const char *word, size_t sz) { size_t i; if (p->col + 2 + (sz * 5) >= p->maxcols) adjbuf(p, p->col + 2 + (sz * 5)); for (i = 0; i < sz; i++) { if (ASCII_HYPH == word[i] || isgraph((unsigned char)word[i])) encode1(p, word[i]); else p->buf[p->col++] = word[i]; } } void term_setwidth(struct termp *p, const char *wstr) { struct roffsu su; int iop, width; iop = 0; width = 0; if (NULL != wstr) { switch (*wstr) { case '+': iop = 1; wstr++; break; case '-': iop = -1; wstr++; break; default: break; } if (a2roffsu(wstr, &su, SCALE_MAX)) width = term_hspan(p, &su); else iop = 0; } (*p->setwidth)(p, iop, width); } size_t term_len(const struct termp *p, size_t sz) { return (*p->width)(p, ' ') * sz; } static size_t cond_width(const struct termp *p, int c, int *skip) { if (*skip) { (*skip) = 0; return 0; } else return (*p->width)(p, c); } size_t term_strlen(const struct termp *p, const char *cp) { size_t sz, rsz, i; int ssz, skip, uc; const char *seq, *rhs; enum mandoc_esc esc; static const char rej[] = { '\\', ASCII_NBRSP, ASCII_HYPH, ASCII_BREAK, '\0' }; /* * Account for escaped sequences within string length * calculations. This follows the logic in term_word() as we * must calculate the width of produced strings. */ sz = 0; skip = 0; while ('\0' != *cp) { rsz = strcspn(cp, rej); for (i = 0; i < rsz; i++) sz += cond_width(p, *cp++, &skip); switch (*cp) { case '\\': cp++; esc = mandoc_escape(&cp, &seq, &ssz); if (ESCAPE_ERROR == esc) continue; rhs = NULL; switch (esc) { case ESCAPE_UNICODE: uc = mchars_num2uc(seq + 1, ssz - 1); break; case ESCAPE_NUMBERED: uc = mchars_num2char(seq, ssz); if (uc < 0) continue; break; case ESCAPE_SPECIAL: if (p->enc == TERMENC_ASCII) { rhs = mchars_spec2str(seq, ssz, &rsz); if (rhs != NULL) break; } else { uc = mchars_spec2cp(seq, ssz); if (uc > 0) sz += cond_width(p, uc, &skip); } continue; case ESCAPE_SKIPCHAR: skip = 1; continue; case ESCAPE_OVERSTRIKE: rsz = 0; rhs = seq + ssz; while (seq < rhs) { if (*seq == '\\') { mandoc_escape(&seq, NULL, NULL); continue; } i = (*p->width)(p, *seq++); if (rsz < i) rsz = i; } sz += rsz; continue; default: continue; } /* * Common handling for Unicode and numbered * character escape sequences. */ if (rhs == NULL) { if (p->enc == TERMENC_ASCII) { rhs = ascii_uc2str(uc); rsz = strlen(rhs); } else { if ((uc < 0x20 && uc != 0x09) || (uc > 0x7E && uc < 0xA0)) uc = 0xFFFD; sz += cond_width(p, uc, &skip); continue; } } if (skip) { skip = 0; break; } /* * Common handling for all escape sequences * printing more than one character. */ for (i = 0; i < rsz; i++) sz += (*p->width)(p, *rhs++); break; case ASCII_NBRSP: sz += cond_width(p, ' ', &skip); cp++; break; case ASCII_HYPH: sz += cond_width(p, '-', &skip); cp++; break; default: break; } } return sz; } int term_vspan(const struct termp *p, const struct roffsu *su) { double r; int ri; switch (su->unit) { case SCALE_BU: r = su->scale / 40.0; break; case SCALE_CM: r = su->scale * 6.0 / 2.54; break; case SCALE_FS: r = su->scale * 65536.0 / 40.0; break; case SCALE_IN: r = su->scale * 6.0; break; case SCALE_MM: r = su->scale * 0.006; break; case SCALE_PC: r = su->scale; break; case SCALE_PT: r = su->scale / 12.0; break; case SCALE_EN: case SCALE_EM: r = su->scale * 0.6; break; case SCALE_VS: r = su->scale; break; default: abort(); } ri = r > 0.0 ? r + 0.4995 : r - 0.4995; return ri < 66 ? ri : 1; } /* * Convert a scaling width to basic units, rounding down. */ int term_hspan(const struct termp *p, const struct roffsu *su) { return (*p->hspan)(p, su); } Index: vendor/mdocml/dist/term_ascii.c =================================================================== --- vendor/mdocml/dist/term_ascii.c (revision 303220) +++ vendor/mdocml/dist/term_ascii.c (revision 303221) @@ -1,383 +1,382 @@ -/* $Id: term_ascii.c,v 1.52 2015/11/12 21:50:03 schwarze Exp $ */ +/* $Id: term_ascii.c,v 1.53 2016/07/08 22:29:05 schwarze Exp $ */ /* * Copyright (c) 2010, 2011 Kristaps Dzonsons <kristaps@bsd.lv> * Copyright (c) 2014, 2015 Ingo Schwarze <schwarze@openbsd.org> * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHORS DISCLAIM ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include "config.h" #include <sys/types.h> #include <assert.h> #if HAVE_WCHAR #include <locale.h> #endif #include <stdint.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #if HAVE_WCHAR #include <wchar.h> #endif #include "mandoc.h" #include "mandoc_aux.h" #include "out.h" #include "term.h" #include "manconf.h" #include "main.h" static struct termp *ascii_init(enum termenc, const struct manoutput *); static int ascii_hspan(const struct termp *, const struct roffsu *); static size_t ascii_width(const struct termp *, int); static void ascii_advance(struct termp *, size_t); static void ascii_begin(struct termp *); static void ascii_end(struct termp *); static void ascii_endline(struct termp *); static void ascii_letter(struct termp *, int); static void ascii_setwidth(struct termp *, int, int); #if HAVE_WCHAR static void locale_advance(struct termp *, size_t); static void locale_endline(struct termp *); static void locale_letter(struct termp *, int); static size_t locale_width(const struct termp *, int); #endif static struct termp * ascii_init(enum termenc enc, const struct manoutput *outopts) { #if HAVE_WCHAR char *v; #endif struct termp *p; p = mandoc_calloc(1, sizeof(struct termp)); p->line = 1; p->tabwidth = 5; p->defrmargin = p->lastrmargin = 78; p->fontq = mandoc_reallocarray(NULL, (p->fontsz = 8), sizeof(enum termfont)); p->fontq[0] = p->fontl = TERMFONT_NONE; p->begin = ascii_begin; p->end = ascii_end; p->hspan = ascii_hspan; p->type = TERMTYPE_CHAR; p->enc = TERMENC_ASCII; p->advance = ascii_advance; p->endline = ascii_endline; p->letter = ascii_letter; p->setwidth = ascii_setwidth; p->width = ascii_width; #if HAVE_WCHAR if (TERMENC_ASCII != enc) { /* * Do not change any of this to LC_ALL. It might break * the formatting by subtly changing the behaviour of * various functions, for example strftime(3). As a * worst case, it might even cause buffer overflows. */ v = TERMENC_LOCALE == enc ? setlocale(LC_CTYPE, "") : setlocale(LC_CTYPE, "en_US.UTF-8"); if (NULL != v && MB_CUR_MAX > 1) { p->enc = enc; p->advance = locale_advance; p->endline = locale_endline; p->letter = locale_letter; p->width = locale_width; } } #endif if (outopts->mdoc) { p->mdocstyle = 1; p->defindent = 5; } if (outopts->indent) p->defindent = outopts->indent; if (outopts->width) p->defrmargin = outopts->width; if (outopts->synopsisonly) p->synopsisonly = 1; return p; } void * ascii_alloc(const struct manoutput *outopts) { return ascii_init(TERMENC_ASCII, outopts); } void * utf8_alloc(const struct manoutput *outopts) { return ascii_init(TERMENC_UTF8, outopts); } void * locale_alloc(const struct manoutput *outopts) { return ascii_init(TERMENC_LOCALE, outopts); } static void ascii_setwidth(struct termp *p, int iop, int width) { width /= 24; p->rmargin = p->defrmargin; if (iop > 0) p->defrmargin += width; else if (iop == 0) p->defrmargin = width ? (size_t)width : p->lastrmargin; else if (p->defrmargin > (size_t)width) p->defrmargin -= width; else p->defrmargin = 0; p->lastrmargin = p->rmargin; p->rmargin = p->maxrmargin = p->defrmargin; } void -ascii_sepline(void *arg) +terminal_sepline(void *arg) { struct termp *p; size_t i; p = (struct termp *)arg; - p->line += 3; - putchar('\n'); + (*p->endline)(p); for (i = 0; i < p->defrmargin; i++) - putchar('-'); - putchar('\n'); - putchar('\n'); + (*p->letter)(p, '-'); + (*p->endline)(p); + (*p->endline)(p); } static size_t ascii_width(const struct termp *p, int c) { return 1; } void ascii_free(void *arg) { term_free((struct termp *)arg); } static void ascii_letter(struct termp *p, int c) { putchar(c); } static void ascii_begin(struct termp *p) { (*p->headf)(p, p->argf); } static void ascii_end(struct termp *p) { (*p->footf)(p, p->argf); } static void ascii_endline(struct termp *p) { p->line++; putchar('\n'); } static void ascii_advance(struct termp *p, size_t len) { size_t i; for (i = 0; i < len; i++) putchar(' '); } static int ascii_hspan(const struct termp *p, const struct roffsu *su) { double r; switch (su->unit) { case SCALE_BU: r = su->scale; break; case SCALE_CM: r = su->scale * 240.0 / 2.54; break; case SCALE_FS: r = su->scale * 65536.0; break; case SCALE_IN: r = su->scale * 240.0; break; case SCALE_MM: r = su->scale * 0.24; break; case SCALE_VS: case SCALE_PC: r = su->scale * 40.0; break; case SCALE_PT: r = su->scale * 10.0 / 3.0; break; case SCALE_EN: case SCALE_EM: r = su->scale * 24.0; break; default: abort(); } return r > 0.0 ? r + 0.01 : r - 0.01; } const char * ascii_uc2str(int uc) { static const char nbrsp[2] = { ASCII_NBRSP, '\0' }; static const char *tab[] = { "<NUL>","<SOH>","<STX>","<ETX>","<EOT>","<ENQ>","<ACK>","<BEL>", "<BS>", "\t", "<LF>", "<VT>", "<FF>", "<CR>", "<SO>", "<SI>", "<DLE>","<DC1>","<DC2>","<DC3>","<DC4>","<NAK>","<SYN>","<ETB>", "<CAN>","<EM>", "<SUB>","<ESC>","<FS>", "<GS>", "<RS>", "<US>", " ", "!", "\"", "#", "$", "%", "&", "'", "(", ")", "*", "+", ",", "-", ".", "/", "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", ":", ";", "<", "=", ">", "?", "@", "A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V", "W", "X", "Y", "Z", "[", "\\", "]", "^", "_", "`", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "{", "|", "}", "~", "<DEL>", "<80>", "<81>", "<82>", "<83>", "<84>", "<85>", "<86>", "<87>", "<88>", "<89>", "<8A>", "<8B>", "<8C>", "<8D>", "<8E>", "<8F>", "<90>", "<91>", "<92>", "<93>", "<94>", "<95>", "<96>", "<97>", "<99>", "<99>", "<9A>", "<9B>", "<9C>", "<9D>", "<9E>", "<9F>", nbrsp, "!", "/\bc", "GBP", "o\bx", "=\bY", "|", "<sec>", "\"", "(C)", "_\ba", "<<", "~", "", "(R)", "-", "<deg>","+-", "2", "3", "'", ",\bu", "<par>",".", ",", "1", "_\bo", ">>", "1/4", "1/2", "3/4", "?", "`\bA", "'\bA", "^\bA", "~\bA", "\"\bA","o\bA", "AE", ",\bC", "`\bE", "'\bE", "^\bE", "\"\bE","`\bI", "'\bI", "^\bI", "\"\bI", "-\bD", "~\bN", "`\bO", "'\bO", "^\bO", "~\bO", "\"\bO","x", "/\bO", "`\bU", "'\bU", "^\bU", "\"\bU","'\bY", "Th", "ss", "`\ba", "'\ba", "^\ba", "~\ba", "\"\ba","o\ba", "ae", ",\bc", "`\be", "'\be", "^\be", "\"\be","`\bi", "'\bi", "^\bi", "\"\bi", "d", "~\bn", "`\bo", "'\bo", "^\bo", "~\bo", "\"\bo","-:-", "/\bo", "`\bu", "'\bu", "^\bu", "\"\bu","'\by", "th", "\"\by", "A", "a", "A", "a", "A", "a", "'\bC", "'\bc", "^\bC", "^\bc", "C", "c", "C", "c", "D", "d", "/\bD", "/\bd", "E", "e", "E", "e", "E", "e", "E", "e", "E", "e", "^\bG", "^\bg", "G", "g", "G", "g", ",\bG", ",\bg", "^\bH", "^\bh", "/\bH", "/\bh", "~\bI", "~\bi", "I", "i", "I", "i", "I", "i", "I", "i", "IJ", "ij", "^\bJ", "^\bj", ",\bK", ",\bk", "q", "'\bL", "'\bl", ",\bL", ",\bl", "L", "l", "L", "l", "/\bL", "/\bl", "'\bN", "'\bn", ",\bN", ",\bn", "N", "n", "'n", "Ng", "ng", "O", "o", "O", "o", "O", "o", "OE", "oe", "'\bR", "'\br", ",\bR", ",\br", "R", "r", "'\bS", "'\bs", "^\bS", "^\bs", ",\bS", ",\bs", "S", "s", ",\bT", ",\bt", "T", "t", "/\bT", "/\bt", "~\bU", "~\bu", "U", "u", "U", "u", "U", "u", "U", "u", "U", "u", "^\bW", "^\bw", "^\bY", "^\by", "\"\bY","'\bZ", "'\bz", "Z", "z", "Z", "z", "s", "b", "B", "B", "b", "6", "6", "O", "C", "c", "D", "D", "D", "d", "d", "3", "@", "E", "F", ",\bf", "G", "G", "hv", "I", "/\bI", "K", "k", "/\bl", "l", "W", "N", "n", "~\bO", "O", "o", "OI", "oi", "P", "p", "YR", "2", "2", "SH", "sh", "t", "T", "t", "T", "U", "u", "Y", "V", "Y", "y", "/\bZ", "/\bz", "ZH", "ZH", "zh", "zh", "/\b2", "5", "5", "ts", "w", "|", "||", "|=", "!", "DZ", "Dz", "dz", "LJ", "Lj", "lj", "NJ", "Nj", "nj", "A", "a", "I", "i", "O", "o", "U", "u", "U", "u", "U", "u", "U", "u", "U", "u", "@", "A", "a", "A", "a", "AE", "ae", "/\bG", "/\bg", "G", "g", "K", "k", "O", "o", "O", "o", "ZH", "zh", "j", "DZ", "Dz", "dz", "'\bG", "'\bg", "HV", "W", "`\bN", "`\bn", "A", "a", "'\bAE","'\bae","O", "o"}; assert(uc >= 0); if ((size_t)uc < sizeof(tab)/sizeof(tab[0])) return tab[uc]; return mchars_uc2str(uc); } #if HAVE_WCHAR static size_t locale_width(const struct termp *p, int c) { int rc; if (c == ASCII_NBRSP) c = ' '; rc = wcwidth(c); if (rc < 0) rc = 0; return rc; } static void locale_advance(struct termp *p, size_t len) { size_t i; for (i = 0; i < len; i++) putwchar(L' '); } static void locale_endline(struct termp *p) { p->line++; putwchar(L'\n'); } static void locale_letter(struct termp *p, int c) { putwchar(c); } #endif Index: vendor/mdocml/dist/test-rewb-bsd.c =================================================================== --- vendor/mdocml/dist/test-rewb-bsd.c (revision 303220) +++ vendor/mdocml/dist/test-rewb-bsd.c (revision 303221) @@ -1,26 +1,27 @@ #include <sys/types.h> +#include <stddef.h> #include <regex.h> int main(void) { regex_t re; if (regcomp(&re, "[[:<:]]word[[:>:]]", REG_EXTENDED | REG_NOSUB)) return 1; if (regexec(&re, "the word is here", 0, NULL, 0)) return 2; if (regexec(&re, "same word", 0, NULL, 0)) return 3; if (regexec(&re, "word again", 0, NULL, 0)) return 4; if (regexec(&re, "word", 0, NULL, 0)) return 5; if (regexec(&re, "wordy", 0, NULL, 0) != REG_NOMATCH) return 6; if (regexec(&re, "sword", 0, NULL, 0) != REG_NOMATCH) return 7; if (regexec(&re, "reworded", 0, NULL, 0) != REG_NOMATCH) return 8; return 0; } Index: vendor/mdocml/dist/test-rewb-sysv.c =================================================================== --- vendor/mdocml/dist/test-rewb-sysv.c (revision 303220) +++ vendor/mdocml/dist/test-rewb-sysv.c (revision 303221) @@ -1,26 +1,27 @@ #include <sys/types.h> +#include <stddef.h> #include <regex.h> int main(void) { regex_t re; if (regcomp(&re, "\\<word\\>", REG_EXTENDED | REG_NOSUB)) return 1; if (regexec(&re, "the word is here", 0, NULL, 0)) return 2; if (regexec(&re, "same word", 0, NULL, 0)) return 3; if (regexec(&re, "word again", 0, NULL, 0)) return 4; if (regexec(&re, "word", 0, NULL, 0)) return 5; if (regexec(&re, "wordy", 0, NULL, 0) != REG_NOMATCH) return 6; if (regexec(&re, "sword", 0, NULL, 0) != REG_NOMATCH) return 7; if (regexec(&re, "reworded", 0, NULL, 0) != REG_NOMATCH) return 8; return 0; }