Index: stable/10/etc/ntp/leap-seconds =================================================================== --- stable/10/etc/ntp/leap-seconds (revision 299825) +++ stable/10/etc/ntp/leap-seconds (revision 299826) @@ -1,221 +1,221 @@ # # $FreeBSD$ # # In the following text, the symbol '#' introduces # a comment, which continues from that symbol until # the end of the line. A plain comment line has a # whitespace character following the comment indicator. # There are also special comment lines defined below. # A special comment will always have a non-whitespace # character in column 2. # # A blank line should be ignored. # # The following table shows the corrections that must # be applied to compute International Atomic Time (TAI) # from the Coordinated Universal Time (UTC) values that # are transmitted by almost all time services. # # The first column shows an epoch as a number of seconds # since 1900.0 and the second column shows the number of # seconds that must be added to UTC to compute TAI for # any timestamp at or after that epoch. The value on # each line is valid from the indicated initial instant # until the epoch given on the next one or indefinitely # into the future if there is no next line. # (The comment on each line shows the representation of # the corresponding initial epoch in the usual # day-month-year format. The epoch always begins at # 00:00:00 UTC on the indicated day. See Note 5 below.) # # Important notes: # # 1. Coordinated Universal Time (UTC) is often referred to # as Greenwich Mean Time (GMT). The GMT time scale is no # longer used, and the use of GMT to designate UTC is # discouraged. # # 2. The UTC time scale is realized by many national # laboratories and timing centers. Each laboratory # identifies its realization with its name: Thus # UTC(NIST), UTC(USNO), etc. The differences among # these different realizations are typically on the # order of a few nanoseconds (i.e., 0.000 000 00x s) # and can be ignored for many purposes. These differences # are tabulated in Circular T, which is published monthly # by the International Bureau of Weights and Measures # (BIPM). See www.bipm.fr for more information. # -# 3. The current defintion of the relationship between UTC +# 3. The current definition of the relationship between UTC # and TAI dates from 1 January 1972. A number of different # time scales were in use before than epoch, and it can be # quite difficult to compute precise timestamps and time # intervals in those "prehistoric" days. For more information, # consult: # # The Explanatory Supplement to the Astronomical # Ephemeris. # or # Terry Quinn, "The BIPM and the Accurate Measurement # of Time," Proc. of the IEEE, Vol. 79, pp. 894-905, # July, 1991. # # 4. The insertion of leap seconds into UTC is currently the # responsibility of the International Earth Rotation Service, # which is located at the Paris Observatory: # # Central Bureau of IERS # 61, Avenue de l'Observatoire # 75014 Paris, France. # # Leap seconds are announced by the IERS in its Bulletin C # # See hpiers.obspm.fr or www.iers.org for more details. # # All national laboratories and timing centers use the # data from the BIPM and the IERS to construct their # local realizations of UTC. # # Although the definition also includes the possibility # of dropping seconds ("negative" leap seconds), this has # never been done and is unlikely to be necessary in the # foreseeable future. # # 5. If your system keeps time as the number of seconds since # some epoch (e.g., NTP timestamps), then the algorithm for # assigning a UTC time stamp to an event that happens during a positive # leap second is not well defined. The official name of that leap # second is 23:59:60, but there is no way of representing that time # in these systems. # Many systems of this type effectively stop the system clock for # one second during the leap second and use a time that is equivalent # to 23:59:59 UTC twice. For these systems, the corresponding TAI # timestamp would be obtained by advancing to the next entry in the # following table when the time equivalent to 23:59:59 UTC # is used for the second time. Thus the leap second which # occurred on 30 June 1972 at 23:59:59 UTC would have TAI # timestamps computed as follows: # # ... # 30 June 1972 23:59:59 (2287785599, first time): TAI= UTC + 10 seconds # 30 June 1972 23:59:60 (2287785599,second time): TAI= UTC + 11 seconds # 1 July 1972 00:00:00 (2287785600) TAI= UTC + 11 seconds # ... # # If your system realizes the leap second by repeating 00:00:00 UTC twice # (this is possible but not usual), then the advance to the next entry # in the table must occur the second time that a time equivlent to # 00:00:00 UTC is used. Thus, using the same example as above: # # ... # 30 June 1972 23:59:59 (2287785599): TAI= UTC + 10 seconds # 30 June 1972 23:59:60 (2287785600, first time): TAI= UTC + 10 seconds # 1 July 1972 00:00:00 (2287785600,second time): TAI= UTC + 11 seconds # ... # # in both cases the use of timestamps based on TAI produces a smooth # time scale with no discontinuity in the time interval. # # This complexity would not be needed for negative leap seconds (if they # are ever used). The UTC time would skip 23:59:59 and advance from # 23:59:58 to 00:00:00 in that case. The TAI offset would decrease by # 1 second at the same instant. This is a much easier situation to deal # with, since the difficulty of unambiguously representing the epoch # during the leap second does not arise. # # Questions or comments to: # Jeff Prillaman # Time Service Department # US Naval Observatory # Washington, DC # jeffrey.prillaman@usno.navy.mil # # Last Update of leap second values: 31 Dec 2015 # # The following line shows this last update date in NTP timestamp # format. This is the date on which the most recent change to # the leap second data was added to the file. This line can # be identified by the unique pair of characters in the first two # columns as shown below. # #$ 3660508800 # # The data in this file will be updated periodically as new leap # seconds are announced. In addition to being entered on the line # above, the update time (in NTP format) will be added to the basic # file name leap-seconds to form the name leap-seconds.. # In addition, the generic name leap-seconds.list will always point to # the most recent version of the file. # # This update procedure will be performed only when a new leap second # is announced. # # The following entry specifies the expiration date of the data # in this file in units of seconds since 1900.0. This expiration date # will be changed at least twice per year whether or not a new leap # second is announced. These semi-annual changes will be made no # later than 1 June and 1 December of each year to indicate what # action (if any) is to be taken on 30 June and 31 December, # respectively. (These are the customary effective dates for new # leap seconds.) This expiration date will be identified by a # unique pair of characters in columns 1 and 2 as shown below. # In the unlikely event that a leap second is announced with an # effective date other than 30 June or 31 December, then this # file will be edited to include that leap second as soon as it is # announced or at least one month before the effective date # (whichever is later). # If an announcement by the IERS specifies that no leap second is # scheduled, then only the expiration date of the file will # be advanced to show that the information in the file is still # current -- the update time stamp, the data and the name of the file # will not change. # # Updated through IERS Bulletin C 50 # File expires on: 1 Jun 2016 # #@ 3673728000 # 2272060800 10 # 1 Jan 1972 2287785600 11 # 1 Jul 1972 2303683200 12 # 1 Jan 1973 2335219200 13 # 1 Jan 1974 2366755200 14 # 1 Jan 1975 2398291200 15 # 1 Jan 1976 2429913600 16 # 1 Jan 1977 2461449600 17 # 1 Jan 1978 2492985600 18 # 1 Jan 1979 2524521600 19 # 1 Jan 1980 2571782400 20 # 1 Jul 1981 2603318400 21 # 1 Jul 1982 2634854400 22 # 1 Jul 1983 2698012800 23 # 1 Jul 1985 2776982400 24 # 1 Jan 1988 2840140800 25 # 1 Jan 1990 2871676800 26 # 1 Jan 1991 2918937600 27 # 1 Jul 1992 2950473600 28 # 1 Jul 1993 2982009600 29 # 1 Jul 1994 3029443200 30 # 1 Jan 1996 3076704000 31 # 1 Jul 1997 3124137600 32 # 1 Jan 1999 3345062400 33 # 1 Jan 2006 3439756800 34 # 1 Jan 2009 3550089600 35 # 1 Jul 2012 3644697600 36 # 1 Jul 2015 # # the following special comment contains the # hash value of the data in this file computed # use the secure hash algorithm as specified # by FIPS 180-1. See the files in ~/sha for # the details of how this hash value is # computed. Note that the hash computation # ignores comments and whitespace characters # in data lines. It includes the NTP values # of both the last modification time and the # expiration time of the file, but not the # white space on those lines. # the hash line is also ignored in the # computation. # #h 44a44c49 35b22601 a9c7054c 8c56cf57 9b6f6ed5 # Index: stable/10/etc/protocols =================================================================== --- stable/10/etc/protocols (revision 299825) +++ stable/10/etc/protocols (revision 299826) @@ -1,158 +1,158 @@ # # Internet protocols # # $FreeBSD$ # from: @(#)protocols 5.1 (Berkeley) 4/17/89 # # See also http://www.iana.org/assignments/protocol-numbers # ip 0 IP # internet protocol, pseudo protocol number #hopopt 0 HOPOPT # hop-by-hop options for ipv6 icmp 1 ICMP # internet control message protocol igmp 2 IGMP # internet group management protocol ggp 3 GGP # gateway-gateway protocol ipencap 4 IP-ENCAP # IP encapsulated in IP (officially ``IP'') st2 5 ST2 # ST2 datagram mode (RFC 1819) (officially ``ST'') tcp 6 TCP # transmission control protocol cbt 7 CBT # CBT, Tony Ballardie egp 8 EGP # exterior gateway protocol igp 9 IGP # any private interior gateway (Cisco: for IGRP) bbn-rcc 10 BBN-RCC-MON # BBN RCC Monitoring nvp 11 NVP-II # Network Voice Protocol pup 12 PUP # PARC universal packet protocol argus 13 ARGUS # ARGUS emcon 14 EMCON # EMCON xnet 15 XNET # Cross Net Debugger chaos 16 CHAOS # Chaos udp 17 UDP # user datagram protocol mux 18 MUX # Multiplexing protocol dcn 19 DCN-MEAS # DCN Measurement Subsystems hmp 20 HMP # host monitoring protocol prm 21 PRM # packet radio measurement protocol xns-idp 22 XNS-IDP # Xerox NS IDP trunk-1 23 TRUNK-1 # Trunk-1 trunk-2 24 TRUNK-2 # Trunk-2 leaf-1 25 LEAF-1 # Leaf-1 leaf-2 26 LEAF-2 # Leaf-2 rdp 27 RDP # "reliable datagram" protocol irtp 28 IRTP # Internet Reliable Transaction Protocol iso-tp4 29 ISO-TP4 # ISO Transport Protocol Class 4 netblt 30 NETBLT # Bulk Data Transfer Protocol mfe-nsp 31 MFE-NSP # MFE Network Services Protocol merit-inp 32 MERIT-INP # MERIT Internodal Protocol dccp 33 DCCP # Datagram Congestion Control Protocol 3pc 34 3PC # Third Party Connect Protocol idpr 35 IDPR # Inter-Domain Policy Routing Protocol -xtp 36 XTP # Xpress Tranfer Protocol +xtp 36 XTP # Xpress Transfer Protocol ddp 37 DDP # Datagram Delivery Protocol idpr-cmtp 38 IDPR-CMTP # IDPR Control Message Transport Proto tp++ 39 TP++ # TP++ Transport Protocol il 40 IL # IL Transport Protocol ipv6 41 IPV6 # ipv6 sdrp 42 SDRP # Source Demand Routing Protocol ipv6-route 43 IPV6-ROUTE # routing header for ipv6 ipv6-frag 44 IPV6-FRAG # fragment header for ipv6 idrp 45 IDRP # Inter-Domain Routing Protocol rsvp 46 RSVP # Resource ReSerVation Protocol gre 47 GRE # Generic Routing Encapsulation dsr 48 DSR # Dynamic Source Routing Protocol bna 49 BNA # BNA esp 50 ESP # encapsulating security payload ah 51 AH # authentication header i-nlsp 52 I-NLSP # Integrated Net Layer Security TUBA swipe 53 SWIPE # IP with Encryption narp 54 NARP # NBMA Address Resolution Protocol mobile 55 MOBILE # IP Mobility tlsp 56 TLSP # Transport Layer Security Protocol skip 57 SKIP # SKIP ipv6-icmp 58 IPV6-ICMP icmp6 # ICMP for IPv6 ipv6-nonxt 59 IPV6-NONXT # no next header for ipv6 ipv6-opts 60 IPV6-OPTS # destination options for ipv6 # 61 # any host internal protocol cftp 62 CFTP # CFTP # 63 # any local network sat-expak 64 SAT-EXPAK # SATNET and Backroom EXPAK kryptolan 65 KRYPTOLAN # Kryptolan rvd 66 RVD # MIT Remote Virtual Disk Protocol ippc 67 IPPC # Internet Pluribus Packet Core # 68 # any distributed filesystem sat-mon 69 SAT-MON # SATNET Monitoring visa 70 VISA # VISA Protocol ipcv 71 IPCV # Internet Packet Core Utility cpnx 72 CPNX # Computer Protocol Network Executive cphb 73 CPHB # Computer Protocol Heart Beat wsn 74 WSN # Wang Span Network pvp 75 PVP # Packet Video Protocol br-sat-mon 76 BR-SAT-MON # Backroom SATNET Monitoring sun-nd 77 SUN-ND # SUN ND PROTOCOL-Temporary wb-mon 78 WB-MON # WIDEBAND Monitoring wb-expak 79 WB-EXPAK # WIDEBAND EXPAK iso-ip 80 ISO-IP # ISO Internet Protocol vmtp 81 VMTP # Versatile Message Transport secure-vmtp 82 SECURE-VMTP # SECURE-VMTP vines 83 VINES # VINES ttp 84 TTP # TTP #iptm 84 IPTM # Protocol Internet Protocol Traffic nsfnet-igp 85 NSFNET-IGP # NSFNET-IGP dgp 86 DGP # Dissimilar Gateway Protocol tcf 87 TCF # TCF eigrp 88 EIGRP # Enhanced Interior Routing Protocol (Cisco) ospf 89 OSPFIGP # Open Shortest Path First IGP sprite-rpc 90 Sprite-RPC # Sprite RPC Protocol larp 91 LARP # Locus Address Resolution Protocol mtp 92 MTP # Multicast Transport Protocol ax.25 93 AX.25 # AX.25 Frames ipip 94 IPIP # Yet Another IP encapsulation micp 95 MICP # Mobile Internetworking Control Pro. scc-sp 96 SCC-SP # Semaphore Communications Sec. Pro. etherip 97 ETHERIP # Ethernet-within-IP Encapsulation encap 98 ENCAP # Yet Another IP encapsulation # 99 # any private encryption scheme gmtp 100 GMTP # GMTP ifmp 101 IFMP # Ipsilon Flow Management Protocol pnni 102 PNNI # PNNI over IP pim 103 PIM # Protocol Independent Multicast aris 104 ARIS # ARIS scps 105 SCPS # SCPS qnx 106 QNX # QNX a/n 107 A/N # Active Networks ipcomp 108 IPComp # IP Payload Compression Protocol snp 109 SNP # Sitara Networks Protocol compaq-peer 110 Compaq-Peer # Compaq Peer Protocol ipx-in-ip 111 IPX-in-IP # IPX in IP carp 112 CARP vrrp # Common Address Redundancy Protocol pgm 113 PGM # PGM Reliable Transport Protocol # 114 # any 0-hop protocol l2tp 115 L2TP # Layer Two Tunneling Protocol ddx 116 DDX # D-II Data Exchange iatp 117 IATP # Interactive Agent Transfer Protocol stp 118 STP # Schedule Transfer Protocol srp 119 SRP # SpectraLink Radio Protocol uti 120 UTI # UTI smp 121 SMP # Simple Message Protocol sm 122 SM # SM ptp 123 PTP # Performance Transparency Protocol isis 124 ISIS # ISIS over IPv4 fire 125 FIRE crtp 126 CRTP # Combat Radio Transport Protocol crudp 127 CRUDP # Combat Radio User Datagram sscopmce 128 SSCOPMCE iplt 129 IPLT sps 130 SPS # Secure Packet Shield pipe 131 PIPE # Private IP Encapsulation within IP sctp 132 SCTP # Stream Control Transmission Protocol fc 133 FC # Fibre Channel rsvp-e2e-ignore 134 RSVP-E2E-IGNORE # Aggregation of RSVP for IP reservations mobility-header 135 Mobility-Header # Mobility Support in IPv6 udplite 136 UDPLite # The UDP-Lite Protocol mpls-in-ip 137 MPLS-IN-IP # Encapsulating MPLS in IP manet 138 MANET # MANET Protocols (RFC5498) hip 139 HIP # Host Identity Protocol (RFC5201) shim6 140 SHIM6 # Shim6 Protocol (RFC5533) wesp 141 WESP # Wrapped Encapsulating Security Payload (RFC5840) rohc 142 ROHC # Robust Header Compression (RFC5858) # 138-254 # Unassigned pfsync 240 PFSYNC # PF Synchronization # 253-254 # Use for experimentation and testing (RFC3692) # 255 # Reserved divert 258 DIVERT # Divert pseudo-protocol [non IANA] Index: stable/10/etc/rc.d/sendmail =================================================================== --- stable/10/etc/rc.d/sendmail (revision 299825) +++ stable/10/etc/rc.d/sendmail (revision 299826) @@ -1,221 +1,221 @@ #!/bin/sh # # $FreeBSD$ # # PROVIDE: mail # REQUIRE: LOGIN FILESYSTEMS # we make mail start late, so that things like .forward's are not # processed until the system is fully operational # KEYWORD: shutdown # XXX - Get together with sendmail mantainer to figure out how to # better handle SENDMAIL_ENABLE and 3rd party MTAs. # . /etc/rc.subr name="sendmail" rcvar="sendmail_enable" required_files="/etc/mail/${name}.cf" start_precmd="sendmail_precmd" load_rc_config $name command=${sendmail_program:-/usr/sbin/${name}} pidfile=${sendmail_pidfile:-/var/run/${name}.pid} procname=${sendmail_procname:-/usr/sbin/${name}} CERTDIR=/etc/mail/certs case ${sendmail_enable} in [Nn][Oo][Nn][Ee]) sendmail_enable="NO" sendmail_submit_enable="NO" sendmail_outbound_enable="NO" sendmail_msp_queue_enable="NO" ;; esac # If sendmail_enable=yes, don't need submit or outbound daemon if checkyesno sendmail_enable; then sendmail_submit_enable="NO" sendmail_outbound_enable="NO" fi # If sendmail_submit_enable=yes, don't need outbound daemon if checkyesno sendmail_submit_enable; then sendmail_outbound_enable="NO" fi sendmail_cert_create() { cnname="${sendmail_cert_cn:-`hostname`}" cnname="${cnname:-amnesiac}" # based upon: # http://www.sendmail.org/~ca/email/other/cagreg.html CAdir=`mktemp -d` && certpass=`(date; ps ax ; hostname) | md5 -q` # make certificate authority ( cd "$CAdir" && chmod 700 "$CAdir" && mkdir certs crl newcerts && echo "01" > serial && :> index.txt && cat <<-OPENSSL_CNF > openssl.cnf && RANDFILE = $CAdir/.rnd [ ca ] default_ca = CA_default [ CA_default ] dir = . certs = \$dir/certs # Where the issued certs are kept crl_dir = \$dir/crl # Where the issued crl are kept database = \$dir/index.txt # database index file. new_certs_dir = \$dir/newcerts # default place for new certs. certificate = \$dir/cacert.pem # The CA certificate serial = \$dir/serial # The current serial number crlnumber = \$dir/crlnumber # the current crl number crl = \$dir/crl.pem # The current CRL private_key = \$dir/cakey.pem - x509_extensions = usr_cert # The extentions to add to the cert + x509_extensions = usr_cert # The extensions to add to the cert name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options default_days = 365 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = default # use public key default MD preserve = no # keep passed DN ordering policy = policy_anything [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional [ req ] default_bits = 2048 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes - x509_extensions = v3_ca # The extentions to add to the self signed cert + x509_extensions = v3_ca # The extensions to add to the self signed cert string_mask = utf8only prompt = no [ req_distinguished_name ] countryName = XX stateOrProvinceName = Some-state localityName = Some-city 0.organizationName = Some-org CN = $cnname [ req_attributes ] challengePassword = foobar unstructuredName = An optional company name [ usr_cert ] basicConstraints=CA:FALSE nsComment = "OpenSSL Generated Certificate" subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer [ v3_req ] basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer basicConstraints = CA:true OPENSSL_CNF # though we use a password, the key is discarded and never used openssl req -batch -passout pass:"$certpass" -new -x509 \ -keyout cakey.pem -out cacert.pem -days 3650 \ -config openssl.cnf -newkey rsa:2048 >/dev/null 2>&1 && # make new certificate openssl req -batch -nodes -new -x509 -keyout newkey.pem \ -out newreq.pem -days 365 -config openssl.cnf \ -newkey rsa:2048 >/dev/null 2>&1 && # sign certificate openssl x509 -x509toreq -in newreq.pem -signkey newkey.pem \ -out tmp.pem >/dev/null 2>&1 && openssl ca -notext -config openssl.cnf \ -out newcert.pem -keyfile cakey.pem -cert cacert.pem \ -key "$certpass" -batch -infiles tmp.pem >/dev/null 2>&1 && mkdir -p "$CERTDIR" && chmod 0755 "$CERTDIR" && chmod 644 newcert.pem cacert.pem && chmod 600 newkey.pem && cp -p newcert.pem "$CERTDIR"/host.cert && cp -p cacert.pem "$CERTDIR"/cacert.pem && cp -p newkey.pem "$CERTDIR"/host.key && ln -s cacert.pem "$CERTDIR"/`openssl x509 -hash -noout \ -in cacert.pem`.0) retVal="$?" rm -rf "$CAdir" return "$retVal" } sendmail_precmd() { # Die if there's pre-8.10 custom configuration file. This check is # mandatory for smooth upgrade. See NetBSD PR 10100 for details. # if checkyesno ${rcvar} && [ -f "/etc/${name}.cf" ]; then if ! cmp -s "/etc/mail/${name}.cf" "/etc/${name}.cf"; then warn \ "${name} was not started; you have multiple copies of sendmail.cf." return 1 fi fi # check modifications on /etc/mail/aliases if checkyesno sendmail_rebuild_aliases; then if [ -f "/etc/mail/aliases.db" ]; then if [ "/etc/mail/aliases" -nt "/etc/mail/aliases.db" ]; then echo \ "${name}: /etc/mail/aliases newer than /etc/mail/aliases.db, regenerating" /usr/bin/newaliases fi else echo \ "${name}: /etc/mail/aliases.db not present, generating" /usr/bin/newaliases fi fi if checkyesno sendmail_cert_create && [ ! \( \ -f "$CERTDIR/host.cert" -o -f "$CERTDIR/host.key" -o \ -f "$CERTDIR/cacert.pem" \) ]; then if ! openssl version >/dev/null 2>&1; then warn "OpenSSL not available, but sendmail_cert_create is YES." else info Creating certificate for sendmail. sendmail_cert_create fi fi } run_rc_command "$1" required_files= if checkyesno sendmail_submit_enable; then name="sendmail_submit" rcvar="sendmail_submit_enable" run_rc_command "$1" fi if checkyesno sendmail_outbound_enable; then name="sendmail_outbound" rcvar="sendmail_outbound_enable" run_rc_command "$1" fi name="sendmail_msp_queue" rcvar="sendmail_msp_queue_enable" pidfile="${sendmail_msp_queue_pidfile:-/var/spool/clientmqueue/sm-client.pid}" required_files="/etc/mail/submit.cf" run_rc_command "$1" Index: stable/10/etc/rc.initdiskless =================================================================== --- stable/10/etc/rc.initdiskless (revision 299825) +++ stable/10/etc/rc.initdiskless (revision 299826) @@ -1,381 +1,381 @@ #!/bin/sh # # Copyright (c) 1999 Matt Dillon # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # $FreeBSD$ # On entry to this script the entire system consists of a read-only root # mounted via NFS. The kernel has run BOOTP and configured an interface # (otherwise it would not have been able to mount the NFS root!) # # We use the contents of /conf to create and populate memory filesystems # that are mounted on top of this root to implement the writable # (and host-specific) parts of the root filesystem, and other volatile # filesystems. # # The hierarchy in /conf has the form /conf/T/M/ where M are directories # for which memory filesystems will be created and filled, # and T is one of the "template" directories below: # # base universal base, typically a replica of the original root; # default secondary universal base, typically overriding some # of the files in the original root; # ${ipba} where ${ipba} is the assigned broadcast IP address # bcast/${ipba} same as above # ${class} where ${class} is a list of directories supplied by # bootp/dhcp through the T134 option. # ${ipba} and ${class} are typically used to configure features # for group of diskless clients, or even individual features; # ${ip} where ${ip} is the machine's assigned IP address, typically # used to set host-specific features; # ip/${ip} same as above # # Template directories are scanned in the order they are listed above, # with each successive directory overriding (merged into) the previous one; # non-existing directories are ignored. The subdirectory forms exist to # help keep the top level /conf manageable in large installations. # # The existence of a directory /conf/T/M causes this script to create a # memory filesystem mounted as /M on the client. # # Some files in /conf have special meaning, namely: # # Filename Action # ---------------------------------------------------------------- # /conf/T/M/remount # The contents of the file is a mount command. E.g. if # /conf/1.2.3.4/foo/remount contains "mount -o ro /dev/ad0s3", # then /dev/ad0s3 will be be mounted on /conf/1.2.3.4/foo/ # # /conf/T/M/remount_optional # If this file exists, then failure to execute the mount # command contained in /conf/T/M/remount is non-fatal. # # /conf/T/M/remount_subdir # If this file exists, then the behaviour of /conf/T/M/remount # changes as follows: # 1. /conf/T/M/remount is invoked to mount the root of the # filesystem where the configuration data exists on a # temporary mountpoint. # 2. /conf/T/M/remount_subdir is then invoked to mount a # *subdirectory* of the filesystem mounted by # /conf/T/M/remount on /conf/T/M/. # # /conf/T/M/diskless_remount # The contents of the file points to an NFS filesystem, # possibly followed by mount_nfs options. If the server name # is omitted, the script will prepend the root path used when # booting. E.g. if you booted from foo.com:/path/to/root, # an entry for /conf/base/etc/diskless_remount could be any of # foo.com:/path/to/root/etc # /etc -o ro # Because mount_nfs understands ".." in paths, it is # possible to mount from locations above the NFS root with # paths such as "/../../etc". # # /conf/T/M/md_size # The contents of the file specifies the size of the memory # filesystem to be created, in 512 byte blocks. # The default size is 10240 blocks (5MB). E.g. if # /conf/base/etc/md_size contains "30000" then a 15MB MFS # will be created. In case of multiple entries for the same # directory M, the last one in the scanning order is used. # NOTE: If you only need to create a memory filesystem but not -# initialize it from a template, it is preferrable to specify +# initialize it from a template, it is preferable to specify # it in fstab e.g. as "md /tmp mfs -s=30m,rw 0 0" # # /conf/T/SUBDIR.cpio.gz # The file is cpio'd into /SUBDIR (and a memory filesystem is # created for /SUBDIR if necessary). The presence of this file # prevents the copy from /conf/T/SUBDIR/ # # /conf/T/SUBDIR.remove # The list of paths contained in the file are rm -rf'd # relative to /SUBDIR. # # /conf/diskless_remount # Similar to /conf/T/M/diskless_remount above, but allows # all of /conf to be remounted. This can be used to allow # multiple roots to share the same /conf. # # # You will almost universally want to create the following files under /conf # # File Content # ---------------------------- ---------------------------------- # /conf/base/etc/md_size size of /etc filesystem # /conf/base/etc/diskless_remount "/etc" # /conf/default/etc/rc.conf generic diskless config parameters # /conf/default/etc/fstab generic diskless fstab e.g. like this # # foo:/root_part / nfs ro 0 0 # foo:/usr_part /usr nfs ro 0 0 # foo:/home_part /home nfs rw 0 0 # md /tmp mfs -s=30m,rw 0 0 # md /var mfs -s=30m,rw 0 0 # proc /proc procfs rw 0 0 # # plus, possibly, overrides for password files etc. # # NOTE! /var, /tmp, and /dev will be typically created elsewhere, e.g. # as entries in the fstab as above. # Those filesystems should not be specified in /conf. # # (end of documentation, now get to the real code) dlv=`/sbin/sysctl -n vfs.nfs.diskless_valid 2> /dev/null` # DEBUGGING # log something on stdout if verbose. o_verbose=0 # set to 1 or 2 if you want more debugging log() { [ ${o_verbose} -gt 0 ] && echo "*** $* ***" [ ${o_verbose} -gt 1 ] && read -p "=== Press enter to continue" foo } # chkerr: # # Routine to check for error # # checks error code and drops into shell on failure. # if shell exits, terminates script as well as /etc/rc. # if remount_optional exists under the mountpoint, skip this check. # chkerr() { lastitem () ( n=$(($# - 1)) ; shift $n ; echo $1 ) mountpoint="$(lastitem $2)" [ -r $mountpoint/remount_optional ] && ( echo "$2 failed: ignoring due to remount_optional" ; return ) case $1 in 0) ;; *) echo "$2 failed: dropping into /bin/sh" /bin/sh # RESUME ;; esac } # The list of filesystems to umount after the copy to_umount="" handle_remount() { # $1 = mount point local nfspt mountopts b b=$1 log handle_remount $1 [ -d $b -a -f $b/diskless_remount ] || return read nfspt mountopts < $b/diskless_remount log "nfspt ${nfspt} mountopts ${mountopts}" # prepend the nfs root if not present [ `expr "$nfspt" : '\(.\)'` = "/" ] && nfspt="${nfsroot}${nfspt}" mount_nfs $mountopts $nfspt $b chkerr $? "mount_nfs $nfspt $b" to_umount="$b ${to_umount}" } # Create a generic memory disk # mount_md() { /sbin/mdmfs -S -i 4096 -s $1 -M md $2 } # Create the memory filesystem if it has not already been created # create_md() { [ "x`eval echo \\$md_created_$1`" = "x" ] || return # only once if [ "x`eval echo \\$md_size_$1`" = "x" ]; then md_size=10240 else md_size=`eval echo \\$md_size_$1` fi log create_md $1 with size $md_size mount_md $md_size /$1 /bin/chmod 755 /$1 eval md_created_$1=created } # DEBUGGING # # set -v # Figure out our interface and IP. # bootp_ifc="" bootp_ipa="" bootp_ipbca="" class="" if [ ${dlv:=0} -ne 0 ] ; then iflist=`ifconfig -l` for i in ${iflist} ; do set -- `ifconfig ${i}` while [ $# -ge 1 ] ; do if [ "${bootp_ifc}" = "" -a "$1" = "inet" ] ; then bootp_ifc=${i} ; bootp_ipa=${2} ; shift fi if [ "${bootp_ipbca}" = "" -a "$1" = "broadcast" ] ; then bootp_ipbca=$2; shift fi shift done if [ "${bootp_ifc}" != "" ] ; then break fi done # Get the values passed with the T134 bootp cookie. class="`/sbin/sysctl -qn kern.bootp_cookie`" echo "Interface ${bootp_ifc} IP-Address ${bootp_ipa} Broadcast ${bootp_ipbca} ${class}" fi log Figure out our NFS root path # set -- `mount -t nfs` while [ $# -ge 1 ] ; do if [ "$2" = "on" -a "$3" = "/" ]; then nfsroot="$1" break fi shift done # The list of directories with template files templates="base default" if [ -n "${bootp_ipbca}" ]; then templates="${templates} ${bootp_ipbca} bcast/${bootp_ipbca}" fi if [ -n "${class}" ]; then templates="${templates} ${class}" fi if [ -n "${bootp_ipa}" ]; then templates="${templates} ${bootp_ipa} ip/${bootp_ipa}" fi # If /conf/diskless_remount exists, remount all of /conf. handle_remount /conf # Resolve templates in /conf/base, /conf/default, /conf/${bootp_ipbca}, # and /conf/${bootp_ipa}. For each subdirectory found within these # directories: # # - calculate memory filesystem sizes. If the subdirectory (prior to # NFS remounting) contains the file 'md_size', the contents specified # in 512 byte sectors will be used to size the memory filesystem. Otherwise # 8192 sectors (4MB) is used. # # - handle NFS remounts. If the subdirectory contains the file # diskless_remount, the contents of the file is NFS mounted over # the directory. For example /conf/base/etc/diskless_remount # might contain 'myserver:/etc'. NFS remounts allow you to avoid # having to dup your system directories in /conf. Your server must # be sure to export those filesystems -alldirs, however. # If the diskless_remount file contains a string beginning with a # '/' it is assumed that the local nfsroot should be prepended to # it before attemping to the remount. This allows the root to be # relocated without needing to change the remount files. # log "templates are ${templates}" for i in ${templates} ; do for j in /conf/$i/* ; do [ -d $j ] || continue # memory filesystem size specification subdir=${j##*/} [ -f $j/md_size ] && eval md_size_$subdir=`cat $j/md_size` # remount. Beware, the command is in the file itself! if [ -f $j/remount ]; then if [ -f $j/remount_subdir ]; then k="/conf.tmp/$i/$subdir" [ -d $k ] || continue # Mount the filesystem root where the config data is # on the temporary mount point. nfspt=`/bin/cat $j/remount` $nfspt $k chkerr $? "$nfspt $k" # Now use a nullfs mount to get the data where we # really want to see it. remount_subdir=`/bin/cat $j/remount_subdir` remount_subdir_cmd="mount -t nullfs $k/$remount_subdir" $remount_subdir_cmd $j chkerr $? "$remount_subdir_cmd $j" # XXX check order -- we must force $k to be unmounted # after j, as j depends on k. to_umount="$j $k ${to_umount}" else nfspt=`/bin/cat $j/remount` $nfspt $j chkerr $? "$nfspt $j" to_umount="$j ${to_umount}" # XXX hope it is really a mount! fi fi # NFS remount handle_remount $j done done # - Create all required MFS filesystems and populate them from # our templates. Support both a direct template and a dir.cpio.gz # archive. Support dir.remove files containing a list of relative # paths to remove. # # The dir.cpio.gz form is there to make the copy process more efficient, # so if the cpio archive is present, it prevents the files from dir/ # from being copied. for i in ${templates} ; do for j in /conf/$i/* ; do subdir=${j##*/} if [ -d $j -a ! -f $j.cpio.gz ]; then create_md $subdir cp -Rp $j/ /$subdir fi done for j in /conf/$i/*.cpio.gz ; do subdir=${j%*.cpio.gz} subdir=${subdir##*/} if [ -f $j ]; then create_md $subdir echo "Loading /$subdir from cpio archive $j" (cd / ; /rescue/tar -xpf $j) fi done for j in /conf/$i/*.remove ; do subdir=${j%*.remove} subdir=${subdir##*/} if [ -f $j ]; then # doubly sure it is a memory disk before rm -rf'ing create_md $subdir (cd /$subdir; rm -rf `/bin/cat $j`) fi done done # umount partitions used to fill the memory filesystems [ -n "${to_umount}" ] && umount $to_umount Index: stable/10/share/examples/ipfilter/ipf-howto.txt =================================================================== --- stable/10/share/examples/ipfilter/ipf-howto.txt (revision 299825) +++ stable/10/share/examples/ipfilter/ipf-howto.txt (revision 299826) @@ -1,3168 +1,3168 @@ IP Filter Based Firewalls HOWTO Brendan Conoboy Erik Fichtner $FreeBSD$ Fri Apr 20 09:31:14 EDT 2001 Abstract: This document is intended to introduce a new user to the IP Filter firewalling package and, at the same time, teach the user some basic fundamentals of good firewall design. 1. Introduction IP Filter is a great little firewall package. It does just about everything other free firewalls (ipfwadm, ipchains, ipfw) do, but it's also portable and does neat stuff the others don't. This document is intended to make some cohesive sense of the sparse documentation presently available for ipfilter. Some prior familiarity with packet filtering will be useful, however too much familiarity may make this document a waste of your time. For greater under- standing of firewalls, the authors recommend reading Build- ing Internet Firewalls, Chapman & Zwicky, O'Reilly and Asso- ciates; and TCP/IP Illustrated, Volume 1, Stevens, Addison- Wesley. 1.1. Disclaimer The authors of this document are not responsible for any damages incurred due to actions taken based on this doc- ument. This document is meant as an introduction to building a firewall based on IP-Filter. If you do not feel -2- comfortable taking responsibility for your own actions, you should stop reading this document and hire a qualified secu- rity professional to install your firewall for you. 1.2. Copyright Unless otherwise stated, HOWTO documents are copy- righted by their respective authors. HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the authors would like to be notified of any such distributions. All translations, derivative works, or aggregate works incorporating any HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the HOWTO coordinator. In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs. 1.3. Where to obtain the important pieces The official IPF homepage is at: The most up-to-date version of this document can be found at: 2. Basic Firewalling This section is designed to familiarize you with ipfil- ter's syntax, and firewall theory in general. The features discussed here are features you'll find in any good firewall package. This section will give you a good foundation to make reading and understanding the advanced section very easy. It must be emphasized that this section alone is not enough to build a good firewall, and that the advanced sec- tion really is required reading for anybody who wants to build an effective security system. -3- 2.1. Config File Dynamics, Order and Precedence IPF (IP Filter) has a config file (as opposed to say, running some command again and again for each new rule). The config file drips with Unix: There's one rule per line, the "#" mark denotes a comment, and you can have a rule and a comment on the same line. Extraneous whitespace is allowed, and is encouraged to keep the rules readable. 2.2. Basic Rule Processing The rules are processed from top to bottom, each one appended after another. This quite simply means that if the entirety of your config file is: block in all pass in all The computer sees it as: block in all pass in all Which is to say that when a packet comes in, the first thing IPF applies is: block in all Should IPF deem it necessary to move on to the next rule, it would then apply the second rule: pass in all At this point, you might want to ask yourself "would IPF move on to the second rule?" If you're familiar with ipfwadm or ipfw, you probably won't ask yourself this. Shortly after, you will become bewildered at the weird way packets are always getting denied or passed when they shouldn't. Many packet filters stop comparing packets to rulesets the moment the first match is made; IPF is not one of them. Unlike the other packet filters, IPF keeps a flag on whether or not it's going to pass the packet. Unless you interrupt the flow, IPF will go through the entire ruleset, making its decision on whether or not to pass or drop the packet based on the last matching rule. The scene: IP Fil- ter's on duty. It's been been scheduled a slice of CPU time. It has a checkpoint clipboard that reads: block in all pass in all -4- A packet comes in the interface and it's time to go to work. It takes a look at the packet, it takes a look at the first rule: block in all "So far I think I will block this packet" says IPF. It takes a look at the second rule: pass in all "So far I think I will pass this packet" says IPF. It takes a look at a third rule. There is no third rule, so it goes with what its last motivation was, to pass the packet onward. It's a good time to point out that even if the ruleset had been block in all block in all block in all block in all pass in all that the packet would still have gone through. There is no cumulative effect. The last matching rule always takes precedence. 2.3. Controlling Rule Processing If you have experience with other packet filters, you may find this layout to be confusing, and you may be specu- lating that there are problems with portability with other filters and speed of rule matching. Imagine if you had 100 rules and most of the applicable ones were the first 10. There would be a terrible overhead for every packet coming in to go through 100 rules every time. Fortunately, there is a simple keyword you can add to any rule that makes it take action at that match. That keyword is quick. Here's a modified copy of the original ruleset using the quick keyword: block in quick all pass in all In this case, IPF looks at the first rule: block in quick all The packet matches and the search is over. The packet is expunged without a peep. There are no notices, no logs, no memorial service. Cake will not be served. So what about -5- the next rule? pass in all This rule is never encountered. It could just as eas- ily not be in the config file at all. The sweeping match of all and the terminal keyword quick from the previous rule make certain that no rules are followed afterward. Having half a config file laid to waste is rarely a desirable state. On the other hand, IPF is here to block packets and as configured, it's doing a very good job. Nonetheless, IPF is also here to let some packets through, so a change to the ruleset to make this possible is called for. 2.4. Basic filtering by IP address IPF will match packets on many criteria. The one that we most commonly think of is the IP address. There are some blocks of address space from which we should never get traf- fic. One such block is from the unroutable networks, 192.168.0.0/16 (/16 is the CIDR notation for a netmask. You may be more familiar with the dotted decimal format, 255.255.0.0. IPF accepts both). If you wanted to block 192.168.0.0/16, this is one way to do it: block in quick from 192.168.0.0/16 to any pass in all Now we have a less stringent ruleset that actually does something for us. Let's imagine a packet comes in from 1.2.3.4. The first rule is applied: block in quick from 192.168.0.0/16 to any The packet is from 1.2.3.4, not 192.168.*.*, so there is no match. The second rule is applied: pass in all The packet from 1.2.3.4 is definitely a part of all, so the packet is sent to whatever it's destination happened to be. On the other hand, suppose we have a packet that comes in from 192.168.1.2. The first rule is applied: block in quick from 192.168.0.0/16 to any There's a match, the packet is dropped, and that's the end. Again, it doesn't move to the second rule because the first rule matches and contains the quick keyword. -6- At this point you can build a fairly extensive set of definitive addresses which are passed or blocked. Since we've already started blocking private address space from entering our firewall, let's take care of the rest of it: block in quick from 192.168.0.0/16 to any block in quick from 172.16.0.0/12 to any block in quick from 10.0.0.0/8 to any pass in all The first three address blocks are some of the private IP space. 2.5. Controlling Your Interfaces It seems very frequent that companies have internal networks before they want a link to the outside world. In fact, it's probably reasonable to say that's the main reason people consider firewalls in the first place. The machine that bridges the outside world to the inside world and vice versa is the router. What separates the router from any other machine is simple: It has more than one interface. Every packet you receive comes from a network inter- face; every packet you transmit goes out a network inter- face. Say your machine has 3 interfaces, lo0 (loopback), xl0 (3com ethernet), and tun0 (FreeBSD's generic tunnel interface that PPP uses), but you don't want packets coming in on the tun0 interface? block in quick on tun0 all pass in all In this case, the on keyword means that that data is coming in on the named interface. If a packet comes in on tun0, the first rule will block it. If a packet comes in on lo0 or in on xl0, the first rule will not match, the second rule will, the packet will be passed. 2.6. Using IP Address and Interface Together It's an odd state of affairs when one decides it best to have the tun0 interface up, but not allow any data to be received from it. The more criteria the firewall matches against, the tighter (or looser) the firewall can become. Maybe you want data from tun0, but not from 192.168.0.0/16? This is the start of a powerful firewall. block in quick on tun0 from 192.168.0.0/16 to any ----------- See rfc1918 at and -7- pass in all Compare this to our previous rule: block in quick from 192.168.0.0/16 to any pass in all The old way, all traffic from 192.168.0.0/16, regardless of interface, was completely blocked. The new way, using on tun0 means that it's only blocked if it comes in on the tun0 interface. If a packet arrived on the xl0 interface from 192.168.0.0/16, it would be passed. At this point you can build a fairly extensive set of definitive addresses which are passed or blocked. Since we've already started blocking private address space from entering tun0, let's take care of the rest of it: block in quick on tun0 from 192.168.0.0/16 to any block in quick on tun0 from 172.16.0.0/12 to any block in quick on tun0 from 10.0.0.0/8 to any block in quick on tun0 from 127.0.0.0/8 to any block in quick on tun0 from 0.0.0.0/8 to any block in quick on tun0 from 169.254.0.0/16 to any block in quick on tun0 from 192.0.2.0/24 to any block in quick on tun0 from 204.152.64.0/23 to any block in quick on tun0 from 224.0.0.0/3 to any pass in all You've already seen the first three blocks, but not the rest. The fourth is a largely wasted class-A network used for loopback. Much software communicates with itself on 127.0.0.1 so blocking it from an external source is a good idea. The fifth, 0.0.0.0/8, should never be seen on the internet. Most IP stacks treat "0.0.0.0/32" as the default gateway, and the rest of the 0.*.*.* network gets handled strangely by various systems as a byproduct of how routing decisions are made. You should treat 0.0.0.0/8 just like 127.0.0.0/8. 169.254.0.0/16 has been assigned by the IANA for use in auto-configuration when systems have not yet been able to obtain an IP address via DHCP or the like. Most notably, Microsoft Windows will use addresses in this range if they are set to DHCP and cannot find a DHCP server. 192.0.2.0/24 has also been reserved for use as an example IP netblock for documentation authors. We specifically do not use this range as it would cause confusion when we tell you to block it, and thus all our examples come from 20.20.20.0/24. 204.152.64.0/23 is an odd netblock reserved by Sun Microsystems for private cluster interconnects, and blocking this is up to your own judgement. Lastly, 224.0.0.0/3 wipes out the "Class D and E" networks which is used mostly for multicast traffic, although further defini- tion of "Class E" space can be found in RFC 1166. -8- There's a very important principle in packet filtering which has only been alluded to with the private network blocking and that is this: When you know there's certain types of data that only comes from certain places, you setup the system to only allow that kind of data from those places. In the case of the unroutable addresses, you know that nothing from 10.0.0.0/8 should be arriving on tun0 because you have no way to reply to it. It's an illegiti- mate packet. The same goes for the other unroutables as well as 127.0.0.0/8. Many pieces of software do all their authentication based upon the packet's originating IP address. When you have an internal network, say 20.20.20.0/24, you know that the only traffic for that internal network is going to come off the local ethernet. Should a packet from 20.20.20.0/24 arrive over a PPP dialup, it's perfectly reasonable to drop it on the floor, or put it in a dark room for interrogation. It should by no means be allowed to get to its final desti- nation. You can accomplish this particularly easily with what you already know of IPF. The new ruleset would be: block in quick on tun0 from 192.168.0.0/16 to any block in quick on tun0 from 172.16.0.0/12 to any block in quick on tun0 from 10.0.0.0/8 to any block in quick on tun0 from 127.0.0.0/8 to any block in quick on tun0 from 0.0.0.0/8 to any block in quick on tun0 from 169.254.0.0/16 to any block in quick on tun0 from 192.0.2.0/24 to any block in quick on tun0 from 204.152.64.0/23 to any block in quick on tun0 from 224.0.0.0/3 to any block in quick on tun0 from 20.20.20.0/24 to any pass in all 2.7. Bi-Directional Filtering; The "out" Keyword Up until now, we've been passing or blocking inbound traffic. To clarify, inbound traffic is all traffic that enters the firewall on any interface. Conversely, outbound traffic is all traffic that leaves on any interface (whether locally generated or simply passing through). This means that all packets coming in are not only filtered as they enter the firewall, they're also filtered as they exit. Thusfar there's been an implied pass out all that may or may not be desirable. Just as you may pass and block incoming traffic, you may do the same with outgoing traffic. Now that we know there's a way to filter outbound pack- ets just like inbound, it's up to us to find a conceivable use for such a thing. One possible use of this idea is to keep spoofed packets from exiting your own network. Instead of passing any traffic out the router, you could instead limit permitted traffic to packets originating at -9- 20.20.20.0/24. You might do it like this: pass out quick on tun0 from 20.20.20.0/24 to any block out quick on tun0 from any to any If a packet comes from 20.20.20.1/32, it gets sent out by the first rule. If a packet comes from 1.2.3.4/32 it gets blocked by the second. You can also make similar rules for the unroutable addresses. If some machine tries to route a packet through IPF with a destination in 192.168.0.0/16, why not drop it? The worst that can happen is that you'll spare yourself some bandwidth: block out quick on tun0 from any to 192.168.0.0/16 block out quick on tun0 from any to 172.16.0.0/12 block out quick on tun0 from any to 10.0.0.0/8 block out quick on tun0 from any to 0.0.0.0/8 block out quick on tun0 from any to 127.0.0.0/8 block out quick on tun0 from any to 169.254.0.0/16 block out quick on tun0 from any to 192.0.2.0/24 block out quick on tun0 from any to 204.152.64.0/23 block out quick on tun0 from any to 224.0.0.0/3 block out quick on tun0 from !20.20.20.0/24 to any In the narrowest viewpoint, this doesn't enhance your secu- rity. It enhances everybody else's security, and that's a nice thing to do. As another viewpoint, one might suppose that because nobody can send spoofed packets from your site, that your site has less value as a relay for crackers, and as such is less of a target. You'll likely find a number of uses for blocking out- bound packets. One thing to always keep in mind is that in and out directions are in reference to your firewall, never any other machine. 2.8. Logging What Happens; The "log" Keyword Up to this point, all blocked and passed packets have been silently blocked and silently passed. Usually you want to know if you're being attacked rather than wonder if that firewall is really buying you any added benefits. While I wouldn't want to log every passed packet, and in some cases every blocked packet, I would want to know about the blocked packets from 20.20.20.0/24. To do this, we add the log key- word: block in quick on tun0 from 192.168.0.0/16 to any ----------- This can, of course, be changed by using -DIPFIL- TER_DEFAULT_BLOCK when compiling ipfilter on your system. -10- block in quick on tun0 from 172.16.0.0/12 to any block in quick on tun0 from 10.0.0.0/8 to any block in quick on tun0 from 127.0.0.0/8 to any block in quick on tun0 from 0.0.0.0/8 to any block in quick on tun0 from 169.254.0.0/16 to any block in quick on tun0 from 192.0.2.0/24 to any block in quick on tun0 from 204.152.64.0/23 to any block in quick on tun0 from 224.0.0.0/3 to any block in log quick on tun0 from 20.20.20.0/24 to any pass in all So far, our firewall is pretty good at blocking packets com- ing to it from suspect places, but there's still more to be done. For one thing, we're accepting packets destined any- where. One thing we ought to do is make sure packets to 20.20.20.0/32 and 20.20.20.255/32 get dropped on the floor. To do otherwise opens the internal network for a smurf attack. These two lines would prevent our hypothetical net- work from being used as a smurf relay: block in log quick on tun0 from any to 20.20.20.0/32 block in log quick on tun0 from any to 20.20.20.255/32 This brings our total ruleset to look something like this: block in quick on tun0 from 192.168.0.0/16 to any block in quick on tun0 from 172.16.0.0/12 to any block in quick on tun0 from 10.0.0.0/8 to any block in quick on tun0 from 127.0.0.0/8 to any block in quick on tun0 from 0.0.0.0/8 to any block in quick on tun0 from 169.254.0.0/16 to any block in quick on tun0 from 192.0.2.0/24 to any block in quick on tun0 from 204.152.64.0/23 to any block in quick on tun0 from 224.0.0.0/3 to any block in log quick on tun0 from 20.20.20.0/24 to any block in log quick on tun0 from any to 20.20.20.0/32 block in log quick on tun0 from any to 20.20.20.255/32 pass in all 2.9. Complete Bi-Directional Filtering By Interface So far we have only presented fragments of a complete ruleset. When you're actually creating your ruleset, you should setup rules for every direction and every interface. The default state of ipfilter is to pass packets. It is as though there were an invisible rule at the beginning which states pass in all and pass out all. Rather than rely on some default behaviour, make everything as specific as pos- sible, interface by interface, until every base is covered. First we'll start with the lo0 interface, which wants to run wild and free. Since these are programs talking to others on the local system, go ahead and keep it unre- stricted: -11- pass out quick on lo0 pass in quick on lo0 Next, there's the xl0 interface. Later on we'll begin plac- ing restrictions on the xl0 interface, but to start with, we'll act as though everything on our local network is trustworthy and give it much the same treatment as lo0: pass out quick on xl0 pass in quick on xl0 Finally, there's the tun0 interface, which we've been half- filtering with up until now: block out quick on tun0 from any to 192.168.0.0/16 block out quick on tun0 from any to 172.16.0.0/12 block out quick on tun0 from any to 127.0.0.0/8 block out quick on tun0 from any to 10.0.0.0/8 block out quick on tun0 from any to 0.0.0.0/8 block out quick on tun0 from any to 169.254.0.0/16 block out quick on tun0 from any to 192.0.2.0/24 block out quick on tun0 from any to 204.152.64.0/23 block out quick on tun0 from any to 224.0.0.0/3 pass out quick on tun0 from 20.20.20.0/24 to any block out quick on tun0 from any to any block in quick on tun0 from 192.168.0.0/16 to any block in quick on tun0 from 172.16.0.0/12 to any block in quick on tun0 from 10.0.0.0/8 to any block in quick on tun0 from 127.0.0.0/8 to any block in quick on tun0 from 0.0.0.0/8 to any block in quick on tun0 from 169.254.0.0/16 to any block in quick on tun0 from 192.0.2.0/24 to any block in quick on tun0 from 204.152.64.0/23 to any block in quick on tun0 from 224.0.0.0/3 to any block in log quick on tun0 from 20.20.20.0/24 to any block in log quick on tun0 from any to 20.20.20.0/32 block in log quick on tun0 from any to 20.20.20.255/32 pass in all This is a pretty significant amount of filtering already, protecting 20.20.20.0/24 from being spoofed or being used for spoofing. Future examples will continue to show one- sideness, but keep in mind that it's for brevity's sake, and when setting up your own ruleset, adding rules for every direction and every interface is necessary. 2.10. Controlling Specific Protocols; The "proto" Keyword Denial of Service attacks are as rampant as buffer overflow exploits. Many denial of service attacks rely on glitches in the OS's TCP/IP stack. Frequently, this has come in the form of ICMP packets. Why not block them -12- entirely? block in log quick on tun0 proto icmp from any to any Now any ICMP traffic coming in from tun0 will be logged and discarded. 2.11. Filtering ICMP with the "icmp-type" Keyword; Merging Rulesets Of course, dropping all ICMP isn't really an ideal sit- uation. Why not drop all ICMP? Well, because it's useful to have partially enabled. So maybe you want to keep some types of ICMP traffic and drop other kinds. If you want ping and traceroute to work, you need to let in ICMP types 0 and 11. Strictly speaking, this might not be a good idea, but if you need to weigh security against convenience, IPF lets you do it. pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 0 pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 11 Remember that ruleset order is important. Since we're doing everything quick we must have our passes before our blocks, so we really want the last three rules in this order: pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 0 pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 11 block in log quick on tun0 proto icmp from any to any Adding these 3 rules to the anti-spoofing rules is a bit tricky. One error might be to put the new ICMP rules at the beginning: pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 0 pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 11 block in log quick on tun0 proto icmp from any to any block in quick on tun0 from 192.168.0.0/16 to any block in quick on tun0 from 172.16.0.0/12 to any block in quick on tun0 from 10.0.0.0/8 to any block in quick on tun0 from 127.0.0.0/8 to any block in quick on tun0 from 0.0.0.0/8 to any block in quick on tun0 from 169.254.0.0/16 to any block in quick on tun0 from 192.0.2.0/24 to any block in quick on tun0 from 204.152.64.0/23 to any block in quick on tun0 from 224.0.0.0/3 to any block in log quick on tun0 from 20.20.20.0/24 to any block in log quick on tun0 from any to 20.20.20.0/32 block in log quick on tun0 from any to 20.20.20.255/32 pass in all The problem with this is that an ICMP type 0 packet from 192.168.0.0/16 will get passed by the first rule, and never blocked by the fourth rule. Also, since we quickly pass an -13- ICMP ECHO_REPLY (type 0) to 20.20.20.0/24, we've just opened ourselves back up to a nasty smurf attack and nullified those last two block rules. Oops. To avoid this, we place the ICMP rules after the anti-spoofing rules: block in quick on tun0 from 192.168.0.0/16 to any block in quick on tun0 from 172.16.0.0/12 to any block in quick on tun0 from 10.0.0.0/8 to any block in quick on tun0 from 127.0.0.0/8 to any block in quick on tun0 from 0.0.0.0/8 to any block in quick on tun0 from 169.254.0.0/16 to any block in quick on tun0 from 192.0.2.0/24 to any block in quick on tun0 from 204.152.64.0/23 to any block in quick on tun0 from 224.0.0.0/3 to any block in log quick on tun0 from 20.20.20.0/24 to any block in log quick on tun0 from any to 20.20.20.0/32 block in log quick on tun0 from any to 20.20.20.255/32 pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 0 pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 11 block in log quick on tun0 proto icmp from any to any pass in all Because we block spoofed traffic before the ICMP rules are processed, a spoofed packet never makes it to the ICMP rule- set. It's very important to keep such situations in mind when merging rules. 2.12. TCP and UDP Ports; The "port" Keyword Now that we've started blocking packets based on proto- col, we can start blocking packets based on specific aspects of each protocol. The most frequently used of these aspects is the port number. Services such as rsh, rlogin, and tel- net are all very convenient to have, but also hideously insecure against network sniffing and spoofing. One great compromise is to only allow the services to run internally, then block them externally. This is easy to do because rlogin, rsh, and telnet use specific TCP ports (513, 514, and 23 respectively). As such, creating rules to block them is easy: block in log quick on tun0 proto tcp from any to 20.20.20.0/24 port = 513 block in log quick on tun0 proto tcp from any to 20.20.20.0/24 port = 514 block in log quick on tun0 proto tcp from any to 20.20.20.0/24 port = 23 Make sure all 3 are before the pass in all and they'll be closed off from the outside (leaving out spoofing for brevity's sake): pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 0 pass in quick on tun0 proto icmp from any to 20.20.20.0/24 icmp-type 11 block in log quick on tun0 proto icmp from any to any block in log quick on tun0 proto tcp from any to 20.20.20.0/24 port = 513 block in log quick on tun0 proto tcp from any to 20.20.20.0/24 port = 514 -14- block in log quick on tun0 proto tcp from any to 20.20.20.0/24 port = 23 pass in all You might also want to block 514/udp (syslog), 111/tcp & 111/udp (portmap), 515/tcp (lpd), 2049/tcp and 2049/udp (NFS), 6000/tcp (X11) and so on and so forth. You can get a complete listing of the ports being listened to by using netstat -a (or lsof -i, if you have it installed). Blocking UDP instead of TCP only requires replacing proto tcp with proto udp. The rule for syslog would be: block in log quick on tun0 proto udp from any to 20.20.20.0/24 port = 514 IPF also has a shorthand way to write rules that apply to both proto tcp and proto udp at the same time, such as portmap or NFS. The rule for portmap would be: block in log quick on tun0 proto tcp/udp from any to 20.20.20.0/24 port = 111 3. Advanced Firewalling Introduction This section is designed as an immediate followup to the basic section. Contained below are both concepts for advanced firewall design, and advanced features contained only within ipfilter. Once you are comfortable with this section, you should be able to build a very strong firewall. 3.1. Rampant Paranoia; or The Default-Deny Stance There's a big problem with blocking services by the port: sometimes they move. RPC based programs are terrible about this, lockd, statd, even nfsd listens places other than 2049. It's awfully hard to predict, and even worse to automate adjusting all the time. What if you miss a ser- vice? Instead of dealing with all that hassle, let's start over with a clean slate. The current ruleset looks like this: Yes, we really are starting over. The first rule we're going to use is this: block in all No network traffic gets through. None. Not a peep. You're rather secure with this setup. Not terribly useful, but quite secure. The great thing is that it doesn't take much more to make your box rather secure, yet useful too. Let's -15- say the machine this is running on is a web server, nothing more, nothing less. It doesn't even do DNS lookups. It just wants to take connections on 80/tcp and that's it. We can do that. We can do that with a second rule, and you already know how: block in on tun0 all pass in quick on tun0 proto tcp from any to 20.20.20.1/32 port = 80 This machine will pass in port 80 traffic for 20.20.20.1, and deny everything else. For basic firewalling, this is all one needs. 3.2. Implicit Allow; The "keep state" Rule The job of your firewall is to prevent unwanted traffic getting to point B from point A. We have general rules which say "as long as this packet is to port 23, it's okay." We have general rules which say "as long as this packet has its FIN flag set, it's okay." Our firewalls don't know the beginning, middle, or end of any TCP/UDP/ICMP session. They merely have vague rules that are applied to all packets. We're left to hope that the packet with its FIN flag set isn't really a FIN scan, mapping our services. We hope that the packet to port 23 isn't an attempted hijack of our tel- net session. What if there was a way to identify and autho- rize individual TCP/UDP/ICMP sessions and distinguish them from port scanners and DoS attacks? There is a way, it's called keeping state. We want convenience and security in one. Lots of peo- ple do, that's why Ciscos have an "established" clause that lets established tcp sessions go through. Ipfw has estab- lished. Ipfwadm has setup/established. They all have this feature, but the name is very misleading. When we first saw it, we thought it meant our packet filter was keeping track of what was going on, that it knew if a connection was really established or not. The fact is, they're all taking the packet's word for it from a part of the packet anybody can lie about. They read the TCP packet's flags section and there's the reason UDP/ICMP don't work with it, they have no such thing. Anybody who can create a packet with bogus flags can get by a firewall with this setup. Where does IPF come in to play here, you ask? Well, unlike the other firewalls, IPF really can keep track of whether or not a connection is established. And it'll do it with TCP, UDP and ICMP, not just TCP. Ipf calls it keeping state. The keyword for the ruleset is keep state. Up until now, we've told you that packets come in, then the ruleset gets checked; packets go out, then the ruleset gets checked. Actually, what happens is packets come in, the state table gets checked, then *maybe* the inbound -16- ruleset gets checked; packets go out, the state table gets checked, then *maybe* the outbound ruleset gets checked. The state table is a list of TCP/UDP/ICMP sessions that are unquestionadely passed through the firewall, circumventing the entire ruleset. Sound like a serious security hole? Hang on, it's the best thing that ever happened to your firewall. All TCP/IP sessions have a start, a middle, and an end (even though they're sometimes all in the same packet). You can't have an end without a middle and you can't have a mid- dle without a start. This means that all you really need to filter on is the beginning of a TCP/UDP/ICMP session. If the beginning of the session is allowed by your firewall rules, you really want the middle and end to be allowed too (lest your IP stack should overflow and your machines become useless). Keeping state allows you to ignore the middle and end and simply focus on blocking/passing new sessions. If the new session is passed, all its subsequent packets will be allowed through. If it's blocked, none of its subsequent packets will be allowed through. Here's an example for run- ning an ssh server (and nothing but an ssh server): block out quick on tun0 all pass in quick on tun0 proto tcp from any to 20.20.20.1/32 port = 22 keep state The first thing you might notice is that there's no "pass out" provision. In fact, there's only an all-inclusive "block out" rule. Despite this, the ruleset is complete. This is because by keeping state, the entire ruleset is cir- cumvented. Once the first SYN packet hits the ssh server, state is created and the remainder of the ssh session is allowed to take place without interference from the fire- wall. Here's another example: block in quick on tun0 all pass out quick on tun0 proto tcp from 20.20.20.1/32 to any keep state In this case, the server is running no services. Infact, it's not a server, it's a client. And this client doesn't want unauthorized packets entering its IP stack at all. However, the client wants full access to the internet and -the reply packets that such privledge entails. This simple +the reply packets that such privilege entails. This simple ruleset creates state entries for every new outgoing TCP session. Again, since a state entry is created, these new TCP sessions are free to talk back and forth as they please -without the hinderance or inspection of the firewall rule- +without the hindrance or inspection of the firewall rule- set. We mentioned that this also works for UDP and ICMP: block in quick on tun0 all pass out quick on tun0 proto tcp from 20.20.20.1/32 to any keep state pass out quick on tun0 proto udp from 20.20.20.1/32 to any keep state pass out quick on tun0 proto icmp from 20.20.20.1/32 to any keep state -17- Yes Virginia, we can ping. Now we're keeping state on TCP, UDP, ICMP. Now we can make outgoing connections as though there's no firewall at all, yet would-be attackers can't get back in. This is very handy because there's no need to track down what ports we're listening to, only the ports we want people to be able to get to. State is pretty handy, but it's also a bit tricky. You can shoot yourself in the foot in strange and mysterious ways. Consider the following ruleset: pass in quick on tun0 proto tcp from any to 20.20.20.1/32 port = 23 pass out quick on tun0 proto tcp from any to any keep state block in quick all block out quick all At first glance, this seems to be a good setup. We allow incoming sessions to port 23, and outgoing sessions any- where. Naturally packets going to port 23 will have reply packets, but the ruleset is setup in such a way that the pass out rule will generate a state entry and everything will work perfectly. At least, you'd think so. The unfortunate truth is that after 60 seconds of idle time the state entry will be closed (as opposed to the nor- mal 5 days). This is because the state tracker never saw the original SYN packet destined to port 23, it only saw the SYN ACK. IPF is very good about following TCP sessions from start to finish, but it's not very good about coming into the middle of a connection, so rewrite the rule to look like this: pass in quick on tun0 proto tcp from any to 20.20.20.1/32 port = 23 keep state pass out quick on tun0 proto tcp from any to any keep state block in quick all block out quick all The additional of this rule will enter the very first packet into the state table and everything will work as expected. Once the 3-way handshake has been witness by the state engine, it is marked in 4/4 mode, which means it's setup for long-term data exchange until such time as the connection is torn down (wherein the mode changes again. You can see the current modes of your state table with ipfstat -s. 3.3. Stateful UDP UDP is stateless so naturally it's a bit harder to do a reliable job of keeping state on it. Nonetheless, ipf does a pretty good job. When machine A sends a UDP packet to machine B with source port X and destination port Y, ipf will allow a reply from machine B to machine A with source port Y and destination port X. This is a short term state entry, a mere 60 seconds. -18- Here's an example of what happens if we use nslookup to get the IP address of www.3com.com: $ nslookup www.3com.com A DNS packet is generated: 17:54:25.499852 20.20.20.1.2111 > 198.41.0.5.53: 51979+ The packet is from 20.20.20.1, port 2111, destined for 198.41.0.5, port 53. A 60 second state entry is created. If a packet comes back from 198.41.0.5 port 53 destined for 20.20.20.1 port 2111 within that period of time, the reply packet will be let through. As you can see, milliseconds later: 17:54:25.501209 198.41.0.5.53 > 20.20.20.1.2111: 51979 q: www.3com.com The reply packet matches the state criteria and is let through. At that same moment that packet is let through, the state gateway is closed and no new incoming packets will be allowed in, even if they claim to be from the same place. 3.4. Stateful ICMP IPFilter handles ICMP states in the manner that one would expect from understanding how ICMP is used with TCP and UDP, and with your understanding of how keep state works. There are two general types of ICMP messages; requests and replies. When you write a rule such as: pass out on tun0 proto icmp from any to any icmp-type 8 keep state to allow outbound echo requests (a typical ping), the resul- tant icmp-type 0 packet that comes back will be allowed in. This state entry has a default timeout of an incomplete 0/0 state of 60 seconds. Thus, if you are keeping state on any outbound icmp message that will elicit an icmp message in reply, you need a proto icmp [...] keep state rule. However, the majority of ICMP messages are status mes- sages generated by some failure in UDP (and sometimes TCP), and in 3.4.x and greater IPFilters, any ICMP error status message (say icmp-type 3 code 3 port unreachable, or icmp- type 11 time exceeded) that matches an active state table entry that could have generated that message, the ICMP packet is let in. For example, in older IPFilters, if you wanted traceroute to work, you needed to use: pass out on tun0 proto udp from any to any port 33434><33690 keep state pass in on tun0 proto icmp from any to any icmp-type timex whereas now you can do the right thing and just keep state on udp with: -19- pass out on tun0 proto udp from any to any port 33434><33690 keep state To provide some protection against a third-party sneaking ICMP messages through your firewall when an active connec- tion is known to be in your state table, the incoming ICMP packet is checked not only for matching source and destina- tion addresses (and ports, when applicable) but a tiny part of the payload of the packet that the ICMP message is claim- ing it was generated by. 3.5. FIN Scan Detection; "flags" Keyword, "keep frags" Key- word Let's go back to the 4 rule set from the previous section: pass in quick on tun0 proto tcp from any to 20.20.20.1/32 port = 23 keep state pass out quick on tun0 proto tcp from any to any keep state block in quick all block out quick all This is almost, but not quite, satisfactory. The problem is that it's not just SYN packets that're allowed to go to port 23, any old packet can get through. We can change this by using the flags option: pass in quick on tun0 proto tcp from any to 20.20.20.1/32 port = 23 flags S keep state pass out quick on tun0 proto tcp from any to any flags S keep state block in quick all block out quick all Now only TCP packets, destined for 20.20.20.1, at port 23, with a lone SYN flag will be allowed in and entered into the state table. A lone SYN flag is only present as the very first packet in a TCP session (called the TCP handshake) and that's really what we wanted all along. There's at least two advantages to this: No arbitrary packets can come in and make a mess of your state table. Also, FIN and XMAS scans will fail since they set flags other than the SYN flag. Now all incoming packets must either be handshakes or have state already. If anything else comes in, it's proba- bly a port scan or a forged packet. There's one exception to that, which is when a packet comes in that's fragmented from its journey. IPF has provisions for this as well, the ----------- Some examples use flags S/SA instead of flags S. flags S actually equates to flags S/AUPRFS and matches against only the SYN packet out of all six possible flags, while flags S/SA will allow pack- ets that may or may not have the URG, PSH, FIN, or RST flags set. Some protocols demand the URG or PSH flags, and S/SAFR would be a better choice for these, however we feel that it is less secure to blindly use S/SA when it isn't required. But it's your firewall. -20- keep frags keyword. With it, IPF will notice and keep track of packets that are fragmented, allowing the expected frag- ments to to go through. Let's rewrite the 3 rules to log forgeries and allow fragments: pass in quick on tun0 proto tcp from any to 20.20.20.1/32 port = 23 flags S keep state keep frags pass out quick on tun0 proto tcp from any to any keep state flags S keep frags block in log quick all block out log quick all This works because every packet that should be allowed through makes it into the state table before the blocking rules are reached. The only scan this won't detect is a SYN -scan itself. If you're truely worried about that, you might +scan itself. If you're truly worried about that, you might even want to log all initial SYN packets. 3.6. Responding To a Blocked Packet So far, all of our blocked packets have been dumped on the floor, logged or not, we've never sent anything back to the originating host. Sometimes this isn't the most desir- able of responses because in doing so, we actually tell the attacker that a packet filter is present. It seems a far better thing to misguide the attacker into believing that, while there's no packet filter running, there's likewise no services to break into. This is where fancier blocking comes into play. When a service isn't running on a Unix system, it nor- mally lets the remote host know with some sort of return packet. In TCP, this is done with an RST (Reset) packet. When blocking a TCP packet, IPF can actually return an RST to the origin by using the return-rst keyword. Where once we did: block in log on tun0 proto tcp from any to 20.20.20.0/24 port = 23 pass in all We might now do: block return-rst in log proto tcp from any to 20.20.20.0/24 port = 23 block in log quick on tun0 pass in all We need two block statements since return-rst only works with TCP, and we still want to block protocols such as UDP, ICMP, and others. Now that this is done, the remote side will get "connection refused" instead of "connection timed out". It's also possible to send an error message when some- body sends a packet to a UDP port on your system. Whereas once you might have used: -21- block in log quick on tun0 proto udp from any to 20.20.20.0/24 port = 111 You could instead use the return-icmp keyword to send a reply: block return-icmp(port-unr) in log quick on tun0 proto udp from any to 20.20.20.0/24 port = 111 According to TCP/IP Illustrated, port-unreachable is the correct ICMP type to return when no service is listening on the port in question. You can use any ICMP type you like, but port-unreachable is probably your best bet. It's also the default ICMP type for return-icmp. However, when using return-icmp, you'll notice that it's not very stealthy, and it returns the ICMP packet with the IP address of the firewall, not the original destination of the packet. This was fixed in ipfilter 3.3, and a new keyword; return-icmp-as-dest, has been added. The new for- mat is: block return-icmp-as-dest(port-unr) in log on tun0 proto udp from any to 20.20.20.0/24 port = 111 3.7. Fancy Logging Techniques It is important to note that the presence of the log keyword only ensures that the packet will be available to the ipfilter logging device; /dev/ipl. In order to actu- ally see this log information, one must be running the ipmon utility (or some other utility that reads from /dev/ipl). The typical usage of log is coupled with ipmon -s to log the information to syslog. As of ipfilter 3.3, one can now even control the logging behavior of syslog by using log level keywords, as in rules such as this: block in log level auth.info quick on tun0 from 20.20.20.0/24 to any block in log level auth.alert quick on tun0 proto tcp from any to 20.20.20.0/24 port = 21 In addition to this, you can tailor what information is being logged. For example, you may not be interested that someone attempted to probe your telnet port 500 times, but you are interested that they probed you once. You can use the log first keyword to only log the first example of a packet. Of course, the notion of "first-ness" only applies to packets in a specific session, and for the typical blocked packet, you will be hard pressed to encounter situa- tions where this does what you expect. However, if used in conjunction with pass and keep state, this can be a valuable keyword for keeping tabs on traffic. Another useful thing you can do with the logs is to keep track of interesting pieces of the packet in addition to the header information normally being logged. Ipfilter will give you the first 128 bytes of the packet if you use the log body keyword. You should limit the use of body -22- logging, as it makes your logs very verbose, but for certain applications, it is often handy to be able to go back and take a look at the packet, or to send this data to another application that can examine it further. 3.8. Putting It All Together So now we have a pretty tight firewall, but it can still be tighter. Some of the original ruleset we wiped clean is actually very useful. I'd suggest bringing back all the anti-spoofing stuff. This leaves us with: block in on tun0 block in quick on tun0 from 192.168.0.0/16 to any block in quick on tun0 from 172.16.0.0/12 to any block in quick on tun0 from 10.0.0.0/8 to any block in quick on tun0 from 127.0.0.0/8 to any block in quick on tun0 from 0.0.0.0/8 to any block in quick on tun0 from 169.254.0.0/16 to any block in quick on tun0 from 192.0.2.0/24 to any block in quick on tun0 from 204.152.64.0/23 to any block in quick on tun0 from 224.0.0.0/3 to any block in log quick on tun0 from 20.20.20.0/24 to any block in log quick on tun0 from any to 20.20.20.0/32 block in log quick on tun0 from any to 20.20.20.255/32 pass out quick on tun0 proto tcp/udp from 20.20.20.1/32 to any keep state pass out quick on tun0 proto icmp from 20.20.20.1/32 to any keep state pass in quick on tun0 proto tcp from any to 20.20.20.1/32 port = 80 flags S keep state 3.9. Improving Performance With Rule Groups Let's extend our use of our firewall by creating a much more complicated, and we hope more applicable to the real world, example configuration For this example, we're going to change the interface names, and network numbers. Let's assume that we have three interfaces in our firewall with interfaces xl0, xl1, and xl2. xl0 is connected to our external network 20.20.20.0/26 xl1 is connected to our "DMZ" network 20.20.20.64/26 xl2 is connected to our protected network 20.20.20.128/25 We'll define the entire ruleset in one swoop, since we fig- ure that you can read these rules by now: block in quick on xl0 from 192.168.0.0/16 to any block in quick on xl0 from 172.16.0.0/12 to any block in quick on xl0 from 10.0.0.0/8 to any block in quick on xl0 from 127.0.0.0/8 to any block in quick on xl0 from 0.0.0.0/8 to any block in quick on xl0 from 169.254.0.0/16 to any block in quick on xl0 from 192.0.2.0/24 to any block in quick on xl0 from 204.152.64.0/23 to any block in quick on xl0 from 224.0.0.0/3 to any -23- block in log quick on xl0 from 20.20.20.0/24 to any block in log quick on xl0 from any to 20.20.20.0/32 block in log quick on xl0 from any to 20.20.20.63/32 block in log quick on xl0 from any to 20.20.20.64/32 block in log quick on xl0 from any to 20.20.20.127/32 block in log quick on xl0 from any to 20.20.20.128/32 block in log quick on xl0 from any to 20.20.20.255/32 pass out on xl0 all pass out quick on xl1 proto tcp from any to 20.20.20.64/26 port = 80 flags S keep state pass out quick on xl1 proto tcp from any to 20.20.20.64/26 port = 21 flags S keep state pass out quick on xl1 proto tcp from any to 20.20.20.64/26 port = 20 flags S keep state pass out quick on xl1 proto tcp from any to 20.20.20.65/32 port = 53 flags S keep state pass out quick on xl1 proto udp from any to 20.20.20.65/32 port = 53 keep state pass out quick on xl1 proto tcp from any to 20.20.20.66/32 port = 53 flags S keep state pass out quick on xl1 proto udp from any to 20.20.20.66/32 port = 53 keep state block out on xl1 all pass in quick on xl1 proto tcp/udp from 20.20.20.64/26 to any keep state block out on xl2 all pass in quick on xl2 proto tcp/udp from 20.20.20.128/25 to any keep state From this arbitarary example, we can already see that our ruleset is becoming unwieldy. To make matters worse, as we add more specific rules to our DMZ network, we add addi- tional tests that must be parsed for every packet, which affects the performance of the xl0 <-> xl2 connections. If you set up a firewall with a ruleset like this, and you have lots of bandwidth and a moderate amount of cpu, everyone that has a workstation on the xl2 network is going to come looking for your head to place on a platter. So, to keep your head <-> torso network intact, you can speed things along by creating rule groups. Rule groups allow you to write your ruleset in a tree fashion, instead of as a linear list, so that if your packet has nothing to do with the set of tests (say, all those xl1 rules) those rules will never be consulted. It's somewhat like having multiple firewalls all on the same machine. Here's a simple example to get us started: block out quick on xl1 all head 10 pass out quick proto tcp from any to 20.20.20.64/26 port = 80 flags S keep state group 10 block out on xl2 all In this simplistic example, we can see a small hint of the power of the rule group. If the packet is not destined for xl1, the head of rule group 10 will not match, and we will go on with our tests. If the packet does match for xl1, the quick keyword will short-circuit all further processing at the root level (rule group 0), and focus the testing on rules which belong to group 10; namely, the SYN check for 80/tcp. In this way, we can re-write the above rules so that we can maximize performance of our firewall. -24- block in quick on xl0 all head 1 block in quick on xl0 from 192.168.0.0/16 to any group 1 block in quick on xl0 from 172.16.0.0/12 to any group 1 block in quick on xl0 from 10.0.0.0/8 to any group 1 block in quick on xl0 from 127.0.0.0/8 to any group 1 block in quick on xl0 from 0.0.0.0/8 to any group 1 block in quick on xl0 from 169.254.0.0/16 to any group 1 block in quick on xl0 from 192.0.2.0/24 to any group 1 block in quick on xl0 from 204.152.64.0/23 to any group 1 block in quick on xl0 from 224.0.0.0/3 to any group 1 block in log quick on xl0 from 20.20.20.0/24 to any group 1 block in log quick on xl0 from any to 20.20.20.0/32 group 1 block in log quick on xl0 from any to 20.20.20.63/32 group 1 block in log quick on xl0 from any to 20.20.20.64/32 group 1 block in log quick on xl0 from any to 20.20.20.127/32 group 1 block in log quick on xl0 from any to 20.20.20.128/32 group 1 block in log quick on xl0 from any to 20.20.20.255/32 group 1 pass in on xl0 all group 1 pass out on xl0 all block out quick on xl1 all head 10 pass out quick on xl1 proto tcp from any to 20.20.20.64/26 port = 80 flags S keep state group 10 pass out quick on xl1 proto tcp from any to 20.20.20.64/26 port = 21 flags S keep state group 10 pass out quick on xl1 proto tcp from any to 20.20.20.64/26 port = 20 flags S keep state group 10 pass out quick on xl1 proto tcp from any to 20.20.20.65/32 port = 53 flags S keep state group 10 pass out quick on xl1 proto udp from any to 20.20.20.65/32 port = 53 keep state group 10 pass out quick on xl1 proto tcp from any to 20.20.20.66/32 port = 53 flags S keep state pass out quick on xl1 proto udp from any to 20.20.20.66/32 port = 53 keep state group 10 pass in quick on xl1 proto tcp/udp from 20.20.20.64/26 to any keep state block out on xl2 all pass in quick on xl2 proto tcp/udp from 20.20.20.128/25 to any keep state Now you can see the rule groups in action. For a host on the xl2 network, we can completely bypass all the checks in group 10 when we're not communicating with hosts on that network. Depending on your situation, it may be prudent to group your rules by protocol, or various machines, or netblocks, or whatever makes it flow smoothly. 3.10. "Fastroute"; The Keyword of Stealthiness Even though we're forwarding some packets, and blocking other packets, we're typically behaving like a well behaved router should by decrementing the TTL on the packet and acknowledging to the entire world that yes, there is a hop here. But we can hide our presence from inquisitive appli- cations like unix traceroute which uses UDP packets with various TTL values to map the hops between two sites. If we -25- want incoming traceroutes to work, but we do not want to announce the presence of our firewall as a hop, we can do so with a rule like this: block in quick on xl0 fastroute proto udp from any to any port 33434 >< 33465 The presence of the fastroute keyword will signal ipfilter to not pass the packet into the Unix IP stack for routing which results in a TTL decrement. The packet will be placed gently on the output interface by ipfilter itself and no such decrement will happen. Ipfilter will of course use the system's routing table to figure out what the appropriate output interface really is, but it will take care of the actual task of routing itself. There's a reason we used block quick in our example, too. If we had used pass, and if we had IP Forwarding enabled in our kernel, we would end up having two paths for a packet to come out of, and we would probably panic our kernel. It should be noted, however, that most Unix kernels (and certainly the ones underlying the systems that ipfilter usually runs on) have far more efficient routing code than what exists in ipfilter, and this keyword should not be thought of as a way to improve the operating speed of your firewall, and should only be used in places where stealth is an issue. 4. NAT and Proxies Outside of the corporate environment, one of the biggest enticements of firewall technology to the end user is the ability to connect several computers through a common external interface, often without the approval, knowledge, or even consent of their service provider. To those famil- iar with Linux, this concept is called IP Masquerading, but to the rest of the world it is known by the more obscure name of Network Address Translation, or NAT for short. 4.1. Mapping Many Addresses Into One Address The basic use of NAT accomplishes much the same thing that Linux's IP Masquerading function does, and it does it ----------- To be pedantic, what IPFilter provides is really called NPAT, for Network and Port Address Transla- tion, which means we can change any of the source and destination IP Addresses and their source and destination ports. True NAT only allows one to change the addresses. -26- with one simple rule: map tun0 192.168.1.0/24 -> 20.20.20.1/32 Very simple. Whenever a packet goes out the tun0 interface with a source address matching the CIDR network mask of 192.168.1.0/24 this packet will be rewritten within the IP stack such that its source address is 20.20.20.1, and it will be sent on to its original destination. The system also keeps a list of what translated connections are in progress so that it can perform the reverse and remap the response (which will be directed to 20.20.20.1) to the internal host that really generated the packet. There is a drawback to the rule we have just written, though. In a large number of cases, we do not happen to know what the IP address of our outside link is (if we're using tun0 or ppp0 and a typical ISP) so it makes setting up our NAT tables a chore. Luckily, NAT is smart enough to accept an address of 0/32 as a signal that it needs to go look at what the address of that interface really is and we can rewrite our rule as follows: map tun0 192.168.1.0/24 -> 0/32 Now we can load our ipnat rules with impunity and connect to the outside world without having to edit anything. You do have to run ipf -y to refresh the address if you get discon- nected and redial or if your DHCP lease changes, though. Some of you may be wondering what happens to the source port when the mapping happens. With our current rule, the packet's source port is unchanged from the original source port. There can be instances where we do not desire this behavior; maybe we have another firewall further upstream we have to pass through, or perhaps many hosts are trying to use the same source port, causing a collision where the rule doesn't match and the packet is passed untranslated. ipnat helps us here with the portmap keyword: map tun0 192.168.1.0/24 -> 0/32 portmap tcp/udp 20000:30000 Our rule now shoehorns all the translated connections (which can be tcp, udp, or tcp/udp) into the port range of 20000 to 30000. ----------- This is a typical internal address space, since it's non-routable on the Real Internet it is often used for internal networks. You should still block these packets coming in from the outside world as discussed earlier. -27- 4.2. Mapping Many Addresses Into a Pool of Addresses Another use common use of NAT is to take a small stati- cally allocated block of addresses and map many computers into this smaller address space. This is easy to accom- plish using what you already know about the map and portmap keywords by writing a rule like so: map tun0 192.168.0.0/16 -> 20.20.20.0/24 portmap tcp/udp 20000:60000 Also, there may be instances where a remote application requires that multiple connections all come from the same IP address. We can help with these situations by telling NAT to statically map sessions from a host into the pool of addresses and work some magic to choose a port. This uses a the keyword map-block as follows: map-block tun0 192.168.1.0/24 -> 20.20.20.0/24 4.3. One to One Mappings Occasionally it is desirable to have a system with one IP address behind the firewall to appear to have a com- pletely different IP address. One example of how this would work would be a lab of computers which are then attached to various networks that are to be put under some kind of test. In this example, you would not want to have to reconfigure the entire lab when you could place a NAT system in front and change the addresses in one simple place. We can do that with the bimap keyword, for bidirectional mapping. Bimap has some additional protections on it to ensure a known state for the connection, whereas the map keyword is designed to allocate an address and a source port and rewrite the packet and go on with life. bimap tun0 192.168.1.1/32 -> 20.20.20.1/32 will accomplish the mapping for one host. 4.4. Spoofing Services Spoofing services? What does that have to do with any- thing? Plenty. Let's pretend that we have a web server running on 20.20.20.5, and since we've gotten increasingly suspicious of our network security, we desire to not run this server on port 80 since that requires a brief lifespan as the root user. But how do we run it on a less privledged port of 8000 in this world of "anything dot com"? How will anyone find our server? We can use the redirection facilities of NAT to solve this problem by instructing it to remap any connections destined for 20.20.20.5:80 to really point to 20.20.20.5:8000. This uses the rdr keyword: rdr tun0 20.20.20.5/32 port 80 -> 192.168.0.5 port 8000 -28- We can also specify the protocol here, if we wanted to redi- rect a UDP service, instead of a TCP service (which is the default). For example, if we had a honeypot on our firewall to impersonate the popular Back Orifice for Windows, we could shovel our entire network into this one place with a simple rule: rdr tun0 20.20.20.0/24 port 31337 -> 127.0.0.1 port 31337 udp An extremely important point must be made about rdr: You cannot easily use this feature as a "reflector". E.g: rdr tun0 20.20.20.5/32 port 80 -> 20.20.20.6 port 80 tcp will not work in the situation where .5 and .6 are on the same LAN segment. The rdr function is applied to packets that enter the firewall on the specified interface. When a packet comes in that matches a rdr rule, its destination address is then rewritten, it is pushed into ipf for filter- ing, and should it successfully run the gauntlet of filter rules, it is then sent to the unix routing code. Since this packet is still inbound on the same interface that it will need to leave the system on to reach a host, the system gets confused. Reflectors don't work. Neither does specifying the address of the interface the packet just came in on. Always remember that rdr destinations must exit out of the firewall host on a different interface. 4.5. Transparent Proxy Support; Redirection Made Useful Since you're installing a firewall, you may have decided that it is prudent to use a proxy for many of your outgoing connections so that you can further tighten your filter rules protecting your internal network, or you may have run into a situation that the NAT mapping process does not currently handle properly. This can also be accom- plished with a redirection statement: rdr xl0 0.0.0.0/0 port 21 -> 127.0.0.1 port 21 This statement says that any packet coming in on the xl0 interface destined for any address (0.0.0.0/0) on the ftp port should be rewritten to connect it with a proxy that is running on the NAT system on port 21. ----------- Yes. There is a way to do this. It's so convo- luted that I refuse to use it, though. Smart peo- ple who require this functionality will transpar- ently redirect into something like TIS plug-gw on 127.0.0.1. Stupid people will set up a dummy loop interface pair and double rewrite. This includes 127.0.0.1, by the way. That's on lo0. Neat, huh? -29- This specific example of FTP proxying does lead to some complications when used with web browsers or other auto- matic-login type clients that are unaware of the require- ments of communicating with the proxy. There are patches for TIS Firewall Toolkit'sftp-gw to mate it with the nat process so that it can determine where you were trying to go and automatically send you there. Many proxy packages now work in a transparent proxy environment (Squid for example, located at http://squid.nlanr.net, works fine.) This application of the rdr keyword is often more use- ful when you wish to force users to authenticate themselves with the proxy. (For example, you desire your engineers to be able to surf the web, but you would rather not have your call-center staff doing so.) 4.6. Magic Hidden Within NAT; Application Proxies Since ipnat provides a method to rewrite packets as they traverse the firewall, it becomes a convenient place to build in some application level proxies to make up for well known deficiencies of that application and typical fire- walls. For example; FTP. We can make our firewall pay attention to the packets going across it and when it notices that it's dealing with an Active FTP session, it can write itself some temporary rules, much like what happens with keep state, so that the FTP data connection works. To do this, we use a rule like so: map tun0 192.168.1.0/24 -> 20.20.20.1/32 proxy port ftp ftp/tcp You must always remember to place this proxy rule before any portmap rules, otherwise when portmap comes along and matches the packet and rewrites it before the proxy gets a chance to work on it. Remember that ipnat rules are first- match. There also exist proxies for "rcmd" (which we suspect is berkeley r-* commands which should be forbidden anyway, thus we haven't looked at what this proxy does) and "raudio" for Real Audio PNM streams. Likewise, both of these rules should be put before any portmap rules, if you're doing NAT. 5. Loading and Manipulating Filter Rules; The ipf Utility IP Filter rules are loaded by using the ipf utility. The filter rules can be stored in any file on the system, but typically these rules are stored in /etc/ipf.rules, /usr/local/etc/ipf.rules, or /etc/opt/ipf/ipf.rules. IP Filter has two sets of rules, the active set and the inactive set. By default, all operations are performed on -30- the active set. You can manipulate the inactive set by adding -I to the ipf command line. The two sets can be toggled by using the -s command line option. This is very useful for testing new rule sets without wiping out the old rule set. Rules can also be removed from the list instead of added by using the -r command line option, but it is gener- ally a safer idea to flush the rule set that you're working on with -F and completely reload it when making changes. In summary, the easiest way to load a rule set is ipf -Fa -f /etc/ipf.rules. For more complicated manipulations of the rule set, please see the ipf(1) man page. 6. Loading and Manipulating NAT Rules; The ipnat Utility NAT rules are loaded by using the ipnat utility. The NAT rules can be stored in any file on the system, but typi- cally these rules are stored in /etc/ipnat.rules, /usr/local/etc/ipnat.rules, or /etc/opt/ipf/ipnat.rules. Rules can also be removed from the list instead of added by using the -r command line option, but it is gener- ally a safer idea to flush the rule set that you're working on with -C and completely reload it when making changes. Any active mappings are not affected by -C, and can be removed with -F. NAT rules and active mappings can be examined with the -l command line option. In summary, the easiest way to load a NAT rule set is ipnat -CF -f /etc/ipnat.rules. 7. Monitoring and Debugging There will come a time when you are interested in what your firewall is actually doing, and ipfilter would be incomplete if it didn't have a full suite of status monitor- ing tools. 7.1. The ipfstat utility In its simplest form, ipfstat displays a table of interesting data about how your firewall is performing, such as how many packets have been passed or blocked, if they were logged or not, how many state entries have been made, and so on. Here's an example of something you might see from running the tool: # ipfstat input packets: blocked 99286 passed 1255609 nomatch 14686 counted 0 output packets: blocked 4200 passed 1284345 nomatch 14687 counted 0 -31- input packets logged: blocked 99286 passed 0 output packets logged: blocked 0 passed 0 packets logged: input 0 output 0 log failures: input 3898 output 0 fragment state(in): kept 0 lost 0 fragment state(out): kept 0 lost 0 packet state(in): kept 169364 lost 0 packet state(out): kept 431395 lost 0 ICMP replies: 0 TCP RSTs sent: 0 Result cache hits(in): 1215208 (out): 1098963 IN Pullups succeeded: 2 failed: 0 OUT Pullups succeeded: 0 failed: 0 Fastroute successes: 0 failures: 0 TCP cksum fails(in): 0 (out): 0 Packet log flags set: (0) none ipfstat is also capable of showing you your current rule list. Using the -i or the -o flag will show the currently loaded rules for in or out, respectively. Adding a -h to this will provide more useful information at the same time by showing you a "hit count" on each rule. For example: # ipfstat -ho 2451423 pass out on xl0 from any to any 354727 block out on ppp0 from any to any 430918 pass out quick on ppp0 proto tcp/udp from 20.20.20.0/24 to any keep state keep frags From this, we can see that perhaps there's something abnor- mal going on, since we've got a lot of blocked packets out- bound, even with a very permissive pass out rule. Something here may warrant further investigation, or it may be func- tioning perfectly by design. ipfstat can't tell you if your rules are right or wrong, it can only tell you what is hap- pening because of your rules. To further debug your rules, you may want to use the -n flag, which will show the rule number next to each rule. # ipfstat -on @1 pass out on xl0 from any to any @2 block out on ppp0 from any to any @3 pass out quick on ppp0 proto tcp/udp from 20.20.20.0/24 to any keep state keep frags The final piece of really interesting information that ipfs- tat can provide us is a dump of the state table. This is done with the -s flag: # ipfstat -s 281458 TCP 319349 UDP 0 ICMP 19780145 hits 5723648 misses -32- 0 maximum 0 no memory 1 active 319349 expired 281419 closed 100.100.100.1 -> 20.20.20.1 ttl 864000 pass 20490 pr 6 state 4/4 pkts 196 bytes 17394 987 -> 22 585538471:2213225493 16592:16500 pass in log quick keep state pkt_flags & b = 2, pkt_options & ffffffff = 0 pkt_security & ffff = 0, pkt_auth & ffff = 0 Here we see that we have one state entry for a TCP connec- tion. The output will vary slightly from version to ver- sion, but the basic information is the same. We can see in this connection that we have a fully established connection (represented by the 4/4 state. Other states are incomplete and will be documented fully later.) We can see that the state entry has a time to live of 240 hours, which is an absurdly long time, but is the default for an established TCP connection. This TTL counter is decremented every sec- ond that the state entry is not used, and will finally result in the connection being purged if it has been left idle. The TTL is also reset to 864000 whenever the state IS used, ensuring that the entry will not time out while it is being actively used. We can also see that we have passed 196 packets consisting of about 17kB worth of data over this connection. We can see the ports for both endpoints, in this case 987 and 22; which means that this state entry rep- resents a connection from 100.100.100.1 port 987 to 20.20.20.1 port 22. The really big numbers in the second line are the TCP sequence numbers for this connection, which helps to ensure that someone isn't easily able to inject a forged packet into your session. The TCP window is also shown. The third line is a synopsis of the implicit rule that was generated by the keep state code, showing that this connection is an inbound connection. 7.2. The ipmon utility ipfstat is great for collecting snapshots of what's going on on the system, but it's often handy to have some kind of log to look at and watch events as they happen in time. ipmon is this tool. ipmon is capable of watching the packet log (as created with the log keyword in your rules), the state log, or the nat log, or any combination of the three. This tool can either be run in the foreground, or as a daemon which logs to syslog or a file. If we wanted to watch the state table in action, ipmon -o S would show this: # ipmon -o S 01/08/1999 15:58:57.836053 STATE:NEW 100.100.100.1,53 -> 20.20.20.15,53 PR udp 01/08/1999 15:58:58.030815 STATE:NEW 20.20.20.15,123 -> 128.167.1.69,123 PR udp 01/08/1999 15:59:18.032174 STATE:NEW 20.20.20.15,123 -> 128.173.14.71,123 PR udp -33- 01/08/1999 15:59:24.570107 STATE:EXPIRE 100.100.100.1,53 -> 20.20.20.15,53 PR udp Pkts 4 Bytes 356 01/08/1999 16:03:51.754867 STATE:NEW 20.20.20.13,1019 -> 100.100.100.10,22 PR tcp 01/08/1999 16:04:03.070127 STATE:EXPIRE 20.20.20.13,1019 -> 100.100.100.10,22 PR tcp Pkts 63 Bytes 4604 Here we see a state entry for an external dns request off our nameserver, two xntp pings to well-known time servers, and a very short lived outbound ssh connection. ipmon is also capable of showing us what packets have been logged. For example, when using state, you'll often run into packets like this: # ipmon -o I 15:57:33.803147 ppp0 @0:2 b 100.100.100.103,443 -> 20.20.20.10,4923 PR tcp len 20 1488 -A What does this mean? The first field is obvious, it's a timestamp. The second field is also pretty obvious, it's the interface that this event happened on. The third field @0:2 is something most people miss. This is the rule that caused the event to happen. Remember ipfstat -in? If you wanted to know where this came from, you could look there for rule 2 in rule group 0. The fourth field, the little "b" says that this packet was blocked, and you'll generally ignore this unless you're logging passed packets as well, which would be a little "p" instead. The fifth and sixth fields are pretty self-explanatory, they say where this packet came from and where it was going. The seventh ("PR") and eighth fields tell you the protocol and the ninth field tells you the size of the packet. The last part, the "-A" in this case, tells you the flags that were on the packet; This one was an ACK packet. Why did I mention state ear- lier? Due to the often laggy nature of the Internet, some- times packets will be regenerated. Sometimes, you'll get two copies of the same packet, and your state rule which keeps track of sequence numbers will have already seen this packet, so it will assume that the packet is part of a dif- ferent connection. Eventually this packet will run into a real rule and have to be dealt with. You'll often see the last packet of a session being closed get logged because the keep state code has already torn down the connection before the last packet has had a chance to make it to your fire- wall. This is normal, do not be alarmed. Another example packet that might be logged: 12:46:12.470951 xl0 @0:1 S 20.20.20.254 -> 255.255.255.255 PR icmp len 20 9216 icmp 9/0 ----------- For a technical presentation of the IP Filter stateful inspection engine, please see the white paper Real Stateful TCP Packet Filtering in IP Filter, by Guido van Rooij. This paper may be found at -34- This is an ICMP router discovery broadcast. We can tell by the ICMP type 9/0. Finally, ipmon also lets us look at the NAT table in action. # ipmon -o N 01/08/1999 05:30:02.466114 @2 NAT:RDR 20.20.20.253,113 <- -> 20.20.20.253,113 [100.100.100.13,45816] 01/08/1999 05:30:31.990037 @2 NAT:EXPIRE 20.20.20.253,113 <- -> 20.20.20.253,113 [100.100.100.13,45816] Pkts 10 Bytes 455 This would be a redirection to an identd that lies to pro- vide ident service for the hosts behind our NAT, since they are typically unable to provide this service for themselves with ordinary natting. 8. Specific Applications of IP Filter - Things that don't fit, but should be mentioned anyway. 8.1. Keep State With Servers and Flags. Keeping state is a good thing, but it's quite easy to make a mistake in the direction that you want to keep state in. Generally, you want to have a keep state keyword on the first rule that interacts with a packet for the connec- tion. One common mistake that is made when mixing state tracking with filtering on flags is this: block in all pass in quick proto tcp from any to 20.20.20.20/32 port = 23 flags S pass out all keep state That certainly appears to allow a connection to be created to the telnet server on 20.20.20.20, and the replies to go back. If you try using this rule, you'll see that it does work--Momentarily. Since we're filtering for the SYN flag, the state entry never fully gets completed, and the default time to live for an incomplete state is 60 seconds. We can solve this by rewriting the rules in one of two ways: 1) block in all pass in quick proto tcp from any to 20.20.20.20/32 port = 23 keep state block out all or: 2) block in all pass in quick proto tcp from any to 20.20.20.20/32 port = 23 flags S keep state -35- pass out all keep state Either of these sets of rules will result in a fully estab- lished state entry for a connection to your server. 8.2. Coping With FTP FTP is one of those protocols that you just have to sit back and ask "What the heck were they thinking?" FTP has many problems that the firewall administrator needs to deal with. What's worse, the problems the administrator must face are different between making ftp clients work and mak- ing ftp servers work. Within the FTP protocol, there are two forms of data transfer, called active and passive. Active transfers are those where the server connects to an open port on the client to send data. Conversely, passive transfers are those where the client connects to the server to receive data. 8.2.1. Running an FTP Server In running an FTP server, handling Active FTP sessions is easy to setup. At the same time, handling Passive FTP sessions is a big problem. First we'll cover how to handle Active FTP, then move on to Passive. Generally, we can han- dle Active FTP sessions like we would an incoming HTTP or SMTP connection; just open the ftp port and let keep state do the rest: pass in quick proto tcp from any to 20.20.20.20/32 port = 21 flags S keep state pass out proto tcp all keep state These rules will allow Active FTP sessions, the most common type, to your ftp server on 20.20.20.20. The next challenge becomes handling Passive FTP connec- tions. Web browsers default to this mode, so it's becoming quite popular and as such it should be supported. The prob- lem with passive connections are that for every passive con- nection, the server starts listening on a new port (usually above 1023). This is essentially like creating a new unknown service on the server. Assuming we have a good firewall with a default-deny policy, that new service will be blocked, and thus Active FTP sessions are broken. Don't despair! There's hope yet to be had. A person's first inclination to solving this problem might be to just open up all ports above 1023. In truth, this will work: pass in quick proto tcp from any to 20.20.20.20/32 port > 1023 flags S keep state pass out proto tcp all keep state -36- This is somewhat unsatisfactory, though. By letting every- thing above 1023 in, we actually open ourselves up for a number of potential problems. While 1-1023 is the desig- nated area for server services to run, numerous programs decided to use numbers higher than 1023, such as nfsd and X. The good news is that your FTP server gets to decide which ports get assigned to passive sessions. This means that instead of opening all ports above 1023, you can allo- cate ports 15001-19999 as ftp ports and only open that range of your firewall up. In wu-ftpd, this is done with the pas- sive ports option in ftpaccess. Please see the man page on ftpaccess for details in wu-ftpd configuration. On the ipfilter side, all we need do is setup corresponding rules: pass in quick proto tcp from any to 20.20.20.20/32 port 15000 >< 20000 flags S keep state pass out proto tcp all keep state If even this solution doesn't satisfy you, you can always hack IPF support into your FTP server, or FTP server support into IPF. 8.2.2. Running an FTP Client While FTP server support is still less than perfect in IPF, FTP client support has been working well since 3.3.3. As with FTP servers, there are two types of ftp client transfers: passive and active. The simplest type of client transfer from the fire- wall's standpoint is the passive transfer. Assuming you're keeping state on all outbound tcp sessions, passive trans- fers will work already. If you're not doing this already, please consider the following: pass out proto tcp all keep state The second type of client transfer, active, is a bit more troublesome, but nonetheless a solved problem. Active transfers cause the server to open up a second connection back to the client for data to flow through. This is nor- mally a problem when there's a firewall in the middle, stop- ping outside connections from coming back in. To solve this, ipfilter includes an ipnat proxy which temporarily opens up a hole in the firewall just for the FTP server to get back to the client. Even if you're not using ipnat to do nat, the proxy is still effective. The following rules is the bare minimum to add to the ipnat configuration file (ep0 should be the interface name of the outbound network connection): map ep0 0/0 -> 0/32 proxy port 21 ftp/tcp -37- For more details on ipfilter's internal proxies, see section 3.6 8.3. Assorted Kernel Variables There are some useful kernel tunes that either need to be set for ipf to function, or are just generally handy to know about for building firewalls. The first major one you must set is to enable IP Forwarding, otherwise ipf will do very little, as the underlying ip stack won't actually route packets. IP Forwarding: openbsd: net.inet.ip.forwarding=1 freebsd: net.inet.ip.forwarding=1 netbsd: net.inet.ip.forwarding=1 solaris: ndd -set /dev/ip ip_forwarding 1 Ephemeral Port Adjustment: openbsd: net.inet.ip.portfirst = 25000 freebsd: net.inet.ip.portrange.first = 25000 net.inet.ip.por- trange.last = 49151 netbsd: net.inet.ip.anonportmin = 25000 net.inet.ip.anonportmax = 49151 solaris: ndd -set /dev/tcp tcp_smallest_anon_port 25000 ndd -set /dev/tcp tcp_largest_anon_port 65535 Other Useful Values: openbsd: net.inet.ip.sourceroute = 0 net.inet.ip.directed-broadcast = 0 -38- freebsd: net.inet.ip.sourceroute=0 net.ip.accept_sourceroute=0 netbsd: net.inet.ip.allowsrcrt=0 net.inet.ip.forwsrcrt=0 net.inet.ip.directed-broadcast=0 net.inet.ip.redirect=0 solaris: ndd -set /dev/ip ip_forward_directed_broadcasts 0 ndd -set /dev/ip ip_forward_src_routed 0 ndd -set /dev/ip ip_respond_to_echo_broadcast 0 In addition, freebsd has some ipf specific sysctl variables. net.inet.ipf.fr_flags: 0 net.inet.ipf.fr_pass: 514 net.inet.ipf.fr_active: 0 net.inet.ipf.fr_tcpidletimeout: 864000 net.inet.ipf.fr_tcpclosewait: 60 net.inet.ipf.fr_tcplastack: 20 net.inet.ipf.fr_tcptimeout: 120 net.inet.ipf.fr_tcpclosed: 1 net.inet.ipf.fr_udptimeout: 120 net.inet.ipf.fr_icmptimeout: 120 net.inet.ipf.fr_defnatage: 1200 net.inet.ipf.fr_ipfrttl: 120 net.inet.ipf.ipl_unreach: 13 net.inet.ipf.ipl_inited: 1 net.inet.ipf.fr_authsize: 32 net.inet.ipf.fr_authused: 0 net.inet.ipf.fr_defaultauthage: 600 9. Fun with ipf! This section doesn't necessarily teach you anything new about ipf, but it may raise an issue or two that you haven't yet thought up on your own, or tickle your brain in a way that you invent something interesting that we haven't thought of. 9.1. Localhost Filtering A long time ago at a university far, far away, Wietse Venema created the tcp-wrapper package, and ever since, it's been used to add a layer of protection to network services all over the world. This is good. But, tcp-wrappers have -39- flaws. For starters, they only protect TCP services, as the name suggests. Also, unless you run your service from inetd, or you have specifically compiled it with libwrap and the appropriate hooks, your service isn't protected. This leaves gigantic holes in your host security. We can plug these up by using ipf on the local host. For example, my laptop often gets plugged into or dialed into networks that I don't specifically trust, and so, I use the following rule set: pass in quick on lo0 all pass out quick on lo0 all block in log all block out all pass in quick proto tcp from any to any port = 113 flags S keep state pass in quick proto tcp from any to any port = 22 flags S keep state pass in quick proto tcp from any port = 20 to any port 39999 >< 45000 flags S keep state pass out quick proto icmp from any to any keep state pass out quick proto tcp/udp from any to any keep state keep frags It's been like that for quite a while, and I haven't suf- fered any pain or anguish as a result of having ipf loaded up all the time. If I wanted to tighten it up more, I could switch to using the NAT ftp proxy and I could add in some rules to prevent spoofing. But even as it stands now, this box is far more restrictive about what it presents to the local network and beyond than the typical host does. This is a good thing if you happen to run a machine that allows a lot of users on it, and you want to make sure one of them doesn't happen to start up a service they wern't supposed to. It won't stop a malicious hacker with root access from adjusting your ipf rules and starting a service anyway, but it will keep the "honest" folks honest, and your weird ser- vices safe, cozy and warm even on a malicious LAN. A big win, in my opinion. Using local host filtering in addition to a somewhat less-restrictive "main firewall" machine can solve many performance issues as well as political night- mares like "Why doesn't ICQ work?" and "Why can't I put a web server on my own workstation! It's MY WORKSTATION!!" Another very big win. Who says you can't have security and convienence at the same time? 9.2. What Firewall? Transparent filtering. One major concern in setting up a firewall is the integrity of the firewall itself. Can somebody break into your firewall, thereby subverting its ruleset? This is a common problem administrators must face, particularly when they're using firewall solutions on top of their Unix/NT machines. Some use it as an argument for blackbox hardware solutions, under the flawed notion that inherent obscurity -40- of their closed system increases their security. We have a better way. Many network admins are familiar with the common ether- net bridge. This is a device that connects two separate ethernet segments to make them one. An ethernet bridge is typically used to connect separate buildings, switch network speeds, and extend maximum wire lengths. Hubs and switches are common bridges, sometimes they're just 2 ported devices called repeaters. Recent versions of Linux, OpenBSD, NetBSD, and FreeBSD include code to convert $1000 PCs into $10 bridges, too! What all bridges tend to have in common is that though they sit in the middle of a connection between two machines, the two machines don't know the bridge is there. Enter ipfilter and OpenBSD. Ethernet bridging takes place at Layer2 on the ISO stack. IP takes place on Layer3. IP Filter in primarily concerned with Layer3, but dabbles in Layer2 by working with interfaces. By mixing IP filter with OpenBSD's bridge device, we can create a firewall that is both invisible and unreachable. The system needs no IP address, it doesn't even need to reveal its ethernet address. The only telltale sign that the filter might be there is that latency is some- what higher than a piece of cat5 would normally make it, and that packets don't seem to make it to their final destina- tion. The setup for this sort of ruleset is surprisingly sim- ple, too. In OpenBSD, the first bridge device is named bridge0. Say we have two ethernet cards in our machine as well, xl0 and xl1. To turn this machine into a bridge, all one need do is enter the following three commands: brconfig bridge0 add xl0 add xl1 up ifconfig xl0 up ifconfig xl1 up At ths point, all traffic ariving on xl0 is sent out xl1 and all traffic on xl1 is sent out xl0. You'll note that nei- ther interface has been assigned an IP address, nor do we need assign one. All things considered, it's likely best we not add one at all. Rulesets behave essentially the as the always have. Though there is a bridge0 interface, we don't filter based on it. Rules continue to be based upon the particular interface we're using, making it important which network cable is plugged into which network card in the back of the machine. Let's start with some basic filtering to illis- trate what's happened. Assume the network used to look like this: -41- 20.20.20.1 <---------------------------------> 20.20.20.0/24 network hub That is, we have a router at 20.20.20.1 connected to the 20.20.20.0/24 network. All packets from the 20.20.20.0/24 network go through 20.20.20.1 to get to the outside world and vice versa. Now we add the Ipf Bridge: 20.20.20.1 <-------/xl0 IpfBridge xl1/-------> 20.20.20.0/24 network hub We also have the following ruleset loaded on the IpfBridge host: pass in quick all pass out quick all With this ruleset loaded, the network is functionally iden- tical. As far as the 20.20.20.1 router is concerned, and as far as the 20.20.20.0/24 hosts are concerned, the two net- work diagrams are identical. Now let's change the ruleset some: block in quick on xl0 proto icmp pass in quick all pass out quick all Still, 20.20.20.1 and 20.20.20.0/24 think the network is identical, but if 20.20.20.1 attempts to ping 20.20.20.2, it will never get a reply. What's more, 20.20.20.2 won't even get the packet in the first place. IPfilter will intercept the packet before it even gets to the other end of the vir- tual wire. We can put a bridged filter anywhere. Using this method we can shrink the network trust circle down an individual host level (given enough ethernet cards:-) Blocking icmp from the world seems kind of silly, espe- cially if you're a sysadmin and like pinging the world, to traceroute, or to resize your MTU. Let's construct a better ruleset and take advantage of the original key feature of ipf: stateful inspection. pass in quick on xl1 proto tcp keep state pass in quick on xl1 proto udp keep state pass in quick on xl1 proto icmp keep state block in quick on xl0 In this situation, the 20.20.20.0/24 network (perhaps more aptly called the xl1 network) can now reach the outside world, but the outside world can't reach it, and it can't figure out why, either. The router is accessible, the hosts are active, but the outside world just can't get in. Even if the router itself were compromised, the firewall would still be active and successful. -42- So far, we've been filtering by interface and protocol only. Even though bridging is concerned layer2, we can still discriminate based on IP address. Normally we have a few services running, so our ruleset may look like this: pass in quick on xl1 proto tcp keep state pass in quick on xl1 proto udp keep state pass in quick on xl1 proto icmp keep state block in quick on xl1 # nuh-uh, we're only passing tcp/udp/icmp sir. pass in quick on xl0 proto udp from any to 20.20.20.2/32 port=53 keep state pass in quick on xl0 proto tcp from any to 20.20.20.2/32 port=53 flags S keep state pass in quick on xl0 proto tcp from any to 20.20.20.3/32 port=25 flags S keep state pass in quick on xl0 proto tcp from any to 20.20.20.7/32 port=80 flags S keep state block in quick on xl0 Now we have a network where 20.20.20.2 is a zone serving name server, 20.20.20.3 is an incoming mail server, and 20.20.20.7 is a web server. Bridged IP Filter is not yet perfect, we must confess. First, You'll note that all the rules are setup using the in direction instead of a combination of in and out. This is because the out direction is presently unimplemented with bridging in OpenBSD. This was originally done to pre- vent vast performance drops using multiple interfaces. Work has been done in speeding it up, but it remains unimple- mented. If you really want this feature, you might try your hand at working on the code or asking the OpenBSD people how you can help. Second, using IP Filter with bridging makes the use of IPF's NAT features inadvisable, if not downright dangerous. The first problem is that it would give away that there's a filtering bridge. The second problem would be that the bridge has no IP address to masquerade with, which will most assuredly lead to confusion and perhaps a kernel panic to boot. You can, of course, put an IP address on the outbound interface to make NAT work, but part of the glee of bridging is thus diminished. 9.2.1. Using Transparent Filtering to Fix Network Design Mistakes Many organizations started using IP well before they thought a firewall or a subnet would be a good idea. Now they have class-C sized networks or larger that include all their servers, their workstations, their routers, coffee makers, everything. The horror! Renumbering with proper subnets, trust levels, filters, and so are in both time con- suming and expensive. The expense in hardware and man hours alone is enough to make most organizations unwilling to really solve the problem, not to mention the downtime involved. The typical problem network looks like this: -43- 20.20.20.1 router 20.20.20.6 unix server 20.20.20.2 unix server 20.20.20.7 nt workstation 20.20.20.3 unix server 20.20.20.8 nt server 20.20.20.4 win98 workstation 20.20.20.9 unix workstation 20.20.20.5 intelligent switch 20.20.20.10 win95 workstation Only it's about 20 times larger and messier and frequently undocumented. Ideally, you'd have all the trusting servers in one subnet, all the work- stations in another, and the network switches in a third. Then the router would filter packets between the subnets, giving the workstations limited access to the servers, nothing access to the switches, and only the sysadmin's workstation access to the coffee pot. I've never seen a class-C sized network with such coherence. IP Filter can help. To start with, we're going to separate the router, the workstations, and the servers. To do this we're going to need 2 hubs (or switches) which we probably already have, and an IPF machine with 3 ethernet cards. We're going to put all the servers on one hub and all the workstations on the other. Normally we'd then connect the hubs to each other, then to the router. Instead, we're going to plug the router into IPF's xl0 interface, the servers into IPF's xl1 interface, and the workstations into IPF's xl2 interface. Our network diagram looks something like this: | 20.20.20.2 unix server router (20.20.20.1) ____________| 20.20.20.3 unix server | / | 20.20.20.6 unix server | /xl1 | 20.20.20.7 nt server ------------/xl0 IPF Bridge < xl2 | 20.20.20.4 win98 workstation ____________| 20.20.20.8 nt workstation | 20.20.20.9 unix workstation | 20.20.20.10 win95 workstation Where once there was nothing but interconnecting wires, now there's a filtering bridge that not a single host needs to be modified to take advantage of. Presumably we've already enabled bridging so the network is behaving perfectly nor- mally. Further, we're starting off with a ruleset much like our last ruleset: pass in quick on xl0 proto udp from any to 20.20.20.2/32 port=53 keep state pass in quick on xl0 proto tcp from any to 20.20.20.2/32 port=53 flags S keep state pass in quick on xl0 proto tcp from any to 20.20.20.3/32 port=25 flags S keep state pass in quick on xl0 proto tcp from any to 20.20.20.7/32 port=80 flags S keep state block in quick on xl0 pass in quick on xl1 proto tcp keep state pass in quick on xl1 proto udp keep state pass in quick on xl1 proto icmp keep state block in quick on xl1 # nuh-uh, we're only passing tcp/udp/icmp sir. pass in quick on xl2 proto tcp keep state -44- pass in quick on xl2 proto udp keep state pass in quick on xl2 proto icmp keep state block in quick on xl2 # nuh-uh, we're only passing tcp/udp/icmp sir. Once again, traffic coming from the router is restricted to DNS, SMTP, and HTTP. At the moment, the servers and the workstations can exchange traffic freely. Depending on what kind of organization you are, there might be something about this network dynamic you don't like. Perhaps you don't want your workstations getting access to your servers at all? Take the xl2 ruleset of: pass in quick on xl2 proto tcp keep state pass in quick on xl2 proto udp keep state pass in quick on xl2 proto icmp keep state block in quick on xl2 # nuh-uh, we're only passing tcp/udp/icmp sir. And change it to: block in quick on xl2 from any to 20.20.20.0/24 pass in quick on xl2 proto tcp keep state pass in quick on xl2 proto udp keep state pass in quick on xl2 proto icmp keep state block in quick on xl2 # nuh-uh, we're only passing tcp/udp/icmp sir. Perhaps you want them to just get to the servers to get and send their mail with IMAP? Easily done: pass in quick on xl2 proto tcp from any to 20.20.20.3/32 port=25 pass in quick on xl2 proto tcp from any to 20.20.20.3/32 port=143 block in quick on xl2 from any to 20.20.20.0/24 pass in quick on xl2 proto tcp keep state pass in quick on xl2 proto udp keep state pass in quick on xl2 proto icmp keep state block in quick on xl2 # nuh-uh, we're only passing tcp/udp/icmp sir. Now your workstations and servers are protected from the outside world, and the servers are protected from your work- stations. Perhaps the opposite is true, maybe you want your work- stations to be able to get to the servers, but not the out- side world. After all, the next generation of exploits is breaking the clients, not the servers. In this case, you'd change the xl2 rules to look more like this: pass in quick on xl2 from any to 20.20.20.0/24 block in quick on xl2 Now the servers have free reign, but the clients can only connect to the servers. We might want to batten down the hatches on the servers, too: pass in quick on xl1 from any to 20.20.20.0/24 -45- block in quick on xl1 With the combination of these two, the clients and servers can talk to each other, but neither can access the outside world (though the outside world can get to the few services from earlier). The whole ruleset would look something like this: pass in quick on xl0 proto udp from any to 20.20.20.2/32 port=53 keep state pass in quick on xl0 proto tcp from any to 20.20.20.2/32 port=53 flags S keep state pass in quick on xl0 proto tcp from any to 20.20.20.3/32 port=25 flags S keep state pass in quick on xl0 proto tcp from any to 20.20.20.7/32 port=80 flags S keep state block in quick on xl0 pass in quick on xl1 from any to 20.20.20.0/24 block in quick on xl1 pass in quick on xl2 from any to 20.20.20.0/24 block in quick on xl2 So remember, when your network is a mess of twisty IP addresses and machine classes, transparent filtered bridges can solve a problem that would otherwise be lived with and perhaps someday exploited. 9.3. Drop-Safe Logging With dup-to and to. Until now, we've been using the filter to drop packets. Instead of dropping them, let's consider passing them on to another system that can do something useful with this infor- mation beyond the logging we can perform with ipmon. Our firewall system, be it a bridge or a router, can have as many interfaces as we can cram into the system. We can use this information to create a "drop-safe" for our packets. A good example of a use for this would be to implement an intrusion detection network. For starters, it might be desirable to hide the presence of our intrusion detection systems from our real network so that we can keep them from being detected. Before we get started, there are some operational char- acteristics that we need to make note of. If we are only going to deal with blocked packets, we can use either the to keyword or the fastroute keyword. (We'll cover the differ- ences between these two later) If we're going to pass the packets like we normally would, we need to make a copy of the packet for our drop-safe log with the dup-to keyword. 9.3.1. The dup-to Method If, for example, we wanted to send a copy of everything going out the xl3 interface off to our drop-safe network on ed0, we would use this rule in our filter list: pass out on xl3 dup-to ed0 from any to any -46- You might also have a need to send the packet directly to a specific IP address on your drop-safe network instead of just making a copy of the packet out there and hoping for the best. To do this, we modify our rule slightly: pass out on xl3 dup-to ed0:192.168.254.2 from any to any But be warned that this method will alter the copied packet's destination address, and may thus destroy the use- fulness of the log. For this reason, we recommend only using the known address method of logging when you can be certain that the address that you're logging to corresponds in some way to what you're logging for (e.g.: don't use "192.168.254.2" for logging for both your web server and your mail server, since you'll have a hard time later trying to figure out which system was the target of a specific set of packets.) This technique can be used quite effectively if you treat an IP Address on your drop-safe network in much the same way that you would treat a Multicast Group on the real internet. (e.g.: "192.168.254.2" could be the channel for your http traffic analysis system, "23.23.23.23" could be your channel for telnet sessions, and so on.) You don't even need to actually have this address set as an address or alias on any of your analysis systems. Normally, your ipfilter machine would need to ARP for the new destination address (using dup-to ed0:192.168.254.2 style, of course) but we can avoid that issue by creating a static arp entry for this "channel" on our ipfilter system. In general, though, dup-to ed0 is all that is required to get a new copy of the packet over to our drop-safe net- work for logging and examination. 9.3.2. The to Method The dup-to method does have an immediate drawback, though. Since it has to make a copy of the packet and optionally modify it for its new destination, it's going to take a while to complete all this work and be ready to deal with the next packet coming in to the ipfilter system. If we don't care about passing the packet to its normal destination and we were going to block it anyway, we can just use the to keyword to push this packet past the normal routing table and force it to go out a different interface than it would normally go out. block in quick on xl0 to ed0 proto tcp from any to any port < 1024 we use block quick for to interface routing, because like fastroute, the to interface code will generate two packet paths through ipfilter when used with pass, and likely cause -47- your system to panic. 10. Bogus Network Filtering, the ultimate in current anti- spoofing technology. We've spent a little bit of time tracking down the cur- rent vast tracts of IP address space that have been reserved by the IANA for various reasons, or are otherwise not cur- rently in use at the time this document was written. Since none of these address ranges should be in use currently, there should be no legitimate reason to ever see them as a source address, or to send them traffic as a destination address, right? Right! So without further ado, the complete list of bogus net- works: # # s/OUTSIDE/outside-interface (eg: fxp0) # s/MYNET/network-cidr-address (eg: 1.2.3.0/24) # block in on OUTSIDE all block in quick on OUTSIDE from 0.0.0.0/7 to any block in quick on OUTSIDE from 2.0.0.0/8 to any block in quick on OUTSIDE from 5.0.0.0/8 to any block in quick on OUTSIDE from 10.0.0.0/8 to any block in quick on OUTSIDE from 23.0.0.0/8 to any block in quick on OUTSIDE from 27.0.0.0/8 to any block in quick on OUTSIDE from 31.0.0.0/8 to any block in quick on OUTSIDE from 67.0.0.0/8 to any block in quick on OUTSIDE from 68.0.0.0/6 to any block in quick on OUTSIDE from 72.0.0.0/5 to any block in quick on OUTSIDE from 80.0.0.0/4 to any block in quick on OUTSIDE from 96.0.0.0/3 to any block in quick on OUTSIDE from 127.0.0.0/8 to any block in quick on OUTSIDE from 128.0.0.0/16 to any block in quick on OUTSIDE from 128.66.0.0/16 to any block in quick on OUTSIDE from 169.254.0.0/16 to any block in quick on OUTSIDE from 172.16.0.0/12 to any block in quick on OUTSIDE from 191.255.0.0/16 to any block in quick on OUTSIDE from 192.0.0.0/16 to any block in quick on OUTSIDE from 192.168.0.0/16 to any block in quick on OUTSIDE from 197.0.0.0/8 to any block in quick on OUTSIDE from 201.0.0.0/8 to any block in quick on OUTSIDE from 204.152.64.0/23 to any block in quick on OUTSIDE from 224.0.0.0/3 to any block in quick on OUTSIDE from MYNET to any # Your pass rules come here... block out on OUTSIDE all block out quick on OUTSIDE from !MYNET to any block out quick on OUTSIDE from MYNET to 0.0.0.0/7 -48- block out quick on OUTSIDE from MYNET to 2.0.0.0/8 block out quick on OUTSIDE from MYNET to 5.0.0.0/8 block out quick on OUTSIDE from MYNET to 10.0.0.0/8 block out quick on OUTSIDE from MYNET to 23.0.0.0/8 block out quick on OUTSIDE from MYNET to 27.0.0.0/8 block out quick on OUTSIDE from MYNET to 31.0.0.0/8 block out quick on OUTSIDE from MYNET to 67.0.0.0/8 block out quick on OUTSIDE from MYNET to 68.0.0.0/6 block out quick on OUTSIDE from MYNET to 72.0.0.0/5 block out quick on OUTSIDE from MYNET to 80.0.0.0/4 block out quick on OUTSIDE from MYNET to 96.0.0.0/3 block out quick on OUTSIDE from MYNET to 127.0.0.0/8 block out quick on OUTSIDE from MYNET to 128.0.0.0/16 block out quick on OUTSIDE from MYNET to 128.66.0.0/16 block out quick on OUTSIDE from MYNET to 169.254.0.0/16 block out quick on OUTSIDE from MYNET to 172.16.0.0/12 block out quick on OUTSIDE from MYNET to 191.255.0.0/16 block out quick on OUTSIDE from MYNET to 192.0.0.0/16 block out quick on OUTSIDE from MYNET to 192.168.0.0/16 block out quick on OUTSIDE from MYNET to 197.0.0.0/8 block out quick on OUTSIDE from MYNET to 201.0.0.0/8 block out quick on OUTSIDE from MYNET to 204.152.64.0/23 block out quick on OUTSIDE from MYNET to 224.0.0.0/3 # Your pass rules come here... If you're going to use these, we suggest that you become familiar with whois.arin.net and keep an occasional eye on these, as the IANA isn't going to notify you when they allo- cate one of these to a new corporation or something. You have been warned. We'd also like to thank Frank DiGennaro for greatly contributing to this filter list. Index: stable/10/share/examples/kld/cdev/test/Makefile =================================================================== --- stable/10/share/examples/kld/cdev/test/Makefile (revision 299825) +++ stable/10/share/examples/kld/cdev/test/Makefile (revision 299826) @@ -1,95 +1,95 @@ # 05 Jun 93 # # Makefile for testmisc # # 05 Jun 93 Rajesh Vaidheeswarran Original # # Copyright (c) 1993 Rajesh Vaidheeswarran. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. All advertising materials mentioning features or use of this software # must display the following acknowledgement: # This product includes software developed by Rajesh Vaidheeswarran. # 4. The name Rajesh Vaidheeswarran may not be used to endorse or promote # products derived from this software without specific prior written # permission. # # THIS SOFTWARE IS PROVIDED BY RAJESH VAIDHEESWARRAN ``AS IS'' AND ANY # EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE RAJESH VAIDHEESWARRAN BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # Copyright (c) 1993 Terrence R. Lambert. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. All advertising materials mentioning features or use of this software # must display the following acknowledgement: # This product includes software developed by Terrence R. Lambert. # 4. The name Terrence R. Lambert may not be used to endorse or promote # products derived from this software without specific prior written # permission. # # THIS SOFTWARE IS PROVIDED BY TERRENCE R. LAMBERT ``AS IS'' AND ANY # EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE TERRENCE R. LAMBERT BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # $FreeBSD$ # PROG= testcdev MAN= WARNS?= 5 MODSTAT= /sbin/kldstat load: - @echo "This test program will call the sample kld characer device "; + @echo "This test program will call the sample kld character device "; @echo "driver." @echo @echo "The sample driver will display a message on the" @echo "system console each time an ioctl is sent to it." @echo @echo @echo @./testcdev unload: @echo "This test program will cause an error if the driver" @echo "has been successfully unloaded by building 'unload' in" @echo "the 'module' subdirectory." @echo ${MODSTAT} -n cdev install: .include Index: stable/10/tools/regression/sockets/so_setfib/so_setfib.c =================================================================== --- stable/10/tools/regression/sockets/so_setfib/so_setfib.c (revision 299825) +++ stable/10/tools/regression/sockets/so_setfib/so_setfib.c (revision 299826) @@ -1,200 +1,200 @@ /*- * Copyright (c) 2012 Cisco Systems, Inc. * All rights reserved. * * This software was developed by Bjoern Zeeb under contract to * Cisco Systems, Inc.. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * Regression test on SO_SETFIB setsockopt(2). * * Check that the expected domain(9) families all handle the socket option * correctly and do proper bounds checks. * * Test plan: * 1. Get system wide number of FIBs from sysctl and convert to index (-= 1). * 2. For each protocol family (INET, INET6, ROUTE and LOCAL) open socketes of * type (STREAM, DGRAM and RAW) as supported. * 3. Do a sequence of -2, -1, 0, .. n, n+1, n+2 SO_SETFIB sockopt calls, * expecting the first two and last two to fail (valid 0 ... n). * 4. Try 3 random numbers. Calculate result based on valid range. * 5. Repeat for next domain family and type from (2) on. */ #include #include #include #include #include #include #include #include #include #include static struct t_dom { int domain; const char *name; } t_dom[] = { #ifdef INET6 { .domain = PF_INET6, .name = "PF_INET6" }, #endif #ifdef INET { .domain = PF_INET, .name = "PF_INET" }, #endif { .domain = PF_ROUTE, .name = "PF_ROUTE" }, { .domain = PF_LOCAL, .name = "PF_LOCAL" }, }; static struct t_type { int type; const char *name; } t_type[] = { { .type = SOCK_STREAM, .name = "SOCK_STREAM" }, { .type = SOCK_DGRAM, .name = "SOCK_DGRAM" }, { .type = SOCK_RAW, .name = "SOCK_RAW" }, }; /* * Number of FIBs as read from net.fibs sysctl - 1. Initialize to clear out of * bounds value to not accidentally run on a limited range. */ static int rt_numfibs = -42; /* Number of test case. */ static int testno = 1; /* * Try the setsockopt with given FIB number i on socket s. * Handle result given on error and valid range and errno. */ static void so_setfib(int s, int i, u_int dom, u_int type) { int error; error = setsockopt(s, SOL_SOCKET, SO_SETFIB, &i, sizeof(i)); /* For out of bounds we expect an error. */ if (error == -1 && (i < 0 || i > rt_numfibs)) printf("ok %d %s_%s_%d\n", testno, t_dom[dom].name, t_type[type].name, i); else if (error != -1 && (i < 0 || i > rt_numfibs)) printf("not ok %d %s_%s_%d # setsockopt(%d, SOL_SOCKET, " "SO_SETFIB, %d, ..) unexpectedly succeeded\n", testno, t_dom[dom].name, t_type[type].name, i, s, i); else if (error == 0) printf("ok %d %s_%s_%d\n", testno, t_dom[dom].name, t_type[type].name, i); else if (errno != EINVAL) printf("not ok %d %s_%s_%d # setsockopt(%d, SOL_SOCKET, " "SO_SETFIB, %d, ..) unexpected error: %s\n", testno, t_dom[dom].name, t_type[type].name, i, s, i, strerror(errno)); else printf("not ok %d %s_%s_%d\n", testno, t_dom[dom].name, t_type[type].name, i); /* Test run done, next please. */ testno++; } /* * Main test. Open socket given domain family and type. For each FIB, out of * bounds FIB numbers and 3 random FIB numbers set the socket option. */ static void t(u_int dom, u_int type) { int i, s; /* PF_ROUTE only supports RAW socket types, while PF_LOCAL does not. */ if (t_dom[dom].domain == PF_ROUTE && t_type[type].type != SOCK_RAW) return; if (t_dom[dom].domain == PF_LOCAL && t_type[type].type == SOCK_RAW) return; /* Open socket for given combination. */ s = socket(t_dom[dom].domain, t_type[type].type, 0); if (s == -1) { printf("not ok %d %s_%s # socket(): %s\n", testno, t_dom[dom].name, t_type[type].name, strerror(errno)); testno++; return; } /* Test FIBs -2, -1, 0, .. n, n + 1, n + 2. */ for (i = -2; i <= (rt_numfibs + 2); i++) so_setfib(s, i, dom, type); /* Test 3 random FIB numbers. */ for (i = 0; i < 3; i++) so_setfib(s, (int)random(), dom, type); /* Close socket. */ close(s); } /* * Returns 0 if no program error, 1 on sysctlbyname error. * Test results are communicated by printf("[not ]ok .."). */ int main(int argc __unused, char *argv[] __unused) { u_int i, j; size_t s; if (geteuid() != 0) { printf("1..0 # SKIP: must be root\n"); return (0); } - /* Initalize randomness. */ + /* Initialize randomness. */ srandomdev(); /* Get number of FIBs supported by kernel. */ s = sizeof(rt_numfibs); if (sysctlbyname("net.fibs", &rt_numfibs, &s, NULL, 0) == -1) err(1, "sysctlbyname(net.fibs, ..)"); printf("1..%lu\n", (nitems(t_dom) - 1) * nitems(t_type) * (2 + rt_numfibs + 2 + 3)); /* Adjust from number to index. */ rt_numfibs -= 1; /* Run tests. */ for (i = 0; i < sizeof(t_dom) / sizeof(struct t_dom); i++) for (j = 0; j < sizeof(t_type) / sizeof(struct t_type); j++) t(i, j); return (0); } /* end */ Index: stable/10/tools/tools/fixwhite/fixwhite.1 =================================================================== --- stable/10/tools/tools/fixwhite/fixwhite.1 (revision 299825) +++ stable/10/tools/tools/fixwhite/fixwhite.1 (revision 299826) @@ -1,48 +1,48 @@ .\" Copyright (c) 2012 Ed Schouten .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd February 6, 2012 .Dt FIXWHITE 1 .Os .Sh NAME .Nm fixwhite .Nd remove unneeded whitespace from text files .Sh SYNOPSIS .Nm .Sh DESCRIPTION The .Nm utility removes unneeded whitespace from text passed to standard input and prints the result to standard output. .Pp It removes leading and trailing empty lines from the input, as well as trailing whitespace characters from ever line of text. Multiple successive empty lines are merged together. If the whitespace at the beginning of a sentence is exactly a multiple of eight spaces, the whitespace is replaced by tabs. -Also, spaces preceeding tabs will be merged into the tab character. +Also, spaces preceding tabs will be merged into the tab character. .Sh AUTHORS .An Ed Schouten Aq ed@FreeBSD.org Index: stable/10/tools/tools/fixwhite/fixwhite.c =================================================================== --- stable/10/tools/tools/fixwhite/fixwhite.c (revision 299825) +++ stable/10/tools/tools/fixwhite/fixwhite.c (revision 299826) @@ -1,180 +1,180 @@ /*- * Copyright (c) 2012 Ed Schouten * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include static char *queue = NULL; static size_t queuelen = 0, queuesize = 0; static off_t column = 0; static void savebyte(char c) { if (queuelen >= queuesize) { queuesize += 128; queue = realloc(queue, queuesize); if (queue == NULL) { perror("malloc"); exit(1); } } queue[queuelen++] = c; switch (c) { case '\n': column = 0; break; case ' ': column++; break; case '\t': column = (column / 8 + 1) * 8; break; } } static bool peekbyte(size_t back, char c) { return (queuelen >= back && queue[queuelen - back] == c); } static void savewhite(char c, bool leading) { off_t ncolumn; switch (c) { case '\n': if (leading) { /* Remove empty lines before input. */ queuelen = 0; column = 0; } else { /* Remove trailing whitespace. */ while (peekbyte(1, ' ') || peekbyte(1, '\t')) queuelen--; /* Remove redundant empty lines. */ if (peekbyte(2, '\n') && peekbyte(1, '\n')) return; savebyte('\n'); } break; case ' ': savebyte(' '); break; case '\t': - /* Convert preceeding spaces to tabs. */ + /* Convert preceding spaces to tabs. */ ncolumn = (column / 8 + 1) * 8; while (peekbyte(1, ' ')) { queuelen--; column--; } while (column < ncolumn) savebyte('\t'); break; } } static void printwhite(void) { off_t i; /* Merge spaces at the start of a sentence to tabs if possible. */ if ((column % 8) == 0) { for (i = 0; i < column; i++) if (!peekbyte(i + 1, ' ')) break; if (i == column) { queuelen -= column; for (i = 0; i < column; i += 8) queue[queuelen++] = '\t'; } } if (fwrite(queue, 1, queuelen, stdout) != queuelen) { perror("write"); exit(1); } queuelen = 0; } static char readchar(void) { int c; c = getchar(); if (c == EOF && ferror(stdin)) { perror("read"); exit(1); } return (c); } static void writechar(char c) { if (putchar(c) == EOF) { perror("write"); exit(1); } /* XXX: Multi-byte characters. */ column++; } int main(void) { int c; bool leading = true; while ((c = readchar()) != EOF) { if (isspace(c)) /* Save whitespace. */ savewhite(c, leading); else { /* Reprint whitespace and print regular character. */ printwhite(); writechar(c); leading = false; } } /* Terminate non-empty files with a newline. */ if (!leading) writechar('\n'); return (0); } Index: stable/10/tools/tools/ipw/ipwstats.c =================================================================== --- stable/10/tools/tools/ipw/ipwstats.c (revision 299825) +++ stable/10/tools/tools/ipw/ipwstats.c (revision 299826) @@ -1,277 +1,277 @@ /*- * Copyright (c) 2006 Damien Bergamini * Copyright (c) 2006 Sam Leffler, Errno Consulting * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include static void get_statistics(const char *); int main(int argc, char **argv) { get_statistics((argc > 1) ? argv[1] : "ipw0"); return EX_OK; } struct statistic { int index; const char *desc; int unit; #define INT 1 #define HEX 2 #define MASK HEX #define PERCENTAGE 3 #define BOOL 4 }; /*- * TIM = Traffic Information Message * DTIM = Delivery TIM * ATIM = Announcement TIM * PSP = Power Save Poll * RTS = Request To Send * CTS = Clear To Send * RSSI = Received Signal Strength Indicator */ static const struct statistic tbl[] = { { 1, "Number of frames submitted for transfer", INT }, { 2, "Number of frames transmitted", INT }, { 3, "Number of unicast frames transmitted", INT }, { 4, "Number of unicast frames transmitted at 1Mb/s", INT }, { 5, "Number of unicast frames transmitted at 2Mb/s", INT }, { 6, "Number of unicast frames transmitted at 5.5Mb/s", INT }, { 7, "Number of unicast frames transmitted at 11Mb/s", INT }, { 13, "Number of multicast frames transmitted at 1Mb/s", INT }, { 14, "Number of multicast frames transmitted at 2Mb/s", INT }, { 15, "Number of multicast frames transmitted at 5.5Mb/s", INT }, { 16, "Number of multicast frames transmitted at 11Mb/s", INT }, { 21, "Number of null frames transmitted", INT }, { 22, "Number of RTS frames transmitted", INT }, { 23, "Number of CTS frames transmitted", INT }, { 24, "Number of ACK frames transmitted", INT }, { 25, "Number of association requests transmitted", INT }, { 26, "Number of association responses transmitted", INT }, { 27, "Number of reassociation requests transmitted", INT }, { 28, "Number of reassociation responses transmitted", INT }, { 29, "Number of probe requests transmitted", INT }, - { 30, "Number of probe reponses transmitted", INT }, + { 30, "Number of probe responses transmitted", INT }, { 31, "Number of beacons transmitted", INT }, { 32, "Number of ATIM frames transmitted", INT }, { 33, "Number of disassociation requests transmitted", INT }, { 34, "Number of authentication requests transmitted", INT }, { 35, "Number of deauthentication requests transmitted", INT }, { 41, "Number of bytes transmitted", INT }, { 42, "Number of transmission retries", INT }, { 43, "Number of transmission retries at 1Mb/s", INT }, { 44, "Number of transmission retries at 2Mb/s", INT }, { 45, "Number of transmission retries at 5.5Mb/s", INT }, { 46, "Number of transmission retries at 11Mb/s", INT }, { 51, "Number of transmission failures", INT }, { 54, "Number of transmission aborted due to slow DMA setup", INT }, { 56, "Number of disassociation failures", INT }, { 58, "Number of spanning tree frames transmitted", INT }, { 59, "Number of transmission errors due to missing ACK", INT }, { 61, "Number of frames received", INT }, { 62, "Number of unicast frames received", INT }, { 63, "Number of unicast frames received at 1Mb/s", INT }, { 64, "Number of unicast frames received at 2Mb/s", INT }, { 65, "Number of unicast frames received at 5.5Mb/s", INT }, { 66, "Number of unicast frames received at 11Mb/s", INT }, { 71, "Number of multicast frames received", INT }, { 72, "Number of multicast frames received at 1Mb/s", INT }, { 73, "Number of multicast frames received at 2Mb/s", INT }, { 74, "Number of multicast frames received at 5.5Mb/s", INT }, { 75, "Number of multicast frames received at 11Mb/s", INT }, { 80, "Number of null frames received", INT }, { 81, "Number of poll frames received", INT }, { 82, "Number of RTS frames received", INT }, { 83, "Number of CTS frames received", INT }, { 84, "Number of ACK frames received", INT }, { 85, "Number of CF-End frames received", INT }, { 86, "Number of CF-End + CF-Ack frames received", INT }, { 87, "Number of association requests received", INT }, { 88, "Number of association responses received", INT }, { 89, "Number of reassociation requests received", INT }, { 90, "Number of reassociation responses received", INT }, { 91, "Number of probe requests received", INT }, - { 92, "Number of probe reponses received", INT }, + { 92, "Number of probe responses received", INT }, { 93, "Number of beacons received", INT }, { 94, "Number of ATIM frames received", INT }, { 95, "Number of disassociation requests received", INT }, { 96, "Number of authentication requests received", INT }, { 97, "Number of deauthentication requests received", INT }, { 101, "Number of bytes received", INT }, { 102, "Number of frames with a bad CRC received", INT }, { 103, "Number of frames with a bad CRC received at 1Mb/s", INT }, { 104, "Number of frames with a bad CRC received at 2Mb/s", INT }, { 105, "Number of frames with a bad CRC received at 5.5Mb/s", INT }, { 106, "Number of frames with a bad CRC received at 11Mb/s", INT }, { 112, "Number of duplicated frames received at 1Mb/s", INT }, { 113, "Number of duplicated frames received at 2Mb/s", INT }, { 114, "Number of duplicated frames received at 5.5Mb/s", INT }, { 115, "Number of duplicated frames received at 11Mb/s", INT }, { 119, "Number of duplicated frames received", INT }, { 123, "Number of frames with a bad protocol received", INT }, { 124, "Boot time", INT }, { 125, "Number of frames dropped due to no buffer", INT }, { 126, "Number of frames dropped due to slow DMA setup", INT }, { 128, "Number of frames dropped due to missing fragment", INT }, { 129, "Number of frames dropped due to non-seq fragment", INT }, { 130, "Number of frames dropped due to missing first frame", INT }, { 131, "Number of frames dropped due to incomplete frame", INT }, { 137, "Number of PSP adapter suspends", INT }, { 138, "Number of PSP beacon timeouts", INT }, { 139, "Number of PSP PsPollResponse timeouts", INT }, { 140, "Number of PSP mcast frame timeouts", INT }, { 141, "Number of PSP DTIM frames received", INT }, { 142, "Number of PSP TIM frames received", INT }, { 143, "PSP station Id", INT }, { 147, "RTC time of last association", INT }, { 148, "Percentage of missed beacons", PERCENTAGE }, { 149, "Percentage of missed transmission retries", PERCENTAGE }, { 151, "Number of access points in access points table", INT }, { 153, "Number of associations", INT }, { 154, "Number of association failures", INT }, { 156, "Number of full scans", INT }, { 157, "Card disabled", BOOL }, { 160, "RSSI at time of association", INT }, { 161, "Number of reassociations due to no probe response", INT }, { 162, "Number of reassociations due to poor line quality", INT }, { 163, "Number of reassociations due to load", INT }, { 164, "Number of reassociations due to access point RSSI level", INT }, { 165, "Number of reassociations due to load leveling", INT }, { 170, "Number of times authentication failed", INT }, { 171, "Number of times authentication response failed", INT }, { 172, "Number of entries in association table", INT }, { 173, "Average RSSI", INT }, { 176, "Self test status", INT }, { 177, "Power mode", INT }, { 178, "Power index", INT }, { 179, "IEEE country code", HEX }, { 180, "Channels supported for this country", MASK }, { 181, "Number of adapter warm resets", INT }, { 182, "Beacon interval", INT }, { 184, "Princeton version", INT }, { 185, "Antenna diversity disabled", BOOL }, #if notset { 186, "CCA RSSI", INT }, { 187, "Number of times EEPROM updated", INT }, #endif { 188, "Beacon intervals between DTIM", INT }, { 189, "Current channel", INT }, { 190, "RTC time", INT }, { 191, "Operating mode", INT }, { 192, "Transmission rate", HEX }, { 193, "Supported transmission rates", MASK }, { 194, "ATIM window", INT }, { 195, "Supported basic transmission rates", MASK }, { 196, "Adapter highest rate", HEX }, { 197, "Access point highest rate", HEX }, { 198, "Management frame capability", BOOL }, { 199, "Type of authentication", INT }, { 200, "Adapter card platform type", INT }, { 201, "RTS threshold", INT }, { 202, "International mode", BOOL }, { 203, "Fragmentation threshold", INT }, { 209, "MAC version", INT }, { 210, "MAC revision", INT }, { 211, "Radio version", INT }, { 212, "NIC manufacturing date+time", INT }, { 213, "Microcode version", INT }, { 214, "RF switch state", INT }, { 0, NULL, 0 } }; static void get_statistics(const char *iface) { static uint32_t stats[256]; const struct statistic *stat; char oid[32]; int ifaceno, len; if (sscanf(iface, "ipw%u", &ifaceno) != 1) errx(EX_DATAERR, "Invalid interface name '%s'", iface); len = sizeof stats; snprintf(oid, sizeof oid, "dev.ipw.%u.stats", ifaceno); if (sysctlbyname(oid, stats, &len, NULL, 0) == -1) err(EX_OSERR, "Can't retrieve statistics"); for (stat = tbl; stat->index != 0; stat++) { printf("%-60s[", stat->desc); switch (stat->unit) { case INT: printf("%u", stats[stat->index]); break; case BOOL: printf(stats[stat->index] ? "true" : "false"); break; case PERCENTAGE: printf("%u%%", stats[stat->index]); break; case HEX: default: printf("0x%08X", stats[stat->index]); } printf("]\n"); } } Index: stable/10/tools/tools/nanobsd/dhcpd/common =================================================================== --- stable/10/tools/tools/nanobsd/dhcpd/common (revision 299825) +++ stable/10/tools/tools/nanobsd/dhcpd/common (revision 299826) @@ -1,275 +1,275 @@ # $FreeBSD$ #- # Copyright (c) 2014 Warner Losh. All Rights Reserved. # Copyright (c) 2010 iXsystems, Inc., All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL iXsystems, Inc OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # # This file is heavily derived from both Sam Leffler's Avilia config, # as well as the BSDRP project's config file. Neither of these have # an explicit copyright/license statement, but are implicitly BSDL. This # example has been taken from the FreeNAS project (an early version) and # simplified to meet the needs of the example. # # NB: You want the other file NANO_PMAKE="make -j $(sysctl -n hw.ncpu)" NANO_CFG_BASE=$(pwd) NANO_CFG_BASE=${NANO_CFG_BASE%/dhcpd} NANO_SRC=$(pwd) NANO_SRC=${NANO_SRC%/tools/tools/nanobsd/dhcpd} NANO_OBJ=${NANO_SRC}/../dhcpd/obj # Where cust_pkg() finds packages to install #XXX: Is this the right place? #NANO_PORTS=$(realpath ${NANO_SRC}/../ports) NANO_PORTS=/usr/ports NANO_PACKAGE_DIR=${NANO_SRC}/${NANO_TOOLS}/Pkg NANO_DATADIR=${NANO_OBJ}/_.data NANO_DATASIZE=40960 NANO_INIT_IMG2=0 unset MAKEOBJDIRPREFIX # this to go into nanobsd.sh NANO_PORTS=${NANO_PORTS:-/usr/ports} customize_cmd cust_allow_ssh_root add_etc_make_conf() { touch ${NANO_WORLDDIR}/etc/make.conf } customize_cmd add_etc_make_conf clean_usr_local() { LOCAL_DIR=${NANO_WORLDDIR}/usr/local pprint 2 "Clean and create world directory (${LOCAL_DIR})" if rm -rf ${LOCAL_DIR}/ > /dev/null 2>&1 ; then true else chflags -R noschg ${LOCAL_DIR}/ rm -rf ${LOCAL_DIR}/ fi for f in bin etc lib libdata libexec sbin share; do mkdir -p ${LOCAL_DIR}/$f done } customize_cmd clean_usr_local cust_install_machine_files() { echo "cd ${NANO_CFG_BASE}/Files" cd ${NANO_CFG_BASE}/Files find . -print | grep -Ev '/(CVS|\.svn)' | cpio -dumpv ${NANO_WORLDDIR} } customize_cmd cust_install_files customize_cmd cust_install_machine_files buildenv() { cd ${NANO_SRC} env __MAKE_CONF=${NANO_MAKE_CONF_BUILD} DESTDIR=${NANO_WORLDDIR} make buildenv } NANO_MAKEFS="makefs -B big \ -o bsize=4096,fsize=512,density=8192,optimization=space" export NANO_MAKEFS # NB: leave c++ enabled so devd can be built CONF_BUILD=" WITHOUT_ACPI=true WITHOUT_ATM=true WITHOUT_AUDIT=true WITHOUT_BIND_DNSSEC=true WITHOUT_BIND_ETC=true WITHOUT_BIND_LIBS_LWRES=true WITHOUT_BLUETOOTH=true WITHOUT_CALENDAR=true WITHOUT_CVS=true WITHOUT_DICT=true WITHOUT_EXAMPLES=true WITHOUT_FORTRAN=true WITHOUT_GAMES=true WITHOUT_GCOV=true WITHOUT_GPIB=true WITHOUT_HTML=true WITHOUT_I4B=true WITHOUT_IPFILTER=true WITHOUT_IPX=true WITHOUT_LIBKSE=true WITHOUT_LOCALES=true WITHOUT_LPR=true WITHOUT_MAN=true WITHOUT_NETCAT=true WITHOUT_NIS=true WITHOUT_NLS=true WITHOUT_NS_CACHING=true WITHOUT_OBJC=true WITHOUT_PROFILE=true WITHOUT_RCMDS=true WITHOUT_SENDMAIL=true WITHOUT_SHAREDOCS=true WITHOUT_SYSCONS=true WITHOUT_LIB32=true " CONF_INSTALL="$CONF_BUILD INSTALL_NODEBUG=t NOPORTDOCS=t NO_INSTALL_MANPAGES=t " # The following would help... # WITHOUT_TOOLCHAIN=true can't build ports # WITHOUT_INSTALLLIB=true libgcc.a # # from the build # WITHOUT_INFO=true makeinfo # WITHOUT_RCS=true PKG_ONLY_MAKE_CONF=" WITHOUT_TOOLCHAIN=true WITHOUT_INSTALLLIB=true WITHOUT_INFO=true WITHOUT_RCS=true " NANO_PACKAGE_ONLY=1 # install a package from a pre-built binary do_add_pkg () { # Need to create ${NANO_OBJ}/ports in this add_pkg_${port} function set -x mkdir -p ${NANO_OBJ}/ports/distfiles mkdir -p ${NANO_OBJ}/ports/packages mkdir -p ${NANO_WORLDDIR}/usr/ports/packages mkdir -p ${NANO_WORLDDIR}/usr/ports/distfiles mount -t nullfs -o noatime ${NANO_OBJ}/ports/packages \ ${NANO_WORLDDIR}/usr/ports/packages mount -t nullfs -o noatime ${NANO_OBJ}/ports/distfiles \ ${NANO_WORLDDIR}/usr/ports/distfiles CR env ASSUME_ALWAYS_YES=YES SIGNATURE_TYPE=none /usr/sbin/pkg add /usr/ports/packages/All/$1.txz umount ${NANO_WORLDDIR}/usr/ports/distfiles umount ${NANO_WORLDDIR}/usr/ports/packages rmdir ${NANO_WORLDDIR}/usr/ports/packages rmdir ${NANO_WORLDDIR}/usr/ports/distfiles rmdir ${NANO_WORLDDIR}/usr/ports set +x } # Build a port (with the side effect of creating a package) do_add_port () { local port_path port_path=$1 shift set -x # Need to create ${NANO_OBJ}/ports in this add_port_${port} function mkdir -p ${NANO_OBJ}/ports/distfiles mkdir -p ${NANO_OBJ}/ports/packages mkdir -p ${NANO_PORTS}/packages mkdir -p ${NANO_PORTS}/distfiles mkdir -p ${NANO_WORLDDIR}/usr/src mkdir -p ${NANO_WORLDDIR}/usr/ports mount -t nullfs -o noatime ${NANO_SRC} ${NANO_WORLDDIR}/usr/src mount -t nullfs -o noatime ${NANO_PORTS} ${NANO_WORLDDIR}/usr/ports mount -t nullfs -o noatime ${NANO_OBJ}/ports/packages \ ${NANO_WORLDDIR}/usr/ports/packages mount -t nullfs -o noatime ${NANO_OBJ}/ports/distfiles \ ${NANO_WORLDDIR}/usr/ports/distfiles mkdir -p ${NANO_WORLDDIR}/dev mount -t devfs devfs ${NANO_WORLDDIR}/dev mkdir -p ${NANO_WORLDDIR}/usr/workdir cp /etc/resolv.conf ${NANO_WORLDDIR}/etc/resolv.conf # OK, a little inefficient, but likely not enough to worry about. CR ldconfig /lib /usr/lib /usr/local/lib CR ldconfig -R CR ldconfig -r # Improvement: Don't know why package-recursive don't works here CR "env UNAME_p=${NANO_ARCH} TARGET=${NANO_ARCH} \ TARGET_ARCH=${NANO_ARCH} PORTSDIR=${NANO_PORTS} make \ __MAKE_CONF=${NANO_MAKE_CONF_BUILD} \ WRKDIRPREFIX=/usr/workdir -C /usr/ports/$port_path \ package-recursive BATCH=yes $* clean FORCE_PKG_REGISTER=t" rm ${NANO_WORLDDIR}/etc/resolv.conf rm -rf ${NANO_WORLDDIR}/usr/obj rm -rf ${NANO_WORLDDIR}/usr/workdir umount ${NANO_WORLDDIR}/dev umount ${NANO_WORLDDIR}/usr/ports/packages umount ${NANO_WORLDDIR}/usr/ports/distfiles umount ${NANO_WORLDDIR}/usr/ports umount ${NANO_WORLDDIR}/usr/src set +x } # Need to check if this function works with cross-compiling architecture!!!! # Recursive complex fonction: Generate one function for each ports add_port () { local port_path=$1 local port=`echo $1 | sed -e 's/\//_/'` shift - # Check if package allready exist + # Check if package already exist # Need to: # 1. check ARCH of this package! # 2. Add a trap cd ${NANO_PORTS}/${port_path} PKG_NAME=`env PORTSDIR=${NANO_PORTS} make __MAKE_CONF=${NANO_MAKE_CONF_BUILD} package-name` if [ -f ${NANO_OBJ}/ports/packages/All/${PKG_NAME}.txz ]; then # Pkg file found: Generate add_pkg_NAME function eval " add_pkg_${port} () { do_add_pkg ${PKG_NAME} } customize_cmd add_pkg_${port} " else # No pkg file: Generate add_port_NAME function eval " add_port_${port} () { do_add_port ${port_path} $* } customize_cmd add_port_${port} " NANO_PACKAGE_ONLY=0 fi } die() { echo "$*" exit 1 } # Automatically include the packaging port here so it is always first so it # builds the port and adds the package so we can add other packages. add_port ports-mgmt/pkg rp=$(realpath ${NANO_OBJ}/) __a=`mount | grep ${rp} | awk '{print length($3), $3;}' | sort -rn | awk '{$1=""; print;}'` if [ -n "$__a" ]; then echo "unmounting $__a" umount $__a fi NANO_BOOTLOADER="boot/boot0" Index: stable/10/tools/tools/shlib-compat/shlib-compat.py =================================================================== --- stable/10/tools/tools/shlib-compat/shlib-compat.py (revision 299825) +++ stable/10/tools/tools/shlib-compat/shlib-compat.py (revision 299826) @@ -1,1097 +1,1097 @@ #!/usr/bin/env python #- # Copyright (c) 2010 Gleb Kurtsou # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # $FreeBSD$ import os import sys import re import optparse class Config(object): version = '0.1' # controlled by user verbose = 0 dump = False no_dump = False version_filter = None symbol_filter = None alias_prefixes = [] # misc opts objdump = 'objdump' dwarfdump = 'dwarfdump' # debug cmpcache_enabled = True dwarfcache_enabled = True w_alias = True w_cached = False w_symbol = True class FileConfig(object): filename = None out = sys.stdout def init(self, outname): if outname and outname != '-': self.out = open(outname, "w") origfile = FileConfig() newfile = FileConfig() @classmethod def init(cls): cls.version_filter = StrFilter() cls.symbol_filter = StrFilter() class App(object): result_code = 0 def warn(cond, msg): if cond: print >> sys.stderr, "WARN: " + msg # {{{ misc class StrFilter(object): def __init__(self): self.exclude = [] self.include = [] def compile(self): self.re_exclude = [ re.compile(x) for x in self.exclude ] self.re_include = [ re.compile(x) for x in self.include ] def match(self, s): if len(self.re_include): matched = False for r in self.re_include: if r.match(s): matched = True break if not matched: return False for r in self.re_exclude: if r.match(s): return False return True class Cache(object): class CacheStats(object): def __init__(self): self.hit = 0 self.miss = 0 def show(self, name): total = self.hit + self.miss if total == 0: ratio = '(undef)' else: ratio = '%f' % (self.hit/float(total)) return '%s cache stats: hit: %d; miss: %d; ratio: %s' % \ (name, self.hit, self.miss, ratio) def __init__(self, enabled=True, stats=None): self.enabled = enabled self.items = {} if stats == None: self.stats = Cache.CacheStats() else: self.stats = stats def get(self, id): if self.enabled and self.items.has_key(id): self.stats.hit += 1 return self.items[id] else: self.stats.miss += 1 return None def put(self, id, obj): if self.enabled: if self.items.has_key(id) and obj is not self.items[id]: #raise ValueError("Item is already cached: %d (%s, %s)" % # (id, self.items[id], obj)) warn(Config.w_cached, "Item is already cached: %d (%s, %s)" % \ (id, self.items[id], obj)) self.items[id] = obj def replace(self, id, obj): if self.enabled: assert self.items.has_key(id) self.items[id] = obj class ListDiff(object): def __init__(self, orig, new): self.orig = set(orig) self.new = set(new) self.common = self.orig & self.new self.added = self.new - self.common self.removed = self.orig - self.common class PrettyPrinter(object): def __init__(self): self.stack = [] def run_nested(self, obj): ex = obj._pp_ex(self) self.stack.append(ex) def run(self, obj): self._result = obj._pp(self) return self._result def nested(self): return sorted(set(self.stack)) def result(self): return self._result; # }}} #{{{ symbols and version maps class Symbol(object): def __init__(self, name, offset, version, lib): self.name = name self.offset = offset self.version = version self.lib = lib self.definition = None @property def name_ver(self): return self.name + '@' + self.version def __repr__(self): return "Symbol(%s, 0x%x, %s)" % (self.name, self.offset, self.version) class CommonSymbol(object): def __init__(self, origsym, newsym): if origsym.name != newsym.name or origsym.version != newsym.version: raise RuntimeError("Symbols have different names: %s", [origsym, newsym]) self.origsym = origsym self.newsym = newsym self.name = newsym.name self.version = newsym.version def __repr__(self): return "CommonSymbol(%s, %s)" % (self.name, self.version) class SymbolAlias(object): def __init__(self, alias, prefix, offset): assert alias.startswith(prefix) self.alias = alias self.name = alias[len(prefix):] self.offset = offset def __repr__(self): return "SymbolAlias(%s, 0x%x)" % (self.alias, self.offset) class VersionMap(object): def __init__(self, name): self.name = name self.symbols = {} def append(self, symbol): if (self.symbols.has_key(symbol.name)): raise ValueError("Symbol is already defined %s@%s" % (symbol.name, self.name)) self.symbols[symbol.name] = symbol def names(self): return self.symbols.keys() def __repr__(self): return repr(self.symbols.values()) # }}} # {{{ types and definitions class Def(object): _is_alias = False def __init__(self, id, name, **kwargs): self.id = id self.name = name self.attrs = kwargs def __getattr__(self, attr): if not self.attrs.has_key(attr): raise AttributeError('%s in %s' % (attr, str(self))) return self.attrs[attr] def _name_opt(self, default=''): if not self.name: return default return self.name def _alias(self): if self._is_alias: return self.type._alias() return self def __cmp__(self, other): # TODO assert 'self' and 'other' belong to different libraries #print 'cmp defs: %s, %s' % (self, other) a = self._alias() try: b = other._alias() except AttributeError: return 1 r = cmp(a.__class__, b.__class__) if r == 0: if a.id != 0 and b.id != 0: ind = (long(a.id) << 32) + b.id r = Dwarf.cmpcache.get(ind) if r != None: return r else: ind = 0 r = cmp(a.attrs, b.attrs) if ind != 0: Dwarf.cmpcache.put(ind, r) else: r = 0 #raise RuntimeError('Comparing different classes: %s, %s' % # (a.__class__.__name__, b.__class__.__name__)) return r def __repr__(self): p = [] if hasattr(self, 'name'): p.append("name=%s" % self.name) for (k, v) in self.attrs.items(): if isinstance(v, Def): v = v.__class__.__name__ + '(...)' p.append("%s=%s" % (k, v)) return self.__class__.__name__ + '(' + ', '.join(p) + ')' def _mapval(self, param, vals): if param not in vals.keys(): raise NotImplementedError("Invalid value '%s': %s" % (param, str(self))) return vals[param] def _pp_ex(self, pp): raise NotImplementedError('Extended pretty print not implemeted: %s' % str(self)) def _pp(self, pp): raise NotImplementedError('Pretty print not implemeted: %s' % str(self)) class AnonymousDef(Def): def __init__(self, id, **kwargs): Def.__init__(self, id, None, **kwargs) class Void(AnonymousDef): _instance = None def __new__(cls, *args, **kwargs): if not cls._instance: cls._instance = super(Void, cls).__new__( cls, *args, **kwargs) return cls._instance def __init__(self): AnonymousDef.__init__(self, 0) def _pp(self, pp): return "void" class VarArgs(AnonymousDef): def _pp(self, pp): return "..." class PointerDef(AnonymousDef): def _pp(self, pp): t = pp.run(self.type) return "%s*" % (t,) class BaseTypeDef(Def): inttypes = ['DW_ATE_signed', 'DW_ATE_unsigned', 'DW_ATE_unsigned_char'] def _pp(self, pp): if self.encoding in self.inttypes: sign = '' if self.encoding == 'DW_ATE_signed' else 'u' bits = int(self.byte_size) * 8 return '%sint%s_t' % (sign, bits) elif self.encoding == 'DW_ATE_signed_char' and int(self.byte_size) == 1: return 'char'; elif self.encoding == 'DW_ATE_float': return self._mapval(self.byte_size, { '16': 'long double', '8': 'double', '4': 'float', }) raise NotImplementedError('Invalid encoding: %s' % self) class TypeAliasDef(Def): _is_alias = True def _pp(self, pp): alias = self._alias() # push typedef name if self.name and not alias.name: alias.name = 'T(%s)' % self.name # return type with modifiers return self.type._pp(pp) class EnumerationTypeDef(Def): def _pp(self, pp): return 'enum ' + self._name_opt('UNKNOWN') class ConstTypeDef(AnonymousDef): _is_alias = True def _pp(self, pp): return 'const ' + self.type._pp(pp) class VolatileTypeDef(AnonymousDef): _is_alias = True def _pp(self, pp): return 'volatile ' + self.type._pp(pp) class ArrayDef(AnonymousDef): def _pp(self, pp): t = pp.run(self.type) assert len(self.subranges) == 1 try: sz = int(self.subranges[0].upper_bound) + 1 except ValueError: s = re.sub(r'\(.+\)', '', self.subranges[0].upper_bound) sz = int(s) + 1 return '%s[%s]' % (t, sz) class ArraySubrangeDef(AnonymousDef): pass class FunctionDef(Def): def _pp(self, pp): result = pp.run(self.result) if not self.params: params = "void" else: params = ', '.join([ pp.run(x) for x in self.params ]) return "%s %s(%s);" % (result, self.name, params) class FunctionTypeDef(Def): def _pp(self, pp): result = pp.run(self.result) if not self.params: params = "void" else: params = ', '.join([ pp.run(x) for x in self.params ]) return "F(%s, %s, (%s))" % (self._name_opt(), result, params) class ParameterDef(Def): def _pp(self, pp): t = pp.run(self.type) return "%s %s" % (t, self._name_opt()) # TODO class StructForwardDef(Def): pass class IncompleteDef(Def): def update(self, complete, cache=None): self.complete = complete complete.incomplete = self if cache != None: cached = cache.get(self.id) if cached != None and isinstance(cached, IncompleteDef): cache.replace(self.id, complete) class StructIncompleteDef(IncompleteDef): def _pp(self, pp): return "struct %s" % (self.name,) class UnionIncompleteDef(IncompleteDef): def _pp(self, pp): return "union %s" % (self.name,) class StructDef(Def): def _pp_ex(self, pp, suffix=';'): members = [ pp.run(x) for x in self.members ] return "struct %s { %s }%s" % \ (self._name_opt(), ' '.join(members), suffix) def _pp(self, pp): if self.name: pp.run_nested(self) return "struct %s" % (self.name,) else: return self._pp_ex(pp, suffix='') class UnionDef(Def): def _pp_ex(self, pp, suffix=';'): members = [ pp.run(x) for x in self.members ] return "union %s { %s }%s" % \ (self._name_opt(), ' '.join(members), suffix) def _pp(self, pp): if self.name: pp.run_nested(self) return "union %s" % (self.name,) else: return self._pp_ex(pp, suffix='') class MemberDef(Def): def _pp(self, pp): t = pp.run(self.type) if self.bit_size: bits = ":%s" % self.bit_size else: bits = "" return "%s %s%s;" % (t, self._name_opt(), bits) class Dwarf(object): cmpcache = Cache(enabled=Config.cmpcache_enabled) def __init__(self, dump): self.dump = dump def _build_optarg_type(self, praw): type = praw.optarg('type', Void()) if type != Void(): type = self.buildref(praw.unit, type) return type def build_subprogram(self, raw): if raw.optname == None: raw.setname('SUBPROGRAM_NONAME_' + raw.arg('low_pc')); params = [ self.build(x) for x in raw.nested ] result = self._build_optarg_type(raw) return FunctionDef(raw.id, raw.name, params=params, result=result) def build_subroutine_type(self, raw): params = [ self.build(x) for x in raw.nested ] result = self._build_optarg_type(raw) return FunctionTypeDef(raw.id, raw.optname, params=params, result=result) def build_formal_parameter(self, raw): type = self._build_optarg_type(raw) return ParameterDef(raw.id, raw.optname, type=type) def build_pointer_type(self, raw): type = self._build_optarg_type(raw) return PointerDef(raw.id, type=type) def build_member(self, raw): type = self.buildref(raw.unit, raw.arg('type')) return MemberDef(raw.id, raw.name, type=type, bit_size=raw.optarg('bit_size', None)) def build_structure_type(self, raw): incomplete = raw.unit.incomplete.get(raw.id) if incomplete == None: incomplete = StructIncompleteDef(raw.id, raw.optname) raw.unit.incomplete.put(raw.id, incomplete) else: return incomplete members = [ self.build(x) for x in raw.nested ] byte_size = raw.optarg('byte_size', None) if byte_size == None: obj = StructForwardDef(raw.id, raw.name, members=members, forcename=raw.name) obj = StructDef(raw.id, raw.optname, members=members, byte_size=byte_size) incomplete.update(obj, cache=raw.unit.cache) return obj def build_union_type(self, raw): incomplete = raw.unit.incomplete.get(raw.id) if incomplete == None: incomplete = UnionIncompleteDef(raw.id, raw.optname) raw.unit.incomplete.put(raw.id, incomplete) else: return incomplete members = [ self.build(x) for x in raw.nested ] byte_size = raw.optarg('byte_size', None) obj = UnionDef(raw.id, raw.optname, members=members, byte_size=byte_size) obj.incomplete = incomplete incomplete.complete = obj return obj def build_typedef(self, raw): type = self._build_optarg_type(raw) return TypeAliasDef(raw.id, raw.name, type=type) def build_const_type(self, raw): type = self._build_optarg_type(raw) return ConstTypeDef(raw.id, type=type) def build_volatile_type(self, raw): type = self._build_optarg_type(raw) return VolatileTypeDef(raw.id, type=type) def build_enumeration_type(self, raw): # TODO handle DW_TAG_enumerator ??? return EnumerationTypeDef(raw.id, name=raw.optname, byte_size=raw.arg('byte_size')) def build_base_type(self, raw): return BaseTypeDef(raw.id, raw.optname, byte_size=raw.arg('byte_size'), encoding=raw.arg('encoding')) def build_array_type(self, raw): type = self.buildref(raw.unit, raw.arg('type')) subranges = [ self.build(x) for x in raw.nested ] return ArrayDef(raw.id, type=type, subranges=subranges) def build_subrange_type(self, raw): type = self.buildref(raw.unit, raw.arg('type')) return ArraySubrangeDef(raw.id, type=type, upper_bound=raw.optarg('upper_bound', 0)) def build_unspecified_parameters(self, raw): return VarArgs(raw.id) def _get_id(self, id): try: return int(id) except ValueError: if (id.startswith('<') and id.endswith('>')): return int(id[1:-1]) else: raise ValueError("Invalid dwarf id: %s" % id) def build(self, raw): obj = raw.unit.cache.get(raw.id) if obj != None: return obj builder_name = raw.tag.replace('DW_TAG_', 'build_') try: builder = getattr(self, builder_name) except AttributeError: raise AttributeError("Unknown dwarf tag: %s" % raw) obj = builder(raw) raw.unit.cache.put(obj.id, obj) return obj def buildref(self, unit, id): id = self._get_id(id) raw = unit.tags[id] obj = self.build(raw) return obj # }}} class Shlib(object): def __init__(self, libfile): self.libfile = libfile self.versions = {} self.alias_syms = {} def parse_objdump(self): objdump = ObjdumpParser(self.libfile) objdump.run() for p in objdump.dynamic_symbols: vername = p['ver'] if vername.startswith('(') and vername.endswith(')'): vername = vername[1:-1] if not Config.version_filter.match(vername): continue if not Config.symbol_filter.match(p['symbol']): continue sym = Symbol(p['symbol'], p['offset'], vername, self) if not self.versions.has_key(vername): self.versions[vername] = VersionMap(vername) self.versions[vername].append(sym) if Config.alias_prefixes: self.local_offsetmap = objdump.local_offsetmap for p in objdump.local_symbols: for prefix in Config.alias_prefixes: if not p['symbol'].startswith(prefix): continue alias = SymbolAlias(p['symbol'], prefix, p['offset']) if self.alias_syms.has_key(alias.name): prevalias = self.alias_syms[alias.name] if alias.name != prevalias.name or \ alias.offset != prevalias.offset: warn(Config.w_alias, "Symbol alias is " \ "already defined: %s: %s at %08x -- %s at %08x" % \ (alias.alias, alias.name, alias.offset, prevalias.name, prevalias.offset)) self.alias_syms[alias.name] = alias def parse_dwarfdump(self): dwarfdump = DwarfdumpParser(self.libfile) def lookup(sym): raw = None try: raw = dwarfdump.offsetmap[sym.offset] except: try: localnames = self.local_offsetmap[sym.offset] localnames.sort(key=lambda x: -len(x)) for localname in localnames: if not self.alias_syms.has_key(localname): continue alias = self.alias_syms[localname] raw = dwarfdump.offsetmap[alias.offset] break except: pass return raw dwarfdump.run() dwarf = Dwarf(dwarfdump) for ver in self.versions.values(): for sym in ver.symbols.values(): raw = lookup(sym); if not raw: warn(Config.w_symbol, "Symbol %s (%s) not found at offset 0x%x" % \ (sym.name_ver, self.libfile, sym.offset)) continue if Config.verbose >= 3: print "Parsing symbol %s (%s)" % (sym.name_ver, self.libfile) sym.definition = dwarf.build(raw) def parse(self): if not os.path.isfile(self.libfile): print >> sys.stderr, ("No such file: %s" % self.libfile) sys.exit(1) self.parse_objdump() self.parse_dwarfdump() # {{{ parsers class Parser(object): def __init__(self, proc): self.proc = proc self.parser = self.parse_begin def run(self): fd = os.popen(self.proc, 'r') while True: line = fd.readline() if (not line): break line = line.strip() if (line): self.parser(line) err = fd.close() if err: print >> sys.stderr, ("Execution failed: %s" % self.proc) sys.exit(2) def parse_begin(self, line): print(line) class ObjdumpParser(Parser): re_header = re.compile('(?P\w*)\s*SYMBOL TABLE:') re_local_symbol = re.compile('(?P[0-9a-fA-F]+)\s+(?P\w+)\s+(?P\w+)\s+(?P
[^\s]+)\s+(?P[0-9a-fA-F]+)\s*(?P[^\s]*)') re_lame_symbol = re.compile('(?P[0-9a-fA-F]+)\s+(?P\w+)\s+\*[A-Z]+\*') re_dynamic_symbol = re.compile('(?P[0-9a-fA-F]+)\s+(?P\w+)\s+(?P\w+)\s+(?P
[^\s]+)\s+(?P[0-9a-fA-F]+)\s*(?P[^\s]*)\s*(?P[^\s]*)') def __init__(self, libfile): Parser.__init__(self, "%s -wtT %s" % (Config.objdump, libfile)) self.dynamic_symbols = [] self.local_symbols = [] self.local_offsetmap = {} def parse_begin(self, line): self.parse_header(line) def add_symbol(self, table, symbol, offsetmap = None): offset = int(symbol['offset'], 16); symbol['offset'] = offset if (offset == 0): return table.append(symbol) if offsetmap != None: if not offsetmap.has_key(offset): offsetmap[offset] = [symbol['symbol']] else: offsetmap[offset].append(symbol['symbol']) def parse_header(self, line): m = self.re_header.match(line) if (m): table = m.group('table') if (table == "DYNAMIC"): self.parser = self.parse_dynamic elif table == '': self.parser = self.parse_local else: raise ValueError("Invalid symbol table: %s" % table) return True return False def parse_local(self, line): if (self.parse_header(line)): return if (self.re_lame_symbol.match(line)): return m = self.re_local_symbol.match(line) if (not m): return #raise ValueError("Invalid symbol definition: %s" % line) p = m.groupdict() if (p['symbol'] and p['symbol'].find('@') == -1): self.add_symbol(self.local_symbols, p, self.local_offsetmap); def parse_dynamic(self, line): if (self.parse_header(line)): return if (self.re_lame_symbol.match(line)): return m = self.re_dynamic_symbol.match(line) if (not m): raise ValueError("Invalid symbol definition: %s" % line) p = m.groupdict() if (p['symbol'] and p['ver']): self.add_symbol(self.dynamic_symbols, p); class DwarfdumpParser(Parser): tagcache_stats = Cache.CacheStats() class Unit(object): def __init__(self): self.cache = Cache(enabled=Config.dwarfcache_enabled, stats=DwarfdumpParser.tagcache_stats) self.incomplete = Cache() self.tags = {} class Tag(object): def __init__(self, unit, data): self.unit = unit self.id = int(data['id']) self.level = int(data['level']) self.tag = data['tag'] self.args = {} self.nested = [] @property def name(self): return self.arg('name') @property def optname(self): return self.optarg('name', None) def setname(self, name): self.args['DW_AT_name'] = name def arg(self, a): name = 'DW_AT_' + a try: return self.args[name] except KeyError: raise KeyError("Argument '%s' not found in %s: %s" % (name, self, self.args)) def optarg(self, a, default): try: return self.arg(a) except KeyError: return default def __repr__(self): return "Tag(%d, %d, %s)" % (self.level, self.id, self.tag) re_header = re.compile('<(?P\d+)><(?P\d+\+*\d*)><(?P\w+)>') re_argname = re.compile('(?P\w+)<') re_argunknown = re.compile('<[^<>]+>') skip_tags = set([ 'DW_TAG_lexical_block', 'DW_TAG_inlined_subroutine', 'DW_TAG_label', 'DW_TAG_variable', ]) def __init__(self, libfile): Parser.__init__(self, "%s -di %s" % (Config.dwarfdump, libfile)) self.current_unit = None self.offsetmap = {} self.stack = [] def parse_begin(self, line): if line == '.debug_info': self.parser = self.parse_debuginfo else: raise ValueError("Invalid dwarfdump header: %s" % line) def parse_argvalue(self, args): assert args.startswith('<') i = 1 cnt = 1 while i < len(args) and args[i]: if args[i] == '<': cnt += 1 elif args[i] == '>': cnt -= 1 if cnt == 0: break i = i + 1 value = args[1:i] args = args[i+1:] return (args, value) def parse_arg(self, tag, args): m = self.re_argname.match(args) if not m: m = self.re_argunknown.match(args) if not m: raise ValueError("Invalid dwarfdump: couldn't parse arguments: %s" % args) args = args[len(m.group(0)):].lstrip() return args argname = m.group('arg') args = args[len(argname):] value = [] while len(args) > 0 and args.startswith('<'): (args, v) = self.parse_argvalue(args) value.append(v) args = args.lstrip() if len(value) == 1: value = value[0] tag.args[argname] = value return args def parse_debuginfo(self, line): m = self.re_header.match(line) if not m: raise ValueError("Invalid dwarfdump: %s" % line) if m.group('level') == '0': self.current_unit = DwarfdumpParser.Unit() return tag = DwarfdumpParser.Tag(self.current_unit, m.groupdict()) args = line[len(m.group(0)):].lstrip() while args: args = self.parse_arg(tag, args) tag.unit.tags[tag.id] = tag if tag.args.has_key('DW_AT_low_pc') and \ tag.tag not in DwarfdumpParser.skip_tags: offset = int(tag.args['DW_AT_low_pc'], 16) if self.offsetmap.has_key(offset): raise ValueError("Dwarf dump parse error: " + - "symbol is aleady defined at offset 0x%x" % offset) + "symbol is already defined at offset 0x%x" % offset) self.offsetmap[offset] = tag if len(self.stack) > 0: prev = self.stack.pop() while prev.level >= tag.level and len(self.stack) > 0: prev = self.stack.pop() if prev.level < tag.level: assert prev.level == tag.level - 1 # TODO check DW_AT_sibling ??? if tag.tag not in DwarfdumpParser.skip_tags: prev.nested.append(tag) self.stack.append(prev) self.stack.append(tag) assert len(self.stack) == tag.level # }}} def list_str(l): l = [ str(x) for x in l ] l.sort() return ', '.join(l) def names_ver_str(vername, names): return list_str([ x + "@" + vername for x in names ]) def common_symbols(origlib, newlib): result = [] verdiff = ListDiff(origlib.versions.keys(), newlib.versions.keys()) if Config.verbose >= 1: print 'Original versions: ', list_str(verdiff.orig) print 'New versions: ', list_str(verdiff.new) for vername in verdiff.added: print 'Added version: ', vername print ' Added symbols: ', \ names_ver_str(vername, newlib.versions[vername].names()) for vername in verdiff.removed: print 'Removed version: ', vername print ' Removed symbols: ', \ names_ver_str(vername, origlib.versions[vername].names()) added = [] removed = [] for vername in verdiff.common: origver = origlib.versions[vername] newver = newlib.versions[vername] namediff = ListDiff(origver.names(), newver.names()) if namediff.added: added.append(names_ver_str(vername, namediff.added)) if namediff.removed: removed.append(names_ver_str(vername, namediff.removed)) commonver = VersionMap(vername) result.append(commonver) for n in namediff.common: sym = CommonSymbol(origver.symbols[n], newver.symbols[n]) commonver.append(sym) if added: print 'Added symbols:' for i in added: print ' ', i if removed: print 'Removed symbols:' for i in removed: print ' ', i return result def cmp_symbols(commonver): for ver in commonver: names = ver.names(); names.sort() for symname in names: sym = ver.symbols[symname] match = sym.origsym.definition == sym.newsym.definition if not match: App.result_code = 1 if Config.verbose >= 1 or not match: print '%s: definitions %smatch' % \ (sym.origsym.name_ver, "" if match else "mis") if Config.dump or (not match and not Config.no_dump): for x in [(sym.origsym, Config.origfile), (sym.newsym, Config.newfile)]: xsym = x[0] xout = x[1].out if not xsym.definition: print >> xout, '\n// Definition not found: %s %s' % \ (xsym.name_ver, xsym.lib.libfile) continue print >> xout, '\n// Definitions mismatch: %s %s' % \ (xsym.name_ver, xsym.lib.libfile) pp = PrettyPrinter() pp.run(xsym.definition) for i in pp.nested(): print >> xout, i print >> xout, pp.result() def dump_symbols(commonver): class SymbolDump(object): def __init__(self, io_conf): self.io_conf = io_conf self.pp = PrettyPrinter() self.res = [] def run(self, sym): r = self.pp.run(sym.definition) self.res.append('/* %s@%s */ %s' % (sym.name, sym.version, r)) def finish(self): print >> self.io_conf.out, '\n// Symbol dump: version %s, library %s' % \ (ver.name, self.io_conf.filename) for i in self.pp.nested(): print >> self.io_conf.out, i print >> self.io_conf.out, '' for i in self.res: print >> self.io_conf.out, i for ver in commonver: names = sorted(ver.names()); d_orig = SymbolDump(Config.origfile) d_new = SymbolDump(Config.newfile) for symname in names: sym = ver.symbols[symname] if not sym.origsym.definition or not sym.newsym.definition: # XXX warn(Config.w_symbol, 'Missing symbol definition: %s@%s' % \ (symname, ver.name)) continue d_orig.run(sym.origsym) d_new.run(sym.newsym) d_orig.finish() d_new.finish() if __name__ == '__main__': Config.init() parser = optparse.OptionParser(usage="usage: %prog origlib newlib", version="%prog " + Config.version) parser.add_option('-v', '--verbose', action='count', help="verbose mode, may be specified several times") parser.add_option('--alias-prefix', action='append', help="name prefix to try for symbol alias lookup", metavar="STR") parser.add_option('--dump', action='store_true', help="dump symbol definitions") parser.add_option('--no-dump', action='store_true', help="disable dump for mismatched symbols") parser.add_option('--out-orig', action='store', help="result output file for original library", metavar="ORIGFILE") parser.add_option('--out-new', action='store', help="result output file for new library", metavar="NEWFILE") parser.add_option('--exclude-ver', action='append', metavar="RE") parser.add_option('--include-ver', action='append', metavar="RE") parser.add_option('--exclude-sym', action='append', metavar="RE") parser.add_option('--include-sym', action='append', metavar="RE") for opt in ['alias', 'cached', 'symbol']: parser.add_option("--w-" + opt, action="store_true", dest="w_" + opt) parser.add_option("--w-no-" + opt, action="store_false", dest="w_" + opt) (opts, args) = parser.parse_args() if len(args) != 2: parser.print_help() sys.exit(-1) if opts.out_orig: Config.origfile.init(opts.out_orig) if opts.out_new: Config.newfile.init(opts.out_new) if opts.no_dump: Config.dump = False Config.no_dump = True if opts.dump: Config.dump = True Config.no_dump = False Config.verbose = 1 if opts.verbose: Config.verbose = opts.verbose if opts.alias_prefix: Config.alias_prefixes = opts.alias_prefix Config.alias_prefixes.sort(key=lambda x: -len(x)) for (k, v) in ({ '_sym': Config.symbol_filter, '_ver': Config.version_filter }).items(): for a in [ 'exclude', 'include' ]: opt = getattr(opts, a + k) if opt: getattr(v, a).extend(opt) Config.version_filter.compile() Config.symbol_filter.compile() for w in ['w_alias', 'w_cached', 'w_symbol']: if hasattr(opts, w): v = getattr(opts, w) if v != None: setattr(Config, w, v) (Config.origfile.filename, Config.newfile.filename) = (args[0], args[1]) origlib = Shlib(Config.origfile.filename) origlib.parse() newlib = Shlib(Config.newfile.filename) newlib.parse() commonver = common_symbols(origlib, newlib) if Config.dump: dump_symbols(commonver) cmp_symbols(commonver) if Config.verbose >= 4: print Dwarf.cmpcache.stats.show('Cmp') print DwarfdumpParser.tagcache_stats.show('Dwarf tag') sys.exit(App.result_code) Index: stable/10/tools/tools/wtap/vis_map/vis_map.c =================================================================== --- stable/10/tools/tools/wtap/vis_map/vis_map.c (revision 299825) +++ stable/10/tools/tools/wtap/vis_map/vis_map.c (revision 299826) @@ -1,117 +1,117 @@ /*- * Copyright (c) 2010-2011 Monthadar Al Jaberi, TerraNet AB * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * similar to the "NO WARRANTY" disclaimer below ("Disclaimer") and any * redistribution must be conditioned upon including a substantially * similar Disclaimer requirement for further binary redistribution. * * NO WARRANTY * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTIBILITY * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL * THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, * OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER * IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF * THE POSSIBILITY OF SUCH DAMAGES. * * $FreeBSD$ */ #include #include #include #include /* * From the driver itself */ #include static int dev = -1; static void toggle_medium(int op) { if (ioctl(dev, VISIOCTLOPEN, &op) < 0) { printf("error opening/closing medium\n"); } } static void link_op(struct link *l) { if (ioctl(dev, VISIOCTLLINK, l) < 0) { printf("error making a link operation\n"); } } static void usage(const char *argv[]) { printf("usage: %s [o | c | [ [a|d] wtap_id1 wtap_id2]]\n", argv[0]); } int main(int argc, const char* argv[]) { struct link l; char cmd; if (argc < 2) { usage(argv); exit(1); } dev = open("/dev/visctl", O_RDONLY); if (dev < 0) { printf("error opening visctl cdev\n"); exit(1); } cmd = (char)*argv[1]; switch (cmd) { case 'o': toggle_medium(1); break; case 'c': toggle_medium(0); break; case 'a': if (argc < 4) { usage(argv); exit(1); } l.op = 1; l.id1 = atoi(argv[2]); l.id2 = atoi(argv[3]); link_op(&l); break; case 'd': if (argc < 4) { usage(argv); exit(1); } l.op = 0; l.id1 = atoi(argv[2]); l.id2 = atoi(argv[3]); link_op(&l); break; default: - printf("wtap ioctl: unkown command '%c'\n", *argv[1]); + printf("wtap ioctl: unknown command '%c'\n", *argv[1]); exit(1); } exit(0); } Index: stable/10/tools/tools/wtap/wtap/wtap.c =================================================================== --- stable/10/tools/tools/wtap/wtap/wtap.c (revision 299825) +++ stable/10/tools/tools/wtap/wtap/wtap.c (revision 299826) @@ -1,82 +1,82 @@ /*- * Copyright (c) 2010-2011 Monthadar Al Jaberi, TerraNet AB * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * similar to the "NO WARRANTY" disclaimer below ("Disclaimer") and any * redistribution must be conditioned upon including a substantially * similar Disclaimer requirement for further binary redistribution. * * NO WARRANTY * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTIBILITY * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL * THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, * OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER * IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF * THE POSSIBILITY OF SUCH DAMAGES. * * $FreeBSD$ */ #include #include #include #include #include "if_wtapioctl.h" static int dev = -1; static void create(int id) { if(ioctl(dev, WTAPIOCTLCRT, &id) < 0){ printf("error creating wtap with id=%d\n", id); } } static void delete(int id) { if(ioctl(dev, WTAPIOCTLDEL, &id) < 0){ printf("error deleting wtap with id=%d\n", id); } } int main( int argc, const char* argv[]) { if(argc != 3){ printf("usage: %s [c | d] wtap_id\n", argv[0]); return -1; } int id = atoi(argv[2]); if(!(id >= 0 && id < 64)){ printf("wtap_id must be between 0 and 7\n"); return -1; } dev = open("/dev/wtapctl", O_RDONLY); if(dev < 0){ printf("error opening wtapctl cdev\n"); return -1; } switch((char)*argv[1]){ case 'c': create(id); break; case 'd': delete(id); break; default: - printf("wtap ioctl: unkown command '%c'\n", *argv[1]); + printf("wtap ioctl: unknown command '%c'\n", *argv[1]); return -1; } return 0; } Index: stable/10/usr.sbin/bluetooth/btpand/btpand.c =================================================================== --- stable/10/usr.sbin/bluetooth/btpand/btpand.c (revision 299825) +++ stable/10/usr.sbin/bluetooth/btpand/btpand.c (revision 299826) @@ -1,293 +1,293 @@ /* $NetBSD: btpand.c,v 1.1 2008/08/17 13:20:57 plunky Exp $ */ /*- * Copyright (c) 2008 Iain Hibbert * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* $FreeBSD$ */ #include __COPYRIGHT("@(#) Copyright (c) 2008 Iain Hibbert. All rights reserved."); __RCSID("$NetBSD: btpand.c,v 1.1 2008/08/17 13:20:57 plunky Exp $"); #include #include #include #include #include #include #include #include #include #include #include #include "btpand.h" /* global variables */ const char * control_path; /* -c */ const char * interface_name; /* -i */ const char * service_name; /* -s */ uint16_t service_class; bdaddr_t local_bdaddr; /* -d */ bdaddr_t remote_bdaddr; /* -a */ uint16_t l2cap_psm; /* -p */ int l2cap_mode; /* -m */ int server_limit; /* -n */ static const struct { const char * name; uint16_t class; const char * desc; } services[] = { { "PANU", SDP_SERVICE_CLASS_PANU, "Personal Area Networking User" }, - { "NAP", SDP_SERVICE_CLASS_NAP, "Network Acess Point" }, + { "NAP", SDP_SERVICE_CLASS_NAP, "Network Access Point" }, { "GN", SDP_SERVICE_CLASS_GN, "Group Network" }, }; static void main_exit(int); static void main_detach(void); static void usage(void); int main(int argc, char *argv[]) { unsigned long ul; char * ep; int ch, status; while ((ch = getopt(argc, argv, "a:c:d:i:l:m:p:S:s:")) != -1) { switch (ch) { case 'a': /* remote address */ if (!bt_aton(optarg, &remote_bdaddr)) { struct hostent *he; if ((he = bt_gethostbyname(optarg)) == NULL) errx(EXIT_FAILURE, "%s: %s", optarg, hstrerror(h_errno)); bdaddr_copy(&remote_bdaddr, (bdaddr_t *)he->h_addr); } break; case 'c': /* control socket path */ control_path = optarg; break; case 'd': /* local address */ if (!bt_devaddr(optarg, &local_bdaddr)) { struct hostent *he; if ((he = bt_gethostbyname(optarg)) == NULL) errx(EXIT_FAILURE, "%s: %s", optarg, hstrerror(h_errno)); bdaddr_copy(&local_bdaddr, (bdaddr_t *)he->h_addr); } break; case 'i': /* tap interface name */ if (strchr(optarg, '/') == NULL) { asprintf(&ep, "/dev/%s", optarg); interface_name = ep; } else interface_name = optarg; break; case 'l': /* limit server sessions */ ul = strtoul(optarg, &ep, 10); if (*optarg == '\0' || *ep != '\0' || ul == 0) errx(EXIT_FAILURE, "%s: invalid session limit", optarg); server_limit = ul; break; case 'm': /* link mode */ warnx("Setting link mode is not yet supported"); break; case 'p': /* protocol/service multiplexer */ ul = strtoul(optarg, &ep, 0); if (*optarg == '\0' || *ep != '\0' || ul > 0xffff || L2CAP_PSM_INVALID(ul)) errx(EXIT_FAILURE, "%s: invalid PSM", optarg); l2cap_psm = ul; break; case 's': /* service */ case 'S': /* service (no SDP) */ for (ul = 0; strcasecmp(optarg, services[ul].name); ul++) { if (ul == __arraycount(services)) errx(EXIT_FAILURE, "%s: unknown service", optarg); } if (ch == 's') service_name = services[ul].name; service_class = services[ul].class; break; default: usage(); /* NOTREACHED */ } } argc -= optind; argv += optind; /* validate options */ if (bdaddr_any(&local_bdaddr) || service_class == 0) usage(); if (!bdaddr_any(&remote_bdaddr) && (server_limit != 0 || control_path != 0 || (service_name != NULL && l2cap_psm != 0))) usage(); /* default options */ if (interface_name == NULL) interface_name = "/dev/tap"; if (l2cap_psm == 0) l2cap_psm = L2CAP_PSM_BNEP; if (bdaddr_any(&remote_bdaddr) && server_limit == 0) { if (service_class == SDP_SERVICE_CLASS_PANU) server_limit = 1; else server_limit = 7; } #ifdef L2CAP_LM_MASTER if (server_limit > 1 && service_class != SDP_SERVICE_CLASS_PANU) l2cap_mode |= L2CAP_LM_MASTER; #endif /* * fork() now so that the setup can be done in the child process * (as kqueue is not inherited) but block in the parent until the * setup is finished so we can return an error if necessary. */ switch(fork()) { case -1: /* bad */ err(EXIT_FAILURE, "fork() failed"); case 0: /* child */ signal(SIGPIPE, SIG_IGN); openlog(getprogname(), LOG_NDELAY | LOG_PERROR | LOG_PID, LOG_DAEMON); channel_init(); server_init(); event_init(); client_init(); tap_init(); main_detach(); event_dispatch(); break; default: /* parent */ signal(SIGUSR1, main_exit); wait(&status); if (WIFEXITED(status)) exit(WEXITSTATUS(status)); break; } err(EXIT_FAILURE, "exiting"); } static void main_exit(int s) { /* child is all grown up */ _exit(EXIT_SUCCESS); } static void main_detach(void) { int fd; if (kill(getppid(), SIGUSR1) == -1) log_err("Could not signal main process: %m"); if (setsid() == -1) log_err("setsid() failed"); fd = open(_PATH_DEVNULL, O_RDWR, 0); if (fd == -1) { log_err("Could not open %s", _PATH_DEVNULL); } else { (void)dup2(fd, STDIN_FILENO); (void)dup2(fd, STDOUT_FILENO); (void)dup2(fd, STDERR_FILENO); close(fd); } } static void usage(void) { const char *p = getprogname(); int n = strlen(p); fprintf(stderr, "usage: %s [-i ifname] [-m mode] -a address -d device\n" " %*s {-s service | -S service [-p psm]}\n" " %s [-c path] [-i ifname] [-l limit] [-m mode] [-p psm] -d device\n" " %*s {-s service | -S service}\n" "\n" "Where:\n" "\t-a address remote bluetooth device\n" "\t-c path SDP server socket\n" "\t-d device local bluetooth device\n" "\t-i ifname tap interface\n" "\t-l limit limit server sessions\n" "\t-m mode L2CAP link mode (NOT YET SUPPORTED)\n" "\t-p psm L2CAP PSM\n" "\t-S service service name (no SDP)\n" "\t-s service service name\n" "\n" "Known services:\n" "", p, n, "", p, n, ""); for (n = 0; n < __arraycount(services); n++) fprintf(stderr, "\t%s\t%s\n", services[n].name, services[n].desc); exit(EXIT_FAILURE); } Index: stable/10/usr.sbin/bluetooth/hccontrol/node.c =================================================================== --- stable/10/usr.sbin/bluetooth/hccontrol/node.c (revision 299825) +++ stable/10/usr.sbin/bluetooth/hccontrol/node.c (revision 299826) @@ -1,607 +1,607 @@ /* * node.c * * Copyright (c) 2001-2002 Maksim Yevmenkin * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $Id: node.c,v 1.6 2003/07/22 21:14:02 max Exp $ * $FreeBSD$ */ #include #include #include #include #include #include #include #include #include "hccontrol.h" /* Send Read_Node_State command to the node */ static int hci_read_node_state(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_state r; memset(&r, 0, sizeof(r)); if (ioctl(s, SIOC_HCI_RAW_NODE_GET_STATE, &r, sizeof(r)) < 0) return (ERROR); fprintf(stdout, "State: %#x\n", r.state); return (OK); } /* hci_read_node_state */ /* Send Intitialize command to the node */ static int hci_node_initialize(int s, int argc, char **argv) { if (ioctl(s, SIOC_HCI_RAW_NODE_INIT) < 0) return (ERROR); return (OK); } /* hci_node_initialize */ /* Send Read_Debug_Level command to the node */ static int hci_read_debug_level(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_debug r; memset(&r, 0, sizeof(r)); if (ioctl(s, SIOC_HCI_RAW_NODE_GET_DEBUG, &r, sizeof(r)) < 0) return (ERROR); fprintf(stdout, "Debug level: %d\n", r.debug); return (OK); } /* hci_read_debug_level */ /* Send Write_Debug_Level command to the node */ static int hci_write_debug_level(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_debug r; memset(&r, 0, sizeof(r)); switch (argc) { case 1: r.debug = atoi(argv[0]); break; default: return (USAGE); } if (ioctl(s, SIOC_HCI_RAW_NODE_SET_DEBUG, &r, sizeof(r)) < 0) return (ERROR); return (OK); } /* hci_write_debug_level */ /* Send Read_Node_Buffer_Size command to the node */ static int hci_read_node_buffer_size(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_buffer r; memset(&r, 0, sizeof(r)); if (ioctl(s, SIOC_HCI_RAW_NODE_GET_BUFFER, &r, sizeof(r)) < 0) return (ERROR); fprintf(stdout, "Number of free command buffers: %d\n", r.buffer.cmd_free); fprintf(stdout, "Max. ACL packet size: %d\n", r.buffer.acl_size); fprintf(stdout, "Numbef of free ACL buffers: %d\n", r.buffer.acl_free); fprintf(stdout, "Total number of ACL buffers: %d\n", r.buffer.acl_pkts); fprintf(stdout, "Max. SCO packet size: %d\n", r.buffer.sco_size); fprintf(stdout, "Numbef of free SCO buffers: %d\n", r.buffer.sco_free); fprintf(stdout, "Total number of SCO buffers: %d\n", r.buffer.sco_pkts); return (OK); } /* hci_read_node_buffer_size */ /* Send Read_Node_BD_ADDR command to the node */ static int hci_read_node_bd_addr(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_bdaddr r; memset(&r, 0, sizeof(r)); if (ioctl(s, SIOC_HCI_RAW_NODE_GET_BDADDR, &r, sizeof(r)) < 0) return (ERROR); fprintf(stdout, "BD_ADDR: %s\n", bt_ntoa(&r.bdaddr, NULL)); return (OK); } /* hci_read_node_bd_addr */ /* Send Read_Node_Features command to the node */ static int hci_read_node_features(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_features r; int n; char buffer[1024]; memset(&r, 0, sizeof(r)); if (ioctl(s, SIOC_HCI_RAW_NODE_GET_FEATURES, &r, sizeof(r)) < 0) return (ERROR); fprintf(stdout, "Features: "); for (n = 0; n < sizeof(r.features)/sizeof(r.features[0]); n++) fprintf(stdout, "%#02x ", r.features[n]); fprintf(stdout, "\n%s\n", hci_features2str(r.features, buffer, sizeof(buffer))); return (OK); } /* hci_read_node_features */ /* Send Read_Node_Stat command to the node */ static int hci_read_node_stat(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_stat r; memset(&r, 0, sizeof(r)); if (ioctl(s, SIOC_HCI_RAW_NODE_GET_STAT, &r, sizeof(r)) < 0) return (ERROR); fprintf(stdout, "Commands sent: %d\n", r.stat.cmd_sent); fprintf(stdout, "Events received: %d\n", r.stat.evnt_recv); fprintf(stdout, "ACL packets received: %d\n", r.stat.acl_recv); fprintf(stdout, "ACL packets sent: %d\n", r.stat.acl_sent); fprintf(stdout, "SCO packets received: %d\n", r.stat.sco_recv); fprintf(stdout, "SCO packets sent: %d\n", r.stat.sco_sent); fprintf(stdout, "Bytes received: %d\n", r.stat.bytes_recv); fprintf(stdout, "Bytes sent: %d\n", r.stat.bytes_sent); return (OK); } /* hci_read_node_stat */ /* Send Reset_Node_Stat command to the node */ static int hci_reset_node_stat(int s, int argc, char **argv) { if (ioctl(s, SIOC_HCI_RAW_NODE_RESET_STAT) < 0) return (ERROR); return (OK); } /* hci_reset_node_stat */ /* Send Flush_Neighbor_Cache command to the node */ static int hci_flush_neighbor_cache(int s, int argc, char **argv) { if (ioctl(s, SIOC_HCI_RAW_NODE_FLUSH_NEIGHBOR_CACHE) < 0) return (ERROR); return (OK); } /* hci_flush_neighbor_cache */ /* Send Read_Neighbor_Cache command to the node */ static int hci_read_neighbor_cache(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_neighbor_cache r; int n, error = OK; memset(&r, 0, sizeof(r)); r.num_entries = NG_HCI_MAX_NEIGHBOR_NUM; r.entries = calloc(NG_HCI_MAX_NEIGHBOR_NUM, sizeof(ng_hci_node_neighbor_cache_entry_ep)); if (r.entries == NULL) { errno = ENOMEM; return (ERROR); } if (ioctl(s, SIOC_HCI_RAW_NODE_GET_NEIGHBOR_CACHE, &r, sizeof(r)) < 0) { error = ERROR; goto out; } fprintf(stdout, "BD_ADDR " \ "Features " \ "Clock offset " \ "Page scan " \ "Rep. scan\n"); for (n = 0; n < r.num_entries; n++) { fprintf(stdout, "%-17.17s " \ "%02x %02x %02x %02x %02x %02x %02x %02x " \ "%#12x " \ "%#9x " \ "%#9x\n", hci_bdaddr2str(&r.entries[n].bdaddr), r.entries[n].features[0], r.entries[n].features[1], r.entries[n].features[2], r.entries[n].features[3], r.entries[n].features[4], r.entries[n].features[5], r.entries[n].features[6], r.entries[n].features[7], r.entries[n].clock_offset, r.entries[n].page_scan_mode, r.entries[n].page_scan_rep_mode); } out: free(r.entries); return (error); } /* hci_read_neightbor_cache */ /* Send Read_Connection_List command to the node */ static int hci_read_connection_list(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_con_list r; int n, error = OK; memset(&r, 0, sizeof(r)); r.num_connections = NG_HCI_MAX_CON_NUM; r.connections = calloc(NG_HCI_MAX_CON_NUM, sizeof(ng_hci_node_con_ep)); if (r.connections == NULL) { errno = ENOMEM; return (ERROR); } if (ioctl(s, SIOC_HCI_RAW_NODE_GET_CON_LIST, &r, sizeof(r)) < 0) { error = ERROR; goto out; } fprintf(stdout, "Remote BD_ADDR " \ "Handle " \ "Type " \ "Mode " \ "Role " \ "Encrypt " \ "Pending " \ "Queue " \ "State\n"); for (n = 0; n < r.num_connections; n++) { fprintf(stdout, "%-17.17s " \ "%6d " \ "%4.4s " \ "%4d " \ "%4.4s " \ "%7.7s " \ "%7d " \ "%5d " \ "%s\n", hci_bdaddr2str(&r.connections[n].bdaddr), r.connections[n].con_handle, (r.connections[n].link_type == NG_HCI_LINK_ACL)? "ACL" : "SCO", r.connections[n].mode, (r.connections[n].role == NG_HCI_ROLE_MASTER)? "MAST" : "SLAV", hci_encrypt2str(r.connections[n].encryption_mode, 1), r.connections[n].pending, r.connections[n].queue_len, hci_con_state2str(r.connections[n].state)); } out: free(r.connections); return (error); } /* hci_read_connection_list */ /* Send Read_Node_Link_Policy_Settings_Mask command to the node */ int hci_read_node_link_policy_settings_mask(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_link_policy_mask r; memset(&r, 0, sizeof(r)); if (ioctl(s, SIOC_HCI_RAW_NODE_GET_LINK_POLICY_MASK, &r, sizeof(r)) < 0) return (ERROR); fprintf(stdout, "Link Policy Settings mask: %#04x\n", r.policy_mask); return (OK); } /* hci_read_node_link_policy_settings_mask */ /* Send Write_Node_Link_Policy_Settings_Mask command to the node */ int hci_write_node_link_policy_settings_mask(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_link_policy_mask r; int m; memset(&r, 0, sizeof(r)); switch (argc) { case 1: if (sscanf(argv[0], "%x", &m) != 1) return (USAGE); r.policy_mask = (m & 0xffff); break; default: return (USAGE); } if (ioctl(s, SIOC_HCI_RAW_NODE_SET_LINK_POLICY_MASK, &r, sizeof(r)) < 0) return (ERROR); return (OK); } /* hci_write_node_link_policy_settings_mask */ /* Send Read_Node_Packet_Mask command to the node */ int hci_read_node_packet_mask(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_packet_mask r; memset(&r, 0, sizeof(r)); if (ioctl(s, SIOC_HCI_RAW_NODE_GET_PACKET_MASK, &r, sizeof(r)) < 0) return (ERROR); fprintf(stdout, "Packet mask: %#04x\n", r.packet_mask); return (OK); } /* hci_read_node_packet_mask */ /* Send Write_Node_Packet_Mask command to the node */ int hci_write_node_packet_mask(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_packet_mask r; int m; memset(&r, 0, sizeof(r)); switch (argc) { case 1: if (sscanf(argv[0], "%x", &m) != 1) return (USAGE); r.packet_mask = (m & 0xffff); break; default: return (USAGE); } if (ioctl(s, SIOC_HCI_RAW_NODE_SET_PACKET_MASK, &r, sizeof(r)) < 0) return (ERROR); return (OK); } /* hci_write_node_packet_mask */ /* Send Read_Node_Role_Switch command to the node */ int hci_read_node_role_switch(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_role_switch r; memset(&r, 0, sizeof(r)); if (ioctl(s, SIOC_HCI_RAW_NODE_GET_ROLE_SWITCH, &r, sizeof(r)) < 0) return (ERROR); fprintf(stdout, "Role switch: %d\n", r.role_switch); return (OK); } /* hci_read_node_role_switch */ /* Send Write_Node_Role_Switch command to the node */ int hci_write_node_role_switch(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_role_switch r; int m; memset(&r, 0, sizeof(r)); switch (argc) { case 1: if (sscanf(argv[0], "%d", &m) != 1) return (USAGE); r.role_switch = m? 1 : 0; break; default: return (USAGE); } if (ioctl(s, SIOC_HCI_RAW_NODE_SET_ROLE_SWITCH, &r, sizeof(r)) < 0) return (ERROR); return (OK); } /* hci_write_node_role_switch */ /* Send Read_Node_List command to the node */ int hci_read_node_list(int s, int argc, char **argv) { struct ng_btsocket_hci_raw_node_list_names r; int i; r.num_names = MAX_NODE_NUM; r.names = (struct nodeinfo*)calloc(MAX_NODE_NUM, sizeof(struct nodeinfo)); if (r.names == NULL) return (ERROR); if (ioctl(s, SIOC_HCI_RAW_NODE_LIST_NAMES, &r, sizeof(r)) < 0) { free(r.names); return (ERROR); } fprintf(stdout, "Name ID Num hooks\n"); for (i = 0; i < r.num_names; ++i) fprintf(stdout, "%-15s %08x %9d\n", r.names[i].name, r.names[i].id, r.names[i].hooks); free(r.names); return (OK); } /* hci_read_node_list */ struct hci_command node_commands[] = { { "read_node_state", "Get the HCI node state", &hci_read_node_state }, { "initialize", "Initialize the HCI node", &hci_node_initialize }, { "read_debug_level", "Read the HCI node debug level", &hci_read_debug_level }, { "write_debug_level ", "Write the HCI node debug level", &hci_write_debug_level }, { "read_node_buffer_size", "Read the HCI node buffer information. This will return current state of the\n"\ "HCI buffer for the HCI node", &hci_read_node_buffer_size }, { "read_node_bd_addr", "Read the HCI node BD_ADDR. Returns device BD_ADDR as cached by the HCI node", &hci_read_node_bd_addr }, { "read_node_features", "Read the HCI node features. This will return list of supported features as\n" \ "cached by the HCI node", &hci_read_node_features }, { "read_node_stat", "Read packets and bytes counters for the HCI node", &hci_read_node_stat }, { "reset_node_stat", "Reset packets and bytes counters for the HCI node", &hci_reset_node_stat }, { "flush_neighbor_cache", "Flush content of the HCI node neighbor cache", &hci_flush_neighbor_cache }, { "read_neighbor_cache", "Read content of the HCI node neighbor cache", &hci_read_neighbor_cache }, { "read_connection_list", "Read the baseband connection descriptors list for the HCI node", &hci_read_connection_list }, { "read_node_link_policy_settings_mask", "Read the value of the Link Policy Settinngs mask for the HCI node", &hci_read_node_link_policy_settings_mask }, { "write_node_link_policy_settings_mask ", "Write the value of the Link Policy Settings mask for the HCI node. By default\n" \ "all supported Link Policy modes (as reported by the local device features) are\n"\ "enabled. The particular Link Policy mode is enabled if local device supports\n"\ "it and correspinding bit in the mask was set\n\n" \ "\t - xxxx; Link Policy mask\n" \ "\t\t0x0000 - Disable All LM Modes\n" \ "\t\t0x0001 - Enable Master Slave Switch\n" \ "\t\t0x0002 - Enable Hold Mode\n" \ "\t\t0x0004 - Enable Sniff Mode\n" \ "\t\t0x0008 - Enable Park Mode\n", &hci_write_node_link_policy_settings_mask }, { "read_node_packet_mask", "Read the value of the Packet mask for the HCI node", &hci_read_node_packet_mask }, { "write_node_packet_mask ", "Write the value of the Packet mask for the HCI node. By default all supported\n" \ "packet types (as reported by the local device features) are enabled. The\n" \ "particular packet type is enabled if local device supports it and corresponding\n" \ "bit in the mask was set\n\n" \ "\t - xxxx; packet type mask\n" \ "" \ "\t\tACL packets\n" \ "\t\t-----------\n" \ "\t\t0x0008 DM1\n" \ "\t\t0x0010 DH1\n" \ "\t\t0x0400 DM3\n" \ "\t\t0x0800 DH3\n" \ "\t\t0x4000 DM5\n" \ "\t\t0x8000 DH5\n" \ "\n" \ "\t\tSCO packets\n" \ "\t\t-----------\n" \ "\t\t0x0020 HV1\n" \ "\t\t0x0040 HV2\n" \ "\t\t0x0080 HV3\n", &hci_write_node_packet_mask }, { "read_node_role_switch", "Read the value of the Role Switch parameter for the HCI node", &hci_read_node_role_switch }, { "write_node_role_switch {0|1}", "Write the value of the Role Switch parameter for the HCI node. By default,\n" \ "if Role Switch is supported, local device will try to perform Role Switch\n" \ "and become Master on incoming connection. Some devices do not support Role\n" \ -"Switch and thus incomming connections from such devices will fail. Setting\n" \ +"Switch and thus incoming connections from such devices will fail. Setting\n" \ "this parameter to zero will prevent Role Switch and thus accepting device\n" \ "will remain Slave", &hci_write_node_role_switch }, { "read_node_list", "Get a list of HCI nodes, their Netgraph IDs and connected hooks.", &hci_read_node_list }, { NULL, }}; Index: stable/10/usr.sbin/pmcstat/pmcpl_calltree.c =================================================================== --- stable/10/usr.sbin/pmcstat/pmcpl_calltree.c (revision 299825) +++ stable/10/usr.sbin/pmcstat/pmcpl_calltree.c (revision 299826) @@ -1,1197 +1,1197 @@ /*- * Copyright (c) 2012, Fabien Thomas * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * Process hwpmc(4) samples as calltree. * * Output file format compatible with Kcachegrind (kdesdk). * Handle top mode with a sorted tree display. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "pmcstat.h" #include "pmcstat_log.h" #include "pmcstat_top.h" #include "pmcpl_calltree.h" #define PMCPL_CT_GROWSIZE 4 static int pmcstat_skiplink = 0; struct pmcpl_ct_node; /* Get the sample value for PMC a. */ #define PMCPL_CT_SAMPLE(a, b) \ ((a) < (b)->npmcs ? (b)->sb[a] : 0) /* Get the sample value in percent related to rsamples. */ #define PMCPL_CT_SAMPLEP(a, b) \ (PMCPL_CT_SAMPLE(a, b) * 100.0 / rsamples->sb[a]) struct pmcpl_ct_sample { int npmcs; /* Max pmc index available. */ unsigned *sb; /* Sample buffer for 0..npmcs. */ }; struct pmcpl_ct_arc { struct pmcpl_ct_sample pcta_samples; struct pmcpl_ct_sample pcta_callid; unsigned pcta_call; struct pmcpl_ct_node *pcta_child; }; struct pmcpl_ct_instr { uintfptr_t pctf_func; struct pmcpl_ct_sample pctf_samples; }; /* * Each calltree node is tracked by a pmcpl_ct_node struct. */ struct pmcpl_ct_node { struct pmcstat_image *pct_image; uintfptr_t pct_func; struct pmcstat_symbol *pct_sym; pmcstat_interned_string pct_ifl; pmcstat_interned_string pct_ifn; struct pmcpl_ct_sample pct_samples; int pct_narc; int pct_arc_c; struct pmcpl_ct_arc *pct_arc; /* TODO: optimize for large number of items. */ int pct_ninstr; int pct_instr_c; struct pmcpl_ct_instr *pct_instr; #define PMCPL_PCT_ADDR 0 #define PMCPL_PCT_NAME 1 char pct_type; #define PMCPL_PCT_WHITE 0 #define PMCPL_PCT_GREY 1 #define PMCPL_PCT_BLACK 2 char pct_color; }; struct pmcpl_ct_node_hash { struct pmcpl_ct_node *pch_ctnode; STAILQ_ENTRY(pmcpl_ct_node_hash) pch_next; }; static struct pmcpl_ct_sample pmcpl_ct_callid; #define PMCPL_CT_MAXCOL PMC_CALLCHAIN_DEPTH_MAX #define PMCPL_CT_MAXLINE 1024 /* TODO: dynamic. */ struct pmcpl_ct_line { unsigned ln_sum; unsigned ln_index; }; static struct pmcpl_ct_line pmcpl_ct_topmax[PMCPL_CT_MAXLINE+1]; static struct pmcpl_ct_node *pmcpl_ct_topscreen[PMCPL_CT_MAXCOL+1][PMCPL_CT_MAXLINE+1]; /* * All nodes indexed by function/image name are placed in a hash table. */ static STAILQ_HEAD(,pmcpl_ct_node_hash) pmcpl_ct_node_hash[PMCSTAT_NHASH]; /* * Root node for the graph. */ static struct pmcpl_ct_node *pmcpl_ct_root; /* * Prototypes */ /* * Initialize a samples. */ static void pmcpl_ct_samples_init(struct pmcpl_ct_sample *samples) { samples->npmcs = 0; samples->sb = NULL; } /* * Free a samples. */ static void pmcpl_ct_samples_free(struct pmcpl_ct_sample *samples) { samples->npmcs = 0; free(samples->sb); samples->sb = NULL; } /* * Grow a sample block to store pmcstat_npmcs PMCs. */ static void pmcpl_ct_samples_grow(struct pmcpl_ct_sample *samples) { int npmcs; /* Enough storage. */ if (pmcstat_npmcs <= samples->npmcs) return; npmcs = samples->npmcs + max(pmcstat_npmcs - samples->npmcs, PMCPL_CT_GROWSIZE); samples->sb = realloc(samples->sb, npmcs * sizeof(unsigned)); if (samples->sb == NULL) errx(EX_SOFTWARE, "ERROR: out of memory"); bzero((char *)samples->sb + samples->npmcs * sizeof(unsigned), (npmcs - samples->npmcs) * sizeof(unsigned)); samples->npmcs = npmcs; } /* * Compute the sum of all root arcs. */ static void pmcpl_ct_samples_root(struct pmcpl_ct_sample *samples) { int i, pmcin; pmcpl_ct_samples_init(samples); pmcpl_ct_samples_grow(samples); for (i = 0; i < pmcpl_ct_root->pct_narc; i++) for (pmcin = 0; pmcin < pmcstat_npmcs; pmcin++) samples->sb[pmcin] += PMCPL_CT_SAMPLE(pmcin, &pmcpl_ct_root->pct_arc[i].pcta_samples); } /* * Grow the arc table. */ static void pmcpl_ct_arc_grow(int cursize, int *maxsize, struct pmcpl_ct_arc **items) { int nmaxsize; if (cursize < *maxsize) return; nmaxsize = *maxsize + max(cursize + 1 - *maxsize, PMCPL_CT_GROWSIZE); *items = realloc(*items, nmaxsize * sizeof(struct pmcpl_ct_arc)); if (*items == NULL) errx(EX_SOFTWARE, "ERROR: out of memory"); bzero((char *)*items + *maxsize * sizeof(struct pmcpl_ct_arc), (nmaxsize - *maxsize) * sizeof(struct pmcpl_ct_arc)); *maxsize = nmaxsize; } /* * Grow the instr table. */ static void pmcpl_ct_instr_grow(int cursize, int *maxsize, struct pmcpl_ct_instr **items) { int nmaxsize; if (cursize < *maxsize) return; nmaxsize = *maxsize + max(cursize + 1 - *maxsize, PMCPL_CT_GROWSIZE); *items = realloc(*items, nmaxsize * sizeof(struct pmcpl_ct_instr)); if (*items == NULL) errx(EX_SOFTWARE, "ERROR: out of memory"); bzero((char *)*items + *maxsize * sizeof(struct pmcpl_ct_instr), (nmaxsize - *maxsize) * sizeof(struct pmcpl_ct_instr)); *maxsize = nmaxsize; } /* * Add a new instruction sample to given node. */ static void pmcpl_ct_instr_add(struct pmcpl_ct_node *ct, int pmcin, uintfptr_t pc, unsigned v) { int i; struct pmcpl_ct_instr *in; for (i = 0; ipct_ninstr; i++) { if (ct->pct_instr[i].pctf_func == pc) { in = &ct->pct_instr[i]; pmcpl_ct_samples_grow(&in->pctf_samples); in->pctf_samples.sb[pmcin] += v; return; } } pmcpl_ct_instr_grow(ct->pct_ninstr, &ct->pct_instr_c, &ct->pct_instr); in = &ct->pct_instr[ct->pct_ninstr]; in->pctf_func = pc; pmcpl_ct_samples_init(&in->pctf_samples); pmcpl_ct_samples_grow(&in->pctf_samples); in->pctf_samples.sb[pmcin] = v; ct->pct_ninstr++; } /* * Allocate a new node. */ static struct pmcpl_ct_node * pmcpl_ct_node_allocate(void) { struct pmcpl_ct_node *ct; if ((ct = malloc(sizeof(*ct))) == NULL) err(EX_OSERR, "ERROR: Cannot allocate callgraph node"); pmcpl_ct_samples_init(&ct->pct_samples); ct->pct_sym = NULL; ct->pct_image = NULL; ct->pct_func = 0; ct->pct_narc = 0; ct->pct_arc_c = 0; ct->pct_arc = NULL; ct->pct_ninstr = 0; ct->pct_instr_c = 0; ct->pct_instr = NULL; ct->pct_color = PMCPL_PCT_WHITE; return (ct); } /* * Free a node. */ static void pmcpl_ct_node_free(struct pmcpl_ct_node *ct) { int i; for (i = 0; i < ct->pct_narc; i++) { pmcpl_ct_samples_free(&ct->pct_arc[i].pcta_samples); pmcpl_ct_samples_free(&ct->pct_arc[i].pcta_callid); } pmcpl_ct_samples_free(&ct->pct_samples); free(ct->pct_arc); free(ct->pct_instr); free(ct); } /* * Clear the graph tag on each node. */ static void pmcpl_ct_node_cleartag(void) { int i; struct pmcpl_ct_node_hash *pch; for (i = 0; i < PMCSTAT_NHASH; i++) STAILQ_FOREACH(pch, &pmcpl_ct_node_hash[i], pch_next) pch->pch_ctnode->pct_color = PMCPL_PCT_WHITE; pmcpl_ct_root->pct_color = PMCPL_PCT_WHITE; } /* * Print the callchain line by line with maximum cost at top. */ static int pmcpl_ct_node_dumptop(int pmcin, struct pmcpl_ct_node *ct, struct pmcpl_ct_sample *rsamples, int x, int *y) { int i, terminal; struct pmcpl_ct_arc *arc; if (ct->pct_color == PMCPL_PCT_GREY) return 0; if (x >= PMCPL_CT_MAXCOL) { pmcpl_ct_topscreen[x][*y] = NULL; return 1; } pmcpl_ct_topscreen[x][*y] = ct; /* * Check if this is a terminal node. * We need to check that some samples exist * for at least one arc for that PMC. */ terminal = 1; for (i = 0; i < ct->pct_narc; i++) { arc = &ct->pct_arc[i]; if (arc->pcta_child->pct_color != PMCPL_PCT_GREY && PMCPL_CT_SAMPLE(pmcin, &arc->pcta_samples) != 0 && PMCPL_CT_SAMPLEP(pmcin, &arc->pcta_samples) > pmcstat_threshold) { terminal = 0; break; } } if (ct->pct_narc == 0 || terminal) { pmcpl_ct_topscreen[x+1][*y] = NULL; if (*y >= PMCPL_CT_MAXLINE) return 1; *y = *y + 1; for (i=0; i < x; i++) pmcpl_ct_topscreen[i][*y] = pmcpl_ct_topscreen[i][*y - 1]; return 0; } ct->pct_color = PMCPL_PCT_GREY; for (i = 0; i < ct->pct_narc; i++) { if (PMCPL_CT_SAMPLE(pmcin, &ct->pct_arc[i].pcta_samples) == 0) continue; if (PMCPL_CT_SAMPLEP(pmcin, &ct->pct_arc[i].pcta_samples) > pmcstat_threshold) { if (pmcpl_ct_node_dumptop(pmcin, ct->pct_arc[i].pcta_child, rsamples, x+1, y)) { ct->pct_color = PMCPL_PCT_BLACK; return 1; } } } ct->pct_color = PMCPL_PCT_BLACK; return 0; } /* * Compare two top line by sum. */ static int pmcpl_ct_line_compare(const void *a, const void *b) { const struct pmcpl_ct_line *ct1, *ct2; ct1 = (const struct pmcpl_ct_line *) a; ct2 = (const struct pmcpl_ct_line *) b; /* Sort in reverse order */ if (ct1->ln_sum < ct2->ln_sum) return (1); if (ct1->ln_sum > ct2->ln_sum) return (-1); return (0); } /* * Format and display given PMC index. */ static void pmcpl_ct_node_printtop(struct pmcpl_ct_sample *rsamples, int pmcin, int maxy) { #undef TS #undef TSI #define TS(x, y) (pmcpl_ct_topscreen[x][y]) #define TSI(x, y) (pmcpl_ct_topscreen[x][pmcpl_ct_topmax[y].ln_index]) int v_attrs, ns_len, vs_len, is_len, width, indentwidth, x, y; float v; char ns[30], vs[10], is[20]; struct pmcpl_ct_node *ct; const char *space = " "; /* * Sort by line cost. */ for (y = 0; ; y++) { ct = TS(1, y); if (ct == NULL) break; pmcpl_ct_topmax[y].ln_sum = 0; pmcpl_ct_topmax[y].ln_index = y; for (x = 1; TS(x, y) != NULL; x++) { pmcpl_ct_topmax[y].ln_sum += PMCPL_CT_SAMPLE(pmcin, &TS(x, y)->pct_samples); } } qsort(pmcpl_ct_topmax, y, sizeof(pmcpl_ct_topmax[0]), pmcpl_ct_line_compare); pmcpl_ct_topmax[y].ln_index = y; for (y = 0; y < maxy; y++) { ct = TSI(1, y); if (ct == NULL) break; if (y > 0) PMCSTAT_PRINTW("\n"); /* Output sum. */ v = pmcpl_ct_topmax[y].ln_sum * 100.0 / rsamples->sb[pmcin]; snprintf(vs, sizeof(vs), "%.1f", v); v_attrs = PMCSTAT_ATTRPERCENT(v); PMCSTAT_ATTRON(v_attrs); PMCSTAT_PRINTW("%5.5s ", vs); PMCSTAT_ATTROFF(v_attrs); width = indentwidth = 5 + 1; for (x = 1; (ct = TSI(x, y)) != NULL; x++) { vs[0] = '\0'; vs_len = 0; is[0] = '\0'; is_len = 0; /* Format value. */ v = PMCPL_CT_SAMPLEP(pmcin, &ct->pct_samples); if (v > pmcstat_threshold) vs_len = snprintf(vs, sizeof(vs), "(%.1f%%)", v); v_attrs = PMCSTAT_ATTRPERCENT(v); if (pmcstat_skiplink && v <= pmcstat_threshold) { strlcpy(ns, ".", sizeof(ns)); ns_len = 1; } else { if (ct->pct_sym != NULL) { ns_len = snprintf(ns, sizeof(ns), "%s", pmcstat_string_unintern(ct->pct_sym->ps_name)); } else ns_len = snprintf(ns, sizeof(ns), "%p", (void *)ct->pct_func); /* Format image. */ if (x == 1 || TSI(x-1, y)->pct_image != ct->pct_image) is_len = snprintf(is, sizeof(is), "@%s", pmcstat_string_unintern(ct->pct_image->pi_name)); /* Check for line wrap. */ width += ns_len + is_len + vs_len + 1; } if (width >= pmcstat_displaywidth) { maxy--; if (y >= maxy) break; PMCSTAT_PRINTW("\n%*s", indentwidth, space); width = indentwidth + ns_len + is_len + vs_len; } PMCSTAT_ATTRON(v_attrs); PMCSTAT_PRINTW("%s%s%s ", ns, is, vs); PMCSTAT_ATTROFF(v_attrs); } } } /* * Output top mode snapshot. */ void pmcpl_ct_topdisplay(void) { int y; struct pmcpl_ct_sample r, *rsamples; rsamples = &r; pmcpl_ct_samples_root(rsamples); pmcpl_ct_node_cleartag(); PMCSTAT_PRINTW("%5.5s %s\n", "%SAMP", "CALLTREE"); y = 0; if (pmcpl_ct_node_dumptop(pmcstat_pmcinfilter, pmcpl_ct_root, rsamples, 0, &y)) PMCSTAT_PRINTW("...\n"); pmcpl_ct_topscreen[1][y] = NULL; pmcpl_ct_node_printtop(rsamples, pmcstat_pmcinfilter, pmcstat_displayheight - 2); pmcpl_ct_samples_free(rsamples); } /* * Handle top mode keypress. */ int pmcpl_ct_topkeypress(int c, WINDOW *w) { switch (c) { case 'f': pmcstat_skiplink = !pmcstat_skiplink; wprintw(w, "skip empty link %s", pmcstat_skiplink ? "on" : "off"); break; } return 0; } /* * Look for a callgraph node associated with pmc `pmcid' in the global * hash table that corresponds to the given `pc' value in the process map * `ppm'. */ static void pmcpl_ct_node_update(struct pmcpl_ct_node *parent, struct pmcpl_ct_node *child, int pmcin, unsigned v, int cd) { struct pmcpl_ct_arc *arc; int i; assert(parent != NULL); /* * Find related arc in parent node and * increment the sample count. */ for (i = 0; i < parent->pct_narc; i++) { if (parent->pct_arc[i].pcta_child == child) { arc = &parent->pct_arc[i]; pmcpl_ct_samples_grow(&arc->pcta_samples); arc->pcta_samples.sb[pmcin] += v; /* Estimate call count. */ if (cd) { pmcpl_ct_samples_grow(&arc->pcta_callid); if (pmcpl_ct_callid.sb[pmcin] - arc->pcta_callid.sb[pmcin] > 1) arc->pcta_call++; arc->pcta_callid.sb[pmcin] = pmcpl_ct_callid.sb[pmcin]; } return; } } /* * No arc found for us, add ourself to the parent. */ pmcpl_ct_arc_grow(parent->pct_narc, &parent->pct_arc_c, &parent->pct_arc); arc = &parent->pct_arc[parent->pct_narc]; pmcpl_ct_samples_grow(&arc->pcta_samples); arc->pcta_samples.sb[pmcin] = v; arc->pcta_call = 1; if (cd) { pmcpl_ct_samples_grow(&arc->pcta_callid); arc->pcta_callid.sb[pmcin] = pmcpl_ct_callid.sb[pmcin]; } arc->pcta_child = child; parent->pct_narc++; } /* * Lookup by image/pc. */ static struct pmcpl_ct_node * pmcpl_ct_node_hash_lookup(struct pmcstat_image *image, uintfptr_t pc, struct pmcstat_symbol *sym, char *fl, char *fn) { int i; unsigned int hash; struct pmcpl_ct_node *ct; struct pmcpl_ct_node_hash *h; pmcstat_interned_string ifl, ifn; if (fn != NULL) { ifl = pmcstat_string_intern(fl); ifn = pmcstat_string_intern(fn); } else { ifl = 0; ifn = 0; } for (hash = i = 0; i < (int)sizeof(uintfptr_t); i++) hash += (pc >> i) & 0xFF; hash &= PMCSTAT_HASH_MASK; STAILQ_FOREACH(h, &pmcpl_ct_node_hash[hash], pch_next) { ct = h->pch_ctnode; assert(ct != NULL); if (ct->pct_image == image && ct->pct_func == pc) { if (fn == NULL) return (ct); if (ct->pct_type == PMCPL_PCT_NAME && ct->pct_ifl == ifl && ct->pct_ifn == ifn) return (ct); } } /* * We haven't seen this (pmcid, pc) tuple yet, so allocate a * new callgraph node and a new hash table entry for it. */ ct = pmcpl_ct_node_allocate(); if ((h = malloc(sizeof(*h))) == NULL) err(EX_OSERR, "ERROR: Could not allocate callgraph node"); if (fn != NULL) { ct->pct_type = PMCPL_PCT_NAME; ct->pct_ifl = ifl; ct->pct_ifn = ifn; } else ct->pct_type = PMCPL_PCT_ADDR; ct->pct_image = image; ct->pct_func = pc; ct->pct_sym = sym; h->pch_ctnode = ct; STAILQ_INSERT_HEAD(&pmcpl_ct_node_hash[hash], h, pch_next); return (ct); } /* * Record a callchain. */ void pmcpl_ct_process(struct pmcstat_process *pp, struct pmcstat_pmcrecord *pmcr, uint32_t nsamples, uintfptr_t *cc, int usermode, uint32_t cpu) { int i, n, pmcin; uintfptr_t pc, loadaddress; struct pmcstat_image *image; struct pmcstat_symbol *sym; struct pmcstat_pcmap *ppm[PMC_CALLCHAIN_DEPTH_MAX]; struct pmcstat_process *km; struct pmcpl_ct_node *ct; struct pmcpl_ct_node *ctl[PMC_CALLCHAIN_DEPTH_MAX+1]; (void) cpu; assert(nsamples>0 && nsamples<=PMC_CALLCHAIN_DEPTH_MAX); /* Get the PMC index. */ pmcin = pmcr->pr_pmcin; /* * Validate mapping for the callchain. * Go from bottom to first invalid entry. */ km = pmcstat_kernproc; for (n = 0; n < (int)nsamples; n++) { ppm[n] = pmcstat_process_find_map(usermode ? pp : km, cc[n]); if (ppm[n] == NULL) { /* Detect full frame capture (kernel + user). */ if (!usermode) { ppm[n] = pmcstat_process_find_map(pp, cc[n]); if (ppm[n] != NULL) km = pp; } } if (ppm[n] == NULL) break; } if (n-- == 0) { pmcstat_stats.ps_callchain_dubious_frames++; pmcr->pr_dubious_frames++; return; } /* Increase the call generation counter. */ pmcpl_ct_samples_grow(&pmcpl_ct_callid); pmcpl_ct_callid.sb[pmcin]++; /* * Build node list. */ ctl[0] = pmcpl_ct_root; for (i = 1; n >= 0; n--) { image = ppm[n]->ppm_image; loadaddress = ppm[n]->ppm_lowpc + image->pi_vaddr - image->pi_start; /* Convert to an offset in the image. */ pc = cc[n] - loadaddress; /* * Try determine the function at this offset. If we can't * find a function round leave the `pc' value alone. */ if ((sym = pmcstat_symbol_search(image, pc)) != NULL) pc = sym->ps_start; else pmcstat_stats.ps_samples_unknown_function++; ct = pmcpl_ct_node_hash_lookup(image, pc, sym, NULL, NULL); if (ct == NULL) { pmcstat_stats.ps_callchain_dubious_frames++; continue; } ctl[i++] = ct; } /* No valid node found. */ if (i == 1) return; n = i; ct = ctl[0]; for (i = 1; i < n; i++) pmcpl_ct_node_update(ctl[i-1], ctl[i], pmcin, 1, 1); /* * Increment the sample count for this PMC. */ pmcpl_ct_samples_grow(&ctl[n-1]->pct_samples); ctl[n-1]->pct_samples.sb[pmcin]++; /* Update per instruction sample if required. */ if (args.pa_ctdumpinstr) pmcpl_ct_instr_add(ctl[n-1], pmcin, cc[0] - (ppm[0]->ppm_lowpc + ppm[0]->ppm_image->pi_vaddr - ppm[0]->ppm_image->pi_start), 1); } /* * Print node child cost. */ static void pmcpl_ct_node_printchild(struct pmcpl_ct_node *ct, uintfptr_t paddr, int pline) { int i, j, line; uintfptr_t addr; struct pmcpl_ct_node *child; char sourcefile[PATH_MAX]; char funcname[PATH_MAX]; /* * Child cost. - * TODO: attach child cost to the real position in the funtion. + * TODO: attach child cost to the real position in the function. * TODO: cfn= / call addr() / addr(call ) */ for (i=0 ; ipct_narc; i++) { child = ct->pct_arc[i].pcta_child; /* Object binary. */ fprintf(args.pa_graphfile, "cob=%s\n", pmcstat_string_unintern(child->pct_image->pi_fullpath)); /* Child function name. */ addr = child->pct_image->pi_vaddr + child->pct_func; line = 0; /* Child function source file. */ if (child->pct_type == PMCPL_PCT_NAME) { fprintf(args.pa_graphfile, "cfi=%s\ncfn=%s\n", pmcstat_string_unintern(child->pct_ifl), pmcstat_string_unintern(child->pct_ifn)); } else if (pmcstat_image_addr2line(child->pct_image, addr, sourcefile, sizeof(sourcefile), &line, funcname, sizeof(funcname))) { fprintf(args.pa_graphfile, "cfi=%s\ncfn=%s\n", sourcefile, funcname); } else { if (child->pct_sym != NULL) fprintf(args.pa_graphfile, "cfi=???\ncfn=%s\n", pmcstat_string_unintern( child->pct_sym->ps_name)); else fprintf(args.pa_graphfile, "cfi=???\ncfn=%p\n", (void *)addr); } /* Child function address, line and call count. */ fprintf(args.pa_graphfile, "calls=%u %p %u\n", ct->pct_arc[i].pcta_call, (void *)addr, line); /* * Call address, line, sample. * TODO: Associate call address to the right location. */ fprintf(args.pa_graphfile, "%p %u", (void *)paddr, pline); for (j = 0; jpct_arc[i].pcta_samples)); fprintf(args.pa_graphfile, "\n"); } } /* * Print node self cost. */ static void pmcpl_ct_node_printself(struct pmcpl_ct_node *ct) { int i, j, fline, line; uintfptr_t faddr, addr; char sourcefile[PATH_MAX]; char funcname[PATH_MAX]; /* * Object binary. */ fprintf(args.pa_graphfile, "ob=%s\n", pmcstat_string_unintern(ct->pct_image->pi_fullpath)); /* * Function name. */ faddr = ct->pct_image->pi_vaddr + ct->pct_func; fline = 0; if (ct->pct_type == PMCPL_PCT_NAME) { fprintf(args.pa_graphfile, "fl=%s\nfn=%s\n", pmcstat_string_unintern(ct->pct_ifl), pmcstat_string_unintern(ct->pct_ifn)); } else if (pmcstat_image_addr2line(ct->pct_image, faddr, sourcefile, sizeof(sourcefile), &fline, funcname, sizeof(funcname))) { fprintf(args.pa_graphfile, "fl=%s\nfn=%s\n", sourcefile, funcname); } else { if (ct->pct_sym != NULL) fprintf(args.pa_graphfile, "fl=???\nfn=%s\n", pmcstat_string_unintern(ct->pct_sym->ps_name)); else fprintf(args.pa_graphfile, "fl=???\nfn=%p\n", (void *)(ct->pct_image->pi_vaddr + ct->pct_func)); } /* * Self cost. */ if (ct->pct_ninstr > 0) { /* * Per location cost. */ for (i = 0; i < ct->pct_ninstr; i++) { addr = ct->pct_image->pi_vaddr + ct->pct_instr[i].pctf_func; line = 0; pmcstat_image_addr2line(ct->pct_image, addr, sourcefile, sizeof(sourcefile), &line, funcname, sizeof(funcname)); fprintf(args.pa_graphfile, "%p %u", (void *)addr, line); for (j = 0; jpct_instr[i].pctf_samples)); fprintf(args.pa_graphfile, "\n"); } } else { /* Global cost function cost. */ fprintf(args.pa_graphfile, "%p %u", (void *)faddr, fline); for (i = 0; ipct_samples)); fprintf(args.pa_graphfile, "\n"); } pmcpl_ct_node_printchild(ct, faddr, fline); } static void pmcpl_ct_printnode(struct pmcpl_ct_node *ct) { int i; if (ct == pmcpl_ct_root) { fprintf(args.pa_graphfile, "fn=root\n"); fprintf(args.pa_graphfile, "0x0 1"); for (i = 0; ipch_ctnode = ct; STAILQ_INSERT_TAIL(&q, pch, pch_next); ct->pct_color = PMCPL_PCT_BLACK; while (!STAILQ_EMPTY(&q)) { pch = STAILQ_FIRST(&q); STAILQ_REMOVE_HEAD(&q, pch_next); pmcpl_ct_printnode(pch->pch_ctnode); for (i = 0; ipch_ctnode->pct_narc; i++) { child = pch->pch_ctnode->pct_arc[i].pcta_child; if (child->pct_color == PMCPL_PCT_WHITE) { child->pct_color = PMCPL_PCT_BLACK; if ((pchc = malloc(sizeof(*pchc))) == NULL) err(EX_OSERR, "ERROR: Cannot allocate queue"); pchc->pch_ctnode = child; STAILQ_INSERT_TAIL(&q, pchc, pch_next); } } free(pch); } } /* * Detect and fix inlined location. */ static void _pmcpl_ct_expand_inline(struct pmcpl_ct_node *ct) { int i, j; unsigned fline, line, v; uintfptr_t faddr, addr, pc; char sourcefile[PATH_MAX]; char ffuncname[PATH_MAX], funcname[PATH_MAX]; char buffer[PATH_MAX]; struct pmcpl_ct_node *child; /* * Resolve parent and compare to each instr location. */ faddr = ct->pct_image->pi_vaddr + ct->pct_func; fline = 0; if (!pmcstat_image_addr2line(ct->pct_image, faddr, sourcefile, sizeof(sourcefile), &fline, ffuncname, sizeof(ffuncname))) return; for (i = 0; i < ct->pct_ninstr; i++) { addr = ct->pct_image->pi_vaddr + ct->pct_instr[i].pctf_func; line = 0; if (!pmcstat_image_addr2line(ct->pct_image, addr, sourcefile, sizeof(sourcefile), &line, funcname, sizeof(funcname))) continue; if (strcmp(funcname, ffuncname) == 0) continue; /* * - Lookup/create inline node by function name. * - Move instr PMCs to the inline node. * - Link nodes. * The lookup create a specific node per image/pc. */ if (args.pa_verbosity >= 2) fprintf(args.pa_printfile, "WARNING: inlined function at %p %s in %s\n", (void *)addr, funcname, ffuncname); snprintf(buffer, sizeof(buffer), "%s@%s", funcname, ffuncname); child = pmcpl_ct_node_hash_lookup(ct->pct_image, ct->pct_func, ct->pct_sym, sourcefile, buffer); assert(child != NULL); pc = ct->pct_instr[i].pctf_func; for (j = 0; jpct_instr[i].pctf_samples); if (v == 0) continue; pmcpl_ct_instr_add(child, j, pc, v); pmcpl_ct_node_update(ct, child, j, v, 0); if (j < ct->pct_samples.npmcs) ct->pct_samples.sb[j] -= ct->pct_instr[i].pctf_samples.sb[j]; ct->pct_instr[i].pctf_samples.sb[j] = 0; } } } static void pmcpl_ct_expand_inline(void) { int i; struct pmcpl_ct_node_hash *pch; if (!args.pa_ctdumpinstr) return; for (i = 0; i < PMCSTAT_NHASH; i++) STAILQ_FOREACH(pch, &pmcpl_ct_node_hash[i], pch_next) if (pch->pch_ctnode->pct_type == PMCPL_PCT_ADDR) _pmcpl_ct_expand_inline(pch->pch_ctnode); } /* * Clean the PMC name for Kcachegrind formula */ static void pmcpl_ct_fixup_pmcname(char *s) { char *p; for (p = s; *p; p++) if (!isalnum(*p)) *p = '_'; } /* * Print a calltree (KCachegrind) for all PMCs. */ static void pmcpl_ct_print(void) { int i; char name[40]; struct pmcpl_ct_sample rsamples; pmcpl_ct_samples_root(&rsamples); pmcpl_ct_expand_inline(); fprintf(args.pa_graphfile, "version: 1\n" "creator: pmcstat\n" "positions: instr line\n" "events:"); for (i=0; i #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "pmcstat.h" #include "pmcstat_log.h" #include "pmcstat_top.h" #define PMCSTAT_ALLOCATE 1 /* * PUBLIC INTERFACES * * pmcstat_initialize_logging() initialize this module, called first * pmcstat_shutdown_logging() orderly shutdown, called last * pmcstat_open_log() open an eventlog for processing * pmcstat_process_log() print/convert an event log * pmcstat_display_log() top mode display for the log * pmcstat_close_log() finish processing an event log * * IMPLEMENTATION NOTES * * We correlate each 'callchain' or 'sample' entry seen in the event * log back to an executable object in the system. Executable objects * include: * - program executables, * - shared libraries loaded by the runtime loader, * - dlopen()'ed objects loaded by the program, * - the runtime loader itself, * - the kernel and kernel modules. * * Each process that we know about is treated as a set of regions that * map to executable objects. Processes are described by * 'pmcstat_process' structures. Executable objects are tracked by * 'pmcstat_image' structures. The kernel and kernel modules are * common to all processes (they reside at the same virtual addresses * for all processes). Individual processes can have their text * segments and shared libraries loaded at process-specific locations. * * A given executable object can be in use by multiple processes * (e.g., libc.so) and loaded at a different address in each. * pmcstat_pcmap structures track per-image mappings. * * The sample log could have samples from multiple PMCs; we * generate one 'gmon.out' profile per PMC. * * IMPLEMENTATION OF GMON OUTPUT * * Each executable object gets one 'gmon.out' profile, per PMC in * use. Creation of 'gmon.out' profiles is done lazily. The * 'gmon.out' profiles generated for a given sampling PMC are * aggregates of all the samples for that particular executable * object. * * IMPLEMENTATION OF SYSTEM-WIDE CALLGRAPH OUTPUT * * Each active pmcid has its own callgraph structure, described by a * 'struct pmcstat_callgraph'. Given a process id and a list of pc * values, we map each pc value to a tuple (image, symbol), where * 'image' denotes an executable object and 'symbol' is the closest * symbol that precedes the pc value. Each pc value in the list is * also given a 'rank' that reflects its depth in the call stack. */ struct pmcstat_pmcs pmcstat_pmcs = LIST_HEAD_INITIALIZER(pmcstat_pmcs); /* * All image descriptors are kept in a hash table. */ struct pmcstat_image_hash_list pmcstat_image_hash[PMCSTAT_NHASH]; /* * All process descriptors are kept in a hash table. */ struct pmcstat_process_hash_list pmcstat_process_hash[PMCSTAT_NHASH]; struct pmcstat_stats pmcstat_stats; /* statistics */ static int ps_samples_period; /* samples count between top refresh. */ struct pmcstat_process *pmcstat_kernproc; /* kernel 'process' */ #include "pmcpl_gprof.h" #include "pmcpl_callgraph.h" #include "pmcpl_annotate.h" #include "pmcpl_annotate_cg.h" #include "pmcpl_calltree.h" static struct pmc_plugins { const char *pl_name; /* name */ /* configure */ int (*pl_configure)(char *opt); /* init and shutdown */ int (*pl_init)(void); void (*pl_shutdown)(FILE *mf); /* sample processing */ void (*pl_process)(struct pmcstat_process *pp, struct pmcstat_pmcrecord *pmcr, uint32_t nsamples, uintfptr_t *cc, int usermode, uint32_t cpu); /* image */ void (*pl_initimage)(struct pmcstat_image *pi); void (*pl_shutdownimage)(struct pmcstat_image *pi); /* pmc */ void (*pl_newpmc)(pmcstat_interned_string ps, struct pmcstat_pmcrecord *pr); /* top display */ void (*pl_topdisplay)(void); /* top keypress */ int (*pl_topkeypress)(int c, WINDOW *w); } plugins[] = { { .pl_name = "none", }, { .pl_name = "callgraph", .pl_init = pmcpl_cg_init, .pl_shutdown = pmcpl_cg_shutdown, .pl_process = pmcpl_cg_process, .pl_topkeypress = pmcpl_cg_topkeypress, .pl_topdisplay = pmcpl_cg_topdisplay }, { .pl_name = "gprof", .pl_shutdown = pmcpl_gmon_shutdown, .pl_process = pmcpl_gmon_process, .pl_initimage = pmcpl_gmon_initimage, .pl_shutdownimage = pmcpl_gmon_shutdownimage, .pl_newpmc = pmcpl_gmon_newpmc }, { .pl_name = "annotate", .pl_process = pmcpl_annotate_process }, { .pl_name = "calltree", .pl_configure = pmcpl_ct_configure, .pl_init = pmcpl_ct_init, .pl_shutdown = pmcpl_ct_shutdown, .pl_process = pmcpl_ct_process, .pl_topkeypress = pmcpl_ct_topkeypress, .pl_topdisplay = pmcpl_ct_topdisplay }, { .pl_name = "annotate_cg", .pl_process = pmcpl_annotate_cg_process }, { .pl_name = NULL } }; static int pmcstat_mergepmc; int pmcstat_pmcinfilter = 0; /* PMC filter for top mode. */ float pmcstat_threshold = 0.5; /* Cost filter for top mode. */ /* * Prototypes */ static struct pmcstat_image *pmcstat_image_from_path(pmcstat_interned_string _path, int _iskernelmodule); static void pmcstat_image_get_aout_params(struct pmcstat_image *_image); static void pmcstat_image_get_elf_params(struct pmcstat_image *_image); static void pmcstat_image_link(struct pmcstat_process *_pp, struct pmcstat_image *_i, uintfptr_t _lpc); static void pmcstat_pmcid_add(pmc_id_t _pmcid, pmcstat_interned_string _name); static void pmcstat_process_aout_exec(struct pmcstat_process *_pp, struct pmcstat_image *_image, uintfptr_t _entryaddr); static void pmcstat_process_elf_exec(struct pmcstat_process *_pp, struct pmcstat_image *_image, uintfptr_t _entryaddr); static void pmcstat_process_exec(struct pmcstat_process *_pp, pmcstat_interned_string _path, uintfptr_t _entryaddr); static struct pmcstat_process *pmcstat_process_lookup(pid_t _pid, int _allocate); static int pmcstat_string_compute_hash(const char *_string); static void pmcstat_string_initialize(void); static int pmcstat_string_lookup_hash(pmcstat_interned_string _is); static void pmcstat_string_shutdown(void); static void pmcstat_stats_reset(int _reset_global); /* * A simple implementation of interned strings. Each interned string * is assigned a unique address, so that subsequent string compares * can be done by a simple pointer comparison instead of using * strcmp(). This speeds up hash table lookups and saves memory if * duplicate strings are the norm. */ struct pmcstat_string { LIST_ENTRY(pmcstat_string) ps_next; /* hash link */ int ps_len; int ps_hash; char *ps_string; }; static LIST_HEAD(,pmcstat_string) pmcstat_string_hash[PMCSTAT_NHASH]; /* * PMC count. */ int pmcstat_npmcs; /* * PMC Top mode pause state. */ static int pmcstat_pause; static void pmcstat_stats_reset(int reset_global) { struct pmcstat_pmcrecord *pr; /* Flush PMCs stats. */ LIST_FOREACH(pr, &pmcstat_pmcs, pr_next) { pr->pr_samples = 0; pr->pr_dubious_frames = 0; } ps_samples_period = 0; /* Flush global stats. */ if (reset_global) bzero(&pmcstat_stats, sizeof(struct pmcstat_stats)); } /* * Compute a 'hash' value for a string. */ static int pmcstat_string_compute_hash(const char *s) { unsigned hash; for (hash = 2166136261; *s; s++) hash = (hash ^ *s) * 16777619; return (hash & PMCSTAT_HASH_MASK); } /* * Intern a copy of string 's', and return a pointer to the * interned structure. */ pmcstat_interned_string pmcstat_string_intern(const char *s) { struct pmcstat_string *ps; const struct pmcstat_string *cps; int hash, len; if ((cps = pmcstat_string_lookup(s)) != NULL) return (cps); hash = pmcstat_string_compute_hash(s); len = strlen(s); if ((ps = malloc(sizeof(*ps))) == NULL) err(EX_OSERR, "ERROR: Could not intern string"); ps->ps_len = len; ps->ps_hash = hash; ps->ps_string = strdup(s); LIST_INSERT_HEAD(&pmcstat_string_hash[hash], ps, ps_next); return ((pmcstat_interned_string) ps); } const char * pmcstat_string_unintern(pmcstat_interned_string str) { const char *s; s = ((const struct pmcstat_string *) str)->ps_string; return (s); } pmcstat_interned_string pmcstat_string_lookup(const char *s) { struct pmcstat_string *ps; int hash, len; hash = pmcstat_string_compute_hash(s); len = strlen(s); LIST_FOREACH(ps, &pmcstat_string_hash[hash], ps_next) if (ps->ps_len == len && ps->ps_hash == hash && strcmp(ps->ps_string, s) == 0) return (ps); return (NULL); } static int pmcstat_string_lookup_hash(pmcstat_interned_string s) { const struct pmcstat_string *ps; ps = (const struct pmcstat_string *) s; return (ps->ps_hash); } /* * Initialize the string interning facility. */ static void pmcstat_string_initialize(void) { int i; for (i = 0; i < PMCSTAT_NHASH; i++) LIST_INIT(&pmcstat_string_hash[i]); } /* * Destroy the string table, free'ing up space. */ static void pmcstat_string_shutdown(void) { int i; struct pmcstat_string *ps, *pstmp; for (i = 0; i < PMCSTAT_NHASH; i++) LIST_FOREACH_SAFE(ps, &pmcstat_string_hash[i], ps_next, pstmp) { LIST_REMOVE(ps, ps_next); free(ps->ps_string); free(ps); } } /* * Determine whether a given executable image is an A.OUT object, and * if so, fill in its parameters from the text file. * Sets image->pi_type. */ static void pmcstat_image_get_aout_params(struct pmcstat_image *image) { int fd; ssize_t nbytes; struct exec ex; const char *path; char buffer[PATH_MAX]; path = pmcstat_string_unintern(image->pi_execpath); assert(path != NULL); if (image->pi_iskernelmodule) errx(EX_SOFTWARE, "ERROR: a.out kernel modules are unsupported \"%s\"", path); (void) snprintf(buffer, sizeof(buffer), "%s%s", args.pa_fsroot, path); if ((fd = open(buffer, O_RDONLY, 0)) < 0 || (nbytes = read(fd, &ex, sizeof(ex))) < 0) { if (args.pa_verbosity >= 2) warn("WARNING: Cannot determine type of \"%s\"", path); image->pi_type = PMCSTAT_IMAGE_INDETERMINABLE; if (fd != -1) (void) close(fd); return; } (void) close(fd); if ((unsigned) nbytes != sizeof(ex) || N_BADMAG(ex)) return; image->pi_type = PMCSTAT_IMAGE_AOUT; /* TODO: the rest of a.out processing */ return; } /* * Helper function. */ static int pmcstat_symbol_compare(const void *a, const void *b) { const struct pmcstat_symbol *sym1, *sym2; sym1 = (const struct pmcstat_symbol *) a; sym2 = (const struct pmcstat_symbol *) b; if (sym1->ps_end <= sym2->ps_start) return (-1); if (sym1->ps_start >= sym2->ps_end) return (1); return (0); } /* * Map an address to a symbol in an image. */ struct pmcstat_symbol * pmcstat_symbol_search(struct pmcstat_image *image, uintfptr_t addr) { struct pmcstat_symbol sym; if (image->pi_symbols == NULL) return (NULL); sym.ps_name = NULL; sym.ps_start = addr; sym.ps_end = addr + 1; return (bsearch((void *) &sym, image->pi_symbols, image->pi_symcount, sizeof(struct pmcstat_symbol), pmcstat_symbol_compare)); } /* * Add the list of symbols in the given section to the list associated * with the object. */ static void pmcstat_image_add_symbols(struct pmcstat_image *image, Elf *e, Elf_Scn *scn, GElf_Shdr *sh) { int firsttime; size_t n, newsyms, nshsyms, nfuncsyms; struct pmcstat_symbol *symptr; char *fnname; GElf_Sym sym; Elf_Data *data; if ((data = elf_getdata(scn, NULL)) == NULL) return; /* * Determine the number of functions named in this * section. */ nshsyms = sh->sh_size / sh->sh_entsize; for (n = nfuncsyms = 0; n < nshsyms; n++) { if (gelf_getsym(data, (int) n, &sym) != &sym) return; if (GELF_ST_TYPE(sym.st_info) == STT_FUNC) nfuncsyms++; } if (nfuncsyms == 0) return; /* * Allocate space for the new entries. */ firsttime = image->pi_symbols == NULL; symptr = realloc(image->pi_symbols, sizeof(*symptr) * (image->pi_symcount + nfuncsyms)); if (symptr == image->pi_symbols) /* realloc() failed. */ return; image->pi_symbols = symptr; /* * Append new symbols to the end of the current table. */ symptr += image->pi_symcount; for (n = newsyms = 0; n < nshsyms; n++) { if (gelf_getsym(data, (int) n, &sym) != &sym) return; if (GELF_ST_TYPE(sym.st_info) != STT_FUNC) continue; if (sym.st_shndx == STN_UNDEF) continue; if (!firsttime && pmcstat_symbol_search(image, sym.st_value)) continue; /* We've seen this symbol already. */ if ((fnname = elf_strptr(e, sh->sh_link, sym.st_name)) == NULL) continue; #ifdef __arm__ /* Remove spurious ARM function name. */ if (fnname[0] == '$' && (fnname[1] == 'a' || fnname[1] == 't' || fnname[1] == 'd') && fnname[2] == '\0') continue; #endif symptr->ps_name = pmcstat_string_intern(fnname); symptr->ps_start = sym.st_value - image->pi_vaddr; symptr->ps_end = symptr->ps_start + sym.st_size; symptr++; newsyms++; } image->pi_symcount += newsyms; if (image->pi_symcount == 0) return; assert(newsyms <= nfuncsyms); /* * Return space to the system if there were duplicates. */ if (newsyms < nfuncsyms) image->pi_symbols = realloc(image->pi_symbols, sizeof(*symptr) * image->pi_symcount); /* * Keep the list of symbols sorted. */ qsort(image->pi_symbols, image->pi_symcount, sizeof(*symptr), pmcstat_symbol_compare); /* * Deal with function symbols that have a size of 'zero' by * making them extend to the next higher address. These * symbols are usually defined in assembly code. */ for (symptr = image->pi_symbols; symptr < image->pi_symbols + (image->pi_symcount - 1); symptr++) if (symptr->ps_start == symptr->ps_end) symptr->ps_end = (symptr+1)->ps_start; } /* * Examine an ELF file to determine the size of its text segment. * Sets image->pi_type if anything conclusive can be determined about * this image. */ static void pmcstat_image_get_elf_params(struct pmcstat_image *image) { int fd; size_t i, nph, nsh; const char *path, *elfbase; char *p, *endp; uintfptr_t minva, maxva; Elf *e; Elf_Scn *scn; GElf_Ehdr eh; GElf_Phdr ph; GElf_Shdr sh; enum pmcstat_image_type image_type; char buffer[PATH_MAX]; assert(image->pi_type == PMCSTAT_IMAGE_UNKNOWN); image->pi_start = minva = ~(uintfptr_t) 0; image->pi_end = maxva = (uintfptr_t) 0; image->pi_type = image_type = PMCSTAT_IMAGE_INDETERMINABLE; image->pi_isdynamic = 0; image->pi_dynlinkerpath = NULL; image->pi_vaddr = 0; path = pmcstat_string_unintern(image->pi_execpath); assert(path != NULL); /* * Look for kernel modules under FSROOT/KERNELPATH/NAME, * and user mode executable objects under FSROOT/PATHNAME. */ if (image->pi_iskernelmodule) (void) snprintf(buffer, sizeof(buffer), "%s%s/%s", args.pa_fsroot, args.pa_kernel, path); else (void) snprintf(buffer, sizeof(buffer), "%s%s", args.pa_fsroot, path); e = NULL; if ((fd = open(buffer, O_RDONLY, 0)) < 0 || (e = elf_begin(fd, ELF_C_READ, NULL)) == NULL || (elf_kind(e) != ELF_K_ELF)) { if (args.pa_verbosity >= 2) warnx("WARNING: Cannot determine the type of \"%s\".", buffer); goto done; } if (gelf_getehdr(e, &eh) != &eh) { warnx( "WARNING: Cannot retrieve the ELF Header for \"%s\": %s.", buffer, elf_errmsg(-1)); goto done; } if (eh.e_type != ET_EXEC && eh.e_type != ET_DYN && !(image->pi_iskernelmodule && eh.e_type == ET_REL)) { warnx("WARNING: \"%s\" is of an unsupported ELF type.", buffer); goto done; } image_type = eh.e_ident[EI_CLASS] == ELFCLASS32 ? PMCSTAT_IMAGE_ELF32 : PMCSTAT_IMAGE_ELF64; /* * Determine the virtual address where an executable would be * loaded. Additionally, for dynamically linked executables, * save the pathname to the runtime linker. */ if (eh.e_type == ET_EXEC) { if (elf_getphnum(e, &nph) == 0) { warnx( "WARNING: Could not determine the number of program headers in \"%s\": %s.", buffer, elf_errmsg(-1)); goto done; } for (i = 0; i < eh.e_phnum; i++) { if (gelf_getphdr(e, i, &ph) != &ph) { warnx( "WARNING: Retrieval of PHDR entry #%ju in \"%s\" failed: %s.", (uintmax_t) i, buffer, elf_errmsg(-1)); goto done; } switch (ph.p_type) { case PT_DYNAMIC: image->pi_isdynamic = 1; break; case PT_INTERP: if ((elfbase = elf_rawfile(e, NULL)) == NULL) { warnx( "WARNING: Cannot retrieve the interpreter for \"%s\": %s.", buffer, elf_errmsg(-1)); goto done; } image->pi_dynlinkerpath = pmcstat_string_intern(elfbase + ph.p_offset); break; case PT_LOAD: if ((ph.p_flags & PF_X) != 0 && (ph.p_offset & (-ph.p_align)) == 0) image->pi_vaddr = ph.p_vaddr & (-ph.p_align); break; } } } /* * Get the min and max VA associated with this ELF object. */ if (elf_getshnum(e, &nsh) == 0) { warnx( "WARNING: Could not determine the number of sections for \"%s\": %s.", buffer, elf_errmsg(-1)); goto done; } for (i = 0; i < nsh; i++) { if ((scn = elf_getscn(e, i)) == NULL || gelf_getshdr(scn, &sh) != &sh) { warnx( "WARNING: Could not retrieve section header #%ju in \"%s\": %s.", (uintmax_t) i, buffer, elf_errmsg(-1)); goto done; } if (sh.sh_flags & SHF_EXECINSTR) { minva = min(minva, sh.sh_addr); maxva = max(maxva, sh.sh_addr + sh.sh_size); } if (sh.sh_type == SHT_SYMTAB || sh.sh_type == SHT_DYNSYM) pmcstat_image_add_symbols(image, e, scn, &sh); } image->pi_start = minva; image->pi_end = maxva; image->pi_type = image_type; image->pi_fullpath = pmcstat_string_intern(buffer); /* Build display name */ endp = buffer; for (p = buffer; *p; p++) if (*p == '/') endp = p+1; image->pi_name = pmcstat_string_intern(endp); done: (void) elf_end(e); if (fd >= 0) (void) close(fd); return; } /* * Given an image descriptor, determine whether it is an ELF, or AOUT. * If no handler claims the image, set its type to 'INDETERMINABLE'. */ void pmcstat_image_determine_type(struct pmcstat_image *image) { assert(image->pi_type == PMCSTAT_IMAGE_UNKNOWN); /* Try each kind of handler in turn */ if (image->pi_type == PMCSTAT_IMAGE_UNKNOWN) pmcstat_image_get_elf_params(image); if (image->pi_type == PMCSTAT_IMAGE_UNKNOWN) pmcstat_image_get_aout_params(image); /* * Otherwise, remember that we tried to determine * the object's type and had failed. */ if (image->pi_type == PMCSTAT_IMAGE_UNKNOWN) image->pi_type = PMCSTAT_IMAGE_INDETERMINABLE; } /* * Locate an image descriptor given an interned path, adding a fresh * descriptor to the cache if necessary. This function also finds a * suitable name for this image's sample file. * * We defer filling in the file format specific parts of the image * structure till the time we actually see a sample that would fall * into this image. */ static struct pmcstat_image * pmcstat_image_from_path(pmcstat_interned_string internedpath, int iskernelmodule) { int hash; struct pmcstat_image *pi; hash = pmcstat_string_lookup_hash(internedpath); /* First, look for an existing entry. */ LIST_FOREACH(pi, &pmcstat_image_hash[hash], pi_next) if (pi->pi_execpath == internedpath && pi->pi_iskernelmodule == iskernelmodule) return (pi); /* * Allocate a new entry and place it at the head of the hash * and LRU lists. */ pi = malloc(sizeof(*pi)); if (pi == NULL) return (NULL); pi->pi_type = PMCSTAT_IMAGE_UNKNOWN; pi->pi_execpath = internedpath; pi->pi_start = ~0; pi->pi_end = 0; pi->pi_entry = 0; pi->pi_vaddr = 0; pi->pi_isdynamic = 0; pi->pi_iskernelmodule = iskernelmodule; pi->pi_dynlinkerpath = NULL; pi->pi_symbols = NULL; pi->pi_symcount = 0; pi->pi_addr2line = NULL; if (plugins[args.pa_pplugin].pl_initimage != NULL) plugins[args.pa_pplugin].pl_initimage(pi); if (plugins[args.pa_plugin].pl_initimage != NULL) plugins[args.pa_plugin].pl_initimage(pi); LIST_INSERT_HEAD(&pmcstat_image_hash[hash], pi, pi_next); return (pi); } /* * Record the fact that PC values from 'start' to 'end' come from * image 'image'. */ static void pmcstat_image_link(struct pmcstat_process *pp, struct pmcstat_image *image, uintfptr_t start) { struct pmcstat_pcmap *pcm, *pcmnew; uintfptr_t offset; assert(image->pi_type != PMCSTAT_IMAGE_UNKNOWN && image->pi_type != PMCSTAT_IMAGE_INDETERMINABLE); if ((pcmnew = malloc(sizeof(*pcmnew))) == NULL) err(EX_OSERR, "ERROR: Cannot create a map entry"); /* * Adjust the map entry to only cover the text portion * of the object. */ offset = start - image->pi_vaddr; pcmnew->ppm_lowpc = image->pi_start + offset; pcmnew->ppm_highpc = image->pi_end + offset; pcmnew->ppm_image = image; assert(pcmnew->ppm_lowpc < pcmnew->ppm_highpc); /* Overlapped mmap()'s are assumed to never occur. */ TAILQ_FOREACH(pcm, &pp->pp_map, ppm_next) if (pcm->ppm_lowpc >= pcmnew->ppm_highpc) break; if (pcm == NULL) TAILQ_INSERT_TAIL(&pp->pp_map, pcmnew, ppm_next); else TAILQ_INSERT_BEFORE(pcm, pcmnew, ppm_next); } /* * Unmap images in the range [start..end) associated with process * 'pp'. */ static void pmcstat_image_unmap(struct pmcstat_process *pp, uintfptr_t start, uintfptr_t end) { struct pmcstat_pcmap *pcm, *pcmtmp, *pcmnew; assert(pp != NULL); assert(start < end); /* * Cases: * - we could have the range completely in the middle of an * existing pcmap; in this case we have to split the pcmap * structure into two (i.e., generate a 'hole'). * - we could have the range covering multiple pcmaps; these * will have to be removed. * - we could have either 'start' or 'end' falling in the * middle of a pcmap; in this case shorten the entry. */ TAILQ_FOREACH_SAFE(pcm, &pp->pp_map, ppm_next, pcmtmp) { assert(pcm->ppm_lowpc < pcm->ppm_highpc); if (pcm->ppm_highpc <= start) continue; if (pcm->ppm_lowpc >= end) return; if (pcm->ppm_lowpc >= start && pcm->ppm_highpc <= end) { /* * The current pcmap is completely inside the * unmapped range: remove it entirely. */ TAILQ_REMOVE(&pp->pp_map, pcm, ppm_next); free(pcm); } else if (pcm->ppm_lowpc < start && pcm->ppm_highpc > end) { /* * Split this pcmap into two; curtail the * current map to end at [start-1], and start * the new one at [end]. */ if ((pcmnew = malloc(sizeof(*pcmnew))) == NULL) err(EX_OSERR, "ERROR: Cannot split a map entry"); pcmnew->ppm_image = pcm->ppm_image; pcmnew->ppm_lowpc = end; pcmnew->ppm_highpc = pcm->ppm_highpc; pcm->ppm_highpc = start; TAILQ_INSERT_AFTER(&pp->pp_map, pcm, pcmnew, ppm_next); return; } else if (pcm->ppm_lowpc < start && pcm->ppm_highpc <= end) pcm->ppm_highpc = start; else if (pcm->ppm_lowpc >= start && pcm->ppm_highpc > end) pcm->ppm_lowpc = end; else assert(0); } } /* * Resolve file name and line number for the given address. */ int pmcstat_image_addr2line(struct pmcstat_image *image, uintfptr_t addr, char *sourcefile, size_t sourcefile_len, unsigned *sourceline, char *funcname, size_t funcname_len) { static int addr2line_warn = 0; unsigned l; char *sep, cmdline[PATH_MAX], imagepath[PATH_MAX]; int fd; if (image->pi_addr2line == NULL) { snprintf(imagepath, sizeof(imagepath), "%s%s.symbols", args.pa_fsroot, pmcstat_string_unintern(image->pi_fullpath)); fd = open(imagepath, O_RDONLY); if (fd < 0) { snprintf(imagepath, sizeof(imagepath), "%s%s", args.pa_fsroot, pmcstat_string_unintern(image->pi_fullpath)); } else close(fd); /* * New addr2line support recursive inline function with -i * but the format does not add a marker when no more entries * are available. */ snprintf(cmdline, sizeof(cmdline), "addr2line -Cfe \"%s\"", imagepath); image->pi_addr2line = popen(cmdline, "r+"); if (image->pi_addr2line == NULL) { if (!addr2line_warn) { addr2line_warn = 1; warnx( "WARNING: addr2line is needed for source code information." ); } return (0); } } if (feof(image->pi_addr2line) || ferror(image->pi_addr2line)) { warnx("WARNING: addr2line pipe error"); pclose(image->pi_addr2line); image->pi_addr2line = NULL; return (0); } fprintf(image->pi_addr2line, "%p\n", (void *)addr); if (fgets(funcname, funcname_len, image->pi_addr2line) == NULL) { warnx("WARNING: addr2line function name read error"); return (0); } sep = strchr(funcname, '\n'); if (sep != NULL) *sep = '\0'; if (fgets(sourcefile, sourcefile_len, image->pi_addr2line) == NULL) { warnx("WARNING: addr2line source file read error"); return (0); } sep = strchr(sourcefile, ':'); if (sep == NULL) { warnx("WARNING: addr2line source line separator missing"); return (0); } *sep = '\0'; l = atoi(sep+1); if (l == 0) return (0); *sourceline = l; return (1); } /* * Add a {pmcid,name} mapping. */ static void pmcstat_pmcid_add(pmc_id_t pmcid, pmcstat_interned_string ps) { struct pmcstat_pmcrecord *pr, *prm; /* Replace an existing name for the PMC. */ prm = NULL; LIST_FOREACH(pr, &pmcstat_pmcs, pr_next) if (pr->pr_pmcid == pmcid) { pr->pr_pmcname = ps; return; } else if (pr->pr_pmcname == ps) prm = pr; /* * Otherwise, allocate a new descriptor and call the * plugins hook. */ if ((pr = malloc(sizeof(*pr))) == NULL) err(EX_OSERR, "ERROR: Cannot allocate pmc record"); pr->pr_pmcid = pmcid; pr->pr_pmcname = ps; pr->pr_pmcin = pmcstat_npmcs++; pr->pr_samples = 0; pr->pr_dubious_frames = 0; pr->pr_merge = prm == NULL ? pr : prm; LIST_INSERT_HEAD(&pmcstat_pmcs, pr, pr_next); if (plugins[args.pa_pplugin].pl_newpmc != NULL) plugins[args.pa_pplugin].pl_newpmc(ps, pr); if (plugins[args.pa_plugin].pl_newpmc != NULL) plugins[args.pa_plugin].pl_newpmc(ps, pr); } /* * Given a pmcid in use, find its human-readable name. */ const char * pmcstat_pmcid_to_name(pmc_id_t pmcid) { struct pmcstat_pmcrecord *pr; LIST_FOREACH(pr, &pmcstat_pmcs, pr_next) if (pr->pr_pmcid == pmcid) return (pmcstat_string_unintern(pr->pr_pmcname)); return NULL; } /* * Convert PMC index to name. */ const char * pmcstat_pmcindex_to_name(int pmcin) { struct pmcstat_pmcrecord *pr; LIST_FOREACH(pr, &pmcstat_pmcs, pr_next) if (pr->pr_pmcin == pmcin) return pmcstat_string_unintern(pr->pr_pmcname); return NULL; } /* * Return PMC record with given index. */ struct pmcstat_pmcrecord * pmcstat_pmcindex_to_pmcr(int pmcin) { struct pmcstat_pmcrecord *pr; LIST_FOREACH(pr, &pmcstat_pmcs, pr_next) if (pr->pr_pmcin == pmcin) return pr; return NULL; } /* * Get PMC record by id, apply merge policy. */ static struct pmcstat_pmcrecord * pmcstat_lookup_pmcid(pmc_id_t pmcid) { struct pmcstat_pmcrecord *pr; LIST_FOREACH(pr, &pmcstat_pmcs, pr_next) { if (pr->pr_pmcid == pmcid) { if (pmcstat_mergepmc) return pr->pr_merge; return pr; } } return NULL; } /* * Associate an AOUT image with a process. */ static void pmcstat_process_aout_exec(struct pmcstat_process *pp, struct pmcstat_image *image, uintfptr_t entryaddr) { (void) pp; (void) image; (void) entryaddr; /* TODO Implement a.out handling */ } /* * Associate an ELF image with a process. */ static void pmcstat_process_elf_exec(struct pmcstat_process *pp, struct pmcstat_image *image, uintfptr_t entryaddr) { uintmax_t libstart; struct pmcstat_image *rtldimage; assert(image->pi_type == PMCSTAT_IMAGE_ELF32 || image->pi_type == PMCSTAT_IMAGE_ELF64); /* Create a map entry for the base executable. */ pmcstat_image_link(pp, image, image->pi_vaddr); /* * For dynamically linked executables we need to determine * where the dynamic linker was mapped to for this process, * Subsequent executable objects that are mapped in by the * dynamic linker will be tracked by log events of type * PMCLOG_TYPE_MAP_IN. */ if (image->pi_isdynamic) { /* * The runtime loader gets loaded just after the maximum * possible heap address. Like so: * * [ TEXT DATA BSS HEAP -->*RTLD SHLIBS <--STACK] * ^ ^ * 0 VM_MAXUSER_ADDRESS * * The exact address where the loader gets mapped in * will vary according to the size of the executable * and the limits on the size of the process'es data * segment at the time of exec(). The entry address * recorded at process exec time corresponds to the * 'start' address inside the dynamic linker. From * this we can figure out the address where the * runtime loader's file object had been mapped to. */ rtldimage = pmcstat_image_from_path(image->pi_dynlinkerpath, 0); if (rtldimage == NULL) { warnx("WARNING: Cannot find image for \"%s\".", pmcstat_string_unintern(image->pi_dynlinkerpath)); pmcstat_stats.ps_exec_errors++; return; } if (rtldimage->pi_type == PMCSTAT_IMAGE_UNKNOWN) pmcstat_image_get_elf_params(rtldimage); if (rtldimage->pi_type != PMCSTAT_IMAGE_ELF32 && rtldimage->pi_type != PMCSTAT_IMAGE_ELF64) { warnx("WARNING: rtld not an ELF object \"%s\".", pmcstat_string_unintern(image->pi_dynlinkerpath)); return; } libstart = entryaddr - rtldimage->pi_entry; pmcstat_image_link(pp, rtldimage, libstart); } } /* * Find the process descriptor corresponding to a PID. If 'allocate' * is zero, we return a NULL if a pid descriptor could not be found or * a process descriptor process. If 'allocate' is non-zero, then we * will attempt to allocate a fresh process descriptor. Zombie * process descriptors are only removed if a fresh allocation for the * same PID is requested. */ static struct pmcstat_process * pmcstat_process_lookup(pid_t pid, int allocate) { uint32_t hash; struct pmcstat_pcmap *ppm, *ppmtmp; struct pmcstat_process *pp, *pptmp; hash = (uint32_t) pid & PMCSTAT_HASH_MASK; /* simplicity wins */ LIST_FOREACH_SAFE(pp, &pmcstat_process_hash[hash], pp_next, pptmp) if (pp->pp_pid == pid) { /* Found a descriptor, check and process zombies */ if (allocate && pp->pp_isactive == 0) { /* remove maps */ TAILQ_FOREACH_SAFE(ppm, &pp->pp_map, ppm_next, ppmtmp) { TAILQ_REMOVE(&pp->pp_map, ppm, ppm_next); free(ppm); } /* remove process entry */ LIST_REMOVE(pp, pp_next); free(pp); break; } return (pp); } if (!allocate) return (NULL); if ((pp = malloc(sizeof(*pp))) == NULL) err(EX_OSERR, "ERROR: Cannot allocate pid descriptor"); pp->pp_pid = pid; pp->pp_isactive = 1; TAILQ_INIT(&pp->pp_map); LIST_INSERT_HEAD(&pmcstat_process_hash[hash], pp, pp_next); return (pp); } /* * Associate an image and a process. */ static void pmcstat_process_exec(struct pmcstat_process *pp, pmcstat_interned_string path, uintfptr_t entryaddr) { struct pmcstat_image *image; if ((image = pmcstat_image_from_path(path, 0)) == NULL) { pmcstat_stats.ps_exec_errors++; return; } if (image->pi_type == PMCSTAT_IMAGE_UNKNOWN) pmcstat_image_determine_type(image); assert(image->pi_type != PMCSTAT_IMAGE_UNKNOWN); switch (image->pi_type) { case PMCSTAT_IMAGE_ELF32: case PMCSTAT_IMAGE_ELF64: pmcstat_stats.ps_exec_elf++; pmcstat_process_elf_exec(pp, image, entryaddr); break; case PMCSTAT_IMAGE_AOUT: pmcstat_stats.ps_exec_aout++; pmcstat_process_aout_exec(pp, image, entryaddr); break; case PMCSTAT_IMAGE_INDETERMINABLE: pmcstat_stats.ps_exec_indeterminable++; break; default: err(EX_SOFTWARE, "ERROR: Unsupported executable type for \"%s\"", pmcstat_string_unintern(path)); } } /* * Find the map entry associated with process 'p' at PC value 'pc'. */ struct pmcstat_pcmap * pmcstat_process_find_map(struct pmcstat_process *p, uintfptr_t pc) { struct pmcstat_pcmap *ppm; TAILQ_FOREACH(ppm, &p->pp_map, ppm_next) { if (pc >= ppm->ppm_lowpc && pc < ppm->ppm_highpc) return (ppm); if (pc < ppm->ppm_lowpc) return (NULL); } return (NULL); } /* * Convert a hwpmc(4) log to profile information. A system-wide * callgraph is generated if FLAG_DO_CALLGRAPHS is set. gmon.out * files usable by gprof(1) are created if FLAG_DO_GPROF is set. */ static int pmcstat_analyze_log(void) { uint32_t cpu, cpuflags; uintfptr_t pc; pid_t pid; struct pmcstat_image *image; struct pmcstat_process *pp, *ppnew; struct pmcstat_pcmap *ppm, *ppmtmp; struct pmclog_ev ev; struct pmcstat_pmcrecord *pmcr; pmcstat_interned_string image_path; assert(args.pa_flags & FLAG_DO_ANALYSIS); if (elf_version(EV_CURRENT) == EV_NONE) - err(EX_UNAVAILABLE, "Elf library intialization failed"); + err(EX_UNAVAILABLE, "Elf library initialization failed"); while (pmclog_read(args.pa_logparser, &ev) == 0) { assert(ev.pl_state == PMCLOG_OK); switch (ev.pl_type) { case PMCLOG_TYPE_INITIALIZE: if ((ev.pl_u.pl_i.pl_version & 0xFF000000) != PMC_VERSION_MAJOR << 24 && args.pa_verbosity > 0) warnx( "WARNING: Log version 0x%x does not match compiled version 0x%x.", ev.pl_u.pl_i.pl_version, PMC_VERSION_MAJOR); break; case PMCLOG_TYPE_MAP_IN: /* * Introduce an address range mapping for a * userland process or the kernel (pid == -1). * * We always allocate a process descriptor so * that subsequent samples seen for this * address range are mapped to the current * object being mapped in. */ pid = ev.pl_u.pl_mi.pl_pid; if (pid == -1) pp = pmcstat_kernproc; else pp = pmcstat_process_lookup(pid, PMCSTAT_ALLOCATE); assert(pp != NULL); image_path = pmcstat_string_intern(ev.pl_u.pl_mi. pl_pathname); image = pmcstat_image_from_path(image_path, pid == -1); if (image->pi_type == PMCSTAT_IMAGE_UNKNOWN) pmcstat_image_determine_type(image); if (image->pi_type != PMCSTAT_IMAGE_INDETERMINABLE) pmcstat_image_link(pp, image, ev.pl_u.pl_mi.pl_start); break; case PMCLOG_TYPE_MAP_OUT: /* * Remove an address map. */ pid = ev.pl_u.pl_mo.pl_pid; if (pid == -1) pp = pmcstat_kernproc; else pp = pmcstat_process_lookup(pid, 0); if (pp == NULL) /* unknown process */ break; pmcstat_image_unmap(pp, ev.pl_u.pl_mo.pl_start, ev.pl_u.pl_mo.pl_end); break; case PMCLOG_TYPE_PCSAMPLE: /* * Note: the `PCSAMPLE' log entry is not * generated by hpwmc(4) after version 2. */ /* * We bring in the gmon file for the image * currently associated with the PMC & pid * pair and increment the appropriate entry * bin inside this. */ pmcstat_stats.ps_samples_total++; ps_samples_period++; pc = ev.pl_u.pl_s.pl_pc; pp = pmcstat_process_lookup(ev.pl_u.pl_s.pl_pid, PMCSTAT_ALLOCATE); /* Get PMC record. */ pmcr = pmcstat_lookup_pmcid(ev.pl_u.pl_s.pl_pmcid); assert(pmcr != NULL); pmcr->pr_samples++; /* * Call the plugins processing * TODO: move pmcstat_process_find_map inside plugins */ if (plugins[args.pa_pplugin].pl_process != NULL) plugins[args.pa_pplugin].pl_process( pp, pmcr, 1, &pc, pmcstat_process_find_map(pp, pc) != NULL, 0); plugins[args.pa_plugin].pl_process( pp, pmcr, 1, &pc, pmcstat_process_find_map(pp, pc) != NULL, 0); break; case PMCLOG_TYPE_CALLCHAIN: pmcstat_stats.ps_samples_total++; ps_samples_period++; cpuflags = ev.pl_u.pl_cc.pl_cpuflags; cpu = PMC_CALLCHAIN_CPUFLAGS_TO_CPU(cpuflags); /* Filter on the CPU id. */ if (!CPU_ISSET(cpu, &(args.pa_cpumask))) { pmcstat_stats.ps_samples_skipped++; break; } pp = pmcstat_process_lookup(ev.pl_u.pl_cc.pl_pid, PMCSTAT_ALLOCATE); /* Get PMC record. */ pmcr = pmcstat_lookup_pmcid(ev.pl_u.pl_cc.pl_pmcid); assert(pmcr != NULL); pmcr->pr_samples++; /* * Call the plugins processing */ if (plugins[args.pa_pplugin].pl_process != NULL) plugins[args.pa_pplugin].pl_process( pp, pmcr, ev.pl_u.pl_cc.pl_npc, ev.pl_u.pl_cc.pl_pc, PMC_CALLCHAIN_CPUFLAGS_TO_USERMODE(cpuflags), cpu); plugins[args.pa_plugin].pl_process( pp, pmcr, ev.pl_u.pl_cc.pl_npc, ev.pl_u.pl_cc.pl_pc, PMC_CALLCHAIN_CPUFLAGS_TO_USERMODE(cpuflags), cpu); break; case PMCLOG_TYPE_PMCALLOCATE: /* * Record the association pmc id between this * PMC and its name. */ pmcstat_pmcid_add(ev.pl_u.pl_a.pl_pmcid, pmcstat_string_intern(ev.pl_u.pl_a.pl_evname)); break; case PMCLOG_TYPE_PMCALLOCATEDYN: /* * Record the association pmc id between this * PMC and its name. */ pmcstat_pmcid_add(ev.pl_u.pl_ad.pl_pmcid, pmcstat_string_intern(ev.pl_u.pl_ad.pl_evname)); break; case PMCLOG_TYPE_PROCEXEC: /* * Change the executable image associated with * a process. */ pp = pmcstat_process_lookup(ev.pl_u.pl_x.pl_pid, PMCSTAT_ALLOCATE); /* delete the current process map */ TAILQ_FOREACH_SAFE(ppm, &pp->pp_map, ppm_next, ppmtmp) { TAILQ_REMOVE(&pp->pp_map, ppm, ppm_next); free(ppm); } /* associate this process image */ image_path = pmcstat_string_intern( ev.pl_u.pl_x.pl_pathname); assert(image_path != NULL); pmcstat_process_exec(pp, image_path, ev.pl_u.pl_x.pl_entryaddr); break; case PMCLOG_TYPE_PROCEXIT: /* * Due to the way the log is generated, the * last few samples corresponding to a process * may appear in the log after the process * exit event is recorded. Thus we keep the * process' descriptor and associated data * structures around, but mark the process as * having exited. */ pp = pmcstat_process_lookup(ev.pl_u.pl_e.pl_pid, 0); if (pp == NULL) break; pp->pp_isactive = 0; /* mark as a zombie */ break; case PMCLOG_TYPE_SYSEXIT: pp = pmcstat_process_lookup(ev.pl_u.pl_se.pl_pid, 0); if (pp == NULL) break; pp->pp_isactive = 0; /* make a zombie */ break; case PMCLOG_TYPE_PROCFORK: /* * Allocate a process descriptor for the new * (child) process. */ ppnew = pmcstat_process_lookup(ev.pl_u.pl_f.pl_newpid, PMCSTAT_ALLOCATE); /* * If we had been tracking the parent, clone * its address maps. */ pp = pmcstat_process_lookup(ev.pl_u.pl_f.pl_oldpid, 0); if (pp == NULL) break; TAILQ_FOREACH(ppm, &pp->pp_map, ppm_next) pmcstat_image_link(ppnew, ppm->ppm_image, ppm->ppm_lowpc); break; default: /* other types of entries are not relevant */ break; } } if (ev.pl_state == PMCLOG_EOF) return (PMCSTAT_FINISHED); else if (ev.pl_state == PMCLOG_REQUIRE_DATA) return (PMCSTAT_RUNNING); err(EX_DATAERR, "ERROR: event parsing failed (record %jd, offset 0x%jx)", (uintmax_t) ev.pl_count + 1, ev.pl_offset); } /* * Print log entries as text. */ static int pmcstat_print_log(void) { struct pmclog_ev ev; uint32_t npc; while (pmclog_read(args.pa_logparser, &ev) == 0) { assert(ev.pl_state == PMCLOG_OK); switch (ev.pl_type) { case PMCLOG_TYPE_CALLCHAIN: PMCSTAT_PRINT_ENTRY("callchain", "%d 0x%x %d %d %c", ev.pl_u.pl_cc.pl_pid, ev.pl_u.pl_cc.pl_pmcid, PMC_CALLCHAIN_CPUFLAGS_TO_CPU(ev.pl_u.pl_cc. \ pl_cpuflags), ev.pl_u.pl_cc.pl_npc, PMC_CALLCHAIN_CPUFLAGS_TO_USERMODE(ev.pl_u.pl_cc.\ pl_cpuflags) ? 'u' : 's'); for (npc = 0; npc < ev.pl_u.pl_cc.pl_npc; npc++) PMCSTAT_PRINT_ENTRY("...", "%p", (void *) ev.pl_u.pl_cc.pl_pc[npc]); break; case PMCLOG_TYPE_CLOSELOG: PMCSTAT_PRINT_ENTRY("closelog",); break; case PMCLOG_TYPE_DROPNOTIFY: PMCSTAT_PRINT_ENTRY("drop",); break; case PMCLOG_TYPE_INITIALIZE: PMCSTAT_PRINT_ENTRY("initlog","0x%x \"%s\"", ev.pl_u.pl_i.pl_version, pmc_name_of_cputype(ev.pl_u.pl_i.pl_arch)); if ((ev.pl_u.pl_i.pl_version & 0xFF000000) != PMC_VERSION_MAJOR << 24 && args.pa_verbosity > 0) warnx( "WARNING: Log version 0x%x != expected version 0x%x.", ev.pl_u.pl_i.pl_version, PMC_VERSION); break; case PMCLOG_TYPE_MAP_IN: PMCSTAT_PRINT_ENTRY("map-in","%d %p \"%s\"", ev.pl_u.pl_mi.pl_pid, (void *) ev.pl_u.pl_mi.pl_start, ev.pl_u.pl_mi.pl_pathname); break; case PMCLOG_TYPE_MAP_OUT: PMCSTAT_PRINT_ENTRY("map-out","%d %p %p", ev.pl_u.pl_mo.pl_pid, (void *) ev.pl_u.pl_mo.pl_start, (void *) ev.pl_u.pl_mo.pl_end); break; case PMCLOG_TYPE_PCSAMPLE: PMCSTAT_PRINT_ENTRY("sample","0x%x %d %p %c", ev.pl_u.pl_s.pl_pmcid, ev.pl_u.pl_s.pl_pid, (void *) ev.pl_u.pl_s.pl_pc, ev.pl_u.pl_s.pl_usermode ? 'u' : 's'); break; case PMCLOG_TYPE_PMCALLOCATE: PMCSTAT_PRINT_ENTRY("allocate","0x%x \"%s\" 0x%x", ev.pl_u.pl_a.pl_pmcid, ev.pl_u.pl_a.pl_evname, ev.pl_u.pl_a.pl_flags); break; case PMCLOG_TYPE_PMCALLOCATEDYN: PMCSTAT_PRINT_ENTRY("allocatedyn","0x%x \"%s\" 0x%x", ev.pl_u.pl_ad.pl_pmcid, ev.pl_u.pl_ad.pl_evname, ev.pl_u.pl_ad.pl_flags); break; case PMCLOG_TYPE_PMCATTACH: PMCSTAT_PRINT_ENTRY("attach","0x%x %d \"%s\"", ev.pl_u.pl_t.pl_pmcid, ev.pl_u.pl_t.pl_pid, ev.pl_u.pl_t.pl_pathname); break; case PMCLOG_TYPE_PMCDETACH: PMCSTAT_PRINT_ENTRY("detach","0x%x %d", ev.pl_u.pl_d.pl_pmcid, ev.pl_u.pl_d.pl_pid); break; case PMCLOG_TYPE_PROCCSW: PMCSTAT_PRINT_ENTRY("cswval","0x%x %d %jd", ev.pl_u.pl_c.pl_pmcid, ev.pl_u.pl_c.pl_pid, ev.pl_u.pl_c.pl_value); break; case PMCLOG_TYPE_PROCEXEC: PMCSTAT_PRINT_ENTRY("exec","0x%x %d %p \"%s\"", ev.pl_u.pl_x.pl_pmcid, ev.pl_u.pl_x.pl_pid, (void *) ev.pl_u.pl_x.pl_entryaddr, ev.pl_u.pl_x.pl_pathname); break; case PMCLOG_TYPE_PROCEXIT: PMCSTAT_PRINT_ENTRY("exitval","0x%x %d %jd", ev.pl_u.pl_e.pl_pmcid, ev.pl_u.pl_e.pl_pid, ev.pl_u.pl_e.pl_value); break; case PMCLOG_TYPE_PROCFORK: PMCSTAT_PRINT_ENTRY("fork","%d %d", ev.pl_u.pl_f.pl_oldpid, ev.pl_u.pl_f.pl_newpid); break; case PMCLOG_TYPE_USERDATA: PMCSTAT_PRINT_ENTRY("userdata","0x%x", ev.pl_u.pl_u.pl_userdata); break; case PMCLOG_TYPE_SYSEXIT: PMCSTAT_PRINT_ENTRY("exit","%d", ev.pl_u.pl_se.pl_pid); break; default: fprintf(args.pa_printfile, "unknown event (type %d).\n", ev.pl_type); } } if (ev.pl_state == PMCLOG_EOF) return (PMCSTAT_FINISHED); else if (ev.pl_state == PMCLOG_REQUIRE_DATA) return (PMCSTAT_RUNNING); errx(EX_DATAERR, "ERROR: event parsing failed (record %jd, offset 0x%jx).", (uintmax_t) ev.pl_count + 1, ev.pl_offset); /*NOTREACHED*/ } /* * Public Interfaces. */ /* * Close a logfile, after first flushing all in-module queued data. */ int pmcstat_close_log(void) { /* If a local logfile is configured ask the kernel to stop * and flush data. Kernel will close the file when data is flushed * so keep the status to EXITING. */ if (args.pa_logfd != -1) { if (pmc_close_logfile() < 0) err(EX_OSERR, "ERROR: logging failed"); } return (args.pa_flags & FLAG_HAS_PIPE ? PMCSTAT_EXITING : PMCSTAT_FINISHED); } /* * Open a log file, for reading or writing. * * The function returns the fd of a successfully opened log or -1 in * case of failure. */ int pmcstat_open_log(const char *path, int mode) { int error, fd, cfd; size_t hlen; const char *p, *errstr; struct addrinfo hints, *res, *res0; char hostname[MAXHOSTNAMELEN]; errstr = NULL; fd = -1; /* * If 'path' is "-" then open one of stdin or stdout depending * on the value of 'mode'. * * If 'path' contains a ':' and does not start with a '/' or '.', * and is being opened for writing, treat it as a "host:port" * specification and open a network socket. * * Otherwise, treat 'path' as a file name and open that. */ if (path[0] == '-' && path[1] == '\0') fd = (mode == PMCSTAT_OPEN_FOR_READ) ? 0 : 1; else if (path[0] != '/' && path[0] != '.' && strchr(path, ':') != NULL) { p = strrchr(path, ':'); hlen = p - path; if (p == path || hlen >= sizeof(hostname)) { errstr = strerror(EINVAL); goto done; } assert(hlen < sizeof(hostname)); (void) strncpy(hostname, path, hlen); hostname[hlen] = '\0'; (void) memset(&hints, 0, sizeof(hints)); hints.ai_family = AF_UNSPEC; hints.ai_socktype = SOCK_STREAM; if ((error = getaddrinfo(hostname, p+1, &hints, &res0)) != 0) { errstr = gai_strerror(error); goto done; } fd = -1; for (res = res0; res; res = res->ai_next) { if ((fd = socket(res->ai_family, res->ai_socktype, res->ai_protocol)) < 0) { errstr = strerror(errno); continue; } if (mode == PMCSTAT_OPEN_FOR_READ) { if (bind(fd, res->ai_addr, res->ai_addrlen) < 0) { errstr = strerror(errno); (void) close(fd); fd = -1; continue; } listen(fd, 1); cfd = accept(fd, NULL, NULL); (void) close(fd); if (cfd < 0) { errstr = strerror(errno); fd = -1; break; } fd = cfd; } else { if (connect(fd, res->ai_addr, res->ai_addrlen) < 0) { errstr = strerror(errno); (void) close(fd); fd = -1; continue; } } errstr = NULL; break; } freeaddrinfo(res0); } else if ((fd = open(path, mode == PMCSTAT_OPEN_FOR_READ ? O_RDONLY : (O_WRONLY|O_CREAT|O_TRUNC), S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH)) < 0) errstr = strerror(errno); done: if (errstr) errx(EX_OSERR, "ERROR: Cannot open \"%s\" for %s: %s.", path, (mode == PMCSTAT_OPEN_FOR_READ ? "reading" : "writing"), errstr); return (fd); } /* * Process a log file in offline analysis mode. */ int pmcstat_process_log(void) { /* * If analysis has not been asked for, just print the log to * the current output file. */ if (args.pa_flags & FLAG_DO_PRINT) return (pmcstat_print_log()); else return (pmcstat_analyze_log()); } /* * Refresh top display. */ static void pmcstat_refresh_top(void) { int v_attrs; float v; char pmcname[40]; struct pmcstat_pmcrecord *pmcpr; /* If in pause mode do not refresh display. */ if (pmcstat_pause) return; /* Wait until PMC pop in the log. */ pmcpr = pmcstat_pmcindex_to_pmcr(pmcstat_pmcinfilter); if (pmcpr == NULL) return; /* Format PMC name. */ if (pmcstat_mergepmc) snprintf(pmcname, sizeof(pmcname), "[%s]", pmcstat_string_unintern(pmcpr->pr_pmcname)); else snprintf(pmcname, sizeof(pmcname), "%s.%d", pmcstat_string_unintern(pmcpr->pr_pmcname), pmcstat_pmcinfilter); /* Format samples count. */ if (ps_samples_period > 0) v = (pmcpr->pr_samples * 100.0) / ps_samples_period; else v = 0.; v_attrs = PMCSTAT_ATTRPERCENT(v); PMCSTAT_PRINTBEGIN(); PMCSTAT_PRINTW("PMC: %s Samples: %u ", pmcname, pmcpr->pr_samples); PMCSTAT_ATTRON(v_attrs); PMCSTAT_PRINTW("(%.1f%%) ", v); PMCSTAT_ATTROFF(v_attrs); PMCSTAT_PRINTW(", %u unresolved\n\n", pmcpr->pr_dubious_frames); if (plugins[args.pa_plugin].pl_topdisplay != NULL) plugins[args.pa_plugin].pl_topdisplay(); PMCSTAT_PRINTEND(); } /* * Find the next pmc index to display. */ static void pmcstat_changefilter(void) { int pmcin; struct pmcstat_pmcrecord *pmcr; /* * Find the next merge target. */ if (pmcstat_mergepmc) { pmcin = pmcstat_pmcinfilter; do { pmcr = pmcstat_pmcindex_to_pmcr(pmcstat_pmcinfilter); if (pmcr == NULL || pmcr == pmcr->pr_merge) break; pmcstat_pmcinfilter++; if (pmcstat_pmcinfilter >= pmcstat_npmcs) pmcstat_pmcinfilter = 0; } while (pmcstat_pmcinfilter != pmcin); } } /* * Top mode keypress. */ int pmcstat_keypress_log(void) { int c, ret = 0; WINDOW *w; w = newwin(1, 0, 1, 0); c = wgetch(w); wprintw(w, "Key: %c => ", c); switch (c) { case 'c': wprintw(w, "enter mode 'd' or 'a' => "); c = wgetch(w); if (c == 'd') { args.pa_topmode = PMCSTAT_TOP_DELTA; wprintw(w, "switching to delta mode"); } else { args.pa_topmode = PMCSTAT_TOP_ACCUM; wprintw(w, "switching to accumulation mode"); } break; case 'm': pmcstat_mergepmc = !pmcstat_mergepmc; /* * Changing merge state require data reset. */ if (plugins[args.pa_plugin].pl_shutdown != NULL) plugins[args.pa_plugin].pl_shutdown(NULL); pmcstat_stats_reset(0); if (plugins[args.pa_plugin].pl_init != NULL) plugins[args.pa_plugin].pl_init(); /* Update filter to be on a merge target. */ pmcstat_changefilter(); wprintw(w, "merge PMC %s", pmcstat_mergepmc ? "on" : "off"); break; case 'n': /* Close current plugin. */ if (plugins[args.pa_plugin].pl_shutdown != NULL) plugins[args.pa_plugin].pl_shutdown(NULL); /* Find next top display available. */ do { args.pa_plugin++; if (plugins[args.pa_plugin].pl_name == NULL) args.pa_plugin = 0; } while (plugins[args.pa_plugin].pl_topdisplay == NULL); /* Open new plugin. */ pmcstat_stats_reset(0); if (plugins[args.pa_plugin].pl_init != NULL) plugins[args.pa_plugin].pl_init(); wprintw(w, "switching to plugin %s", plugins[args.pa_plugin].pl_name); break; case 'p': pmcstat_pmcinfilter++; if (pmcstat_pmcinfilter >= pmcstat_npmcs) pmcstat_pmcinfilter = 0; pmcstat_changefilter(); wprintw(w, "switching to PMC %s.%d", pmcstat_pmcindex_to_name(pmcstat_pmcinfilter), pmcstat_pmcinfilter); break; case ' ': pmcstat_pause = !pmcstat_pause; if (pmcstat_pause) wprintw(w, "pause => press space again to continue"); break; case 'q': wprintw(w, "exiting..."); ret = 1; break; default: if (plugins[args.pa_plugin].pl_topkeypress != NULL) if (plugins[args.pa_plugin].pl_topkeypress(c, w)) ret = 1; } wrefresh(w); delwin(w); return ret; } /* * Top mode display. */ void pmcstat_display_log(void) { pmcstat_refresh_top(); /* Reset everythings if delta mode. */ if (args.pa_topmode == PMCSTAT_TOP_DELTA) { if (plugins[args.pa_plugin].pl_shutdown != NULL) plugins[args.pa_plugin].pl_shutdown(NULL); pmcstat_stats_reset(0); if (plugins[args.pa_plugin].pl_init != NULL) plugins[args.pa_plugin].pl_init(); } } /* * Configure a plugins. */ void pmcstat_pluginconfigure_log(char *opt) { if (strncmp(opt, "threshold=", 10) == 0) { pmcstat_threshold = atof(opt+10); } else { if (plugins[args.pa_plugin].pl_configure != NULL) { if (!plugins[args.pa_plugin].pl_configure(opt)) err(EX_USAGE, "ERROR: unknown option <%s>.", opt); } } } /* * Initialize module. */ void pmcstat_initialize_logging(void) { int i; /* use a convenient format for 'ldd' output */ if (setenv("LD_TRACE_LOADED_OBJECTS_FMT1","%o \"%p\" %x\n",1) != 0) err(EX_OSERR, "ERROR: Cannot setenv"); /* Initialize hash tables */ pmcstat_string_initialize(); for (i = 0; i < PMCSTAT_NHASH; i++) { LIST_INIT(&pmcstat_image_hash[i]); LIST_INIT(&pmcstat_process_hash[i]); } /* * Create a fake 'process' entry for the kernel with pid -1. * hwpmc(4) will subsequently inform us about where the kernel * and any loaded kernel modules are mapped. */ if ((pmcstat_kernproc = pmcstat_process_lookup((pid_t) -1, PMCSTAT_ALLOCATE)) == NULL) err(EX_OSERR, "ERROR: Cannot initialize logging"); /* PMC count. */ pmcstat_npmcs = 0; /* Merge PMC with same name. */ pmcstat_mergepmc = args.pa_mergepmc; /* * Initialize plugins */ if (plugins[args.pa_pplugin].pl_init != NULL) plugins[args.pa_pplugin].pl_init(); if (plugins[args.pa_plugin].pl_init != NULL) plugins[args.pa_plugin].pl_init(); } /* * Shutdown module. */ void pmcstat_shutdown_logging(void) { int i; FILE *mf; struct pmcstat_image *pi, *pitmp; struct pmcstat_process *pp, *pptmp; struct pmcstat_pcmap *ppm, *ppmtmp; /* determine where to send the map file */ mf = NULL; if (args.pa_mapfilename != NULL) mf = (strcmp(args.pa_mapfilename, "-") == 0) ? args.pa_printfile : fopen(args.pa_mapfilename, "w"); if (mf == NULL && args.pa_flags & FLAG_DO_GPROF && args.pa_verbosity >= 2) mf = args.pa_printfile; if (mf) (void) fprintf(mf, "MAP:\n"); /* * Shutdown the plugins */ if (plugins[args.pa_plugin].pl_shutdown != NULL) plugins[args.pa_plugin].pl_shutdown(mf); if (plugins[args.pa_pplugin].pl_shutdown != NULL) plugins[args.pa_pplugin].pl_shutdown(mf); for (i = 0; i < PMCSTAT_NHASH; i++) { LIST_FOREACH_SAFE(pi, &pmcstat_image_hash[i], pi_next, pitmp) { if (plugins[args.pa_plugin].pl_shutdownimage != NULL) plugins[args.pa_plugin].pl_shutdownimage(pi); if (plugins[args.pa_pplugin].pl_shutdownimage != NULL) plugins[args.pa_pplugin].pl_shutdownimage(pi); free(pi->pi_symbols); if (pi->pi_addr2line != NULL) pclose(pi->pi_addr2line); LIST_REMOVE(pi, pi_next); free(pi); } LIST_FOREACH_SAFE(pp, &pmcstat_process_hash[i], pp_next, pptmp) { TAILQ_FOREACH_SAFE(ppm, &pp->pp_map, ppm_next, ppmtmp) { TAILQ_REMOVE(&pp->pp_map, ppm, ppm_next); free(ppm); } LIST_REMOVE(pp, pp_next); free(pp); } } pmcstat_string_shutdown(); /* * Print errors unless -q was specified. Print all statistics * if verbosity > 1. */ #define PRINT(N,V) do { \ if (pmcstat_stats.ps_##V || args.pa_verbosity >= 2) \ (void) fprintf(args.pa_printfile, " %-40s %d\n",\ N, pmcstat_stats.ps_##V); \ } while (0) if (args.pa_verbosity >= 1 && (args.pa_flags & FLAG_DO_ANALYSIS)) { (void) fprintf(args.pa_printfile, "CONVERSION STATISTICS:\n"); PRINT("#exec/a.out", exec_aout); PRINT("#exec/elf", exec_elf); PRINT("#exec/unknown", exec_indeterminable); PRINT("#exec handling errors", exec_errors); PRINT("#samples/total", samples_total); PRINT("#samples/unclaimed", samples_unknown_offset); PRINT("#samples/unknown-object", samples_indeterminable); PRINT("#samples/unknown-function", samples_unknown_function); PRINT("#callchain/dubious-frames", callchain_dubious_frames); } if (mf) (void) fclose(mf); } Index: stable/10 =================================================================== --- stable/10 (revision 299825) +++ stable/10 (revision 299826) Property changes on: stable/10 ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head:r298881-298883,298885,298887