Index: projects/clang1000-import/crypto/openssh/FREEBSD-upgrade =================================================================== --- projects/clang1000-import/crypto/openssh/FREEBSD-upgrade (revision 357965) +++ projects/clang1000-import/crypto/openssh/FREEBSD-upgrade (revision 357966) @@ -1,181 +1,181 @@ FreeBSD maintainer's guide to OpenSSH-portable ============================================== 00) Make sure your mail spool has plenty of free space. It'll fill up pretty fast once you're done with this checklist. 01) Download the latest OpenSSH-portable tarball and signature from - OpenBSD (ftp://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/). + OpenBSD (https://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/). 02) Verify the signature: $ gpg --verify openssh-X.YpZ.tar.gz.asc 03) Unpack the tarball in a suitable directory: $ tar xf openssh-X.YpZ.tar.gz 04) Copy to the vendor directory: $ svn co svn+ssh://repo.freebsd.org/base/vendor-crypto/openssh/dist $ rsync --archive --delete openssh-X.YpZ/ dist/ 05) Take care of added / deleted files: $ svn rm $(svn stat dist | awk '$1 == "!" { print $2 }') $ svn add --no-auto-props $(svn stat dist | awk '$1 == "?" { print $2 }') 06) Commit: $ svn commit -m "Vendor import of OpenSSH X.YpZ." dist 07) Tag: $ svn copy -m "Tag OpenSSH X.YpZ." \ svn+ssh://repo.freebsd.org/base/vendor-crypto/openssh/dist \ svn+ssh://repo.freebsd.org/base/vendor-crypto/openssh/X.YpZ 08) Check out head and run the pre-merge script, which strips our RCS tags from files that have them: $ svn co svn+ssh://repo.freebsd.org/base/head $ cd head/crypto/openssh $ sh freebsd-pre-merge.sh 09) Merge from the vendor branch: $ svn merge -cNNNNNN \^/vendor-crypto/openssh/dist . 0A) Resolve conflicts. Remember to bump the version addendum in version.h, and update the default value in ssh{,d}_config and ssh{,d}_config.5. 0B) Diff against the vendor branch: $ svn diff --no-diff-deleted --no-diff-added \ --ignore-properties \^/vendor-crypto/openssh/X.YpZ . Files that have modifications relative to the vendor code, and only those files, must have the svn:keywords property set to FreeBSD=%H and be listed in the 'keywords' file created by the pre-merge script. 0C) Run the post-merge script, which re-adds RCS tags to files that need them: $ sh freebsd-post-merge.sh 0D) Run the configure script: $ sh freebsd-configure.sh 0E) Review changes to config.h very carefully. Note that libwrap should not be defined in config.h; as of r311585 it is conditional on MK_TCP_WRAPPERS. 0F) If source files have been added or removed, update the appropriate makefiles to reflect changes in the vendor's Makefile.in. 10) Update ssh_namespace.h: $ sh freebsd-namespace.sh 11) Build and install world, reboot, test. Pay particular attention to pam_ssh(8), which gropes inside libssh and will break if something significant changes or if ssh_namespace.h is out of whack. 12) Commit, and hunker down for the inevitable storm of complaints. An overview of FreeBSD changes to OpenSSH-portable ================================================== 0) VersionAddendum The SSH protocol allows for a human-readable version string of up to 40 characters to be appended to the protocol version string. FreeBSD takes advantage of this to include a date indicating the "patch level", so people can easily determine whether their system is vulnerable when an OpenSSH advisory goes out. Some people, however, dislike advertising their patch level in the protocol handshake, so we've added a VersionAddendum configuration variable to allow them to change or disable it. Upstream added support for VersionAddendum on the server side, but we also support it on the client side. 1) Modified server-side defaults We've modified some configuration defaults in sshd: - UsePAM defaults to "yes". - PermitRootLogin defaults to "no". - X11Forwarding defaults to "yes". - PasswordAuthentication defaults to "no". - VersionAddendum defaults to "FreeBSD-YYYYMMDD". - PrivilegeSeparation defaults to "sandbox". - UseDNS defaults to "yes". 2) Modified client-side defaults We've modified some configuration defaults in ssh: - CheckHostIP defaults to "no". - VerifyHostKeyDNS defaults to "yes" if built with LDNS. - VersionAddendum defaults to "FreeBSD-YYYYMMDD". 3) Canonic host names We've added code to ssh.c to canonicize the target host name after reading options but before trying to connect. This eliminates the usual problem with duplicate known_hosts entries. 4) setusercontext() environment Our setusercontext(3) can set environment variables, which we must take care to transfer to the child's environment. 5) TCP wrappers Support for TCP wrappers was removed in upstream 6.7p1. We've added it back by porting the 6.6p1 code forward. TCP wrappers support in sshd will be disabled in HEAD and will be removed from FreeBSD in the future. 6) Agent client reference counting We've added code to ssh-agent.c to implement client reference counting; the agent will automatically exit when the last client disconnects. 7) Class-based login restrictions We've added code to auth2.c to enforce the host.allow, host.deny, times.allow and times.deny login class capabilities. 8) HPN We no longer have the HPN patches (adaptive buffer size for increased throughput on high-BxD links), but we recognize and ignore HPN-related configuration options to avoid breaking existing configurations. 9) AES-CBC The AES-CBC ciphers were removed from the server-side proposal list in 6.7p1 due to theoretical weaknesses and the availability of superior ciphers (including AES-CTR and AES-GCM). We have re-added them for compatibility with third-party clients. This port was brought to you by (in no particular order) DARPA, NAI Labs, ThinkSec, Nescafé, the Aberlour Glenlivet Distillery Co., Suzanne Vega, and a Sanford's #69 Deluxe Marker. -- des@FreeBSD.org $FreeBSD$ Index: projects/clang1000-import/crypto/openssh/FREEBSD-vendor =================================================================== --- projects/clang1000-import/crypto/openssh/FREEBSD-vendor (revision 357965) +++ projects/clang1000-import/crypto/openssh/FREEBSD-vendor (revision 357966) @@ -1,6 +1,6 @@ # $FreeBSD$ Project: Portable OpenSSH ProjectURL: http://www.openssh.com/portable.html -Version: 5.2p1 +Version: 7.9p1 License: BSD Maintainer: des Index: projects/clang1000-import/crypto/openssh =================================================================== --- projects/clang1000-import/crypto/openssh (revision 357965) +++ projects/clang1000-import/crypto/openssh (revision 357966) Property changes on: projects/clang1000-import/crypto/openssh ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/crypto/openssh:r357931-357965 Index: projects/clang1000-import/sys/amd64/amd64/pmap.c =================================================================== --- projects/clang1000-import/sys/amd64/amd64/pmap.c (revision 357965) +++ projects/clang1000-import/sys/amd64/amd64/pmap.c (revision 357966) @@ -1,10667 +1,10667 @@ /*- * SPDX-License-Identifier: BSD-4-Clause * * Copyright (c) 1991 Regents of the University of California. * All rights reserved. * Copyright (c) 1994 John S. Dyson * All rights reserved. * Copyright (c) 1994 David Greenman * All rights reserved. * Copyright (c) 2003 Peter Wemm * All rights reserved. * Copyright (c) 2005-2010 Alan L. Cox * All rights reserved. * * This code is derived from software contributed to Berkeley by * the Systems Programming Group of the University of Utah Computer * Science Department and William Jolitz of UUNET Technologies Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by the University of * California, Berkeley and its contributors. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * from: @(#)pmap.c 7.7 (Berkeley) 5/12/91 */ /*- * Copyright (c) 2003 Networks Associates Technology, Inc. * Copyright (c) 2014-2019 The FreeBSD Foundation * All rights reserved. * * This software was developed for the FreeBSD Project by Jake Burkholder, * Safeport Network Services, and Network Associates Laboratories, the * Security Research Division of Network Associates, Inc. under * DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA * CHATS research program. * * Portions of this software were developed by * Konstantin Belousov under sponsorship from * the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #define AMD64_NPT_AWARE #include __FBSDID("$FreeBSD$"); /* * Manages physical address maps. * * Since the information managed by this module is * also stored by the logical address mapping module, * this module may throw away valid virtual-to-physical * mappings at almost any time. However, invalidations * of virtual-to-physical mappings must be done as * requested. * * In order to cope with hardware architectures which * make virtual-to-physical map invalidates expensive, * this module may delay invalidate or reduced protection * operations until such time as they are actually * necessary. This module is given full information as * to which processors are currently using which maps, * and to when physical maps must be made correct. */ #include "opt_ddb.h" #include "opt_pmap.h" #include "opt_vm.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef DDB #include #include #endif #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef SMP #include #endif #include #include #ifdef NUMA #define PMAP_MEMDOM MAXMEMDOM #else #define PMAP_MEMDOM 1 #endif static __inline boolean_t pmap_type_guest(pmap_t pmap) { return ((pmap->pm_type == PT_EPT) || (pmap->pm_type == PT_RVI)); } static __inline boolean_t pmap_emulate_ad_bits(pmap_t pmap) { return ((pmap->pm_flags & PMAP_EMULATE_AD_BITS) != 0); } static __inline pt_entry_t pmap_valid_bit(pmap_t pmap) { pt_entry_t mask; switch (pmap->pm_type) { case PT_X86: case PT_RVI: mask = X86_PG_V; break; case PT_EPT: if (pmap_emulate_ad_bits(pmap)) mask = EPT_PG_EMUL_V; else mask = EPT_PG_READ; break; default: panic("pmap_valid_bit: invalid pm_type %d", pmap->pm_type); } return (mask); } static __inline pt_entry_t pmap_rw_bit(pmap_t pmap) { pt_entry_t mask; switch (pmap->pm_type) { case PT_X86: case PT_RVI: mask = X86_PG_RW; break; case PT_EPT: if (pmap_emulate_ad_bits(pmap)) mask = EPT_PG_EMUL_RW; else mask = EPT_PG_WRITE; break; default: panic("pmap_rw_bit: invalid pm_type %d", pmap->pm_type); } return (mask); } static pt_entry_t pg_g; static __inline pt_entry_t pmap_global_bit(pmap_t pmap) { pt_entry_t mask; switch (pmap->pm_type) { case PT_X86: mask = pg_g; break; case PT_RVI: case PT_EPT: mask = 0; break; default: panic("pmap_global_bit: invalid pm_type %d", pmap->pm_type); } return (mask); } static __inline pt_entry_t pmap_accessed_bit(pmap_t pmap) { pt_entry_t mask; switch (pmap->pm_type) { case PT_X86: case PT_RVI: mask = X86_PG_A; break; case PT_EPT: if (pmap_emulate_ad_bits(pmap)) mask = EPT_PG_READ; else mask = EPT_PG_A; break; default: panic("pmap_accessed_bit: invalid pm_type %d", pmap->pm_type); } return (mask); } static __inline pt_entry_t pmap_modified_bit(pmap_t pmap) { pt_entry_t mask; switch (pmap->pm_type) { case PT_X86: case PT_RVI: mask = X86_PG_M; break; case PT_EPT: if (pmap_emulate_ad_bits(pmap)) mask = EPT_PG_WRITE; else mask = EPT_PG_M; break; default: panic("pmap_modified_bit: invalid pm_type %d", pmap->pm_type); } return (mask); } static __inline pt_entry_t pmap_pku_mask_bit(pmap_t pmap) { return (pmap->pm_type == PT_X86 ? X86_PG_PKU_MASK : 0); } #if !defined(DIAGNOSTIC) #ifdef __GNUC_GNU_INLINE__ #define PMAP_INLINE __attribute__((__gnu_inline__)) inline #else #define PMAP_INLINE extern inline #endif #else #define PMAP_INLINE #endif #ifdef PV_STATS #define PV_STAT(x) do { x ; } while (0) #else #define PV_STAT(x) do { } while (0) #endif #undef pa_index #define pa_index(pa) ({ \ KASSERT((pa) <= vm_phys_segs[vm_phys_nsegs - 1].end, \ ("address %lx beyond the last segment", (pa))); \ (pa) >> PDRSHIFT; \ }) #ifdef NUMA #define pa_to_pmdp(pa) (&pv_table[pa_index(pa)]) #define pa_to_pvh(pa) (&(pa_to_pmdp(pa)->pv_page)) #define PHYS_TO_PV_LIST_LOCK(pa) ({ \ struct rwlock *_lock; \ if (__predict_false((pa) > pmap_last_pa)) \ _lock = &pv_dummy_large.pv_lock; \ else \ _lock = &(pa_to_pmdp(pa)->pv_lock); \ _lock; \ }) #else #define pa_to_pvh(pa) (&pv_table[pa_index(pa)]) #define NPV_LIST_LOCKS MAXCPU #define PHYS_TO_PV_LIST_LOCK(pa) \ (&pv_list_locks[pa_index(pa) % NPV_LIST_LOCKS]) #endif #define CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, pa) do { \ struct rwlock **_lockp = (lockp); \ struct rwlock *_new_lock; \ \ _new_lock = PHYS_TO_PV_LIST_LOCK(pa); \ if (_new_lock != *_lockp) { \ if (*_lockp != NULL) \ rw_wunlock(*_lockp); \ *_lockp = _new_lock; \ rw_wlock(*_lockp); \ } \ } while (0) #define CHANGE_PV_LIST_LOCK_TO_VM_PAGE(lockp, m) \ CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, VM_PAGE_TO_PHYS(m)) #define RELEASE_PV_LIST_LOCK(lockp) do { \ struct rwlock **_lockp = (lockp); \ \ if (*_lockp != NULL) { \ rw_wunlock(*_lockp); \ *_lockp = NULL; \ } \ } while (0) #define VM_PAGE_TO_PV_LIST_LOCK(m) \ PHYS_TO_PV_LIST_LOCK(VM_PAGE_TO_PHYS(m)) struct pmap kernel_pmap_store; vm_offset_t virtual_avail; /* VA of first avail page (after kernel bss) */ vm_offset_t virtual_end; /* VA of last avail page (end of kernel AS) */ int nkpt; SYSCTL_INT(_machdep, OID_AUTO, nkpt, CTLFLAG_RD, &nkpt, 0, "Number of kernel page table pages allocated on bootup"); static int ndmpdp; vm_paddr_t dmaplimit; vm_offset_t kernel_vm_end = VM_MIN_KERNEL_ADDRESS; pt_entry_t pg_nx; static SYSCTL_NODE(_vm, OID_AUTO, pmap, CTLFLAG_RD, 0, "VM/pmap parameters"); static int pg_ps_enabled = 1; SYSCTL_INT(_vm_pmap, OID_AUTO, pg_ps_enabled, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, &pg_ps_enabled, 0, "Are large page mappings enabled?"); #define PAT_INDEX_SIZE 8 static int pat_index[PAT_INDEX_SIZE]; /* cache mode to PAT index conversion */ static u_int64_t KPTphys; /* phys addr of kernel level 1 */ static u_int64_t KPDphys; /* phys addr of kernel level 2 */ u_int64_t KPDPphys; /* phys addr of kernel level 3 */ u_int64_t KPML4phys; /* phys addr of kernel level 4 */ static u_int64_t DMPDphys; /* phys addr of direct mapped level 2 */ static u_int64_t DMPDPphys; /* phys addr of direct mapped level 3 */ static int ndmpdpphys; /* number of DMPDPphys pages */ static vm_paddr_t KERNend; /* phys addr of end of bootstrap data */ /* * pmap_mapdev support pre initialization (i.e. console) */ #define PMAP_PREINIT_MAPPING_COUNT 8 static struct pmap_preinit_mapping { vm_paddr_t pa; vm_offset_t va; vm_size_t sz; int mode; } pmap_preinit_mapping[PMAP_PREINIT_MAPPING_COUNT]; static int pmap_initialized; /* * Data for the pv entry allocation mechanism. * Updates to pv_invl_gen are protected by the pv list lock but reads are not. */ #ifdef NUMA static __inline int pc_to_domain(struct pv_chunk *pc) { return (_vm_phys_domain(DMAP_TO_PHYS((vm_offset_t)pc))); } #else static __inline int pc_to_domain(struct pv_chunk *pc __unused) { return (0); } #endif struct pv_chunks_list { struct mtx pvc_lock; TAILQ_HEAD(pch, pv_chunk) pvc_list; int active_reclaims; } __aligned(CACHE_LINE_SIZE); struct pv_chunks_list __exclusive_cache_line pv_chunks[PMAP_MEMDOM]; #ifdef NUMA struct pmap_large_md_page { struct rwlock pv_lock; struct md_page pv_page; u_long pv_invl_gen; }; __exclusive_cache_line static struct pmap_large_md_page pv_dummy_large; #define pv_dummy pv_dummy_large.pv_page __read_mostly static struct pmap_large_md_page *pv_table; __read_mostly vm_paddr_t pmap_last_pa; #else static struct rwlock __exclusive_cache_line pv_list_locks[NPV_LIST_LOCKS]; static u_long pv_invl_gen[NPV_LIST_LOCKS]; static struct md_page *pv_table; static struct md_page pv_dummy; #endif /* * All those kernel PT submaps that BSD is so fond of */ pt_entry_t *CMAP1 = NULL; caddr_t CADDR1 = 0; static vm_offset_t qframe = 0; static struct mtx qframe_mtx; static int pmap_flags = PMAP_PDE_SUPERPAGE; /* flags for x86 pmaps */ static vmem_t *large_vmem; static u_int lm_ents; #define PMAP_ADDRESS_IN_LARGEMAP(va) ((va) >= LARGEMAP_MIN_ADDRESS && \ (va) < LARGEMAP_MIN_ADDRESS + NBPML4 * (u_long)lm_ents) int pmap_pcid_enabled = 1; SYSCTL_INT(_vm_pmap, OID_AUTO, pcid_enabled, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, &pmap_pcid_enabled, 0, "Is TLB Context ID enabled ?"); int invpcid_works = 0; SYSCTL_INT(_vm_pmap, OID_AUTO, invpcid_works, CTLFLAG_RD, &invpcid_works, 0, "Is the invpcid instruction available ?"); int __read_frequently pti = 0; SYSCTL_INT(_vm_pmap, OID_AUTO, pti, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, &pti, 0, "Page Table Isolation enabled"); static vm_object_t pti_obj; static pml4_entry_t *pti_pml4; static vm_pindex_t pti_pg_idx; static bool pti_finalized; struct pmap_pkru_range { struct rs_el pkru_rs_el; u_int pkru_keyidx; int pkru_flags; }; static uma_zone_t pmap_pkru_ranges_zone; static bool pmap_pkru_same(pmap_t pmap, vm_offset_t sva, vm_offset_t eva); static pt_entry_t pmap_pkru_get(pmap_t pmap, vm_offset_t va); static void pmap_pkru_on_remove(pmap_t pmap, vm_offset_t sva, vm_offset_t eva); static void *pkru_dup_range(void *ctx, void *data); static void pkru_free_range(void *ctx, void *node); static int pmap_pkru_copy(pmap_t dst_pmap, pmap_t src_pmap); static int pmap_pkru_deassign(pmap_t pmap, vm_offset_t sva, vm_offset_t eva); static void pmap_pkru_deassign_all(pmap_t pmap); static int pmap_pcid_save_cnt_proc(SYSCTL_HANDLER_ARGS) { int i; uint64_t res; res = 0; CPU_FOREACH(i) { res += cpuid_to_pcpu[i]->pc_pm_save_cnt; } return (sysctl_handle_64(oidp, &res, 0, req)); } SYSCTL_PROC(_vm_pmap, OID_AUTO, pcid_save_cnt, CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, 0, pmap_pcid_save_cnt_proc, "QU", "Count of saved TLB context on switch"); static LIST_HEAD(, pmap_invl_gen) pmap_invl_gen_tracker = LIST_HEAD_INITIALIZER(&pmap_invl_gen_tracker); static struct mtx invl_gen_mtx; /* Fake lock object to satisfy turnstiles interface. */ static struct lock_object invl_gen_ts = { .lo_name = "invlts", }; static struct pmap_invl_gen pmap_invl_gen_head = { .gen = 1, .next = NULL, }; static u_long pmap_invl_gen = 1; static int pmap_invl_waiters; static struct callout pmap_invl_callout; static bool pmap_invl_callout_inited; #define PMAP_ASSERT_NOT_IN_DI() \ KASSERT(pmap_not_in_di(), ("DI already started")) static bool pmap_di_locked(void) { int tun; if ((cpu_feature2 & CPUID2_CX16) == 0) return (true); tun = 0; TUNABLE_INT_FETCH("vm.pmap.di_locked", &tun); return (tun != 0); } static int sysctl_pmap_di_locked(SYSCTL_HANDLER_ARGS) { int locked; locked = pmap_di_locked(); return (sysctl_handle_int(oidp, &locked, 0, req)); } SYSCTL_PROC(_vm_pmap, OID_AUTO, di_locked, CTLTYPE_INT | CTLFLAG_RDTUN | CTLFLAG_MPSAFE, 0, 0, sysctl_pmap_di_locked, "", "Locked delayed invalidation"); static bool pmap_not_in_di_l(void); static bool pmap_not_in_di_u(void); DEFINE_IFUNC(, bool, pmap_not_in_di, (void)) { return (pmap_di_locked() ? pmap_not_in_di_l : pmap_not_in_di_u); } static bool pmap_not_in_di_l(void) { struct pmap_invl_gen *invl_gen; invl_gen = &curthread->td_md.md_invl_gen; return (invl_gen->gen == 0); } static void pmap_thread_init_invl_gen_l(struct thread *td) { struct pmap_invl_gen *invl_gen; invl_gen = &td->td_md.md_invl_gen; invl_gen->gen = 0; } static void pmap_delayed_invl_wait_block(u_long *m_gen, u_long *invl_gen) { struct turnstile *ts; ts = turnstile_trywait(&invl_gen_ts); if (*m_gen > atomic_load_long(invl_gen)) turnstile_wait(ts, NULL, TS_SHARED_QUEUE); else turnstile_cancel(ts); } static void pmap_delayed_invl_finish_unblock(u_long new_gen) { struct turnstile *ts; turnstile_chain_lock(&invl_gen_ts); ts = turnstile_lookup(&invl_gen_ts); if (new_gen != 0) pmap_invl_gen = new_gen; if (ts != NULL) { turnstile_broadcast(ts, TS_SHARED_QUEUE); turnstile_unpend(ts); } turnstile_chain_unlock(&invl_gen_ts); } /* * Start a new Delayed Invalidation (DI) block of code, executed by * the current thread. Within a DI block, the current thread may * destroy both the page table and PV list entries for a mapping and * then release the corresponding PV list lock before ensuring that * the mapping is flushed from the TLBs of any processors with the * pmap active. */ static void pmap_delayed_invl_start_l(void) { struct pmap_invl_gen *invl_gen; u_long currgen; invl_gen = &curthread->td_md.md_invl_gen; PMAP_ASSERT_NOT_IN_DI(); mtx_lock(&invl_gen_mtx); if (LIST_EMPTY(&pmap_invl_gen_tracker)) currgen = pmap_invl_gen; else currgen = LIST_FIRST(&pmap_invl_gen_tracker)->gen; invl_gen->gen = currgen + 1; LIST_INSERT_HEAD(&pmap_invl_gen_tracker, invl_gen, link); mtx_unlock(&invl_gen_mtx); } /* * Finish the DI block, previously started by the current thread. All * required TLB flushes for the pages marked by * pmap_delayed_invl_page() must be finished before this function is * called. * * This function works by bumping the global DI generation number to * the generation number of the current thread's DI, unless there is a * pending DI that started earlier. In the latter case, bumping the * global DI generation number would incorrectly signal that the * earlier DI had finished. Instead, this function bumps the earlier * DI's generation number to match the generation number of the * current thread's DI. */ static void pmap_delayed_invl_finish_l(void) { struct pmap_invl_gen *invl_gen, *next; invl_gen = &curthread->td_md.md_invl_gen; KASSERT(invl_gen->gen != 0, ("missed invl_start")); mtx_lock(&invl_gen_mtx); next = LIST_NEXT(invl_gen, link); if (next == NULL) pmap_delayed_invl_finish_unblock(invl_gen->gen); else next->gen = invl_gen->gen; LIST_REMOVE(invl_gen, link); mtx_unlock(&invl_gen_mtx); invl_gen->gen = 0; } static bool pmap_not_in_di_u(void) { struct pmap_invl_gen *invl_gen; invl_gen = &curthread->td_md.md_invl_gen; return (((uintptr_t)invl_gen->next & PMAP_INVL_GEN_NEXT_INVALID) != 0); } static void pmap_thread_init_invl_gen_u(struct thread *td) { struct pmap_invl_gen *invl_gen; invl_gen = &td->td_md.md_invl_gen; invl_gen->gen = 0; invl_gen->next = (void *)PMAP_INVL_GEN_NEXT_INVALID; } static bool pmap_di_load_invl(struct pmap_invl_gen *ptr, struct pmap_invl_gen *out) { uint64_t new_high, new_low, old_high, old_low; char res; old_low = new_low = 0; old_high = new_high = (uintptr_t)0; __asm volatile("lock;cmpxchg16b\t%1;sete\t%0" : "=r" (res), "+m" (*ptr), "+a" (old_low), "+d" (old_high) : "b"(new_low), "c" (new_high) : "memory", "cc"); if (res == 0) { if ((old_high & PMAP_INVL_GEN_NEXT_INVALID) != 0) return (false); out->gen = old_low; out->next = (void *)old_high; } else { out->gen = new_low; out->next = (void *)new_high; } return (true); } static bool pmap_di_store_invl(struct pmap_invl_gen *ptr, struct pmap_invl_gen *old_val, struct pmap_invl_gen *new_val) { uint64_t new_high, new_low, old_high, old_low; char res; new_low = new_val->gen; new_high = (uintptr_t)new_val->next; old_low = old_val->gen; old_high = (uintptr_t)old_val->next; __asm volatile("lock;cmpxchg16b\t%1;sete\t%0" : "=r" (res), "+m" (*ptr), "+a" (old_low), "+d" (old_high) : "b"(new_low), "c" (new_high) : "memory", "cc"); return (res); } #ifdef PV_STATS static long invl_start_restart; SYSCTL_LONG(_vm_pmap, OID_AUTO, invl_start_restart, CTLFLAG_RD, &invl_start_restart, 0, ""); static long invl_finish_restart; SYSCTL_LONG(_vm_pmap, OID_AUTO, invl_finish_restart, CTLFLAG_RD, &invl_finish_restart, 0, ""); static int invl_max_qlen; SYSCTL_INT(_vm_pmap, OID_AUTO, invl_max_qlen, CTLFLAG_RD, &invl_max_qlen, 0, ""); #endif #define di_delay locks_delay static void pmap_delayed_invl_start_u(void) { struct pmap_invl_gen *invl_gen, *p, prev, new_prev; struct thread *td; struct lock_delay_arg lda; uintptr_t prevl; u_char pri; #ifdef PV_STATS int i, ii; #endif td = curthread; invl_gen = &td->td_md.md_invl_gen; PMAP_ASSERT_NOT_IN_DI(); lock_delay_arg_init(&lda, &di_delay); invl_gen->saved_pri = 0; pri = td->td_base_pri; if (pri > PVM) { thread_lock(td); pri = td->td_base_pri; if (pri > PVM) { invl_gen->saved_pri = pri; sched_prio(td, PVM); } thread_unlock(td); } again: PV_STAT(i = 0); for (p = &pmap_invl_gen_head;; p = prev.next) { PV_STAT(i++); - prevl = atomic_load_ptr(&p->next); + prevl = (uintptr_t)atomic_load_ptr(&p->next); if ((prevl & PMAP_INVL_GEN_NEXT_INVALID) != 0) { PV_STAT(atomic_add_long(&invl_start_restart, 1)); lock_delay(&lda); goto again; } if (prevl == 0) break; prev.next = (void *)prevl; } #ifdef PV_STATS if ((ii = invl_max_qlen) < i) atomic_cmpset_int(&invl_max_qlen, ii, i); #endif if (!pmap_di_load_invl(p, &prev) || prev.next != NULL) { PV_STAT(atomic_add_long(&invl_start_restart, 1)); lock_delay(&lda); goto again; } new_prev.gen = prev.gen; new_prev.next = invl_gen; invl_gen->gen = prev.gen + 1; /* Formal fence between store to invl->gen and updating *p. */ atomic_thread_fence_rel(); /* * After inserting an invl_gen element with invalid bit set, * this thread blocks any other thread trying to enter the * delayed invalidation block. Do not allow to remove us from * the CPU, because it causes starvation for other threads. */ critical_enter(); /* * ABA for *p is not possible there, since p->gen can only * increase. So if the *p thread finished its di, then * started a new one and got inserted into the list at the * same place, its gen will appear greater than the previously * read gen. */ if (!pmap_di_store_invl(p, &prev, &new_prev)) { critical_exit(); PV_STAT(atomic_add_long(&invl_start_restart, 1)); lock_delay(&lda); goto again; } /* * There we clear PMAP_INVL_GEN_NEXT_INVALID in * invl_gen->next, allowing other threads to iterate past us. * pmap_di_store_invl() provides fence between the generation * write and the update of next. */ invl_gen->next = NULL; critical_exit(); } static bool pmap_delayed_invl_finish_u_crit(struct pmap_invl_gen *invl_gen, struct pmap_invl_gen *p) { struct pmap_invl_gen prev, new_prev; u_long mygen; /* * Load invl_gen->gen after setting invl_gen->next * PMAP_INVL_GEN_NEXT_INVALID. This prevents larger * generations to propagate to our invl_gen->gen. Lock prefix * in atomic_set_ptr() worked as seq_cst fence. */ mygen = atomic_load_long(&invl_gen->gen); if (!pmap_di_load_invl(p, &prev) || prev.next != invl_gen) return (false); KASSERT(prev.gen < mygen, ("invalid di gen sequence %lu %lu", prev.gen, mygen)); new_prev.gen = mygen; new_prev.next = (void *)((uintptr_t)invl_gen->next & ~PMAP_INVL_GEN_NEXT_INVALID); /* Formal fence between load of prev and storing update to it. */ atomic_thread_fence_rel(); return (pmap_di_store_invl(p, &prev, &new_prev)); } static void pmap_delayed_invl_finish_u(void) { struct pmap_invl_gen *invl_gen, *p; struct thread *td; struct lock_delay_arg lda; uintptr_t prevl; td = curthread; invl_gen = &td->td_md.md_invl_gen; KASSERT(invl_gen->gen != 0, ("missed invl_start: gen 0")); KASSERT(((uintptr_t)invl_gen->next & PMAP_INVL_GEN_NEXT_INVALID) == 0, ("missed invl_start: INVALID")); lock_delay_arg_init(&lda, &di_delay); again: for (p = &pmap_invl_gen_head; p != NULL; p = (void *)prevl) { - prevl = atomic_load_ptr(&p->next); + prevl = (uintptr_t)atomic_load_ptr(&p->next); if ((prevl & PMAP_INVL_GEN_NEXT_INVALID) != 0) { PV_STAT(atomic_add_long(&invl_finish_restart, 1)); lock_delay(&lda); goto again; } if ((void *)prevl == invl_gen) break; } /* * It is legitimate to not find ourself on the list if a * thread before us finished its DI and started it again. */ if (__predict_false(p == NULL)) { PV_STAT(atomic_add_long(&invl_finish_restart, 1)); lock_delay(&lda); goto again; } critical_enter(); atomic_set_ptr((uintptr_t *)&invl_gen->next, PMAP_INVL_GEN_NEXT_INVALID); if (!pmap_delayed_invl_finish_u_crit(invl_gen, p)) { atomic_clear_ptr((uintptr_t *)&invl_gen->next, PMAP_INVL_GEN_NEXT_INVALID); critical_exit(); PV_STAT(atomic_add_long(&invl_finish_restart, 1)); lock_delay(&lda); goto again; } critical_exit(); if (atomic_load_int(&pmap_invl_waiters) > 0) pmap_delayed_invl_finish_unblock(0); if (invl_gen->saved_pri != 0) { thread_lock(td); sched_prio(td, invl_gen->saved_pri); thread_unlock(td); } } #ifdef DDB DB_SHOW_COMMAND(di_queue, pmap_di_queue) { struct pmap_invl_gen *p, *pn; struct thread *td; uintptr_t nextl; bool first; for (p = &pmap_invl_gen_head, first = true; p != NULL; p = pn, first = false) { - nextl = atomic_load_ptr(&p->next); + nextl = (uintptr_t)atomic_load_ptr(&p->next); pn = (void *)(nextl & ~PMAP_INVL_GEN_NEXT_INVALID); td = first ? NULL : __containerof(p, struct thread, td_md.md_invl_gen); db_printf("gen %lu inv %d td %p tid %d\n", p->gen, (nextl & PMAP_INVL_GEN_NEXT_INVALID) != 0, td, td != NULL ? td->td_tid : -1); } } #endif #ifdef PV_STATS static long invl_wait; SYSCTL_LONG(_vm_pmap, OID_AUTO, invl_wait, CTLFLAG_RD, &invl_wait, 0, "Number of times DI invalidation blocked pmap_remove_all/write"); static long invl_wait_slow; SYSCTL_LONG(_vm_pmap, OID_AUTO, invl_wait_slow, CTLFLAG_RD, &invl_wait_slow, 0, "Number of slow invalidation waits for lockless DI"); #endif #ifdef NUMA static u_long * pmap_delayed_invl_genp(vm_page_t m) { vm_paddr_t pa; u_long *gen; pa = VM_PAGE_TO_PHYS(m); if (__predict_false((pa) > pmap_last_pa)) gen = &pv_dummy_large.pv_invl_gen; else gen = &(pa_to_pmdp(pa)->pv_invl_gen); return (gen); } #else static u_long * pmap_delayed_invl_genp(vm_page_t m) { return (&pv_invl_gen[pa_index(VM_PAGE_TO_PHYS(m)) % NPV_LIST_LOCKS]); } #endif static void pmap_delayed_invl_callout_func(void *arg __unused) { if (atomic_load_int(&pmap_invl_waiters) == 0) return; pmap_delayed_invl_finish_unblock(0); } static void pmap_delayed_invl_callout_init(void *arg __unused) { if (pmap_di_locked()) return; callout_init(&pmap_invl_callout, 1); pmap_invl_callout_inited = true; } SYSINIT(pmap_di_callout, SI_SUB_CPU + 1, SI_ORDER_ANY, pmap_delayed_invl_callout_init, NULL); /* * Ensure that all currently executing DI blocks, that need to flush * TLB for the given page m, actually flushed the TLB at the time the * function returned. If the page m has an empty PV list and we call * pmap_delayed_invl_wait(), upon its return we know that no CPU has a * valid mapping for the page m in either its page table or TLB. * * This function works by blocking until the global DI generation * number catches up with the generation number associated with the * given page m and its PV list. Since this function's callers * typically own an object lock and sometimes own a page lock, it * cannot sleep. Instead, it blocks on a turnstile to relinquish the * processor. */ static void pmap_delayed_invl_wait_l(vm_page_t m) { u_long *m_gen; #ifdef PV_STATS bool accounted = false; #endif m_gen = pmap_delayed_invl_genp(m); while (*m_gen > pmap_invl_gen) { #ifdef PV_STATS if (!accounted) { atomic_add_long(&invl_wait, 1); accounted = true; } #endif pmap_delayed_invl_wait_block(m_gen, &pmap_invl_gen); } } static void pmap_delayed_invl_wait_u(vm_page_t m) { u_long *m_gen; struct lock_delay_arg lda; bool fast; fast = true; m_gen = pmap_delayed_invl_genp(m); lock_delay_arg_init(&lda, &di_delay); while (*m_gen > atomic_load_long(&pmap_invl_gen_head.gen)) { if (fast || !pmap_invl_callout_inited) { PV_STAT(atomic_add_long(&invl_wait, 1)); lock_delay(&lda); fast = false; } else { /* * The page's invalidation generation number * is still below the current thread's number. * Prepare to block so that we do not waste * CPU cycles or worse, suffer livelock. * * Since it is impossible to block without * racing with pmap_delayed_invl_finish_u(), * prepare for the race by incrementing * pmap_invl_waiters and arming a 1-tick * callout which will unblock us if we lose * the race. */ atomic_add_int(&pmap_invl_waiters, 1); /* * Re-check the current thread's invalidation * generation after incrementing * pmap_invl_waiters, so that there is no race * with pmap_delayed_invl_finish_u() setting * the page generation and checking * pmap_invl_waiters. The only race allowed * is for a missed unblock, which is handled * by the callout. */ if (*m_gen > atomic_load_long(&pmap_invl_gen_head.gen)) { callout_reset(&pmap_invl_callout, 1, pmap_delayed_invl_callout_func, NULL); PV_STAT(atomic_add_long(&invl_wait_slow, 1)); pmap_delayed_invl_wait_block(m_gen, &pmap_invl_gen_head.gen); } atomic_add_int(&pmap_invl_waiters, -1); } } } DEFINE_IFUNC(, void, pmap_thread_init_invl_gen, (struct thread *)) { return (pmap_di_locked() ? pmap_thread_init_invl_gen_l : pmap_thread_init_invl_gen_u); } DEFINE_IFUNC(static, void, pmap_delayed_invl_start, (void)) { return (pmap_di_locked() ? pmap_delayed_invl_start_l : pmap_delayed_invl_start_u); } DEFINE_IFUNC(static, void, pmap_delayed_invl_finish, (void)) { return (pmap_di_locked() ? pmap_delayed_invl_finish_l : pmap_delayed_invl_finish_u); } DEFINE_IFUNC(static, void, pmap_delayed_invl_wait, (vm_page_t)) { return (pmap_di_locked() ? pmap_delayed_invl_wait_l : pmap_delayed_invl_wait_u); } /* * Mark the page m's PV list as participating in the current thread's * DI block. Any threads concurrently using m's PV list to remove or * restrict all mappings to m will wait for the current thread's DI * block to complete before proceeding. * * The function works by setting the DI generation number for m's PV * list to at least the DI generation number of the current thread. * This forces a caller of pmap_delayed_invl_wait() to block until * current thread calls pmap_delayed_invl_finish(). */ static void pmap_delayed_invl_page(vm_page_t m) { u_long gen, *m_gen; rw_assert(VM_PAGE_TO_PV_LIST_LOCK(m), RA_WLOCKED); gen = curthread->td_md.md_invl_gen.gen; if (gen == 0) return; m_gen = pmap_delayed_invl_genp(m); if (*m_gen < gen) *m_gen = gen; } /* * Crashdump maps. */ static caddr_t crashdumpmap; /* * Internal flags for pmap_enter()'s helper functions. */ #define PMAP_ENTER_NORECLAIM 0x1000000 /* Don't reclaim PV entries. */ #define PMAP_ENTER_NOREPLACE 0x2000000 /* Don't replace mappings. */ /* * Internal flags for pmap_mapdev_internal() and * pmap_change_props_locked(). */ #define MAPDEV_FLUSHCACHE 0x00000001 /* Flush cache after mapping. */ #define MAPDEV_SETATTR 0x00000002 /* Modify existing attrs. */ #define MAPDEV_ASSERTVALID 0x00000004 /* Assert mapping validity. */ TAILQ_HEAD(pv_chunklist, pv_chunk); static void free_pv_chunk(struct pv_chunk *pc); static void free_pv_chunk_batch(struct pv_chunklist *batch); static void free_pv_entry(pmap_t pmap, pv_entry_t pv); static pv_entry_t get_pv_entry(pmap_t pmap, struct rwlock **lockp); static int popcnt_pc_map_pq(uint64_t *map); static vm_page_t reclaim_pv_chunk(pmap_t locked_pmap, struct rwlock **lockp); static void reserve_pv_entries(pmap_t pmap, int needed, struct rwlock **lockp); static void pmap_pv_demote_pde(pmap_t pmap, vm_offset_t va, vm_paddr_t pa, struct rwlock **lockp); static bool pmap_pv_insert_pde(pmap_t pmap, vm_offset_t va, pd_entry_t pde, u_int flags, struct rwlock **lockp); #if VM_NRESERVLEVEL > 0 static void pmap_pv_promote_pde(pmap_t pmap, vm_offset_t va, vm_paddr_t pa, struct rwlock **lockp); #endif static void pmap_pvh_free(struct md_page *pvh, pmap_t pmap, vm_offset_t va); static pv_entry_t pmap_pvh_remove(struct md_page *pvh, pmap_t pmap, vm_offset_t va); static void pmap_abort_ptp(pmap_t pmap, vm_offset_t va, vm_page_t mpte); static int pmap_change_props_locked(vm_offset_t va, vm_size_t size, vm_prot_t prot, int mode, int flags); static boolean_t pmap_demote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va); static boolean_t pmap_demote_pde_locked(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, struct rwlock **lockp); static boolean_t pmap_demote_pdpe(pmap_t pmap, pdp_entry_t *pdpe, vm_offset_t va); static bool pmap_enter_2mpage(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, struct rwlock **lockp); static int pmap_enter_pde(pmap_t pmap, vm_offset_t va, pd_entry_t newpde, u_int flags, vm_page_t m, struct rwlock **lockp); static vm_page_t pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, vm_page_t mpte, struct rwlock **lockp); static void pmap_fill_ptp(pt_entry_t *firstpte, pt_entry_t newpte); static int pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte, bool promoted); static void pmap_invalidate_cache_range_selfsnoop(vm_offset_t sva, vm_offset_t eva); static void pmap_invalidate_cache_range_all(vm_offset_t sva, vm_offset_t eva); static void pmap_invalidate_pde_page(pmap_t pmap, vm_offset_t va, pd_entry_t pde); static void pmap_kenter_attr(vm_offset_t va, vm_paddr_t pa, int mode); static vm_page_t pmap_large_map_getptp_unlocked(void); static vm_paddr_t pmap_large_map_kextract(vm_offset_t va); #if VM_NRESERVLEVEL > 0 static void pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, struct rwlock **lockp); #endif static boolean_t pmap_protect_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t sva, vm_prot_t prot); static void pmap_pte_props(pt_entry_t *pte, u_long bits, u_long mask); static void pmap_pti_add_kva_locked(vm_offset_t sva, vm_offset_t eva, bool exec); static pdp_entry_t *pmap_pti_pdpe(vm_offset_t va); static pd_entry_t *pmap_pti_pde(vm_offset_t va); static void pmap_pti_wire_pte(void *pte); static int pmap_remove_pde(pmap_t pmap, pd_entry_t *pdq, vm_offset_t sva, struct spglist *free, struct rwlock **lockp); static int pmap_remove_pte(pmap_t pmap, pt_entry_t *ptq, vm_offset_t sva, pd_entry_t ptepde, struct spglist *free, struct rwlock **lockp); static vm_page_t pmap_remove_pt_page(pmap_t pmap, vm_offset_t va); static void pmap_remove_page(pmap_t pmap, vm_offset_t va, pd_entry_t *pde, struct spglist *free); static bool pmap_remove_ptes(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, pd_entry_t *pde, struct spglist *free, struct rwlock **lockp); static boolean_t pmap_try_insert_pv_entry(pmap_t pmap, vm_offset_t va, vm_page_t m, struct rwlock **lockp); static void pmap_update_pde(pmap_t pmap, vm_offset_t va, pd_entry_t *pde, pd_entry_t newpde); static void pmap_update_pde_invalidate(pmap_t, vm_offset_t va, pd_entry_t pde); static vm_page_t _pmap_allocpte(pmap_t pmap, vm_pindex_t ptepindex, struct rwlock **lockp); static pd_entry_t *pmap_alloc_pde(pmap_t pmap, vm_offset_t va, vm_page_t *pdpgp, struct rwlock **lockp); static vm_page_t pmap_allocpte(pmap_t pmap, vm_offset_t va, struct rwlock **lockp); static void _pmap_unwire_ptp(pmap_t pmap, vm_offset_t va, vm_page_t m, struct spglist *free); static int pmap_unuse_pt(pmap_t, vm_offset_t, pd_entry_t, struct spglist *); /********************/ /* Inline functions */ /********************/ /* Return a non-clipped PD index for a given VA */ static __inline vm_pindex_t pmap_pde_pindex(vm_offset_t va) { return (va >> PDRSHIFT); } /* Return a pointer to the PML4 slot that corresponds to a VA */ static __inline pml4_entry_t * pmap_pml4e(pmap_t pmap, vm_offset_t va) { return (&pmap->pm_pml4[pmap_pml4e_index(va)]); } /* Return a pointer to the PDP slot that corresponds to a VA */ static __inline pdp_entry_t * pmap_pml4e_to_pdpe(pml4_entry_t *pml4e, vm_offset_t va) { pdp_entry_t *pdpe; pdpe = (pdp_entry_t *)PHYS_TO_DMAP(*pml4e & PG_FRAME); return (&pdpe[pmap_pdpe_index(va)]); } /* Return a pointer to the PDP slot that corresponds to a VA */ static __inline pdp_entry_t * pmap_pdpe(pmap_t pmap, vm_offset_t va) { pml4_entry_t *pml4e; pt_entry_t PG_V; PG_V = pmap_valid_bit(pmap); pml4e = pmap_pml4e(pmap, va); if ((*pml4e & PG_V) == 0) return (NULL); return (pmap_pml4e_to_pdpe(pml4e, va)); } /* Return a pointer to the PD slot that corresponds to a VA */ static __inline pd_entry_t * pmap_pdpe_to_pde(pdp_entry_t *pdpe, vm_offset_t va) { pd_entry_t *pde; pde = (pd_entry_t *)PHYS_TO_DMAP(*pdpe & PG_FRAME); return (&pde[pmap_pde_index(va)]); } /* Return a pointer to the PD slot that corresponds to a VA */ static __inline pd_entry_t * pmap_pde(pmap_t pmap, vm_offset_t va) { pdp_entry_t *pdpe; pt_entry_t PG_V; PG_V = pmap_valid_bit(pmap); pdpe = pmap_pdpe(pmap, va); if (pdpe == NULL || (*pdpe & PG_V) == 0) return (NULL); return (pmap_pdpe_to_pde(pdpe, va)); } /* Return a pointer to the PT slot that corresponds to a VA */ static __inline pt_entry_t * pmap_pde_to_pte(pd_entry_t *pde, vm_offset_t va) { pt_entry_t *pte; pte = (pt_entry_t *)PHYS_TO_DMAP(*pde & PG_FRAME); return (&pte[pmap_pte_index(va)]); } /* Return a pointer to the PT slot that corresponds to a VA */ static __inline pt_entry_t * pmap_pte(pmap_t pmap, vm_offset_t va) { pd_entry_t *pde; pt_entry_t PG_V; PG_V = pmap_valid_bit(pmap); pde = pmap_pde(pmap, va); if (pde == NULL || (*pde & PG_V) == 0) return (NULL); if ((*pde & PG_PS) != 0) /* compat with i386 pmap_pte() */ return ((pt_entry_t *)pde); return (pmap_pde_to_pte(pde, va)); } static __inline void pmap_resident_count_inc(pmap_t pmap, int count) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); pmap->pm_stats.resident_count += count; } static __inline void pmap_resident_count_dec(pmap_t pmap, int count) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); KASSERT(pmap->pm_stats.resident_count >= count, ("pmap %p resident count underflow %ld %d", pmap, pmap->pm_stats.resident_count, count)); pmap->pm_stats.resident_count -= count; } PMAP_INLINE pt_entry_t * vtopte(vm_offset_t va) { u_int64_t mask = ((1ul << (NPTEPGSHIFT + NPDEPGSHIFT + NPDPEPGSHIFT + NPML4EPGSHIFT)) - 1); KASSERT(va >= VM_MAXUSER_ADDRESS, ("vtopte on a uva/gpa 0x%0lx", va)); return (PTmap + ((va >> PAGE_SHIFT) & mask)); } static __inline pd_entry_t * vtopde(vm_offset_t va) { u_int64_t mask = ((1ul << (NPDEPGSHIFT + NPDPEPGSHIFT + NPML4EPGSHIFT)) - 1); KASSERT(va >= VM_MAXUSER_ADDRESS, ("vtopde on a uva/gpa 0x%0lx", va)); return (PDmap + ((va >> PDRSHIFT) & mask)); } static u_int64_t allocpages(vm_paddr_t *firstaddr, int n) { u_int64_t ret; ret = *firstaddr; bzero((void *)ret, n * PAGE_SIZE); *firstaddr += n * PAGE_SIZE; return (ret); } CTASSERT(powerof2(NDMPML4E)); /* number of kernel PDP slots */ #define NKPDPE(ptpgs) howmany(ptpgs, NPDEPG) static void nkpt_init(vm_paddr_t addr) { int pt_pages; #ifdef NKPT pt_pages = NKPT; #else pt_pages = howmany(addr, 1 << PDRSHIFT); pt_pages += NKPDPE(pt_pages); /* * Add some slop beyond the bare minimum required for bootstrapping * the kernel. * * This is quite important when allocating KVA for kernel modules. * The modules are required to be linked in the negative 2GB of * the address space. If we run out of KVA in this region then * pmap_growkernel() will need to allocate page table pages to map * the entire 512GB of KVA space which is an unnecessary tax on * physical memory. * * Secondly, device memory mapped as part of setting up the low- * level console(s) is taken from KVA, starting at virtual_avail. * This is because cninit() is called after pmap_bootstrap() but * before vm_init() and pmap_init(). 20MB for a frame buffer is * not uncommon. */ pt_pages += 32; /* 64MB additional slop. */ #endif nkpt = pt_pages; } /* * Returns the proper write/execute permission for a physical page that is * part of the initial boot allocations. * * If the page has kernel text, it is marked as read-only. If the page has * kernel read-only data, it is marked as read-only/not-executable. If the * page has only read-write data, it is marked as read-write/not-executable. * If the page is below/above the kernel range, it is marked as read-write. * * This function operates on 2M pages, since we map the kernel space that * way. */ static inline pt_entry_t bootaddr_rwx(vm_paddr_t pa) { /* * The kernel is loaded at a 2MB-aligned address, and memory below that * need not be executable. The .bss section is padded to a 2MB * boundary, so memory following the kernel need not be executable * either. Preloaded kernel modules have their mapping permissions * fixed up by the linker. */ if (pa < trunc_2mpage(btext - KERNBASE) || pa >= trunc_2mpage(_end - KERNBASE)) return (X86_PG_RW | pg_nx); /* * The linker should ensure that the read-only and read-write * portions don't share the same 2M page, so this shouldn't * impact read-only data. However, in any case, any page with * read-write data needs to be read-write. */ if (pa >= trunc_2mpage(brwsection - KERNBASE)) return (X86_PG_RW | pg_nx); /* * Mark any 2M page containing kernel text as read-only. Mark * other pages with read-only data as read-only and not executable. * (It is likely a small portion of the read-only data section will * be marked as read-only, but executable. This should be acceptable * since the read-only protection will keep the data from changing.) * Note that fixups to the .text section will still work until we * set CR0.WP. */ if (pa < round_2mpage(etext - KERNBASE)) return (0); return (pg_nx); } static void create_pagetables(vm_paddr_t *firstaddr) { int i, j, ndm1g, nkpdpe, nkdmpde; pd_entry_t *pd_p; pdp_entry_t *pdp_p; pml4_entry_t *p4_p; uint64_t DMPDkernphys; /* Allocate page table pages for the direct map */ ndmpdp = howmany(ptoa(Maxmem), NBPDP); if (ndmpdp < 4) /* Minimum 4GB of dirmap */ ndmpdp = 4; ndmpdpphys = howmany(ndmpdp, NPDPEPG); if (ndmpdpphys > NDMPML4E) { /* * Each NDMPML4E allows 512 GB, so limit to that, * and then readjust ndmpdp and ndmpdpphys. */ printf("NDMPML4E limits system to %d GB\n", NDMPML4E * 512); Maxmem = atop(NDMPML4E * NBPML4); ndmpdpphys = NDMPML4E; ndmpdp = NDMPML4E * NPDEPG; } DMPDPphys = allocpages(firstaddr, ndmpdpphys); ndm1g = 0; if ((amd_feature & AMDID_PAGE1GB) != 0) { /* * Calculate the number of 1G pages that will fully fit in * Maxmem. */ ndm1g = ptoa(Maxmem) >> PDPSHIFT; /* * Allocate 2M pages for the kernel. These will be used in * place of the first one or more 1G pages from ndm1g. */ nkdmpde = howmany((vm_offset_t)(brwsection - KERNBASE), NBPDP); DMPDkernphys = allocpages(firstaddr, nkdmpde); } if (ndm1g < ndmpdp) DMPDphys = allocpages(firstaddr, ndmpdp - ndm1g); dmaplimit = (vm_paddr_t)ndmpdp << PDPSHIFT; /* Allocate pages */ KPML4phys = allocpages(firstaddr, 1); KPDPphys = allocpages(firstaddr, NKPML4E); /* * Allocate the initial number of kernel page table pages required to * bootstrap. We defer this until after all memory-size dependent * allocations are done (e.g. direct map), so that we don't have to * build in too much slop in our estimate. * * Note that when NKPML4E > 1, we have an empty page underneath * all but the KPML4I'th one, so we need NKPML4E-1 extra (zeroed) * pages. (pmap_enter requires a PD page to exist for each KPML4E.) */ nkpt_init(*firstaddr); nkpdpe = NKPDPE(nkpt); KPTphys = allocpages(firstaddr, nkpt); KPDphys = allocpages(firstaddr, nkpdpe); /* * Connect the zero-filled PT pages to their PD entries. This * implicitly maps the PT pages at their correct locations within * the PTmap. */ pd_p = (pd_entry_t *)KPDphys; for (i = 0; i < nkpt; i++) pd_p[i] = (KPTphys + ptoa(i)) | X86_PG_RW | X86_PG_V; /* * Map from physical address zero to the end of loader preallocated * memory using 2MB pages. This replaces some of the PD entries * created above. */ for (i = 0; (i << PDRSHIFT) < KERNend; i++) /* Preset PG_M and PG_A because demotion expects it. */ pd_p[i] = (i << PDRSHIFT) | X86_PG_V | PG_PS | pg_g | X86_PG_M | X86_PG_A | bootaddr_rwx(i << PDRSHIFT); /* * Because we map the physical blocks in 2M pages, adjust firstaddr * to record the physical blocks we've actually mapped into kernel * virtual address space. */ if (*firstaddr < round_2mpage(KERNend)) *firstaddr = round_2mpage(KERNend); /* And connect up the PD to the PDP (leaving room for L4 pages) */ pdp_p = (pdp_entry_t *)(KPDPphys + ptoa(KPML4I - KPML4BASE)); for (i = 0; i < nkpdpe; i++) pdp_p[i + KPDPI] = (KPDphys + ptoa(i)) | X86_PG_RW | X86_PG_V; /* * Now, set up the direct map region using 2MB and/or 1GB pages. If * the end of physical memory is not aligned to a 1GB page boundary, * then the residual physical memory is mapped with 2MB pages. Later, * if pmap_mapdev{_attr}() uses the direct map for non-write-back * memory, pmap_change_attr() will demote any 2MB or 1GB page mappings * that are partially used. */ pd_p = (pd_entry_t *)DMPDphys; for (i = NPDEPG * ndm1g, j = 0; i < NPDEPG * ndmpdp; i++, j++) { pd_p[j] = (vm_paddr_t)i << PDRSHIFT; /* Preset PG_M and PG_A because demotion expects it. */ pd_p[j] |= X86_PG_RW | X86_PG_V | PG_PS | pg_g | X86_PG_M | X86_PG_A | pg_nx; } pdp_p = (pdp_entry_t *)DMPDPphys; for (i = 0; i < ndm1g; i++) { pdp_p[i] = (vm_paddr_t)i << PDPSHIFT; /* Preset PG_M and PG_A because demotion expects it. */ pdp_p[i] |= X86_PG_RW | X86_PG_V | PG_PS | pg_g | X86_PG_M | X86_PG_A | pg_nx; } for (j = 0; i < ndmpdp; i++, j++) { pdp_p[i] = DMPDphys + ptoa(j); pdp_p[i] |= X86_PG_RW | X86_PG_V | pg_nx; } /* * Instead of using a 1G page for the memory containing the kernel, * use 2M pages with read-only and no-execute permissions. (If using 1G * pages, this will partially overwrite the PDPEs above.) */ if (ndm1g) { pd_p = (pd_entry_t *)DMPDkernphys; for (i = 0; i < (NPDEPG * nkdmpde); i++) pd_p[i] = (i << PDRSHIFT) | X86_PG_V | PG_PS | pg_g | X86_PG_M | X86_PG_A | pg_nx | bootaddr_rwx(i << PDRSHIFT); for (i = 0; i < nkdmpde; i++) pdp_p[i] = (DMPDkernphys + ptoa(i)) | X86_PG_RW | X86_PG_V | pg_nx; } /* And recursively map PML4 to itself in order to get PTmap */ p4_p = (pml4_entry_t *)KPML4phys; p4_p[PML4PML4I] = KPML4phys; p4_p[PML4PML4I] |= X86_PG_RW | X86_PG_V | pg_nx; /* Connect the Direct Map slot(s) up to the PML4. */ for (i = 0; i < ndmpdpphys; i++) { p4_p[DMPML4I + i] = DMPDPphys + ptoa(i); p4_p[DMPML4I + i] |= X86_PG_RW | X86_PG_V | pg_nx; } /* Connect the KVA slots up to the PML4 */ for (i = 0; i < NKPML4E; i++) { p4_p[KPML4BASE + i] = KPDPphys + ptoa(i); p4_p[KPML4BASE + i] |= X86_PG_RW | X86_PG_V; } } /* * Bootstrap the system enough to run with virtual memory. * * On amd64 this is called after mapping has already been enabled * and just syncs the pmap module with what has already been done. * [We can't call it easily with mapping off since the kernel is not * mapped with PA == VA, hence we would have to relocate every address * from the linked base (virtual) address "KERNBASE" to the actual * (physical) address starting relative to 0] */ void pmap_bootstrap(vm_paddr_t *firstaddr) { vm_offset_t va; pt_entry_t *pte, *pcpu_pte; struct region_descriptor r_gdt; uint64_t cr4, pcpu_phys; u_long res; int i; KERNend = *firstaddr; res = atop(KERNend - (vm_paddr_t)kernphys); if (!pti) pg_g = X86_PG_G; /* * Create an initial set of page tables to run the kernel in. */ create_pagetables(firstaddr); pcpu_phys = allocpages(firstaddr, MAXCPU); /* * Add a physical memory segment (vm_phys_seg) corresponding to the * preallocated kernel page table pages so that vm_page structures * representing these pages will be created. The vm_page structures * are required for promotion of the corresponding kernel virtual * addresses to superpage mappings. */ vm_phys_add_seg(KPTphys, KPTphys + ptoa(nkpt)); /* * Account for the virtual addresses mapped by create_pagetables(). */ virtual_avail = (vm_offset_t)KERNBASE + round_2mpage(KERNend); virtual_end = VM_MAX_KERNEL_ADDRESS; /* * Enable PG_G global pages, then switch to the kernel page * table from the bootstrap page table. After the switch, it * is possible to enable SMEP and SMAP since PG_U bits are * correct now. */ cr4 = rcr4(); cr4 |= CR4_PGE; load_cr4(cr4); load_cr3(KPML4phys); if (cpu_stdext_feature & CPUID_STDEXT_SMEP) cr4 |= CR4_SMEP; if (cpu_stdext_feature & CPUID_STDEXT_SMAP) cr4 |= CR4_SMAP; load_cr4(cr4); /* * Initialize the kernel pmap (which is statically allocated). * Count bootstrap data as being resident in case any of this data is * later unmapped (using pmap_remove()) and freed. */ PMAP_LOCK_INIT(kernel_pmap); kernel_pmap->pm_pml4 = (pdp_entry_t *)PHYS_TO_DMAP(KPML4phys); kernel_pmap->pm_cr3 = KPML4phys; kernel_pmap->pm_ucr3 = PMAP_NO_CR3; CPU_FILL(&kernel_pmap->pm_active); /* don't allow deactivation */ TAILQ_INIT(&kernel_pmap->pm_pvchunk); kernel_pmap->pm_stats.resident_count = res; kernel_pmap->pm_flags = pmap_flags; /* * Initialize the TLB invalidations generation number lock. */ mtx_init(&invl_gen_mtx, "invlgn", NULL, MTX_DEF); /* * Reserve some special page table entries/VA space for temporary * mapping of pages. */ #define SYSMAP(c, p, v, n) \ v = (c)va; va += ((n)*PAGE_SIZE); p = pte; pte += (n); va = virtual_avail; pte = vtopte(va); /* * Crashdump maps. The first page is reused as CMAP1 for the * memory test. */ SYSMAP(caddr_t, CMAP1, crashdumpmap, MAXDUMPPGS) CADDR1 = crashdumpmap; SYSMAP(struct pcpu *, pcpu_pte, __pcpu, MAXCPU); virtual_avail = va; for (i = 0; i < MAXCPU; i++) { pcpu_pte[i] = (pcpu_phys + ptoa(i)) | X86_PG_V | X86_PG_RW | pg_g | pg_nx | X86_PG_M | X86_PG_A; } /* * Re-initialize PCPU area for BSP after switching. * Make hardware use gdt and common_tss from the new PCPU. */ STAILQ_INIT(&cpuhead); wrmsr(MSR_GSBASE, (uint64_t)&__pcpu[0]); pcpu_init(&__pcpu[0], 0, sizeof(struct pcpu)); amd64_bsp_pcpu_init1(&__pcpu[0]); amd64_bsp_ist_init(&__pcpu[0]); __pcpu[0].pc_common_tss.tss_iobase = sizeof(struct amd64tss) + IOPERM_BITMAP_SIZE; memcpy(__pcpu[0].pc_gdt, temp_bsp_pcpu.pc_gdt, NGDT * sizeof(struct user_segment_descriptor)); gdt_segs[GPROC0_SEL].ssd_base = (uintptr_t)&__pcpu[0].pc_common_tss; ssdtosyssd(&gdt_segs[GPROC0_SEL], (struct system_segment_descriptor *)&__pcpu[0].pc_gdt[GPROC0_SEL]); r_gdt.rd_limit = NGDT * sizeof(struct user_segment_descriptor) - 1; r_gdt.rd_base = (long)__pcpu[0].pc_gdt; lgdt(&r_gdt); wrmsr(MSR_GSBASE, (uint64_t)&__pcpu[0]); ltr(GSEL(GPROC0_SEL, SEL_KPL)); __pcpu[0].pc_dynamic = temp_bsp_pcpu.pc_dynamic; __pcpu[0].pc_acpi_id = temp_bsp_pcpu.pc_acpi_id; /* * Initialize the PAT MSR. * pmap_init_pat() clears and sets CR4_PGE, which, as a * side-effect, invalidates stale PG_G TLB entries that might * have been created in our pre-boot environment. */ pmap_init_pat(); /* Initialize TLB Context Id. */ if (pmap_pcid_enabled) { for (i = 0; i < MAXCPU; i++) { kernel_pmap->pm_pcids[i].pm_pcid = PMAP_PCID_KERN; kernel_pmap->pm_pcids[i].pm_gen = 1; } /* * PMAP_PCID_KERN + 1 is used for initialization of * proc0 pmap. The pmap' pcid state might be used by * EFIRT entry before first context switch, so it * needs to be valid. */ PCPU_SET(pcid_next, PMAP_PCID_KERN + 2); PCPU_SET(pcid_gen, 1); /* * pcpu area for APs is zeroed during AP startup. * pc_pcid_next and pc_pcid_gen are initialized by AP * during pcpu setup. */ load_cr4(rcr4() | CR4_PCIDE); } } /* * Setup the PAT MSR. */ void pmap_init_pat(void) { uint64_t pat_msr; u_long cr0, cr4; int i; /* Bail if this CPU doesn't implement PAT. */ if ((cpu_feature & CPUID_PAT) == 0) panic("no PAT??"); /* Set default PAT index table. */ for (i = 0; i < PAT_INDEX_SIZE; i++) pat_index[i] = -1; pat_index[PAT_WRITE_BACK] = 0; pat_index[PAT_WRITE_THROUGH] = 1; pat_index[PAT_UNCACHEABLE] = 3; pat_index[PAT_WRITE_COMBINING] = 6; pat_index[PAT_WRITE_PROTECTED] = 5; pat_index[PAT_UNCACHED] = 2; /* * Initialize default PAT entries. * Leave the indices 0-3 at the default of WB, WT, UC-, and UC. * Program 5 and 6 as WP and WC. * * Leave 4 and 7 as WB and UC. Note that a recursive page table * mapping for a 2M page uses a PAT value with the bit 3 set due * to its overload with PG_PS. */ pat_msr = PAT_VALUE(0, PAT_WRITE_BACK) | PAT_VALUE(1, PAT_WRITE_THROUGH) | PAT_VALUE(2, PAT_UNCACHED) | PAT_VALUE(3, PAT_UNCACHEABLE) | PAT_VALUE(4, PAT_WRITE_BACK) | PAT_VALUE(5, PAT_WRITE_PROTECTED) | PAT_VALUE(6, PAT_WRITE_COMBINING) | PAT_VALUE(7, PAT_UNCACHEABLE); /* Disable PGE. */ cr4 = rcr4(); load_cr4(cr4 & ~CR4_PGE); /* Disable caches (CD = 1, NW = 0). */ cr0 = rcr0(); load_cr0((cr0 & ~CR0_NW) | CR0_CD); /* Flushes caches and TLBs. */ wbinvd(); invltlb(); /* Update PAT and index table. */ wrmsr(MSR_PAT, pat_msr); /* Flush caches and TLBs again. */ wbinvd(); invltlb(); /* Restore caches and PGE. */ load_cr0(cr0); load_cr4(cr4); } /* * Initialize a vm_page's machine-dependent fields. */ void pmap_page_init(vm_page_t m) { TAILQ_INIT(&m->md.pv_list); m->md.pat_mode = PAT_WRITE_BACK; } static int pmap_allow_2m_x_ept; SYSCTL_INT(_vm_pmap, OID_AUTO, allow_2m_x_ept, CTLFLAG_RWTUN | CTLFLAG_NOFETCH, &pmap_allow_2m_x_ept, 0, "Allow executable superpage mappings in EPT"); void pmap_allow_2m_x_ept_recalculate(void) { /* * SKL002, SKL012S. Since the EPT format is only used by * Intel CPUs, the vendor check is merely a formality. */ if (!(cpu_vendor_id != CPU_VENDOR_INTEL || (cpu_ia32_arch_caps & IA32_ARCH_CAP_IF_PSCHANGE_MC_NO) != 0 || (CPUID_TO_FAMILY(cpu_id) == 0x6 && (CPUID_TO_MODEL(cpu_id) == 0x26 || /* Atoms */ CPUID_TO_MODEL(cpu_id) == 0x27 || CPUID_TO_MODEL(cpu_id) == 0x35 || CPUID_TO_MODEL(cpu_id) == 0x36 || CPUID_TO_MODEL(cpu_id) == 0x37 || CPUID_TO_MODEL(cpu_id) == 0x86 || CPUID_TO_MODEL(cpu_id) == 0x1c || CPUID_TO_MODEL(cpu_id) == 0x4a || CPUID_TO_MODEL(cpu_id) == 0x4c || CPUID_TO_MODEL(cpu_id) == 0x4d || CPUID_TO_MODEL(cpu_id) == 0x5a || CPUID_TO_MODEL(cpu_id) == 0x5c || CPUID_TO_MODEL(cpu_id) == 0x5d || CPUID_TO_MODEL(cpu_id) == 0x5f || CPUID_TO_MODEL(cpu_id) == 0x6e || CPUID_TO_MODEL(cpu_id) == 0x7a || CPUID_TO_MODEL(cpu_id) == 0x57 || /* Knights */ CPUID_TO_MODEL(cpu_id) == 0x85)))) pmap_allow_2m_x_ept = 1; TUNABLE_INT_FETCH("hw.allow_2m_x_ept", &pmap_allow_2m_x_ept); } static bool pmap_allow_2m_x_page(pmap_t pmap, bool executable) { return (pmap->pm_type != PT_EPT || !executable || !pmap_allow_2m_x_ept); } #ifdef NUMA static void pmap_init_pv_table(void) { struct pmap_large_md_page *pvd; vm_size_t s; long start, end, highest, pv_npg; int domain, i, j, pages; /* * We strongly depend on the size being a power of two, so the assert * is overzealous. However, should the struct be resized to a * different power of two, the code below needs to be revisited. */ CTASSERT((sizeof(*pvd) == 64)); /* * Calculate the size of the array. */ pmap_last_pa = vm_phys_segs[vm_phys_nsegs - 1].end; pv_npg = howmany(pmap_last_pa, NBPDR); s = (vm_size_t)pv_npg * sizeof(struct pmap_large_md_page); s = round_page(s); pv_table = (struct pmap_large_md_page *)kva_alloc(s); if (pv_table == NULL) panic("%s: kva_alloc failed\n", __func__); /* * Iterate physical segments to allocate space for respective pages. */ highest = -1; s = 0; for (i = 0; i < vm_phys_nsegs; i++) { end = vm_phys_segs[i].end / NBPDR; domain = vm_phys_segs[i].domain; if (highest >= end) continue; start = highest + 1; pvd = &pv_table[start]; pages = end - start + 1; s = round_page(pages * sizeof(*pvd)); highest = start + (s / sizeof(*pvd)) - 1; for (j = 0; j < s; j += PAGE_SIZE) { vm_page_t m = vm_page_alloc_domain(NULL, 0, domain, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ); if (m == NULL) panic("vm_page_alloc_domain failed for %lx\n", (vm_offset_t)pvd + j); pmap_qenter((vm_offset_t)pvd + j, &m, 1); } for (j = 0; j < s / sizeof(*pvd); j++) { rw_init_flags(&pvd->pv_lock, "pmap pv list", RW_NEW); TAILQ_INIT(&pvd->pv_page.pv_list); pvd->pv_page.pv_gen = 0; pvd->pv_page.pat_mode = 0; pvd->pv_invl_gen = 0; pvd++; } } pvd = &pv_dummy_large; rw_init_flags(&pvd->pv_lock, "pmap pv list dummy", RW_NEW); TAILQ_INIT(&pvd->pv_page.pv_list); pvd->pv_page.pv_gen = 0; pvd->pv_page.pat_mode = 0; pvd->pv_invl_gen = 0; } #else static void pmap_init_pv_table(void) { vm_size_t s; long i, pv_npg; /* * Initialize the pool of pv list locks. */ for (i = 0; i < NPV_LIST_LOCKS; i++) rw_init(&pv_list_locks[i], "pmap pv list"); /* * Calculate the size of the pv head table for superpages. */ pv_npg = howmany(vm_phys_segs[vm_phys_nsegs - 1].end, NBPDR); /* * Allocate memory for the pv head table for superpages. */ s = (vm_size_t)pv_npg * sizeof(struct md_page); s = round_page(s); pv_table = (struct md_page *)kmem_malloc(s, M_WAITOK | M_ZERO); for (i = 0; i < pv_npg; i++) TAILQ_INIT(&pv_table[i].pv_list); TAILQ_INIT(&pv_dummy.pv_list); } #endif /* * Initialize the pmap module. * Called by vm_init, to initialize any structures that the pmap * system needs to map virtual memory. */ void pmap_init(void) { struct pmap_preinit_mapping *ppim; vm_page_t m, mpte; int error, i, ret, skz63; /* L1TF, reserve page @0 unconditionally */ vm_page_blacklist_add(0, bootverbose); /* Detect bare-metal Skylake Server and Skylake-X. */ if (vm_guest == VM_GUEST_NO && cpu_vendor_id == CPU_VENDOR_INTEL && CPUID_TO_FAMILY(cpu_id) == 0x6 && CPUID_TO_MODEL(cpu_id) == 0x55) { /* * Skylake-X errata SKZ63. Processor May Hang When * Executing Code In an HLE Transaction Region between * 40000000H and 403FFFFFH. * * Mark the pages in the range as preallocated. It * seems to be impossible to distinguish between * Skylake Server and Skylake X. */ skz63 = 1; TUNABLE_INT_FETCH("hw.skz63_enable", &skz63); if (skz63 != 0) { if (bootverbose) printf("SKZ63: skipping 4M RAM starting " "at physical 1G\n"); for (i = 0; i < atop(0x400000); i++) { ret = vm_page_blacklist_add(0x40000000 + ptoa(i), FALSE); if (!ret && bootverbose) printf("page at %#lx already used\n", 0x40000000 + ptoa(i)); } } } /* IFU */ pmap_allow_2m_x_ept_recalculate(); /* * Initialize the vm page array entries for the kernel pmap's * page table pages. */ PMAP_LOCK(kernel_pmap); for (i = 0; i < nkpt; i++) { mpte = PHYS_TO_VM_PAGE(KPTphys + (i << PAGE_SHIFT)); KASSERT(mpte >= vm_page_array && mpte < &vm_page_array[vm_page_array_size], ("pmap_init: page table page is out of range")); mpte->pindex = pmap_pde_pindex(KERNBASE) + i; mpte->phys_addr = KPTphys + (i << PAGE_SHIFT); mpte->ref_count = 1; /* * Collect the page table pages that were replaced by a 2MB * page in create_pagetables(). They are zero filled. */ if (i << PDRSHIFT < KERNend && pmap_insert_pt_page(kernel_pmap, mpte, false)) panic("pmap_init: pmap_insert_pt_page failed"); } PMAP_UNLOCK(kernel_pmap); vm_wire_add(nkpt); /* * If the kernel is running on a virtual machine, then it must assume * that MCA is enabled by the hypervisor. Moreover, the kernel must * be prepared for the hypervisor changing the vendor and family that * are reported by CPUID. Consequently, the workaround for AMD Family * 10h Erratum 383 is enabled if the processor's feature set does not * include at least one feature that is only supported by older Intel * or newer AMD processors. */ if (vm_guest != VM_GUEST_NO && (cpu_feature & CPUID_SS) == 0 && (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI | CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP | AMDID2_FMA4)) == 0) workaround_erratum383 = 1; /* * Are large page mappings enabled? */ TUNABLE_INT_FETCH("vm.pmap.pg_ps_enabled", &pg_ps_enabled); if (pg_ps_enabled) { KASSERT(MAXPAGESIZES > 1 && pagesizes[1] == 0, ("pmap_init: can't assign to pagesizes[1]")); pagesizes[1] = NBPDR; } /* * Initialize pv chunk lists. */ for (i = 0; i < PMAP_MEMDOM; i++) { mtx_init(&pv_chunks[i].pvc_lock, "pmap pv chunk list", NULL, MTX_DEF); TAILQ_INIT(&pv_chunks[i].pvc_list); } pmap_init_pv_table(); pmap_initialized = 1; for (i = 0; i < PMAP_PREINIT_MAPPING_COUNT; i++) { ppim = pmap_preinit_mapping + i; if (ppim->va == 0) continue; /* Make the direct map consistent */ if (ppim->pa < dmaplimit && ppim->pa + ppim->sz <= dmaplimit) { (void)pmap_change_attr(PHYS_TO_DMAP(ppim->pa), ppim->sz, ppim->mode); } if (!bootverbose) continue; printf("PPIM %u: PA=%#lx, VA=%#lx, size=%#lx, mode=%#x\n", i, ppim->pa, ppim->va, ppim->sz, ppim->mode); } mtx_init(&qframe_mtx, "qfrmlk", NULL, MTX_SPIN); error = vmem_alloc(kernel_arena, PAGE_SIZE, M_BESTFIT | M_WAITOK, (vmem_addr_t *)&qframe); if (error != 0) panic("qframe allocation failed"); lm_ents = 8; TUNABLE_INT_FETCH("vm.pmap.large_map_pml4_entries", &lm_ents); if (lm_ents > LMEPML4I - LMSPML4I + 1) lm_ents = LMEPML4I - LMSPML4I + 1; if (bootverbose) printf("pmap: large map %u PML4 slots (%lu GB)\n", lm_ents, (u_long)lm_ents * (NBPML4 / 1024 / 1024 / 1024)); if (lm_ents != 0) { large_vmem = vmem_create("large", LARGEMAP_MIN_ADDRESS, (vmem_size_t)lm_ents * NBPML4, PAGE_SIZE, 0, M_WAITOK); if (large_vmem == NULL) { printf("pmap: cannot create large map\n"); lm_ents = 0; } for (i = 0; i < lm_ents; i++) { m = pmap_large_map_getptp_unlocked(); kernel_pmap->pm_pml4[LMSPML4I + i] = X86_PG_V | X86_PG_RW | X86_PG_A | X86_PG_M | pg_nx | VM_PAGE_TO_PHYS(m); } } } SYSCTL_UINT(_vm_pmap, OID_AUTO, large_map_pml4_entries, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, &lm_ents, 0, "Maximum number of PML4 entries for use by large map (tunable). " "Each entry corresponds to 512GB of address space."); static SYSCTL_NODE(_vm_pmap, OID_AUTO, pde, CTLFLAG_RD, 0, "2MB page mapping counters"); static u_long pmap_pde_demotions; SYSCTL_ULONG(_vm_pmap_pde, OID_AUTO, demotions, CTLFLAG_RD, &pmap_pde_demotions, 0, "2MB page demotions"); static u_long pmap_pde_mappings; SYSCTL_ULONG(_vm_pmap_pde, OID_AUTO, mappings, CTLFLAG_RD, &pmap_pde_mappings, 0, "2MB page mappings"); static u_long pmap_pde_p_failures; SYSCTL_ULONG(_vm_pmap_pde, OID_AUTO, p_failures, CTLFLAG_RD, &pmap_pde_p_failures, 0, "2MB page promotion failures"); static u_long pmap_pde_promotions; SYSCTL_ULONG(_vm_pmap_pde, OID_AUTO, promotions, CTLFLAG_RD, &pmap_pde_promotions, 0, "2MB page promotions"); static SYSCTL_NODE(_vm_pmap, OID_AUTO, pdpe, CTLFLAG_RD, 0, "1GB page mapping counters"); static u_long pmap_pdpe_demotions; SYSCTL_ULONG(_vm_pmap_pdpe, OID_AUTO, demotions, CTLFLAG_RD, &pmap_pdpe_demotions, 0, "1GB page demotions"); /*************************************************** * Low level helper routines..... ***************************************************/ static pt_entry_t pmap_swap_pat(pmap_t pmap, pt_entry_t entry) { int x86_pat_bits = X86_PG_PTE_PAT | X86_PG_PDE_PAT; switch (pmap->pm_type) { case PT_X86: case PT_RVI: /* Verify that both PAT bits are not set at the same time */ KASSERT((entry & x86_pat_bits) != x86_pat_bits, ("Invalid PAT bits in entry %#lx", entry)); /* Swap the PAT bits if one of them is set */ if ((entry & x86_pat_bits) != 0) entry ^= x86_pat_bits; break; case PT_EPT: /* * Nothing to do - the memory attributes are represented * the same way for regular pages and superpages. */ break; default: panic("pmap_switch_pat_bits: bad pm_type %d", pmap->pm_type); } return (entry); } boolean_t pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) { return (mode >= 0 && mode < PAT_INDEX_SIZE && pat_index[(int)mode] >= 0); } /* * Determine the appropriate bits to set in a PTE or PDE for a specified * caching mode. */ int pmap_cache_bits(pmap_t pmap, int mode, boolean_t is_pde) { int cache_bits, pat_flag, pat_idx; if (!pmap_is_valid_memattr(pmap, mode)) panic("Unknown caching mode %d\n", mode); switch (pmap->pm_type) { case PT_X86: case PT_RVI: /* The PAT bit is different for PTE's and PDE's. */ pat_flag = is_pde ? X86_PG_PDE_PAT : X86_PG_PTE_PAT; /* Map the caching mode to a PAT index. */ pat_idx = pat_index[mode]; /* Map the 3-bit index value into the PAT, PCD, and PWT bits. */ cache_bits = 0; if (pat_idx & 0x4) cache_bits |= pat_flag; if (pat_idx & 0x2) cache_bits |= PG_NC_PCD; if (pat_idx & 0x1) cache_bits |= PG_NC_PWT; break; case PT_EPT: cache_bits = EPT_PG_IGNORE_PAT | EPT_PG_MEMORY_TYPE(mode); break; default: panic("unsupported pmap type %d", pmap->pm_type); } return (cache_bits); } static int pmap_cache_mask(pmap_t pmap, boolean_t is_pde) { int mask; switch (pmap->pm_type) { case PT_X86: case PT_RVI: mask = is_pde ? X86_PG_PDE_CACHE : X86_PG_PTE_CACHE; break; case PT_EPT: mask = EPT_PG_IGNORE_PAT | EPT_PG_MEMORY_TYPE(0x7); break; default: panic("pmap_cache_mask: invalid pm_type %d", pmap->pm_type); } return (mask); } static int pmap_pat_index(pmap_t pmap, pt_entry_t pte, bool is_pde) { int pat_flag, pat_idx; pat_idx = 0; switch (pmap->pm_type) { case PT_X86: case PT_RVI: /* The PAT bit is different for PTE's and PDE's. */ pat_flag = is_pde ? X86_PG_PDE_PAT : X86_PG_PTE_PAT; if ((pte & pat_flag) != 0) pat_idx |= 0x4; if ((pte & PG_NC_PCD) != 0) pat_idx |= 0x2; if ((pte & PG_NC_PWT) != 0) pat_idx |= 0x1; break; case PT_EPT: if ((pte & EPT_PG_IGNORE_PAT) != 0) panic("EPT PTE %#lx has no PAT memory type", pte); pat_idx = (pte & EPT_PG_MEMORY_TYPE(0x7)) >> 3; break; } /* See pmap_init_pat(). */ if (pat_idx == 4) pat_idx = 0; if (pat_idx == 7) pat_idx = 3; return (pat_idx); } bool pmap_ps_enabled(pmap_t pmap) { return (pg_ps_enabled && (pmap->pm_flags & PMAP_PDE_SUPERPAGE) != 0); } static void pmap_update_pde_store(pmap_t pmap, pd_entry_t *pde, pd_entry_t newpde) { switch (pmap->pm_type) { case PT_X86: break; case PT_RVI: case PT_EPT: /* * XXX * This is a little bogus since the generation number is * supposed to be bumped up when a region of the address * space is invalidated in the page tables. * * In this case the old PDE entry is valid but yet we want * to make sure that any mappings using the old entry are * invalidated in the TLB. * * The reason this works as expected is because we rendezvous * "all" host cpus and force any vcpu context to exit as a * side-effect. */ atomic_add_acq_long(&pmap->pm_eptgen, 1); break; default: panic("pmap_update_pde_store: bad pm_type %d", pmap->pm_type); } pde_store(pde, newpde); } /* * After changing the page size for the specified virtual address in the page * table, flush the corresponding entries from the processor's TLB. Only the * calling processor's TLB is affected. * * The calling thread must be pinned to a processor. */ static void pmap_update_pde_invalidate(pmap_t pmap, vm_offset_t va, pd_entry_t newpde) { pt_entry_t PG_G; if (pmap_type_guest(pmap)) return; KASSERT(pmap->pm_type == PT_X86, ("pmap_update_pde_invalidate: invalid type %d", pmap->pm_type)); PG_G = pmap_global_bit(pmap); if ((newpde & PG_PS) == 0) /* Demotion: flush a specific 2MB page mapping. */ invlpg(va); else if ((newpde & PG_G) == 0) /* * Promotion: flush every 4KB page mapping from the TLB * because there are too many to flush individually. */ invltlb(); else { /* * Promotion: flush every 4KB page mapping from the TLB, * including any global (PG_G) mappings. */ invltlb_glob(); } } #ifdef SMP /* * For SMP, these functions have to use the IPI mechanism for coherence. * * N.B.: Before calling any of the following TLB invalidation functions, * the calling processor must ensure that all stores updating a non- * kernel page table are globally performed. Otherwise, another * processor could cache an old, pre-update entry without being * invalidated. This can happen one of two ways: (1) The pmap becomes * active on another processor after its pm_active field is checked by * one of the following functions but before a store updating the page * table is globally performed. (2) The pmap becomes active on another * processor before its pm_active field is checked but due to * speculative loads one of the following functions stills reads the * pmap as inactive on the other processor. * * The kernel page table is exempt because its pm_active field is * immutable. The kernel page table is always active on every * processor. */ /* * Interrupt the cpus that are executing in the guest context. * This will force the vcpu to exit and the cached EPT mappings * will be invalidated by the host before the next vmresume. */ static __inline void pmap_invalidate_ept(pmap_t pmap) { int ipinum; sched_pin(); KASSERT(!CPU_ISSET(curcpu, &pmap->pm_active), ("pmap_invalidate_ept: absurd pm_active")); /* * The TLB mappings associated with a vcpu context are not * flushed each time a different vcpu is chosen to execute. * * This is in contrast with a process's vtop mappings that * are flushed from the TLB on each context switch. * * Therefore we need to do more than just a TLB shootdown on * the active cpus in 'pmap->pm_active'. To do this we keep * track of the number of invalidations performed on this pmap. * * Each vcpu keeps a cache of this counter and compares it * just before a vmresume. If the counter is out-of-date an * invept will be done to flush stale mappings from the TLB. */ atomic_add_acq_long(&pmap->pm_eptgen, 1); /* * Force the vcpu to exit and trap back into the hypervisor. */ ipinum = pmap->pm_flags & PMAP_NESTED_IPIMASK; ipi_selected(pmap->pm_active, ipinum); sched_unpin(); } static cpuset_t pmap_invalidate_cpu_mask(pmap_t pmap) { return (pmap == kernel_pmap ? all_cpus : pmap->pm_active); } static inline void pmap_invalidate_page_pcid(pmap_t pmap, vm_offset_t va, const bool invpcid_works1) { struct invpcid_descr d; uint64_t kcr3, ucr3; uint32_t pcid; u_int cpuid, i; cpuid = PCPU_GET(cpuid); if (pmap == PCPU_GET(curpmap)) { if (pmap->pm_ucr3 != PMAP_NO_CR3) { /* * Because pm_pcid is recalculated on a * context switch, we must disable switching. * Otherwise, we might use a stale value * below. */ critical_enter(); pcid = pmap->pm_pcids[cpuid].pm_pcid; if (invpcid_works1) { d.pcid = pcid | PMAP_PCID_USER_PT; d.pad = 0; d.addr = va; invpcid(&d, INVPCID_ADDR); } else { kcr3 = pmap->pm_cr3 | pcid | CR3_PCID_SAVE; ucr3 = pmap->pm_ucr3 | pcid | PMAP_PCID_USER_PT | CR3_PCID_SAVE; pmap_pti_pcid_invlpg(ucr3, kcr3, va); } critical_exit(); } } else pmap->pm_pcids[cpuid].pm_gen = 0; CPU_FOREACH(i) { if (cpuid != i) pmap->pm_pcids[i].pm_gen = 0; } /* * The fence is between stores to pm_gen and the read of the * pm_active mask. We need to ensure that it is impossible * for us to miss the bit update in pm_active and * simultaneously observe a non-zero pm_gen in * pmap_activate_sw(), otherwise TLB update is missed. * Without the fence, IA32 allows such an outcome. Note that * pm_active is updated by a locked operation, which provides * the reciprocal fence. */ atomic_thread_fence_seq_cst(); } static void pmap_invalidate_page_pcid_invpcid(pmap_t pmap, vm_offset_t va) { pmap_invalidate_page_pcid(pmap, va, true); } static void pmap_invalidate_page_pcid_noinvpcid(pmap_t pmap, vm_offset_t va) { pmap_invalidate_page_pcid(pmap, va, false); } static void pmap_invalidate_page_nopcid(pmap_t pmap, vm_offset_t va) { } DEFINE_IFUNC(static, void, pmap_invalidate_page_mode, (pmap_t, vm_offset_t)) { if (pmap_pcid_enabled) return (invpcid_works ? pmap_invalidate_page_pcid_invpcid : pmap_invalidate_page_pcid_noinvpcid); return (pmap_invalidate_page_nopcid); } void pmap_invalidate_page(pmap_t pmap, vm_offset_t va) { if (pmap_type_guest(pmap)) { pmap_invalidate_ept(pmap); return; } KASSERT(pmap->pm_type == PT_X86, ("pmap_invalidate_page: invalid type %d", pmap->pm_type)); sched_pin(); if (pmap == kernel_pmap) { invlpg(va); } else { if (pmap == PCPU_GET(curpmap)) invlpg(va); pmap_invalidate_page_mode(pmap, va); } smp_masked_invlpg(pmap_invalidate_cpu_mask(pmap), va, pmap); sched_unpin(); } /* 4k PTEs -- Chosen to exceed the total size of Broadwell L2 TLB */ #define PMAP_INVLPG_THRESHOLD (4 * 1024 * PAGE_SIZE) static void pmap_invalidate_range_pcid(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, const bool invpcid_works1) { struct invpcid_descr d; uint64_t kcr3, ucr3; uint32_t pcid; u_int cpuid, i; cpuid = PCPU_GET(cpuid); if (pmap == PCPU_GET(curpmap)) { if (pmap->pm_ucr3 != PMAP_NO_CR3) { critical_enter(); pcid = pmap->pm_pcids[cpuid].pm_pcid; if (invpcid_works1) { d.pcid = pcid | PMAP_PCID_USER_PT; d.pad = 0; d.addr = sva; for (; d.addr < eva; d.addr += PAGE_SIZE) invpcid(&d, INVPCID_ADDR); } else { kcr3 = pmap->pm_cr3 | pcid | CR3_PCID_SAVE; ucr3 = pmap->pm_ucr3 | pcid | PMAP_PCID_USER_PT | CR3_PCID_SAVE; pmap_pti_pcid_invlrng(ucr3, kcr3, sva, eva); } critical_exit(); } } else pmap->pm_pcids[cpuid].pm_gen = 0; CPU_FOREACH(i) { if (cpuid != i) pmap->pm_pcids[i].pm_gen = 0; } /* See the comment in pmap_invalidate_page_pcid(). */ atomic_thread_fence_seq_cst(); } static void pmap_invalidate_range_pcid_invpcid(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { pmap_invalidate_range_pcid(pmap, sva, eva, true); } static void pmap_invalidate_range_pcid_noinvpcid(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { pmap_invalidate_range_pcid(pmap, sva, eva, false); } static void pmap_invalidate_range_nopcid(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { } DEFINE_IFUNC(static, void, pmap_invalidate_range_mode, (pmap_t, vm_offset_t, vm_offset_t)) { if (pmap_pcid_enabled) return (invpcid_works ? pmap_invalidate_range_pcid_invpcid : pmap_invalidate_range_pcid_noinvpcid); return (pmap_invalidate_range_nopcid); } void pmap_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { vm_offset_t addr; if (eva - sva >= PMAP_INVLPG_THRESHOLD) { pmap_invalidate_all(pmap); return; } if (pmap_type_guest(pmap)) { pmap_invalidate_ept(pmap); return; } KASSERT(pmap->pm_type == PT_X86, ("pmap_invalidate_range: invalid type %d", pmap->pm_type)); sched_pin(); if (pmap == kernel_pmap) { for (addr = sva; addr < eva; addr += PAGE_SIZE) invlpg(addr); } else { if (pmap == PCPU_GET(curpmap)) { for (addr = sva; addr < eva; addr += PAGE_SIZE) invlpg(addr); } pmap_invalidate_range_mode(pmap, sva, eva); } smp_masked_invlpg_range(pmap_invalidate_cpu_mask(pmap), sva, eva, pmap); sched_unpin(); } static inline void pmap_invalidate_all_pcid(pmap_t pmap, bool invpcid_works1) { struct invpcid_descr d; uint64_t kcr3, ucr3; uint32_t pcid; u_int cpuid, i; if (pmap == kernel_pmap) { if (invpcid_works1) { bzero(&d, sizeof(d)); invpcid(&d, INVPCID_CTXGLOB); } else { invltlb_glob(); } } else { cpuid = PCPU_GET(cpuid); if (pmap == PCPU_GET(curpmap)) { critical_enter(); pcid = pmap->pm_pcids[cpuid].pm_pcid; if (invpcid_works1) { d.pcid = pcid; d.pad = 0; d.addr = 0; invpcid(&d, INVPCID_CTX); if (pmap->pm_ucr3 != PMAP_NO_CR3) { d.pcid |= PMAP_PCID_USER_PT; invpcid(&d, INVPCID_CTX); } } else { kcr3 = pmap->pm_cr3 | pcid; ucr3 = pmap->pm_ucr3; if (ucr3 != PMAP_NO_CR3) { ucr3 |= pcid | PMAP_PCID_USER_PT; pmap_pti_pcid_invalidate(ucr3, kcr3); } else { load_cr3(kcr3); } } critical_exit(); } else pmap->pm_pcids[cpuid].pm_gen = 0; CPU_FOREACH(i) { if (cpuid != i) pmap->pm_pcids[i].pm_gen = 0; } } /* See the comment in pmap_invalidate_page_pcid(). */ atomic_thread_fence_seq_cst(); } static void pmap_invalidate_all_pcid_invpcid(pmap_t pmap) { pmap_invalidate_all_pcid(pmap, true); } static void pmap_invalidate_all_pcid_noinvpcid(pmap_t pmap) { pmap_invalidate_all_pcid(pmap, false); } static void pmap_invalidate_all_nopcid(pmap_t pmap) { if (pmap == kernel_pmap) invltlb_glob(); else if (pmap == PCPU_GET(curpmap)) invltlb(); } DEFINE_IFUNC(static, void, pmap_invalidate_all_mode, (pmap_t)) { if (pmap_pcid_enabled) return (invpcid_works ? pmap_invalidate_all_pcid_invpcid : pmap_invalidate_all_pcid_noinvpcid); return (pmap_invalidate_all_nopcid); } void pmap_invalidate_all(pmap_t pmap) { if (pmap_type_guest(pmap)) { pmap_invalidate_ept(pmap); return; } KASSERT(pmap->pm_type == PT_X86, ("pmap_invalidate_all: invalid type %d", pmap->pm_type)); sched_pin(); pmap_invalidate_all_mode(pmap); smp_masked_invltlb(pmap_invalidate_cpu_mask(pmap), pmap); sched_unpin(); } void pmap_invalidate_cache(void) { sched_pin(); wbinvd(); smp_cache_flush(); sched_unpin(); } struct pde_action { cpuset_t invalidate; /* processors that invalidate their TLB */ pmap_t pmap; vm_offset_t va; pd_entry_t *pde; pd_entry_t newpde; u_int store; /* processor that updates the PDE */ }; static void pmap_update_pde_action(void *arg) { struct pde_action *act = arg; if (act->store == PCPU_GET(cpuid)) pmap_update_pde_store(act->pmap, act->pde, act->newpde); } static void pmap_update_pde_teardown(void *arg) { struct pde_action *act = arg; if (CPU_ISSET(PCPU_GET(cpuid), &act->invalidate)) pmap_update_pde_invalidate(act->pmap, act->va, act->newpde); } /* * Change the page size for the specified virtual address in a way that * prevents any possibility of the TLB ever having two entries that map the * same virtual address using different page sizes. This is the recommended * workaround for Erratum 383 on AMD Family 10h processors. It prevents a * machine check exception for a TLB state that is improperly diagnosed as a * hardware error. */ static void pmap_update_pde(pmap_t pmap, vm_offset_t va, pd_entry_t *pde, pd_entry_t newpde) { struct pde_action act; cpuset_t active, other_cpus; u_int cpuid; sched_pin(); cpuid = PCPU_GET(cpuid); other_cpus = all_cpus; CPU_CLR(cpuid, &other_cpus); if (pmap == kernel_pmap || pmap_type_guest(pmap)) active = all_cpus; else { active = pmap->pm_active; } if (CPU_OVERLAP(&active, &other_cpus)) { act.store = cpuid; act.invalidate = active; act.va = va; act.pmap = pmap; act.pde = pde; act.newpde = newpde; CPU_SET(cpuid, &active); smp_rendezvous_cpus(active, smp_no_rendezvous_barrier, pmap_update_pde_action, pmap_update_pde_teardown, &act); } else { pmap_update_pde_store(pmap, pde, newpde); if (CPU_ISSET(cpuid, &active)) pmap_update_pde_invalidate(pmap, va, newpde); } sched_unpin(); } #else /* !SMP */ /* * Normal, non-SMP, invalidation functions. */ void pmap_invalidate_page(pmap_t pmap, vm_offset_t va) { struct invpcid_descr d; uint64_t kcr3, ucr3; uint32_t pcid; if (pmap->pm_type == PT_RVI || pmap->pm_type == PT_EPT) { pmap->pm_eptgen++; return; } KASSERT(pmap->pm_type == PT_X86, ("pmap_invalidate_range: unknown type %d", pmap->pm_type)); if (pmap == kernel_pmap || pmap == PCPU_GET(curpmap)) { invlpg(va); if (pmap == PCPU_GET(curpmap) && pmap_pcid_enabled && pmap->pm_ucr3 != PMAP_NO_CR3) { critical_enter(); pcid = pmap->pm_pcids[0].pm_pcid; if (invpcid_works) { d.pcid = pcid | PMAP_PCID_USER_PT; d.pad = 0; d.addr = va; invpcid(&d, INVPCID_ADDR); } else { kcr3 = pmap->pm_cr3 | pcid | CR3_PCID_SAVE; ucr3 = pmap->pm_ucr3 | pcid | PMAP_PCID_USER_PT | CR3_PCID_SAVE; pmap_pti_pcid_invlpg(ucr3, kcr3, va); } critical_exit(); } } else if (pmap_pcid_enabled) pmap->pm_pcids[0].pm_gen = 0; } void pmap_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { struct invpcid_descr d; vm_offset_t addr; uint64_t kcr3, ucr3; if (pmap->pm_type == PT_RVI || pmap->pm_type == PT_EPT) { pmap->pm_eptgen++; return; } KASSERT(pmap->pm_type == PT_X86, ("pmap_invalidate_range: unknown type %d", pmap->pm_type)); if (pmap == kernel_pmap || pmap == PCPU_GET(curpmap)) { for (addr = sva; addr < eva; addr += PAGE_SIZE) invlpg(addr); if (pmap == PCPU_GET(curpmap) && pmap_pcid_enabled && pmap->pm_ucr3 != PMAP_NO_CR3) { critical_enter(); if (invpcid_works) { d.pcid = pmap->pm_pcids[0].pm_pcid | PMAP_PCID_USER_PT; d.pad = 0; d.addr = sva; for (; d.addr < eva; d.addr += PAGE_SIZE) invpcid(&d, INVPCID_ADDR); } else { kcr3 = pmap->pm_cr3 | pmap->pm_pcids[0]. pm_pcid | CR3_PCID_SAVE; ucr3 = pmap->pm_ucr3 | pmap->pm_pcids[0]. pm_pcid | PMAP_PCID_USER_PT | CR3_PCID_SAVE; pmap_pti_pcid_invlrng(ucr3, kcr3, sva, eva); } critical_exit(); } } else if (pmap_pcid_enabled) { pmap->pm_pcids[0].pm_gen = 0; } } void pmap_invalidate_all(pmap_t pmap) { struct invpcid_descr d; uint64_t kcr3, ucr3; if (pmap->pm_type == PT_RVI || pmap->pm_type == PT_EPT) { pmap->pm_eptgen++; return; } KASSERT(pmap->pm_type == PT_X86, ("pmap_invalidate_all: unknown type %d", pmap->pm_type)); if (pmap == kernel_pmap) { if (pmap_pcid_enabled && invpcid_works) { bzero(&d, sizeof(d)); invpcid(&d, INVPCID_CTXGLOB); } else { invltlb_glob(); } } else if (pmap == PCPU_GET(curpmap)) { if (pmap_pcid_enabled) { critical_enter(); if (invpcid_works) { d.pcid = pmap->pm_pcids[0].pm_pcid; d.pad = 0; d.addr = 0; invpcid(&d, INVPCID_CTX); if (pmap->pm_ucr3 != PMAP_NO_CR3) { d.pcid |= PMAP_PCID_USER_PT; invpcid(&d, INVPCID_CTX); } } else { kcr3 = pmap->pm_cr3 | pmap->pm_pcids[0].pm_pcid; if (pmap->pm_ucr3 != PMAP_NO_CR3) { ucr3 = pmap->pm_ucr3 | pmap->pm_pcids[ 0].pm_pcid | PMAP_PCID_USER_PT; pmap_pti_pcid_invalidate(ucr3, kcr3); } else load_cr3(kcr3); } critical_exit(); } else { invltlb(); } } else if (pmap_pcid_enabled) { pmap->pm_pcids[0].pm_gen = 0; } } PMAP_INLINE void pmap_invalidate_cache(void) { wbinvd(); } static void pmap_update_pde(pmap_t pmap, vm_offset_t va, pd_entry_t *pde, pd_entry_t newpde) { pmap_update_pde_store(pmap, pde, newpde); if (pmap == kernel_pmap || pmap == PCPU_GET(curpmap)) pmap_update_pde_invalidate(pmap, va, newpde); else pmap->pm_pcids[0].pm_gen = 0; } #endif /* !SMP */ static void pmap_invalidate_pde_page(pmap_t pmap, vm_offset_t va, pd_entry_t pde) { /* * When the PDE has PG_PROMOTED set, the 2MB page mapping was created * by a promotion that did not invalidate the 512 4KB page mappings * that might exist in the TLB. Consequently, at this point, the TLB * may hold both 4KB and 2MB page mappings for the address range [va, * va + NBPDR). Therefore, the entire range must be invalidated here. * In contrast, when PG_PROMOTED is clear, the TLB will not hold any * 4KB page mappings for the address range [va, va + NBPDR), and so a * single INVLPG suffices to invalidate the 2MB page mapping from the * TLB. */ if ((pde & PG_PROMOTED) != 0) pmap_invalidate_range(pmap, va, va + NBPDR - 1); else pmap_invalidate_page(pmap, va); } DEFINE_IFUNC(, void, pmap_invalidate_cache_range, (vm_offset_t sva, vm_offset_t eva)) { if ((cpu_feature & CPUID_SS) != 0) return (pmap_invalidate_cache_range_selfsnoop); if ((cpu_feature & CPUID_CLFSH) != 0) return (pmap_force_invalidate_cache_range); return (pmap_invalidate_cache_range_all); } #define PMAP_CLFLUSH_THRESHOLD (2 * 1024 * 1024) static void pmap_invalidate_cache_range_check_align(vm_offset_t sva, vm_offset_t eva) { KASSERT((sva & PAGE_MASK) == 0, ("pmap_invalidate_cache_range: sva not page-aligned")); KASSERT((eva & PAGE_MASK) == 0, ("pmap_invalidate_cache_range: eva not page-aligned")); } static void pmap_invalidate_cache_range_selfsnoop(vm_offset_t sva, vm_offset_t eva) { pmap_invalidate_cache_range_check_align(sva, eva); } void pmap_force_invalidate_cache_range(vm_offset_t sva, vm_offset_t eva) { sva &= ~(vm_offset_t)(cpu_clflush_line_size - 1); /* * XXX: Some CPUs fault, hang, or trash the local APIC * registers if we use CLFLUSH on the local APIC range. The * local APIC is always uncached, so we don't need to flush * for that range anyway. */ if (pmap_kextract(sva) == lapic_paddr) return; if ((cpu_stdext_feature & CPUID_STDEXT_CLFLUSHOPT) != 0) { /* * Do per-cache line flush. Use a locked * instruction to insure that previous stores are * included in the write-back. The processor * propagates flush to other processors in the cache * coherence domain. */ atomic_thread_fence_seq_cst(); for (; sva < eva; sva += cpu_clflush_line_size) clflushopt(sva); atomic_thread_fence_seq_cst(); } else { /* * Writes are ordered by CLFLUSH on Intel CPUs. */ if (cpu_vendor_id != CPU_VENDOR_INTEL) mfence(); for (; sva < eva; sva += cpu_clflush_line_size) clflush(sva); if (cpu_vendor_id != CPU_VENDOR_INTEL) mfence(); } } static void pmap_invalidate_cache_range_all(vm_offset_t sva, vm_offset_t eva) { pmap_invalidate_cache_range_check_align(sva, eva); pmap_invalidate_cache(); } /* * Remove the specified set of pages from the data and instruction caches. * * In contrast to pmap_invalidate_cache_range(), this function does not * rely on the CPU's self-snoop feature, because it is intended for use * when moving pages into a different cache domain. */ void pmap_invalidate_cache_pages(vm_page_t *pages, int count) { vm_offset_t daddr, eva; int i; bool useclflushopt; useclflushopt = (cpu_stdext_feature & CPUID_STDEXT_CLFLUSHOPT) != 0; if (count >= PMAP_CLFLUSH_THRESHOLD / PAGE_SIZE || ((cpu_feature & CPUID_CLFSH) == 0 && !useclflushopt)) pmap_invalidate_cache(); else { if (useclflushopt) atomic_thread_fence_seq_cst(); else if (cpu_vendor_id != CPU_VENDOR_INTEL) mfence(); for (i = 0; i < count; i++) { daddr = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(pages[i])); eva = daddr + PAGE_SIZE; for (; daddr < eva; daddr += cpu_clflush_line_size) { if (useclflushopt) clflushopt(daddr); else clflush(daddr); } } if (useclflushopt) atomic_thread_fence_seq_cst(); else if (cpu_vendor_id != CPU_VENDOR_INTEL) mfence(); } } void pmap_flush_cache_range(vm_offset_t sva, vm_offset_t eva) { pmap_invalidate_cache_range_check_align(sva, eva); if ((cpu_stdext_feature & CPUID_STDEXT_CLWB) == 0) { pmap_force_invalidate_cache_range(sva, eva); return; } /* See comment in pmap_force_invalidate_cache_range(). */ if (pmap_kextract(sva) == lapic_paddr) return; atomic_thread_fence_seq_cst(); for (; sva < eva; sva += cpu_clflush_line_size) clwb(sva); atomic_thread_fence_seq_cst(); } void pmap_flush_cache_phys_range(vm_paddr_t spa, vm_paddr_t epa, vm_memattr_t mattr) { pt_entry_t *pte; vm_offset_t vaddr; int error, pte_bits; KASSERT((spa & PAGE_MASK) == 0, ("pmap_flush_cache_phys_range: spa not page-aligned")); KASSERT((epa & PAGE_MASK) == 0, ("pmap_flush_cache_phys_range: epa not page-aligned")); if (spa < dmaplimit) { pmap_flush_cache_range(PHYS_TO_DMAP(spa), PHYS_TO_DMAP(MIN( dmaplimit, epa))); if (dmaplimit >= epa) return; spa = dmaplimit; } pte_bits = pmap_cache_bits(kernel_pmap, mattr, 0) | X86_PG_RW | X86_PG_V; error = vmem_alloc(kernel_arena, PAGE_SIZE, M_BESTFIT | M_WAITOK, &vaddr); KASSERT(error == 0, ("vmem_alloc failed: %d", error)); pte = vtopte(vaddr); for (; spa < epa; spa += PAGE_SIZE) { sched_pin(); pte_store(pte, spa | pte_bits); invlpg(vaddr); /* XXXKIB atomic inside flush_cache_range are excessive */ pmap_flush_cache_range(vaddr, vaddr + PAGE_SIZE); sched_unpin(); } vmem_free(kernel_arena, vaddr, PAGE_SIZE); } /* * Routine: pmap_extract * Function: * Extract the physical page address associated * with the given map/virtual_address pair. */ vm_paddr_t pmap_extract(pmap_t pmap, vm_offset_t va) { pdp_entry_t *pdpe; pd_entry_t *pde; pt_entry_t *pte, PG_V; vm_paddr_t pa; pa = 0; PG_V = pmap_valid_bit(pmap); PMAP_LOCK(pmap); pdpe = pmap_pdpe(pmap, va); if (pdpe != NULL && (*pdpe & PG_V) != 0) { if ((*pdpe & PG_PS) != 0) pa = (*pdpe & PG_PS_FRAME) | (va & PDPMASK); else { pde = pmap_pdpe_to_pde(pdpe, va); if ((*pde & PG_V) != 0) { if ((*pde & PG_PS) != 0) { pa = (*pde & PG_PS_FRAME) | (va & PDRMASK); } else { pte = pmap_pde_to_pte(pde, va); pa = (*pte & PG_FRAME) | (va & PAGE_MASK); } } } } PMAP_UNLOCK(pmap); return (pa); } /* * Routine: pmap_extract_and_hold * Function: * Atomically extract and hold the physical page * with the given pmap and virtual address pair * if that mapping permits the given protection. */ vm_page_t pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, vm_prot_t prot) { pd_entry_t pde, *pdep; pt_entry_t pte, PG_RW, PG_V; vm_page_t m; m = NULL; PG_RW = pmap_rw_bit(pmap); PG_V = pmap_valid_bit(pmap); PMAP_LOCK(pmap); pdep = pmap_pde(pmap, va); if (pdep != NULL && (pde = *pdep)) { if (pde & PG_PS) { if ((pde & PG_RW) != 0 || (prot & VM_PROT_WRITE) == 0) m = PHYS_TO_VM_PAGE((pde & PG_PS_FRAME) | (va & PDRMASK)); } else { pte = *pmap_pde_to_pte(pdep, va); if ((pte & PG_V) != 0 && ((pte & PG_RW) != 0 || (prot & VM_PROT_WRITE) == 0)) m = PHYS_TO_VM_PAGE(pte & PG_FRAME); } if (m != NULL && !vm_page_wire_mapped(m)) m = NULL; } PMAP_UNLOCK(pmap); return (m); } vm_paddr_t pmap_kextract(vm_offset_t va) { pd_entry_t pde; vm_paddr_t pa; if (va >= DMAP_MIN_ADDRESS && va < DMAP_MAX_ADDRESS) { pa = DMAP_TO_PHYS(va); } else if (PMAP_ADDRESS_IN_LARGEMAP(va)) { pa = pmap_large_map_kextract(va); } else { pde = *vtopde(va); if (pde & PG_PS) { pa = (pde & PG_PS_FRAME) | (va & PDRMASK); } else { /* * Beware of a concurrent promotion that changes the * PDE at this point! For example, vtopte() must not * be used to access the PTE because it would use the * new PDE. It is, however, safe to use the old PDE * because the page table page is preserved by the * promotion. */ pa = *pmap_pde_to_pte(&pde, va); pa = (pa & PG_FRAME) | (va & PAGE_MASK); } } return (pa); } /*************************************************** * Low level mapping routines..... ***************************************************/ /* * Add a wired page to the kva. * Note: not SMP coherent. */ PMAP_INLINE void pmap_kenter(vm_offset_t va, vm_paddr_t pa) { pt_entry_t *pte; pte = vtopte(va); pte_store(pte, pa | X86_PG_RW | X86_PG_V | pg_g | pg_nx); } static __inline void pmap_kenter_attr(vm_offset_t va, vm_paddr_t pa, int mode) { pt_entry_t *pte; int cache_bits; pte = vtopte(va); cache_bits = pmap_cache_bits(kernel_pmap, mode, 0); pte_store(pte, pa | X86_PG_RW | X86_PG_V | pg_g | pg_nx | cache_bits); } /* * Remove a page from the kernel pagetables. * Note: not SMP coherent. */ PMAP_INLINE void pmap_kremove(vm_offset_t va) { pt_entry_t *pte; pte = vtopte(va); pte_clear(pte); } /* * Used to map a range of physical addresses into kernel * virtual address space. * * The value passed in '*virt' is a suggested virtual address for * the mapping. Architectures which can support a direct-mapped * physical to virtual region can return the appropriate address * within that region, leaving '*virt' unchanged. Other * architectures should map the pages starting at '*virt' and * update '*virt' with the first usable address after the mapped * region. */ vm_offset_t pmap_map(vm_offset_t *virt, vm_paddr_t start, vm_paddr_t end, int prot) { return PHYS_TO_DMAP(start); } /* * Add a list of wired pages to the kva * this routine is only used for temporary * kernel mappings that do not need to have * page modification or references recorded. * Note that old mappings are simply written * over. The page *must* be wired. * Note: SMP coherent. Uses a ranged shootdown IPI. */ void pmap_qenter(vm_offset_t sva, vm_page_t *ma, int count) { pt_entry_t *endpte, oldpte, pa, *pte; vm_page_t m; int cache_bits; oldpte = 0; pte = vtopte(sva); endpte = pte + count; while (pte < endpte) { m = *ma++; cache_bits = pmap_cache_bits(kernel_pmap, m->md.pat_mode, 0); pa = VM_PAGE_TO_PHYS(m) | cache_bits; if ((*pte & (PG_FRAME | X86_PG_PTE_CACHE)) != pa) { oldpte |= *pte; pte_store(pte, pa | pg_g | pg_nx | X86_PG_RW | X86_PG_V); } pte++; } if (__predict_false((oldpte & X86_PG_V) != 0)) pmap_invalidate_range(kernel_pmap, sva, sva + count * PAGE_SIZE); } /* * This routine tears out page mappings from the * kernel -- it is meant only for temporary mappings. * Note: SMP coherent. Uses a ranged shootdown IPI. */ void pmap_qremove(vm_offset_t sva, int count) { vm_offset_t va; va = sva; while (count-- > 0) { KASSERT(va >= VM_MIN_KERNEL_ADDRESS, ("usermode va %lx", va)); pmap_kremove(va); va += PAGE_SIZE; } pmap_invalidate_range(kernel_pmap, sva, va); } /*************************************************** * Page table page management routines..... ***************************************************/ /* * Schedule the specified unused page table page to be freed. Specifically, * add the page to the specified list of pages that will be released to the * physical memory manager after the TLB has been updated. */ static __inline void pmap_add_delayed_free_list(vm_page_t m, struct spglist *free, boolean_t set_PG_ZERO) { if (set_PG_ZERO) m->flags |= PG_ZERO; else m->flags &= ~PG_ZERO; SLIST_INSERT_HEAD(free, m, plinks.s.ss); } /* * Inserts the specified page table page into the specified pmap's collection * of idle page table pages. Each of a pmap's page table pages is responsible * for mapping a distinct range of virtual addresses. The pmap's collection is * ordered by this virtual address range. * * If "promoted" is false, then the page table page "mpte" must be zero filled. */ static __inline int pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte, bool promoted) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); mpte->valid = promoted ? VM_PAGE_BITS_ALL : 0; return (vm_radix_insert(&pmap->pm_root, mpte)); } /* * Removes the page table page mapping the specified virtual address from the * specified pmap's collection of idle page table pages, and returns it. * Otherwise, returns NULL if there is no page table page corresponding to the * specified virtual address. */ static __inline vm_page_t pmap_remove_pt_page(pmap_t pmap, vm_offset_t va) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); return (vm_radix_remove(&pmap->pm_root, pmap_pde_pindex(va))); } /* * Decrements a page table page's reference count, which is used to record the * number of valid page table entries within the page. If the reference count * drops to zero, then the page table page is unmapped. Returns TRUE if the * page table page was unmapped and FALSE otherwise. */ static inline boolean_t pmap_unwire_ptp(pmap_t pmap, vm_offset_t va, vm_page_t m, struct spglist *free) { --m->ref_count; if (m->ref_count == 0) { _pmap_unwire_ptp(pmap, va, m, free); return (TRUE); } else return (FALSE); } static void _pmap_unwire_ptp(pmap_t pmap, vm_offset_t va, vm_page_t m, struct spglist *free) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); /* * unmap the page table page */ if (m->pindex >= NUPDE + NUPDPE) { /* PDP page */ pml4_entry_t *pml4; pml4 = pmap_pml4e(pmap, va); *pml4 = 0; if (pmap->pm_pml4u != NULL && va <= VM_MAXUSER_ADDRESS) { pml4 = &pmap->pm_pml4u[pmap_pml4e_index(va)]; *pml4 = 0; } } else if (m->pindex >= NUPDE) { /* PD page */ pdp_entry_t *pdp; pdp = pmap_pdpe(pmap, va); *pdp = 0; } else { /* PTE page */ pd_entry_t *pd; pd = pmap_pde(pmap, va); *pd = 0; } pmap_resident_count_dec(pmap, 1); if (m->pindex < NUPDE) { /* We just released a PT, unhold the matching PD */ vm_page_t pdpg; pdpg = PHYS_TO_VM_PAGE(*pmap_pdpe(pmap, va) & PG_FRAME); pmap_unwire_ptp(pmap, va, pdpg, free); } else if (m->pindex < NUPDE + NUPDPE) { /* We just released a PD, unhold the matching PDP */ vm_page_t pdppg; pdppg = PHYS_TO_VM_PAGE(*pmap_pml4e(pmap, va) & PG_FRAME); pmap_unwire_ptp(pmap, va, pdppg, free); } /* * Put page on a list so that it is released after * *ALL* TLB shootdown is done */ pmap_add_delayed_free_list(m, free, TRUE); } /* * After removing a page table entry, this routine is used to * conditionally free the page, and manage the reference count. */ static int pmap_unuse_pt(pmap_t pmap, vm_offset_t va, pd_entry_t ptepde, struct spglist *free) { vm_page_t mpte; if (va >= VM_MAXUSER_ADDRESS) return (0); KASSERT(ptepde != 0, ("pmap_unuse_pt: ptepde != 0")); mpte = PHYS_TO_VM_PAGE(ptepde & PG_FRAME); return (pmap_unwire_ptp(pmap, va, mpte, free)); } /* * Release a page table page reference after a failed attempt to create a * mapping. */ static void pmap_abort_ptp(pmap_t pmap, vm_offset_t va, vm_page_t mpte) { struct spglist free; SLIST_INIT(&free); if (pmap_unwire_ptp(pmap, va, mpte, &free)) { /* * Although "va" was never mapped, paging-structure caches * could nonetheless have entries that refer to the freed * page table pages. Invalidate those entries. */ pmap_invalidate_page(pmap, va); vm_page_free_pages_toq(&free, true); } } void pmap_pinit0(pmap_t pmap) { struct proc *p; struct thread *td; int i; PMAP_LOCK_INIT(pmap); pmap->pm_pml4 = (pml4_entry_t *)PHYS_TO_DMAP(KPML4phys); pmap->pm_pml4u = NULL; pmap->pm_cr3 = KPML4phys; /* hack to keep pmap_pti_pcid_invalidate() alive */ pmap->pm_ucr3 = PMAP_NO_CR3; pmap->pm_root.rt_root = 0; CPU_ZERO(&pmap->pm_active); TAILQ_INIT(&pmap->pm_pvchunk); bzero(&pmap->pm_stats, sizeof pmap->pm_stats); pmap->pm_flags = pmap_flags; CPU_FOREACH(i) { pmap->pm_pcids[i].pm_pcid = PMAP_PCID_KERN + 1; pmap->pm_pcids[i].pm_gen = 1; } pmap_activate_boot(pmap); td = curthread; if (pti) { p = td->td_proc; PROC_LOCK(p); p->p_md.md_flags |= P_MD_KPTI; PROC_UNLOCK(p); } pmap_thread_init_invl_gen(td); if ((cpu_stdext_feature2 & CPUID_STDEXT2_PKU) != 0) { pmap_pkru_ranges_zone = uma_zcreate("pkru ranges", sizeof(struct pmap_pkru_range), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); } } void pmap_pinit_pml4(vm_page_t pml4pg) { pml4_entry_t *pm_pml4; int i; pm_pml4 = (pml4_entry_t *)PHYS_TO_DMAP(VM_PAGE_TO_PHYS(pml4pg)); /* Wire in kernel global address entries. */ for (i = 0; i < NKPML4E; i++) { pm_pml4[KPML4BASE + i] = (KPDPphys + ptoa(i)) | X86_PG_RW | X86_PG_V; } for (i = 0; i < ndmpdpphys; i++) { pm_pml4[DMPML4I + i] = (DMPDPphys + ptoa(i)) | X86_PG_RW | X86_PG_V; } /* install self-referential address mapping entry(s) */ pm_pml4[PML4PML4I] = VM_PAGE_TO_PHYS(pml4pg) | X86_PG_V | X86_PG_RW | X86_PG_A | X86_PG_M; /* install large map entries if configured */ for (i = 0; i < lm_ents; i++) pm_pml4[LMSPML4I + i] = kernel_pmap->pm_pml4[LMSPML4I + i]; } static void pmap_pinit_pml4_pti(vm_page_t pml4pg) { pml4_entry_t *pm_pml4; int i; pm_pml4 = (pml4_entry_t *)PHYS_TO_DMAP(VM_PAGE_TO_PHYS(pml4pg)); for (i = 0; i < NPML4EPG; i++) pm_pml4[i] = pti_pml4[i]; } /* * Initialize a preallocated and zeroed pmap structure, * such as one in a vmspace structure. */ int pmap_pinit_type(pmap_t pmap, enum pmap_type pm_type, int flags) { vm_page_t pml4pg, pml4pgu; vm_paddr_t pml4phys; int i; /* * allocate the page directory page */ pml4pg = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO | VM_ALLOC_WAITOK); pml4phys = VM_PAGE_TO_PHYS(pml4pg); pmap->pm_pml4 = (pml4_entry_t *)PHYS_TO_DMAP(pml4phys); CPU_FOREACH(i) { pmap->pm_pcids[i].pm_pcid = PMAP_PCID_NONE; pmap->pm_pcids[i].pm_gen = 0; } pmap->pm_cr3 = PMAP_NO_CR3; /* initialize to an invalid value */ pmap->pm_ucr3 = PMAP_NO_CR3; pmap->pm_pml4u = NULL; pmap->pm_type = pm_type; if ((pml4pg->flags & PG_ZERO) == 0) pagezero(pmap->pm_pml4); /* * Do not install the host kernel mappings in the nested page * tables. These mappings are meaningless in the guest physical * address space. * Install minimal kernel mappings in PTI case. */ if (pm_type == PT_X86) { pmap->pm_cr3 = pml4phys; pmap_pinit_pml4(pml4pg); if ((curproc->p_md.md_flags & P_MD_KPTI) != 0) { pml4pgu = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_WAITOK); pmap->pm_pml4u = (pml4_entry_t *)PHYS_TO_DMAP( VM_PAGE_TO_PHYS(pml4pgu)); pmap_pinit_pml4_pti(pml4pgu); pmap->pm_ucr3 = VM_PAGE_TO_PHYS(pml4pgu); } if ((cpu_stdext_feature2 & CPUID_STDEXT2_PKU) != 0) { rangeset_init(&pmap->pm_pkru, pkru_dup_range, pkru_free_range, pmap, M_NOWAIT); } } pmap->pm_root.rt_root = 0; CPU_ZERO(&pmap->pm_active); TAILQ_INIT(&pmap->pm_pvchunk); bzero(&pmap->pm_stats, sizeof pmap->pm_stats); pmap->pm_flags = flags; pmap->pm_eptgen = 0; return (1); } int pmap_pinit(pmap_t pmap) { return (pmap_pinit_type(pmap, PT_X86, pmap_flags)); } /* * This routine is called if the desired page table page does not exist. * * If page table page allocation fails, this routine may sleep before * returning NULL. It sleeps only if a lock pointer was given. * * Note: If a page allocation fails at page table level two or three, * one or two pages may be held during the wait, only to be released * afterwards. This conservative approach is easily argued to avoid * race conditions. */ static vm_page_t _pmap_allocpte(pmap_t pmap, vm_pindex_t ptepindex, struct rwlock **lockp) { vm_page_t m, pdppg, pdpg; pt_entry_t PG_A, PG_M, PG_RW, PG_V; PMAP_LOCK_ASSERT(pmap, MA_OWNED); PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); /* * Allocate a page table page. */ if ((m = vm_page_alloc(NULL, ptepindex, VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO)) == NULL) { if (lockp != NULL) { RELEASE_PV_LIST_LOCK(lockp); PMAP_UNLOCK(pmap); PMAP_ASSERT_NOT_IN_DI(); vm_wait(NULL); PMAP_LOCK(pmap); } /* * Indicate the need to retry. While waiting, the page table * page may have been allocated. */ return (NULL); } if ((m->flags & PG_ZERO) == 0) pmap_zero_page(m); /* * Map the pagetable page into the process address space, if * it isn't already there. */ if (ptepindex >= (NUPDE + NUPDPE)) { pml4_entry_t *pml4, *pml4u; vm_pindex_t pml4index; /* Wire up a new PDPE page */ pml4index = ptepindex - (NUPDE + NUPDPE); pml4 = &pmap->pm_pml4[pml4index]; *pml4 = VM_PAGE_TO_PHYS(m) | PG_U | PG_RW | PG_V | PG_A | PG_M; if (pmap->pm_pml4u != NULL && pml4index < NUPML4E) { /* * PTI: Make all user-space mappings in the * kernel-mode page table no-execute so that * we detect any programming errors that leave * the kernel-mode page table active on return * to user space. */ if (pmap->pm_ucr3 != PMAP_NO_CR3) *pml4 |= pg_nx; pml4u = &pmap->pm_pml4u[pml4index]; *pml4u = VM_PAGE_TO_PHYS(m) | PG_U | PG_RW | PG_V | PG_A | PG_M; } } else if (ptepindex >= NUPDE) { vm_pindex_t pml4index; vm_pindex_t pdpindex; pml4_entry_t *pml4; pdp_entry_t *pdp; /* Wire up a new PDE page */ pdpindex = ptepindex - NUPDE; pml4index = pdpindex >> NPML4EPGSHIFT; pml4 = &pmap->pm_pml4[pml4index]; if ((*pml4 & PG_V) == 0) { /* Have to allocate a new pdp, recurse */ if (_pmap_allocpte(pmap, NUPDE + NUPDPE + pml4index, lockp) == NULL) { vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } } else { /* Add reference to pdp page */ pdppg = PHYS_TO_VM_PAGE(*pml4 & PG_FRAME); pdppg->ref_count++; } pdp = (pdp_entry_t *)PHYS_TO_DMAP(*pml4 & PG_FRAME); /* Now find the pdp page */ pdp = &pdp[pdpindex & ((1ul << NPDPEPGSHIFT) - 1)]; *pdp = VM_PAGE_TO_PHYS(m) | PG_U | PG_RW | PG_V | PG_A | PG_M; } else { vm_pindex_t pml4index; vm_pindex_t pdpindex; pml4_entry_t *pml4; pdp_entry_t *pdp; pd_entry_t *pd; /* Wire up a new PTE page */ pdpindex = ptepindex >> NPDPEPGSHIFT; pml4index = pdpindex >> NPML4EPGSHIFT; /* First, find the pdp and check that its valid. */ pml4 = &pmap->pm_pml4[pml4index]; if ((*pml4 & PG_V) == 0) { /* Have to allocate a new pd, recurse */ if (_pmap_allocpte(pmap, NUPDE + pdpindex, lockp) == NULL) { vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } pdp = (pdp_entry_t *)PHYS_TO_DMAP(*pml4 & PG_FRAME); pdp = &pdp[pdpindex & ((1ul << NPDPEPGSHIFT) - 1)]; } else { pdp = (pdp_entry_t *)PHYS_TO_DMAP(*pml4 & PG_FRAME); pdp = &pdp[pdpindex & ((1ul << NPDPEPGSHIFT) - 1)]; if ((*pdp & PG_V) == 0) { /* Have to allocate a new pd, recurse */ if (_pmap_allocpte(pmap, NUPDE + pdpindex, lockp) == NULL) { vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } } else { /* Add reference to the pd page */ pdpg = PHYS_TO_VM_PAGE(*pdp & PG_FRAME); pdpg->ref_count++; } } pd = (pd_entry_t *)PHYS_TO_DMAP(*pdp & PG_FRAME); /* Now we know where the page directory page is */ pd = &pd[ptepindex & ((1ul << NPDEPGSHIFT) - 1)]; *pd = VM_PAGE_TO_PHYS(m) | PG_U | PG_RW | PG_V | PG_A | PG_M; } pmap_resident_count_inc(pmap, 1); return (m); } static pd_entry_t * pmap_alloc_pde(pmap_t pmap, vm_offset_t va, vm_page_t *pdpgp, struct rwlock **lockp) { pdp_entry_t *pdpe, PG_V; pd_entry_t *pde; vm_page_t pdpg; vm_pindex_t pdpindex; PG_V = pmap_valid_bit(pmap); retry: pdpe = pmap_pdpe(pmap, va); if (pdpe != NULL && (*pdpe & PG_V) != 0) { pde = pmap_pdpe_to_pde(pdpe, va); if (va < VM_MAXUSER_ADDRESS) { /* Add a reference to the pd page. */ pdpg = PHYS_TO_VM_PAGE(*pdpe & PG_FRAME); pdpg->ref_count++; } else pdpg = NULL; } else if (va < VM_MAXUSER_ADDRESS) { /* Allocate a pd page. */ pdpindex = pmap_pde_pindex(va) >> NPDPEPGSHIFT; pdpg = _pmap_allocpte(pmap, NUPDE + pdpindex, lockp); if (pdpg == NULL) { if (lockp != NULL) goto retry; else return (NULL); } pde = (pd_entry_t *)PHYS_TO_DMAP(VM_PAGE_TO_PHYS(pdpg)); pde = &pde[pmap_pde_index(va)]; } else panic("pmap_alloc_pde: missing page table page for va %#lx", va); *pdpgp = pdpg; return (pde); } static vm_page_t pmap_allocpte(pmap_t pmap, vm_offset_t va, struct rwlock **lockp) { vm_pindex_t ptepindex; pd_entry_t *pd, PG_V; vm_page_t m; PG_V = pmap_valid_bit(pmap); /* * Calculate pagetable page index */ ptepindex = pmap_pde_pindex(va); retry: /* * Get the page directory entry */ pd = pmap_pde(pmap, va); /* * This supports switching from a 2MB page to a * normal 4K page. */ if (pd != NULL && (*pd & (PG_PS | PG_V)) == (PG_PS | PG_V)) { if (!pmap_demote_pde_locked(pmap, pd, va, lockp)) { /* * Invalidation of the 2MB page mapping may have caused * the deallocation of the underlying PD page. */ pd = NULL; } } /* * If the page table page is mapped, we just increment the * hold count, and activate it. */ if (pd != NULL && (*pd & PG_V) != 0) { m = PHYS_TO_VM_PAGE(*pd & PG_FRAME); m->ref_count++; } else { /* * Here if the pte page isn't mapped, or if it has been * deallocated. */ m = _pmap_allocpte(pmap, ptepindex, lockp); if (m == NULL && lockp != NULL) goto retry; } return (m); } /*************************************************** * Pmap allocation/deallocation routines. ***************************************************/ /* * Release any resources held by the given physical map. * Called when a pmap initialized by pmap_pinit is being released. * Should only be called if the map contains no valid mappings. */ void pmap_release(pmap_t pmap) { vm_page_t m; int i; KASSERT(pmap->pm_stats.resident_count == 0, ("pmap_release: pmap resident count %ld != 0", pmap->pm_stats.resident_count)); KASSERT(vm_radix_is_empty(&pmap->pm_root), ("pmap_release: pmap has reserved page table page(s)")); KASSERT(CPU_EMPTY(&pmap->pm_active), ("releasing active pmap %p", pmap)); m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pmap->pm_pml4)); for (i = 0; i < NKPML4E; i++) /* KVA */ pmap->pm_pml4[KPML4BASE + i] = 0; for (i = 0; i < ndmpdpphys; i++)/* Direct Map */ pmap->pm_pml4[DMPML4I + i] = 0; pmap->pm_pml4[PML4PML4I] = 0; /* Recursive Mapping */ for (i = 0; i < lm_ents; i++) /* Large Map */ pmap->pm_pml4[LMSPML4I + i] = 0; vm_page_unwire_noq(m); vm_page_free_zero(m); if (pmap->pm_pml4u != NULL) { m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pmap->pm_pml4u)); vm_page_unwire_noq(m); vm_page_free(m); } if (pmap->pm_type == PT_X86 && (cpu_stdext_feature2 & CPUID_STDEXT2_PKU) != 0) rangeset_fini(&pmap->pm_pkru); } static int kvm_size(SYSCTL_HANDLER_ARGS) { unsigned long ksize = VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS; return sysctl_handle_long(oidp, &ksize, 0, req); } SYSCTL_PROC(_vm, OID_AUTO, kvm_size, CTLTYPE_LONG | CTLFLAG_RD | CTLFLAG_MPSAFE, 0, 0, kvm_size, "LU", "Size of KVM"); static int kvm_free(SYSCTL_HANDLER_ARGS) { unsigned long kfree = VM_MAX_KERNEL_ADDRESS - kernel_vm_end; return sysctl_handle_long(oidp, &kfree, 0, req); } SYSCTL_PROC(_vm, OID_AUTO, kvm_free, CTLTYPE_LONG | CTLFLAG_RD | CTLFLAG_MPSAFE, 0, 0, kvm_free, "LU", "Amount of KVM free"); /* * Allocate physical memory for the vm_page array and map it into KVA, * attempting to back the vm_pages with domain-local memory. */ void pmap_page_array_startup(long pages) { pdp_entry_t *pdpe; pd_entry_t *pde, newpdir; vm_offset_t va, start, end; vm_paddr_t pa; long pfn; int domain, i; vm_page_array_size = pages; start = VM_MIN_KERNEL_ADDRESS; end = start + pages * sizeof(struct vm_page); for (va = start; va < end; va += NBPDR) { pfn = first_page + (va - start) / sizeof(struct vm_page); domain = _vm_phys_domain(ptoa(pfn)); pdpe = pmap_pdpe(kernel_pmap, va); if ((*pdpe & X86_PG_V) == 0) { pa = vm_phys_early_alloc(domain, PAGE_SIZE); dump_add_page(pa); pagezero((void *)PHYS_TO_DMAP(pa)); *pdpe = (pdp_entry_t)(pa | X86_PG_V | X86_PG_RW | X86_PG_A | X86_PG_M); } pde = pmap_pdpe_to_pde(pdpe, va); if ((*pde & X86_PG_V) != 0) panic("Unexpected pde"); pa = vm_phys_early_alloc(domain, NBPDR); for (i = 0; i < NPDEPG; i++) dump_add_page(pa + i * PAGE_SIZE); newpdir = (pd_entry_t)(pa | X86_PG_V | X86_PG_RW | X86_PG_A | X86_PG_M | PG_PS | pg_g | pg_nx); pde_store(pde, newpdir); } vm_page_array = (vm_page_t)start; } /* * grow the number of kernel page table entries, if needed */ void pmap_growkernel(vm_offset_t addr) { vm_paddr_t paddr; vm_page_t nkpg; pd_entry_t *pde, newpdir; pdp_entry_t *pdpe; mtx_assert(&kernel_map->system_mtx, MA_OWNED); /* * Return if "addr" is within the range of kernel page table pages * that were preallocated during pmap bootstrap. Moreover, leave * "kernel_vm_end" and the kernel page table as they were. * * The correctness of this action is based on the following * argument: vm_map_insert() allocates contiguous ranges of the * kernel virtual address space. It calls this function if a range * ends after "kernel_vm_end". If the kernel is mapped between * "kernel_vm_end" and "addr", then the range cannot begin at * "kernel_vm_end". In fact, its beginning address cannot be less * than the kernel. Thus, there is no immediate need to allocate * any new kernel page table pages between "kernel_vm_end" and * "KERNBASE". */ if (KERNBASE < addr && addr <= KERNBASE + nkpt * NBPDR) return; addr = roundup2(addr, NBPDR); if (addr - 1 >= vm_map_max(kernel_map)) addr = vm_map_max(kernel_map); while (kernel_vm_end < addr) { pdpe = pmap_pdpe(kernel_pmap, kernel_vm_end); if ((*pdpe & X86_PG_V) == 0) { /* We need a new PDP entry */ nkpg = vm_page_alloc(NULL, kernel_vm_end >> PDPSHIFT, VM_ALLOC_INTERRUPT | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO); if (nkpg == NULL) panic("pmap_growkernel: no memory to grow kernel"); if ((nkpg->flags & PG_ZERO) == 0) pmap_zero_page(nkpg); paddr = VM_PAGE_TO_PHYS(nkpg); *pdpe = (pdp_entry_t)(paddr | X86_PG_V | X86_PG_RW | X86_PG_A | X86_PG_M); continue; /* try again */ } pde = pmap_pdpe_to_pde(pdpe, kernel_vm_end); if ((*pde & X86_PG_V) != 0) { kernel_vm_end = (kernel_vm_end + NBPDR) & ~PDRMASK; if (kernel_vm_end - 1 >= vm_map_max(kernel_map)) { kernel_vm_end = vm_map_max(kernel_map); break; } continue; } nkpg = vm_page_alloc(NULL, pmap_pde_pindex(kernel_vm_end), VM_ALLOC_INTERRUPT | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO); if (nkpg == NULL) panic("pmap_growkernel: no memory to grow kernel"); if ((nkpg->flags & PG_ZERO) == 0) pmap_zero_page(nkpg); paddr = VM_PAGE_TO_PHYS(nkpg); newpdir = paddr | X86_PG_V | X86_PG_RW | X86_PG_A | X86_PG_M; pde_store(pde, newpdir); kernel_vm_end = (kernel_vm_end + NBPDR) & ~PDRMASK; if (kernel_vm_end - 1 >= vm_map_max(kernel_map)) { kernel_vm_end = vm_map_max(kernel_map); break; } } } /*************************************************** * page management routines. ***************************************************/ CTASSERT(sizeof(struct pv_chunk) == PAGE_SIZE); CTASSERT(_NPCM == 3); CTASSERT(_NPCPV == 168); static __inline struct pv_chunk * pv_to_chunk(pv_entry_t pv) { return ((struct pv_chunk *)((uintptr_t)pv & ~(uintptr_t)PAGE_MASK)); } #define PV_PMAP(pv) (pv_to_chunk(pv)->pc_pmap) #define PC_FREE0 0xfffffffffffffffful #define PC_FREE1 0xfffffffffffffffful #define PC_FREE2 0x000000fffffffffful static const uint64_t pc_freemask[_NPCM] = { PC_FREE0, PC_FREE1, PC_FREE2 }; #ifdef PV_STATS static int pc_chunk_count, pc_chunk_allocs, pc_chunk_frees, pc_chunk_tryfail; SYSCTL_INT(_vm_pmap, OID_AUTO, pc_chunk_count, CTLFLAG_RD, &pc_chunk_count, 0, "Current number of pv entry chunks"); SYSCTL_INT(_vm_pmap, OID_AUTO, pc_chunk_allocs, CTLFLAG_RD, &pc_chunk_allocs, 0, "Current number of pv entry chunks allocated"); SYSCTL_INT(_vm_pmap, OID_AUTO, pc_chunk_frees, CTLFLAG_RD, &pc_chunk_frees, 0, "Current number of pv entry chunks frees"); SYSCTL_INT(_vm_pmap, OID_AUTO, pc_chunk_tryfail, CTLFLAG_RD, &pc_chunk_tryfail, 0, "Number of times tried to get a chunk page but failed."); static long pv_entry_frees, pv_entry_allocs, pv_entry_count; static int pv_entry_spare; SYSCTL_LONG(_vm_pmap, OID_AUTO, pv_entry_frees, CTLFLAG_RD, &pv_entry_frees, 0, "Current number of pv entry frees"); SYSCTL_LONG(_vm_pmap, OID_AUTO, pv_entry_allocs, CTLFLAG_RD, &pv_entry_allocs, 0, "Current number of pv entry allocs"); SYSCTL_LONG(_vm_pmap, OID_AUTO, pv_entry_count, CTLFLAG_RD, &pv_entry_count, 0, "Current number of pv entries"); SYSCTL_INT(_vm_pmap, OID_AUTO, pv_entry_spare, CTLFLAG_RD, &pv_entry_spare, 0, "Current number of spare pv entries"); #endif static void reclaim_pv_chunk_leave_pmap(pmap_t pmap, pmap_t locked_pmap, bool start_di) { if (pmap == NULL) return; pmap_invalidate_all(pmap); if (pmap != locked_pmap) PMAP_UNLOCK(pmap); if (start_di) pmap_delayed_invl_finish(); } /* * We are in a serious low memory condition. Resort to * drastic measures to free some pages so we can allocate * another pv entry chunk. * * Returns NULL if PV entries were reclaimed from the specified pmap. * * We do not, however, unmap 2mpages because subsequent accesses will * allocate per-page pv entries until repromotion occurs, thereby * exacerbating the shortage of free pv entries. */ static vm_page_t reclaim_pv_chunk_domain(pmap_t locked_pmap, struct rwlock **lockp, int domain) { struct pv_chunks_list *pvc; struct pv_chunk *pc, *pc_marker, *pc_marker_end; struct pv_chunk_header pc_marker_b, pc_marker_end_b; struct md_page *pvh; pd_entry_t *pde; pmap_t next_pmap, pmap; pt_entry_t *pte, tpte; pt_entry_t PG_G, PG_A, PG_M, PG_RW; pv_entry_t pv; vm_offset_t va; vm_page_t m, m_pc; struct spglist free; uint64_t inuse; int bit, field, freed; bool start_di; PMAP_LOCK_ASSERT(locked_pmap, MA_OWNED); KASSERT(lockp != NULL, ("reclaim_pv_chunk: lockp is NULL")); pmap = NULL; m_pc = NULL; PG_G = PG_A = PG_M = PG_RW = 0; SLIST_INIT(&free); bzero(&pc_marker_b, sizeof(pc_marker_b)); bzero(&pc_marker_end_b, sizeof(pc_marker_end_b)); pc_marker = (struct pv_chunk *)&pc_marker_b; pc_marker_end = (struct pv_chunk *)&pc_marker_end_b; /* * A delayed invalidation block should already be active if * pmap_advise() or pmap_remove() called this function by way * of pmap_demote_pde_locked(). */ start_di = pmap_not_in_di(); pvc = &pv_chunks[domain]; mtx_lock(&pvc->pvc_lock); pvc->active_reclaims++; TAILQ_INSERT_HEAD(&pvc->pvc_list, pc_marker, pc_lru); TAILQ_INSERT_TAIL(&pvc->pvc_list, pc_marker_end, pc_lru); while ((pc = TAILQ_NEXT(pc_marker, pc_lru)) != pc_marker_end && SLIST_EMPTY(&free)) { next_pmap = pc->pc_pmap; if (next_pmap == NULL) { /* * The next chunk is a marker. However, it is * not our marker, so active_reclaims must be * > 1. Consequently, the next_chunk code * will not rotate the pv_chunks list. */ goto next_chunk; } mtx_unlock(&pvc->pvc_lock); /* * A pv_chunk can only be removed from the pc_lru list * when both pc_chunks_mutex is owned and the * corresponding pmap is locked. */ if (pmap != next_pmap) { reclaim_pv_chunk_leave_pmap(pmap, locked_pmap, start_di); pmap = next_pmap; /* Avoid deadlock and lock recursion. */ if (pmap > locked_pmap) { RELEASE_PV_LIST_LOCK(lockp); PMAP_LOCK(pmap); if (start_di) pmap_delayed_invl_start(); mtx_lock(&pvc->pvc_lock); continue; } else if (pmap != locked_pmap) { if (PMAP_TRYLOCK(pmap)) { if (start_di) pmap_delayed_invl_start(); mtx_lock(&pvc->pvc_lock); continue; } else { pmap = NULL; /* pmap is not locked */ mtx_lock(&pvc->pvc_lock); pc = TAILQ_NEXT(pc_marker, pc_lru); if (pc == NULL || pc->pc_pmap != next_pmap) continue; goto next_chunk; } } else if (start_di) pmap_delayed_invl_start(); PG_G = pmap_global_bit(pmap); PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); } /* * Destroy every non-wired, 4 KB page mapping in the chunk. */ freed = 0; for (field = 0; field < _NPCM; field++) { for (inuse = ~pc->pc_map[field] & pc_freemask[field]; inuse != 0; inuse &= ~(1UL << bit)) { bit = bsfq(inuse); pv = &pc->pc_pventry[field * 64 + bit]; va = pv->pv_va; pde = pmap_pde(pmap, va); if ((*pde & PG_PS) != 0) continue; pte = pmap_pde_to_pte(pde, va); if ((*pte & PG_W) != 0) continue; tpte = pte_load_clear(pte); if ((tpte & PG_G) != 0) pmap_invalidate_page(pmap, va); m = PHYS_TO_VM_PAGE(tpte & PG_FRAME); if ((tpte & (PG_M | PG_RW)) == (PG_M | PG_RW)) vm_page_dirty(m); if ((tpte & PG_A) != 0) vm_page_aflag_set(m, PGA_REFERENCED); CHANGE_PV_LIST_LOCK_TO_VM_PAGE(lockp, m); TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; if (TAILQ_EMPTY(&m->md.pv_list) && (m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); if (TAILQ_EMPTY(&pvh->pv_list)) { vm_page_aflag_clear(m, PGA_WRITEABLE); } } pmap_delayed_invl_page(m); pc->pc_map[field] |= 1UL << bit; pmap_unuse_pt(pmap, va, *pde, &free); freed++; } } if (freed == 0) { mtx_lock(&pvc->pvc_lock); goto next_chunk; } /* Every freed mapping is for a 4 KB page. */ pmap_resident_count_dec(pmap, freed); PV_STAT(atomic_add_long(&pv_entry_frees, freed)); PV_STAT(atomic_add_int(&pv_entry_spare, freed)); PV_STAT(atomic_subtract_long(&pv_entry_count, freed)); TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); if (pc->pc_map[0] == PC_FREE0 && pc->pc_map[1] == PC_FREE1 && pc->pc_map[2] == PC_FREE2) { PV_STAT(atomic_subtract_int(&pv_entry_spare, _NPCPV)); PV_STAT(atomic_subtract_int(&pc_chunk_count, 1)); PV_STAT(atomic_add_int(&pc_chunk_frees, 1)); /* Entire chunk is free; return it. */ m_pc = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pc)); dump_drop_page(m_pc->phys_addr); mtx_lock(&pvc->pvc_lock); TAILQ_REMOVE(&pvc->pvc_list, pc, pc_lru); break; } TAILQ_INSERT_HEAD(&pmap->pm_pvchunk, pc, pc_list); mtx_lock(&pvc->pvc_lock); /* One freed pv entry in locked_pmap is sufficient. */ if (pmap == locked_pmap) break; next_chunk: TAILQ_REMOVE(&pvc->pvc_list, pc_marker, pc_lru); TAILQ_INSERT_AFTER(&pvc->pvc_list, pc, pc_marker, pc_lru); if (pvc->active_reclaims == 1 && pmap != NULL) { /* * Rotate the pv chunks list so that we do not * scan the same pv chunks that could not be * freed (because they contained a wired * and/or superpage mapping) on every * invocation of reclaim_pv_chunk(). */ while ((pc = TAILQ_FIRST(&pvc->pvc_list)) != pc_marker) { MPASS(pc->pc_pmap != NULL); TAILQ_REMOVE(&pvc->pvc_list, pc, pc_lru); TAILQ_INSERT_TAIL(&pvc->pvc_list, pc, pc_lru); } } } TAILQ_REMOVE(&pvc->pvc_list, pc_marker, pc_lru); TAILQ_REMOVE(&pvc->pvc_list, pc_marker_end, pc_lru); pvc->active_reclaims--; mtx_unlock(&pvc->pvc_lock); reclaim_pv_chunk_leave_pmap(pmap, locked_pmap, start_di); if (m_pc == NULL && !SLIST_EMPTY(&free)) { m_pc = SLIST_FIRST(&free); SLIST_REMOVE_HEAD(&free, plinks.s.ss); /* Recycle a freed page table page. */ m_pc->ref_count = 1; } vm_page_free_pages_toq(&free, true); return (m_pc); } static vm_page_t reclaim_pv_chunk(pmap_t locked_pmap, struct rwlock **lockp) { vm_page_t m; int i, domain; domain = PCPU_GET(domain); for (i = 0; i < vm_ndomains; i++) { m = reclaim_pv_chunk_domain(locked_pmap, lockp, domain); if (m != NULL) break; domain = (domain + 1) % vm_ndomains; } return (m); } /* * free the pv_entry back to the free list */ static void free_pv_entry(pmap_t pmap, pv_entry_t pv) { struct pv_chunk *pc; int idx, field, bit; PMAP_LOCK_ASSERT(pmap, MA_OWNED); PV_STAT(atomic_add_long(&pv_entry_frees, 1)); PV_STAT(atomic_add_int(&pv_entry_spare, 1)); PV_STAT(atomic_subtract_long(&pv_entry_count, 1)); pc = pv_to_chunk(pv); idx = pv - &pc->pc_pventry[0]; field = idx / 64; bit = idx % 64; pc->pc_map[field] |= 1ul << bit; if (pc->pc_map[0] != PC_FREE0 || pc->pc_map[1] != PC_FREE1 || pc->pc_map[2] != PC_FREE2) { /* 98% of the time, pc is already at the head of the list. */ if (__predict_false(pc != TAILQ_FIRST(&pmap->pm_pvchunk))) { TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_HEAD(&pmap->pm_pvchunk, pc, pc_list); } return; } TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); free_pv_chunk(pc); } static void free_pv_chunk_dequeued(struct pv_chunk *pc) { vm_page_t m; PV_STAT(atomic_subtract_int(&pv_entry_spare, _NPCPV)); PV_STAT(atomic_subtract_int(&pc_chunk_count, 1)); PV_STAT(atomic_add_int(&pc_chunk_frees, 1)); /* entire chunk is free, return it */ m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pc)); dump_drop_page(m->phys_addr); vm_page_unwire_noq(m); vm_page_free(m); } static void free_pv_chunk(struct pv_chunk *pc) { struct pv_chunks_list *pvc; pvc = &pv_chunks[pc_to_domain(pc)]; mtx_lock(&pvc->pvc_lock); TAILQ_REMOVE(&pvc->pvc_list, pc, pc_lru); mtx_unlock(&pvc->pvc_lock); free_pv_chunk_dequeued(pc); } static void free_pv_chunk_batch(struct pv_chunklist *batch) { struct pv_chunks_list *pvc; struct pv_chunk *pc, *npc; int i; for (i = 0; i < vm_ndomains; i++) { if (TAILQ_EMPTY(&batch[i])) continue; pvc = &pv_chunks[i]; mtx_lock(&pvc->pvc_lock); TAILQ_FOREACH(pc, &batch[i], pc_list) { TAILQ_REMOVE(&pvc->pvc_list, pc, pc_lru); } mtx_unlock(&pvc->pvc_lock); } for (i = 0; i < vm_ndomains; i++) { TAILQ_FOREACH_SAFE(pc, &batch[i], pc_list, npc) { free_pv_chunk_dequeued(pc); } } } /* * Returns a new PV entry, allocating a new PV chunk from the system when * needed. If this PV chunk allocation fails and a PV list lock pointer was * given, a PV chunk is reclaimed from an arbitrary pmap. Otherwise, NULL is * returned. * * The given PV list lock may be released. */ static pv_entry_t get_pv_entry(pmap_t pmap, struct rwlock **lockp) { struct pv_chunks_list *pvc; int bit, field; pv_entry_t pv; struct pv_chunk *pc; vm_page_t m; PMAP_LOCK_ASSERT(pmap, MA_OWNED); PV_STAT(atomic_add_long(&pv_entry_allocs, 1)); retry: pc = TAILQ_FIRST(&pmap->pm_pvchunk); if (pc != NULL) { for (field = 0; field < _NPCM; field++) { if (pc->pc_map[field]) { bit = bsfq(pc->pc_map[field]); break; } } if (field < _NPCM) { pv = &pc->pc_pventry[field * 64 + bit]; pc->pc_map[field] &= ~(1ul << bit); /* If this was the last item, move it to tail */ if (pc->pc_map[0] == 0 && pc->pc_map[1] == 0 && pc->pc_map[2] == 0) { TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_TAIL(&pmap->pm_pvchunk, pc, pc_list); } PV_STAT(atomic_add_long(&pv_entry_count, 1)); PV_STAT(atomic_subtract_int(&pv_entry_spare, 1)); return (pv); } } /* No free items, allocate another chunk */ m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED); if (m == NULL) { if (lockp == NULL) { PV_STAT(pc_chunk_tryfail++); return (NULL); } m = reclaim_pv_chunk(pmap, lockp); if (m == NULL) goto retry; } PV_STAT(atomic_add_int(&pc_chunk_count, 1)); PV_STAT(atomic_add_int(&pc_chunk_allocs, 1)); dump_add_page(m->phys_addr); pc = (void *)PHYS_TO_DMAP(m->phys_addr); pc->pc_pmap = pmap; pc->pc_map[0] = PC_FREE0 & ~1ul; /* preallocated bit 0 */ pc->pc_map[1] = PC_FREE1; pc->pc_map[2] = PC_FREE2; pvc = &pv_chunks[_vm_phys_domain(m->phys_addr)]; mtx_lock(&pvc->pvc_lock); TAILQ_INSERT_TAIL(&pvc->pvc_list, pc, pc_lru); mtx_unlock(&pvc->pvc_lock); pv = &pc->pc_pventry[0]; TAILQ_INSERT_HEAD(&pmap->pm_pvchunk, pc, pc_list); PV_STAT(atomic_add_long(&pv_entry_count, 1)); PV_STAT(atomic_add_int(&pv_entry_spare, _NPCPV - 1)); return (pv); } /* * Returns the number of one bits within the given PV chunk map. * * The erratas for Intel processors state that "POPCNT Instruction May * Take Longer to Execute Than Expected". It is believed that the * issue is the spurious dependency on the destination register. * Provide a hint to the register rename logic that the destination * value is overwritten, by clearing it, as suggested in the * optimization manual. It should be cheap for unaffected processors * as well. * * Reference numbers for erratas are * 4th Gen Core: HSD146 * 5th Gen Core: BDM85 * 6th Gen Core: SKL029 */ static int popcnt_pc_map_pq(uint64_t *map) { u_long result, tmp; __asm __volatile("xorl %k0,%k0;popcntq %2,%0;" "xorl %k1,%k1;popcntq %3,%1;addl %k1,%k0;" "xorl %k1,%k1;popcntq %4,%1;addl %k1,%k0" : "=&r" (result), "=&r" (tmp) : "m" (map[0]), "m" (map[1]), "m" (map[2])); return (result); } /* * Ensure that the number of spare PV entries in the specified pmap meets or * exceeds the given count, "needed". * * The given PV list lock may be released. */ static void reserve_pv_entries(pmap_t pmap, int needed, struct rwlock **lockp) { struct pv_chunks_list *pvc; struct pch new_tail[PMAP_MEMDOM]; struct pv_chunk *pc; vm_page_t m; int avail, free, i; bool reclaimed; PMAP_LOCK_ASSERT(pmap, MA_OWNED); KASSERT(lockp != NULL, ("reserve_pv_entries: lockp is NULL")); /* * Newly allocated PV chunks must be stored in a private list until * the required number of PV chunks have been allocated. Otherwise, * reclaim_pv_chunk() could recycle one of these chunks. In * contrast, these chunks must be added to the pmap upon allocation. */ for (i = 0; i < PMAP_MEMDOM; i++) TAILQ_INIT(&new_tail[i]); retry: avail = 0; TAILQ_FOREACH(pc, &pmap->pm_pvchunk, pc_list) { #ifndef __POPCNT__ if ((cpu_feature2 & CPUID2_POPCNT) == 0) bit_count((bitstr_t *)pc->pc_map, 0, sizeof(pc->pc_map) * NBBY, &free); else #endif free = popcnt_pc_map_pq(pc->pc_map); if (free == 0) break; avail += free; if (avail >= needed) break; } for (reclaimed = false; avail < needed; avail += _NPCPV) { m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED); if (m == NULL) { m = reclaim_pv_chunk(pmap, lockp); if (m == NULL) goto retry; reclaimed = true; } PV_STAT(atomic_add_int(&pc_chunk_count, 1)); PV_STAT(atomic_add_int(&pc_chunk_allocs, 1)); dump_add_page(m->phys_addr); pc = (void *)PHYS_TO_DMAP(m->phys_addr); pc->pc_pmap = pmap; pc->pc_map[0] = PC_FREE0; pc->pc_map[1] = PC_FREE1; pc->pc_map[2] = PC_FREE2; TAILQ_INSERT_HEAD(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_TAIL(&new_tail[pc_to_domain(pc)], pc, pc_lru); PV_STAT(atomic_add_int(&pv_entry_spare, _NPCPV)); /* * The reclaim might have freed a chunk from the current pmap. * If that chunk contained available entries, we need to * re-count the number of available entries. */ if (reclaimed) goto retry; } for (i = 0; i < vm_ndomains; i++) { if (TAILQ_EMPTY(&new_tail[i])) continue; pvc = &pv_chunks[i]; mtx_lock(&pvc->pvc_lock); TAILQ_CONCAT(&pvc->pvc_list, &new_tail[i], pc_lru); mtx_unlock(&pvc->pvc_lock); } } /* * First find and then remove the pv entry for the specified pmap and virtual * address from the specified pv list. Returns the pv entry if found and NULL * otherwise. This operation can be performed on pv lists for either 4KB or * 2MB page mappings. */ static __inline pv_entry_t pmap_pvh_remove(struct md_page *pvh, pmap_t pmap, vm_offset_t va) { pv_entry_t pv; TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) { if (pmap == PV_PMAP(pv) && va == pv->pv_va) { TAILQ_REMOVE(&pvh->pv_list, pv, pv_next); pvh->pv_gen++; break; } } return (pv); } /* * After demotion from a 2MB page mapping to 512 4KB page mappings, * destroy the pv entry for the 2MB page mapping and reinstantiate the pv * entries for each of the 4KB page mappings. */ static void pmap_pv_demote_pde(pmap_t pmap, vm_offset_t va, vm_paddr_t pa, struct rwlock **lockp) { struct md_page *pvh; struct pv_chunk *pc; pv_entry_t pv; vm_offset_t va_last; vm_page_t m; int bit, field; PMAP_LOCK_ASSERT(pmap, MA_OWNED); KASSERT((pa & PDRMASK) == 0, ("pmap_pv_demote_pde: pa is not 2mpage aligned")); CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, pa); /* * Transfer the 2mpage's pv entry for this mapping to the first * page's pv list. Once this transfer begins, the pv list lock * must not be released until the last pv entry is reinstantiated. */ pvh = pa_to_pvh(pa); va = trunc_2mpage(va); pv = pmap_pvh_remove(pvh, pmap, va); KASSERT(pv != NULL, ("pmap_pv_demote_pde: pv not found")); m = PHYS_TO_VM_PAGE(pa); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; /* Instantiate the remaining NPTEPG - 1 pv entries. */ PV_STAT(atomic_add_long(&pv_entry_allocs, NPTEPG - 1)); va_last = va + NBPDR - PAGE_SIZE; for (;;) { pc = TAILQ_FIRST(&pmap->pm_pvchunk); KASSERT(pc->pc_map[0] != 0 || pc->pc_map[1] != 0 || pc->pc_map[2] != 0, ("pmap_pv_demote_pde: missing spare")); for (field = 0; field < _NPCM; field++) { while (pc->pc_map[field]) { bit = bsfq(pc->pc_map[field]); pc->pc_map[field] &= ~(1ul << bit); pv = &pc->pc_pventry[field * 64 + bit]; va += PAGE_SIZE; pv->pv_va = va; m++; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_pv_demote_pde: page %p is not managed", m)); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; if (va == va_last) goto out; } } TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_TAIL(&pmap->pm_pvchunk, pc, pc_list); } out: if (pc->pc_map[0] == 0 && pc->pc_map[1] == 0 && pc->pc_map[2] == 0) { TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_TAIL(&pmap->pm_pvchunk, pc, pc_list); } PV_STAT(atomic_add_long(&pv_entry_count, NPTEPG - 1)); PV_STAT(atomic_subtract_int(&pv_entry_spare, NPTEPG - 1)); } #if VM_NRESERVLEVEL > 0 /* * After promotion from 512 4KB page mappings to a single 2MB page mapping, * replace the many pv entries for the 4KB page mappings by a single pv entry * for the 2MB page mapping. */ static void pmap_pv_promote_pde(pmap_t pmap, vm_offset_t va, vm_paddr_t pa, struct rwlock **lockp) { struct md_page *pvh; pv_entry_t pv; vm_offset_t va_last; vm_page_t m; KASSERT((pa & PDRMASK) == 0, ("pmap_pv_promote_pde: pa is not 2mpage aligned")); CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, pa); /* * Transfer the first page's pv entry for this mapping to the 2mpage's * pv list. Aside from avoiding the cost of a call to get_pv_entry(), * a transfer avoids the possibility that get_pv_entry() calls * reclaim_pv_chunk() and that reclaim_pv_chunk() removes one of the * mappings that is being promoted. */ m = PHYS_TO_VM_PAGE(pa); va = trunc_2mpage(va); pv = pmap_pvh_remove(&m->md, pmap, va); KASSERT(pv != NULL, ("pmap_pv_promote_pde: pv not found")); pvh = pa_to_pvh(pa); TAILQ_INSERT_TAIL(&pvh->pv_list, pv, pv_next); pvh->pv_gen++; /* Free the remaining NPTEPG - 1 pv entries. */ va_last = va + NBPDR - PAGE_SIZE; do { m++; va += PAGE_SIZE; pmap_pvh_free(&m->md, pmap, va); } while (va < va_last); } #endif /* VM_NRESERVLEVEL > 0 */ /* * First find and then destroy the pv entry for the specified pmap and virtual * address. This operation can be performed on pv lists for either 4KB or 2MB * page mappings. */ static void pmap_pvh_free(struct md_page *pvh, pmap_t pmap, vm_offset_t va) { pv_entry_t pv; pv = pmap_pvh_remove(pvh, pmap, va); KASSERT(pv != NULL, ("pmap_pvh_free: pv not found")); free_pv_entry(pmap, pv); } /* * Conditionally create the PV entry for a 4KB page mapping if the required * memory can be allocated without resorting to reclamation. */ static boolean_t pmap_try_insert_pv_entry(pmap_t pmap, vm_offset_t va, vm_page_t m, struct rwlock **lockp) { pv_entry_t pv; PMAP_LOCK_ASSERT(pmap, MA_OWNED); /* Pass NULL instead of the lock pointer to disable reclamation. */ if ((pv = get_pv_entry(pmap, NULL)) != NULL) { pv->pv_va = va; CHANGE_PV_LIST_LOCK_TO_VM_PAGE(lockp, m); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; return (TRUE); } else return (FALSE); } /* * Create the PV entry for a 2MB page mapping. Always returns true unless the * flag PMAP_ENTER_NORECLAIM is specified. If that flag is specified, returns * false if the PV entry cannot be allocated without resorting to reclamation. */ static bool pmap_pv_insert_pde(pmap_t pmap, vm_offset_t va, pd_entry_t pde, u_int flags, struct rwlock **lockp) { struct md_page *pvh; pv_entry_t pv; vm_paddr_t pa; PMAP_LOCK_ASSERT(pmap, MA_OWNED); /* Pass NULL instead of the lock pointer to disable reclamation. */ if ((pv = get_pv_entry(pmap, (flags & PMAP_ENTER_NORECLAIM) != 0 ? NULL : lockp)) == NULL) return (false); pv->pv_va = va; pa = pde & PG_PS_FRAME; CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, pa); pvh = pa_to_pvh(pa); TAILQ_INSERT_TAIL(&pvh->pv_list, pv, pv_next); pvh->pv_gen++; return (true); } /* * Fills a page table page with mappings to consecutive physical pages. */ static void pmap_fill_ptp(pt_entry_t *firstpte, pt_entry_t newpte) { pt_entry_t *pte; for (pte = firstpte; pte < firstpte + NPTEPG; pte++) { *pte = newpte; newpte += PAGE_SIZE; } } /* * Tries to demote a 2MB page mapping. If demotion fails, the 2MB page * mapping is invalidated. */ static boolean_t pmap_demote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va) { struct rwlock *lock; boolean_t rv; lock = NULL; rv = pmap_demote_pde_locked(pmap, pde, va, &lock); if (lock != NULL) rw_wunlock(lock); return (rv); } static void pmap_demote_pde_check(pt_entry_t *firstpte __unused, pt_entry_t newpte __unused) { #ifdef INVARIANTS #ifdef DIAGNOSTIC pt_entry_t *xpte, *ypte; for (xpte = firstpte; xpte < firstpte + NPTEPG; xpte++, newpte += PAGE_SIZE) { if ((*xpte & PG_FRAME) != (newpte & PG_FRAME)) { printf("pmap_demote_pde: xpte %zd and newpte map " "different pages: found %#lx, expected %#lx\n", xpte - firstpte, *xpte, newpte); printf("page table dump\n"); for (ypte = firstpte; ypte < firstpte + NPTEPG; ypte++) printf("%zd %#lx\n", ypte - firstpte, *ypte); panic("firstpte"); } } #else KASSERT((*firstpte & PG_FRAME) == (newpte & PG_FRAME), ("pmap_demote_pde: firstpte and newpte map different physical" " addresses")); #endif #endif } static void pmap_demote_pde_abort(pmap_t pmap, vm_offset_t va, pd_entry_t *pde, pd_entry_t oldpde, struct rwlock **lockp) { struct spglist free; vm_offset_t sva; SLIST_INIT(&free); sva = trunc_2mpage(va); pmap_remove_pde(pmap, pde, sva, &free, lockp); if ((oldpde & pmap_global_bit(pmap)) == 0) pmap_invalidate_pde_page(pmap, sva, oldpde); vm_page_free_pages_toq(&free, true); CTR2(KTR_PMAP, "pmap_demote_pde: failure for va %#lx in pmap %p", va, pmap); } static boolean_t pmap_demote_pde_locked(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, struct rwlock **lockp) { pd_entry_t newpde, oldpde; pt_entry_t *firstpte, newpte; pt_entry_t PG_A, PG_G, PG_M, PG_PKU_MASK, PG_RW, PG_V; vm_paddr_t mptepa; vm_page_t mpte; int PG_PTE_CACHE; bool in_kernel; PG_A = pmap_accessed_bit(pmap); PG_G = pmap_global_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_PTE_CACHE = pmap_cache_mask(pmap, 0); PG_PKU_MASK = pmap_pku_mask_bit(pmap); PMAP_LOCK_ASSERT(pmap, MA_OWNED); in_kernel = va >= VM_MAXUSER_ADDRESS; oldpde = *pde; KASSERT((oldpde & (PG_PS | PG_V)) == (PG_PS | PG_V), ("pmap_demote_pde: oldpde is missing PG_PS and/or PG_V")); /* * Invalidate the 2MB page mapping and return "failure" if the * mapping was never accessed. */ if ((oldpde & PG_A) == 0) { KASSERT((oldpde & PG_W) == 0, ("pmap_demote_pde: a wired mapping is missing PG_A")); pmap_demote_pde_abort(pmap, va, pde, oldpde, lockp); return (FALSE); } mpte = pmap_remove_pt_page(pmap, va); if (mpte == NULL) { KASSERT((oldpde & PG_W) == 0, ("pmap_demote_pde: page table page for a wired mapping" " is missing")); /* * If the page table page is missing and the mapping * is for a kernel address, the mapping must belong to * the direct map. Page table pages are preallocated * for every other part of the kernel address space, * so the direct map region is the only part of the * kernel address space that must be handled here. */ KASSERT(!in_kernel || (va >= DMAP_MIN_ADDRESS && va < DMAP_MAX_ADDRESS), ("pmap_demote_pde: No saved mpte for va %#lx", va)); /* * If the 2MB page mapping belongs to the direct map * region of the kernel's address space, then the page * allocation request specifies the highest possible * priority (VM_ALLOC_INTERRUPT). Otherwise, the * priority is normal. */ mpte = vm_page_alloc(NULL, pmap_pde_pindex(va), (in_kernel ? VM_ALLOC_INTERRUPT : VM_ALLOC_NORMAL) | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED); /* * If the allocation of the new page table page fails, * invalidate the 2MB page mapping and return "failure". */ if (mpte == NULL) { pmap_demote_pde_abort(pmap, va, pde, oldpde, lockp); return (FALSE); } if (!in_kernel) { mpte->ref_count = NPTEPG; pmap_resident_count_inc(pmap, 1); } } mptepa = VM_PAGE_TO_PHYS(mpte); firstpte = (pt_entry_t *)PHYS_TO_DMAP(mptepa); newpde = mptepa | PG_M | PG_A | (oldpde & PG_U) | PG_RW | PG_V; KASSERT((oldpde & (PG_M | PG_RW)) != PG_RW, ("pmap_demote_pde: oldpde is missing PG_M")); newpte = oldpde & ~PG_PS; newpte = pmap_swap_pat(pmap, newpte); /* * If the page table page is not leftover from an earlier promotion, * initialize it. */ if (mpte->valid == 0) pmap_fill_ptp(firstpte, newpte); pmap_demote_pde_check(firstpte, newpte); /* * If the mapping has changed attributes, update the page table * entries. */ if ((*firstpte & PG_PTE_PROMOTE) != (newpte & PG_PTE_PROMOTE)) pmap_fill_ptp(firstpte, newpte); /* * The spare PV entries must be reserved prior to demoting the * mapping, that is, prior to changing the PDE. Otherwise, the state * of the PDE and the PV lists will be inconsistent, which can result * in reclaim_pv_chunk() attempting to remove a PV entry from the * wrong PV list and pmap_pv_demote_pde() failing to find the expected * PV entry for the 2MB page mapping that is being demoted. */ if ((oldpde & PG_MANAGED) != 0) reserve_pv_entries(pmap, NPTEPG - 1, lockp); /* * Demote the mapping. This pmap is locked. The old PDE has * PG_A set. If the old PDE has PG_RW set, it also has PG_M * set. Thus, there is no danger of a race with another * processor changing the setting of PG_A and/or PG_M between * the read above and the store below. */ if (workaround_erratum383) pmap_update_pde(pmap, va, pde, newpde); else pde_store(pde, newpde); /* * Invalidate a stale recursive mapping of the page table page. */ if (in_kernel) pmap_invalidate_page(pmap, (vm_offset_t)vtopte(va)); /* * Demote the PV entry. */ if ((oldpde & PG_MANAGED) != 0) pmap_pv_demote_pde(pmap, va, oldpde & PG_PS_FRAME, lockp); atomic_add_long(&pmap_pde_demotions, 1); CTR2(KTR_PMAP, "pmap_demote_pde: success for va %#lx in pmap %p", va, pmap); return (TRUE); } /* * pmap_remove_kernel_pde: Remove a kernel superpage mapping. */ static void pmap_remove_kernel_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va) { pd_entry_t newpde; vm_paddr_t mptepa; vm_page_t mpte; KASSERT(pmap == kernel_pmap, ("pmap %p is not kernel_pmap", pmap)); PMAP_LOCK_ASSERT(pmap, MA_OWNED); mpte = pmap_remove_pt_page(pmap, va); if (mpte == NULL) panic("pmap_remove_kernel_pde: Missing pt page."); mptepa = VM_PAGE_TO_PHYS(mpte); newpde = mptepa | X86_PG_M | X86_PG_A | X86_PG_RW | X86_PG_V; /* * If this page table page was unmapped by a promotion, then it * contains valid mappings. Zero it to invalidate those mappings. */ if (mpte->valid != 0) pagezero((void *)PHYS_TO_DMAP(mptepa)); /* * Demote the mapping. */ if (workaround_erratum383) pmap_update_pde(pmap, va, pde, newpde); else pde_store(pde, newpde); /* * Invalidate a stale recursive mapping of the page table page. */ pmap_invalidate_page(pmap, (vm_offset_t)vtopte(va)); } /* * pmap_remove_pde: do the things to unmap a superpage in a process */ static int pmap_remove_pde(pmap_t pmap, pd_entry_t *pdq, vm_offset_t sva, struct spglist *free, struct rwlock **lockp) { struct md_page *pvh; pd_entry_t oldpde; vm_offset_t eva, va; vm_page_t m, mpte; pt_entry_t PG_G, PG_A, PG_M, PG_RW; PG_G = pmap_global_bit(pmap); PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); PMAP_LOCK_ASSERT(pmap, MA_OWNED); KASSERT((sva & PDRMASK) == 0, ("pmap_remove_pde: sva is not 2mpage aligned")); oldpde = pte_load_clear(pdq); if (oldpde & PG_W) pmap->pm_stats.wired_count -= NBPDR / PAGE_SIZE; if ((oldpde & PG_G) != 0) pmap_invalidate_pde_page(kernel_pmap, sva, oldpde); pmap_resident_count_dec(pmap, NBPDR / PAGE_SIZE); if (oldpde & PG_MANAGED) { CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, oldpde & PG_PS_FRAME); pvh = pa_to_pvh(oldpde & PG_PS_FRAME); pmap_pvh_free(pvh, pmap, sva); eva = sva + NBPDR; for (va = sva, m = PHYS_TO_VM_PAGE(oldpde & PG_PS_FRAME); va < eva; va += PAGE_SIZE, m++) { if ((oldpde & (PG_M | PG_RW)) == (PG_M | PG_RW)) vm_page_dirty(m); if (oldpde & PG_A) vm_page_aflag_set(m, PGA_REFERENCED); if (TAILQ_EMPTY(&m->md.pv_list) && TAILQ_EMPTY(&pvh->pv_list)) vm_page_aflag_clear(m, PGA_WRITEABLE); pmap_delayed_invl_page(m); } } if (pmap == kernel_pmap) { pmap_remove_kernel_pde(pmap, pdq, sva); } else { mpte = pmap_remove_pt_page(pmap, sva); if (mpte != NULL) { KASSERT(mpte->valid == VM_PAGE_BITS_ALL, ("pmap_remove_pde: pte page not promoted")); pmap_resident_count_dec(pmap, 1); KASSERT(mpte->ref_count == NPTEPG, ("pmap_remove_pde: pte page ref count error")); mpte->ref_count = 0; pmap_add_delayed_free_list(mpte, free, FALSE); } } return (pmap_unuse_pt(pmap, sva, *pmap_pdpe(pmap, sva), free)); } /* * pmap_remove_pte: do the things to unmap a page in a process */ static int pmap_remove_pte(pmap_t pmap, pt_entry_t *ptq, vm_offset_t va, pd_entry_t ptepde, struct spglist *free, struct rwlock **lockp) { struct md_page *pvh; pt_entry_t oldpte, PG_A, PG_M, PG_RW; vm_page_t m; PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); PMAP_LOCK_ASSERT(pmap, MA_OWNED); oldpte = pte_load_clear(ptq); if (oldpte & PG_W) pmap->pm_stats.wired_count -= 1; pmap_resident_count_dec(pmap, 1); if (oldpte & PG_MANAGED) { m = PHYS_TO_VM_PAGE(oldpte & PG_FRAME); if ((oldpte & (PG_M | PG_RW)) == (PG_M | PG_RW)) vm_page_dirty(m); if (oldpte & PG_A) vm_page_aflag_set(m, PGA_REFERENCED); CHANGE_PV_LIST_LOCK_TO_VM_PAGE(lockp, m); pmap_pvh_free(&m->md, pmap, va); if (TAILQ_EMPTY(&m->md.pv_list) && (m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); if (TAILQ_EMPTY(&pvh->pv_list)) vm_page_aflag_clear(m, PGA_WRITEABLE); } pmap_delayed_invl_page(m); } return (pmap_unuse_pt(pmap, va, ptepde, free)); } /* * Remove a single page from a process address space */ static void pmap_remove_page(pmap_t pmap, vm_offset_t va, pd_entry_t *pde, struct spglist *free) { struct rwlock *lock; pt_entry_t *pte, PG_V; PG_V = pmap_valid_bit(pmap); PMAP_LOCK_ASSERT(pmap, MA_OWNED); if ((*pde & PG_V) == 0) return; pte = pmap_pde_to_pte(pde, va); if ((*pte & PG_V) == 0) return; lock = NULL; pmap_remove_pte(pmap, pte, va, *pde, free, &lock); if (lock != NULL) rw_wunlock(lock); pmap_invalidate_page(pmap, va); } /* * Removes the specified range of addresses from the page table page. */ static bool pmap_remove_ptes(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, pd_entry_t *pde, struct spglist *free, struct rwlock **lockp) { pt_entry_t PG_G, *pte; vm_offset_t va; bool anyvalid; PMAP_LOCK_ASSERT(pmap, MA_OWNED); PG_G = pmap_global_bit(pmap); anyvalid = false; va = eva; for (pte = pmap_pde_to_pte(pde, sva); sva != eva; pte++, sva += PAGE_SIZE) { if (*pte == 0) { if (va != eva) { pmap_invalidate_range(pmap, va, sva); va = eva; } continue; } if ((*pte & PG_G) == 0) anyvalid = true; else if (va == eva) va = sva; if (pmap_remove_pte(pmap, pte, sva, *pde, free, lockp)) { sva += PAGE_SIZE; break; } } if (va != eva) pmap_invalidate_range(pmap, va, sva); return (anyvalid); } /* * Remove the given range of addresses from the specified map. * * It is assumed that the start and end are properly * rounded to the page size. */ void pmap_remove(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { struct rwlock *lock; vm_offset_t va_next; pml4_entry_t *pml4e; pdp_entry_t *pdpe; pd_entry_t ptpaddr, *pde; pt_entry_t PG_G, PG_V; struct spglist free; int anyvalid; PG_G = pmap_global_bit(pmap); PG_V = pmap_valid_bit(pmap); /* * Perform an unsynchronized read. This is, however, safe. */ if (pmap->pm_stats.resident_count == 0) return; anyvalid = 0; SLIST_INIT(&free); pmap_delayed_invl_start(); PMAP_LOCK(pmap); pmap_pkru_on_remove(pmap, sva, eva); /* * special handling of removing one page. a very * common operation and easy to short circuit some * code. */ if (sva + PAGE_SIZE == eva) { pde = pmap_pde(pmap, sva); if (pde && (*pde & PG_PS) == 0) { pmap_remove_page(pmap, sva, pde, &free); goto out; } } lock = NULL; for (; sva < eva; sva = va_next) { if (pmap->pm_stats.resident_count == 0) break; pml4e = pmap_pml4e(pmap, sva); if ((*pml4e & PG_V) == 0) { va_next = (sva + NBPML4) & ~PML4MASK; if (va_next < sva) va_next = eva; continue; } pdpe = pmap_pml4e_to_pdpe(pml4e, sva); if ((*pdpe & PG_V) == 0) { va_next = (sva + NBPDP) & ~PDPMASK; if (va_next < sva) va_next = eva; continue; } /* * Calculate index for next page table. */ va_next = (sva + NBPDR) & ~PDRMASK; if (va_next < sva) va_next = eva; pde = pmap_pdpe_to_pde(pdpe, sva); ptpaddr = *pde; /* * Weed out invalid mappings. */ if (ptpaddr == 0) continue; /* * Check for large page. */ if ((ptpaddr & PG_PS) != 0) { /* * Are we removing the entire large page? If not, * demote the mapping and fall through. */ if (sva + NBPDR == va_next && eva >= va_next) { /* * The TLB entry for a PG_G mapping is * invalidated by pmap_remove_pde(). */ if ((ptpaddr & PG_G) == 0) anyvalid = 1; pmap_remove_pde(pmap, pde, sva, &free, &lock); continue; } else if (!pmap_demote_pde_locked(pmap, pde, sva, &lock)) { /* The large page mapping was destroyed. */ continue; } else ptpaddr = *pde; } /* * Limit our scan to either the end of the va represented * by the current page table page, or to the end of the * range being removed. */ if (va_next > eva) va_next = eva; if (pmap_remove_ptes(pmap, sva, va_next, pde, &free, &lock)) anyvalid = 1; } if (lock != NULL) rw_wunlock(lock); out: if (anyvalid) pmap_invalidate_all(pmap); PMAP_UNLOCK(pmap); pmap_delayed_invl_finish(); vm_page_free_pages_toq(&free, true); } /* * Routine: pmap_remove_all * Function: * Removes this physical page from * all physical maps in which it resides. * Reflects back modify bits to the pager. * * Notes: * Original versions of this routine were very * inefficient because they iteratively called * pmap_remove (slow...) */ void pmap_remove_all(vm_page_t m) { struct md_page *pvh; pv_entry_t pv; pmap_t pmap; struct rwlock *lock; pt_entry_t *pte, tpte, PG_A, PG_M, PG_RW; pd_entry_t *pde; vm_offset_t va; struct spglist free; int pvh_gen, md_gen; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_remove_all: page %p is not managed", m)); SLIST_INIT(&free); lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(VM_PAGE_TO_PHYS(m)); retry: rw_wlock(lock); while ((pv = TAILQ_FIRST(&pvh->pv_list)) != NULL) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { rw_wunlock(lock); PMAP_UNLOCK(pmap); goto retry; } } va = pv->pv_va; pde = pmap_pde(pmap, va); (void)pmap_demote_pde_locked(pmap, pde, va, &lock); PMAP_UNLOCK(pmap); } while ((pv = TAILQ_FIRST(&m->md.pv_list)) != NULL) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; md_gen = m->md.pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { rw_wunlock(lock); PMAP_UNLOCK(pmap); goto retry; } } PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); pmap_resident_count_dec(pmap, 1); pde = pmap_pde(pmap, pv->pv_va); KASSERT((*pde & PG_PS) == 0, ("pmap_remove_all: found" " a 2mpage in page %p's pv list", m)); pte = pmap_pde_to_pte(pde, pv->pv_va); tpte = pte_load_clear(pte); if (tpte & PG_W) pmap->pm_stats.wired_count--; if (tpte & PG_A) vm_page_aflag_set(m, PGA_REFERENCED); /* * Update the vm_page_t clean and reference bits. */ if ((tpte & (PG_M | PG_RW)) == (PG_M | PG_RW)) vm_page_dirty(m); pmap_unuse_pt(pmap, pv->pv_va, *pde, &free); pmap_invalidate_page(pmap, pv->pv_va); TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; free_pv_entry(pmap, pv); PMAP_UNLOCK(pmap); } vm_page_aflag_clear(m, PGA_WRITEABLE); rw_wunlock(lock); pmap_delayed_invl_wait(m); vm_page_free_pages_toq(&free, true); } /* * pmap_protect_pde: do the things to protect a 2mpage in a process */ static boolean_t pmap_protect_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t sva, vm_prot_t prot) { pd_entry_t newpde, oldpde; vm_page_t m, mt; boolean_t anychanged; pt_entry_t PG_G, PG_M, PG_RW; PG_G = pmap_global_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); PMAP_LOCK_ASSERT(pmap, MA_OWNED); KASSERT((sva & PDRMASK) == 0, ("pmap_protect_pde: sva is not 2mpage aligned")); anychanged = FALSE; retry: oldpde = newpde = *pde; if ((prot & VM_PROT_WRITE) == 0) { if ((oldpde & (PG_MANAGED | PG_M | PG_RW)) == (PG_MANAGED | PG_M | PG_RW)) { m = PHYS_TO_VM_PAGE(oldpde & PG_PS_FRAME); for (mt = m; mt < &m[NBPDR / PAGE_SIZE]; mt++) vm_page_dirty(mt); } newpde &= ~(PG_RW | PG_M); } if ((prot & VM_PROT_EXECUTE) == 0) newpde |= pg_nx; if (newpde != oldpde) { /* * As an optimization to future operations on this PDE, clear * PG_PROMOTED. The impending invalidation will remove any * lingering 4KB page mappings from the TLB. */ if (!atomic_cmpset_long(pde, oldpde, newpde & ~PG_PROMOTED)) goto retry; if ((oldpde & PG_G) != 0) pmap_invalidate_pde_page(kernel_pmap, sva, oldpde); else anychanged = TRUE; } return (anychanged); } /* * Set the physical protection on the * specified range of this map as requested. */ void pmap_protect(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, vm_prot_t prot) { vm_offset_t va_next; pml4_entry_t *pml4e; pdp_entry_t *pdpe; pd_entry_t ptpaddr, *pde; pt_entry_t *pte, PG_G, PG_M, PG_RW, PG_V; boolean_t anychanged; KASSERT((prot & ~VM_PROT_ALL) == 0, ("invalid prot %x", prot)); if (prot == VM_PROT_NONE) { pmap_remove(pmap, sva, eva); return; } if ((prot & (VM_PROT_WRITE|VM_PROT_EXECUTE)) == (VM_PROT_WRITE|VM_PROT_EXECUTE)) return; PG_G = pmap_global_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); anychanged = FALSE; /* * Although this function delays and batches the invalidation * of stale TLB entries, it does not need to call * pmap_delayed_invl_start() and * pmap_delayed_invl_finish(), because it does not * ordinarily destroy mappings. Stale TLB entries from * protection-only changes need only be invalidated before the * pmap lock is released, because protection-only changes do * not destroy PV entries. Even operations that iterate over * a physical page's PV list of mappings, like * pmap_remove_write(), acquire the pmap lock for each * mapping. Consequently, for protection-only changes, the * pmap lock suffices to synchronize both page table and TLB * updates. * * This function only destroys a mapping if pmap_demote_pde() * fails. In that case, stale TLB entries are immediately * invalidated. */ PMAP_LOCK(pmap); for (; sva < eva; sva = va_next) { pml4e = pmap_pml4e(pmap, sva); if ((*pml4e & PG_V) == 0) { va_next = (sva + NBPML4) & ~PML4MASK; if (va_next < sva) va_next = eva; continue; } pdpe = pmap_pml4e_to_pdpe(pml4e, sva); if ((*pdpe & PG_V) == 0) { va_next = (sva + NBPDP) & ~PDPMASK; if (va_next < sva) va_next = eva; continue; } va_next = (sva + NBPDR) & ~PDRMASK; if (va_next < sva) va_next = eva; pde = pmap_pdpe_to_pde(pdpe, sva); ptpaddr = *pde; /* * Weed out invalid mappings. */ if (ptpaddr == 0) continue; /* * Check for large page. */ if ((ptpaddr & PG_PS) != 0) { /* * Are we protecting the entire large page? If not, * demote the mapping and fall through. */ if (sva + NBPDR == va_next && eva >= va_next) { /* * The TLB entry for a PG_G mapping is * invalidated by pmap_protect_pde(). */ if (pmap_protect_pde(pmap, pde, sva, prot)) anychanged = TRUE; continue; } else if (!pmap_demote_pde(pmap, pde, sva)) { /* * The large page mapping was destroyed. */ continue; } } if (va_next > eva) va_next = eva; for (pte = pmap_pde_to_pte(pde, sva); sva != va_next; pte++, sva += PAGE_SIZE) { pt_entry_t obits, pbits; vm_page_t m; retry: obits = pbits = *pte; if ((pbits & PG_V) == 0) continue; if ((prot & VM_PROT_WRITE) == 0) { if ((pbits & (PG_MANAGED | PG_M | PG_RW)) == (PG_MANAGED | PG_M | PG_RW)) { m = PHYS_TO_VM_PAGE(pbits & PG_FRAME); vm_page_dirty(m); } pbits &= ~(PG_RW | PG_M); } if ((prot & VM_PROT_EXECUTE) == 0) pbits |= pg_nx; if (pbits != obits) { if (!atomic_cmpset_long(pte, obits, pbits)) goto retry; if (obits & PG_G) pmap_invalidate_page(pmap, sva); else anychanged = TRUE; } } } if (anychanged) pmap_invalidate_all(pmap); PMAP_UNLOCK(pmap); } #if VM_NRESERVLEVEL > 0 static bool pmap_pde_ept_executable(pmap_t pmap, pd_entry_t pde) { if (pmap->pm_type != PT_EPT) return (false); return ((pde & EPT_PG_EXECUTE) != 0); } /* * Tries to promote the 512, contiguous 4KB page mappings that are within a * single page table page (PTP) to a single 2MB page mapping. For promotion * to occur, two conditions must be met: (1) the 4KB page mappings must map * aligned, contiguous physical memory and (2) the 4KB page mappings must have * identical characteristics. */ static void pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, struct rwlock **lockp) { pd_entry_t newpde; pt_entry_t *firstpte, oldpte, pa, *pte; pt_entry_t PG_G, PG_A, PG_M, PG_RW, PG_V, PG_PKU_MASK; vm_page_t mpte; int PG_PTE_CACHE; PG_A = pmap_accessed_bit(pmap); PG_G = pmap_global_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); PG_PKU_MASK = pmap_pku_mask_bit(pmap); PG_PTE_CACHE = pmap_cache_mask(pmap, 0); PMAP_LOCK_ASSERT(pmap, MA_OWNED); /* * Examine the first PTE in the specified PTP. Abort if this PTE is * either invalid, unused, or does not map the first 4KB physical page * within a 2MB page. */ firstpte = (pt_entry_t *)PHYS_TO_DMAP(*pde & PG_FRAME); setpde: newpde = *firstpte; if ((newpde & ((PG_FRAME & PDRMASK) | PG_A | PG_V)) != (PG_A | PG_V) || !pmap_allow_2m_x_page(pmap, pmap_pde_ept_executable(pmap, newpde))) { atomic_add_long(&pmap_pde_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_pde: failure for va %#lx" " in pmap %p", va, pmap); return; } if ((newpde & (PG_M | PG_RW)) == PG_RW) { /* * When PG_M is already clear, PG_RW can be cleared without * a TLB invalidation. */ if (!atomic_cmpset_long(firstpte, newpde, newpde & ~PG_RW)) goto setpde; newpde &= ~PG_RW; } /* * Examine each of the other PTEs in the specified PTP. Abort if this * PTE maps an unexpected 4KB physical page or does not have identical * characteristics to the first PTE. */ pa = (newpde & (PG_PS_FRAME | PG_A | PG_V)) + NBPDR - PAGE_SIZE; for (pte = firstpte + NPTEPG - 1; pte > firstpte; pte--) { setpte: oldpte = *pte; if ((oldpte & (PG_FRAME | PG_A | PG_V)) != pa) { atomic_add_long(&pmap_pde_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_pde: failure for va %#lx" " in pmap %p", va, pmap); return; } if ((oldpte & (PG_M | PG_RW)) == PG_RW) { /* * When PG_M is already clear, PG_RW can be cleared * without a TLB invalidation. */ if (!atomic_cmpset_long(pte, oldpte, oldpte & ~PG_RW)) goto setpte; oldpte &= ~PG_RW; CTR2(KTR_PMAP, "pmap_promote_pde: protect for va %#lx" " in pmap %p", (oldpte & PG_FRAME & PDRMASK) | (va & ~PDRMASK), pmap); } if ((oldpte & PG_PTE_PROMOTE) != (newpde & PG_PTE_PROMOTE)) { atomic_add_long(&pmap_pde_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_pde: failure for va %#lx" " in pmap %p", va, pmap); return; } pa -= PAGE_SIZE; } /* * Save the page table page in its current state until the PDE * mapping the superpage is demoted by pmap_demote_pde() or * destroyed by pmap_remove_pde(). */ mpte = PHYS_TO_VM_PAGE(*pde & PG_FRAME); KASSERT(mpte >= vm_page_array && mpte < &vm_page_array[vm_page_array_size], ("pmap_promote_pde: page table page is out of range")); KASSERT(mpte->pindex == pmap_pde_pindex(va), ("pmap_promote_pde: page table page's pindex is wrong")); if (pmap_insert_pt_page(pmap, mpte, true)) { atomic_add_long(&pmap_pde_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_pde: failure for va %#lx in pmap %p", va, pmap); return; } /* * Promote the pv entries. */ if ((newpde & PG_MANAGED) != 0) pmap_pv_promote_pde(pmap, va, newpde & PG_PS_FRAME, lockp); /* * Propagate the PAT index to its proper position. */ newpde = pmap_swap_pat(pmap, newpde); /* * Map the superpage. */ if (workaround_erratum383) pmap_update_pde(pmap, va, pde, PG_PS | newpde); else pde_store(pde, PG_PROMOTED | PG_PS | newpde); atomic_add_long(&pmap_pde_promotions, 1); CTR2(KTR_PMAP, "pmap_promote_pde: success for va %#lx" " in pmap %p", va, pmap); } #endif /* VM_NRESERVLEVEL > 0 */ /* * Insert the given physical page (p) at * the specified virtual address (v) in the * target physical map with the protection requested. * * If specified, the page will be wired down, meaning * that the related pte can not be reclaimed. * * NB: This is the only routine which MAY NOT lazy-evaluate * or lose information. That is, this routine must actually * insert this page into the given map NOW. * * When destroying both a page table and PV entry, this function * performs the TLB invalidation before releasing the PV list * lock, so we do not need pmap_delayed_invl_page() calls here. */ int pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, u_int flags, int8_t psind) { struct rwlock *lock; pd_entry_t *pde; pt_entry_t *pte, PG_G, PG_A, PG_M, PG_RW, PG_V; pt_entry_t newpte, origpte; pv_entry_t pv; vm_paddr_t opa, pa; vm_page_t mpte, om; int rv; boolean_t nosleep; PG_A = pmap_accessed_bit(pmap); PG_G = pmap_global_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); va = trunc_page(va); KASSERT(va <= VM_MAX_KERNEL_ADDRESS, ("pmap_enter: toobig")); KASSERT(va < UPT_MIN_ADDRESS || va >= UPT_MAX_ADDRESS, ("pmap_enter: invalid to pmap_enter page table pages (va: 0x%lx)", va)); KASSERT((m->oflags & VPO_UNMANAGED) != 0 || va < kmi.clean_sva || va >= kmi.clean_eva, ("pmap_enter: managed mapping within the clean submap")); if ((m->oflags & VPO_UNMANAGED) == 0) VM_PAGE_OBJECT_BUSY_ASSERT(m); KASSERT((flags & PMAP_ENTER_RESERVED) == 0, ("pmap_enter: flags %u has reserved bits set", flags)); pa = VM_PAGE_TO_PHYS(m); newpte = (pt_entry_t)(pa | PG_A | PG_V); if ((flags & VM_PROT_WRITE) != 0) newpte |= PG_M; if ((prot & VM_PROT_WRITE) != 0) newpte |= PG_RW; KASSERT((newpte & (PG_M | PG_RW)) != PG_M, ("pmap_enter: flags includes VM_PROT_WRITE but prot doesn't")); if ((prot & VM_PROT_EXECUTE) == 0) newpte |= pg_nx; if ((flags & PMAP_ENTER_WIRED) != 0) newpte |= PG_W; if (va < VM_MAXUSER_ADDRESS) newpte |= PG_U; if (pmap == kernel_pmap) newpte |= PG_G; newpte |= pmap_cache_bits(pmap, m->md.pat_mode, psind > 0); /* * Set modified bit gratuitously for writeable mappings if * the page is unmanaged. We do not want to take a fault * to do the dirty bit accounting for these mappings. */ if ((m->oflags & VPO_UNMANAGED) != 0) { if ((newpte & PG_RW) != 0) newpte |= PG_M; } else newpte |= PG_MANAGED; lock = NULL; PMAP_LOCK(pmap); if (psind == 1) { /* Assert the required virtual and physical alignment. */ KASSERT((va & PDRMASK) == 0, ("pmap_enter: va unaligned")); KASSERT(m->psind > 0, ("pmap_enter: m->psind < psind")); rv = pmap_enter_pde(pmap, va, newpte | PG_PS, flags, m, &lock); goto out; } mpte = NULL; /* * In the case that a page table page is not * resident, we are creating it here. */ retry: pde = pmap_pde(pmap, va); if (pde != NULL && (*pde & PG_V) != 0 && ((*pde & PG_PS) == 0 || pmap_demote_pde_locked(pmap, pde, va, &lock))) { pte = pmap_pde_to_pte(pde, va); if (va < VM_MAXUSER_ADDRESS && mpte == NULL) { mpte = PHYS_TO_VM_PAGE(*pde & PG_FRAME); mpte->ref_count++; } } else if (va < VM_MAXUSER_ADDRESS) { /* * Here if the pte page isn't mapped, or if it has been * deallocated. */ nosleep = (flags & PMAP_ENTER_NOSLEEP) != 0; mpte = _pmap_allocpte(pmap, pmap_pde_pindex(va), nosleep ? NULL : &lock); if (mpte == NULL && nosleep) { rv = KERN_RESOURCE_SHORTAGE; goto out; } goto retry; } else panic("pmap_enter: invalid page directory va=%#lx", va); origpte = *pte; pv = NULL; if (va < VM_MAXUSER_ADDRESS && pmap->pm_type == PT_X86) newpte |= pmap_pkru_get(pmap, va); /* * Is the specified virtual address already mapped? */ if ((origpte & PG_V) != 0) { /* * Wiring change, just update stats. We don't worry about * wiring PT pages as they remain resident as long as there * are valid mappings in them. Hence, if a user page is wired, * the PT page will be also. */ if ((newpte & PG_W) != 0 && (origpte & PG_W) == 0) pmap->pm_stats.wired_count++; else if ((newpte & PG_W) == 0 && (origpte & PG_W) != 0) pmap->pm_stats.wired_count--; /* * Remove the extra PT page reference. */ if (mpte != NULL) { mpte->ref_count--; KASSERT(mpte->ref_count > 0, ("pmap_enter: missing reference to page table page," " va: 0x%lx", va)); } /* * Has the physical page changed? */ opa = origpte & PG_FRAME; if (opa == pa) { /* * No, might be a protection or wiring change. */ if ((origpte & PG_MANAGED) != 0 && (newpte & PG_RW) != 0) vm_page_aflag_set(m, PGA_WRITEABLE); if (((origpte ^ newpte) & ~(PG_M | PG_A)) == 0) goto unchanged; goto validate; } /* * The physical page has changed. Temporarily invalidate * the mapping. This ensures that all threads sharing the * pmap keep a consistent view of the mapping, which is * necessary for the correct handling of COW faults. It * also permits reuse of the old mapping's PV entry, * avoiding an allocation. * * For consistency, handle unmanaged mappings the same way. */ origpte = pte_load_clear(pte); KASSERT((origpte & PG_FRAME) == opa, ("pmap_enter: unexpected pa update for %#lx", va)); if ((origpte & PG_MANAGED) != 0) { om = PHYS_TO_VM_PAGE(opa); /* * The pmap lock is sufficient to synchronize with * concurrent calls to pmap_page_test_mappings() and * pmap_ts_referenced(). */ if ((origpte & (PG_M | PG_RW)) == (PG_M | PG_RW)) vm_page_dirty(om); if ((origpte & PG_A) != 0) { pmap_invalidate_page(pmap, va); vm_page_aflag_set(om, PGA_REFERENCED); } CHANGE_PV_LIST_LOCK_TO_PHYS(&lock, opa); pv = pmap_pvh_remove(&om->md, pmap, va); KASSERT(pv != NULL, ("pmap_enter: no PV entry for %#lx", va)); if ((newpte & PG_MANAGED) == 0) free_pv_entry(pmap, pv); if ((om->a.flags & PGA_WRITEABLE) != 0 && TAILQ_EMPTY(&om->md.pv_list) && ((om->flags & PG_FICTITIOUS) != 0 || TAILQ_EMPTY(&pa_to_pvh(opa)->pv_list))) vm_page_aflag_clear(om, PGA_WRITEABLE); } else { /* * Since this mapping is unmanaged, assume that PG_A * is set. */ pmap_invalidate_page(pmap, va); } origpte = 0; } else { /* * Increment the counters. */ if ((newpte & PG_W) != 0) pmap->pm_stats.wired_count++; pmap_resident_count_inc(pmap, 1); } /* * Enter on the PV list if part of our managed memory. */ if ((newpte & PG_MANAGED) != 0) { if (pv == NULL) { pv = get_pv_entry(pmap, &lock); pv->pv_va = va; } CHANGE_PV_LIST_LOCK_TO_PHYS(&lock, pa); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; if ((newpte & PG_RW) != 0) vm_page_aflag_set(m, PGA_WRITEABLE); } /* * Update the PTE. */ if ((origpte & PG_V) != 0) { validate: origpte = pte_load_store(pte, newpte); KASSERT((origpte & PG_FRAME) == pa, ("pmap_enter: unexpected pa update for %#lx", va)); if ((newpte & PG_M) == 0 && (origpte & (PG_M | PG_RW)) == (PG_M | PG_RW)) { if ((origpte & PG_MANAGED) != 0) vm_page_dirty(m); /* * Although the PTE may still have PG_RW set, TLB * invalidation may nonetheless be required because * the PTE no longer has PG_M set. */ } else if ((origpte & PG_NX) != 0 || (newpte & PG_NX) == 0) { /* * This PTE change does not require TLB invalidation. */ goto unchanged; } if ((origpte & PG_A) != 0) pmap_invalidate_page(pmap, va); } else pte_store(pte, newpte); unchanged: #if VM_NRESERVLEVEL > 0 /* * If both the page table page and the reservation are fully * populated, then attempt promotion. */ if ((mpte == NULL || mpte->ref_count == NPTEPG) && pmap_ps_enabled(pmap) && (m->flags & PG_FICTITIOUS) == 0 && vm_reserv_level_iffullpop(m) == 0) pmap_promote_pde(pmap, pde, va, &lock); #endif rv = KERN_SUCCESS; out: if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(pmap); return (rv); } /* * Tries to create a read- and/or execute-only 2MB page mapping. Returns true * if successful. Returns false if (1) a page table page cannot be allocated * without sleeping, (2) a mapping already exists at the specified virtual * address, or (3) a PV entry cannot be allocated without reclaiming another * PV entry. */ static bool pmap_enter_2mpage(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, struct rwlock **lockp) { pd_entry_t newpde; pt_entry_t PG_V; PMAP_LOCK_ASSERT(pmap, MA_OWNED); PG_V = pmap_valid_bit(pmap); newpde = VM_PAGE_TO_PHYS(m) | pmap_cache_bits(pmap, m->md.pat_mode, 1) | PG_PS | PG_V; if ((m->oflags & VPO_UNMANAGED) == 0) newpde |= PG_MANAGED; if ((prot & VM_PROT_EXECUTE) == 0) newpde |= pg_nx; if (va < VM_MAXUSER_ADDRESS) newpde |= PG_U; return (pmap_enter_pde(pmap, va, newpde, PMAP_ENTER_NOSLEEP | PMAP_ENTER_NOREPLACE | PMAP_ENTER_NORECLAIM, NULL, lockp) == KERN_SUCCESS); } /* * Returns true if every page table entry in the specified page table page is * zero. */ static bool pmap_every_pte_zero(vm_paddr_t pa) { pt_entry_t *pt_end, *pte; KASSERT((pa & PAGE_MASK) == 0, ("pa is misaligned")); pte = (pt_entry_t *)PHYS_TO_DMAP(pa); for (pt_end = pte + NPTEPG; pte < pt_end; pte++) { if (*pte != 0) return (false); } return (true); } /* * Tries to create the specified 2MB page mapping. Returns KERN_SUCCESS if * the mapping was created, and either KERN_FAILURE or KERN_RESOURCE_SHORTAGE * otherwise. Returns KERN_FAILURE if PMAP_ENTER_NOREPLACE was specified and * a mapping already exists at the specified virtual address. Returns * KERN_RESOURCE_SHORTAGE if PMAP_ENTER_NOSLEEP was specified and a page table * page allocation failed. Returns KERN_RESOURCE_SHORTAGE if * PMAP_ENTER_NORECLAIM was specified and a PV entry allocation failed. * * The parameter "m" is only used when creating a managed, writeable mapping. */ static int pmap_enter_pde(pmap_t pmap, vm_offset_t va, pd_entry_t newpde, u_int flags, vm_page_t m, struct rwlock **lockp) { struct spglist free; pd_entry_t oldpde, *pde; pt_entry_t PG_G, PG_RW, PG_V; vm_page_t mt, pdpg; KASSERT(pmap == kernel_pmap || (newpde & PG_W) == 0, ("pmap_enter_pde: cannot create wired user mapping")); PG_G = pmap_global_bit(pmap); PG_RW = pmap_rw_bit(pmap); KASSERT((newpde & (pmap_modified_bit(pmap) | PG_RW)) != PG_RW, ("pmap_enter_pde: newpde is missing PG_M")); PG_V = pmap_valid_bit(pmap); PMAP_LOCK_ASSERT(pmap, MA_OWNED); if (!pmap_allow_2m_x_page(pmap, pmap_pde_ept_executable(pmap, newpde))) { CTR2(KTR_PMAP, "pmap_enter_pde: 2m x blocked for va %#lx" " in pmap %p", va, pmap); return (KERN_FAILURE); } if ((pde = pmap_alloc_pde(pmap, va, &pdpg, (flags & PMAP_ENTER_NOSLEEP) != 0 ? NULL : lockp)) == NULL) { CTR2(KTR_PMAP, "pmap_enter_pde: failure for va %#lx" " in pmap %p", va, pmap); return (KERN_RESOURCE_SHORTAGE); } /* * If pkru is not same for the whole pde range, return failure * and let vm_fault() cope. Check after pde allocation, since * it could sleep. */ if (!pmap_pkru_same(pmap, va, va + NBPDR)) { pmap_abort_ptp(pmap, va, pdpg); return (KERN_FAILURE); } if (va < VM_MAXUSER_ADDRESS && pmap->pm_type == PT_X86) { newpde &= ~X86_PG_PKU_MASK; newpde |= pmap_pkru_get(pmap, va); } /* * If there are existing mappings, either abort or remove them. */ oldpde = *pde; if ((oldpde & PG_V) != 0) { KASSERT(pdpg == NULL || pdpg->ref_count > 1, ("pmap_enter_pde: pdpg's reference count is too low")); if ((flags & PMAP_ENTER_NOREPLACE) != 0 && (va < VM_MAXUSER_ADDRESS || (oldpde & PG_PS) != 0 || !pmap_every_pte_zero(oldpde & PG_FRAME))) { if (pdpg != NULL) pdpg->ref_count--; CTR2(KTR_PMAP, "pmap_enter_pde: failure for va %#lx" " in pmap %p", va, pmap); return (KERN_FAILURE); } /* Break the existing mapping(s). */ SLIST_INIT(&free); if ((oldpde & PG_PS) != 0) { /* * The reference to the PD page that was acquired by * pmap_alloc_pde() ensures that it won't be freed. * However, if the PDE resulted from a promotion, then * a reserved PT page could be freed. */ (void)pmap_remove_pde(pmap, pde, va, &free, lockp); if ((oldpde & PG_G) == 0) pmap_invalidate_pde_page(pmap, va, oldpde); } else { pmap_delayed_invl_start(); if (pmap_remove_ptes(pmap, va, va + NBPDR, pde, &free, lockp)) pmap_invalidate_all(pmap); pmap_delayed_invl_finish(); } if (va < VM_MAXUSER_ADDRESS) { vm_page_free_pages_toq(&free, true); KASSERT(*pde == 0, ("pmap_enter_pde: non-zero pde %p", pde)); } else { KASSERT(SLIST_EMPTY(&free), ("pmap_enter_pde: freed kernel page table page")); /* * Both pmap_remove_pde() and pmap_remove_ptes() will * leave the kernel page table page zero filled. */ mt = PHYS_TO_VM_PAGE(*pde & PG_FRAME); if (pmap_insert_pt_page(pmap, mt, false)) panic("pmap_enter_pde: trie insert failed"); } } if ((newpde & PG_MANAGED) != 0) { /* * Abort this mapping if its PV entry could not be created. */ if (!pmap_pv_insert_pde(pmap, va, newpde, flags, lockp)) { if (pdpg != NULL) pmap_abort_ptp(pmap, va, pdpg); CTR2(KTR_PMAP, "pmap_enter_pde: failure for va %#lx" " in pmap %p", va, pmap); return (KERN_RESOURCE_SHORTAGE); } if ((newpde & PG_RW) != 0) { for (mt = m; mt < &m[NBPDR / PAGE_SIZE]; mt++) vm_page_aflag_set(mt, PGA_WRITEABLE); } } /* * Increment counters. */ if ((newpde & PG_W) != 0) pmap->pm_stats.wired_count += NBPDR / PAGE_SIZE; pmap_resident_count_inc(pmap, NBPDR / PAGE_SIZE); /* * Map the superpage. (This is not a promoted mapping; there will not * be any lingering 4KB page mappings in the TLB.) */ pde_store(pde, newpde); atomic_add_long(&pmap_pde_mappings, 1); CTR2(KTR_PMAP, "pmap_enter_pde: success for va %#lx in pmap %p", va, pmap); return (KERN_SUCCESS); } /* * Maps a sequence of resident pages belonging to the same object. * The sequence begins with the given page m_start. This page is * mapped at the given virtual address start. Each subsequent page is * mapped at a virtual address that is offset from start by the same * amount as the page is offset from m_start within the object. The * last page in the sequence is the page with the largest offset from * m_start that can be mapped at a virtual address less than the given * virtual address end. Not every virtual page between start and end * is mapped; only those for which a resident page exists with the * corresponding offset from m_start are mapped. */ void pmap_enter_object(pmap_t pmap, vm_offset_t start, vm_offset_t end, vm_page_t m_start, vm_prot_t prot) { struct rwlock *lock; vm_offset_t va; vm_page_t m, mpte; vm_pindex_t diff, psize; VM_OBJECT_ASSERT_LOCKED(m_start->object); psize = atop(end - start); mpte = NULL; m = m_start; lock = NULL; PMAP_LOCK(pmap); while (m != NULL && (diff = m->pindex - m_start->pindex) < psize) { va = start + ptoa(diff); if ((va & PDRMASK) == 0 && va + NBPDR <= end && m->psind == 1 && pmap_ps_enabled(pmap) && pmap_allow_2m_x_page(pmap, (prot & VM_PROT_EXECUTE) != 0) && pmap_enter_2mpage(pmap, va, m, prot, &lock)) m = &m[NBPDR / PAGE_SIZE - 1]; else mpte = pmap_enter_quick_locked(pmap, va, m, prot, mpte, &lock); m = TAILQ_NEXT(m, listq); } if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(pmap); } /* * this code makes some *MAJOR* assumptions: * 1. Current pmap & pmap exists. * 2. Not wired. * 3. Read access. * 4. No page table pages. * but is *MUCH* faster than pmap_enter... */ void pmap_enter_quick(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot) { struct rwlock *lock; lock = NULL; PMAP_LOCK(pmap); (void)pmap_enter_quick_locked(pmap, va, m, prot, NULL, &lock); if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(pmap); } static vm_page_t pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, vm_page_t mpte, struct rwlock **lockp) { pt_entry_t newpte, *pte, PG_V; KASSERT(va < kmi.clean_sva || va >= kmi.clean_eva || (m->oflags & VPO_UNMANAGED) != 0, ("pmap_enter_quick_locked: managed mapping within the clean submap")); PG_V = pmap_valid_bit(pmap); PMAP_LOCK_ASSERT(pmap, MA_OWNED); /* * In the case that a page table page is not * resident, we are creating it here. */ if (va < VM_MAXUSER_ADDRESS) { vm_pindex_t ptepindex; pd_entry_t *ptepa; /* * Calculate pagetable page index */ ptepindex = pmap_pde_pindex(va); if (mpte && (mpte->pindex == ptepindex)) { mpte->ref_count++; } else { /* * Get the page directory entry */ ptepa = pmap_pde(pmap, va); /* * If the page table page is mapped, we just increment * the hold count, and activate it. Otherwise, we * attempt to allocate a page table page. If this * attempt fails, we don't retry. Instead, we give up. */ if (ptepa && (*ptepa & PG_V) != 0) { if (*ptepa & PG_PS) return (NULL); mpte = PHYS_TO_VM_PAGE(*ptepa & PG_FRAME); mpte->ref_count++; } else { /* * Pass NULL instead of the PV list lock * pointer, because we don't intend to sleep. */ mpte = _pmap_allocpte(pmap, ptepindex, NULL); if (mpte == NULL) return (mpte); } } pte = (pt_entry_t *)PHYS_TO_DMAP(VM_PAGE_TO_PHYS(mpte)); pte = &pte[pmap_pte_index(va)]; } else { mpte = NULL; pte = vtopte(va); } if (*pte) { if (mpte != NULL) mpte->ref_count--; return (NULL); } /* * Enter on the PV list if part of our managed memory. */ if ((m->oflags & VPO_UNMANAGED) == 0 && !pmap_try_insert_pv_entry(pmap, va, m, lockp)) { if (mpte != NULL) pmap_abort_ptp(pmap, va, mpte); return (NULL); } /* * Increment counters */ pmap_resident_count_inc(pmap, 1); newpte = VM_PAGE_TO_PHYS(m) | PG_V | pmap_cache_bits(pmap, m->md.pat_mode, 0); if ((m->oflags & VPO_UNMANAGED) == 0) newpte |= PG_MANAGED; if ((prot & VM_PROT_EXECUTE) == 0) newpte |= pg_nx; if (va < VM_MAXUSER_ADDRESS) newpte |= PG_U | pmap_pkru_get(pmap, va); pte_store(pte, newpte); return (mpte); } /* * Make a temporary mapping for a physical address. This is only intended * to be used for panic dumps. */ void * pmap_kenter_temporary(vm_paddr_t pa, int i) { vm_offset_t va; va = (vm_offset_t)crashdumpmap + (i * PAGE_SIZE); pmap_kenter(va, pa); invlpg(va); return ((void *)crashdumpmap); } /* * This code maps large physical mmap regions into the * processor address space. Note that some shortcuts * are taken, but the code works. */ void pmap_object_init_pt(pmap_t pmap, vm_offset_t addr, vm_object_t object, vm_pindex_t pindex, vm_size_t size) { pd_entry_t *pde; pt_entry_t PG_A, PG_M, PG_RW, PG_V; vm_paddr_t pa, ptepa; vm_page_t p, pdpg; int pat_mode; PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(object->type == OBJT_DEVICE || object->type == OBJT_SG, ("pmap_object_init_pt: non-device object")); if ((addr & (NBPDR - 1)) == 0 && (size & (NBPDR - 1)) == 0) { if (!pmap_ps_enabled(pmap)) return; if (!vm_object_populate(object, pindex, pindex + atop(size))) return; p = vm_page_lookup(object, pindex); KASSERT(p->valid == VM_PAGE_BITS_ALL, ("pmap_object_init_pt: invalid page %p", p)); pat_mode = p->md.pat_mode; /* * Abort the mapping if the first page is not physically * aligned to a 2MB page boundary. */ ptepa = VM_PAGE_TO_PHYS(p); if (ptepa & (NBPDR - 1)) return; /* * Skip the first page. Abort the mapping if the rest of * the pages are not physically contiguous or have differing * memory attributes. */ p = TAILQ_NEXT(p, listq); for (pa = ptepa + PAGE_SIZE; pa < ptepa + size; pa += PAGE_SIZE) { KASSERT(p->valid == VM_PAGE_BITS_ALL, ("pmap_object_init_pt: invalid page %p", p)); if (pa != VM_PAGE_TO_PHYS(p) || pat_mode != p->md.pat_mode) return; p = TAILQ_NEXT(p, listq); } /* * Map using 2MB pages. Since "ptepa" is 2M aligned and * "size" is a multiple of 2M, adding the PAT setting to "pa" * will not affect the termination of this loop. */ PMAP_LOCK(pmap); for (pa = ptepa | pmap_cache_bits(pmap, pat_mode, 1); pa < ptepa + size; pa += NBPDR) { pde = pmap_alloc_pde(pmap, addr, &pdpg, NULL); if (pde == NULL) { /* * The creation of mappings below is only an * optimization. If a page directory page * cannot be allocated without blocking, * continue on to the next mapping rather than * blocking. */ addr += NBPDR; continue; } if ((*pde & PG_V) == 0) { pde_store(pde, pa | PG_PS | PG_M | PG_A | PG_U | PG_RW | PG_V); pmap_resident_count_inc(pmap, NBPDR / PAGE_SIZE); atomic_add_long(&pmap_pde_mappings, 1); } else { /* Continue on if the PDE is already valid. */ pdpg->ref_count--; KASSERT(pdpg->ref_count > 0, ("pmap_object_init_pt: missing reference " "to page directory page, va: 0x%lx", addr)); } addr += NBPDR; } PMAP_UNLOCK(pmap); } } /* * Clear the wired attribute from the mappings for the specified range of * addresses in the given pmap. Every valid mapping within that range * must have the wired attribute set. In contrast, invalid mappings * cannot have the wired attribute set, so they are ignored. * * The wired attribute of the page table entry is not a hardware * feature, so there is no need to invalidate any TLB entries. * Since pmap_demote_pde() for the wired entry must never fail, * pmap_delayed_invl_start()/finish() calls around the * function are not needed. */ void pmap_unwire(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { vm_offset_t va_next; pml4_entry_t *pml4e; pdp_entry_t *pdpe; pd_entry_t *pde; pt_entry_t *pte, PG_V; PG_V = pmap_valid_bit(pmap); PMAP_LOCK(pmap); for (; sva < eva; sva = va_next) { pml4e = pmap_pml4e(pmap, sva); if ((*pml4e & PG_V) == 0) { va_next = (sva + NBPML4) & ~PML4MASK; if (va_next < sva) va_next = eva; continue; } pdpe = pmap_pml4e_to_pdpe(pml4e, sva); if ((*pdpe & PG_V) == 0) { va_next = (sva + NBPDP) & ~PDPMASK; if (va_next < sva) va_next = eva; continue; } va_next = (sva + NBPDR) & ~PDRMASK; if (va_next < sva) va_next = eva; pde = pmap_pdpe_to_pde(pdpe, sva); if ((*pde & PG_V) == 0) continue; if ((*pde & PG_PS) != 0) { if ((*pde & PG_W) == 0) panic("pmap_unwire: pde %#jx is missing PG_W", (uintmax_t)*pde); /* * Are we unwiring the entire large page? If not, * demote the mapping and fall through. */ if (sva + NBPDR == va_next && eva >= va_next) { atomic_clear_long(pde, PG_W); pmap->pm_stats.wired_count -= NBPDR / PAGE_SIZE; continue; } else if (!pmap_demote_pde(pmap, pde, sva)) panic("pmap_unwire: demotion failed"); } if (va_next > eva) va_next = eva; for (pte = pmap_pde_to_pte(pde, sva); sva != va_next; pte++, sva += PAGE_SIZE) { if ((*pte & PG_V) == 0) continue; if ((*pte & PG_W) == 0) panic("pmap_unwire: pte %#jx is missing PG_W", (uintmax_t)*pte); /* * PG_W must be cleared atomically. Although the pmap * lock synchronizes access to PG_W, another processor * could be setting PG_M and/or PG_A concurrently. */ atomic_clear_long(pte, PG_W); pmap->pm_stats.wired_count--; } } PMAP_UNLOCK(pmap); } /* * Copy the range specified by src_addr/len * from the source map to the range dst_addr/len * in the destination map. * * This routine is only advisory and need not do anything. */ void pmap_copy(pmap_t dst_pmap, pmap_t src_pmap, vm_offset_t dst_addr, vm_size_t len, vm_offset_t src_addr) { struct rwlock *lock; pml4_entry_t *pml4e; pdp_entry_t *pdpe; pd_entry_t *pde, srcptepaddr; pt_entry_t *dst_pte, PG_A, PG_M, PG_V, ptetemp, *src_pte; vm_offset_t addr, end_addr, va_next; vm_page_t dst_pdpg, dstmpte, srcmpte; if (dst_addr != src_addr) return; if (dst_pmap->pm_type != src_pmap->pm_type) return; /* * EPT page table entries that require emulation of A/D bits are * sensitive to clearing the PG_A bit (aka EPT_PG_READ). Although * we clear PG_M (aka EPT_PG_WRITE) concomitantly, the PG_U bit * (aka EPT_PG_EXECUTE) could still be set. Since some EPT * implementations flag an EPT misconfiguration for exec-only * mappings we skip this function entirely for emulated pmaps. */ if (pmap_emulate_ad_bits(dst_pmap)) return; end_addr = src_addr + len; lock = NULL; if (dst_pmap < src_pmap) { PMAP_LOCK(dst_pmap); PMAP_LOCK(src_pmap); } else { PMAP_LOCK(src_pmap); PMAP_LOCK(dst_pmap); } PG_A = pmap_accessed_bit(dst_pmap); PG_M = pmap_modified_bit(dst_pmap); PG_V = pmap_valid_bit(dst_pmap); for (addr = src_addr; addr < end_addr; addr = va_next) { KASSERT(addr < UPT_MIN_ADDRESS, ("pmap_copy: invalid to pmap_copy page tables")); pml4e = pmap_pml4e(src_pmap, addr); if ((*pml4e & PG_V) == 0) { va_next = (addr + NBPML4) & ~PML4MASK; if (va_next < addr) va_next = end_addr; continue; } pdpe = pmap_pml4e_to_pdpe(pml4e, addr); if ((*pdpe & PG_V) == 0) { va_next = (addr + NBPDP) & ~PDPMASK; if (va_next < addr) va_next = end_addr; continue; } va_next = (addr + NBPDR) & ~PDRMASK; if (va_next < addr) va_next = end_addr; pde = pmap_pdpe_to_pde(pdpe, addr); srcptepaddr = *pde; if (srcptepaddr == 0) continue; if (srcptepaddr & PG_PS) { if ((addr & PDRMASK) != 0 || addr + NBPDR > end_addr) continue; pde = pmap_alloc_pde(dst_pmap, addr, &dst_pdpg, NULL); if (pde == NULL) break; if (*pde == 0 && ((srcptepaddr & PG_MANAGED) == 0 || pmap_pv_insert_pde(dst_pmap, addr, srcptepaddr, PMAP_ENTER_NORECLAIM, &lock))) { *pde = srcptepaddr & ~PG_W; pmap_resident_count_inc(dst_pmap, NBPDR / PAGE_SIZE); atomic_add_long(&pmap_pde_mappings, 1); } else pmap_abort_ptp(dst_pmap, addr, dst_pdpg); continue; } srcptepaddr &= PG_FRAME; srcmpte = PHYS_TO_VM_PAGE(srcptepaddr); KASSERT(srcmpte->ref_count > 0, ("pmap_copy: source page table page is unused")); if (va_next > end_addr) va_next = end_addr; src_pte = (pt_entry_t *)PHYS_TO_DMAP(srcptepaddr); src_pte = &src_pte[pmap_pte_index(addr)]; dstmpte = NULL; for (; addr < va_next; addr += PAGE_SIZE, src_pte++) { ptetemp = *src_pte; /* * We only virtual copy managed pages. */ if ((ptetemp & PG_MANAGED) == 0) continue; if (dstmpte != NULL) { KASSERT(dstmpte->pindex == pmap_pde_pindex(addr), ("dstmpte pindex/addr mismatch")); dstmpte->ref_count++; } else if ((dstmpte = pmap_allocpte(dst_pmap, addr, NULL)) == NULL) goto out; dst_pte = (pt_entry_t *) PHYS_TO_DMAP(VM_PAGE_TO_PHYS(dstmpte)); dst_pte = &dst_pte[pmap_pte_index(addr)]; if (*dst_pte == 0 && pmap_try_insert_pv_entry(dst_pmap, addr, PHYS_TO_VM_PAGE(ptetemp & PG_FRAME), &lock)) { /* * Clear the wired, modified, and accessed * (referenced) bits during the copy. */ *dst_pte = ptetemp & ~(PG_W | PG_M | PG_A); pmap_resident_count_inc(dst_pmap, 1); } else { pmap_abort_ptp(dst_pmap, addr, dstmpte); goto out; } /* Have we copied all of the valid mappings? */ if (dstmpte->ref_count >= srcmpte->ref_count) break; } } out: if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(src_pmap); PMAP_UNLOCK(dst_pmap); } int pmap_vmspace_copy(pmap_t dst_pmap, pmap_t src_pmap) { int error; if (dst_pmap->pm_type != src_pmap->pm_type || dst_pmap->pm_type != PT_X86 || (cpu_stdext_feature2 & CPUID_STDEXT2_PKU) == 0) return (0); for (;;) { if (dst_pmap < src_pmap) { PMAP_LOCK(dst_pmap); PMAP_LOCK(src_pmap); } else { PMAP_LOCK(src_pmap); PMAP_LOCK(dst_pmap); } error = pmap_pkru_copy(dst_pmap, src_pmap); /* Clean up partial copy on failure due to no memory. */ if (error == ENOMEM) pmap_pkru_deassign_all(dst_pmap); PMAP_UNLOCK(src_pmap); PMAP_UNLOCK(dst_pmap); if (error != ENOMEM) break; vm_wait(NULL); } return (error); } /* * Zero the specified hardware page. */ void pmap_zero_page(vm_page_t m) { vm_offset_t va = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m)); pagezero((void *)va); } /* * Zero an an area within a single hardware page. off and size must not * cover an area beyond a single hardware page. */ void pmap_zero_page_area(vm_page_t m, int off, int size) { vm_offset_t va = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m)); if (off == 0 && size == PAGE_SIZE) pagezero((void *)va); else bzero((char *)va + off, size); } /* * Copy 1 specified hardware page to another. */ void pmap_copy_page(vm_page_t msrc, vm_page_t mdst) { vm_offset_t src = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(msrc)); vm_offset_t dst = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(mdst)); pagecopy((void *)src, (void *)dst); } int unmapped_buf_allowed = 1; void pmap_copy_pages(vm_page_t ma[], vm_offset_t a_offset, vm_page_t mb[], vm_offset_t b_offset, int xfersize) { void *a_cp, *b_cp; vm_page_t pages[2]; vm_offset_t vaddr[2], a_pg_offset, b_pg_offset; int cnt; boolean_t mapped; while (xfersize > 0) { a_pg_offset = a_offset & PAGE_MASK; pages[0] = ma[a_offset >> PAGE_SHIFT]; b_pg_offset = b_offset & PAGE_MASK; pages[1] = mb[b_offset >> PAGE_SHIFT]; cnt = min(xfersize, PAGE_SIZE - a_pg_offset); cnt = min(cnt, PAGE_SIZE - b_pg_offset); mapped = pmap_map_io_transient(pages, vaddr, 2, FALSE); a_cp = (char *)vaddr[0] + a_pg_offset; b_cp = (char *)vaddr[1] + b_pg_offset; bcopy(a_cp, b_cp, cnt); if (__predict_false(mapped)) pmap_unmap_io_transient(pages, vaddr, 2, FALSE); a_offset += cnt; b_offset += cnt; xfersize -= cnt; } } /* * Returns true if the pmap's pv is one of the first * 16 pvs linked to from this page. This count may * be changed upwards or downwards in the future; it * is only necessary that true be returned for a small * subset of pmaps for proper page aging. */ boolean_t pmap_page_exists_quick(pmap_t pmap, vm_page_t m) { struct md_page *pvh; struct rwlock *lock; pv_entry_t pv; int loops = 0; boolean_t rv; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_page_exists_quick: page %p is not managed", m)); rv = FALSE; lock = VM_PAGE_TO_PV_LIST_LOCK(m); rw_rlock(lock); TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) { if (PV_PMAP(pv) == pmap) { rv = TRUE; break; } loops++; if (loops >= 16) break; } if (!rv && loops < 16 && (m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) { if (PV_PMAP(pv) == pmap) { rv = TRUE; break; } loops++; if (loops >= 16) break; } } rw_runlock(lock); return (rv); } /* * pmap_page_wired_mappings: * * Return the number of managed mappings to the given physical page * that are wired. */ int pmap_page_wired_mappings(vm_page_t m) { struct rwlock *lock; struct md_page *pvh; pmap_t pmap; pt_entry_t *pte; pv_entry_t pv; int count, md_gen, pvh_gen; if ((m->oflags & VPO_UNMANAGED) != 0) return (0); lock = VM_PAGE_TO_PV_LIST_LOCK(m); rw_rlock(lock); restart: count = 0; TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { md_gen = m->md.pv_gen; rw_runlock(lock); PMAP_LOCK(pmap); rw_rlock(lock); if (md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } pte = pmap_pte(pmap, pv->pv_va); if ((*pte & PG_W) != 0) count++; PMAP_UNLOCK(pmap); } if ((m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { md_gen = m->md.pv_gen; pvh_gen = pvh->pv_gen; rw_runlock(lock); PMAP_LOCK(pmap); rw_rlock(lock); if (md_gen != m->md.pv_gen || pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } pte = pmap_pde(pmap, pv->pv_va); if ((*pte & PG_W) != 0) count++; PMAP_UNLOCK(pmap); } } rw_runlock(lock); return (count); } /* * Returns TRUE if the given page is mapped individually or as part of * a 2mpage. Otherwise, returns FALSE. */ boolean_t pmap_page_is_mapped(vm_page_t m) { struct rwlock *lock; boolean_t rv; if ((m->oflags & VPO_UNMANAGED) != 0) return (FALSE); lock = VM_PAGE_TO_PV_LIST_LOCK(m); rw_rlock(lock); rv = !TAILQ_EMPTY(&m->md.pv_list) || ((m->flags & PG_FICTITIOUS) == 0 && !TAILQ_EMPTY(&pa_to_pvh(VM_PAGE_TO_PHYS(m))->pv_list)); rw_runlock(lock); return (rv); } /* * Destroy all managed, non-wired mappings in the given user-space * pmap. This pmap cannot be active on any processor besides the * caller. * * This function cannot be applied to the kernel pmap. Moreover, it * is not intended for general use. It is only to be used during * process termination. Consequently, it can be implemented in ways * that make it faster than pmap_remove(). First, it can more quickly * destroy mappings by iterating over the pmap's collection of PV * entries, rather than searching the page table. Second, it doesn't * have to test and clear the page table entries atomically, because * no processor is currently accessing the user address space. In * particular, a page table entry's dirty bit won't change state once * this function starts. * * Although this function destroys all of the pmap's managed, * non-wired mappings, it can delay and batch the invalidation of TLB * entries without calling pmap_delayed_invl_start() and * pmap_delayed_invl_finish(). Because the pmap is not active on * any other processor, none of these TLB entries will ever be used * before their eventual invalidation. Consequently, there is no need * for either pmap_remove_all() or pmap_remove_write() to wait for * that eventual TLB invalidation. */ void pmap_remove_pages(pmap_t pmap) { pd_entry_t ptepde; pt_entry_t *pte, tpte; pt_entry_t PG_M, PG_RW, PG_V; struct spglist free; struct pv_chunklist free_chunks[PMAP_MEMDOM]; vm_page_t m, mpte, mt; pv_entry_t pv; struct md_page *pvh; struct pv_chunk *pc, *npc; struct rwlock *lock; int64_t bit; uint64_t inuse, bitmask; int allfree, field, freed, i, idx; boolean_t superpage; vm_paddr_t pa; /* * Assert that the given pmap is only active on the current * CPU. Unfortunately, we cannot block another CPU from * activating the pmap while this function is executing. */ KASSERT(pmap == PCPU_GET(curpmap), ("non-current pmap %p", pmap)); #ifdef INVARIANTS { cpuset_t other_cpus; other_cpus = all_cpus; critical_enter(); CPU_CLR(PCPU_GET(cpuid), &other_cpus); CPU_AND(&other_cpus, &pmap->pm_active); critical_exit(); KASSERT(CPU_EMPTY(&other_cpus), ("pmap active %p", pmap)); } #endif lock = NULL; PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); for (i = 0; i < PMAP_MEMDOM; i++) TAILQ_INIT(&free_chunks[i]); SLIST_INIT(&free); PMAP_LOCK(pmap); TAILQ_FOREACH_SAFE(pc, &pmap->pm_pvchunk, pc_list, npc) { allfree = 1; freed = 0; for (field = 0; field < _NPCM; field++) { inuse = ~pc->pc_map[field] & pc_freemask[field]; while (inuse != 0) { bit = bsfq(inuse); bitmask = 1UL << bit; idx = field * 64 + bit; pv = &pc->pc_pventry[idx]; inuse &= ~bitmask; pte = pmap_pdpe(pmap, pv->pv_va); ptepde = *pte; pte = pmap_pdpe_to_pde(pte, pv->pv_va); tpte = *pte; if ((tpte & (PG_PS | PG_V)) == PG_V) { superpage = FALSE; ptepde = tpte; pte = (pt_entry_t *)PHYS_TO_DMAP(tpte & PG_FRAME); pte = &pte[pmap_pte_index(pv->pv_va)]; tpte = *pte; } else { /* * Keep track whether 'tpte' is a * superpage explicitly instead of * relying on PG_PS being set. * * This is because PG_PS is numerically * identical to PG_PTE_PAT and thus a * regular page could be mistaken for * a superpage. */ superpage = TRUE; } if ((tpte & PG_V) == 0) { panic("bad pte va %lx pte %lx", pv->pv_va, tpte); } /* * We cannot remove wired pages from a process' mapping at this time */ if (tpte & PG_W) { allfree = 0; continue; } if (superpage) pa = tpte & PG_PS_FRAME; else pa = tpte & PG_FRAME; m = PHYS_TO_VM_PAGE(pa); KASSERT(m->phys_addr == pa, ("vm_page_t %p phys_addr mismatch %016jx %016jx", m, (uintmax_t)m->phys_addr, (uintmax_t)tpte)); KASSERT((m->flags & PG_FICTITIOUS) != 0 || m < &vm_page_array[vm_page_array_size], ("pmap_remove_pages: bad tpte %#jx", (uintmax_t)tpte)); pte_clear(pte); /* * Update the vm_page_t clean/reference bits. */ if ((tpte & (PG_M | PG_RW)) == (PG_M | PG_RW)) { if (superpage) { for (mt = m; mt < &m[NBPDR / PAGE_SIZE]; mt++) vm_page_dirty(mt); } else vm_page_dirty(m); } CHANGE_PV_LIST_LOCK_TO_VM_PAGE(&lock, m); /* Mark free */ pc->pc_map[field] |= bitmask; if (superpage) { pmap_resident_count_dec(pmap, NBPDR / PAGE_SIZE); pvh = pa_to_pvh(tpte & PG_PS_FRAME); TAILQ_REMOVE(&pvh->pv_list, pv, pv_next); pvh->pv_gen++; if (TAILQ_EMPTY(&pvh->pv_list)) { for (mt = m; mt < &m[NBPDR / PAGE_SIZE]; mt++) if ((mt->a.flags & PGA_WRITEABLE) != 0 && TAILQ_EMPTY(&mt->md.pv_list)) vm_page_aflag_clear(mt, PGA_WRITEABLE); } mpte = pmap_remove_pt_page(pmap, pv->pv_va); if (mpte != NULL) { KASSERT(mpte->valid == VM_PAGE_BITS_ALL, ("pmap_remove_pages: pte page not promoted")); pmap_resident_count_dec(pmap, 1); KASSERT(mpte->ref_count == NPTEPG, ("pmap_remove_pages: pte page reference count error")); mpte->ref_count = 0; pmap_add_delayed_free_list(mpte, &free, FALSE); } } else { pmap_resident_count_dec(pmap, 1); TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; if ((m->a.flags & PGA_WRITEABLE) != 0 && TAILQ_EMPTY(&m->md.pv_list) && (m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); if (TAILQ_EMPTY(&pvh->pv_list)) vm_page_aflag_clear(m, PGA_WRITEABLE); } } pmap_unuse_pt(pmap, pv->pv_va, ptepde, &free); freed++; } } PV_STAT(atomic_add_long(&pv_entry_frees, freed)); PV_STAT(atomic_add_int(&pv_entry_spare, freed)); PV_STAT(atomic_subtract_long(&pv_entry_count, freed)); if (allfree) { TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_TAIL(&free_chunks[pc_to_domain(pc)], pc, pc_list); } } if (lock != NULL) rw_wunlock(lock); pmap_invalidate_all(pmap); pmap_pkru_deassign_all(pmap); free_pv_chunk_batch((struct pv_chunklist *)&free_chunks); PMAP_UNLOCK(pmap); vm_page_free_pages_toq(&free, true); } static boolean_t pmap_page_test_mappings(vm_page_t m, boolean_t accessed, boolean_t modified) { struct rwlock *lock; pv_entry_t pv; struct md_page *pvh; pt_entry_t *pte, mask; pt_entry_t PG_A, PG_M, PG_RW, PG_V; pmap_t pmap; int md_gen, pvh_gen; boolean_t rv; rv = FALSE; lock = VM_PAGE_TO_PV_LIST_LOCK(m); rw_rlock(lock); restart: TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { md_gen = m->md.pv_gen; rw_runlock(lock); PMAP_LOCK(pmap); rw_rlock(lock); if (md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } pte = pmap_pte(pmap, pv->pv_va); mask = 0; if (modified) { PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); mask |= PG_RW | PG_M; } if (accessed) { PG_A = pmap_accessed_bit(pmap); PG_V = pmap_valid_bit(pmap); mask |= PG_V | PG_A; } rv = (*pte & mask) == mask; PMAP_UNLOCK(pmap); if (rv) goto out; } if ((m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { md_gen = m->md.pv_gen; pvh_gen = pvh->pv_gen; rw_runlock(lock); PMAP_LOCK(pmap); rw_rlock(lock); if (md_gen != m->md.pv_gen || pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } pte = pmap_pde(pmap, pv->pv_va); mask = 0; if (modified) { PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); mask |= PG_RW | PG_M; } if (accessed) { PG_A = pmap_accessed_bit(pmap); PG_V = pmap_valid_bit(pmap); mask |= PG_V | PG_A; } rv = (*pte & mask) == mask; PMAP_UNLOCK(pmap); if (rv) goto out; } } out: rw_runlock(lock); return (rv); } /* * pmap_is_modified: * * Return whether or not the specified physical page was modified * in any physical maps. */ boolean_t pmap_is_modified(vm_page_t m) { KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_is_modified: page %p is not managed", m)); /* * If the page is not busied then this check is racy. */ if (!pmap_page_is_write_mapped(m)) return (FALSE); return (pmap_page_test_mappings(m, FALSE, TRUE)); } /* * pmap_is_prefaultable: * * Return whether or not the specified virtual address is eligible * for prefault. */ boolean_t pmap_is_prefaultable(pmap_t pmap, vm_offset_t addr) { pd_entry_t *pde; pt_entry_t *pte, PG_V; boolean_t rv; PG_V = pmap_valid_bit(pmap); rv = FALSE; PMAP_LOCK(pmap); pde = pmap_pde(pmap, addr); if (pde != NULL && (*pde & (PG_PS | PG_V)) == PG_V) { pte = pmap_pde_to_pte(pde, addr); rv = (*pte & PG_V) == 0; } PMAP_UNLOCK(pmap); return (rv); } /* * pmap_is_referenced: * * Return whether or not the specified physical page was referenced * in any physical maps. */ boolean_t pmap_is_referenced(vm_page_t m) { KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_is_referenced: page %p is not managed", m)); return (pmap_page_test_mappings(m, TRUE, FALSE)); } /* * Clear the write and modified bits in each of the given page's mappings. */ void pmap_remove_write(vm_page_t m) { struct md_page *pvh; pmap_t pmap; struct rwlock *lock; pv_entry_t next_pv, pv; pd_entry_t *pde; pt_entry_t oldpte, *pte, PG_M, PG_RW; vm_offset_t va; int pvh_gen, md_gen; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_remove_write: page %p is not managed", m)); vm_page_assert_busied(m); if (!pmap_page_is_write_mapped(m)) return; lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(VM_PAGE_TO_PHYS(m)); retry_pv_loop: rw_wlock(lock); TAILQ_FOREACH_SAFE(pv, &pvh->pv_list, pv_next, next_pv) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); rw_wunlock(lock); goto retry_pv_loop; } } PG_RW = pmap_rw_bit(pmap); va = pv->pv_va; pde = pmap_pde(pmap, va); if ((*pde & PG_RW) != 0) (void)pmap_demote_pde_locked(pmap, pde, va, &lock); KASSERT(lock == VM_PAGE_TO_PV_LIST_LOCK(m), ("inconsistent pv lock %p %p for page %p", lock, VM_PAGE_TO_PV_LIST_LOCK(m), m)); PMAP_UNLOCK(pmap); } TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; md_gen = m->md.pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); rw_wunlock(lock); goto retry_pv_loop; } } PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); pde = pmap_pde(pmap, pv->pv_va); KASSERT((*pde & PG_PS) == 0, ("pmap_remove_write: found a 2mpage in page %p's pv list", m)); pte = pmap_pde_to_pte(pde, pv->pv_va); retry: oldpte = *pte; if (oldpte & PG_RW) { if (!atomic_cmpset_long(pte, oldpte, oldpte & ~(PG_RW | PG_M))) goto retry; if ((oldpte & PG_M) != 0) vm_page_dirty(m); pmap_invalidate_page(pmap, pv->pv_va); } PMAP_UNLOCK(pmap); } rw_wunlock(lock); vm_page_aflag_clear(m, PGA_WRITEABLE); pmap_delayed_invl_wait(m); } static __inline boolean_t safe_to_clear_referenced(pmap_t pmap, pt_entry_t pte) { if (!pmap_emulate_ad_bits(pmap)) return (TRUE); KASSERT(pmap->pm_type == PT_EPT, ("invalid pm_type %d", pmap->pm_type)); /* * XWR = 010 or 110 will cause an unconditional EPT misconfiguration * so we don't let the referenced (aka EPT_PG_READ) bit to be cleared * if the EPT_PG_WRITE bit is set. */ if ((pte & EPT_PG_WRITE) != 0) return (FALSE); /* * XWR = 100 is allowed only if the PMAP_SUPPORTS_EXEC_ONLY is set. */ if ((pte & EPT_PG_EXECUTE) == 0 || ((pmap->pm_flags & PMAP_SUPPORTS_EXEC_ONLY) != 0)) return (TRUE); else return (FALSE); } /* * pmap_ts_referenced: * * Return a count of reference bits for a page, clearing those bits. * It is not necessary for every reference bit to be cleared, but it * is necessary that 0 only be returned when there are truly no * reference bits set. * * As an optimization, update the page's dirty field if a modified bit is * found while counting reference bits. This opportunistic update can be * performed at low cost and can eliminate the need for some future calls * to pmap_is_modified(). However, since this function stops after * finding PMAP_TS_REFERENCED_MAX reference bits, it may not detect some * dirty pages. Those dirty pages will only be detected by a future call * to pmap_is_modified(). * * A DI block is not needed within this function, because * invalidations are performed before the PV list lock is * released. */ int pmap_ts_referenced(vm_page_t m) { struct md_page *pvh; pv_entry_t pv, pvf; pmap_t pmap; struct rwlock *lock; pd_entry_t oldpde, *pde; pt_entry_t *pte, PG_A, PG_M, PG_RW; vm_offset_t va; vm_paddr_t pa; int cleared, md_gen, not_cleared, pvh_gen; struct spglist free; boolean_t demoted; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_ts_referenced: page %p is not managed", m)); SLIST_INIT(&free); cleared = 0; pa = VM_PAGE_TO_PHYS(m); lock = PHYS_TO_PV_LIST_LOCK(pa); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(pa); rw_wlock(lock); retry: not_cleared = 0; if ((pvf = TAILQ_FIRST(&pvh->pv_list)) == NULL) goto small_mappings; pv = pvf; do { if (pvf == NULL) pvf = pv; pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); goto retry; } } PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); va = pv->pv_va; pde = pmap_pde(pmap, pv->pv_va); oldpde = *pde; if ((oldpde & (PG_M | PG_RW)) == (PG_M | PG_RW)) { /* * Although "oldpde" is mapping a 2MB page, because * this function is called at a 4KB page granularity, * we only update the 4KB page under test. */ vm_page_dirty(m); } if ((oldpde & PG_A) != 0) { /* * Since this reference bit is shared by 512 4KB * pages, it should not be cleared every time it is * tested. Apply a simple "hash" function on the * physical page number, the virtual superpage number, * and the pmap address to select one 4KB page out of * the 512 on which testing the reference bit will * result in clearing that reference bit. This * function is designed to avoid the selection of the * same 4KB page for every 2MB page mapping. * * On demotion, a mapping that hasn't been referenced * is simply destroyed. To avoid the possibility of a * subsequent page fault on a demoted wired mapping, * always leave its reference bit set. Moreover, * since the superpage is wired, the current state of * its reference bit won't affect page replacement. */ if ((((pa >> PAGE_SHIFT) ^ (pv->pv_va >> PDRSHIFT) ^ (uintptr_t)pmap) & (NPTEPG - 1)) == 0 && (oldpde & PG_W) == 0) { if (safe_to_clear_referenced(pmap, oldpde)) { atomic_clear_long(pde, PG_A); pmap_invalidate_page(pmap, pv->pv_va); demoted = FALSE; } else if (pmap_demote_pde_locked(pmap, pde, pv->pv_va, &lock)) { /* * Remove the mapping to a single page * so that a subsequent access may * repromote. Since the underlying * page table page is fully populated, * this removal never frees a page * table page. */ demoted = TRUE; va += VM_PAGE_TO_PHYS(m) - (oldpde & PG_PS_FRAME); pte = pmap_pde_to_pte(pde, va); pmap_remove_pte(pmap, pte, va, *pde, NULL, &lock); pmap_invalidate_page(pmap, va); } else demoted = TRUE; if (demoted) { /* * The superpage mapping was removed * entirely and therefore 'pv' is no * longer valid. */ if (pvf == pv) pvf = NULL; pv = NULL; } cleared++; KASSERT(lock == VM_PAGE_TO_PV_LIST_LOCK(m), ("inconsistent pv lock %p %p for page %p", lock, VM_PAGE_TO_PV_LIST_LOCK(m), m)); } else not_cleared++; } PMAP_UNLOCK(pmap); /* Rotate the PV list if it has more than one entry. */ if (pv != NULL && TAILQ_NEXT(pv, pv_next) != NULL) { TAILQ_REMOVE(&pvh->pv_list, pv, pv_next); TAILQ_INSERT_TAIL(&pvh->pv_list, pv, pv_next); pvh->pv_gen++; } if (cleared + not_cleared >= PMAP_TS_REFERENCED_MAX) goto out; } while ((pv = TAILQ_FIRST(&pvh->pv_list)) != pvf); small_mappings: if ((pvf = TAILQ_FIRST(&m->md.pv_list)) == NULL) goto out; pv = pvf; do { if (pvf == NULL) pvf = pv; pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; md_gen = m->md.pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); goto retry; } } PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); pde = pmap_pde(pmap, pv->pv_va); KASSERT((*pde & PG_PS) == 0, ("pmap_ts_referenced: found a 2mpage in page %p's pv list", m)); pte = pmap_pde_to_pte(pde, pv->pv_va); if ((*pte & (PG_M | PG_RW)) == (PG_M | PG_RW)) vm_page_dirty(m); if ((*pte & PG_A) != 0) { if (safe_to_clear_referenced(pmap, *pte)) { atomic_clear_long(pte, PG_A); pmap_invalidate_page(pmap, pv->pv_va); cleared++; } else if ((*pte & PG_W) == 0) { /* * Wired pages cannot be paged out so * doing accessed bit emulation for * them is wasted effort. We do the * hard work for unwired pages only. */ pmap_remove_pte(pmap, pte, pv->pv_va, *pde, &free, &lock); pmap_invalidate_page(pmap, pv->pv_va); cleared++; if (pvf == pv) pvf = NULL; pv = NULL; KASSERT(lock == VM_PAGE_TO_PV_LIST_LOCK(m), ("inconsistent pv lock %p %p for page %p", lock, VM_PAGE_TO_PV_LIST_LOCK(m), m)); } else not_cleared++; } PMAP_UNLOCK(pmap); /* Rotate the PV list if it has more than one entry. */ if (pv != NULL && TAILQ_NEXT(pv, pv_next) != NULL) { TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; } } while ((pv = TAILQ_FIRST(&m->md.pv_list)) != pvf && cleared + not_cleared < PMAP_TS_REFERENCED_MAX); out: rw_wunlock(lock); vm_page_free_pages_toq(&free, true); return (cleared + not_cleared); } /* * Apply the given advice to the specified range of addresses within the * given pmap. Depending on the advice, clear the referenced and/or * modified flags in each mapping and set the mapped page's dirty field. */ void pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, int advice) { struct rwlock *lock; pml4_entry_t *pml4e; pdp_entry_t *pdpe; pd_entry_t oldpde, *pde; pt_entry_t *pte, PG_A, PG_G, PG_M, PG_RW, PG_V; vm_offset_t va, va_next; vm_page_t m; bool anychanged; if (advice != MADV_DONTNEED && advice != MADV_FREE) return; /* * A/D bit emulation requires an alternate code path when clearing * the modified and accessed bits below. Since this function is * advisory in nature we skip it entirely for pmaps that require * A/D bit emulation. */ if (pmap_emulate_ad_bits(pmap)) return; PG_A = pmap_accessed_bit(pmap); PG_G = pmap_global_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); anychanged = false; pmap_delayed_invl_start(); PMAP_LOCK(pmap); for (; sva < eva; sva = va_next) { pml4e = pmap_pml4e(pmap, sva); if ((*pml4e & PG_V) == 0) { va_next = (sva + NBPML4) & ~PML4MASK; if (va_next < sva) va_next = eva; continue; } pdpe = pmap_pml4e_to_pdpe(pml4e, sva); if ((*pdpe & PG_V) == 0) { va_next = (sva + NBPDP) & ~PDPMASK; if (va_next < sva) va_next = eva; continue; } va_next = (sva + NBPDR) & ~PDRMASK; if (va_next < sva) va_next = eva; pde = pmap_pdpe_to_pde(pdpe, sva); oldpde = *pde; if ((oldpde & PG_V) == 0) continue; else if ((oldpde & PG_PS) != 0) { if ((oldpde & PG_MANAGED) == 0) continue; lock = NULL; if (!pmap_demote_pde_locked(pmap, pde, sva, &lock)) { if (lock != NULL) rw_wunlock(lock); /* * The large page mapping was destroyed. */ continue; } /* * Unless the page mappings are wired, remove the * mapping to a single page so that a subsequent * access may repromote. Choosing the last page * within the address range [sva, min(va_next, eva)) * generally results in more repromotions. Since the * underlying page table page is fully populated, this * removal never frees a page table page. */ if ((oldpde & PG_W) == 0) { va = eva; if (va > va_next) va = va_next; va -= PAGE_SIZE; KASSERT(va >= sva, ("pmap_advise: no address gap")); pte = pmap_pde_to_pte(pde, va); KASSERT((*pte & PG_V) != 0, ("pmap_advise: invalid PTE")); pmap_remove_pte(pmap, pte, va, *pde, NULL, &lock); anychanged = true; } if (lock != NULL) rw_wunlock(lock); } if (va_next > eva) va_next = eva; va = va_next; for (pte = pmap_pde_to_pte(pde, sva); sva != va_next; pte++, sva += PAGE_SIZE) { if ((*pte & (PG_MANAGED | PG_V)) != (PG_MANAGED | PG_V)) goto maybe_invlrng; else if ((*pte & (PG_M | PG_RW)) == (PG_M | PG_RW)) { if (advice == MADV_DONTNEED) { /* * Future calls to pmap_is_modified() * can be avoided by making the page * dirty now. */ m = PHYS_TO_VM_PAGE(*pte & PG_FRAME); vm_page_dirty(m); } atomic_clear_long(pte, PG_M | PG_A); } else if ((*pte & PG_A) != 0) atomic_clear_long(pte, PG_A); else goto maybe_invlrng; if ((*pte & PG_G) != 0) { if (va == va_next) va = sva; } else anychanged = true; continue; maybe_invlrng: if (va != va_next) { pmap_invalidate_range(pmap, va, sva); va = va_next; } } if (va != va_next) pmap_invalidate_range(pmap, va, sva); } if (anychanged) pmap_invalidate_all(pmap); PMAP_UNLOCK(pmap); pmap_delayed_invl_finish(); } /* * Clear the modify bits on the specified physical page. */ void pmap_clear_modify(vm_page_t m) { struct md_page *pvh; pmap_t pmap; pv_entry_t next_pv, pv; pd_entry_t oldpde, *pde; pt_entry_t *pte, PG_M, PG_RW; struct rwlock *lock; vm_offset_t va; int md_gen, pvh_gen; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_clear_modify: page %p is not managed", m)); vm_page_assert_busied(m); if (!pmap_page_is_write_mapped(m)) return; pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(VM_PAGE_TO_PHYS(m)); lock = VM_PAGE_TO_PV_LIST_LOCK(m); rw_wlock(lock); restart: TAILQ_FOREACH_SAFE(pv, &pvh->pv_list, pv_next, next_pv) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); va = pv->pv_va; pde = pmap_pde(pmap, va); oldpde = *pde; /* If oldpde has PG_RW set, then it also has PG_M set. */ if ((oldpde & PG_RW) != 0 && pmap_demote_pde_locked(pmap, pde, va, &lock) && (oldpde & PG_W) == 0) { /* * Write protect the mapping to a single page so that * a subsequent write access may repromote. */ va += VM_PAGE_TO_PHYS(m) - (oldpde & PG_PS_FRAME); pte = pmap_pde_to_pte(pde, va); atomic_clear_long(pte, PG_M | PG_RW); vm_page_dirty(m); pmap_invalidate_page(pmap, va); } PMAP_UNLOCK(pmap); } TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { md_gen = m->md.pv_gen; pvh_gen = pvh->pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } PG_M = pmap_modified_bit(pmap); PG_RW = pmap_rw_bit(pmap); pde = pmap_pde(pmap, pv->pv_va); KASSERT((*pde & PG_PS) == 0, ("pmap_clear_modify: found" " a 2mpage in page %p's pv list", m)); pte = pmap_pde_to_pte(pde, pv->pv_va); if ((*pte & (PG_M | PG_RW)) == (PG_M | PG_RW)) { atomic_clear_long(pte, PG_M); pmap_invalidate_page(pmap, pv->pv_va); } PMAP_UNLOCK(pmap); } rw_wunlock(lock); } /* * Miscellaneous support routines follow */ /* Adjust the properties for a leaf page table entry. */ static __inline void pmap_pte_props(pt_entry_t *pte, u_long bits, u_long mask) { u_long opte, npte; opte = *(u_long *)pte; do { npte = opte & ~mask; npte |= bits; } while (npte != opte && !atomic_fcmpset_long((u_long *)pte, &opte, npte)); } /* * Map a set of physical memory pages into the kernel virtual * address space. Return a pointer to where it is mapped. This * routine is intended to be used for mapping device memory, * NOT real memory. */ static void * pmap_mapdev_internal(vm_paddr_t pa, vm_size_t size, int mode, int flags) { struct pmap_preinit_mapping *ppim; vm_offset_t va, offset; vm_size_t tmpsize; int i; offset = pa & PAGE_MASK; size = round_page(offset + size); pa = trunc_page(pa); if (!pmap_initialized) { va = 0; for (i = 0; i < PMAP_PREINIT_MAPPING_COUNT; i++) { ppim = pmap_preinit_mapping + i; if (ppim->va == 0) { ppim->pa = pa; ppim->sz = size; ppim->mode = mode; ppim->va = virtual_avail; virtual_avail += size; va = ppim->va; break; } } if (va == 0) panic("%s: too many preinit mappings", __func__); } else { /* * If we have a preinit mapping, re-use it. */ for (i = 0; i < PMAP_PREINIT_MAPPING_COUNT; i++) { ppim = pmap_preinit_mapping + i; if (ppim->pa == pa && ppim->sz == size && (ppim->mode == mode || (flags & MAPDEV_SETATTR) == 0)) return ((void *)(ppim->va + offset)); } /* * If the specified range of physical addresses fits within * the direct map window, use the direct map. */ if (pa < dmaplimit && pa + size <= dmaplimit) { va = PHYS_TO_DMAP(pa); if ((flags & MAPDEV_SETATTR) != 0) { PMAP_LOCK(kernel_pmap); i = pmap_change_props_locked(va, size, PROT_NONE, mode, flags); PMAP_UNLOCK(kernel_pmap); } else i = 0; if (!i) return ((void *)(va + offset)); } va = kva_alloc(size); if (va == 0) panic("%s: Couldn't allocate KVA", __func__); } for (tmpsize = 0; tmpsize < size; tmpsize += PAGE_SIZE) pmap_kenter_attr(va + tmpsize, pa + tmpsize, mode); pmap_invalidate_range(kernel_pmap, va, va + tmpsize); if ((flags & MAPDEV_FLUSHCACHE) != 0) pmap_invalidate_cache_range(va, va + tmpsize); return ((void *)(va + offset)); } void * pmap_mapdev_attr(vm_paddr_t pa, vm_size_t size, int mode) { return (pmap_mapdev_internal(pa, size, mode, MAPDEV_FLUSHCACHE | MAPDEV_SETATTR)); } void * pmap_mapdev(vm_paddr_t pa, vm_size_t size) { return (pmap_mapdev_attr(pa, size, PAT_UNCACHEABLE)); } void * pmap_mapdev_pciecfg(vm_paddr_t pa, vm_size_t size) { return (pmap_mapdev_internal(pa, size, PAT_UNCACHEABLE, MAPDEV_SETATTR)); } void * pmap_mapbios(vm_paddr_t pa, vm_size_t size) { return (pmap_mapdev_internal(pa, size, PAT_WRITE_BACK, MAPDEV_FLUSHCACHE)); } void pmap_unmapdev(vm_offset_t va, vm_size_t size) { struct pmap_preinit_mapping *ppim; vm_offset_t offset; int i; /* If we gave a direct map region in pmap_mapdev, do nothing */ if (va >= DMAP_MIN_ADDRESS && va < DMAP_MAX_ADDRESS) return; offset = va & PAGE_MASK; size = round_page(offset + size); va = trunc_page(va); for (i = 0; i < PMAP_PREINIT_MAPPING_COUNT; i++) { ppim = pmap_preinit_mapping + i; if (ppim->va == va && ppim->sz == size) { if (pmap_initialized) return; ppim->pa = 0; ppim->va = 0; ppim->sz = 0; ppim->mode = 0; if (va + size == virtual_avail) virtual_avail = va; return; } } if (pmap_initialized) kva_free(va, size); } /* * Tries to demote a 1GB page mapping. */ static boolean_t pmap_demote_pdpe(pmap_t pmap, pdp_entry_t *pdpe, vm_offset_t va) { pdp_entry_t newpdpe, oldpdpe; pd_entry_t *firstpde, newpde, *pde; pt_entry_t PG_A, PG_M, PG_RW, PG_V; vm_paddr_t pdpgpa; vm_page_t pdpg; PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); PMAP_LOCK_ASSERT(pmap, MA_OWNED); oldpdpe = *pdpe; KASSERT((oldpdpe & (PG_PS | PG_V)) == (PG_PS | PG_V), ("pmap_demote_pdpe: oldpdpe is missing PG_PS and/or PG_V")); if ((pdpg = vm_page_alloc(NULL, va >> PDPSHIFT, VM_ALLOC_INTERRUPT | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED)) == NULL) { CTR2(KTR_PMAP, "pmap_demote_pdpe: failure for va %#lx" " in pmap %p", va, pmap); return (FALSE); } pdpgpa = VM_PAGE_TO_PHYS(pdpg); firstpde = (pd_entry_t *)PHYS_TO_DMAP(pdpgpa); newpdpe = pdpgpa | PG_M | PG_A | (oldpdpe & PG_U) | PG_RW | PG_V; KASSERT((oldpdpe & PG_A) != 0, ("pmap_demote_pdpe: oldpdpe is missing PG_A")); KASSERT((oldpdpe & (PG_M | PG_RW)) != PG_RW, ("pmap_demote_pdpe: oldpdpe is missing PG_M")); newpde = oldpdpe; /* * Initialize the page directory page. */ for (pde = firstpde; pde < firstpde + NPDEPG; pde++) { *pde = newpde; newpde += NBPDR; } /* * Demote the mapping. */ *pdpe = newpdpe; /* * Invalidate a stale recursive mapping of the page directory page. */ pmap_invalidate_page(pmap, (vm_offset_t)vtopde(va)); pmap_pdpe_demotions++; CTR2(KTR_PMAP, "pmap_demote_pdpe: success for va %#lx" " in pmap %p", va, pmap); return (TRUE); } /* * Sets the memory attribute for the specified page. */ void pmap_page_set_memattr(vm_page_t m, vm_memattr_t ma) { m->md.pat_mode = ma; /* * If "m" is a normal page, update its direct mapping. This update * can be relied upon to perform any cache operations that are * required for data coherence. */ if ((m->flags & PG_FICTITIOUS) == 0 && pmap_change_attr(PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m)), PAGE_SIZE, m->md.pat_mode)) panic("memory attribute change on the direct map failed"); } /* * Changes the specified virtual address range's memory type to that given by * the parameter "mode". The specified virtual address range must be * completely contained within either the direct map or the kernel map. If * the virtual address range is contained within the kernel map, then the * memory type for each of the corresponding ranges of the direct map is also * changed. (The corresponding ranges of the direct map are those ranges that * map the same physical pages as the specified virtual address range.) These * changes to the direct map are necessary because Intel describes the * behavior of their processors as "undefined" if two or more mappings to the * same physical page have different memory types. * * Returns zero if the change completed successfully, and either EINVAL or * ENOMEM if the change failed. Specifically, EINVAL is returned if some part * of the virtual address range was not mapped, and ENOMEM is returned if * there was insufficient memory available to complete the change. In the * latter case, the memory type may have been changed on some part of the * virtual address range or the direct map. */ int pmap_change_attr(vm_offset_t va, vm_size_t size, int mode) { int error; PMAP_LOCK(kernel_pmap); error = pmap_change_props_locked(va, size, PROT_NONE, mode, MAPDEV_FLUSHCACHE); PMAP_UNLOCK(kernel_pmap); return (error); } /* * Changes the specified virtual address range's protections to those * specified by "prot". Like pmap_change_attr(), protections for aliases * in the direct map are updated as well. Protections on aliasing mappings may * be a subset of the requested protections; for example, mappings in the direct * map are never executable. */ int pmap_change_prot(vm_offset_t va, vm_size_t size, vm_prot_t prot) { int error; /* Only supported within the kernel map. */ if (va < VM_MIN_KERNEL_ADDRESS) return (EINVAL); PMAP_LOCK(kernel_pmap); error = pmap_change_props_locked(va, size, prot, -1, MAPDEV_ASSERTVALID); PMAP_UNLOCK(kernel_pmap); return (error); } static int pmap_change_props_locked(vm_offset_t va, vm_size_t size, vm_prot_t prot, int mode, int flags) { vm_offset_t base, offset, tmpva; vm_paddr_t pa_start, pa_end, pa_end1; pdp_entry_t *pdpe; pd_entry_t *pde, pde_bits, pde_mask; pt_entry_t *pte, pte_bits, pte_mask; int error; bool changed; PMAP_LOCK_ASSERT(kernel_pmap, MA_OWNED); base = trunc_page(va); offset = va & PAGE_MASK; size = round_page(offset + size); /* * Only supported on kernel virtual addresses, including the direct * map but excluding the recursive map. */ if (base < DMAP_MIN_ADDRESS) return (EINVAL); /* * Construct our flag sets and masks. "bits" is the subset of * "mask" that will be set in each modified PTE. * * Mappings in the direct map are never allowed to be executable. */ pde_bits = pte_bits = 0; pde_mask = pte_mask = 0; if (mode != -1) { pde_bits |= pmap_cache_bits(kernel_pmap, mode, true); pde_mask |= X86_PG_PDE_CACHE; pte_bits |= pmap_cache_bits(kernel_pmap, mode, false); pte_mask |= X86_PG_PTE_CACHE; } if (prot != VM_PROT_NONE) { if ((prot & VM_PROT_WRITE) != 0) { pde_bits |= X86_PG_RW; pte_bits |= X86_PG_RW; } if ((prot & VM_PROT_EXECUTE) == 0 || va < VM_MIN_KERNEL_ADDRESS) { pde_bits |= pg_nx; pte_bits |= pg_nx; } pde_mask |= X86_PG_RW | pg_nx; pte_mask |= X86_PG_RW | pg_nx; } /* * Pages that aren't mapped aren't supported. Also break down 2MB pages * into 4KB pages if required. */ for (tmpva = base; tmpva < base + size; ) { pdpe = pmap_pdpe(kernel_pmap, tmpva); if (pdpe == NULL || *pdpe == 0) { KASSERT((flags & MAPDEV_ASSERTVALID) == 0, ("%s: addr %#lx is not mapped", __func__, tmpva)); return (EINVAL); } if (*pdpe & PG_PS) { /* * If the current 1GB page already has the required * properties, then we need not demote this page. Just * increment tmpva to the next 1GB page frame. */ if ((*pdpe & pde_mask) == pde_bits) { tmpva = trunc_1gpage(tmpva) + NBPDP; continue; } /* * If the current offset aligns with a 1GB page frame * and there is at least 1GB left within the range, then * we need not break down this page into 2MB pages. */ if ((tmpva & PDPMASK) == 0 && tmpva + PDPMASK < base + size) { tmpva += NBPDP; continue; } if (!pmap_demote_pdpe(kernel_pmap, pdpe, tmpva)) return (ENOMEM); } pde = pmap_pdpe_to_pde(pdpe, tmpva); if (*pde == 0) { KASSERT((flags & MAPDEV_ASSERTVALID) == 0, ("%s: addr %#lx is not mapped", __func__, tmpva)); return (EINVAL); } if (*pde & PG_PS) { /* * If the current 2MB page already has the required * properties, then we need not demote this page. Just * increment tmpva to the next 2MB page frame. */ if ((*pde & pde_mask) == pde_bits) { tmpva = trunc_2mpage(tmpva) + NBPDR; continue; } /* * If the current offset aligns with a 2MB page frame * and there is at least 2MB left within the range, then * we need not break down this page into 4KB pages. */ if ((tmpva & PDRMASK) == 0 && tmpva + PDRMASK < base + size) { tmpva += NBPDR; continue; } if (!pmap_demote_pde(kernel_pmap, pde, tmpva)) return (ENOMEM); } pte = pmap_pde_to_pte(pde, tmpva); if (*pte == 0) { KASSERT((flags & MAPDEV_ASSERTVALID) == 0, ("%s: addr %#lx is not mapped", __func__, tmpva)); return (EINVAL); } tmpva += PAGE_SIZE; } error = 0; /* * Ok, all the pages exist, so run through them updating their * properties if required. */ changed = false; pa_start = pa_end = 0; for (tmpva = base; tmpva < base + size; ) { pdpe = pmap_pdpe(kernel_pmap, tmpva); if (*pdpe & PG_PS) { if ((*pdpe & pde_mask) != pde_bits) { pmap_pte_props(pdpe, pde_bits, pde_mask); changed = true; } if (tmpva >= VM_MIN_KERNEL_ADDRESS && (*pdpe & PG_PS_FRAME) < dmaplimit) { if (pa_start == pa_end) { /* Start physical address run. */ pa_start = *pdpe & PG_PS_FRAME; pa_end = pa_start + NBPDP; } else if (pa_end == (*pdpe & PG_PS_FRAME)) pa_end += NBPDP; else { /* Run ended, update direct map. */ error = pmap_change_props_locked( PHYS_TO_DMAP(pa_start), pa_end - pa_start, prot, mode, flags); if (error != 0) break; /* Start physical address run. */ pa_start = *pdpe & PG_PS_FRAME; pa_end = pa_start + NBPDP; } } tmpva = trunc_1gpage(tmpva) + NBPDP; continue; } pde = pmap_pdpe_to_pde(pdpe, tmpva); if (*pde & PG_PS) { if ((*pde & pde_mask) != pde_bits) { pmap_pte_props(pde, pde_bits, pde_mask); changed = true; } if (tmpva >= VM_MIN_KERNEL_ADDRESS && (*pde & PG_PS_FRAME) < dmaplimit) { if (pa_start == pa_end) { /* Start physical address run. */ pa_start = *pde & PG_PS_FRAME; pa_end = pa_start + NBPDR; } else if (pa_end == (*pde & PG_PS_FRAME)) pa_end += NBPDR; else { /* Run ended, update direct map. */ error = pmap_change_props_locked( PHYS_TO_DMAP(pa_start), pa_end - pa_start, prot, mode, flags); if (error != 0) break; /* Start physical address run. */ pa_start = *pde & PG_PS_FRAME; pa_end = pa_start + NBPDR; } } tmpva = trunc_2mpage(tmpva) + NBPDR; } else { pte = pmap_pde_to_pte(pde, tmpva); if ((*pte & pte_mask) != pte_bits) { pmap_pte_props(pte, pte_bits, pte_mask); changed = true; } if (tmpva >= VM_MIN_KERNEL_ADDRESS && (*pte & PG_FRAME) < dmaplimit) { if (pa_start == pa_end) { /* Start physical address run. */ pa_start = *pte & PG_FRAME; pa_end = pa_start + PAGE_SIZE; } else if (pa_end == (*pte & PG_FRAME)) pa_end += PAGE_SIZE; else { /* Run ended, update direct map. */ error = pmap_change_props_locked( PHYS_TO_DMAP(pa_start), pa_end - pa_start, prot, mode, flags); if (error != 0) break; /* Start physical address run. */ pa_start = *pte & PG_FRAME; pa_end = pa_start + PAGE_SIZE; } } tmpva += PAGE_SIZE; } } if (error == 0 && pa_start != pa_end && pa_start < dmaplimit) { pa_end1 = MIN(pa_end, dmaplimit); if (pa_start != pa_end1) error = pmap_change_props_locked(PHYS_TO_DMAP(pa_start), pa_end1 - pa_start, prot, mode, flags); } /* * Flush CPU caches if required to make sure any data isn't cached that * shouldn't be, etc. */ if (changed) { pmap_invalidate_range(kernel_pmap, base, tmpva); if ((flags & MAPDEV_FLUSHCACHE) != 0) pmap_invalidate_cache_range(base, tmpva); } return (error); } /* * Demotes any mapping within the direct map region that covers more than the * specified range of physical addresses. This range's size must be a power * of two and its starting address must be a multiple of its size. Since the * demotion does not change any attributes of the mapping, a TLB invalidation * is not mandatory. The caller may, however, request a TLB invalidation. */ void pmap_demote_DMAP(vm_paddr_t base, vm_size_t len, boolean_t invalidate) { pdp_entry_t *pdpe; pd_entry_t *pde; vm_offset_t va; boolean_t changed; if (len == 0) return; KASSERT(powerof2(len), ("pmap_demote_DMAP: len is not a power of 2")); KASSERT((base & (len - 1)) == 0, ("pmap_demote_DMAP: base is not a multiple of len")); if (len < NBPDP && base < dmaplimit) { va = PHYS_TO_DMAP(base); changed = FALSE; PMAP_LOCK(kernel_pmap); pdpe = pmap_pdpe(kernel_pmap, va); if ((*pdpe & X86_PG_V) == 0) panic("pmap_demote_DMAP: invalid PDPE"); if ((*pdpe & PG_PS) != 0) { if (!pmap_demote_pdpe(kernel_pmap, pdpe, va)) panic("pmap_demote_DMAP: PDPE failed"); changed = TRUE; } if (len < NBPDR) { pde = pmap_pdpe_to_pde(pdpe, va); if ((*pde & X86_PG_V) == 0) panic("pmap_demote_DMAP: invalid PDE"); if ((*pde & PG_PS) != 0) { if (!pmap_demote_pde(kernel_pmap, pde, va)) panic("pmap_demote_DMAP: PDE failed"); changed = TRUE; } } if (changed && invalidate) pmap_invalidate_page(kernel_pmap, va); PMAP_UNLOCK(kernel_pmap); } } /* * Perform the pmap work for mincore(2). If the page is not both referenced and * modified by this pmap, returns its physical address so that the caller can * find other mappings. */ int pmap_mincore(pmap_t pmap, vm_offset_t addr, vm_paddr_t *pap) { pd_entry_t *pdep; pt_entry_t pte, PG_A, PG_M, PG_RW, PG_V; vm_paddr_t pa; int val; PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); PMAP_LOCK(pmap); pdep = pmap_pde(pmap, addr); if (pdep != NULL && (*pdep & PG_V)) { if (*pdep & PG_PS) { pte = *pdep; /* Compute the physical address of the 4KB page. */ pa = ((*pdep & PG_PS_FRAME) | (addr & PDRMASK)) & PG_FRAME; val = MINCORE_SUPER; } else { pte = *pmap_pde_to_pte(pdep, addr); pa = pte & PG_FRAME; val = 0; } } else { pte = 0; pa = 0; val = 0; } if ((pte & PG_V) != 0) { val |= MINCORE_INCORE; if ((pte & (PG_M | PG_RW)) == (PG_M | PG_RW)) val |= MINCORE_MODIFIED | MINCORE_MODIFIED_OTHER; if ((pte & PG_A) != 0) val |= MINCORE_REFERENCED | MINCORE_REFERENCED_OTHER; } if ((val & (MINCORE_MODIFIED_OTHER | MINCORE_REFERENCED_OTHER)) != (MINCORE_MODIFIED_OTHER | MINCORE_REFERENCED_OTHER) && (pte & (PG_MANAGED | PG_V)) == (PG_MANAGED | PG_V)) { *pap = pa; } PMAP_UNLOCK(pmap); return (val); } static uint64_t pmap_pcid_alloc(pmap_t pmap, u_int cpuid) { uint32_t gen, new_gen, pcid_next; CRITICAL_ASSERT(curthread); gen = PCPU_GET(pcid_gen); if (pmap->pm_pcids[cpuid].pm_pcid == PMAP_PCID_KERN) return (pti ? 0 : CR3_PCID_SAVE); if (pmap->pm_pcids[cpuid].pm_gen == gen) return (CR3_PCID_SAVE); pcid_next = PCPU_GET(pcid_next); KASSERT((!pti && pcid_next <= PMAP_PCID_OVERMAX) || (pti && pcid_next <= PMAP_PCID_OVERMAX_KERN), ("cpu %d pcid_next %#x", cpuid, pcid_next)); if ((!pti && pcid_next == PMAP_PCID_OVERMAX) || (pti && pcid_next == PMAP_PCID_OVERMAX_KERN)) { new_gen = gen + 1; if (new_gen == 0) new_gen = 1; PCPU_SET(pcid_gen, new_gen); pcid_next = PMAP_PCID_KERN + 1; } else { new_gen = gen; } pmap->pm_pcids[cpuid].pm_pcid = pcid_next; pmap->pm_pcids[cpuid].pm_gen = new_gen; PCPU_SET(pcid_next, pcid_next + 1); return (0); } static uint64_t pmap_pcid_alloc_checked(pmap_t pmap, u_int cpuid) { uint64_t cached; cached = pmap_pcid_alloc(pmap, cpuid); KASSERT(pmap->pm_pcids[cpuid].pm_pcid < PMAP_PCID_OVERMAX, ("pmap %p cpu %d pcid %#x", pmap, cpuid, pmap->pm_pcids[cpuid].pm_pcid)); KASSERT(pmap->pm_pcids[cpuid].pm_pcid != PMAP_PCID_KERN || pmap == kernel_pmap, ("non-kernel pmap pmap %p cpu %d pcid %#x", pmap, cpuid, pmap->pm_pcids[cpuid].pm_pcid)); return (cached); } static void pmap_activate_sw_pti_post(struct thread *td, pmap_t pmap) { PCPU_GET(tssp)->tss_rsp0 = pmap->pm_ucr3 != PMAP_NO_CR3 ? PCPU_GET(pti_rsp0) : (uintptr_t)td->td_md.md_stack_base; } static void inline pmap_activate_sw_pcid_pti(pmap_t pmap, u_int cpuid, const bool invpcid_works1) { struct invpcid_descr d; uint64_t cached, cr3, kcr3, ucr3; cached = pmap_pcid_alloc_checked(pmap, cpuid); cr3 = rcr3(); if ((cr3 & ~CR3_PCID_MASK) != pmap->pm_cr3) load_cr3(pmap->pm_cr3 | pmap->pm_pcids[cpuid].pm_pcid); PCPU_SET(curpmap, pmap); kcr3 = pmap->pm_cr3 | pmap->pm_pcids[cpuid].pm_pcid; ucr3 = pmap->pm_ucr3 | pmap->pm_pcids[cpuid].pm_pcid | PMAP_PCID_USER_PT; if (!cached && pmap->pm_ucr3 != PMAP_NO_CR3) { /* * Explicitly invalidate translations cached from the * user page table. They are not automatically * flushed by reload of cr3 with the kernel page table * pointer above. * * Note that the if() condition is resolved statically * by using the function argument instead of * runtime-evaluated invpcid_works value. */ if (invpcid_works1) { d.pcid = PMAP_PCID_USER_PT | pmap->pm_pcids[cpuid].pm_pcid; d.pad = 0; d.addr = 0; invpcid(&d, INVPCID_CTX); } else { pmap_pti_pcid_invalidate(ucr3, kcr3); } } PCPU_SET(kcr3, kcr3 | CR3_PCID_SAVE); PCPU_SET(ucr3, ucr3 | CR3_PCID_SAVE); if (cached) PCPU_INC(pm_save_cnt); } static void pmap_activate_sw_pcid_invpcid_pti(struct thread *td, pmap_t pmap, u_int cpuid) { pmap_activate_sw_pcid_pti(pmap, cpuid, true); pmap_activate_sw_pti_post(td, pmap); } static void pmap_activate_sw_pcid_noinvpcid_pti(struct thread *td, pmap_t pmap, u_int cpuid) { register_t rflags; /* * If the INVPCID instruction is not available, * invltlb_pcid_handler() is used to handle an invalidate_all * IPI, which checks for curpmap == smp_tlb_pmap. The below * sequence of operations has a window where %CR3 is loaded * with the new pmap's PML4 address, but the curpmap value has * not yet been updated. This causes the invltlb IPI handler, * which is called between the updates, to execute as a NOP, * which leaves stale TLB entries. * * Note that the most typical use of pmap_activate_sw(), from * the context switch, is immune to this race, because * interrupts are disabled (while the thread lock is owned), * and the IPI happens after curpmap is updated. Protect * other callers in a similar way, by disabling interrupts * around the %cr3 register reload and curpmap assignment. */ rflags = intr_disable(); pmap_activate_sw_pcid_pti(pmap, cpuid, false); intr_restore(rflags); pmap_activate_sw_pti_post(td, pmap); } static void pmap_activate_sw_pcid_nopti(struct thread *td __unused, pmap_t pmap, u_int cpuid) { uint64_t cached, cr3; cached = pmap_pcid_alloc_checked(pmap, cpuid); cr3 = rcr3(); if (!cached || (cr3 & ~CR3_PCID_MASK) != pmap->pm_cr3) load_cr3(pmap->pm_cr3 | pmap->pm_pcids[cpuid].pm_pcid | cached); PCPU_SET(curpmap, pmap); if (cached) PCPU_INC(pm_save_cnt); } static void pmap_activate_sw_pcid_noinvpcid_nopti(struct thread *td __unused, pmap_t pmap, u_int cpuid) { register_t rflags; rflags = intr_disable(); pmap_activate_sw_pcid_nopti(td, pmap, cpuid); intr_restore(rflags); } static void pmap_activate_sw_nopcid_nopti(struct thread *td __unused, pmap_t pmap, u_int cpuid __unused) { load_cr3(pmap->pm_cr3); PCPU_SET(curpmap, pmap); } static void pmap_activate_sw_nopcid_pti(struct thread *td, pmap_t pmap, u_int cpuid __unused) { pmap_activate_sw_nopcid_nopti(td, pmap, cpuid); PCPU_SET(kcr3, pmap->pm_cr3); PCPU_SET(ucr3, pmap->pm_ucr3); pmap_activate_sw_pti_post(td, pmap); } DEFINE_IFUNC(static, void, pmap_activate_sw_mode, (struct thread *, pmap_t, u_int)) { if (pmap_pcid_enabled && pti && invpcid_works) return (pmap_activate_sw_pcid_invpcid_pti); else if (pmap_pcid_enabled && pti && !invpcid_works) return (pmap_activate_sw_pcid_noinvpcid_pti); else if (pmap_pcid_enabled && !pti && invpcid_works) return (pmap_activate_sw_pcid_nopti); else if (pmap_pcid_enabled && !pti && !invpcid_works) return (pmap_activate_sw_pcid_noinvpcid_nopti); else if (!pmap_pcid_enabled && pti) return (pmap_activate_sw_nopcid_pti); else /* if (!pmap_pcid_enabled && !pti) */ return (pmap_activate_sw_nopcid_nopti); } void pmap_activate_sw(struct thread *td) { pmap_t oldpmap, pmap; u_int cpuid; oldpmap = PCPU_GET(curpmap); pmap = vmspace_pmap(td->td_proc->p_vmspace); if (oldpmap == pmap) { if (cpu_vendor_id != CPU_VENDOR_INTEL) mfence(); return; } cpuid = PCPU_GET(cpuid); #ifdef SMP CPU_SET_ATOMIC(cpuid, &pmap->pm_active); #else CPU_SET(cpuid, &pmap->pm_active); #endif pmap_activate_sw_mode(td, pmap, cpuid); #ifdef SMP CPU_CLR_ATOMIC(cpuid, &oldpmap->pm_active); #else CPU_CLR(cpuid, &oldpmap->pm_active); #endif } void pmap_activate(struct thread *td) { critical_enter(); pmap_activate_sw(td); critical_exit(); } void pmap_activate_boot(pmap_t pmap) { uint64_t kcr3; u_int cpuid; /* * kernel_pmap must be never deactivated, and we ensure that * by never activating it at all. */ MPASS(pmap != kernel_pmap); cpuid = PCPU_GET(cpuid); #ifdef SMP CPU_SET_ATOMIC(cpuid, &pmap->pm_active); #else CPU_SET(cpuid, &pmap->pm_active); #endif PCPU_SET(curpmap, pmap); if (pti) { kcr3 = pmap->pm_cr3; if (pmap_pcid_enabled) kcr3 |= pmap->pm_pcids[cpuid].pm_pcid | CR3_PCID_SAVE; } else { kcr3 = PMAP_NO_CR3; } PCPU_SET(kcr3, kcr3); PCPU_SET(ucr3, PMAP_NO_CR3); } void pmap_sync_icache(pmap_t pm, vm_offset_t va, vm_size_t sz) { } /* * Increase the starting virtual address of the given mapping if a * different alignment might result in more superpage mappings. */ void pmap_align_superpage(vm_object_t object, vm_ooffset_t offset, vm_offset_t *addr, vm_size_t size) { vm_offset_t superpage_offset; if (size < NBPDR) return; if (object != NULL && (object->flags & OBJ_COLORED) != 0) offset += ptoa(object->pg_color); superpage_offset = offset & PDRMASK; if (size - ((NBPDR - superpage_offset) & PDRMASK) < NBPDR || (*addr & PDRMASK) == superpage_offset) return; if ((*addr & PDRMASK) < superpage_offset) *addr = (*addr & ~PDRMASK) + superpage_offset; else *addr = ((*addr + PDRMASK) & ~PDRMASK) + superpage_offset; } #ifdef INVARIANTS static unsigned long num_dirty_emulations; SYSCTL_ULONG(_vm_pmap, OID_AUTO, num_dirty_emulations, CTLFLAG_RW, &num_dirty_emulations, 0, NULL); static unsigned long num_accessed_emulations; SYSCTL_ULONG(_vm_pmap, OID_AUTO, num_accessed_emulations, CTLFLAG_RW, &num_accessed_emulations, 0, NULL); static unsigned long num_superpage_accessed_emulations; SYSCTL_ULONG(_vm_pmap, OID_AUTO, num_superpage_accessed_emulations, CTLFLAG_RW, &num_superpage_accessed_emulations, 0, NULL); static unsigned long ad_emulation_superpage_promotions; SYSCTL_ULONG(_vm_pmap, OID_AUTO, ad_emulation_superpage_promotions, CTLFLAG_RW, &ad_emulation_superpage_promotions, 0, NULL); #endif /* INVARIANTS */ int pmap_emulate_accessed_dirty(pmap_t pmap, vm_offset_t va, int ftype) { int rv; struct rwlock *lock; #if VM_NRESERVLEVEL > 0 vm_page_t m, mpte; #endif pd_entry_t *pde; pt_entry_t *pte, PG_A, PG_M, PG_RW, PG_V; KASSERT(ftype == VM_PROT_READ || ftype == VM_PROT_WRITE, ("pmap_emulate_accessed_dirty: invalid fault type %d", ftype)); if (!pmap_emulate_ad_bits(pmap)) return (-1); PG_A = pmap_accessed_bit(pmap); PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); rv = -1; lock = NULL; PMAP_LOCK(pmap); pde = pmap_pde(pmap, va); if (pde == NULL || (*pde & PG_V) == 0) goto done; if ((*pde & PG_PS) != 0) { if (ftype == VM_PROT_READ) { #ifdef INVARIANTS atomic_add_long(&num_superpage_accessed_emulations, 1); #endif *pde |= PG_A; rv = 0; } goto done; } pte = pmap_pde_to_pte(pde, va); if ((*pte & PG_V) == 0) goto done; if (ftype == VM_PROT_WRITE) { if ((*pte & PG_RW) == 0) goto done; /* * Set the modified and accessed bits simultaneously. * * Intel EPT PTEs that do software emulation of A/D bits map * PG_A and PG_M to EPT_PG_READ and EPT_PG_WRITE respectively. * An EPT misconfiguration is triggered if the PTE is writable * but not readable (WR=10). This is avoided by setting PG_A * and PG_M simultaneously. */ *pte |= PG_M | PG_A; } else { *pte |= PG_A; } #if VM_NRESERVLEVEL > 0 /* try to promote the mapping */ if (va < VM_MAXUSER_ADDRESS) mpte = PHYS_TO_VM_PAGE(*pde & PG_FRAME); else mpte = NULL; m = PHYS_TO_VM_PAGE(*pte & PG_FRAME); if ((mpte == NULL || mpte->ref_count == NPTEPG) && pmap_ps_enabled(pmap) && (m->flags & PG_FICTITIOUS) == 0 && vm_reserv_level_iffullpop(m) == 0) { pmap_promote_pde(pmap, pde, va, &lock); #ifdef INVARIANTS atomic_add_long(&ad_emulation_superpage_promotions, 1); #endif } #endif #ifdef INVARIANTS if (ftype == VM_PROT_WRITE) atomic_add_long(&num_dirty_emulations, 1); else atomic_add_long(&num_accessed_emulations, 1); #endif rv = 0; /* success */ done: if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(pmap); return (rv); } void pmap_get_mapping(pmap_t pmap, vm_offset_t va, uint64_t *ptr, int *num) { pml4_entry_t *pml4; pdp_entry_t *pdp; pd_entry_t *pde; pt_entry_t *pte, PG_V; int idx; idx = 0; PG_V = pmap_valid_bit(pmap); PMAP_LOCK(pmap); pml4 = pmap_pml4e(pmap, va); ptr[idx++] = *pml4; if ((*pml4 & PG_V) == 0) goto done; pdp = pmap_pml4e_to_pdpe(pml4, va); ptr[idx++] = *pdp; if ((*pdp & PG_V) == 0 || (*pdp & PG_PS) != 0) goto done; pde = pmap_pdpe_to_pde(pdp, va); ptr[idx++] = *pde; if ((*pde & PG_V) == 0 || (*pde & PG_PS) != 0) goto done; pte = pmap_pde_to_pte(pde, va); ptr[idx++] = *pte; done: PMAP_UNLOCK(pmap); *num = idx; } /** * Get the kernel virtual address of a set of physical pages. If there are * physical addresses not covered by the DMAP perform a transient mapping * that will be removed when calling pmap_unmap_io_transient. * * \param page The pages the caller wishes to obtain the virtual * address on the kernel memory map. * \param vaddr On return contains the kernel virtual memory address * of the pages passed in the page parameter. * \param count Number of pages passed in. * \param can_fault TRUE if the thread using the mapped pages can take * page faults, FALSE otherwise. * * \returns TRUE if the caller must call pmap_unmap_io_transient when * finished or FALSE otherwise. * */ boolean_t pmap_map_io_transient(vm_page_t page[], vm_offset_t vaddr[], int count, boolean_t can_fault) { vm_paddr_t paddr; boolean_t needs_mapping; pt_entry_t *pte; int cache_bits, error __unused, i; /* * Allocate any KVA space that we need, this is done in a separate * loop to prevent calling vmem_alloc while pinned. */ needs_mapping = FALSE; for (i = 0; i < count; i++) { paddr = VM_PAGE_TO_PHYS(page[i]); if (__predict_false(paddr >= dmaplimit)) { error = vmem_alloc(kernel_arena, PAGE_SIZE, M_BESTFIT | M_WAITOK, &vaddr[i]); KASSERT(error == 0, ("vmem_alloc failed: %d", error)); needs_mapping = TRUE; } else { vaddr[i] = PHYS_TO_DMAP(paddr); } } /* Exit early if everything is covered by the DMAP */ if (!needs_mapping) return (FALSE); /* * NB: The sequence of updating a page table followed by accesses * to the corresponding pages used in the !DMAP case is subject to * the situation described in the "AMD64 Architecture Programmer's * Manual Volume 2: System Programming" rev. 3.23, "7.3.1 Special * Coherency Considerations". Therefore, issuing the INVLPG right * after modifying the PTE bits is crucial. */ if (!can_fault) sched_pin(); for (i = 0; i < count; i++) { paddr = VM_PAGE_TO_PHYS(page[i]); if (paddr >= dmaplimit) { if (can_fault) { /* * Slow path, since we can get page faults * while mappings are active don't pin the * thread to the CPU and instead add a global * mapping visible to all CPUs. */ pmap_qenter(vaddr[i], &page[i], 1); } else { pte = vtopte(vaddr[i]); cache_bits = pmap_cache_bits(kernel_pmap, page[i]->md.pat_mode, 0); pte_store(pte, paddr | X86_PG_RW | X86_PG_V | cache_bits); invlpg(vaddr[i]); } } } return (needs_mapping); } void pmap_unmap_io_transient(vm_page_t page[], vm_offset_t vaddr[], int count, boolean_t can_fault) { vm_paddr_t paddr; int i; if (!can_fault) sched_unpin(); for (i = 0; i < count; i++) { paddr = VM_PAGE_TO_PHYS(page[i]); if (paddr >= dmaplimit) { if (can_fault) pmap_qremove(vaddr[i], 1); vmem_free(kernel_arena, vaddr[i], PAGE_SIZE); } } } vm_offset_t pmap_quick_enter_page(vm_page_t m) { vm_paddr_t paddr; paddr = VM_PAGE_TO_PHYS(m); if (paddr < dmaplimit) return (PHYS_TO_DMAP(paddr)); mtx_lock_spin(&qframe_mtx); KASSERT(*vtopte(qframe) == 0, ("qframe busy")); pte_store(vtopte(qframe), paddr | X86_PG_RW | X86_PG_V | X86_PG_A | X86_PG_M | pmap_cache_bits(kernel_pmap, m->md.pat_mode, 0)); return (qframe); } void pmap_quick_remove_page(vm_offset_t addr) { if (addr != qframe) return; pte_store(vtopte(qframe), 0); invlpg(qframe); mtx_unlock_spin(&qframe_mtx); } /* * Pdp pages from the large map are managed differently from either * kernel or user page table pages. They are permanently allocated at * initialization time, and their reference count is permanently set to * zero. The pml4 entries pointing to those pages are copied into * each allocated pmap. * * In contrast, pd and pt pages are managed like user page table * pages. They are dynamically allocated, and their reference count * represents the number of valid entries within the page. */ static vm_page_t pmap_large_map_getptp_unlocked(void) { vm_page_t m; m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_ZERO); if (m != NULL && (m->flags & PG_ZERO) == 0) pmap_zero_page(m); return (m); } static vm_page_t pmap_large_map_getptp(void) { vm_page_t m; PMAP_LOCK_ASSERT(kernel_pmap, MA_OWNED); m = pmap_large_map_getptp_unlocked(); if (m == NULL) { PMAP_UNLOCK(kernel_pmap); vm_wait(NULL); PMAP_LOCK(kernel_pmap); /* Callers retry. */ } return (m); } static pdp_entry_t * pmap_large_map_pdpe(vm_offset_t va) { vm_pindex_t pml4_idx; vm_paddr_t mphys; pml4_idx = pmap_pml4e_index(va); KASSERT(LMSPML4I <= pml4_idx && pml4_idx < LMSPML4I + lm_ents, ("pmap_large_map_pdpe: va %#jx out of range idx %#jx LMSPML4I " "%#jx lm_ents %d", (uintmax_t)va, (uintmax_t)pml4_idx, LMSPML4I, lm_ents)); KASSERT((kernel_pmap->pm_pml4[pml4_idx] & X86_PG_V) != 0, ("pmap_large_map_pdpe: invalid pml4 for va %#jx idx %#jx " "LMSPML4I %#jx lm_ents %d", (uintmax_t)va, (uintmax_t)pml4_idx, LMSPML4I, lm_ents)); mphys = kernel_pmap->pm_pml4[pml4_idx] & PG_FRAME; return ((pdp_entry_t *)PHYS_TO_DMAP(mphys) + pmap_pdpe_index(va)); } static pd_entry_t * pmap_large_map_pde(vm_offset_t va) { pdp_entry_t *pdpe; vm_page_t m; vm_paddr_t mphys; retry: pdpe = pmap_large_map_pdpe(va); if (*pdpe == 0) { m = pmap_large_map_getptp(); if (m == NULL) goto retry; mphys = VM_PAGE_TO_PHYS(m); *pdpe = mphys | X86_PG_A | X86_PG_RW | X86_PG_V | pg_nx; } else { MPASS((*pdpe & X86_PG_PS) == 0); mphys = *pdpe & PG_FRAME; } return ((pd_entry_t *)PHYS_TO_DMAP(mphys) + pmap_pde_index(va)); } static pt_entry_t * pmap_large_map_pte(vm_offset_t va) { pd_entry_t *pde; vm_page_t m; vm_paddr_t mphys; retry: pde = pmap_large_map_pde(va); if (*pde == 0) { m = pmap_large_map_getptp(); if (m == NULL) goto retry; mphys = VM_PAGE_TO_PHYS(m); *pde = mphys | X86_PG_A | X86_PG_RW | X86_PG_V | pg_nx; PHYS_TO_VM_PAGE(DMAP_TO_PHYS((uintptr_t)pde))->ref_count++; } else { MPASS((*pde & X86_PG_PS) == 0); mphys = *pde & PG_FRAME; } return ((pt_entry_t *)PHYS_TO_DMAP(mphys) + pmap_pte_index(va)); } static vm_paddr_t pmap_large_map_kextract(vm_offset_t va) { pdp_entry_t *pdpe, pdp; pd_entry_t *pde, pd; pt_entry_t *pte, pt; KASSERT(PMAP_ADDRESS_IN_LARGEMAP(va), ("not largemap range %#lx", (u_long)va)); pdpe = pmap_large_map_pdpe(va); pdp = *pdpe; KASSERT((pdp & X86_PG_V) != 0, ("invalid pdp va %#lx pdpe %#lx pdp %#lx", va, (u_long)pdpe, pdp)); if ((pdp & X86_PG_PS) != 0) { KASSERT((amd_feature & AMDID_PAGE1GB) != 0, ("no 1G pages, va %#lx pdpe %#lx pdp %#lx", va, (u_long)pdpe, pdp)); return ((pdp & PG_PS_PDP_FRAME) | (va & PDPMASK)); } pde = pmap_pdpe_to_pde(pdpe, va); pd = *pde; KASSERT((pd & X86_PG_V) != 0, ("invalid pd va %#lx pde %#lx pd %#lx", va, (u_long)pde, pd)); if ((pd & X86_PG_PS) != 0) return ((pd & PG_PS_FRAME) | (va & PDRMASK)); pte = pmap_pde_to_pte(pde, va); pt = *pte; KASSERT((pt & X86_PG_V) != 0, ("invalid pte va %#lx pte %#lx pt %#lx", va, (u_long)pte, pt)); return ((pt & PG_FRAME) | (va & PAGE_MASK)); } static int pmap_large_map_getva(vm_size_t len, vm_offset_t align, vm_offset_t phase, vmem_addr_t *vmem_res) { /* * Large mappings are all but static. Consequently, there * is no point in waiting for an earlier allocation to be * freed. */ return (vmem_xalloc(large_vmem, len, align, phase, 0, VMEM_ADDR_MIN, VMEM_ADDR_MAX, M_NOWAIT | M_BESTFIT, vmem_res)); } int pmap_large_map(vm_paddr_t spa, vm_size_t len, void **addr, vm_memattr_t mattr) { pdp_entry_t *pdpe; pd_entry_t *pde; pt_entry_t *pte; vm_offset_t va, inc; vmem_addr_t vmem_res; vm_paddr_t pa; int error; if (len == 0 || spa + len < spa) return (EINVAL); /* See if DMAP can serve. */ if (spa + len <= dmaplimit) { va = PHYS_TO_DMAP(spa); *addr = (void *)va; return (pmap_change_attr(va, len, mattr)); } /* * No, allocate KVA. Fit the address with best possible * alignment for superpages. Fall back to worse align if * failed. */ error = ENOMEM; if ((amd_feature & AMDID_PAGE1GB) != 0 && rounddown2(spa + len, NBPDP) >= roundup2(spa, NBPDP) + NBPDP) error = pmap_large_map_getva(len, NBPDP, spa & PDPMASK, &vmem_res); if (error != 0 && rounddown2(spa + len, NBPDR) >= roundup2(spa, NBPDR) + NBPDR) error = pmap_large_map_getva(len, NBPDR, spa & PDRMASK, &vmem_res); if (error != 0) error = pmap_large_map_getva(len, PAGE_SIZE, 0, &vmem_res); if (error != 0) return (error); /* * Fill pagetable. PG_M is not pre-set, we scan modified bits * in the pagetable to minimize flushing. No need to * invalidate TLB, since we only update invalid entries. */ PMAP_LOCK(kernel_pmap); for (pa = spa, va = vmem_res; len > 0; pa += inc, va += inc, len -= inc) { if ((amd_feature & AMDID_PAGE1GB) != 0 && len >= NBPDP && (pa & PDPMASK) == 0 && (va & PDPMASK) == 0) { pdpe = pmap_large_map_pdpe(va); MPASS(*pdpe == 0); *pdpe = pa | pg_g | X86_PG_PS | X86_PG_RW | X86_PG_V | X86_PG_A | pg_nx | pmap_cache_bits(kernel_pmap, mattr, TRUE); inc = NBPDP; } else if (len >= NBPDR && (pa & PDRMASK) == 0 && (va & PDRMASK) == 0) { pde = pmap_large_map_pde(va); MPASS(*pde == 0); *pde = pa | pg_g | X86_PG_PS | X86_PG_RW | X86_PG_V | X86_PG_A | pg_nx | pmap_cache_bits(kernel_pmap, mattr, TRUE); PHYS_TO_VM_PAGE(DMAP_TO_PHYS((uintptr_t)pde))-> ref_count++; inc = NBPDR; } else { pte = pmap_large_map_pte(va); MPASS(*pte == 0); *pte = pa | pg_g | X86_PG_RW | X86_PG_V | X86_PG_A | pg_nx | pmap_cache_bits(kernel_pmap, mattr, FALSE); PHYS_TO_VM_PAGE(DMAP_TO_PHYS((uintptr_t)pte))-> ref_count++; inc = PAGE_SIZE; } } PMAP_UNLOCK(kernel_pmap); MPASS(len == 0); *addr = (void *)vmem_res; return (0); } void pmap_large_unmap(void *svaa, vm_size_t len) { vm_offset_t sva, va; vm_size_t inc; pdp_entry_t *pdpe, pdp; pd_entry_t *pde, pd; pt_entry_t *pte; vm_page_t m; struct spglist spgf; sva = (vm_offset_t)svaa; if (len == 0 || sva + len < sva || (sva >= DMAP_MIN_ADDRESS && sva + len <= DMAP_MIN_ADDRESS + dmaplimit)) return; SLIST_INIT(&spgf); KASSERT(PMAP_ADDRESS_IN_LARGEMAP(sva) && PMAP_ADDRESS_IN_LARGEMAP(sva + len - 1), ("not largemap range %#lx %#lx", (u_long)svaa, (u_long)svaa + len)); PMAP_LOCK(kernel_pmap); for (va = sva; va < sva + len; va += inc) { pdpe = pmap_large_map_pdpe(va); pdp = *pdpe; KASSERT((pdp & X86_PG_V) != 0, ("invalid pdp va %#lx pdpe %#lx pdp %#lx", va, (u_long)pdpe, pdp)); if ((pdp & X86_PG_PS) != 0) { KASSERT((amd_feature & AMDID_PAGE1GB) != 0, ("no 1G pages, va %#lx pdpe %#lx pdp %#lx", va, (u_long)pdpe, pdp)); KASSERT((va & PDPMASK) == 0, ("PDPMASK bit set, va %#lx pdpe %#lx pdp %#lx", va, (u_long)pdpe, pdp)); KASSERT(va + NBPDP <= sva + len, ("unmap covers partial 1GB page, sva %#lx va %#lx " "pdpe %#lx pdp %#lx len %#lx", sva, va, (u_long)pdpe, pdp, len)); *pdpe = 0; inc = NBPDP; continue; } pde = pmap_pdpe_to_pde(pdpe, va); pd = *pde; KASSERT((pd & X86_PG_V) != 0, ("invalid pd va %#lx pde %#lx pd %#lx", va, (u_long)pde, pd)); if ((pd & X86_PG_PS) != 0) { KASSERT((va & PDRMASK) == 0, ("PDRMASK bit set, va %#lx pde %#lx pd %#lx", va, (u_long)pde, pd)); KASSERT(va + NBPDR <= sva + len, ("unmap covers partial 2MB page, sva %#lx va %#lx " "pde %#lx pd %#lx len %#lx", sva, va, (u_long)pde, pd, len)); pde_store(pde, 0); inc = NBPDR; m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pde)); m->ref_count--; if (m->ref_count == 0) { *pdpe = 0; SLIST_INSERT_HEAD(&spgf, m, plinks.s.ss); } continue; } pte = pmap_pde_to_pte(pde, va); KASSERT((*pte & X86_PG_V) != 0, ("invalid pte va %#lx pte %#lx pt %#lx", va, (u_long)pte, *pte)); pte_clear(pte); inc = PAGE_SIZE; m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pte)); m->ref_count--; if (m->ref_count == 0) { *pde = 0; SLIST_INSERT_HEAD(&spgf, m, plinks.s.ss); m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pde)); m->ref_count--; if (m->ref_count == 0) { *pdpe = 0; SLIST_INSERT_HEAD(&spgf, m, plinks.s.ss); } } } pmap_invalidate_range(kernel_pmap, sva, sva + len); PMAP_UNLOCK(kernel_pmap); vm_page_free_pages_toq(&spgf, false); vmem_free(large_vmem, sva, len); } static void pmap_large_map_wb_fence_mfence(void) { mfence(); } static void pmap_large_map_wb_fence_atomic(void) { atomic_thread_fence_seq_cst(); } static void pmap_large_map_wb_fence_nop(void) { } DEFINE_IFUNC(static, void, pmap_large_map_wb_fence, (void)) { if (cpu_vendor_id != CPU_VENDOR_INTEL) return (pmap_large_map_wb_fence_mfence); else if ((cpu_stdext_feature & (CPUID_STDEXT_CLWB | CPUID_STDEXT_CLFLUSHOPT)) == 0) return (pmap_large_map_wb_fence_atomic); else /* clflush is strongly enough ordered */ return (pmap_large_map_wb_fence_nop); } static void pmap_large_map_flush_range_clwb(vm_offset_t va, vm_size_t len) { for (; len > 0; len -= cpu_clflush_line_size, va += cpu_clflush_line_size) clwb(va); } static void pmap_large_map_flush_range_clflushopt(vm_offset_t va, vm_size_t len) { for (; len > 0; len -= cpu_clflush_line_size, va += cpu_clflush_line_size) clflushopt(va); } static void pmap_large_map_flush_range_clflush(vm_offset_t va, vm_size_t len) { for (; len > 0; len -= cpu_clflush_line_size, va += cpu_clflush_line_size) clflush(va); } static void pmap_large_map_flush_range_nop(vm_offset_t sva __unused, vm_size_t len __unused) { } DEFINE_IFUNC(static, void, pmap_large_map_flush_range, (vm_offset_t, vm_size_t)) { if ((cpu_stdext_feature & CPUID_STDEXT_CLWB) != 0) return (pmap_large_map_flush_range_clwb); else if ((cpu_stdext_feature & CPUID_STDEXT_CLFLUSHOPT) != 0) return (pmap_large_map_flush_range_clflushopt); else if ((cpu_feature & CPUID_CLFSH) != 0) return (pmap_large_map_flush_range_clflush); else return (pmap_large_map_flush_range_nop); } static void pmap_large_map_wb_large(vm_offset_t sva, vm_offset_t eva) { volatile u_long *pe; u_long p; vm_offset_t va; vm_size_t inc; bool seen_other; for (va = sva; va < eva; va += inc) { inc = 0; if ((amd_feature & AMDID_PAGE1GB) != 0) { pe = (volatile u_long *)pmap_large_map_pdpe(va); p = *pe; if ((p & X86_PG_PS) != 0) inc = NBPDP; } if (inc == 0) { pe = (volatile u_long *)pmap_large_map_pde(va); p = *pe; if ((p & X86_PG_PS) != 0) inc = NBPDR; } if (inc == 0) { pe = (volatile u_long *)pmap_large_map_pte(va); p = *pe; inc = PAGE_SIZE; } seen_other = false; for (;;) { if ((p & X86_PG_AVAIL1) != 0) { /* * Spin-wait for the end of a parallel * write-back. */ cpu_spinwait(); p = *pe; /* * If we saw other write-back * occuring, we cannot rely on PG_M to * indicate state of the cache. The * PG_M bit is cleared before the * flush to avoid ignoring new writes, * and writes which are relevant for * us might happen after. */ seen_other = true; continue; } if ((p & X86_PG_M) != 0 || seen_other) { if (!atomic_fcmpset_long(pe, &p, (p & ~X86_PG_M) | X86_PG_AVAIL1)) /* * If we saw PG_M without * PG_AVAIL1, and then on the * next attempt we do not * observe either PG_M or * PG_AVAIL1, the other * write-back started after us * and finished before us. We * can rely on it doing our * work. */ continue; pmap_large_map_flush_range(va, inc); atomic_clear_long(pe, X86_PG_AVAIL1); } break; } maybe_yield(); } } /* * Write-back cache lines for the given address range. * * Must be called only on the range or sub-range returned from * pmap_large_map(). Must not be called on the coalesced ranges. * * Does nothing on CPUs without CLWB, CLFLUSHOPT, or CLFLUSH * instructions support. */ void pmap_large_map_wb(void *svap, vm_size_t len) { vm_offset_t eva, sva; sva = (vm_offset_t)svap; eva = sva + len; pmap_large_map_wb_fence(); if (sva >= DMAP_MIN_ADDRESS && eva <= DMAP_MIN_ADDRESS + dmaplimit) { pmap_large_map_flush_range(sva, len); } else { KASSERT(sva >= LARGEMAP_MIN_ADDRESS && eva <= LARGEMAP_MIN_ADDRESS + lm_ents * NBPML4, ("pmap_large_map_wb: not largemap %#lx %#lx", sva, len)); pmap_large_map_wb_large(sva, eva); } pmap_large_map_wb_fence(); } static vm_page_t pmap_pti_alloc_page(void) { vm_page_t m; VM_OBJECT_ASSERT_WLOCKED(pti_obj); m = vm_page_grab(pti_obj, pti_pg_idx++, VM_ALLOC_NOBUSY | VM_ALLOC_WIRED | VM_ALLOC_ZERO); return (m); } static bool pmap_pti_free_page(vm_page_t m) { KASSERT(m->ref_count > 0, ("page %p not referenced", m)); if (!vm_page_unwire_noq(m)) return (false); vm_page_free_zero(m); return (true); } static void pmap_pti_init(void) { vm_page_t pml4_pg; pdp_entry_t *pdpe; vm_offset_t va; int i; if (!pti) return; pti_obj = vm_pager_allocate(OBJT_PHYS, NULL, 0, VM_PROT_ALL, 0, NULL); VM_OBJECT_WLOCK(pti_obj); pml4_pg = pmap_pti_alloc_page(); pti_pml4 = (pml4_entry_t *)PHYS_TO_DMAP(VM_PAGE_TO_PHYS(pml4_pg)); for (va = VM_MIN_KERNEL_ADDRESS; va <= VM_MAX_KERNEL_ADDRESS && va >= VM_MIN_KERNEL_ADDRESS && va > NBPML4; va += NBPML4) { pdpe = pmap_pti_pdpe(va); pmap_pti_wire_pte(pdpe); } pmap_pti_add_kva_locked((vm_offset_t)&__pcpu[0], (vm_offset_t)&__pcpu[0] + sizeof(__pcpu[0]) * MAXCPU, false); pmap_pti_add_kva_locked((vm_offset_t)idt, (vm_offset_t)idt + sizeof(struct gate_descriptor) * NIDT, false); CPU_FOREACH(i) { /* Doublefault stack IST 1 */ va = __pcpu[i].pc_common_tss.tss_ist1; pmap_pti_add_kva_locked(va - PAGE_SIZE, va, false); /* NMI stack IST 2 */ va = __pcpu[i].pc_common_tss.tss_ist2 + sizeof(struct nmi_pcpu); pmap_pti_add_kva_locked(va - PAGE_SIZE, va, false); /* MC# stack IST 3 */ va = __pcpu[i].pc_common_tss.tss_ist3 + sizeof(struct nmi_pcpu); pmap_pti_add_kva_locked(va - PAGE_SIZE, va, false); /* DB# stack IST 4 */ va = __pcpu[i].pc_common_tss.tss_ist4 + sizeof(struct nmi_pcpu); pmap_pti_add_kva_locked(va - PAGE_SIZE, va, false); } pmap_pti_add_kva_locked((vm_offset_t)kernphys + KERNBASE, (vm_offset_t)etext, true); pti_finalized = true; VM_OBJECT_WUNLOCK(pti_obj); } SYSINIT(pmap_pti, SI_SUB_CPU + 1, SI_ORDER_ANY, pmap_pti_init, NULL); static pdp_entry_t * pmap_pti_pdpe(vm_offset_t va) { pml4_entry_t *pml4e; pdp_entry_t *pdpe; vm_page_t m; vm_pindex_t pml4_idx; vm_paddr_t mphys; VM_OBJECT_ASSERT_WLOCKED(pti_obj); pml4_idx = pmap_pml4e_index(va); pml4e = &pti_pml4[pml4_idx]; m = NULL; if (*pml4e == 0) { if (pti_finalized) panic("pml4 alloc after finalization\n"); m = pmap_pti_alloc_page(); if (*pml4e != 0) { pmap_pti_free_page(m); mphys = *pml4e & ~PAGE_MASK; } else { mphys = VM_PAGE_TO_PHYS(m); *pml4e = mphys | X86_PG_RW | X86_PG_V; } } else { mphys = *pml4e & ~PAGE_MASK; } pdpe = (pdp_entry_t *)PHYS_TO_DMAP(mphys) + pmap_pdpe_index(va); return (pdpe); } static void pmap_pti_wire_pte(void *pte) { vm_page_t m; VM_OBJECT_ASSERT_WLOCKED(pti_obj); m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((uintptr_t)pte)); m->ref_count++; } static void pmap_pti_unwire_pde(void *pde, bool only_ref) { vm_page_t m; VM_OBJECT_ASSERT_WLOCKED(pti_obj); m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((uintptr_t)pde)); MPASS(m->ref_count > 0); MPASS(only_ref || m->ref_count > 1); pmap_pti_free_page(m); } static void pmap_pti_unwire_pte(void *pte, vm_offset_t va) { vm_page_t m; pd_entry_t *pde; VM_OBJECT_ASSERT_WLOCKED(pti_obj); m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((uintptr_t)pte)); MPASS(m->ref_count > 0); if (pmap_pti_free_page(m)) { pde = pmap_pti_pde(va); MPASS((*pde & (X86_PG_PS | X86_PG_V)) == X86_PG_V); *pde = 0; pmap_pti_unwire_pde(pde, false); } } static pd_entry_t * pmap_pti_pde(vm_offset_t va) { pdp_entry_t *pdpe; pd_entry_t *pde; vm_page_t m; vm_pindex_t pd_idx; vm_paddr_t mphys; VM_OBJECT_ASSERT_WLOCKED(pti_obj); pdpe = pmap_pti_pdpe(va); if (*pdpe == 0) { m = pmap_pti_alloc_page(); if (*pdpe != 0) { pmap_pti_free_page(m); MPASS((*pdpe & X86_PG_PS) == 0); mphys = *pdpe & ~PAGE_MASK; } else { mphys = VM_PAGE_TO_PHYS(m); *pdpe = mphys | X86_PG_RW | X86_PG_V; } } else { MPASS((*pdpe & X86_PG_PS) == 0); mphys = *pdpe & ~PAGE_MASK; } pde = (pd_entry_t *)PHYS_TO_DMAP(mphys); pd_idx = pmap_pde_index(va); pde += pd_idx; return (pde); } static pt_entry_t * pmap_pti_pte(vm_offset_t va, bool *unwire_pde) { pd_entry_t *pde; pt_entry_t *pte; vm_page_t m; vm_paddr_t mphys; VM_OBJECT_ASSERT_WLOCKED(pti_obj); pde = pmap_pti_pde(va); if (unwire_pde != NULL) { *unwire_pde = true; pmap_pti_wire_pte(pde); } if (*pde == 0) { m = pmap_pti_alloc_page(); if (*pde != 0) { pmap_pti_free_page(m); MPASS((*pde & X86_PG_PS) == 0); mphys = *pde & ~(PAGE_MASK | pg_nx); } else { mphys = VM_PAGE_TO_PHYS(m); *pde = mphys | X86_PG_RW | X86_PG_V; if (unwire_pde != NULL) *unwire_pde = false; } } else { MPASS((*pde & X86_PG_PS) == 0); mphys = *pde & ~(PAGE_MASK | pg_nx); } pte = (pt_entry_t *)PHYS_TO_DMAP(mphys); pte += pmap_pte_index(va); return (pte); } static void pmap_pti_add_kva_locked(vm_offset_t sva, vm_offset_t eva, bool exec) { vm_paddr_t pa; pd_entry_t *pde; pt_entry_t *pte, ptev; bool unwire_pde; VM_OBJECT_ASSERT_WLOCKED(pti_obj); sva = trunc_page(sva); MPASS(sva > VM_MAXUSER_ADDRESS); eva = round_page(eva); MPASS(sva < eva); for (; sva < eva; sva += PAGE_SIZE) { pte = pmap_pti_pte(sva, &unwire_pde); pa = pmap_kextract(sva); ptev = pa | X86_PG_RW | X86_PG_V | X86_PG_A | X86_PG_G | (exec ? 0 : pg_nx) | pmap_cache_bits(kernel_pmap, VM_MEMATTR_DEFAULT, FALSE); if (*pte == 0) { pte_store(pte, ptev); pmap_pti_wire_pte(pte); } else { KASSERT(!pti_finalized, ("pti overlap after fin %#lx %#lx %#lx", sva, *pte, ptev)); KASSERT(*pte == ptev, ("pti non-identical pte after fin %#lx %#lx %#lx", sva, *pte, ptev)); } if (unwire_pde) { pde = pmap_pti_pde(sva); pmap_pti_unwire_pde(pde, true); } } } void pmap_pti_add_kva(vm_offset_t sva, vm_offset_t eva, bool exec) { if (!pti) return; VM_OBJECT_WLOCK(pti_obj); pmap_pti_add_kva_locked(sva, eva, exec); VM_OBJECT_WUNLOCK(pti_obj); } void pmap_pti_remove_kva(vm_offset_t sva, vm_offset_t eva) { pt_entry_t *pte; vm_offset_t va; if (!pti) return; sva = rounddown2(sva, PAGE_SIZE); MPASS(sva > VM_MAXUSER_ADDRESS); eva = roundup2(eva, PAGE_SIZE); MPASS(sva < eva); VM_OBJECT_WLOCK(pti_obj); for (va = sva; va < eva; va += PAGE_SIZE) { pte = pmap_pti_pte(va, NULL); KASSERT((*pte & X86_PG_V) != 0, ("invalid pte va %#lx pte %#lx pt %#lx", va, (u_long)pte, *pte)); pte_clear(pte); pmap_pti_unwire_pte(pte, va); } pmap_invalidate_range(kernel_pmap, sva, eva); VM_OBJECT_WUNLOCK(pti_obj); } static void * pkru_dup_range(void *ctx __unused, void *data) { struct pmap_pkru_range *node, *new_node; new_node = uma_zalloc(pmap_pkru_ranges_zone, M_NOWAIT); if (new_node == NULL) return (NULL); node = data; memcpy(new_node, node, sizeof(*node)); return (new_node); } static void pkru_free_range(void *ctx __unused, void *node) { uma_zfree(pmap_pkru_ranges_zone, node); } static int pmap_pkru_assign(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, u_int keyidx, int flags) { struct pmap_pkru_range *ppr; int error; PMAP_LOCK_ASSERT(pmap, MA_OWNED); MPASS(pmap->pm_type == PT_X86); MPASS((cpu_stdext_feature2 & CPUID_STDEXT2_PKU) != 0); if ((flags & AMD64_PKRU_EXCL) != 0 && !rangeset_check_empty(&pmap->pm_pkru, sva, eva)) return (EBUSY); ppr = uma_zalloc(pmap_pkru_ranges_zone, M_NOWAIT); if (ppr == NULL) return (ENOMEM); ppr->pkru_keyidx = keyidx; ppr->pkru_flags = flags & AMD64_PKRU_PERSIST; error = rangeset_insert(&pmap->pm_pkru, sva, eva, ppr); if (error != 0) uma_zfree(pmap_pkru_ranges_zone, ppr); return (error); } static int pmap_pkru_deassign(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); MPASS(pmap->pm_type == PT_X86); MPASS((cpu_stdext_feature2 & CPUID_STDEXT2_PKU) != 0); return (rangeset_remove(&pmap->pm_pkru, sva, eva)); } static void pmap_pkru_deassign_all(pmap_t pmap) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); if (pmap->pm_type == PT_X86 && (cpu_stdext_feature2 & CPUID_STDEXT2_PKU) != 0) rangeset_remove_all(&pmap->pm_pkru); } static bool pmap_pkru_same(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { struct pmap_pkru_range *ppr, *prev_ppr; vm_offset_t va; PMAP_LOCK_ASSERT(pmap, MA_OWNED); if (pmap->pm_type != PT_X86 || (cpu_stdext_feature2 & CPUID_STDEXT2_PKU) == 0 || sva >= VM_MAXUSER_ADDRESS) return (true); MPASS(eva <= VM_MAXUSER_ADDRESS); for (va = sva, prev_ppr = NULL; va < eva;) { ppr = rangeset_lookup(&pmap->pm_pkru, va); if ((ppr == NULL) ^ (prev_ppr == NULL)) return (false); if (ppr == NULL) { va += PAGE_SIZE; continue; } if (prev_ppr->pkru_keyidx != ppr->pkru_keyidx) return (false); va = ppr->pkru_rs_el.re_end; } return (true); } static pt_entry_t pmap_pkru_get(pmap_t pmap, vm_offset_t va) { struct pmap_pkru_range *ppr; PMAP_LOCK_ASSERT(pmap, MA_OWNED); if (pmap->pm_type != PT_X86 || (cpu_stdext_feature2 & CPUID_STDEXT2_PKU) == 0 || va >= VM_MAXUSER_ADDRESS) return (0); ppr = rangeset_lookup(&pmap->pm_pkru, va); if (ppr != NULL) return (X86_PG_PKU(ppr->pkru_keyidx)); return (0); } static bool pred_pkru_on_remove(void *ctx __unused, void *r) { struct pmap_pkru_range *ppr; ppr = r; return ((ppr->pkru_flags & AMD64_PKRU_PERSIST) == 0); } static void pmap_pkru_on_remove(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); if (pmap->pm_type == PT_X86 && (cpu_stdext_feature2 & CPUID_STDEXT2_PKU) != 0) { rangeset_remove_pred(&pmap->pm_pkru, sva, eva, pred_pkru_on_remove); } } static int pmap_pkru_copy(pmap_t dst_pmap, pmap_t src_pmap) { PMAP_LOCK_ASSERT(dst_pmap, MA_OWNED); PMAP_LOCK_ASSERT(src_pmap, MA_OWNED); MPASS(dst_pmap->pm_type == PT_X86); MPASS(src_pmap->pm_type == PT_X86); MPASS((cpu_stdext_feature2 & CPUID_STDEXT2_PKU) != 0); if (src_pmap->pm_pkru.rs_data_ctx == NULL) return (0); return (rangeset_copy(&dst_pmap->pm_pkru, &src_pmap->pm_pkru)); } static void pmap_pkru_update_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, u_int keyidx) { pml4_entry_t *pml4e; pdp_entry_t *pdpe; pd_entry_t newpde, ptpaddr, *pde; pt_entry_t newpte, *ptep, pte; vm_offset_t va, va_next; bool changed; PMAP_LOCK_ASSERT(pmap, MA_OWNED); MPASS(pmap->pm_type == PT_X86); MPASS(keyidx <= PMAP_MAX_PKRU_IDX); for (changed = false, va = sva; va < eva; va = va_next) { pml4e = pmap_pml4e(pmap, va); if ((*pml4e & X86_PG_V) == 0) { va_next = (va + NBPML4) & ~PML4MASK; if (va_next < va) va_next = eva; continue; } pdpe = pmap_pml4e_to_pdpe(pml4e, va); if ((*pdpe & X86_PG_V) == 0) { va_next = (va + NBPDP) & ~PDPMASK; if (va_next < va) va_next = eva; continue; } va_next = (va + NBPDR) & ~PDRMASK; if (va_next < va) va_next = eva; pde = pmap_pdpe_to_pde(pdpe, va); ptpaddr = *pde; if (ptpaddr == 0) continue; MPASS((ptpaddr & X86_PG_V) != 0); if ((ptpaddr & PG_PS) != 0) { if (va + NBPDR == va_next && eva >= va_next) { newpde = (ptpaddr & ~X86_PG_PKU_MASK) | X86_PG_PKU(keyidx); if (newpde != ptpaddr) { *pde = newpde; changed = true; } continue; } else if (!pmap_demote_pde(pmap, pde, va)) { continue; } } if (va_next > eva) va_next = eva; for (ptep = pmap_pde_to_pte(pde, va); va != va_next; ptep++, va += PAGE_SIZE) { pte = *ptep; if ((pte & X86_PG_V) == 0) continue; newpte = (pte & ~X86_PG_PKU_MASK) | X86_PG_PKU(keyidx); if (newpte != pte) { *ptep = newpte; changed = true; } } } if (changed) pmap_invalidate_range(pmap, sva, eva); } static int pmap_pkru_check_uargs(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, u_int keyidx, int flags) { if (pmap->pm_type != PT_X86 || keyidx > PMAP_MAX_PKRU_IDX || (flags & ~(AMD64_PKRU_PERSIST | AMD64_PKRU_EXCL)) != 0) return (EINVAL); if (eva <= sva || eva > VM_MAXUSER_ADDRESS) return (EFAULT); if ((cpu_stdext_feature2 & CPUID_STDEXT2_PKU) == 0) return (ENOTSUP); return (0); } int pmap_pkru_set(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, u_int keyidx, int flags) { int error; sva = trunc_page(sva); eva = round_page(eva); error = pmap_pkru_check_uargs(pmap, sva, eva, keyidx, flags); if (error != 0) return (error); for (;;) { PMAP_LOCK(pmap); error = pmap_pkru_assign(pmap, sva, eva, keyidx, flags); if (error == 0) pmap_pkru_update_range(pmap, sva, eva, keyidx); PMAP_UNLOCK(pmap); if (error != ENOMEM) break; vm_wait(NULL); } return (error); } int pmap_pkru_clear(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { int error; sva = trunc_page(sva); eva = round_page(eva); error = pmap_pkru_check_uargs(pmap, sva, eva, 0, 0); if (error != 0) return (error); for (;;) { PMAP_LOCK(pmap); error = pmap_pkru_deassign(pmap, sva, eva); if (error == 0) pmap_pkru_update_range(pmap, sva, eva, 0); PMAP_UNLOCK(pmap); if (error != ENOMEM) break; vm_wait(NULL); } return (error); } /* * Track a range of the kernel's virtual address space that is contiguous * in various mapping attributes. */ struct pmap_kernel_map_range { vm_offset_t sva; pt_entry_t attrs; int ptes; int pdes; int pdpes; }; static void sysctl_kmaps_dump(struct sbuf *sb, struct pmap_kernel_map_range *range, vm_offset_t eva) { const char *mode; int i, pat_idx; if (eva <= range->sva) return; pat_idx = pmap_pat_index(kernel_pmap, range->attrs, true); for (i = 0; i < PAT_INDEX_SIZE; i++) if (pat_index[i] == pat_idx) break; switch (i) { case PAT_WRITE_BACK: mode = "WB"; break; case PAT_WRITE_THROUGH: mode = "WT"; break; case PAT_UNCACHEABLE: mode = "UC"; break; case PAT_UNCACHED: mode = "U-"; break; case PAT_WRITE_PROTECTED: mode = "WP"; break; case PAT_WRITE_COMBINING: mode = "WC"; break; default: printf("%s: unknown PAT mode %#x for range 0x%016lx-0x%016lx\n", __func__, pat_idx, range->sva, eva); mode = "??"; break; } sbuf_printf(sb, "0x%016lx-0x%016lx r%c%c%c%c %s %d %d %d\n", range->sva, eva, (range->attrs & X86_PG_RW) != 0 ? 'w' : '-', (range->attrs & pg_nx) != 0 ? '-' : 'x', (range->attrs & X86_PG_U) != 0 ? 'u' : 's', (range->attrs & X86_PG_G) != 0 ? 'g' : '-', mode, range->pdpes, range->pdes, range->ptes); /* Reset to sentinel value. */ range->sva = KVADDR(NPML4EPG - 1, NPDPEPG - 1, NPDEPG - 1, NPTEPG - 1); } /* * Determine whether the attributes specified by a page table entry match those * being tracked by the current range. This is not quite as simple as a direct * flag comparison since some PAT modes have multiple representations. */ static bool sysctl_kmaps_match(struct pmap_kernel_map_range *range, pt_entry_t attrs) { pt_entry_t diff, mask; mask = X86_PG_G | X86_PG_RW | X86_PG_U | X86_PG_PDE_CACHE | pg_nx; diff = (range->attrs ^ attrs) & mask; if (diff == 0) return (true); if ((diff & ~X86_PG_PDE_PAT) == 0 && pmap_pat_index(kernel_pmap, range->attrs, true) == pmap_pat_index(kernel_pmap, attrs, true)) return (true); return (false); } static void sysctl_kmaps_reinit(struct pmap_kernel_map_range *range, vm_offset_t va, pt_entry_t attrs) { memset(range, 0, sizeof(*range)); range->sva = va; range->attrs = attrs; } /* * Given a leaf PTE, derive the mapping's attributes. If they do not match * those of the current run, dump the address range and its attributes, and * begin a new run. */ static void sysctl_kmaps_check(struct sbuf *sb, struct pmap_kernel_map_range *range, vm_offset_t va, pml4_entry_t pml4e, pdp_entry_t pdpe, pd_entry_t pde, pt_entry_t pte) { pt_entry_t attrs; attrs = pml4e & (X86_PG_RW | X86_PG_U | pg_nx); attrs |= pdpe & pg_nx; attrs &= pg_nx | (pdpe & (X86_PG_RW | X86_PG_U)); if ((pdpe & PG_PS) != 0) { attrs |= pdpe & (X86_PG_G | X86_PG_PDE_CACHE); } else if (pde != 0) { attrs |= pde & pg_nx; attrs &= pg_nx | (pde & (X86_PG_RW | X86_PG_U)); } if ((pde & PG_PS) != 0) { attrs |= pde & (X86_PG_G | X86_PG_PDE_CACHE); } else if (pte != 0) { attrs |= pte & pg_nx; attrs &= pg_nx | (pte & (X86_PG_RW | X86_PG_U)); attrs |= pte & (X86_PG_G | X86_PG_PTE_CACHE); /* Canonicalize by always using the PDE PAT bit. */ if ((attrs & X86_PG_PTE_PAT) != 0) attrs ^= X86_PG_PDE_PAT | X86_PG_PTE_PAT; } if (range->sva > va || !sysctl_kmaps_match(range, attrs)) { sysctl_kmaps_dump(sb, range, va); sysctl_kmaps_reinit(range, va, attrs); } } static int sysctl_kmaps(SYSCTL_HANDLER_ARGS) { struct pmap_kernel_map_range range; struct sbuf sbuf, *sb; pml4_entry_t pml4e; pdp_entry_t *pdp, pdpe; pd_entry_t *pd, pde; pt_entry_t *pt, pte; vm_offset_t sva; vm_paddr_t pa; int error, i, j, k, l; error = sysctl_wire_old_buffer(req, 0); if (error != 0) return (error); sb = &sbuf; sbuf_new_for_sysctl(sb, NULL, PAGE_SIZE, req); /* Sentinel value. */ range.sva = KVADDR(NPML4EPG - 1, NPDPEPG - 1, NPDEPG - 1, NPTEPG - 1); /* * Iterate over the kernel page tables without holding the kernel pmap * lock. Outside of the large map, kernel page table pages are never * freed, so at worst we will observe inconsistencies in the output. * Within the large map, ensure that PDP and PD page addresses are * valid before descending. */ for (sva = 0, i = pmap_pml4e_index(sva); i < NPML4EPG; i++) { switch (i) { case PML4PML4I: sbuf_printf(sb, "\nRecursive map:\n"); break; case DMPML4I: sbuf_printf(sb, "\nDirect map:\n"); break; case KPML4BASE: sbuf_printf(sb, "\nKernel map:\n"); break; case LMSPML4I: sbuf_printf(sb, "\nLarge map:\n"); break; } /* Convert to canonical form. */ if (sva == 1ul << 47) sva |= -1ul << 48; restart: pml4e = kernel_pmap->pm_pml4[i]; if ((pml4e & X86_PG_V) == 0) { sva = rounddown2(sva, NBPML4); sysctl_kmaps_dump(sb, &range, sva); sva += NBPML4; continue; } pa = pml4e & PG_FRAME; pdp = (pdp_entry_t *)PHYS_TO_DMAP(pa); for (j = pmap_pdpe_index(sva); j < NPDPEPG; j++) { pdpe = pdp[j]; if ((pdpe & X86_PG_V) == 0) { sva = rounddown2(sva, NBPDP); sysctl_kmaps_dump(sb, &range, sva); sva += NBPDP; continue; } pa = pdpe & PG_FRAME; if (PMAP_ADDRESS_IN_LARGEMAP(sva) && vm_phys_paddr_to_vm_page(pa) == NULL) goto restart; if ((pdpe & PG_PS) != 0) { sva = rounddown2(sva, NBPDP); sysctl_kmaps_check(sb, &range, sva, pml4e, pdpe, 0, 0); range.pdpes++; sva += NBPDP; continue; } pd = (pd_entry_t *)PHYS_TO_DMAP(pa); for (k = pmap_pde_index(sva); k < NPDEPG; k++) { pde = pd[k]; if ((pde & X86_PG_V) == 0) { sva = rounddown2(sva, NBPDR); sysctl_kmaps_dump(sb, &range, sva); sva += NBPDR; continue; } pa = pde & PG_FRAME; if (PMAP_ADDRESS_IN_LARGEMAP(sva) && vm_phys_paddr_to_vm_page(pa) == NULL) goto restart; if ((pde & PG_PS) != 0) { sva = rounddown2(sva, NBPDR); sysctl_kmaps_check(sb, &range, sva, pml4e, pdpe, pde, 0); range.pdes++; sva += NBPDR; continue; } pt = (pt_entry_t *)PHYS_TO_DMAP(pa); for (l = pmap_pte_index(sva); l < NPTEPG; l++, sva += PAGE_SIZE) { pte = pt[l]; if ((pte & X86_PG_V) == 0) { sysctl_kmaps_dump(sb, &range, sva); continue; } sysctl_kmaps_check(sb, &range, sva, pml4e, pdpe, pde, pte); range.ptes++; } } } } error = sbuf_finish(sb); sbuf_delete(sb); return (error); } SYSCTL_OID(_vm_pmap, OID_AUTO, kernel_maps, CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, 0, sysctl_kmaps, "A", "Dump kernel address layout"); #ifdef DDB DB_SHOW_COMMAND(pte, pmap_print_pte) { pmap_t pmap; pml4_entry_t *pml4; pdp_entry_t *pdp; pd_entry_t *pde; pt_entry_t *pte, PG_V; vm_offset_t va; if (!have_addr) { db_printf("show pte addr\n"); return; } va = (vm_offset_t)addr; if (kdb_thread != NULL) pmap = vmspace_pmap(kdb_thread->td_proc->p_vmspace); else pmap = PCPU_GET(curpmap); PG_V = pmap_valid_bit(pmap); pml4 = pmap_pml4e(pmap, va); db_printf("VA 0x%016lx pml4e 0x%016lx", va, *pml4); if ((*pml4 & PG_V) == 0) { db_printf("\n"); return; } pdp = pmap_pml4e_to_pdpe(pml4, va); db_printf(" pdpe 0x%016lx", *pdp); if ((*pdp & PG_V) == 0 || (*pdp & PG_PS) != 0) { db_printf("\n"); return; } pde = pmap_pdpe_to_pde(pdp, va); db_printf(" pde 0x%016lx", *pde); if ((*pde & PG_V) == 0 || (*pde & PG_PS) != 0) { db_printf("\n"); return; } pte = pmap_pde_to_pte(pde, va); db_printf(" pte 0x%016lx\n", *pte); } DB_SHOW_COMMAND(phys2dmap, pmap_phys2dmap) { vm_paddr_t a; if (have_addr) { a = (vm_paddr_t)addr; db_printf("0x%jx\n", (uintmax_t)PHYS_TO_DMAP(a)); } else { db_printf("show phys2dmap addr\n"); } } #endif Index: projects/clang1000-import/sys/amd64/amd64/trap.c =================================================================== --- projects/clang1000-import/sys/amd64/amd64/trap.c (revision 357965) +++ projects/clang1000-import/sys/amd64/amd64/trap.c (revision 357966) @@ -1,1201 +1,1201 @@ /*- * SPDX-License-Identifier: BSD-4-Clause * * Copyright (C) 1994, David Greenman * Copyright (c) 1990, 1993 * The Regents of the University of California. All rights reserved. * * This code is derived from software contributed to Berkeley by * the University of Utah, and William Jolitz. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by the University of * California, Berkeley and its contributors. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * from: @(#)trap.c 7.4 (Berkeley) 5/13/91 */ #include __FBSDID("$FreeBSD$"); /* * AMD64 Trap and System call handling */ #include "opt_clock.h" #include "opt_compat.h" #include "opt_cpu.h" #include "opt_hwpmc_hooks.h" #include "opt_isa.h" #include "opt_kdb.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef HWPMC_HOOKS #include PMC_SOFT_DEFINE( , , page_fault, all); PMC_SOFT_DEFINE( , , page_fault, read); PMC_SOFT_DEFINE( , , page_fault, write); #endif #include #include #include #include #include #include #include #include #include #include #include #include #ifdef SMP #include #endif #include #include #include #ifdef KDTRACE_HOOKS #include #endif extern inthand_t IDTVEC(bpt), IDTVEC(bpt_pti), IDTVEC(dbg), IDTVEC(fast_syscall), IDTVEC(fast_syscall_pti), IDTVEC(fast_syscall32), IDTVEC(int0x80_syscall_pti), IDTVEC(int0x80_syscall); void __noinline trap(struct trapframe *frame); void trap_check(struct trapframe *frame); void dblfault_handler(struct trapframe *frame); static int trap_pfault(struct trapframe *, bool, int *, int *); static void trap_fatal(struct trapframe *, vm_offset_t); #ifdef KDTRACE_HOOKS static bool trap_user_dtrace(struct trapframe *, int (**hook)(struct trapframe *)); #endif static const char UNKNOWN[] = "unknown"; static const char *const trap_msg[] = { [0] = UNKNOWN, /* unused */ [T_PRIVINFLT] = "privileged instruction fault", [2] = UNKNOWN, /* unused */ [T_BPTFLT] = "breakpoint instruction fault", [4] = UNKNOWN, /* unused */ [5] = UNKNOWN, /* unused */ [T_ARITHTRAP] = "arithmetic trap", [7] = UNKNOWN, /* unused */ [8] = UNKNOWN, /* unused */ [T_PROTFLT] = "general protection fault", [T_TRCTRAP] = "debug exception", [11] = UNKNOWN, /* unused */ [T_PAGEFLT] = "page fault", [13] = UNKNOWN, /* unused */ [T_ALIGNFLT] = "alignment fault", [15] = UNKNOWN, /* unused */ [16] = UNKNOWN, /* unused */ [17] = UNKNOWN, /* unused */ [T_DIVIDE] = "integer divide fault", [T_NMI] = "non-maskable interrupt trap", [T_OFLOW] = "overflow trap", [T_BOUND] = "FPU bounds check fault", [T_DNA] = "FPU device not available", [T_DOUBLEFLT] = "double fault", [T_FPOPFLT] = "FPU operand fetch fault", [T_TSSFLT] = "invalid TSS fault", [T_SEGNPFLT] = "segment not present fault", [T_STKFLT] = "stack fault", [T_MCHK] = "machine check trap", [T_XMMFLT] = "SIMD floating-point exception", [T_RESERVED] = "reserved (unknown) fault", [31] = UNKNOWN, /* reserved */ [T_DTRACE_RET] = "DTrace pid return trap", }; static int uprintf_signal; SYSCTL_INT(_machdep, OID_AUTO, uprintf_signal, CTLFLAG_RWTUN, &uprintf_signal, 0, "Print debugging information on trap signal to ctty"); /* * Control L1D flush on return from NMI. * * Tunable can be set to the following values: * 0 - only enable flush on return from NMI if required by vmm.ko (default) * >1 - always flush on return from NMI. * * Post-boot, the sysctl indicates if flushing is currently enabled. */ int nmi_flush_l1d_sw; SYSCTL_INT(_machdep, OID_AUTO, nmi_flush_l1d_sw, CTLFLAG_RWTUN, &nmi_flush_l1d_sw, 0, "Flush L1 Data Cache on NMI exit, software bhyve L1TF mitigation assist"); /* * Exception, fault, and trap interface to the FreeBSD kernel. * This common code is called from assembly language IDT gate entry * routines that prepare a suitable stack frame, and restore this * frame after the exception has been processed. */ void trap(struct trapframe *frame) { ksiginfo_t ksi; struct thread *td; struct proc *p; register_t addr, dr6; int pf, signo, ucode; u_int type; td = curthread; p = td->td_proc; dr6 = 0; VM_CNT_INC(v_trap); type = frame->tf_trapno; #ifdef SMP /* Handler for NMI IPIs used for stopping CPUs. */ if (type == T_NMI && ipi_nmi_handler() == 0) return; #endif #ifdef KDB if (kdb_active) { kdb_reenter(); return; } #endif if (type == T_RESERVED) { trap_fatal(frame, 0); return; } if (type == T_NMI) { #ifdef HWPMC_HOOKS /* * CPU PMCs interrupt using an NMI. If the PMC module is * active, pass the 'rip' value to the PMC module's interrupt * handler. A non-zero return value from the handler means that * the NMI was consumed by it and we can return immediately. */ if (pmc_intr != NULL && (*pmc_intr)(frame) != 0) return; #endif } if ((frame->tf_rflags & PSL_I) == 0) { /* * Buggy application or kernel code has disabled * interrupts and then trapped. Enabling interrupts * now is wrong, but it is better than running with * interrupts disabled until they are accidentally * enabled later. */ if (TRAPF_USERMODE(frame)) uprintf( "pid %ld (%s): trap %d with interrupts disabled\n", (long)curproc->p_pid, curthread->td_name, type); else if (type != T_NMI && type != T_BPTFLT && type != T_TRCTRAP) { /* * XXX not quite right, since this may be for a * multiple fault in user mode. */ printf("kernel trap %d with interrupts disabled\n", type); /* * We shouldn't enable interrupts while holding a * spin lock. */ if (td->td_md.md_spinlock_count == 0) enable_intr(); } } if (TRAPF_USERMODE(frame)) { /* user trap */ td->td_pticks = 0; td->td_frame = frame; addr = frame->tf_rip; if (td->td_cowgen != p->p_cowgen) thread_cow_update(td); switch (type) { case T_PRIVINFLT: /* privileged instruction fault */ signo = SIGILL; ucode = ILL_PRVOPC; break; case T_BPTFLT: /* bpt instruction fault */ #ifdef KDTRACE_HOOKS if (trap_user_dtrace(frame, &dtrace_pid_probe_ptr)) return; #else enable_intr(); #endif signo = SIGTRAP; ucode = TRAP_BRKPT; break; case T_TRCTRAP: /* debug exception */ enable_intr(); signo = SIGTRAP; ucode = TRAP_TRACE; dr6 = rdr6(); if ((dr6 & DBREG_DR6_BS) != 0) { PROC_LOCK(td->td_proc); if ((td->td_dbgflags & TDB_STEP) != 0) { td->td_frame->tf_rflags &= ~PSL_T; td->td_dbgflags &= ~TDB_STEP; } PROC_UNLOCK(td->td_proc); } break; case T_ARITHTRAP: /* arithmetic trap */ ucode = fputrap_x87(); if (ucode == -1) return; signo = SIGFPE; break; case T_PROTFLT: /* general protection fault */ signo = SIGBUS; ucode = BUS_OBJERR; break; case T_STKFLT: /* stack fault */ case T_SEGNPFLT: /* segment not present fault */ signo = SIGBUS; ucode = BUS_ADRERR; break; case T_TSSFLT: /* invalid TSS fault */ signo = SIGBUS; ucode = BUS_OBJERR; break; case T_ALIGNFLT: signo = SIGBUS; ucode = BUS_ADRALN; break; case T_DOUBLEFLT: /* double fault */ default: signo = SIGBUS; ucode = BUS_OBJERR; break; case T_PAGEFLT: /* page fault */ /* * Can emulator handle this trap? */ if (*p->p_sysent->sv_trap != NULL && (*p->p_sysent->sv_trap)(td) == 0) return; pf = trap_pfault(frame, true, &signo, &ucode); if (pf == -1) return; if (pf == 0) goto userret; addr = frame->tf_addr; break; case T_DIVIDE: /* integer divide fault */ ucode = FPE_INTDIV; signo = SIGFPE; break; #ifdef DEV_ISA case T_NMI: nmi_handle_intr(type, frame); return; #endif case T_OFLOW: /* integer overflow fault */ ucode = FPE_INTOVF; signo = SIGFPE; break; case T_BOUND: /* bounds check fault */ ucode = FPE_FLTSUB; signo = SIGFPE; break; case T_DNA: /* transparent fault (due to context switch "late") */ KASSERT(PCB_USER_FPU(td->td_pcb), ("kernel FPU ctx has leaked")); fpudna(); return; case T_FPOPFLT: /* FPU operand fetch fault */ ucode = ILL_COPROC; signo = SIGILL; break; case T_XMMFLT: /* SIMD floating-point exception */ ucode = fputrap_sse(); if (ucode == -1) return; signo = SIGFPE; break; #ifdef KDTRACE_HOOKS case T_DTRACE_RET: (void)trap_user_dtrace(frame, &dtrace_return_probe_ptr); return; #endif } } else { /* kernel trap */ KASSERT(cold || td->td_ucred != NULL, ("kernel trap doesn't have ucred")); switch (type) { case T_PAGEFLT: /* page fault */ (void)trap_pfault(frame, false, NULL, NULL); return; case T_DNA: if (PCB_USER_FPU(td->td_pcb)) panic("Unregistered use of FPU in kernel"); fpudna(); return; case T_ARITHTRAP: /* arithmetic trap */ case T_XMMFLT: /* SIMD floating-point exception */ case T_FPOPFLT: /* FPU operand fetch fault */ /* * For now, supporting kernel handler * registration for FPU traps is overkill. */ trap_fatal(frame, 0); return; case T_STKFLT: /* stack fault */ case T_PROTFLT: /* general protection fault */ case T_SEGNPFLT: /* segment not present fault */ if (td->td_intr_nesting_level != 0) break; /* * Invalid segment selectors and out of bounds * %rip's and %rsp's can be set up in user mode. * This causes a fault in kernel mode when the * kernel tries to return to user mode. We want * to get this fault so that we can fix the * problem here and not have to check all the * selectors and pointers when the user changes * them. * * In case of PTI, the IRETQ faulted while the * kernel used the pti stack, and exception * frame records %rsp value pointing to that * stack. If we return normally to * doreti_iret_fault, the trapframe is * reconstructed on pti stack, and calltrap() * called on it as well. Due to the very * limited pti stack size, kernel does not * survive for too long. Switch to the normal * thread stack for the trap handling. * * Magic '5' is the number of qwords occupied by * the hardware trap frame. */ if (frame->tf_rip == (long)doreti_iret) { frame->tf_rip = (long)doreti_iret_fault; if ((PCPU_GET(curpmap)->pm_ucr3 != PMAP_NO_CR3) && (frame->tf_rsp == (uintptr_t)PCPU_GET( pti_rsp0) - 5 * sizeof(register_t))) { frame->tf_rsp = PCPU_GET(rsp0) - 5 * sizeof(register_t); } return; } if (frame->tf_rip == (long)ld_ds) { frame->tf_rip = (long)ds_load_fault; return; } if (frame->tf_rip == (long)ld_es) { frame->tf_rip = (long)es_load_fault; return; } if (frame->tf_rip == (long)ld_fs) { frame->tf_rip = (long)fs_load_fault; return; } if (frame->tf_rip == (long)ld_gs) { frame->tf_rip = (long)gs_load_fault; return; } if (frame->tf_rip == (long)ld_gsbase) { frame->tf_rip = (long)gsbase_load_fault; return; } if (frame->tf_rip == (long)ld_fsbase) { frame->tf_rip = (long)fsbase_load_fault; return; } if (curpcb->pcb_onfault != NULL) { frame->tf_rip = (long)curpcb->pcb_onfault; return; } break; case T_TSSFLT: /* * PSL_NT can be set in user mode and isn't cleared * automatically when the kernel is entered. This * causes a TSS fault when the kernel attempts to * `iret' because the TSS link is uninitialized. We * want to get this fault so that we can fix the * problem here and not every time the kernel is * entered. */ if (frame->tf_rflags & PSL_NT) { frame->tf_rflags &= ~PSL_NT; return; } break; case T_TRCTRAP: /* debug exception */ /* Clear any pending debug events. */ dr6 = rdr6(); load_dr6(0); /* * Ignore debug register exceptions due to * accesses in the user's address space, which * can happen under several conditions such as * if a user sets a watchpoint on a buffer and * then passes that buffer to a system call. * We still want to get TRCTRAPS for addresses * in kernel space because that is useful when * debugging the kernel. */ if (user_dbreg_trap(dr6)) return; /* * Malicious user code can configure a debug * register watchpoint to trap on data access * to the top of stack and then execute 'pop * %ss; int 3'. Due to exception deferral for * 'pop %ss', the CPU will not interrupt 'int * 3' to raise the DB# exception for the debug * register but will postpone the DB# until * execution of the first instruction of the * BP# handler (in kernel mode). Normally the * previous check would ignore DB# exceptions * for watchpoints on user addresses raised in * kernel mode. However, some CPU errata * include cases where DB# exceptions do not * properly set bits in %dr6, e.g. Haswell * HSD23 and Skylake-X SKZ24. * * A deferred DB# can also be raised on the * first instructions of system call entry * points or single-step traps via similar use * of 'pop %ss' or 'mov xxx, %ss'. */ if (pti) { if (frame->tf_rip == (uintptr_t)IDTVEC(fast_syscall_pti) || #ifdef COMPAT_FREEBSD32 frame->tf_rip == (uintptr_t)IDTVEC(int0x80_syscall_pti) || #endif frame->tf_rip == (uintptr_t)IDTVEC(bpt_pti)) return; } else { if (frame->tf_rip == (uintptr_t)IDTVEC(fast_syscall) || #ifdef COMPAT_FREEBSD32 frame->tf_rip == (uintptr_t)IDTVEC(int0x80_syscall) || #endif frame->tf_rip == (uintptr_t)IDTVEC(bpt)) return; } if (frame->tf_rip == (uintptr_t)IDTVEC(dbg) || /* Needed for AMD. */ frame->tf_rip == (uintptr_t)IDTVEC(fast_syscall32)) return; /* * FALLTHROUGH (TRCTRAP kernel mode, kernel address) */ case T_BPTFLT: /* * If KDB is enabled, let it handle the debugger trap. * Otherwise, debugger traps "can't happen". */ #ifdef KDB if (kdb_trap(type, dr6, frame)) return; #endif break; #ifdef DEV_ISA case T_NMI: nmi_handle_intr(type, frame); return; #endif } trap_fatal(frame, 0); return; } /* Translate fault for emulators (e.g. Linux) */ if (*p->p_sysent->sv_transtrap != NULL) signo = (*p->p_sysent->sv_transtrap)(signo, type); ksiginfo_init_trap(&ksi); ksi.ksi_signo = signo; ksi.ksi_code = ucode; ksi.ksi_trapno = type; ksi.ksi_addr = (void *)addr; if (uprintf_signal) { uprintf("pid %d comm %s: signal %d err %lx code %d type %d " "addr 0x%lx rsp 0x%lx rip 0x%lx " "<%02x %02x %02x %02x %02x %02x %02x %02x>\n", p->p_pid, p->p_comm, signo, frame->tf_err, ucode, type, addr, frame->tf_rsp, frame->tf_rip, fubyte((void *)(frame->tf_rip + 0)), fubyte((void *)(frame->tf_rip + 1)), fubyte((void *)(frame->tf_rip + 2)), fubyte((void *)(frame->tf_rip + 3)), fubyte((void *)(frame->tf_rip + 4)), fubyte((void *)(frame->tf_rip + 5)), fubyte((void *)(frame->tf_rip + 6)), fubyte((void *)(frame->tf_rip + 7))); } KASSERT((read_rflags() & PSL_I) != 0, ("interrupts disabled")); trapsignal(td, &ksi); userret: userret(td, frame); KASSERT(PCB_USER_FPU(td->td_pcb), ("Return from trap with kernel FPU ctx leaked")); } /* * Ensure that we ignore any DTrace-induced faults. This function cannot * be instrumented, so it cannot generate such faults itself. */ void trap_check(struct trapframe *frame) { #ifdef KDTRACE_HOOKS if (dtrace_trap_func != NULL && (*dtrace_trap_func)(frame, frame->tf_trapno) != 0) return; #endif trap(frame); } static bool trap_is_smap(struct trapframe *frame) { /* * A page fault on a userspace address is classified as * SMAP-induced if: * - SMAP is supported; * - kernel mode accessed present data page; * - rflags.AC was cleared. * Kernel must never access user space with rflags.AC cleared * if SMAP is enabled. */ return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 && (frame->tf_err & (PGEX_P | PGEX_U | PGEX_I | PGEX_RSV)) == PGEX_P && (frame->tf_rflags & PSL_AC) == 0); } static bool trap_is_pti(struct trapframe *frame) { return (PCPU_GET(curpmap)->pm_ucr3 != PMAP_NO_CR3 && pg_nx != 0 && (frame->tf_err & (PGEX_P | PGEX_W | PGEX_U | PGEX_I)) == (PGEX_P | PGEX_U | PGEX_I) && (curpcb->pcb_saved_ucr3 & ~CR3_PCID_MASK) == (PCPU_GET(curpmap)->pm_cr3 & ~CR3_PCID_MASK)); } /* * Handle all details of a page fault. * Returns: * -1 if this fault was fatal, typically from kernel mode * (cannot happen, but we need to return something). * 0 if this fault was handled by updating either the user or kernel * page table, execution can continue. * 1 if this fault was from usermode and it was not handled, a synchronous * signal should be delivered to the thread. *signo returns the signal * number, *ucode gives si_code. */ static int trap_pfault(struct trapframe *frame, bool usermode, int *signo, int *ucode) { struct thread *td; struct proc *p; vm_map_t map; vm_offset_t eva; int rv; vm_prot_t ftype; MPASS(!usermode || (signo != NULL && ucode != NULL)); td = curthread; p = td->td_proc; eva = frame->tf_addr; if (__predict_false((td->td_pflags & TDP_NOFAULTING) != 0)) { /* * Due to both processor errata and lazy TLB invalidation when * access restrictions are removed from virtual pages, memory * accesses that are allowed by the physical mapping layer may * nonetheless cause one spurious page fault per virtual page. * When the thread is executing a "no faulting" section that * is bracketed by vm_fault_{disable,enable}_pagefaults(), * every page fault is treated as a spurious page fault, * unless it accesses the same virtual address as the most * recent page fault within the same "no faulting" section. */ if (td->td_md.md_spurflt_addr != eva || (td->td_pflags & TDP_RESETSPUR) != 0) { /* * Do nothing to the TLB. A stale TLB entry is * flushed automatically by a page fault. */ td->td_md.md_spurflt_addr = eva; td->td_pflags &= ~TDP_RESETSPUR; return (0); } } else { /* * If we get a page fault while in a critical section, then * it is most likely a fatal kernel page fault. The kernel * is already going to panic trying to get a sleep lock to * do the VM lookup, so just consider it a fatal trap so the * kernel can print out a useful trap message and even get * to the debugger. * * If we get a page fault while holding a non-sleepable * lock, then it is most likely a fatal kernel page fault. * If WITNESS is enabled, then it's going to whine about * bogus LORs with various VM locks, so just skip to the * fatal trap handling directly. */ if (td->td_critnest != 0 || WITNESS_CHECK(WARN_SLEEPOK | WARN_GIANTOK, NULL, "Kernel page fault") != 0) { trap_fatal(frame, eva); return (-1); } } if (eva >= VM_MIN_KERNEL_ADDRESS) { /* * Don't allow user-mode faults in kernel address space. */ if (usermode) { *signo = SIGSEGV; *ucode = SEGV_MAPERR; return (1); } map = kernel_map; } else { map = &p->p_vmspace->vm_map; /* * When accessing a usermode address, kernel must be * ready to accept the page fault, and provide a * handling routine. Since accessing the address * without the handler is a bug, do not try to handle * it normally, and panic immediately. * * If SMAP is enabled, filter SMAP faults also, * because illegal access might occur to the mapped * user address, causing infinite loop. */ if (!usermode && (td->td_intr_nesting_level != 0 || trap_is_smap(frame) || curpcb->pcb_onfault == NULL)) { trap_fatal(frame, eva); return (-1); } } /* * If the trap was caused by errant bits in the PTE then panic. */ if (frame->tf_err & PGEX_RSV) { trap_fatal(frame, eva); return (-1); } /* * User-mode protection key violation (PKU). May happen * either from usermode or from kernel if copyin accessed * key-protected mapping. */ if ((frame->tf_err & PGEX_PK) != 0) { if (eva > VM_MAXUSER_ADDRESS) { trap_fatal(frame, eva); return (-1); } if (usermode) { *signo = SIGSEGV; *ucode = SEGV_PKUERR; return (1); } goto after_vmfault; } /* * If nx protection of the usermode portion of kernel page * tables caused trap, panic. */ if (usermode && trap_is_pti(frame)) panic("PTI: pid %d comm %s tf_err %#lx", p->p_pid, p->p_comm, frame->tf_err); /* * PGEX_I is defined only if the execute disable bit capability is * supported and enabled. */ if (frame->tf_err & PGEX_W) ftype = VM_PROT_WRITE; else if ((frame->tf_err & PGEX_I) && pg_nx != 0) ftype = VM_PROT_EXECUTE; else ftype = VM_PROT_READ; /* Fault in the page. */ rv = vm_fault_trap(map, eva, ftype, VM_FAULT_NORMAL, signo, ucode); if (rv == KERN_SUCCESS) { #ifdef HWPMC_HOOKS if (ftype == VM_PROT_READ || ftype == VM_PROT_WRITE) { PMC_SOFT_CALL_TF( , , page_fault, all, frame); if (ftype == VM_PROT_READ) PMC_SOFT_CALL_TF( , , page_fault, read, frame); else PMC_SOFT_CALL_TF( , , page_fault, write, frame); } #endif return (0); } if (usermode) return (1); after_vmfault: if (td->td_intr_nesting_level == 0 && curpcb->pcb_onfault != NULL) { frame->tf_rip = (long)curpcb->pcb_onfault; return (0); } trap_fatal(frame, eva); return (-1); } static void trap_fatal(frame, eva) struct trapframe *frame; vm_offset_t eva; { int code, ss; u_int type; struct soft_segment_descriptor softseg; struct user_segment_descriptor *gdt; #ifdef KDB bool handled; #endif code = frame->tf_err; type = frame->tf_trapno; gdt = *PCPU_PTR(gdt); sdtossd(&gdt[IDXSEL(frame->tf_cs & 0xffff)], &softseg); printf("\n\nFatal trap %d: %s while in %s mode\n", type, type < nitems(trap_msg) ? trap_msg[type] : UNKNOWN, TRAPF_USERMODE(frame) ? "user" : "kernel"); #ifdef SMP /* two separate prints in case of a trap on an unmapped page */ printf("cpuid = %d; ", PCPU_GET(cpuid)); printf("apic id = %02x\n", PCPU_GET(apic_id)); #endif if (type == T_PAGEFLT) { printf("fault virtual address = 0x%lx\n", eva); printf("fault code = %s %s %s%s%s, %s\n", code & PGEX_U ? "user" : "supervisor", code & PGEX_W ? "write" : "read", code & PGEX_I ? "instruction" : "data", code & PGEX_PK ? " prot key" : "", code & PGEX_SGX ? " SGX" : "", code & PGEX_RSV ? "reserved bits in PTE" : code & PGEX_P ? "protection violation" : "page not present"); } printf("instruction pointer = 0x%lx:0x%lx\n", frame->tf_cs & 0xffff, frame->tf_rip); ss = frame->tf_ss & 0xffff; printf("stack pointer = 0x%x:0x%lx\n", ss, frame->tf_rsp); printf("frame pointer = 0x%x:0x%lx\n", ss, frame->tf_rbp); printf("code segment = base 0x%lx, limit 0x%lx, type 0x%x\n", softseg.ssd_base, softseg.ssd_limit, softseg.ssd_type); printf(" = DPL %d, pres %d, long %d, def32 %d, gran %d\n", softseg.ssd_dpl, softseg.ssd_p, softseg.ssd_long, softseg.ssd_def32, softseg.ssd_gran); printf("processor eflags = "); if (frame->tf_rflags & PSL_T) printf("trace trap, "); if (frame->tf_rflags & PSL_I) printf("interrupt enabled, "); if (frame->tf_rflags & PSL_NT) printf("nested task, "); if (frame->tf_rflags & PSL_RF) printf("resume, "); printf("IOPL = %ld\n", (frame->tf_rflags & PSL_IOPL) >> 12); printf("current process = %d (%s)\n", curproc->p_pid, curthread->td_name); #ifdef KDB if (debugger_on_trap) { kdb_why = KDB_WHY_TRAP; handled = kdb_trap(type, 0, frame); kdb_why = KDB_WHY_UNSET; if (handled) return; } #endif printf("trap number = %d\n", type); panic("%s", type < nitems(trap_msg) ? trap_msg[type] : "unknown/reserved trap"); } #ifdef KDTRACE_HOOKS /* * Invoke a userspace DTrace hook. The hook pointer is cleared when no * userspace probes are enabled, so we must synchronize with DTrace to ensure * that a trapping thread is able to call the hook before it is cleared. */ static bool trap_user_dtrace(struct trapframe *frame, int (**hookp)(struct trapframe *)) { int (*hook)(struct trapframe *); - hook = (int (*)(struct trapframe *))atomic_load_ptr(hookp); + hook = atomic_load_ptr(hookp); enable_intr(); if (hook != NULL) return ((hook)(frame) == 0); return (false); } #endif /* * Double fault handler. Called when a fault occurs while writing * a frame for a trap/exception onto the stack. This usually occurs * when the stack overflows (such is the case with infinite recursion, * for example). */ void dblfault_handler(struct trapframe *frame) { #ifdef KDTRACE_HOOKS if (dtrace_doubletrap_func != NULL) (*dtrace_doubletrap_func)(); #endif printf("\nFatal double fault\n" "rip %#lx rsp %#lx rbp %#lx\n" "rax %#lx rdx %#lx rbx %#lx\n" "rcx %#lx rsi %#lx rdi %#lx\n" "r8 %#lx r9 %#lx r10 %#lx\n" "r11 %#lx r12 %#lx r13 %#lx\n" "r14 %#lx r15 %#lx rflags %#lx\n" "cs %#lx ss %#lx ds %#hx es %#hx fs %#hx gs %#hx\n" "fsbase %#lx gsbase %#lx kgsbase %#lx\n", frame->tf_rip, frame->tf_rsp, frame->tf_rbp, frame->tf_rax, frame->tf_rdx, frame->tf_rbx, frame->tf_rcx, frame->tf_rdi, frame->tf_rsi, frame->tf_r8, frame->tf_r9, frame->tf_r10, frame->tf_r11, frame->tf_r12, frame->tf_r13, frame->tf_r14, frame->tf_r15, frame->tf_rflags, frame->tf_cs, frame->tf_ss, frame->tf_ds, frame->tf_es, frame->tf_fs, frame->tf_gs, rdmsr(MSR_FSBASE), rdmsr(MSR_GSBASE), rdmsr(MSR_KGSBASE)); #ifdef SMP /* two separate prints in case of a trap on an unmapped page */ printf("cpuid = %d; ", PCPU_GET(cpuid)); printf("apic id = %02x\n", PCPU_GET(apic_id)); #endif panic("double fault"); } static int __noinline cpu_fetch_syscall_args_fallback(struct thread *td, struct syscall_args *sa) { struct proc *p; struct trapframe *frame; register_t *argp; caddr_t params; int reg, regcnt, error; p = td->td_proc; frame = td->td_frame; reg = 0; regcnt = NARGREGS; if (sa->code == SYS_syscall || sa->code == SYS___syscall) { sa->code = frame->tf_rdi; reg++; regcnt--; } if (sa->code >= p->p_sysent->sv_size) sa->callp = &p->p_sysent->sv_table[0]; else sa->callp = &p->p_sysent->sv_table[sa->code]; sa->narg = sa->callp->sy_narg; KASSERT(sa->narg <= nitems(sa->args), ("Too many syscall arguments!")); argp = &frame->tf_rdi; argp += reg; memcpy(sa->args, argp, sizeof(sa->args[0]) * NARGREGS); if (sa->narg > regcnt) { params = (caddr_t)frame->tf_rsp + sizeof(register_t); error = copyin(params, &sa->args[regcnt], (sa->narg - regcnt) * sizeof(sa->args[0])); if (__predict_false(error != 0)) return (error); } td->td_retval[0] = 0; td->td_retval[1] = frame->tf_rdx; return (0); } int cpu_fetch_syscall_args(struct thread *td) { struct proc *p; struct trapframe *frame; struct syscall_args *sa; p = td->td_proc; frame = td->td_frame; sa = &td->td_sa; sa->code = frame->tf_rax; if (__predict_false(sa->code == SYS_syscall || sa->code == SYS___syscall || sa->code >= p->p_sysent->sv_size)) return (cpu_fetch_syscall_args_fallback(td, sa)); sa->callp = &p->p_sysent->sv_table[sa->code]; sa->narg = sa->callp->sy_narg; KASSERT(sa->narg <= nitems(sa->args), ("Too many syscall arguments!")); if (__predict_false(sa->narg > NARGREGS)) return (cpu_fetch_syscall_args_fallback(td, sa)); memcpy(sa->args, &frame->tf_rdi, sizeof(sa->args[0]) * NARGREGS); td->td_retval[0] = 0; td->td_retval[1] = frame->tf_rdx; return (0); } #include "../../kern/subr_syscall.c" static void (*syscall_ret_l1d_flush)(void); int syscall_ret_l1d_flush_mode; static void flush_l1d_hw(void) { wrmsr(MSR_IA32_FLUSH_CMD, IA32_FLUSH_CMD_L1D); } static void __noinline amd64_syscall_ret_flush_l1d_check(int error) { void (*p)(void); if (error != EEXIST && error != EAGAIN && error != EXDEV && error != ENOENT && error != ENOTCONN && error != EINPROGRESS) { - p = (void *)atomic_load_ptr(&syscall_ret_l1d_flush); + p = atomic_load_ptr(&syscall_ret_l1d_flush); if (p != NULL) p(); } } static void __inline amd64_syscall_ret_flush_l1d_check_inline(int error) { if (__predict_false(error != 0)) amd64_syscall_ret_flush_l1d_check(error); } void amd64_syscall_ret_flush_l1d(int error) { amd64_syscall_ret_flush_l1d_check_inline(error); } void amd64_syscall_ret_flush_l1d_recalc(void) { bool l1d_hw; l1d_hw = (cpu_stdext_feature3 & CPUID_STDEXT3_L1D_FLUSH) != 0; again: switch (syscall_ret_l1d_flush_mode) { case 0: syscall_ret_l1d_flush = NULL; break; case 1: syscall_ret_l1d_flush = l1d_hw ? flush_l1d_hw : flush_l1d_sw_abi; break; case 2: syscall_ret_l1d_flush = l1d_hw ? flush_l1d_hw : NULL; break; case 3: syscall_ret_l1d_flush = flush_l1d_sw_abi; break; default: syscall_ret_l1d_flush_mode = 1; goto again; } } static int machdep_syscall_ret_flush_l1d(SYSCTL_HANDLER_ARGS) { int error, val; val = syscall_ret_l1d_flush_mode; error = sysctl_handle_int(oidp, &val, 0, req); if (error != 0 || req->newptr == NULL) return (error); syscall_ret_l1d_flush_mode = val; amd64_syscall_ret_flush_l1d_recalc(); return (0); } SYSCTL_PROC(_machdep, OID_AUTO, syscall_ret_flush_l1d, CTLTYPE_INT | CTLFLAG_RWTUN | CTLFLAG_NOFETCH | CTLFLAG_MPSAFE, NULL, 0, machdep_syscall_ret_flush_l1d, "I", "Flush L1D on syscall return with error (0 - off, 1 - on, " "2 - use hw only, 3 - use sw only"); /* * System call handler for native binaries. The trap frame is already * set up by the assembler trampoline and a pointer to it is saved in * td_frame. */ void amd64_syscall(struct thread *td, int traced) { ksiginfo_t ksi; #ifdef DIAGNOSTIC if (!TRAPF_USERMODE(td->td_frame)) { panic("syscall"); /* NOT REACHED */ } #endif syscallenter(td); /* * Traced syscall. */ if (__predict_false(traced)) { td->td_frame->tf_rflags &= ~PSL_T; ksiginfo_init_trap(&ksi); ksi.ksi_signo = SIGTRAP; ksi.ksi_code = TRAP_TRACE; ksi.ksi_addr = (void *)td->td_frame->tf_rip; trapsignal(td, &ksi); } KASSERT(PCB_USER_FPU(td->td_pcb), ("System call %s returning with kernel FPU ctx leaked", syscallname(td->td_proc, td->td_sa.code))); KASSERT(td->td_pcb->pcb_save == get_pcb_user_save_td(td), ("System call %s returning with mangled pcb_save", syscallname(td->td_proc, td->td_sa.code))); KASSERT(pmap_not_in_di(), ("System call %s returning with leaked invl_gen %lu", syscallname(td->td_proc, td->td_sa.code), td->td_md.md_invl_gen.gen)); syscallret(td); /* * If the user-supplied value of %rip is not a canonical * address, then some CPUs will trigger a ring 0 #GP during * the sysret instruction. However, the fault handler would * execute in ring 0 with the user's %gs and %rsp which would * not be safe. Instead, use the full return path which * catches the problem safely. */ if (__predict_false(td->td_frame->tf_rip >= VM_MAXUSER_ADDRESS)) set_pcb_flags(td->td_pcb, PCB_FULL_IRET); amd64_syscall_ret_flush_l1d_check_inline(td->td_errno); } Index: projects/clang1000-import/sys/arm64/arm64/gicv3_its.c =================================================================== --- projects/clang1000-import/sys/arm64/arm64/gicv3_its.c (revision 357965) +++ projects/clang1000-import/sys/arm64/arm64/gicv3_its.c (revision 357966) @@ -1,1796 +1,1795 @@ /*- * Copyright (c) 2015-2016 The FreeBSD Foundation * All rights reserved. * * This software was developed by Andrew Turner under * the sponsorship of the FreeBSD Foundation. * * This software was developed by Semihalf under * the sponsorship of the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include "opt_acpi.h" #include "opt_platform.h" #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef FDT #include #include #include #endif #include #include #include "pcib_if.h" #include "pic_if.h" #include "msi_if.h" MALLOC_DEFINE(M_GICV3_ITS, "GICv3 ITS", "ARM GICv3 Interrupt Translation Service"); #define LPI_NIRQS (64 * 1024) /* The size and alignment of the command circular buffer */ #define ITS_CMDQ_SIZE (64 * 1024) /* Must be a multiple of 4K */ #define ITS_CMDQ_ALIGN (64 * 1024) #define LPI_CONFTAB_SIZE LPI_NIRQS #define LPI_CONFTAB_ALIGN (64 * 1024) #define LPI_CONFTAB_MAX_ADDR ((1ul << 48) - 1) /* We need a 47 bit PA */ /* 1 bit per SPI, PPI, and SGI (8k), and 1 bit per LPI (LPI_CONFTAB_SIZE) */ #define LPI_PENDTAB_SIZE ((LPI_NIRQS + GIC_FIRST_LPI) / 8) #define LPI_PENDTAB_ALIGN (64 * 1024) #define LPI_PENDTAB_MAX_ADDR ((1ul << 48) - 1) /* We need a 47 bit PA */ #define LPI_INT_TRANS_TAB_ALIGN 256 #define LPI_INT_TRANS_TAB_MAX_ADDR ((1ul << 48) - 1) /* ITS commands encoding */ #define ITS_CMD_MOVI (0x01) #define ITS_CMD_SYNC (0x05) #define ITS_CMD_MAPD (0x08) #define ITS_CMD_MAPC (0x09) #define ITS_CMD_MAPTI (0x0a) #define ITS_CMD_MAPI (0x0b) #define ITS_CMD_INV (0x0c) #define ITS_CMD_INVALL (0x0d) /* Command */ #define CMD_COMMAND_MASK (0xFFUL) /* PCI device ID */ #define CMD_DEVID_SHIFT (32) #define CMD_DEVID_MASK (0xFFFFFFFFUL << CMD_DEVID_SHIFT) /* Size of IRQ ID bitfield */ #define CMD_SIZE_MASK (0xFFUL) /* Virtual LPI ID */ #define CMD_ID_MASK (0xFFFFFFFFUL) /* Physical LPI ID */ #define CMD_PID_SHIFT (32) #define CMD_PID_MASK (0xFFFFFFFFUL << CMD_PID_SHIFT) /* Collection */ #define CMD_COL_MASK (0xFFFFUL) /* Target (CPU or Re-Distributor) */ #define CMD_TARGET_SHIFT (16) #define CMD_TARGET_MASK (0xFFFFFFFFUL << CMD_TARGET_SHIFT) /* Interrupt Translation Table address */ #define CMD_ITT_MASK (0xFFFFFFFFFF00UL) /* Valid command bit */ #define CMD_VALID_SHIFT (63) #define CMD_VALID_MASK (1UL << CMD_VALID_SHIFT) #define ITS_TARGET_NONE 0xFBADBEEF /* LPI chunk owned by ITS device */ struct lpi_chunk { u_int lpi_base; u_int lpi_free; /* First free LPI in set */ u_int lpi_num; /* Total number of LPIs in chunk */ u_int lpi_busy; /* Number of busy LPIs in chink */ }; /* ITS device */ struct its_dev { TAILQ_ENTRY(its_dev) entry; /* PCI device */ device_t pci_dev; /* Device ID (i.e. PCI device ID) */ uint32_t devid; /* List of assigned LPIs */ struct lpi_chunk lpis; /* Virtual address of ITT */ vm_offset_t itt; size_t itt_size; }; /* * ITS command descriptor. * Idea for command description passing taken from Linux. */ struct its_cmd_desc { uint8_t cmd_type; union { struct { struct its_dev *its_dev; struct its_col *col; uint32_t id; } cmd_desc_movi; struct { struct its_col *col; } cmd_desc_sync; struct { struct its_col *col; uint8_t valid; } cmd_desc_mapc; struct { struct its_dev *its_dev; struct its_col *col; uint32_t pid; uint32_t id; } cmd_desc_mapvi; struct { struct its_dev *its_dev; struct its_col *col; uint32_t pid; } cmd_desc_mapi; struct { struct its_dev *its_dev; uint8_t valid; } cmd_desc_mapd; struct { struct its_dev *its_dev; struct its_col *col; uint32_t pid; } cmd_desc_inv; struct { struct its_col *col; } cmd_desc_invall; }; }; /* ITS command. Each command is 32 bytes long */ struct its_cmd { uint64_t cmd_dword[4]; /* ITS command double word */ }; /* An ITS private table */ struct its_ptable { vm_offset_t ptab_vaddr; unsigned long ptab_size; }; /* ITS collection description. */ struct its_col { uint64_t col_target; /* Target Re-Distributor */ uint64_t col_id; /* Collection ID */ }; struct gicv3_its_irqsrc { struct intr_irqsrc gi_isrc; u_int gi_id; u_int gi_lpi; struct its_dev *gi_its_dev; }; struct gicv3_its_softc { struct intr_pic *sc_pic; struct resource *sc_its_res; cpuset_t sc_cpus; u_int gic_irq_cpu; struct its_ptable sc_its_ptab[GITS_BASER_NUM]; struct its_col *sc_its_cols[MAXCPU]; /* Per-CPU collections */ /* * TODO: We should get these from the parent as we only want a * single copy of each across the interrupt controller. */ uint8_t *sc_conf_base; vm_offset_t sc_pend_base[MAXCPU]; /* Command handling */ struct mtx sc_its_cmd_lock; struct its_cmd *sc_its_cmd_base; /* Command circular buffer address */ size_t sc_its_cmd_next_idx; vmem_t *sc_irq_alloc; struct gicv3_its_irqsrc *sc_irqs; u_int sc_irq_base; u_int sc_irq_length; struct mtx sc_its_dev_lock; TAILQ_HEAD(its_dev_list, its_dev) sc_its_dev_list; #define ITS_FLAGS_CMDQ_FLUSH 0x00000001 #define ITS_FLAGS_LPI_CONF_FLUSH 0x00000002 #define ITS_FLAGS_ERRATA_CAVIUM_22375 0x00000004 u_int sc_its_flags; }; static void *conf_base; typedef void (its_quirk_func_t)(device_t); static its_quirk_func_t its_quirk_cavium_22375; static const struct { const char *desc; uint32_t iidr; uint32_t iidr_mask; its_quirk_func_t *func; } its_quirks[] = { { /* Cavium ThunderX Pass 1.x */ .desc = "Cavium ThunderX errata: 22375, 24313", .iidr = GITS_IIDR_RAW(GITS_IIDR_IMPL_CAVIUM, GITS_IIDR_PROD_THUNDER, GITS_IIDR_VAR_THUNDER_1, 0), .iidr_mask = ~GITS_IIDR_REVISION_MASK, .func = its_quirk_cavium_22375, }, }; #define gic_its_read_4(sc, reg) \ bus_read_4((sc)->sc_its_res, (reg)) #define gic_its_read_8(sc, reg) \ bus_read_8((sc)->sc_its_res, (reg)) #define gic_its_write_4(sc, reg, val) \ bus_write_4((sc)->sc_its_res, (reg), (val)) #define gic_its_write_8(sc, reg, val) \ bus_write_8((sc)->sc_its_res, (reg), (val)) static device_attach_t gicv3_its_attach; static device_detach_t gicv3_its_detach; static pic_disable_intr_t gicv3_its_disable_intr; static pic_enable_intr_t gicv3_its_enable_intr; static pic_map_intr_t gicv3_its_map_intr; static pic_setup_intr_t gicv3_its_setup_intr; static pic_post_filter_t gicv3_its_post_filter; static pic_post_ithread_t gicv3_its_post_ithread; static pic_pre_ithread_t gicv3_its_pre_ithread; static pic_bind_intr_t gicv3_its_bind_intr; #ifdef SMP static pic_init_secondary_t gicv3_its_init_secondary; #endif static msi_alloc_msi_t gicv3_its_alloc_msi; static msi_release_msi_t gicv3_its_release_msi; static msi_alloc_msix_t gicv3_its_alloc_msix; static msi_release_msix_t gicv3_its_release_msix; static msi_map_msi_t gicv3_its_map_msi; static void its_cmd_movi(device_t, struct gicv3_its_irqsrc *); static void its_cmd_mapc(device_t, struct its_col *, uint8_t); static void its_cmd_mapti(device_t, struct gicv3_its_irqsrc *); static void its_cmd_mapd(device_t, struct its_dev *, uint8_t); static void its_cmd_inv(device_t, struct its_dev *, struct gicv3_its_irqsrc *); static void its_cmd_invall(device_t, struct its_col *); static device_method_t gicv3_its_methods[] = { /* Device interface */ DEVMETHOD(device_detach, gicv3_its_detach), /* Interrupt controller interface */ DEVMETHOD(pic_disable_intr, gicv3_its_disable_intr), DEVMETHOD(pic_enable_intr, gicv3_its_enable_intr), DEVMETHOD(pic_map_intr, gicv3_its_map_intr), DEVMETHOD(pic_setup_intr, gicv3_its_setup_intr), DEVMETHOD(pic_post_filter, gicv3_its_post_filter), DEVMETHOD(pic_post_ithread, gicv3_its_post_ithread), DEVMETHOD(pic_pre_ithread, gicv3_its_pre_ithread), #ifdef SMP DEVMETHOD(pic_bind_intr, gicv3_its_bind_intr), DEVMETHOD(pic_init_secondary, gicv3_its_init_secondary), #endif /* MSI/MSI-X */ DEVMETHOD(msi_alloc_msi, gicv3_its_alloc_msi), DEVMETHOD(msi_release_msi, gicv3_its_release_msi), DEVMETHOD(msi_alloc_msix, gicv3_its_alloc_msix), DEVMETHOD(msi_release_msix, gicv3_its_release_msix), DEVMETHOD(msi_map_msi, gicv3_its_map_msi), /* End */ DEVMETHOD_END }; static DEFINE_CLASS_0(gic, gicv3_its_driver, gicv3_its_methods, sizeof(struct gicv3_its_softc)); static void gicv3_its_cmdq_init(struct gicv3_its_softc *sc) { vm_paddr_t cmd_paddr; uint64_t reg, tmp; /* Set up the command circular buffer */ sc->sc_its_cmd_base = contigmalloc(ITS_CMDQ_SIZE, M_GICV3_ITS, M_WAITOK | M_ZERO, 0, (1ul << 48) - 1, ITS_CMDQ_ALIGN, 0); sc->sc_its_cmd_next_idx = 0; cmd_paddr = vtophys(sc->sc_its_cmd_base); /* Set the base of the command buffer */ reg = GITS_CBASER_VALID | (GITS_CBASER_CACHE_NIWAWB << GITS_CBASER_CACHE_SHIFT) | cmd_paddr | (GITS_CBASER_SHARE_IS << GITS_CBASER_SHARE_SHIFT) | (ITS_CMDQ_SIZE / 4096 - 1); gic_its_write_8(sc, GITS_CBASER, reg); /* Read back to check for fixed value fields */ tmp = gic_its_read_8(sc, GITS_CBASER); if ((tmp & GITS_CBASER_SHARE_MASK) != (GITS_CBASER_SHARE_IS << GITS_CBASER_SHARE_SHIFT)) { /* Check if the hardware reported non-shareable */ if ((tmp & GITS_CBASER_SHARE_MASK) == (GITS_CBASER_SHARE_NS << GITS_CBASER_SHARE_SHIFT)) { /* If so remove the cache attribute */ reg &= ~GITS_CBASER_CACHE_MASK; reg &= ~GITS_CBASER_SHARE_MASK; /* Set to Non-cacheable, Non-shareable */ reg |= GITS_CBASER_CACHE_NIN << GITS_CBASER_CACHE_SHIFT; reg |= GITS_CBASER_SHARE_NS << GITS_CBASER_SHARE_SHIFT; gic_its_write_8(sc, GITS_CBASER, reg); } /* The command queue has to be flushed after each command */ sc->sc_its_flags |= ITS_FLAGS_CMDQ_FLUSH; } /* Get the next command from the start of the buffer */ gic_its_write_8(sc, GITS_CWRITER, 0x0); } static int gicv3_its_table_init(device_t dev, struct gicv3_its_softc *sc) { vm_offset_t table; vm_paddr_t paddr; uint64_t cache, reg, share, tmp, type; size_t esize, its_tbl_size, nidents, nitspages, npages; int i, page_size; int devbits; if ((sc->sc_its_flags & ITS_FLAGS_ERRATA_CAVIUM_22375) != 0) { /* * GITS_TYPER[17:13] of ThunderX reports that device IDs * are to be 21 bits in length. The entry size of the ITS * table can be read from GITS_BASERn[52:48] and on ThunderX * is supposed to be 8 bytes in length (for device table). * Finally the page size that is to be used by ITS to access * this table will be set to 64KB. * * This gives 0x200000 entries of size 0x8 bytes covered by * 256 pages each of which 64KB in size. The number of pages * (minus 1) should then be written to GITS_BASERn[7:0]. In * that case this value would be 0xFF but on ThunderX the * maximum value that HW accepts is 0xFD. * * Set an arbitrary number of device ID bits to 20 in order * to limit the number of entries in ITS device table to * 0x100000 and the table size to 8MB. */ devbits = 20; cache = 0; } else { devbits = GITS_TYPER_DEVB(gic_its_read_8(sc, GITS_TYPER)); cache = GITS_BASER_CACHE_WAWB; } share = GITS_BASER_SHARE_IS; page_size = PAGE_SIZE_64K; for (i = 0; i < GITS_BASER_NUM; i++) { reg = gic_its_read_8(sc, GITS_BASER(i)); /* The type of table */ type = GITS_BASER_TYPE(reg); /* The table entry size */ esize = GITS_BASER_ESIZE(reg); switch(type) { case GITS_BASER_TYPE_DEV: nidents = (1 << devbits); its_tbl_size = esize * nidents; its_tbl_size = roundup2(its_tbl_size, PAGE_SIZE_64K); break; case GITS_BASER_TYPE_VP: case GITS_BASER_TYPE_PP: /* Undocumented? */ case GITS_BASER_TYPE_IC: its_tbl_size = page_size; break; default: continue; } npages = howmany(its_tbl_size, PAGE_SIZE); /* Allocate the table */ table = (vm_offset_t)contigmalloc(npages * PAGE_SIZE, M_GICV3_ITS, M_WAITOK | M_ZERO, 0, (1ul << 48) - 1, PAGE_SIZE_64K, 0); sc->sc_its_ptab[i].ptab_vaddr = table; sc->sc_its_ptab[i].ptab_size = npages * PAGE_SIZE; paddr = vtophys(table); while (1) { nitspages = howmany(its_tbl_size, page_size); /* Clear the fields we will be setting */ reg &= ~(GITS_BASER_VALID | GITS_BASER_CACHE_MASK | GITS_BASER_TYPE_MASK | GITS_BASER_ESIZE_MASK | GITS_BASER_PA_MASK | GITS_BASER_SHARE_MASK | GITS_BASER_PSZ_MASK | GITS_BASER_SIZE_MASK); /* Set the new values */ reg |= GITS_BASER_VALID | (cache << GITS_BASER_CACHE_SHIFT) | (type << GITS_BASER_TYPE_SHIFT) | ((esize - 1) << GITS_BASER_ESIZE_SHIFT) | paddr | (share << GITS_BASER_SHARE_SHIFT) | (nitspages - 1); switch (page_size) { case PAGE_SIZE: /* 4KB */ reg |= GITS_BASER_PSZ_4K << GITS_BASER_PSZ_SHIFT; break; case PAGE_SIZE_16K: /* 16KB */ reg |= GITS_BASER_PSZ_16K << GITS_BASER_PSZ_SHIFT; break; case PAGE_SIZE_64K: /* 64KB */ reg |= GITS_BASER_PSZ_64K << GITS_BASER_PSZ_SHIFT; break; } gic_its_write_8(sc, GITS_BASER(i), reg); /* Read back to check */ tmp = gic_its_read_8(sc, GITS_BASER(i)); /* Do the shareability masks line up? */ if ((tmp & GITS_BASER_SHARE_MASK) != (reg & GITS_BASER_SHARE_MASK)) { share = (tmp & GITS_BASER_SHARE_MASK) >> GITS_BASER_SHARE_SHIFT; continue; } if ((tmp & GITS_BASER_PSZ_MASK) != (reg & GITS_BASER_PSZ_MASK)) { switch (page_size) { case PAGE_SIZE_16K: page_size = PAGE_SIZE; continue; case PAGE_SIZE_64K: page_size = PAGE_SIZE_16K; continue; } } if (tmp != reg) { device_printf(dev, "GITS_BASER%d: " "unable to be updated: %lx != %lx\n", i, reg, tmp); return (ENXIO); } /* We should have made all needed changes */ break; } } return (0); } static void gicv3_its_conftable_init(struct gicv3_its_softc *sc) { void *conf_table; - conf_table = (void *)atomic_load_ptr((uintptr_t *)&conf_base); + conf_table = atomic_load_ptr(&conf_base); if (conf_table == NULL) { conf_table = contigmalloc(LPI_CONFTAB_SIZE, M_GICV3_ITS, M_WAITOK, 0, LPI_CONFTAB_MAX_ADDR, LPI_CONFTAB_ALIGN, 0); if (atomic_cmpset_ptr((uintptr_t *)&conf_base, (uintptr_t)NULL, (uintptr_t)conf_table) == 0) { contigfree(conf_table, LPI_CONFTAB_SIZE, M_GICV3_ITS); - conf_table = - (void *)atomic_load_ptr((uintptr_t *)&conf_base); + conf_table = atomic_load_ptr(&conf_base); } } sc->sc_conf_base = conf_table; /* Set the default configuration */ memset(sc->sc_conf_base, GIC_PRIORITY_MAX | LPI_CONF_GROUP1, LPI_CONFTAB_SIZE); /* Flush the table to memory */ cpu_dcache_wb_range((vm_offset_t)sc->sc_conf_base, LPI_CONFTAB_SIZE); } static void gicv3_its_pendtables_init(struct gicv3_its_softc *sc) { int i; for (i = 0; i <= mp_maxid; i++) { if (CPU_ISSET(i, &sc->sc_cpus) == 0) continue; sc->sc_pend_base[i] = (vm_offset_t)contigmalloc( LPI_PENDTAB_SIZE, M_GICV3_ITS, M_WAITOK | M_ZERO, 0, LPI_PENDTAB_MAX_ADDR, LPI_PENDTAB_ALIGN, 0); /* Flush so the ITS can see the memory */ cpu_dcache_wb_range((vm_offset_t)sc->sc_pend_base[i], LPI_PENDTAB_SIZE); } } static void its_init_cpu_lpi(device_t dev, struct gicv3_its_softc *sc) { device_t gicv3; uint64_t xbaser, tmp; uint32_t ctlr; u_int cpuid; gicv3 = device_get_parent(dev); cpuid = PCPU_GET(cpuid); /* Disable LPIs */ ctlr = gic_r_read_4(gicv3, GICR_CTLR); ctlr &= ~GICR_CTLR_LPI_ENABLE; gic_r_write_4(gicv3, GICR_CTLR, ctlr); /* Make sure changes are observable my the GIC */ dsb(sy); /* * Set the redistributor base */ xbaser = vtophys(sc->sc_conf_base) | (GICR_PROPBASER_SHARE_IS << GICR_PROPBASER_SHARE_SHIFT) | (GICR_PROPBASER_CACHE_NIWAWB << GICR_PROPBASER_CACHE_SHIFT) | (flsl(LPI_CONFTAB_SIZE | GIC_FIRST_LPI) - 1); gic_r_write_8(gicv3, GICR_PROPBASER, xbaser); /* Check the cache attributes we set */ tmp = gic_r_read_8(gicv3, GICR_PROPBASER); if ((tmp & GICR_PROPBASER_SHARE_MASK) != (xbaser & GICR_PROPBASER_SHARE_MASK)) { if ((tmp & GICR_PROPBASER_SHARE_MASK) == (GICR_PROPBASER_SHARE_NS << GICR_PROPBASER_SHARE_SHIFT)) { /* We need to mark as non-cacheable */ xbaser &= ~(GICR_PROPBASER_SHARE_MASK | GICR_PROPBASER_CACHE_MASK); /* Non-cacheable */ xbaser |= GICR_PROPBASER_CACHE_NIN << GICR_PROPBASER_CACHE_SHIFT; /* Non-sareable */ xbaser |= GICR_PROPBASER_SHARE_NS << GICR_PROPBASER_SHARE_SHIFT; gic_r_write_8(gicv3, GICR_PROPBASER, xbaser); } sc->sc_its_flags |= ITS_FLAGS_LPI_CONF_FLUSH; } /* * Set the LPI pending table base */ xbaser = vtophys(sc->sc_pend_base[cpuid]) | (GICR_PENDBASER_CACHE_NIWAWB << GICR_PENDBASER_CACHE_SHIFT) | (GICR_PENDBASER_SHARE_IS << GICR_PENDBASER_SHARE_SHIFT); gic_r_write_8(gicv3, GICR_PENDBASER, xbaser); tmp = gic_r_read_8(gicv3, GICR_PENDBASER); if ((tmp & GICR_PENDBASER_SHARE_MASK) == (GICR_PENDBASER_SHARE_NS << GICR_PENDBASER_SHARE_SHIFT)) { /* Clear the cahce and shareability bits */ xbaser &= ~(GICR_PENDBASER_CACHE_MASK | GICR_PENDBASER_SHARE_MASK); /* Mark as non-shareable */ xbaser |= GICR_PENDBASER_SHARE_NS << GICR_PENDBASER_SHARE_SHIFT; /* And non-cacheable */ xbaser |= GICR_PENDBASER_CACHE_NIN << GICR_PENDBASER_CACHE_SHIFT; } /* Enable LPIs */ ctlr = gic_r_read_4(gicv3, GICR_CTLR); ctlr |= GICR_CTLR_LPI_ENABLE; gic_r_write_4(gicv3, GICR_CTLR, ctlr); /* Make sure the GIC has seen everything */ dsb(sy); } static int its_init_cpu(device_t dev, struct gicv3_its_softc *sc) { device_t gicv3; vm_paddr_t target; u_int cpuid; struct redist_pcpu *rpcpu; gicv3 = device_get_parent(dev); cpuid = PCPU_GET(cpuid); if (!CPU_ISSET(cpuid, &sc->sc_cpus)) return (0); /* Check if the ITS is enabled on this CPU */ if ((gic_r_read_4(gicv3, GICR_TYPER) & GICR_TYPER_PLPIS) == 0) return (ENXIO); rpcpu = gicv3_get_redist(dev); /* Do per-cpu LPI init once */ if (!rpcpu->lpi_enabled) { its_init_cpu_lpi(dev, sc); rpcpu->lpi_enabled = true; } if ((gic_its_read_8(sc, GITS_TYPER) & GITS_TYPER_PTA) != 0) { /* This ITS wants the redistributor physical address */ target = vtophys(rman_get_virtual(&rpcpu->res)); } else { /* This ITS wants the unique processor number */ target = GICR_TYPER_CPUNUM(gic_r_read_8(gicv3, GICR_TYPER)) << CMD_TARGET_SHIFT; } sc->sc_its_cols[cpuid]->col_target = target; sc->sc_its_cols[cpuid]->col_id = cpuid; its_cmd_mapc(dev, sc->sc_its_cols[cpuid], 1); its_cmd_invall(dev, sc->sc_its_cols[cpuid]); return (0); } static int gicv3_its_attach(device_t dev) { struct gicv3_its_softc *sc; const char *name; uint32_t iidr; int domain, err, i, rid; sc = device_get_softc(dev); sc->sc_irq_length = gicv3_get_nirqs(dev); sc->sc_irq_base = GIC_FIRST_LPI; sc->sc_irq_base += device_get_unit(dev) * sc->sc_irq_length; rid = 0; sc->sc_its_res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, RF_ACTIVE); if (sc->sc_its_res == NULL) { device_printf(dev, "Could not allocate memory\n"); return (ENXIO); } iidr = gic_its_read_4(sc, GITS_IIDR); for (i = 0; i < nitems(its_quirks); i++) { if ((iidr & its_quirks[i].iidr_mask) == its_quirks[i].iidr) { if (bootverbose) { device_printf(dev, "Applying %s\n", its_quirks[i].desc); } its_quirks[i].func(dev); break; } } /* Allocate the private tables */ err = gicv3_its_table_init(dev, sc); if (err != 0) return (err); /* Protects access to the device list */ mtx_init(&sc->sc_its_dev_lock, "ITS device lock", NULL, MTX_SPIN); /* Protects access to the ITS command circular buffer. */ mtx_init(&sc->sc_its_cmd_lock, "ITS cmd lock", NULL, MTX_SPIN); CPU_ZERO(&sc->sc_cpus); if (bus_get_domain(dev, &domain) == 0) { if (domain < MAXMEMDOM) CPU_COPY(&cpuset_domain[domain], &sc->sc_cpus); } else { CPU_COPY(&all_cpus, &sc->sc_cpus); } /* Allocate the command circular buffer */ gicv3_its_cmdq_init(sc); /* Allocate the per-CPU collections */ for (int cpu = 0; cpu <= mp_maxid; cpu++) if (CPU_ISSET(cpu, &sc->sc_cpus) != 0) sc->sc_its_cols[cpu] = malloc( sizeof(*sc->sc_its_cols[0]), M_GICV3_ITS, M_WAITOK | M_ZERO); else sc->sc_its_cols[cpu] = NULL; /* Enable the ITS */ gic_its_write_4(sc, GITS_CTLR, gic_its_read_4(sc, GITS_CTLR) | GITS_CTLR_EN); /* Create the LPI configuration table */ gicv3_its_conftable_init(sc); /* And the pending tebles */ gicv3_its_pendtables_init(sc); /* Enable LPIs on this CPU */ its_init_cpu(dev, sc); TAILQ_INIT(&sc->sc_its_dev_list); /* * Create the vmem object to allocate INTRNG IRQs from. We try to * use all IRQs not already used by the GICv3. * XXX: This assumes there are no other interrupt controllers in the * system. */ sc->sc_irq_alloc = vmem_create(device_get_nameunit(dev), 0, gicv3_get_nirqs(dev), 1, 0, M_FIRSTFIT | M_WAITOK); sc->sc_irqs = malloc(sizeof(*sc->sc_irqs) * sc->sc_irq_length, M_GICV3_ITS, M_WAITOK | M_ZERO); name = device_get_nameunit(dev); for (i = 0; i < sc->sc_irq_length; i++) { sc->sc_irqs[i].gi_id = -1; sc->sc_irqs[i].gi_lpi = i + sc->sc_irq_base - GIC_FIRST_LPI; err = intr_isrc_register(&sc->sc_irqs[i].gi_isrc, dev, 0, "%s,%u", name, i); } return (0); } static int gicv3_its_detach(device_t dev) { return (ENXIO); } static void its_quirk_cavium_22375(device_t dev) { struct gicv3_its_softc *sc; sc = device_get_softc(dev); sc->sc_its_flags |= ITS_FLAGS_ERRATA_CAVIUM_22375; } static void gicv3_its_disable_intr(device_t dev, struct intr_irqsrc *isrc) { struct gicv3_its_softc *sc; struct gicv3_its_irqsrc *girq; uint8_t *conf; sc = device_get_softc(dev); girq = (struct gicv3_its_irqsrc *)isrc; conf = sc->sc_conf_base; conf[girq->gi_lpi] &= ~LPI_CONF_ENABLE; if ((sc->sc_its_flags & ITS_FLAGS_LPI_CONF_FLUSH) != 0) { /* Clean D-cache under command. */ cpu_dcache_wb_range((vm_offset_t)&conf[girq->gi_lpi], 1); } else { /* DSB inner shareable, store */ dsb(ishst); } its_cmd_inv(dev, girq->gi_its_dev, girq); } static void gicv3_its_enable_intr(device_t dev, struct intr_irqsrc *isrc) { struct gicv3_its_softc *sc; struct gicv3_its_irqsrc *girq; uint8_t *conf; sc = device_get_softc(dev); girq = (struct gicv3_its_irqsrc *)isrc; conf = sc->sc_conf_base; conf[girq->gi_lpi] |= LPI_CONF_ENABLE; if ((sc->sc_its_flags & ITS_FLAGS_LPI_CONF_FLUSH) != 0) { /* Clean D-cache under command. */ cpu_dcache_wb_range((vm_offset_t)&conf[girq->gi_lpi], 1); } else { /* DSB inner shareable, store */ dsb(ishst); } its_cmd_inv(dev, girq->gi_its_dev, girq); } static int gicv3_its_intr(void *arg, uintptr_t irq) { struct gicv3_its_softc *sc = arg; struct gicv3_its_irqsrc *girq; struct trapframe *tf; irq -= sc->sc_irq_base; girq = &sc->sc_irqs[irq]; if (girq == NULL) panic("gicv3_its_intr: Invalid interrupt %ld", irq + sc->sc_irq_base); tf = curthread->td_intr_frame; intr_isrc_dispatch(&girq->gi_isrc, tf); return (FILTER_HANDLED); } static void gicv3_its_pre_ithread(device_t dev, struct intr_irqsrc *isrc) { struct gicv3_its_irqsrc *girq; struct gicv3_its_softc *sc; sc = device_get_softc(dev); girq = (struct gicv3_its_irqsrc *)isrc; gicv3_its_disable_intr(dev, isrc); gic_icc_write(EOIR1, girq->gi_lpi + GIC_FIRST_LPI); } static void gicv3_its_post_ithread(device_t dev, struct intr_irqsrc *isrc) { gicv3_its_enable_intr(dev, isrc); } static void gicv3_its_post_filter(device_t dev, struct intr_irqsrc *isrc) { struct gicv3_its_irqsrc *girq; struct gicv3_its_softc *sc; sc = device_get_softc(dev); girq = (struct gicv3_its_irqsrc *)isrc; gic_icc_write(EOIR1, girq->gi_lpi + GIC_FIRST_LPI); } static int gicv3_its_select_cpu(device_t dev, struct intr_irqsrc *isrc) { struct gicv3_its_softc *sc; sc = device_get_softc(dev); if (CPU_EMPTY(&isrc->isrc_cpu)) { sc->gic_irq_cpu = intr_irq_next_cpu(sc->gic_irq_cpu, &sc->sc_cpus); CPU_SETOF(sc->gic_irq_cpu, &isrc->isrc_cpu); } return (0); } static int gicv3_its_bind_intr(device_t dev, struct intr_irqsrc *isrc) { struct gicv3_its_irqsrc *girq; gicv3_its_select_cpu(dev, isrc); girq = (struct gicv3_its_irqsrc *)isrc; its_cmd_movi(dev, girq); return (0); } static int gicv3_its_map_intr(device_t dev, struct intr_map_data *data, struct intr_irqsrc **isrcp) { /* * This should never happen, we only call this function to map * interrupts found before the controller driver is ready. */ panic("gicv3_its_map_intr: Unable to map a MSI interrupt"); } static int gicv3_its_setup_intr(device_t dev, struct intr_irqsrc *isrc, struct resource *res, struct intr_map_data *data) { /* Bind the interrupt to a CPU */ gicv3_its_bind_intr(dev, isrc); return (0); } #ifdef SMP static void gicv3_its_init_secondary(device_t dev) { struct gicv3_its_softc *sc; sc = device_get_softc(dev); /* * This is fatal as otherwise we may bind interrupts to this CPU. * We need a way to tell the interrupt framework to only bind to a * subset of given CPUs when it performs the shuffle. */ if (its_init_cpu(dev, sc) != 0) panic("gicv3_its_init_secondary: No usable ITS on CPU%d", PCPU_GET(cpuid)); } #endif static uint32_t its_get_devid(device_t pci_dev) { uintptr_t id; if (pci_get_id(pci_dev, PCI_ID_MSI, &id) != 0) panic("its_get_devid: Unable to get the MSI DeviceID"); return (id); } static struct its_dev * its_device_find(device_t dev, device_t child) { struct gicv3_its_softc *sc; struct its_dev *its_dev = NULL; sc = device_get_softc(dev); mtx_lock_spin(&sc->sc_its_dev_lock); TAILQ_FOREACH(its_dev, &sc->sc_its_dev_list, entry) { if (its_dev->pci_dev == child) break; } mtx_unlock_spin(&sc->sc_its_dev_lock); return (its_dev); } static struct its_dev * its_device_get(device_t dev, device_t child, u_int nvecs) { struct gicv3_its_softc *sc; struct its_dev *its_dev; vmem_addr_t irq_base; size_t esize; sc = device_get_softc(dev); its_dev = its_device_find(dev, child); if (its_dev != NULL) return (its_dev); its_dev = malloc(sizeof(*its_dev), M_GICV3_ITS, M_NOWAIT | M_ZERO); if (its_dev == NULL) return (NULL); its_dev->pci_dev = child; its_dev->devid = its_get_devid(child); its_dev->lpis.lpi_busy = 0; its_dev->lpis.lpi_num = nvecs; its_dev->lpis.lpi_free = nvecs; if (vmem_alloc(sc->sc_irq_alloc, nvecs, M_FIRSTFIT | M_NOWAIT, &irq_base) != 0) { free(its_dev, M_GICV3_ITS); return (NULL); } its_dev->lpis.lpi_base = irq_base; /* Get ITT entry size */ esize = GITS_TYPER_ITTES(gic_its_read_8(sc, GITS_TYPER)); /* * Allocate ITT for this device. * PA has to be 256 B aligned. At least two entries for device. */ its_dev->itt_size = roundup2(MAX(nvecs, 2) * esize, 256); its_dev->itt = (vm_offset_t)contigmalloc(its_dev->itt_size, M_GICV3_ITS, M_NOWAIT | M_ZERO, 0, LPI_INT_TRANS_TAB_MAX_ADDR, LPI_INT_TRANS_TAB_ALIGN, 0); if (its_dev->itt == 0) { vmem_free(sc->sc_irq_alloc, its_dev->lpis.lpi_base, nvecs); free(its_dev, M_GICV3_ITS); return (NULL); } mtx_lock_spin(&sc->sc_its_dev_lock); TAILQ_INSERT_TAIL(&sc->sc_its_dev_list, its_dev, entry); mtx_unlock_spin(&sc->sc_its_dev_lock); /* Map device to its ITT */ its_cmd_mapd(dev, its_dev, 1); return (its_dev); } static void its_device_release(device_t dev, struct its_dev *its_dev) { struct gicv3_its_softc *sc; KASSERT(its_dev->lpis.lpi_busy == 0, ("its_device_release: Trying to release an inuse ITS device")); /* Unmap device in ITS */ its_cmd_mapd(dev, its_dev, 0); sc = device_get_softc(dev); /* Remove the device from the list of devices */ mtx_lock_spin(&sc->sc_its_dev_lock); TAILQ_REMOVE(&sc->sc_its_dev_list, its_dev, entry); mtx_unlock_spin(&sc->sc_its_dev_lock); /* Free ITT */ KASSERT(its_dev->itt != 0, ("Invalid ITT in valid ITS device")); contigfree((void *)its_dev->itt, its_dev->itt_size, M_GICV3_ITS); /* Free the IRQ allocation */ vmem_free(sc->sc_irq_alloc, its_dev->lpis.lpi_base, its_dev->lpis.lpi_num); free(its_dev, M_GICV3_ITS); } static int gicv3_its_alloc_msi(device_t dev, device_t child, int count, int maxcount, device_t *pic, struct intr_irqsrc **srcs) { struct gicv3_its_softc *sc; struct gicv3_its_irqsrc *girq; struct its_dev *its_dev; u_int irq; int i; its_dev = its_device_get(dev, child, count); if (its_dev == NULL) return (ENXIO); KASSERT(its_dev->lpis.lpi_free >= count, ("gicv3_its_alloc_msi: No free LPIs")); sc = device_get_softc(dev); irq = its_dev->lpis.lpi_base + its_dev->lpis.lpi_num - its_dev->lpis.lpi_free; for (i = 0; i < count; i++, irq++) { its_dev->lpis.lpi_free--; girq = &sc->sc_irqs[irq]; girq->gi_id = i; girq->gi_its_dev = its_dev; srcs[i] = (struct intr_irqsrc *)girq; /* Map the message to the given IRQ */ gicv3_its_select_cpu(dev, (struct intr_irqsrc *)girq); its_cmd_mapti(dev, girq); } its_dev->lpis.lpi_busy += count; *pic = dev; return (0); } static int gicv3_its_release_msi(device_t dev, device_t child, int count, struct intr_irqsrc **isrc) { struct gicv3_its_irqsrc *girq; struct its_dev *its_dev; int i; its_dev = its_device_find(dev, child); KASSERT(its_dev != NULL, ("gicv3_its_release_msi: Releasing a MSI interrupt with " "no ITS device")); KASSERT(its_dev->lpis.lpi_busy >= count, ("gicv3_its_release_msi: Releasing more interrupts than " "were allocated: releasing %d, allocated %d", count, its_dev->lpis.lpi_busy)); for (i = 0; i < count; i++) { girq = (struct gicv3_its_irqsrc *)isrc[i]; girq->gi_id = -1; girq->gi_its_dev = NULL; } its_dev->lpis.lpi_busy -= count; if (its_dev->lpis.lpi_busy == 0) its_device_release(dev, its_dev); return (0); } static int gicv3_its_alloc_msix(device_t dev, device_t child, device_t *pic, struct intr_irqsrc **isrcp) { struct gicv3_its_softc *sc; struct gicv3_its_irqsrc *girq; struct its_dev *its_dev; u_int nvecs, irq; nvecs = pci_msix_count(child); its_dev = its_device_get(dev, child, nvecs); if (its_dev == NULL) return (ENXIO); KASSERT(its_dev->lpis.lpi_free > 0, ("gicv3_its_alloc_msix: No free LPIs")); sc = device_get_softc(dev); irq = its_dev->lpis.lpi_base + its_dev->lpis.lpi_num - its_dev->lpis.lpi_free; girq = &sc->sc_irqs[irq]; girq->gi_id = its_dev->lpis.lpi_busy; girq->gi_its_dev = its_dev; its_dev->lpis.lpi_free--; its_dev->lpis.lpi_busy++; /* Map the message to the given IRQ */ gicv3_its_select_cpu(dev, (struct intr_irqsrc *)girq); its_cmd_mapti(dev, girq); *pic = dev; *isrcp = (struct intr_irqsrc *)girq; return (0); } static int gicv3_its_release_msix(device_t dev, device_t child, struct intr_irqsrc *isrc) { struct gicv3_its_irqsrc *girq; struct its_dev *its_dev; its_dev = its_device_find(dev, child); KASSERT(its_dev != NULL, ("gicv3_its_release_msix: Releasing a MSI-X interrupt with " "no ITS device")); KASSERT(its_dev->lpis.lpi_busy > 0, ("gicv3_its_release_msix: Releasing more interrupts than " "were allocated: allocated %d", its_dev->lpis.lpi_busy)); girq = (struct gicv3_its_irqsrc *)isrc; girq->gi_its_dev = NULL; girq->gi_id = -1; its_dev->lpis.lpi_busy--; if (its_dev->lpis.lpi_busy == 0) its_device_release(dev, its_dev); return (0); } static int gicv3_its_map_msi(device_t dev, device_t child, struct intr_irqsrc *isrc, uint64_t *addr, uint32_t *data) { struct gicv3_its_softc *sc; struct gicv3_its_irqsrc *girq; sc = device_get_softc(dev); girq = (struct gicv3_its_irqsrc *)isrc; *addr = vtophys(rman_get_virtual(sc->sc_its_res)) + GITS_TRANSLATER; *data = girq->gi_id; return (0); } /* * Commands handling. */ static __inline void cmd_format_command(struct its_cmd *cmd, uint8_t cmd_type) { /* Command field: DW0 [7:0] */ cmd->cmd_dword[0] &= htole64(~CMD_COMMAND_MASK); cmd->cmd_dword[0] |= htole64(cmd_type); } static __inline void cmd_format_devid(struct its_cmd *cmd, uint32_t devid) { /* Device ID field: DW0 [63:32] */ cmd->cmd_dword[0] &= htole64(~CMD_DEVID_MASK); cmd->cmd_dword[0] |= htole64((uint64_t)devid << CMD_DEVID_SHIFT); } static __inline void cmd_format_size(struct its_cmd *cmd, uint16_t size) { /* Size field: DW1 [4:0] */ cmd->cmd_dword[1] &= htole64(~CMD_SIZE_MASK); cmd->cmd_dword[1] |= htole64((size & CMD_SIZE_MASK)); } static __inline void cmd_format_id(struct its_cmd *cmd, uint32_t id) { /* ID field: DW1 [31:0] */ cmd->cmd_dword[1] &= htole64(~CMD_ID_MASK); cmd->cmd_dword[1] |= htole64(id); } static __inline void cmd_format_pid(struct its_cmd *cmd, uint32_t pid) { /* Physical ID field: DW1 [63:32] */ cmd->cmd_dword[1] &= htole64(~CMD_PID_MASK); cmd->cmd_dword[1] |= htole64((uint64_t)pid << CMD_PID_SHIFT); } static __inline void cmd_format_col(struct its_cmd *cmd, uint16_t col_id) { /* Collection field: DW2 [16:0] */ cmd->cmd_dword[2] &= htole64(~CMD_COL_MASK); cmd->cmd_dword[2] |= htole64(col_id); } static __inline void cmd_format_target(struct its_cmd *cmd, uint64_t target) { /* Target Address field: DW2 [47:16] */ cmd->cmd_dword[2] &= htole64(~CMD_TARGET_MASK); cmd->cmd_dword[2] |= htole64(target & CMD_TARGET_MASK); } static __inline void cmd_format_itt(struct its_cmd *cmd, uint64_t itt) { /* ITT Address field: DW2 [47:8] */ cmd->cmd_dword[2] &= htole64(~CMD_ITT_MASK); cmd->cmd_dword[2] |= htole64(itt & CMD_ITT_MASK); } static __inline void cmd_format_valid(struct its_cmd *cmd, uint8_t valid) { /* Valid field: DW2 [63] */ cmd->cmd_dword[2] &= htole64(~CMD_VALID_MASK); cmd->cmd_dword[2] |= htole64((uint64_t)valid << CMD_VALID_SHIFT); } static inline bool its_cmd_queue_full(struct gicv3_its_softc *sc) { size_t read_idx, next_write_idx; /* Get the index of the next command */ next_write_idx = (sc->sc_its_cmd_next_idx + 1) % (ITS_CMDQ_SIZE / sizeof(struct its_cmd)); /* And the index of the current command being read */ read_idx = gic_its_read_4(sc, GITS_CREADR) / sizeof(struct its_cmd); /* * The queue is full when the write offset points * at the command before the current read offset. */ return (next_write_idx == read_idx); } static inline void its_cmd_sync(struct gicv3_its_softc *sc, struct its_cmd *cmd) { if ((sc->sc_its_flags & ITS_FLAGS_CMDQ_FLUSH) != 0) { /* Clean D-cache under command. */ cpu_dcache_wb_range((vm_offset_t)cmd, sizeof(*cmd)); } else { /* DSB inner shareable, store */ dsb(ishst); } } static inline uint64_t its_cmd_cwriter_offset(struct gicv3_its_softc *sc, struct its_cmd *cmd) { uint64_t off; off = (cmd - sc->sc_its_cmd_base) * sizeof(*cmd); return (off); } static void its_cmd_wait_completion(device_t dev, struct its_cmd *cmd_first, struct its_cmd *cmd_last) { struct gicv3_its_softc *sc; uint64_t first, last, read; size_t us_left; sc = device_get_softc(dev); /* * XXX ARM64TODO: This is obviously a significant delay. * The reason for that is that currently the time frames for * the command to complete are not known. */ us_left = 1000000; first = its_cmd_cwriter_offset(sc, cmd_first); last = its_cmd_cwriter_offset(sc, cmd_last); for (;;) { read = gic_its_read_8(sc, GITS_CREADR); if (first < last) { if (read < first || read >= last) break; } else if (read < first && read >= last) break; if (us_left-- == 0) { /* This means timeout */ device_printf(dev, "Timeout while waiting for CMD completion.\n"); return; } DELAY(1); } } static struct its_cmd * its_cmd_alloc_locked(device_t dev) { struct gicv3_its_softc *sc; struct its_cmd *cmd; size_t us_left; sc = device_get_softc(dev); /* * XXX ARM64TODO: This is obviously a significant delay. * The reason for that is that currently the time frames for * the command to complete (and therefore free the descriptor) * are not known. */ us_left = 1000000; mtx_assert(&sc->sc_its_cmd_lock, MA_OWNED); while (its_cmd_queue_full(sc)) { if (us_left-- == 0) { /* Timeout while waiting for free command */ device_printf(dev, "Timeout while waiting for free command\n"); return (NULL); } DELAY(1); } cmd = &sc->sc_its_cmd_base[sc->sc_its_cmd_next_idx]; sc->sc_its_cmd_next_idx++; sc->sc_its_cmd_next_idx %= ITS_CMDQ_SIZE / sizeof(struct its_cmd); return (cmd); } static uint64_t its_cmd_prepare(struct its_cmd *cmd, struct its_cmd_desc *desc) { uint64_t target; uint8_t cmd_type; u_int size; cmd_type = desc->cmd_type; target = ITS_TARGET_NONE; switch (cmd_type) { case ITS_CMD_MOVI: /* Move interrupt ID to another collection */ target = desc->cmd_desc_movi.col->col_target; cmd_format_command(cmd, ITS_CMD_MOVI); cmd_format_id(cmd, desc->cmd_desc_movi.id); cmd_format_col(cmd, desc->cmd_desc_movi.col->col_id); cmd_format_devid(cmd, desc->cmd_desc_movi.its_dev->devid); break; case ITS_CMD_SYNC: /* Wait for previous commands completion */ target = desc->cmd_desc_sync.col->col_target; cmd_format_command(cmd, ITS_CMD_SYNC); cmd_format_target(cmd, target); break; case ITS_CMD_MAPD: /* Assign ITT to device */ cmd_format_command(cmd, ITS_CMD_MAPD); cmd_format_itt(cmd, vtophys(desc->cmd_desc_mapd.its_dev->itt)); /* * Size describes number of bits to encode interrupt IDs * supported by the device minus one. * When V (valid) bit is zero, this field should be written * as zero. */ if (desc->cmd_desc_mapd.valid != 0) { size = fls(desc->cmd_desc_mapd.its_dev->lpis.lpi_num); size = MAX(1, size) - 1; } else size = 0; cmd_format_size(cmd, size); cmd_format_devid(cmd, desc->cmd_desc_mapd.its_dev->devid); cmd_format_valid(cmd, desc->cmd_desc_mapd.valid); break; case ITS_CMD_MAPC: /* Map collection to Re-Distributor */ target = desc->cmd_desc_mapc.col->col_target; cmd_format_command(cmd, ITS_CMD_MAPC); cmd_format_col(cmd, desc->cmd_desc_mapc.col->col_id); cmd_format_valid(cmd, desc->cmd_desc_mapc.valid); cmd_format_target(cmd, target); break; case ITS_CMD_MAPTI: target = desc->cmd_desc_mapvi.col->col_target; cmd_format_command(cmd, ITS_CMD_MAPTI); cmd_format_devid(cmd, desc->cmd_desc_mapvi.its_dev->devid); cmd_format_id(cmd, desc->cmd_desc_mapvi.id); cmd_format_pid(cmd, desc->cmd_desc_mapvi.pid); cmd_format_col(cmd, desc->cmd_desc_mapvi.col->col_id); break; case ITS_CMD_MAPI: target = desc->cmd_desc_mapi.col->col_target; cmd_format_command(cmd, ITS_CMD_MAPI); cmd_format_devid(cmd, desc->cmd_desc_mapi.its_dev->devid); cmd_format_id(cmd, desc->cmd_desc_mapi.pid); cmd_format_col(cmd, desc->cmd_desc_mapi.col->col_id); break; case ITS_CMD_INV: target = desc->cmd_desc_inv.col->col_target; cmd_format_command(cmd, ITS_CMD_INV); cmd_format_devid(cmd, desc->cmd_desc_inv.its_dev->devid); cmd_format_id(cmd, desc->cmd_desc_inv.pid); break; case ITS_CMD_INVALL: cmd_format_command(cmd, ITS_CMD_INVALL); cmd_format_col(cmd, desc->cmd_desc_invall.col->col_id); break; default: panic("its_cmd_prepare: Invalid command: %x", cmd_type); } return (target); } static int its_cmd_send(device_t dev, struct its_cmd_desc *desc) { struct gicv3_its_softc *sc; struct its_cmd *cmd, *cmd_sync, *cmd_write; struct its_col col_sync; struct its_cmd_desc desc_sync; uint64_t target, cwriter; sc = device_get_softc(dev); mtx_lock_spin(&sc->sc_its_cmd_lock); cmd = its_cmd_alloc_locked(dev); if (cmd == NULL) { device_printf(dev, "could not allocate ITS command\n"); mtx_unlock_spin(&sc->sc_its_cmd_lock); return (EBUSY); } target = its_cmd_prepare(cmd, desc); its_cmd_sync(sc, cmd); if (target != ITS_TARGET_NONE) { cmd_sync = its_cmd_alloc_locked(dev); if (cmd_sync != NULL) { desc_sync.cmd_type = ITS_CMD_SYNC; col_sync.col_target = target; desc_sync.cmd_desc_sync.col = &col_sync; its_cmd_prepare(cmd_sync, &desc_sync); its_cmd_sync(sc, cmd_sync); } } /* Update GITS_CWRITER */ cwriter = sc->sc_its_cmd_next_idx * sizeof(struct its_cmd); gic_its_write_8(sc, GITS_CWRITER, cwriter); cmd_write = &sc->sc_its_cmd_base[sc->sc_its_cmd_next_idx]; mtx_unlock_spin(&sc->sc_its_cmd_lock); its_cmd_wait_completion(dev, cmd, cmd_write); return (0); } /* Handlers to send commands */ static void its_cmd_movi(device_t dev, struct gicv3_its_irqsrc *girq) { struct gicv3_its_softc *sc; struct its_cmd_desc desc; struct its_col *col; sc = device_get_softc(dev); col = sc->sc_its_cols[CPU_FFS(&girq->gi_isrc.isrc_cpu) - 1]; desc.cmd_type = ITS_CMD_MOVI; desc.cmd_desc_movi.its_dev = girq->gi_its_dev; desc.cmd_desc_movi.col = col; desc.cmd_desc_movi.id = girq->gi_id; its_cmd_send(dev, &desc); } static void its_cmd_mapc(device_t dev, struct its_col *col, uint8_t valid) { struct its_cmd_desc desc; desc.cmd_type = ITS_CMD_MAPC; desc.cmd_desc_mapc.col = col; /* * Valid bit set - map the collection. * Valid bit cleared - unmap the collection. */ desc.cmd_desc_mapc.valid = valid; its_cmd_send(dev, &desc); } static void its_cmd_mapti(device_t dev, struct gicv3_its_irqsrc *girq) { struct gicv3_its_softc *sc; struct its_cmd_desc desc; struct its_col *col; u_int col_id; sc = device_get_softc(dev); col_id = CPU_FFS(&girq->gi_isrc.isrc_cpu) - 1; col = sc->sc_its_cols[col_id]; desc.cmd_type = ITS_CMD_MAPTI; desc.cmd_desc_mapvi.its_dev = girq->gi_its_dev; desc.cmd_desc_mapvi.col = col; /* The EventID sent to the device */ desc.cmd_desc_mapvi.id = girq->gi_id; /* The physical interrupt presented to softeware */ desc.cmd_desc_mapvi.pid = girq->gi_lpi + GIC_FIRST_LPI; its_cmd_send(dev, &desc); } static void its_cmd_mapd(device_t dev, struct its_dev *its_dev, uint8_t valid) { struct its_cmd_desc desc; desc.cmd_type = ITS_CMD_MAPD; desc.cmd_desc_mapd.its_dev = its_dev; desc.cmd_desc_mapd.valid = valid; its_cmd_send(dev, &desc); } static void its_cmd_inv(device_t dev, struct its_dev *its_dev, struct gicv3_its_irqsrc *girq) { struct gicv3_its_softc *sc; struct its_cmd_desc desc; struct its_col *col; sc = device_get_softc(dev); col = sc->sc_its_cols[CPU_FFS(&girq->gi_isrc.isrc_cpu) - 1]; desc.cmd_type = ITS_CMD_INV; /* The EventID sent to the device */ desc.cmd_desc_inv.pid = girq->gi_id; desc.cmd_desc_inv.its_dev = its_dev; desc.cmd_desc_inv.col = col; its_cmd_send(dev, &desc); } static void its_cmd_invall(device_t dev, struct its_col *col) { struct its_cmd_desc desc; desc.cmd_type = ITS_CMD_INVALL; desc.cmd_desc_invall.col = col; its_cmd_send(dev, &desc); } #ifdef FDT static device_probe_t gicv3_its_fdt_probe; static device_attach_t gicv3_its_fdt_attach; static device_method_t gicv3_its_fdt_methods[] = { /* Device interface */ DEVMETHOD(device_probe, gicv3_its_fdt_probe), DEVMETHOD(device_attach, gicv3_its_fdt_attach), /* End */ DEVMETHOD_END }; #define its_baseclasses its_fdt_baseclasses DEFINE_CLASS_1(its, gicv3_its_fdt_driver, gicv3_its_fdt_methods, sizeof(struct gicv3_its_softc), gicv3_its_driver); #undef its_baseclasses static devclass_t gicv3_its_fdt_devclass; EARLY_DRIVER_MODULE(its_fdt, gic, gicv3_its_fdt_driver, gicv3_its_fdt_devclass, 0, 0, BUS_PASS_INTERRUPT + BUS_PASS_ORDER_MIDDLE); static int gicv3_its_fdt_probe(device_t dev) { if (!ofw_bus_status_okay(dev)) return (ENXIO); if (!ofw_bus_is_compatible(dev, "arm,gic-v3-its")) return (ENXIO); device_set_desc(dev, "ARM GIC Interrupt Translation Service"); return (BUS_PROBE_DEFAULT); } static int gicv3_its_fdt_attach(device_t dev) { struct gicv3_its_softc *sc; phandle_t xref; int err; sc = device_get_softc(dev); err = gicv3_its_attach(dev); if (err != 0) return (err); /* Register this device as a interrupt controller */ xref = OF_xref_from_node(ofw_bus_get_node(dev)); sc->sc_pic = intr_pic_register(dev, xref); intr_pic_add_handler(device_get_parent(dev), sc->sc_pic, gicv3_its_intr, sc, sc->sc_irq_base, sc->sc_irq_length); /* Register this device to handle MSI interrupts */ intr_msi_register(dev, xref); return (0); } #endif #ifdef DEV_ACPI static device_probe_t gicv3_its_acpi_probe; static device_attach_t gicv3_its_acpi_attach; static device_method_t gicv3_its_acpi_methods[] = { /* Device interface */ DEVMETHOD(device_probe, gicv3_its_acpi_probe), DEVMETHOD(device_attach, gicv3_its_acpi_attach), /* End */ DEVMETHOD_END }; #define its_baseclasses its_acpi_baseclasses DEFINE_CLASS_1(its, gicv3_its_acpi_driver, gicv3_its_acpi_methods, sizeof(struct gicv3_its_softc), gicv3_its_driver); #undef its_baseclasses static devclass_t gicv3_its_acpi_devclass; EARLY_DRIVER_MODULE(its_acpi, gic, gicv3_its_acpi_driver, gicv3_its_acpi_devclass, 0, 0, BUS_PASS_INTERRUPT + BUS_PASS_ORDER_MIDDLE); static int gicv3_its_acpi_probe(device_t dev) { if (gic_get_bus(dev) != GIC_BUS_ACPI) return (EINVAL); if (gic_get_hw_rev(dev) < 3) return (EINVAL); device_set_desc(dev, "ARM GIC Interrupt Translation Service"); return (BUS_PROBE_DEFAULT); } static int gicv3_its_acpi_attach(device_t dev) { struct gicv3_its_softc *sc; struct gic_v3_devinfo *di; int err; sc = device_get_softc(dev); err = gicv3_its_attach(dev); if (err != 0) return (err); di = device_get_ivars(dev); sc->sc_pic = intr_pic_register(dev, di->msi_xref); intr_pic_add_handler(device_get_parent(dev), sc->sc_pic, gicv3_its_intr, sc, sc->sc_irq_base, sc->sc_irq_length); /* Register this device to handle MSI interrupts */ intr_msi_register(dev, di->msi_xref); return (0); } #endif Index: projects/clang1000-import/sys/compat/cloudabi/cloudabi_file.c =================================================================== --- projects/clang1000-import/sys/compat/cloudabi/cloudabi_file.c (revision 357965) +++ projects/clang1000-import/sys/compat/cloudabi/cloudabi_file.c (revision 357966) @@ -1,762 +1,762 @@ /*- * Copyright (c) 2015 Nuxi, https://nuxi.nl/ * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static MALLOC_DEFINE(M_CLOUDABI_PATH, "cloudabipath", "CloudABI pathnames"); /* * Copying pathnames from userspace to kernelspace. * * Unlike most operating systems, CloudABI doesn't use null-terminated * pathname strings. Processes always pass pathnames to the kernel by * providing a base pointer and a length. This has a couple of reasons: * * - It makes it easier to use CloudABI in combination with programming * languages other than C, that may use non-null terminated strings. * - It allows for calling system calls on individual components of the * pathname without modifying the input string. * * The function below copies in pathname strings and null-terminates it. * It also ensure that the string itself does not contain any null * bytes. * * TODO(ed): Add an abstraction to vfs_lookup.c that allows us to pass * in unterminated pathname strings, so we can do away with * the copying. */ static int copyin_path(const char *uaddr, size_t len, char **result) { char *buf; int error; if (len >= PATH_MAX) return (ENAMETOOLONG); buf = malloc(len + 1, M_CLOUDABI_PATH, M_WAITOK); error = copyin(uaddr, buf, len); if (error != 0) { free(buf, M_CLOUDABI_PATH); return (error); } if (memchr(buf, '\0', len) != NULL) { free(buf, M_CLOUDABI_PATH); return (EINVAL); } buf[len] = '\0'; *result = buf; return (0); } static void cloudabi_freestr(char *buf) { free(buf, M_CLOUDABI_PATH); } int cloudabi_sys_file_advise(struct thread *td, struct cloudabi_sys_file_advise_args *uap) { int advice; switch (uap->advice) { case CLOUDABI_ADVICE_DONTNEED: advice = POSIX_FADV_DONTNEED; break; case CLOUDABI_ADVICE_NOREUSE: advice = POSIX_FADV_NOREUSE; break; case CLOUDABI_ADVICE_NORMAL: advice = POSIX_FADV_NORMAL; break; case CLOUDABI_ADVICE_RANDOM: advice = POSIX_FADV_RANDOM; break; case CLOUDABI_ADVICE_SEQUENTIAL: advice = POSIX_FADV_SEQUENTIAL; break; case CLOUDABI_ADVICE_WILLNEED: advice = POSIX_FADV_WILLNEED; break; default: return (EINVAL); } return (kern_posix_fadvise(td, uap->fd, uap->offset, uap->len, advice)); } int cloudabi_sys_file_allocate(struct thread *td, struct cloudabi_sys_file_allocate_args *uap) { return (kern_posix_fallocate(td, uap->fd, uap->offset, uap->len)); } int cloudabi_sys_file_create(struct thread *td, struct cloudabi_sys_file_create_args *uap) { char *path; int error; error = copyin_path(uap->path, uap->path_len, &path); if (error != 0) return (error); /* * CloudABI processes cannot interact with UNIX credentials and * permissions. Depend on the umask that is set prior to * execution to restrict the file permissions. */ switch (uap->type) { case CLOUDABI_FILETYPE_DIRECTORY: error = kern_mkdirat(td, uap->fd, path, UIO_SYSSPACE, 0777); break; default: error = EINVAL; break; } cloudabi_freestr(path); return (error); } int cloudabi_sys_file_link(struct thread *td, struct cloudabi_sys_file_link_args *uap) { char *path1, *path2; int error; error = copyin_path(uap->path1, uap->path1_len, &path1); if (error != 0) return (error); error = copyin_path(uap->path2, uap->path2_len, &path2); if (error != 0) { cloudabi_freestr(path1); return (error); } error = kern_linkat(td, uap->fd1.fd, uap->fd2, path1, path2, UIO_SYSSPACE, (uap->fd1.flags & CLOUDABI_LOOKUP_SYMLINK_FOLLOW) ? FOLLOW : NOFOLLOW); cloudabi_freestr(path1); cloudabi_freestr(path2); return (error); } int cloudabi_sys_file_open(struct thread *td, struct cloudabi_sys_file_open_args *uap) { cloudabi_fdstat_t fds; cap_rights_t rights; struct filecaps fcaps = {}; struct nameidata nd; struct file *fp; struct vnode *vp; char *path; int error, fd, fflags; bool read, write; error = copyin(uap->fds, &fds, sizeof(fds)); if (error != 0) return (error); /* All the requested rights should be set on the descriptor. */ error = cloudabi_convert_rights( fds.fs_rights_base | fds.fs_rights_inheriting, &rights); if (error != 0) return (error); - cap_rights_set(&rights, CAP_LOOKUP); + cap_rights_set_one(&rights, CAP_LOOKUP); /* Convert rights to corresponding access mode. */ read = (fds.fs_rights_base & (CLOUDABI_RIGHT_FD_READ | CLOUDABI_RIGHT_FILE_READDIR | CLOUDABI_RIGHT_MEM_MAP_EXEC)) != 0; write = (fds.fs_rights_base & (CLOUDABI_RIGHT_FD_DATASYNC | CLOUDABI_RIGHT_FD_WRITE | CLOUDABI_RIGHT_FILE_ALLOCATE | CLOUDABI_RIGHT_FILE_STAT_FPUT_SIZE)) != 0; fflags = write ? read ? FREAD | FWRITE : FWRITE : FREAD; /* Convert open flags. */ if ((uap->oflags & CLOUDABI_O_CREAT) != 0) { fflags |= O_CREAT; - cap_rights_set(&rights, CAP_CREATE); + cap_rights_set_one(&rights, CAP_CREATE); } if ((uap->oflags & CLOUDABI_O_DIRECTORY) != 0) fflags |= O_DIRECTORY; if ((uap->oflags & CLOUDABI_O_EXCL) != 0) fflags |= O_EXCL; if ((uap->oflags & CLOUDABI_O_TRUNC) != 0) { fflags |= O_TRUNC; - cap_rights_set(&rights, CAP_FTRUNCATE); + cap_rights_set_one(&rights, CAP_FTRUNCATE); } if ((fds.fs_flags & CLOUDABI_FDFLAG_APPEND) != 0) fflags |= O_APPEND; if ((fds.fs_flags & CLOUDABI_FDFLAG_NONBLOCK) != 0) fflags |= O_NONBLOCK; if ((fds.fs_flags & (CLOUDABI_FDFLAG_SYNC | CLOUDABI_FDFLAG_DSYNC | CLOUDABI_FDFLAG_RSYNC)) != 0) { fflags |= O_SYNC; - cap_rights_set(&rights, CAP_FSYNC); + cap_rights_set_one(&rights, CAP_FSYNC); } if ((uap->dirfd.flags & CLOUDABI_LOOKUP_SYMLINK_FOLLOW) == 0) fflags |= O_NOFOLLOW; if (write && (fflags & (O_APPEND | O_TRUNC)) == 0) - cap_rights_set(&rights, CAP_SEEK); + cap_rights_set_one(&rights, CAP_SEEK); /* Allocate new file descriptor. */ error = falloc_noinstall(td, &fp); if (error != 0) return (error); fp->f_flag = fflags & FMASK; /* Open path. */ error = copyin_path(uap->path, uap->path_len, &path); if (error != 0) { fdrop(fp, td); return (error); } NDINIT_ATRIGHTS(&nd, LOOKUP, FOLLOW, UIO_SYSSPACE, path, uap->dirfd.fd, &rights, td); error = vn_open(&nd, &fflags, 0777 & ~td->td_proc->p_fd->fd_cmask, fp); cloudabi_freestr(path); if (error != 0) { /* Custom operations provided. */ if (error == ENXIO && fp->f_ops != &badfileops) goto success; /* * POSIX compliance: return ELOOP in case openat() is * called on a symbolic link and O_NOFOLLOW is set. */ if (error == EMLINK) error = ELOOP; fdrop(fp, td); return (error); } NDFREE(&nd, NDF_ONLY_PNBUF); filecaps_free(&nd.ni_filecaps); fp->f_vnode = vp = nd.ni_vp; /* Install vnode operations if no custom operations are provided. */ if (fp->f_ops == &badfileops) { fp->f_seqcount = 1; finit(fp, (fflags & FMASK) | (fp->f_flag & FHASLOCK), DTYPE_VNODE, vp, &vnops); } VOP_UNLOCK(vp); /* Truncate file. */ if (fflags & O_TRUNC) { error = fo_truncate(fp, 0, td->td_ucred, td); if (error != 0) { fdrop(fp, td); return (error); } } success: /* Determine which Capsicum rights to set on the file descriptor. */ cloudabi_remove_conflicting_rights(cloudabi_convert_filetype(fp), &fds.fs_rights_base, &fds.fs_rights_inheriting); cloudabi_convert_rights(fds.fs_rights_base | fds.fs_rights_inheriting, &fcaps.fc_rights); if (cap_rights_is_set(&fcaps.fc_rights)) fcaps.fc_fcntls = CAP_FCNTL_SETFL; error = finstall(td, fp, &fd, fflags, &fcaps); fdrop(fp, td); if (error != 0) return (error); td->td_retval[0] = fd; return (0); } /* Converts a FreeBSD directory entry structure and writes it to userspace. */ static int write_dirent(struct dirent *bde, cloudabi_dircookie_t cookie, struct uio *uio) { cloudabi_dirent_t cde = { .d_next = cookie, .d_ino = bde->d_fileno, .d_namlen = bde->d_namlen, }; size_t len; int error; /* Convert file type. */ switch (bde->d_type) { case DT_BLK: cde.d_type = CLOUDABI_FILETYPE_BLOCK_DEVICE; break; case DT_CHR: cde.d_type = CLOUDABI_FILETYPE_CHARACTER_DEVICE; break; case DT_DIR: cde.d_type = CLOUDABI_FILETYPE_DIRECTORY; break; case DT_FIFO: cde.d_type = CLOUDABI_FILETYPE_SOCKET_STREAM; break; case DT_LNK: cde.d_type = CLOUDABI_FILETYPE_SYMBOLIC_LINK; break; case DT_REG: cde.d_type = CLOUDABI_FILETYPE_REGULAR_FILE; break; case DT_SOCK: /* The exact socket type cannot be derived. */ cde.d_type = CLOUDABI_FILETYPE_SOCKET_STREAM; break; default: cde.d_type = CLOUDABI_FILETYPE_UNKNOWN; break; } /* Write directory entry structure. */ len = sizeof(cde) < uio->uio_resid ? sizeof(cde) : uio->uio_resid; error = uiomove(&cde, len, uio); if (error != 0) return (error); /* Write filename. */ len = bde->d_namlen < uio->uio_resid ? bde->d_namlen : uio->uio_resid; return (uiomove(bde->d_name, len, uio)); } int cloudabi_sys_file_readdir(struct thread *td, struct cloudabi_sys_file_readdir_args *uap) { struct iovec iov = { .iov_base = uap->buf, .iov_len = uap->buf_len }; struct uio uio = { .uio_iov = &iov, .uio_iovcnt = 1, .uio_resid = iov.iov_len, .uio_segflg = UIO_USERSPACE, .uio_rw = UIO_READ, .uio_td = td }; struct file *fp; struct vnode *vp; void *readbuf; cloudabi_dircookie_t offset; int error; /* Obtain directory vnode. */ error = getvnode(td, uap->fd, &cap_read_rights, &fp); if (error != 0) { if (error == EINVAL) return (ENOTDIR); return (error); } if ((fp->f_flag & FREAD) == 0) { fdrop(fp, td); return (EBADF); } /* * Call VOP_READDIR() and convert resulting data until the user * provided buffer is filled. */ readbuf = malloc(MAXBSIZE, M_TEMP, M_WAITOK); offset = uap->cookie; vp = fp->f_vnode; while (uio.uio_resid > 0) { struct iovec readiov = { .iov_base = readbuf, .iov_len = MAXBSIZE }; struct uio readuio = { .uio_iov = &readiov, .uio_iovcnt = 1, .uio_rw = UIO_READ, .uio_segflg = UIO_SYSSPACE, .uio_td = td, .uio_resid = MAXBSIZE, .uio_offset = offset }; struct dirent *bde; unsigned long *cookies, *cookie; size_t readbuflen; int eof, ncookies; /* Validate file type. */ vn_lock(vp, LK_SHARED | LK_RETRY); if (vp->v_type != VDIR) { VOP_UNLOCK(vp); error = ENOTDIR; goto done; } #ifdef MAC error = mac_vnode_check_readdir(td->td_ucred, vp); if (error != 0) { VOP_UNLOCK(vp); goto done; } #endif /* MAC */ /* Read new directory entries. */ cookies = NULL; ncookies = 0; error = VOP_READDIR(vp, &readuio, fp->f_cred, &eof, &ncookies, &cookies); VOP_UNLOCK(vp); if (error != 0) goto done; /* Convert entries to CloudABI's format. */ readbuflen = MAXBSIZE - readuio.uio_resid; bde = readbuf; cookie = cookies; while (readbuflen >= offsetof(struct dirent, d_name) && uio.uio_resid > 0 && ncookies > 0) { /* Ensure that the returned offset always increases. */ if (readbuflen >= bde->d_reclen && bde->d_fileno != 0 && *cookie > offset) { error = write_dirent(bde, *cookie, &uio); if (error != 0) { free(cookies, M_TEMP); goto done; } } if (offset < *cookie) offset = *cookie; ++cookie; --ncookies; readbuflen -= bde->d_reclen; bde = (struct dirent *)((char *)bde + bde->d_reclen); } free(cookies, M_TEMP); if (eof) break; } done: fdrop(fp, td); free(readbuf, M_TEMP); if (error != 0) return (error); /* Return number of bytes copied to userspace. */ td->td_retval[0] = uap->buf_len - uio.uio_resid; return (0); } int cloudabi_sys_file_readlink(struct thread *td, struct cloudabi_sys_file_readlink_args *uap) { char *path; int error; error = copyin_path(uap->path, uap->path_len, &path); if (error != 0) return (error); error = kern_readlinkat(td, uap->fd, path, UIO_SYSSPACE, uap->buf, UIO_USERSPACE, uap->buf_len); cloudabi_freestr(path); return (error); } int cloudabi_sys_file_rename(struct thread *td, struct cloudabi_sys_file_rename_args *uap) { char *old, *new; int error; error = copyin_path(uap->path1, uap->path1_len, &old); if (error != 0) return (error); error = copyin_path(uap->path2, uap->path2_len, &new); if (error != 0) { cloudabi_freestr(old); return (error); } error = kern_renameat(td, uap->fd1, old, uap->fd2, new, UIO_SYSSPACE); cloudabi_freestr(old); cloudabi_freestr(new); return (error); } /* Converts a FreeBSD stat structure to a CloudABI stat structure. */ static void convert_stat(const struct stat *sb, cloudabi_filestat_t *csb) { cloudabi_filestat_t res = { .st_dev = sb->st_dev, .st_ino = sb->st_ino, .st_nlink = sb->st_nlink, .st_size = sb->st_size, }; cloudabi_convert_timespec(&sb->st_atim, &res.st_atim); cloudabi_convert_timespec(&sb->st_mtim, &res.st_mtim); cloudabi_convert_timespec(&sb->st_ctim, &res.st_ctim); *csb = res; } int cloudabi_sys_file_stat_fget(struct thread *td, struct cloudabi_sys_file_stat_fget_args *uap) { struct stat sb; cloudabi_filestat_t csb; struct file *fp; cloudabi_filetype_t filetype; int error; memset(&csb, 0, sizeof(csb)); /* Fetch file descriptor attributes. */ error = fget(td, uap->fd, &cap_fstat_rights, &fp); if (error != 0) return (error); error = fo_stat(fp, &sb, td->td_ucred, td); if (error != 0) { fdrop(fp, td); return (error); } filetype = cloudabi_convert_filetype(fp); fdrop(fp, td); /* Convert attributes to CloudABI's format. */ convert_stat(&sb, &csb); csb.st_filetype = filetype; return (copyout(&csb, uap->buf, sizeof(csb))); } /* Converts timestamps to arguments to futimens() and utimensat(). */ static void convert_utimens_arguments(const cloudabi_filestat_t *fs, cloudabi_fsflags_t flags, struct timespec *ts) { if ((flags & CLOUDABI_FILESTAT_ATIM_NOW) != 0) { ts[0].tv_nsec = UTIME_NOW; } else if ((flags & CLOUDABI_FILESTAT_ATIM) != 0) { ts[0].tv_sec = fs->st_atim / 1000000000; ts[0].tv_nsec = fs->st_atim % 1000000000; } else { ts[0].tv_nsec = UTIME_OMIT; } if ((flags & CLOUDABI_FILESTAT_MTIM_NOW) != 0) { ts[1].tv_nsec = UTIME_NOW; } else if ((flags & CLOUDABI_FILESTAT_MTIM) != 0) { ts[1].tv_sec = fs->st_mtim / 1000000000; ts[1].tv_nsec = fs->st_mtim % 1000000000; } else { ts[1].tv_nsec = UTIME_OMIT; } } int cloudabi_sys_file_stat_fput(struct thread *td, struct cloudabi_sys_file_stat_fput_args *uap) { cloudabi_filestat_t fs; struct timespec ts[2]; int error; error = copyin(uap->buf, &fs, sizeof(fs)); if (error != 0) return (error); /* * Only support truncation and timestamp modification separately * for now, to prevent unnecessary code duplication. */ if ((uap->flags & CLOUDABI_FILESTAT_SIZE) != 0) { /* Call into kern_ftruncate() for file truncation. */ if ((uap->flags & ~CLOUDABI_FILESTAT_SIZE) != 0) return (EINVAL); return (kern_ftruncate(td, uap->fd, fs.st_size)); } else if ((uap->flags & (CLOUDABI_FILESTAT_ATIM | CLOUDABI_FILESTAT_ATIM_NOW | CLOUDABI_FILESTAT_MTIM | CLOUDABI_FILESTAT_MTIM_NOW)) != 0) { /* Call into kern_futimens() for timestamp modification. */ if ((uap->flags & ~(CLOUDABI_FILESTAT_ATIM | CLOUDABI_FILESTAT_ATIM_NOW | CLOUDABI_FILESTAT_MTIM | CLOUDABI_FILESTAT_MTIM_NOW)) != 0) return (EINVAL); convert_utimens_arguments(&fs, uap->flags, ts); return (kern_futimens(td, uap->fd, ts, UIO_SYSSPACE)); } return (EINVAL); } int cloudabi_sys_file_stat_get(struct thread *td, struct cloudabi_sys_file_stat_get_args *uap) { struct stat sb; cloudabi_filestat_t csb; char *path; int error; memset(&csb, 0, sizeof(csb)); error = copyin_path(uap->path, uap->path_len, &path); if (error != 0) return (error); error = kern_statat(td, (uap->fd.flags & CLOUDABI_LOOKUP_SYMLINK_FOLLOW) != 0 ? 0 : AT_SYMLINK_NOFOLLOW, uap->fd.fd, path, UIO_SYSSPACE, &sb, NULL); cloudabi_freestr(path); if (error != 0) return (error); /* Convert results and return them. */ convert_stat(&sb, &csb); if (S_ISBLK(sb.st_mode)) csb.st_filetype = CLOUDABI_FILETYPE_BLOCK_DEVICE; else if (S_ISCHR(sb.st_mode)) csb.st_filetype = CLOUDABI_FILETYPE_CHARACTER_DEVICE; else if (S_ISDIR(sb.st_mode)) csb.st_filetype = CLOUDABI_FILETYPE_DIRECTORY; else if (S_ISFIFO(sb.st_mode)) csb.st_filetype = CLOUDABI_FILETYPE_SOCKET_STREAM; else if (S_ISREG(sb.st_mode)) csb.st_filetype = CLOUDABI_FILETYPE_REGULAR_FILE; else if (S_ISSOCK(sb.st_mode)) { /* Inaccurate, but the best that we can do. */ csb.st_filetype = CLOUDABI_FILETYPE_SOCKET_STREAM; } else if (S_ISLNK(sb.st_mode)) csb.st_filetype = CLOUDABI_FILETYPE_SYMBOLIC_LINK; else csb.st_filetype = CLOUDABI_FILETYPE_UNKNOWN; return (copyout(&csb, uap->buf, sizeof(csb))); } int cloudabi_sys_file_stat_put(struct thread *td, struct cloudabi_sys_file_stat_put_args *uap) { cloudabi_filestat_t fs; struct timespec ts[2]; char *path; int error; /* * Only support timestamp modification for now, as there is no * truncateat(). */ if ((uap->flags & ~(CLOUDABI_FILESTAT_ATIM | CLOUDABI_FILESTAT_ATIM_NOW | CLOUDABI_FILESTAT_MTIM | CLOUDABI_FILESTAT_MTIM_NOW)) != 0) return (EINVAL); error = copyin(uap->buf, &fs, sizeof(fs)); if (error != 0) return (error); error = copyin_path(uap->path, uap->path_len, &path); if (error != 0) return (error); convert_utimens_arguments(&fs, uap->flags, ts); error = kern_utimensat(td, uap->fd.fd, path, UIO_SYSSPACE, ts, UIO_SYSSPACE, (uap->fd.flags & CLOUDABI_LOOKUP_SYMLINK_FOLLOW) ? 0 : AT_SYMLINK_NOFOLLOW); cloudabi_freestr(path); return (error); } int cloudabi_sys_file_symlink(struct thread *td, struct cloudabi_sys_file_symlink_args *uap) { char *path1, *path2; int error; error = copyin_path(uap->path1, uap->path1_len, &path1); if (error != 0) return (error); error = copyin_path(uap->path2, uap->path2_len, &path2); if (error != 0) { cloudabi_freestr(path1); return (error); } error = kern_symlinkat(td, path1, uap->fd, path2, UIO_SYSSPACE); cloudabi_freestr(path1); cloudabi_freestr(path2); return (error); } int cloudabi_sys_file_unlink(struct thread *td, struct cloudabi_sys_file_unlink_args *uap) { char *path; int error; error = copyin_path(uap->path, uap->path_len, &path); if (error != 0) return (error); if (uap->flags & CLOUDABI_UNLINK_REMOVEDIR) error = kern_frmdirat(td, uap->fd, path, FD_NONE, UIO_SYSSPACE, 0); else error = kern_funlinkat(td, uap->fd, path, FD_NONE, UIO_SYSSPACE, 0, 0); cloudabi_freestr(path); return (error); } Index: projects/clang1000-import/sys/contrib/dev/acpica/changes.txt =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/changes.txt (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/changes.txt (revision 357966) @@ -1,18777 +1,18875 @@ ---------------------------------------- +14 February 2020. Summary of changes for version 20200214: + + +1) ACPICA kernel-resident subsystem: + +Enable sleep button on ACPI legacy wake: Hibernation (S4) is triggered +in a guest when it receives a sleep trigger from the hypervisor. When the +guest resumes from this power state, it does not see the SleepEnabled +bit. In other words, the sleepHibernation (S4) is triggered in a guest +when it receives a sleep trigger from the hypervisor. When the guest +resumes from this power state, it does not see the SleepEnabled bit. In +other words, the sleep button is not enabled on waking from an S4 state. +This causes subsequent invocation of sleep state to fail since the +guest.button is not enabled on waking from an S4 state. This causes +subsequent invocation of sleep state to fail in the guest. Fix this +problem by enabling the sleep button in ACPI legacy wake. From Anchal +Agarwal . + +Implemented a new external interface, AcpiAnyGpeStatusSet (). To be used +for checking the status bits of all enabled GPEs in one go. It is needed +to distinguish spurious SCIs from genuine ones when deciding whether or +not to wake up the system from suspend-to-idle. + +Generic Makefiles: replace HOST name with ACPI_HOST: Some machines may be +using HOST in their environment to represent the host name for their +machines. Avoid this problem by renaming this variable from HOST to +ACPI_HOST. + +MSVC 2017 project files: Enable multiprocessor generation to improve +build performance. + +Added a macro to get the byte width of a Generic Address structure. New +ACPI_ACCESS_BYTE_WIDTH is in addition to the existing +ACPI_ACCESS_BIT_WIDTH. From Mika Westerberg. + + +2) iASL Compiler/Disassembler and ACPICA tools: + +iASL: Implemented full support for the (optional, rarely used) ReturnType +and ParameterTypesList for the Method, Function, and External operators. +For Method declarations, the number of individual ParameterTypes must +match the declaration of the number of arguments (NumArgs). This also +Fixes a problem with the External operator where extra/extraneous bytes +were emitted in the AML code if the optional ReturnType/ParameterTypes +were specified for a MethodObj declaration. +New error message: +1) Method NumArgs count does not match length of ParameterTypes list + +iASL: Implemented detection of type mismatches between External +declarations and named object declarations. Also, detect type mismatches +between multiple External declarations of the same Name. +New error messages: +1) Type mismatch between external declaration and actual object +declaration detected +2) Type mismatch between multiple external declarations detected + +iASL: Implemented new error messages for External operators that specify +a ReturnType and/or ParameterTypesList for any object type other than +control methods (MethodObj). +New error messages: +1) Return type is only allowed for Externals declared as MethodObj +2) Parameter type is only allowed for Externals declared as MethodObj + +iASL: Implemented two new remark/warning messages for ASL code that +creates named objects from within a control method. This is very +inefficient since the named object must be created and deleted each time +the method is executed. +New messages: +1) Creation of named objects within a method is highly inefficient, use +globals or method local variables instead (remark) +2) Static OperationRegion should be declared outside control method +(warning) + +iASL: Improved illegal forward reference detection by adding support to +detect forward-reference method invocations. + +iASL: Detect and issue an error message for NameStrings that contain too +many individual NameSegs (>255). This is an AML limitation that is +defined in the ACPI specification. +New message: +1) NameString contains too many NameSegs (>255) + +acpidump: windows: use GetSystemFirmwareTable API for all tables except +SSDT. By using this API, acpidump is able to get all tables in the XSDT + +iASL: Removed unused parser file and updated msvc2017 project files. +Removed the obsolete AslCompiler.y from the repository. + +iASL: msvc2017: Fixed macros in the file dependency list to prevent +unnecessary rebuilds. Replace %(Directory) with %(RelativeDir). + +Disassembler: Prevent spilling error messages to the output file. All +errors are directed to the console instead. These error messages +prevented re-compilation of the resulting disassembled ASL output file +(.DSL). + + +---------------------------------------- 10 January 2020. Summary of changes for version 20200110: 1) ACPICA kernel-resident subsystem: Updated all copyrights to 2020. This affects all ACPICA source code modules. 2) iASL Compiler/Disassembler and ACPICA tools: ASL test suite (ASLTS): Updated all copyrights to 2020. Tools and utilities: Updated all signon copyrights to 2020. iASL: fix forward reference analysis for field declarations. Fixes forward reference analysis for field declarations by searching the parent scope for the named object when the object is not present in the current scope. iASL: Improved the error output for ALREADY_EXISTS errors. Now, the full pathname of the name that already exists is printed. iASL: Enhance duplicate Case() detection for buffers. Add check for buffers with no initializer list (these buffers will be filled with zeros at runtime.) ---------------------------------------- 13 December 2019. Summary of changes for version 20191213: 1) ACPICA kernel-resident subsystem: Return a Buffer object for all fields created via the CreateField operator. Previously, an Integer would be returned if the size of the field was less than or equal to the current size of an Integer. Although this goes against the ACPI specification, it provides compatibility with other ACPI implementations. Also updated the ASLTS test suite to reflect this new behavior. 2) iASL Compiler/Disassembler and ACPICA tools: iASL: Implemented detection of (and throw an error for) duplicate values for Case statements within a single Switch statement. Duplicate Integers, Strings, and Buffers are supported. iASL: Fix error logging issue during multiple file compilation -- Switch to the correct input file during error node creation. iASL: For duplicate named object creation, now emit an error instead of a warning - since this will cause a runtime error. AcpiSrc: Add unix line-ending support for non-Windows builds. iASL: Add an error condition for an attempt to create a NameString with > 255 NameSegs (the max allowable via the AML definition). ---------------------------------------- 18 October 2019. Summary of changes for version 20191018: 1) ACPICA kernel-resident subsystem: Debugger: added a new command: ?Fields [address space ID]?. This command dumps the contents of all field units that are defined within the namespace with a particular address space ID. Modified the external interface AcpiLoadTable() to return a table index. This table index can be used for unloading a table for debugging. ACPI_STATUS AcpiLoadTable ( ACPI_TABLE_HEADER *Table, UINT32 *TableIndex)) Implemented a new external interface: AcpiUnloadTable() This new function takes a table index as an argument and unloads the table. Useful for debugging only. ACPI_STATUS AcpiUnloadTable ( UINT32 TableIndex)) Ported the AcpiNames utility to use the new table initialization sequence. The utility was broken before this change. Also, it was required to include most of the AML interpreter into the utility in order to process table initialization (module-level code execution.) Update for results from running Clang V8.0.1. This fixes all "dead assignment" warnings. There are still several "Dereference of NULL pointer" warnings, but these have been found to be false positive warnings. 2) iASL Compiler/Disassembler and ACPICA tools: iASL: numerous table compiler changes to ensure that the usage of yacc/bison syntax is POSIX-compliant. iASL/disassembler: several simple bug fixes in the data table disassembler. Acpiexec: expanded the initialization file (the -fi option) to initialize strings, buffers, packages, and field units. ---------------------------------------- 16 August 2019. Summary of changes for version 20190816: This release is available at https://acpica.org/downloads 1) ACPICA kernel-resident subsystem: Modified the OwnerId mechanism to allow for more Owner Ids. The previous limit was 256 Ids, now it is 4096 Ids. This prevents OWNER_ID_LIMIT exceptions on machines with a large number of initialization threads, many CPU cores and nested initialization control methods. Introduced acpi_dispatch_gpe() as a wrapper around AcpiEvDetectGpe() for checking if the given GPE (as represented by a GPE device handle and a GPE number) is currently active and dispatching it (if that's the case) outside of interrupt context. Table load: exit the interpreter before initializing objects within the new table This prevents re-acquiring the interpreter lock when loading tables Added the "Windows 2019" string to the _OSI support (version 1903). Jung- uk Kim Macros: removed pointer math on a null pointer. Causes warnings on some compilers and/or tools. Changed ACPI_TO_POINTER to use ACPI_CAST_PTR instead of using arithmetic. Fully deployed the ACPI_PRINTF_LIKE macro. This macro was not being used across all "printf-like" internal functions. Also, cleanup all calls to such functions (both in 32-bit mode and 64-bit mode) now that they are analyzed by the gcc compiler via ACPI_PRINTF_LIKE. 2) iASL Compiler/Disassembler and ACPICA tools: iASL: implemented a new data table compiler flex/bison front-end. This change is internal and is not intended to result in changes to the compiled code. This new compiler front-end can be invoked using the -tp option for now, until the old mechanism is removed. ASLTS: Implemented a new data table compiler test suite. This test suite generates all table templates and compile/disassemble/re-compile/binary- compare each file. iASL: return -1 if AML files were not generated due to compiler errors iASL: added a warning on use of the now-legacy ASL Processor () keyword. iASL: added an error on _UID object declaration that returns a String within a Processor () declaration. A _UID for a processor must be an Integer. iASL: added a null terminator to name strings that consist only of multiple parent prefixes (^) iASL: added support to compile both ASL and data table files in a single command. Updated the tool generation project files that were recently migrated to MSVC 2017 to eliminate all new warnings. The new project files appear in the directory \acpica\generate\msvc2017. This change effectively deprecates the older project files in \acpica\generate\msvc9. ---------------------------------------- 03 July 2019. Summary of changes for version 20190703: 1) ACPICA kernel-resident subsystem: Remove legacy module-level support code. There were still some remnants of the legacy module-level code executions. Since we no longer support this option, this is essentially dead code and has been removed from the ACPICA source. iASL: ensure that _WAK, _PTS, _TTS, and _Sx are declared only at the root scope. If these named objects are declared outside the root scope, they will not be invoked by any host Operating System. Clear status of GPEs on first direct enable. ACPI GPEs (other than the EC one) can be enabled in two situations. First, the GPEs with existing _Lxx and _Exx methods are enabled implicitly by ACPICA during system initialization. Second, the GPEs without these methods (like GPEs listed by _PRW objects for wakeup devices) need to be enabled directly by the code that is going to use them (e.g. ACPI power management or device drivers). In the former case, if the status of a given GPE is set to start with, its handler method (either _Lxx or _Exx) needs to be invoked to take care of the events (possibly) signaled before the GPE was enabled. In the latter case, however, the first caller of AcpiEnableGpe() for a given GPE should not be expected to care about any events that might be signaled through it earlier. In that case, it is better to clear the status of the GPE before enabling it, to prevent stale events from triggering unwanted actions (like spurious system resume, for example). For this reason, modify AcpiEvAddGpeReference() to take an additional boolean argument indicating whether or not the GPE status needs to be cleared when its reference counter changes from zero to one and make AcpiEnableGpe() pass TRUE to it through that new argument. 2) iASL Compiler/Disassembler and ACPICA tools: The tool generation process has been migrated to MSVC 2017, and all project files have been upgraded. The new project files appear in the directory \acpica\generate\msvc2017. This change effectively deprecates the older project files in \acpica\generate\msvc9. iASL: ensure that _WAK, _PTS, _TTS, and _Sx are declared only at the root scope. If these named objects are declared outside the root scope, they will not be invoked by any host Operating System ---------------------------------------- 09 May 2019. Summary of changes for version 20190509: 1) ACPICA kernel-resident subsystem: Revert commit 6c43e1a ("ACPICA: Clear status of GPEs before enabling them") that causes problems with Thunderbolt controllers to occur if a dock device is connected at init time (the xhci_hcd and thunderbolt modules crash which prevents peripherals connected through them from working). Commit 6c43e1a effectively causes commit ecc1165b8b74 ("ACPICA: Dispatch active GPEs at init time") to get undone, so the problem addressed by commit ecc1165b8b74 appears again as a result of it. 2) iASL Compiler/Disassembler and ACPICA tools: Reverted iASL: Additional forward reference detection. This change reverts forward reference detection for field declarations. The feature unintentionally emitted AML bytecode with incorrect package lengths for some ASL code related to Fields and OperationRegions. This malformed AML can cause systems to crash during boot. The malformed AML bytecode is emitted in iASL version 20190329 and 20190405. iASL: improve forward reference detection. This change improves forward reference detection for named objects inside of scopes. If a parse object has the OP_NOT_FOUND_DURING_LOAD set, it means that Op is a reference to a named object that is declared later in the AML bytecode. This is allowed if the reference is inside of a method and the declaration is outside of a method like so: DefinitionBlock(...) { Method (TEST) { Return (NUM0) } Name (NUM0,0) } However, if the declaration and reference are both in the same method or outside any methods, this is a forward reference and should be marked as an error because it would result in runtime errors. DefinitionBlock(...) { Name (BUFF, Buffer (NUM0) {}) // Forward reference Name (NUM0, 0x0) Method (TEST) { Local0 = NUM1 Name (NUM1, 0x1) // Forward reference return (Local0) } } iASL: Implemented additional buffer overflow analysis for BufferField declarations. Check if a buffer index argument to a create buffer field operation is beyond the end of the target buffer. This affects these AML operators: AML_CREATE_FIELD_OP AML_CREATE_BIT_FIELD_OP AML_CREATE_BYTE_FIELD_OP AML_CREATE_WORD_FIELD_OP AML_CREATE_DWORD_FIELD_OP AML_CREATE_QWORD_FIELD_OP There are three conditions that must be satisfied in order to allow this validation at compile time: 1) The length of the target buffer must be an integer constant 2) The index specified in the create* must be an integer constant 3) For CreateField, the bit length argument must be non-zero. Example: Name (BUF1, Buffer() {1,2}) CreateField (BUF1, 7, 9, CF03) // 3: ERR dsdt.asl 14: CreateField (BUF1, 7, 9, CF03) // 3: ERR Error 6165 - ^ Buffer index beyond end of target buffer ---------------------------------------- 05 April 2019. Summary of changes for version 20190405: 1) ACPICA kernel-resident subsystem: Event Manager: History: Commit 18996f2db918 ("ACPICA: Events: Stop unconditionally clearing ACPI IRQs during suspend/resume") was added earlier to stop clearing of event status bits unconditionally on suspend and resume paths. Though this change fixed an issue on suspend path, it introduced regressions on several resume paths. In the case of S0ix, events are enabled as part of device suspend path. If status bits for the events are set when they are enabled, it could result in premature wake from S0ix. If status is cleared for any event that is being enabled so that any stale events are cleared out. In case of S0ix, events are enabled as part of device suspend path. If status bits for the events are set when they are enabled, it could result in premature wake from S0ix. This change ensures that status is cleared for any event that is being enabled so that any stale events are cleared out. 2) iASL Compiler/Disassembler and ACPICA tools: iASL: Implemented an enhanced multiple file compilation that combines named objects from all input files to a single namespace. With this feature, any unresolved external declarations as well as duplicate named object declarations can be detected during compilation rather than generating errors much later at runtime. The following commands are examples that utilize this feature: iasl dsdt.asl ssdt.asl iasl dsdt.asl ssdt1.asl ssdt2.asl iasl dsdt.asl ssdt*.asl ---------------------------------------- 29 March 2019. Summary of changes for version 20190329: 1) ACPICA kernel-resident subsystem: Namespace support: Remove the address nodes from global list after method termination. The global address list contains pointers to namespace nodes that represent Operation Regions. This change properly removes Operation Region namespace nodes that are declared dynamically during method execution. Linux: Use a different debug default than ACPICA. There was a divergence between Linux and the ACPICA codebases. In order to resolve this divergence, Linux now declares its own debug default in aclinux.h Renamed some internal macros to improve code understanding and maintenance. The macros below all operate on single 4-character ACPI NameSegs, not generic strings (old -> new): ACPI_NAME_SIZE -> ACPI_NAMESEG_SIZE ACPI_COMPARE_NAME -> ACPI_COMPARE_NAMESEG ACPI_MOVE_NAME -> ACPI_COPY_NAMESEG Fix for missing comma in array declaration for the AcpiGbl_GenericNotify table. Test suite: Update makefiles, add PCC operation region support 2) iASL Compiler/Disassembler and Tools: iASL: Implemented additional illegal forward reference detection. Now detect and emit an error upon detection of a forward reference from a Field to an Operation Region. This will fail at runtime if allowed to pass the compiler. AcpiExec: Add an address list check for dynamic Operation Regions. This feature performs a sanity test for each node the global address list. This is done in order to ensure that all dynamic operation regions are properly removed from the global address list and no dangling pointers are left behind. Disassembler: Improved generation of resource pathnames. This change improves the code that generates resource descriptor and resource tag pathnames. The original code used a bunch of str* C library functions that caused warnings on some compilers. iASL: Removed some uses of strncpy and replaced with memmove. The strncpy function can overwrite buffers if the calling code is not very careful. In the case of generating a module/table header, use of memmove is a better implementation. 3) Status of new features that have not been completed at this time: iASL: Implementing an enhanced multiple file compilation into a single namespace feature (Status): This feature will be released soon, and allows multiple ASL files to be compiled into the same single namespace. By doing so, any unresolved external declarations as well as duplicate named object declarations can be detected during compilation (rather than later during runtime). The following commands are examples that utilize this feature: iasl dsdt.asl ssdt.asl iasl dsdt.asl ssdt1.asl ssdt2.asl iasl dsdt.asl ssdt*.asl ASL tutorial status: Feedback is being gathered internally and the current plan is to publish this tutorial on the ACPICA website after a final review by a tech writer. ---------------------------------------- 15 February 2019. Summary of changes for version 20190215: 0) Support for ACPI specification version 6.3: Add PCC operation region support for the AML interpreter. This adds PCC operation region support in the AML interpreter and a default handler for acpiexec. The change also renames the PCC region address space keyword to PlatformCommChannel. Support for new predefined methods _NBS, _NCH, _NIC, _NIH, and _NIG. These methods provide OSPM with health information and device boot status. PDTT: Add TriggerOrder to the PCC Identifier structure. The field value defines if the trigger needs to be invoked by OSPM before or at the end of kernel crash dump processing/handling operation. SRAT: Add Generic Affinity Structure subtable. This subtable in the SRAT is used for describing devices such as heterogeneous processors, accelerators, GPUs, and IO devices with integrated compute or DMA engines. MADT: Add support for statistical profiling in GICC. Statistical profiling extension (SPE) is an architecture-specific feature for ARM. MADT: Add online capable flag. If this bit is set, system hardware supports enabling this processor during OS runtime. New Error Disconnect Recover Notification value. There are a number of scenarios where system Firmware in collaboration with hardware may disconnect one or more devices from the rest of the system for purposes of error containment. Firmware can use this new notification value to alert OSPM of such a removal. PPTT: New additional fields in Processor Structure Flags. These flags provide more information about processor topology. NFIT/Disassembler: Change a field name from "Address Range" to "Region Type". HMAT updates: make several existing fields to be reserved as well as rename subtable 0 to "memory proximity domain attributes". GTDT: Add support for new GTDT Revision 3. This revision adds information for the EL2 timer. iASL: Update the HMAT example template for new fields. iASL: Add support for the new revision of the GTDT (Rev 3). 1) ACPICA kernel-resident subsystem: AML Parser: fix the main AML parse loop to correctly skip erroneous extended opcodes. AML opcodes come in two lengths: 1-byte opcodes and 2- byte extended opcodes. If an error occurs during an AML table load, the AML parser will continue loading the table by skipping the offending opcode. This implements a "load table at any cost" philosophy. 2) iASL Compiler/Disassembler and Tools: iASL: Add checks for illegal object references, such as a reference outside of method to an object within a method. Such an object is only temporary. iASL: Emit error for creation of a zero-length operation region. Such a region is rather pointless. If encountered, a runtime error is also implemented in the interpeter. Debugger: Fix a possible fault with the "test objects" command. iASL: Makefile: support parent directory filenames containing embedded spaces. iASL: Update the TPM2 template to revision 4. iASL: Add the ability to report specific warnings or remarks as errors. Disassembler: Disassemble OEMx tables as actual AML byte code. Previously, these tables were treated as "unknown table". iASL: Add definition and disassembly for TPM2 revision 3. iASL: Add support for TPM2 rev 3 compilation. ---------------------------------------- 08 January 2019. Summary of changes for version 20190108: 1) ACPICA kernel-resident subsystem: Updated all copyrights to 2019. This affects all source code modules. 2) iASL Compiler/Disassembler and Tools: ASL test suite (ASLTS): Updated all copyrights to 2019. Tools: Updated all signon copyrights to 2019. AcpiExec: Added a new option to dump extra information concerning any memory leaks detected by the internal object/cache tracking mechanism. - va iASL: Updated the table template for the TPM2 table to the newest version of the table (Revision 4) ---------------------------------------- 13 December 2018. Summary of changes for version 20181213: 1) ACPICA Kernel-resident Subsystem: Fixed some buffer length issues with the GenericSerialBus, related to two of the bidirectional protocols: AttribRawProcessBytes and AttribRawBytes, which are rarely seen in the field. For these, the LEN field of the ASL buffer is now ignored. Hans de Goede Implemented a new object evaluation trace mechanism for control methods and data objects. This includes nested control methods. It is particularly useful for examining the ACPI execution during system initialization since the output is relatively terse. The flag below enables the output of the trace via the ACPI_DEBUG_PRINT_RAW interface: #define ACPI_LV_EVALUATION 0x00080000 Examples: Enter evaluation : _SB.PCI0._INI (Method) Exit evaluation : _SB.PCI0._INI Enter evaluation : _OSI (Method) Exit evaluation : _OSI Enter evaluation : _SB.PCI0.TEST (Method) Nested method call : _SB.PCI0.NST1 Exit nested method : _SB.PCI0.NST1 Exit evaluation : _SB.PCI0.TEST Added two recently-defined _OSI strings. See https://docs.microsoft.com/en-us/windows-hardware/drivers/acpi/winacpi- osi. "Windows 2018" "Windows 2018.2" Update for buffer-to-string conversions via the ToHexString ASL operator. A "0x" is now prepended to each of the hex values in the output string. This provides compatibility with other ACPI implementations. The ACPI specification is somewhat vague on this issue. Example output string after conversion: "0x01,0x02,0x03,0x04,0x05,0x06" Return a run-time error for TermArg expressions within individual package elements. Although this is technically supported by the ASL grammar, other ACPI implementations do not support this either. Also, this fixes a fault if this type of construct is ever encountered (it never has been). 2) iASL Compiler/Disassembler and Tools: iASL: Implemented a new compile option (-ww) that will promote individual warnings and remarks to errors. This is intended to enhance the firmware build process. AcpiExec: Implemented a new command-line option (-eo) to support the new object evaluation trace mechanism described above. Disassembler: Added support to disassemble OEMx tables as AML/ASL tables instead of a "unknown table" message. AcpiHelp: Improved support for the "special" predefined names such as _Lxx, _Exx, _EJx, _T_x, etc. For these, any legal hex value can now be used for "xx" and "x". ---------------------------------------- 31 October 2018. Summary of changes for version 20181031: An Operation Region regression was fixed by properly adding address ranges to a global list during initialization. This allows OS to accurately check for overlapping regions between native devices (such as PCI) and Operation regions as well as checking for region conflicts between two Operation Regions. Added support for the 2-byte extended opcodes in the code/feature that attempts to continue parsing during the table load phase. Skip parsing Device declarations (and other extended opcodes) when an error occurs during parsing. Previously, only single-byte opcodes were supported. Cleanup: Simplified the module-level code support by eliminating a useless global variable (AcpiGbl_GroupModuleLeveCode). 2) iASL Compiler/Disassembler and Tools: iASL/Preprocessor: Fixed a regression where an incorrect use of ACPI_FREE could cause a fault in the preprocessor. This was an inadvertent side- effect from moving more allocations/frees to the local cache/memory mechanism. iASL: Enhanced error detection by validating that all NameSeg elements within a NamePatch actually exist. The previous behavior was spotty at best, and such errors could be improperly ignored at compiler time (never at runtime, however. There are two new error messages, as shown in the examples below: dsdt.asl 33: CreateByteField (TTTT.BXXX, 1, CBF1) Error 6161 - ^ One or more objects within the Pathname do not exist (TTTT.BXXX) dsdt.asl 34: CreateByteField (BUF1, UUUU.INT1, BBBB.CBF1) Error 6160 - One or more prefix Scopes do not exist ^ (BBBB.CBF1) iASL: Disassembler/table-compiler: Added support for the static data table TPM2 revision 3 (an older version of TPM2). The support has been added for the compiler and the disassembler. Fixed compilation of DOS format data table file on Unix/Linux systems. iASL now properly detects line continuations (\) for DOS format data table definition language files on when executing on Unix/Linux. ---------------------------------------- 03 October 2018. Summary of changes for version 20181003: 2) iASL Compiler/Disassembler and Tools: Fixed a regression introduced in version 20180927 that could cause the compiler to fault, especially with NamePaths containing one or more carats (^). Such as: ^^_SB_PCI0 Added a new remark for the Sleep() operator when the sleep time operand is larger than one second. This is a very long time for the ASL/BIOS code and may not be what was intended by the ASL writer. ---------------------------------------- 27 September 2018. Summary of changes for version 20180927: 1) ACPICA kernel-resident subsystem: Updated the GPE support to clear the status of all ACPI events when entering any/all sleep states in order to avoid premature wakeups. In theory, this may cause some wakeup events to be missed, but the likelihood of this is small. This change restores the original behavior of the ACPICA code in order to fix a regression seen from the previous "Stop unconditionally clearing ACPI IRQs during suspend/resume" change. This regression could cause some systems to incorrectly wake immediately. Updated the execution of the _REG methods during initialization and namespace loading to bring the behavior into closer conformance to the ACPI specification and other ACPI implementations: From the ACPI specification 6.2A, section 6.5.4 "_REG (Region): "Control methods must assume all operation regions are inaccessible until the _REG(RegionSpace, 1) method is executed" "The exceptions to this rule are: 1. OSPM must guarantee that the following operation regions are always accessible: SystemIO operation regions. SystemMemory operation regions when accessing memory returned by the System Address Map reporting interfaces." Since the state of both the SystemIO and SystemMemory address spaces are defined by the specification to never change, this ACPICA change ensures that now _REG is never called on them. This solves some problems seen in the field and provides compatibility with other ACPI implementations. An update to the upcoming new version of the ACPI specification will help clarify this behavior. Updated the implementation of support for the Generic Serial Bus. For the "bidirectional" protocols, the internal implementation now automatically creates a return data buffer of the maximum size (255). This handles the worst-case for data that is returned from the serial bus handler, and fixes some problems seen in the field. This new buffer is directly returned to the ASL. As such, there is no true "bidirectional" buffer, which matches the ACPI specification. This is the reason for the "double store" seen in the example ASL code in the specification, shown below: Word Process Call (AttribProcessCall): OperationRegion(TOP1, GenericSerialBus, 0x00, 0x100) Field(TOP1, BufferAcc, NoLock, Preserve) { FLD1, 8, // Virtual register at command value 1. } Name(BUFF, Buffer(20){}) // Create GenericSerialBus data buffer // as BUFF CreateWordField(BUFF, 0x02, DATA) // DATA = Data (Word) Store(0x5416, DATA) // Save 0x5416 into the data buffer Store(Store(BUFF, FLD1), BUFF) // Invoke a write/read Process Call transaction // This is the "double store". The write to // FLD1 returns a new buffer, which is stored // back into BUFF with the second Store. 2) iASL Compiler/Disassembler and Tools: iASL: Implemented detection of extraneous/redundant uses of the Offset() operator within a Field Unit list. A remark is now issued for these. For example, the first two of the Offset() operators below are extraneous. Because both the compiler and the interpreter track the offsets automatically, these Offsets simply refer to the current offset and are unnecessary. Note, when optimization is enabled, the iASL compiler will in fact remove the redundant Offset operators and will not emit any AML code for them. OperationRegion (OPR1, SystemMemory, 0x100, 0x100) Field (OPR1) { Offset (0), // Never needed FLD1, 32, Offset (4), // Redundant, offset is already 4 (bytes) FLD2, 8, Offset (64), // OK use of Offset. FLD3, 16, } dsdt.asl 14: Offset (0), Remark 2158 - ^ Unnecessary/redundant use of Offset operator dsdt.asl 16: Offset (4), Remark 2158 - ^ Unnecessary/redundant use of Offset operator ---------------------------------------- 10 August 2018. Summary of changes for version 20180810: 1) ACPICA kernel-resident subsystem: Initial ACPI table loading: Attempt to continue loading ACPI tables regardless of malformed AML. Since migrating table initialization to the new module-level code support, the AML interpreter rejected tables upon any ACPI error encountered during table load. This is a problem because non-serious ACPI errors during table load do not necessarily mean that the entire definition block (DSDT or SSDT) is invalid. This change improves the table loading by ignoring some types of errors that can be generated by incorrect AML. This can range from object type errors, scope errors, and index errors. Suspend/Resume support: Update to stop unconditionally clearing ACPI IRQs during suspend/resume. The status of ACPI events is no longer cleared when entering the ACPI S5 system state (power off) which caused some systems to power up immediately after turning off power in certain situations. This was a functional regression. It was fixed by clearing the status of all ACPI events again when entering S5 (for system-wide suspend or hibernation the clearing of the status of all events is not desirable, as it might cause the kernel to miss wakeup events sometimes). Rafael Wysocki. 2) iASL Compiler/Disassembler and Tools: AcpiExec: Enhanced the -fi option (Namespace initialization file). Field elements listed in the initialization file were previously initialized after the table load and before executing module-level code blocks. Recent changes in the module-level code support means that the table load becomes a large control method execution. If fields are used within module-level code and we are executing with the -fi option, the initialization values were used to initialize the namespace object(s) only after the table was finished loading. This change Provides an early initialization of objects specified in the initialization file so that field unit values are populated during the table load (not after the load). AcpiExec: Fixed a small memory leak regression that could result in warnings during exit of the utility. These warnings were similar to these: 0002D690 Length 0x0006 nsnames-0502 [Not a Descriptor - too small] 0002CD70 Length 0x002C utcache-0453 [Operand] Integer RefCount 0x0001 ---------------------------------------- 29 June 2018. Summary of changes for version 20180629: 1) iASL Compiler/Disassembler and Tools: iASL: Fixed a regression related to the use of the ASL External statement. Error checking for the use of the External() statement has been relaxed. Previously, a restriction on the use of External meant that the referenced named object was required to be defined in a different table (an SSDT). Thus it would be an error to declare an object as an external and then define the same named object in the same table. For example: DefinitionBlock (...) { External (DEV1) Device (DEV1){...} // This was an error } However, this behavior has caused regressions in some existing ASL code, because there is code that depends on named objects and externals (with the same name) being declared in the same table. This change will allow the ASL code above to compile without errors or warnings. iASL: Implemented ASL language extensions for four operators to make some of their arguments optional instead of required: 1) Field (RegionName, AccessType, LockRule, UpdateRule) 2) BankField (RegionName, BankName, BankValue, AccessType, LockRule, UpdateRule) 3) IndexField (IndexName, DataName, AccessType, LockRule, UpdateRule) For the Field operators above, the AccessType, LockRule, and UpdateRule are now optional arguments. The default values are: AccessType: AnyAcc LockRule: NoLock UpdateRule: Preserve 4) Mutex (MutexName, SyncLevel) For this operator, the SyncLevel argument is now optional. This argument is rarely used in any meaningful way by ASL code, and thus it makes sense to make it optional. The default value is: SyncLevel: 0 iASL: Attempted use of the ASL Unload() operator now results in the following warning: "Unload is not supported by all operating systems" This is in fact very true, and the Unload operator may be completely deprecated in the near future. AcpiExec: Fixed a regression for the -fi option (Namespace initialization file. Recent changes in the ACPICA module-level code support altered the table load/initialization sequence . This means that the table load has become a large method execution of the table itself. If Operation Region Fields are used within any module-level code and the -fi option was specified, the initialization values were populated only after the table had completely finished loading (and thus the module-level code had already been executed). This change moves the initialization of objects listed in the initialization file to before the table is executed as a method. Field unit values are now initialized before the table execution is performed. ---------------------------------------- 31 May 2018. Summary of changes for version 20180531: 1) ACPICA kernel-resident Subsystem: Implemented additional support to help ensure that a DSDT or SSDT is fully loaded even if errors are incurred during the load. The majority of the problems that are seen is the failure of individual AML operators that occur during execution of any module-level code (MLC) existing in the table. This support adds a mechanism to abort the current ASL statement (AML opcode), emit an error message, and to simply move on to the next opcode -- instead of aborting the entire table load. This is different than the execution of a control method where the entire method is aborted upon any error. The goal is to perform a very "best effort" to load the ACPI tables. The most common MLC errors that have been seen in the field are direct references to unresolved ASL/AML symbols (referenced directly without the use of the CondRefOf operator to validate the symbol). This new ACPICA behavior is now compatible with other ACPI implementations. Interpreter: The Unload AML operator is no longer supported for the reasons below. An AE_NOT_IMPLEMENTED exception is returned. 1) A correct implementation on at least some hosts may not be possible. 2) Other ACPI implementations do not correctly/fully support it. 3) It requires host device driver support which is not known to exist. (To properly support namespace unload out from underneath.) 4) This AML operator has never been seen in the field. Parser: Added a debug option to dump AML parse sub-trees as they are being executed. Used with ACPI_DEBUG_PRINT, the enabling debug level is ACPI_DB_PARSE_TREES. Debugger: Reduced the verbosity for errors incurred during table load and module-level code execution. Completed an investigation into adding a namespace node "owner list" instead of the current "owner ID" associated with namespace nodes. This list would link together all nodes that are owned by an individual control method. The purpose would be to enhance control method execution by speeding up cleanup during method exit (all namespace nodes created by a method are deleted upon method termination.) Currently, the entire namespace must be searched for matching owner IDs if (and only if) the method creates named objects outside of the local scope. However, by far the most common case is that methods create objects locally, not outside the method scope. There is already an ACPICA optimization in place that only searches the entire namespace in the rare case of a method creating objects elsewhere in the namespace. Therefore, it is felt that the overhead of adding an additional pointer to each namespace node to implement the owner list makes this feature unnecessary. 2) iASL Compiler/Disassembler and Tools: iASL, Disassembler, and Template generator: Implemented support for Revision D of the IORT table. Adds a new subtable that is used to specify SMMUv3 PMCGs. rmurphy-arm. Disassembler: Restored correct table header validation for the "special" ACPI tables -- RSDP and FACS. These tables do not contain a standard ACPI table header and must be special-cased. This was a regression that has been present for apparently a long time. AcpiExec: Reduced verbosity of the local exception handler implemented within acpiexec. This handler is invoked by ACPICA upon any exceptions generated during control method execution. A new option was added: -vh restores the original verbosity level if desired. AcpiExec: Changed the default base from decimal to hex for the -x option (set debug level). This simplifies the use of this option and matches the behavior of the corresponding iASL -x option. AcpiExec: Restored a force-exit on multiple control-c (sigint) interrupts. This allows program termination even if other issues cause the control-c to fail. ASL test suite (ASLTS): Added tests for the recently implemented package element resolution mechanism that allows forward references to named objects from individual package elements (this mechanism provides compatibility with other ACPI implementations.) ---------------------------------------- 8 May 2018. Summary of changes for version 20180508: 1) ACPICA kernel-resident subsystem: Completed the new (recently deployed) package resolution mechanism for the Load and LoadTable ASL/AML operators. This fixes a regression that was introduced in version 20180209 that could result in an AE_AML_INTERNAL exception during the loading of a dynamic ACPI/AML table (SSDT) that contains package objects. 2) iASL Compiler/Disassembler and Tools: AcpiDump and AcpiXtract: Implemented support for ACPI tables larger than 1 MB. This change allows for table offsets within the acpidump file to be up to 8 characters. These changes are backwards compatible with existing acpidump files. ---------------------------------------- 27 April 2018. Summary of changes for version 20180427: 1) ACPICA kernel-resident subsystem: Debugger: Added support for Package objects in the "Test Objects" command. This command walks the entire namespace and evaluates all named data objects (Integers, Strings, Buffers, and now Packages). Improved error messages for the namespace root node. Originally, the root was referred to by the confusing string "\___". This has been replaced by "Namespace Root" for clarification. Fixed a potential infinite loop in the AcpiRsDumpByteList function. Colin Ian King . 2) iASL Compiler/Disassembler and Tools: iASL: Implemented support to detect and flag illegal forward references. For compatibility with other ACPI implementations, these references are now illegal at the root level of the DSDT or SSDTs. Forward references have always been illegal within control methods. This change should not affect existing ASL/AML code because of the fact that these references have always been illegal in the other ACPI implementation. iASL: Added error messages for the case where a table OEM ID and OEM TABLE ID strings are longer than the ACPI-defined length. Previously, these strings were simply silently truncated. iASL: Enhanced the -tc option (which creates an AML hex file in C, suitable for import into a firmware project): 1) Create a unique name for the table, to simplify use of multiple SSDTs. 2) Add a protection #ifdef in the file, similar to a .h header file. With assistance from Sami Mujawar, sami.mujawar@arm.com and Evan Lloyd, evan.lloyd@arm.com AcpiExec: Added a new option, -df, to disable the local fault handler. This is useful during debugging, where it may be desired to drop into a debugger on a fault. ---------------------------------------- 13 March 2018. Summary of changes for version 20180313: 1) ACPICA kernel-resident subsystem: Implemented various improvements to the GPE support: 1) Dispatch all active GPEs at initialization time so that no GPEs are lost. 2) Enable runtime GPEs earlier. Some systems expect GPEs to be enabled before devices are enumerated. 3) Don't unconditionally clear ACPI IRQs during suspend/resume, so that IRQs are not lost. 4) Add parallel GPE handling to eliminate the possibility of dispatching the same GPE twice. 5) Dispatch any pending GPEs after enabling for the first time. AcpiGetObjectInfo - removed support for the _STA method. This was causing problems on some platforms. Added a new _OSI string, "Windows 2017.2". Cleaned up and simplified the module-level code support. These changes are in preparation for the eventual removal of the legacy MLC support (deferred execution), replaced by the new MLC architecture which executes the MLC as a table is loaded (DSDT/SSDTs). Changed a compile-time option to a runtime option. Changes the option to ignore ACPI table load-time package resolution errors into a runtime option. Used only for platforms that generate many AE_NOT_FOUND errors during boot. AcpiGbl_IgnorePackageResolutionErrors. Fixed the ACPI_ERROR_NAMESPACE macro. This change involves putting some ACPI_ERROR_NAMESPACE parameters inside macros. By doing so, we avoid compilation errors from unused variables (seen with some compilers). 2) iASL Compiler/Disassembler and Tools: ASLTS: parallelized execution in order to achieve an (approximately) 2X performance increase. ASLTS: Updated to use the iASL __LINE__ and __METHOD__ macros. Improves error reporting. ---------------------------------------- 09 February 2018. Summary of changes for version 20180209: 1) ACPICA kernel-resident subsystem: Completed the final integration of the recent changes to Package Object handling and the module-level AML code support. This allows forward references from individual package elements when the package object is declared from within module-level code blocks. Provides compatibility with other ACPI implementations. The new architecture for the AML module-level code has been completed and is now the default for the ACPICA code. This new architecture executes the module-level code in-line as the ACPI table is loaded/parsed instead of the previous architecture which deferred this code until after the table was fully loaded. This solves some ASL code ordering issues and provides compatibility with other ACPI implementations. At this time, there is an option to fallback to the earlier architecture, but this support is deprecated and is planned to be completely removed later this year. Added a compile-time option to ignore AE_NOT_FOUND exceptions during resolution of named reference elements within Package objects. Although this is potentially a serious problem, it can generate a lot of noise/errors on platforms whose firmware carries around a bunch of unused Package objects. To disable these errors, define ACPI_IGNORE_PACKAGE_RESOLUTION_ERRORS in the OS-specific header. All errors are always reported for ACPICA applications such as AcpiExec. Fixed a regression related to the explicit type-conversion AML operators (ToXXXX). The regression was introduced early in 2017 but was not seen until recently because these operators are not fully supported by other ACPI implementations and are thus rarely used by firmware developers. The operators are defined by the ACPI specification to not implement the "implicit result object conversion". The regression incorrectly introduced this object conversion for the following explicit conversion operators: ToInteger ToString ToBuffer ToDecimalString ToHexString ToBCD FromBCD 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a problem with the compiler constant folding feature as related to the ToXXXX explicit conversion operators. These operators do not support the "implicit result object conversion" by definition. Thus, ASL expressions that use these operators cannot be folded to a simple Store operator because Store implements the implicit conversion. This change uses the CopyObject operator for the ToXXXX operator folding instead. CopyObject is defined to not implement implicit result conversions and is thus appropriate for folding the ToXXXX operators. iASL: Changed the severity of an error condition to a simple warning for the case where a symbol is declared both locally and as an external symbol. This accommodates existing ASL code. AcpiExec: The -ep option to enable the new architecture for module-level code has been removed. It is replaced by the -dp option which instead has the opposite effect: it disables the new architecture (the default) and enables the legacy architecture. When the legacy code is removed in the future, the -dp option will be removed also. ---------------------------------------- 05 January 2018. Summary of changes for version 20180105: 1) ACPICA kernel-resident subsystem: Updated all copyrights to 2018. This affects all source code modules. Fixed a possible build error caused by an unresolved reference to the AcpiUtSafeStrncpy function. Removed NULL pointer arithmetic in the various pointer manipulation macros. All "(void *) NULL" constructs are converted to "(void *) 0". This eliminates warnings/errors in newer C compilers. Jung-uk Kim. Added support for A32 ABI compilation, which uses the ILP32 model. Anuj Mittal. 2) iASL Compiler/Disassembler and Tools: ASLTS: Updated all copyrights to 2018. Tools: Updated all signon copyrights to 2018. AcpiXtract: Fixed a regression related to ACPI table signatures where the signature was truncated to 3 characters (instead of 4). AcpiExec: Restore the original terminal mode after the use of the -v and -vd options. ASLTS: Deployed the iASL __METHOD__ macro across the test suite. ---------------------------------------- 14 December 2017. Summary of changes for version 20171214: 1) ACPICA kernel-resident subsystem: Fixed a regression in the external (public) AcpiEvaluateObjectTyped interface where the optional "pathname" argument had inadvertently become a required argument returning an error if omitted (NULL pointer argument). Fixed two possible memory leaks related to the recently developed "late resolution" of reference objects within ASL Package Object definitions. Added two recently defined _OSI strings: "Windows 2016" and "Windows 2017". Mario Limonciello. Implemented and deployed a safer version of the C library function strncpy: AcpiUtSafeStrncpy. The intent is to at least prevent the creation of unterminated strings as a possible result of a standard strncpy. Cleaned up and restructured the global variable file (acglobal.h). There are many changes, but no functional changes. 2) iASL Compiler/Disassembler and Tools: iASL Table Compiler: Fixed a problem with the DBG2 ACPI table where the optional OemData field at the end of the table was incorrectly required for proper compilation. It is now correctly an optional field. ASLTS: The entire suite was converted from standard ASL to the ASL+ language, using the ASL-to-ASL+ converter which is integrated into the iASL compiler. A binary compare of all output files has verified the correctness of the conversion. iASL: Fixed the source code build for platforms where "char" is unsigned. This affected the iASL lexer only. Jung-uk Kim. ---------------------------------------- 10 November 2017. Summary of changes for version 20171110: 1) ACPICA kernel-resident subsystem: This release implements full support for ACPI 6.2A: NFIT - Added a new subtable, "Platform Capabilities Structure" No other changes to ACPICA were required, since ACPI 6.2A is primarily an errata release of the specification. Other ACPI table changes: IORT: Added the SMMUv3 Device ID mapping index. Hanjun Guo PPTT: Added cache attribute flag definitions to actbl1.h. Jeremy Linton Utilities: Modified the string/integer conversion functions to use internal 64-bit divide support instead of a native divide. On 32-bit platforms, a 64-bit divide typically requires a library function which may not be present in the build (kernel or otherwise). Implemented a targeted error message for timeouts returned from the Embedded Controller device driver. This is seen frequently enough to special-case an AE_TIME returned from an EC operation region access: "Timeout from EC hardware or EC device driver" Changed the "ACPI Exception" message prefix to "ACPI Error" so that all runtime error messages have the identical prefix. 2) iASL Compiler/Disassembler and Tools: AcpiXtract: Fixed a problem with table header detection within the acpidump file. Processing a table could be ended early if a 0x40 (@) appears in the original binary table, resulting in the @ symbol appearing in the decoded ASCII field at the end of the acpidump text line. The symbol caused acpixtract to incorrectly think it had reached the end of the current table and the beginning of a new table. AcpiXtract: Added an option (-f) to ignore some errors during table extraction. This initial implementation ignores non-ASCII and non- printable characters found in the acpidump text file. TestSuite(ASLTS)/AcpiExec: Fixed and restored the memory usage statistics for ASLTS. This feature is used to track memory allocations from different memory caches within the ACPICA code. At the end of an ASLTS run, these memory statistics are recorded and stored in a log file. Debugger (user-space version): Implemented a simple "Background" command. Creates a new thread to execute a control method in the background, while control returns to the debugger prompt to allow additional commands. Syntax: Background [Arguments] ---------------------------------------- 29 September 2017. Summary of changes for version 20170929: 1) ACPICA kernel-resident subsystem: Redesigned and implemented an improved ASL While() loop timeout mechanism. This mechanism is used to prevent infinite loops in the kernel AML interpreter caused by either non-responsive hardware or incorrect AML code. The new implementation uses AcpiOsGetTimer instead of a simple maximum loop count, and is thus more accurate and constant across different machines. The default timeout is currently 30 seconds, but this may be adjusted later. Renamed the ACPI_AML_INFINITE_LOOP exception to AE_AML_LOOP_TIMEOUT to better reflect the new implementation of the loop timeout mechanism. Updated the AcpiGetTimerDuration interface to cleanup the 64-bit support and to fix an off-by-one error. Jung-uk Kim. Fixed an EFI build problem by updating the makefiles to for a new file that was added, utstrsuppt.c 2) iASL Compiler/Disassembler and Tools: Implemented full support for the PDTT, SDEV, and TPM2 ACPI tables. This includes support in the table disassembler, compiler, and template generator. iASL: Added an exception for an illegal type of recursive method invocation. If a method creates named objects, the first recursive call will fail at runtime. This change adds an error detection at compile time to catch the problem up front. Note: Marking such a method as "serialized" will not help with this problem, because the same thread can acquire the method mutex more than once. Example compiler and runtime output: Method (MTH1) { Name (INT1, 1) MTH1 () } dsdt.asl 22: MTH1 () Error 6152 - ^ Illegal recursive call to method that creates named objects (MTH1) Previous runtime exception: ACPI Error: [INT1] Namespace lookup failure, AE_ALREADY_EXISTS (20170831/dswload2-465) iASL: Updated support for External() opcodes to improve namespace management and error detection. These changes are related to issues seen with multiple-segment namespace pathnames within External declarations, such as below: External(\_SB.PCI0.GFX0, DeviceObj) External(\_SB.PCI0.GFX0.ALSI) iASL: Implemented support for multi-line error/warning messages. This enables more detailed and helpful error messages as below, from the initial deployment for the duplicate names error: DSDT.iiii 1692: Device(PEG2) { Error 6074 - ^ Name already exists in scope (PEG2) Original name creation/declaration below: DSDT.iiii 93: External(\_SB.PCI0.PEG2, DeviceObj) AcpiXtract: Added additional flexibility to support differing input hex dump formats. Specifically, hex dumps that contain partial disassembly and/or comments within the ACPI table data definition. There exist some dump utilities seen in the field that create this type of hex dump (such as Simics). For example: DSDT @ 0xdfffd0c0 (10999 bytes) Signature DSDT Length 10999 Revision 1 Checksum 0xf3 (Ok) OEM_ID BXPC OEM_table_id BXDSDT OEM_revision 1 Creator_id 1280593481 Creator_revision 537399345 0000: 44 53 44 54 f7 2a 00 00 01 f3 42 58 50 43 00 00 ... 2af0: 5f 4c 30 46 00 a4 01 Test suite: Miscellaneous changes/fixes: More cleanup and simplification of makefiles Continue compilation of test cases after a compile failure Do not perform binary compare unless both files actually exist iASL: Performed some code/module restructuring. Moved all memory allocation functions to new modules. Two new files, aslallocate.c and aslcache.c ---------------------------------------- 31 August 2017. Summary of changes for version 20170831: 1) ACPICA kernel-resident subsystem: Implemented internal support for full 64-bit addresses that appear in all Generic Address Structure (GAS) structures. Previously, only the lower 32 bits were used. Affects the use of GAS structures in the FADT and other tables, as well as the GAS structures passed to the AcpiRead and AcpiWrite public external interfaces that are used by drivers. Lv Zheng. Added header support for the PDTT ACPI table (Processor Debug Trigger Table). Full support in the iASL Data Table Compiler and disassembler is forthcoming. 2) iASL Compiler/Disassembler and Tools: iASL/Disassembler: Fixed a problem with the PPTT ACPI table (Processor Properties Topology Table) where a flag bit was specified in the wrong bit position ("Line Size Valid", bit 6). iASL: Implemented support for Octal integer constants as defined by the ASL language grammar, per the ACPI specification. Any integer constant that starts with a zero is an octal constant. For example, Store (037777, Local0) /* Octal constant */ Store (0x3FFF, Local0) /* Hex equivalent */ Store (16383, Local0) /* Decimal equivalent */ iASL: Improved overflow detection for 64-bit string conversions during compilation of integer constants. "Overflow" in this case means a string that represents an integer that is too large to fit into a 64-bit value. Any 64-bit constants within a 32-bit DSDT or SSDT are still truncated to the low-order 32 bits with a warning, as previously implemented. Several new exceptions are defined that indicate a 64-bit overflow, as well as the base (radix) that was used during the attempted conversion. Examples: Local0 = 0xAAAABBBBCCCCDDDDEEEEFFFF // AE_HEX_OVERFLOW Local0 = 01111222233334444555566667777 // AE_OCTAL_OVERFLOW Local0 = 11112222333344445555666677778888 // AE_DECIMAL_OVERFLOW iASL: Added a warning for the case where a ResourceTemplate is declared with no ResourceDescriptor entries (coded as "ResourceTemplate(){}"). In this case, the resulting template is created with a single END_TAG descriptor, which is essentially useless. iASL: Expanded the -vw option (ignore specific warnings/remarks) to include compilation error codes as well. ---------------------------------------- 28 July 2017. Summary of changes for version 20170728: 1) ACPICA kernel-resident subsystem: Fixed a regression seen with small resource descriptors that could cause an inadvertent AE_AML_NO_RESOURCE_END_TAG exception. AML interpreter: Implemented a new feature that allows forward references from individual named references within package objects that are contained within blocks of "module-level code". This provides compatibility with other ACPI implementations and supports existing firmware that depends on this feature. Example: Name (ABCD, 1) If (ABCD) /* An If() at module-level */ { Name (PKG1, Package() { INT1 /* Forward reference to object INT1 */ }) Name (INT1, 0x1234) } AML Interpreter: Fixed a problem with the Alias() operator where aliases to some ASL objects were not handled properly. Objects affected are: Mutex, Event, and OperationRegion. AML Debugger: Enhanced to properly handle AML Alias objects. These objects have one level of indirection which was not fully supported by the debugger. Table Manager: Added support to detect and ignore duplicate SSDTs within the XSDT/RSDT. This error in the XSDT has been seen in the field. EFI and EDK2 support: Enabled /WX flag for MSVC builds Added support for AcpiOsStall, AcpiOsSleep, and AcpiOsGetTimer Added local support for 64-bit multiply and shift operations Added support to compile acpidump.efi on Windows Added OSL function stubs for interfaces not used under EFI Added additional support for the _DMA predefined name. _DMA returns a buffer containing a resource template. This change add support within the resource manager (AcpiWalkResourceBuffer) to walk and parse this list of resource descriptors. Lorenzo Pieralisi 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a problem where the internal input line buffer(s) could overflow if there are very long lines in the input ASL source code file. Implemented buffer management that automatically increases the size of the buffers as necessary. iASL: Added an option (-vx) to "expect" particular remarks, warnings and errors. If the specified exception is not raised during compilation, the compiler emits an error. This is intended to support the ASL test suite, but may be useful in other contexts. iASL: Implemented a new predefined macro, __METHOD__, which returns a string containing the name of the current control method that is being compiled. iASL: Implemented debugger and table compiler support for the SDEI ACPI table (Software Delegated Exception Interface). James Morse Unix/Linux makefiles: Added an option to disable compile optimizations. The disable occurs when the NOOPT flag is set to TRUE. theracermaster@gmail.com Acpidump: Added support for multiple DSDT and FACS tables. This can occur when there are different tables for 32-bit versus 64-bit. Enhanced error reporting for the ASL test suite (ASLTS) by removing unnecessary/verbose text, and emit the actual line number where an error has occurred. These changes are intended to improve the usefulness of the test suite. ---------------------------------------- 29 June 2017. Summary of changes for version 20170629: 1) ACPICA kernel-resident subsystem: Tables: Implemented a deferred ACPI table verification. This is useful for operating systems where the tables cannot be verified in the early initialization stage due to early memory mapping limitations on some architectures. Lv Zheng. Tables: Removed the signature validation for dynamically loaded tables. Provides compatibility with other ACPI implementations. Previously, only SSDT tables were allowed, as per the ACPI specification. Now, any table signature can be used via the Load() operator. Lv Zheng. Tables: Fixed several mutex issues that could cause errors during table acquisition. Lv Zheng. Tables: Fixed a problem where an ACPI warning could be generated if a null pointer was passed to the AcpiPutTable interface. Lv Zheng. Tables: Added a mechanism to handle imbalances for the AcpiGetTable and AcpiPutTable interfaces. This applies to the "late stage" table loading when the use of AcpiPutTable is no longer required (since the system memory manager is fully running and available). Lv Zheng. Fixed/Reverted a regression during processing of resource descriptors that contain only a single EndTag. Fixes an AE_AML_NO_RESOURCE_END_TAG exception in this case. Headers: IORT/SMMU support: Updated the SMMU models for Revision C of the I/O Remapping specification. Robin Murphy Interpreter: Fixed a possible fault if an Alias operator with an invalid or duplicate target is encountered during Alias creation in AcpiExCreateAlias. Alex James Added an option to use designated initializers for function pointers. Kees Cook 2) iASL Compiler/Disassembler and Tools: iASL: Allow compilation of External declarations with target pathnames that refer to existing named objects within the table. Erik Schmauss. iASL: Fixed a regression when compiling FieldUnits. Fixes an error if a FieldUnit name also is declared via External in the same table. Erik Schmauss. iASL: Allow existing scope names within pathnames used in External statements. For example: External (ABCD.EFGH) // ABCD exists, but EFGH is truly external Device (ABCD) iASL: IORT ACPI table: Implemented changes required to decode the new Proximity Domain for the SMMUv3 IORT. Disassembler and Data Table compiler. Ganapatrao Kulkarni Disassembler: Don't abort disassembly on errors from External() statements. Erik Schmauss. Disassembler: fixed a possible fault when one of the Create*Field operators references a Resource Template. ACPICA Bugzilla 1396. iASL: In the source code, resolved some naming inconsistences across the parsing support. Fixes confusion between "Parse Op" and "Parse Node". Adds a new file, aslparseop.c ---------------------------------------- 31 May 2017. Summary of changes for version 20170531: 0) ACPI 6.2 support: The ACPI specification version 6.2 has been released and is available at http://uefi.org/specifications This version of ACPICA fully supports the ACPI 6.2 specification. Changes are summarized below. New ACPI tables (Table Compiler/Disassembler/Templates): HMAT (Heterogeneous Memory Attributes Table) WSMT (Windows SMM Security Mitigation Table) PPTT (Processor Properties Topology Table) New subtables for existing ACPI tables: HEST (New subtable, Arch-deferred machine check) SRAT (New subtable, Arch-specific affinity structure) PCCT (New subtables, Extended PCC subspaces (types 3 and 4)) Simple updates for existing ACPI tables: BGRT (two new flag bits) HEST (New bit defined for several subtables, GHES_ASSIST) New Resource Descriptors and Resource macros (Compiler/Disassembler): PinConfig() PinFunction() PinGroup() PinGroupConfig() PinGroupFunction() New type for hardware error notification (section 18.3.2.9) New predefined names/methods (Compiler/Interpreter): _HMA (Heterogeneous Memory Attributes) _LSI (Label Storage Information) _LSR (Label Storage Read) _LSW (Label Storage Write) ASL grammar/macro changes (Compiler): For() ASL macro, implemented with the AML while operator Extensions to Concatenate operator Support for multiple definition blocks in same ASL file Clarification for Buffer operator Allow executable AML code underneath all scopes (Devices, etc.) Clarification/change for the _OSI return value ASL grammar update for reference operators Allow a zero-length string for AML filename in DefinitionBlock Miscellaneous: New device object notification value Remove a notify value (0x0C) for graceful shutdown New UUIDs for processor/cache properties and physical package property New _HID, ACPI0014 (Wireless Power Calibration Device) 1) ACPICA kernel-resident subsystem: Added support to disable ACPI events on hardware-reduced platforms. Eliminates error messages of the form "Could not enable fixed event". Lv Zheng Fixed a problem using Device/Thermal objects with the ObjectType and DerefOf ASL operators. This support had not been fully/properly implemented. Fixed a problem where if a Buffer object containing a resource template was longer than the actual resource template, an error was generated -- even though the AML is legal. This case has been seen in the field. Fixed a problem with the header definition of the MADT PCAT_COMPAT flag. The values for DUAL_PIC and MULTIPLE_APIC were reversed. Added header file changes for the TPM2 ACPI table. Update to new version of the TCG specification. Adds a new TPM2 subtable for ARM SMC. Exported the external interfaces AcpiAcquireMutex and AcpiReleaseMutex. These interfaces are intended to be used only in conjunction with the predefined _DLM method (Device Lock Method). "This object appears in a device scope when AML access to the device must be synchronized with the OS environment". Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 143.1K Code, 60.0K Data, 203.1K Total Debug Version: 204.0K Code, 84.3K Data, 288.3K Total Previous Release: Non-Debug Version: 141.7K Code, 58.5K Data, 200.2K Total Debug Version: 207.5K Code, 82.7K Data, 290.2K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a problem where an External() declaration could not refer to a Field Unit. Erik Schmauss. Disassembler: Improved support for the Switch/Case operators. This feature will disassemble AML code back to the original Switch operators when possible, instead of an If..Else sequence. David Box iASL and disassembler: Improved the handling of multiple extraneous parentheses for both ASL input and disassembled ASL output. Improved the behavior of the iASL compiler and disassembler to detect improper use of external declarations Disassembler: Now aborts immediately upon detection of an unknown AML opcode. The AML parser has no real way to recover from this, and can result in the creation of an ill-formed parse tree that causes errors later during the disassembly. All tools: Fixed a problem where the Unix application OSL did not handle control-c correctly. For example, a control-c could incorrectly wake the debugger. AcpiExec: Improved the Control-C handling and added a handler for segmentation faults (SIGSEGV). Supports both Windows and Unix-like environments. Reduced the verbosity of the generic unix makefiles. Previously, each compilation displayed the full set of compiler options. This has been eliminated as the options are easily inspected within the makefiles. Each compilation now results in a single line of output. ---------------------------------------- 03 March 2017. Summary of changes for version 20170303: 0) ACPICA licensing: The licensing information at the start of each source code module has been updated. In addition to the Intel license, the dual GPLv2/BSD license has been added for completeness. Now, a single version of the source code should be suitable for all ACPICA customers. This is the major change for this release since it affects all source code modules. 1) ACPICA kernel-resident subsystem: Fixed two issues with the common asltypes.h header that could cause problems in some environments: (Kim Jung-uk) Removed typedef for YY_BUFFER_STATE ? Fixes an error with earlier versions of Flex. Removed use of FILE typedef (which is only defined in stdio.h) 2) iASL Compiler/Disassembler and Tools: Disassembler: fixed a regression introduced in 20170224. A fix for a memory leak related to resource descriptor tags (names) could fault when the disassembler was generated with 64-bit compilers. The ASLTS test suite has been updated to implement a new testing architecture. During generation of the suite from ASL source, both the ASL and ASL+ compilers are now validated, as well as the disassembler itself (Erik Schmauss). The architecture executes as follows: For every ASL source module: Compile (legacy ASL compilation) Disassemble the resulting AML to ASL+ source code Compile the new ASL+ module Perform a binary compare on the legacy AML and the new ASL+ AML The ASLTS suite then executes normally using the AML binaries. ---------------------------------------- 24 February 2017. Summary of changes for version 20170224: 1) ACPICA kernel-resident subsystem: Interpreter: Fixed two issues with the control method return value auto- repair feature, where an attempt to double-delete an internal object could result in an ACPICA warning (for _CID repair and others). No fault occurs, however, because the attempted deletion (actually a release to an internal cache) is detected and ignored via object poisoning. Debugger: Fixed an AML interpreter mutex issue during the single stepping of control methods. If certain debugger commands are executed during stepping, a mutex acquire/release error could occur. Lv Zheng. Fixed some issues generating ACPICA with the Intel C compiler by restoring the original behavior and compiler-specific include file in acenv.h. Lv Zheng. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 141.7K Code, 58.5K Data, 200.2K Total Debug Version: 207.5K Code, 82.7K Data, 290.2K Total Previous Release: Non-Debug Version: 137.4K Code, 52.6K Data, 190.0K Total Debug Version: 201.5K Code, 82.2K Data, 283.7K Total 2) iASL Compiler/Disassembler and Tools: iASL/Disassembler: A preliminary version of a new ASL-to-ASL+ conversion tool has been designed, implemented, and included in this release. The key feature of this utility is that the original comments within the input ASL file are preserved during the conversion process, and included within the converted ASL+ file -- thus creating a transparent conversion of existing ASL files to ASL+ (ASL 2.0). Erik Schmauss. Usage: iasl -ca // Output is a .dsl file with converted code iASL/Disassembler: Improved the detection and correct disassembly of Switch/Case operators. This feature detects sequences of if/elseif/else operators that originated from ASL Switch/Case/Default operators and emits the original operators. David Box. iASL: Improved the IORT ACPI table support in the following areas. Lv Zheng: Clear MappingOffset if the MappingCount is zero. Fix the disassembly of the SMMU GSU interrupt offset. Update the template file for the IORT table. Disassembler: Enhanced the detection and disassembly of resource template/descriptor within a Buffer object. An EndTag descriptor is now required to have a zero second byte, since all known ASL compilers emit this. This helps eliminate incorrect decisions when a buffer is disassembled (false positives on resource templates). ---------------------------------------- 19 January 2017. Summary of changes for version 20170119: 1) General ACPICA software: Entire source code base: Added the 2017 copyright to all source code legal/licensing module headers and utility/tool signons. This includes the standard Linux dual-license header. This affects virtually every file in the ACPICA core subsystem, iASL compiler, all ACPICA utilities, and the ACPICA test suite. 2) iASL Compiler/Disassembler and Tools: iASL: Removed/fixed an inadvertent remark when a method argument containing a reference is used as a target operand within the method (and never used as a simple argument), as in the example below. Jeffrey Hugo. dsdt.asl 1507: Store(0x1, Arg0) Remark 2146 - ^ Method Argument is never used (Arg0) All tools: Removed the bit width of the compiler that generated the tool from the common signon for all user space tools. This proved to be confusing and unnecessary. This includes similar removal of HARDWARE_NAME from the generic makefiles (Thomas Petazzoni). Example below. Old: ASL+ Optimizing Compiler version 20170119-32 ASL+ Optimizing Compiler version 20170119-64 New: ASL+ Optimizing Compiler version 20170119 ---------------------------------------- 22 December 2016. Summary of changes for version 20161222: 1) ACPICA kernel-resident subsystem: AML Debugger: Implemented a new mechanism to simplify and enhance debugger integration into all environments, including kernel debuggers and user-space utilities, as well as remote debug services. This mechanism essentially consists of new OSL interfaces to support debugger initialization/termination, as well as wait/notify interfaces to perform the debugger handshake with the host. Lv Zheng. New OSL interfaces: AcpiOsInitializeDebugger (void) AcpiOsTerminateDebugger (void) AcpiOsWaitCommandReady (void) AcpiOsNotifyCommandComplete (void) New OS services layer: osgendbg.c -- Example implementation, and used for AcpiExec Update for Generic Address Space (GAS) support: Although the AccessWidth and/or BitOffset fields of the GAS are not often used, this change now fully supports these fields. This affects the internal support for FADT registers, registers in other ACPI data tables, and the AcpiRead and AcpiWrite public interfaces. Lv Zheng. Sleep support: In order to simplify integration of ACPI sleep for the various host operating systems, a new OSL interface has been introduced. AcpiOsEnterSleep allows the host to perform any required operations before the final write to the sleep control register(s) is performed by ACPICA. Lv Zheng. New OSL interface: AcpiOsEnterSleep(SleepState, RegisterAValue, RegisterBValue) Called from these internal interfaces: AcpiHwLegacySleep AcpiHwExtendedSleep EFI support: Added a very small EFI/ACPICA example application. Provides a simple demo for EFI integration, as well as assisting with resolution of issues related to customer ACPICA/EFI integration. Lv Zheng. See: source/tools/efihello/efihello.c Local C library: Implemented several new functions to enhance ACPICA portability, for environments where these clib functions are not available (such as EFI). Lv Zheng: putchar getchar strpbrk strtok memmove Fixed a regression where occasionally a valid resource descriptor was incorrectly detected as invalid at runtime, and a AE_AML_NO_RESOURCE_END_TAG was returned. Fixed a problem with the recently implemented support that enables control method invocations as Target operands to many ASL operators. Warnings of this form: "Needed type [Reference], found [Processor]" were seen at runtime for some method invocations. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 141.5K Code, 58.5K Data, 200.0K Total Debug Version: 201.7K Code, 82.7K Data, 284.4K Total Previous Release: Non-Debug Version: 140.5K Code, 58.5K Data, 198.9K Total Debug Version: 201.3K Code, 82.7K Data, 284.0K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Enhanced output by adding the capability to detect and disassemble ASL Switch/Case statements back to the original ASL source code instead of if/else blocks. David Box. AcpiHelp: Split a large file into separate files based upon functionality/purpose. New files are: ahaml.c ahasl.c ---------------------------------------- 17 November 2016. Summary of changes for version 20161117: 1) ACPICA kernel-resident subsystem: Table Manager: Fixed a regression introduced in 20160729, "FADT support cleanup". This was an attempt to remove all references in the source to the FADT version 2, which never was a legal version number. It was skipped because it was an early version of 64-bit support that was eventually abandoned for the current 64-bit support. Interpreter: Fixed a problem where runtime implicit conversion was incorrectly disabled for the ASL operators below. This brings the behavior into compliance with the ACPI specification: FromBCD ToBCD ToDecimalString ToHexString ToInteger ToBuffer Table Manager: Added a new public interface, AcpiPutTable, used to release and free an ACPI table returned by AcpiGetTable and related interfaces. Lv Zheng. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 140.5K Code, 58.5K Data, 198.9K Total Debug Version: 201.3K Code, 82.7K Data, 284.0K Total Previous Release: Non-Debug Version: 140.4K Code, 58.1K Data, 198.5K Total Debug Version: 200.7K Code, 82.1K Data, 282.8K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Fixed a regression for disassembly of Resource Template. Detection of templates in the AML stream missed some types of templates. iASL: Fixed a problem where an Access Size error was returned for the PCC address space when the AccessSize of the GAS register is greater than a DWORD. Hoan Tran. iASL: Implemented several grammar changes for the operators below. These changes are slated for the next version of the ACPI specification: RefOf - Disallow method invocation as an operand CondRefOf - Disallow method invocation as an operand DerefOf - Disallow operands that use the result from operators that do not return a reference (Changed TermArg to SuperName). iASL: Control method invocations are now allowed for Target operands, as per the ACPI specification. Removed error for using a control method invocation as a Target operand. Disassembler: Improved detection of Resource Templates, Unicode, and Strings within Buffer objects. These subtypes do not contain a specific opcode to indicate the originating ASL code, and they must be detected by other means within the disassembler. iASL: Implemented an optimization improvement for 32-bit ACPI tables (DSDT/SSDT). For the 32-bit case only, compute the optimum integer opcode only after 64-bit to 32-bit truncation. A truncation warning message is still emitted, however. AcpiXtract: Implemented handling for both types of line terminators (LF or CR/LF) so that it can accept AcpiDump output files from any system. Peter Wu. AcpiBin: Added two new options for comparing AML files: -a: compare and display ALL mismatches -o: start compare at this offset into the second file ---------------------------------------- 30 September 2016. Summary of changes for version 20160930: 1) ACPICA kernel-resident subsystem: Fixed a regression in the internal AcpiTbFindTable function where a non AE_OK exception could inadvertently be returned even if the function did not fail. This problem affects the following operators: DataTableRegion LoadTable Fixed a regression in the LoadTable operator where a load to any namespace location other than the root no longer worked properly. Increased the maximum loop count value that will result in the AE_AML_INFINITE_LOOP exception. This is a mechanism that is intended to prevent infinite loops within the AML interpreter and thus the host OS kernel. The value is increased from 0xFFFF to 0xFFFFF loops (65,535 to 1,048,575). Moved the AcpiGbl_MaxLoopIterations configuration variable to the public acpixf.h file. This allows hosts to easily configure the maximum loop count at runtime. Removed an illegal character in the strtoul64.c file. This character caused errors with some C compilers. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 140.4K Code, 58.1K Data, 198.5K Total Debug Version: 200.7K Code, 82.1K Data, 282.8K Total Previous Release: Non-Debug Version: 140.0K Code, 58.1K Data, 198.1K Total Debug Version: 200.3K Code, 82.1K Data, 282.4K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Fixed a problem with the conversion of Else{If{ blocks into the simpler ASL ElseIf keyword. During the conversion, a trailing If block could be lost and missing from the disassembled output. iASL: Fixed a missing parser rule for the ObjectType operator. For ASL+, the missing rule caused a parse error when using the Index operator as an operand to ObjectType. This construct now compiles properly. Example: ObjectType(PKG1[4]). iASL: Correctly handle unresolved symbols in the hardware map file (-lm option). Previously, unresolved symbols could cause a protection fault. Such symbols are now marked as unresolved in the map file. iASL: Implemented support to allow control method invocations as an operand to the ASL DeRefOf operator. Example: DeRefOf(MTH1(Local0)) Disassembler: Improved support for the ToPLD ASL macro. Detection of a possible _PLD buffer now includes examination of both the normal buffer length (16 or 20) as well as the surrounding AML package length. Disassembler: Fixed a problem with the decoding of complex expressions within the Divide operator for ASL+. For the case where both the quotient and remainder targets are specified, the entire statement cannot be disassembled. Previously, the output incorrectly contained a mix of ASL- and ASL+ operators. This mixed statement causes a syntax error when compiled. Example: Divide (Add (INT1, 6), 128, RSLT, QUOT) // was incorrectly disassembled to: Divide (INT1 + 6, 128, RSLT, QUOT) iASL/Tools: Added support to process AML and non-AML ACPI tables consistently. For the disassembler and AcpiExec, allow all types of ACPI tables (AML and data tables). For the iASL -e option, allow only AML tables (DSDT/SSDT). ---------------------------------------- 31 August 2016. Summary of changes for version 20160831: 1) ACPICA kernel-resident subsystem: Improve support for the so-called "module-level code", which is defined to be math, logical and control AML opcodes that appear outside of any control method. This change improves the support by adding more opcodes that can be executed in the manner. Some other issues have been solved, and the ASL grammar changes to support such code under all scope operators (Device, etc.) are complete. Lv Zheng. UEFI support: these OSL functions have been implemented. This is an additional step toward supporting the AcpiExec utility natively (with full hardware access) under UEFI. Marcelo Ferreira. AcpiOsReadPciConfiguration AcpiOsWritePciConfiguration Fixed a possible mutex error during control method auto-serialization. Lv Zheng. Updated support for the Generic Address Structure by fully implementing all GAS fields when a 32-bit address is expanded to a 64-bit GAS. Lv Zheng. Updated the return value for the internal _OSI method. Instead of 0xFFFFFFFF, the "Ones" value is now returned, which is 0xFFFFFFFFFFFFFFFF for 64-bit ACPI tables. This fixes an incompatibility with other ACPI implementations, and will be reflected and clarified in the next version of the ACPI specification. Implemented two new table events that can be passed to an ACPICA table handler. These events are used to indicate a table installation or uninstallation. These events are used in addition to existed table load and unload events. Lv Zheng. Implemented a cleanup for all internal string-to-integer conversions. Consolidate multiple versions of this functionality and limit possible bases to either 10 or 16 to simplify the code. Adds a new file, utstrtoul64. Cleanup the inclusion order of the various compiler-specific headers. This simplifies build configuration management. The compiler-specific headers are now split out from the host-specific headers. Lv Zheng. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 140.1K Code, 58.1K Data, 198.1K Total Debug Version: 200.3K Code, 82.1K Data, 282.4K Total 2) iASL Compiler/Disassembler and Tools: iASL/AcpiExec: Added a command line option to display the build date/time of the tool (-vd). This can be useful to verify that the correct version of the tools are being used. AML Debugger: Implemented a new subcommand ("execute predef") to execute all predefined control methods and names within the current namespace. This can be useful for debugging problems with ACPI tables and the ACPI namespace. ---------------------------------------- 29 July 2016. Summary of changes for version 20160729: 1) ACPICA kernel-resident subsystem: Implemented basic UEFI support for the various ACPICA tools. This includes: 1) An OSL to implement the various AcpiOs* interfaces on UEFI. 2) Support to obtain the ACPI tables on UEFI. 3) Local implementation of required C library functions not available on UEFI. 4) A front-end (main) function for the tools for UEFI-related initialization. The initial deployment of this support is the AcpiDump utility executing as an UEFI application via EDK2 (EDKII, "UEFI Firmware Development Kit"). Current environments supported are Linux/Unix. MSVC generation is not supported at this time. See the generate/efi/README file for build instructions. Lv Zheng. Future plans include porting the AcpiExec utility to execute natively on the platform with I/O and memory access. This will allow viewing/dump of the platform namespace and native execution of ACPI control methods that access the actual hardware. To fully implement this support, the OSL functions below must be implemented with UEFI interfaces. Any community help in the implementation of these functions would be appreciated: AcpiOsReadPort AcpiOsWritePort AcpiOsReadMemory AcpiOsWriteMemory AcpiOsReadPciConfiguration AcpiOsWritePciConfiguration Restructured and standardized the C library configuration for ACPICA, resulting in the various configuration options below. This includes a global restructuring of the compiler-dependent and platform-dependent include files. These changes may affect the existing platform-dependent configuration files on some hosts. Lv Zheng. The current C library configuration options appear below. For any issues, it may be helpful to examine the existing compiler-dependent and platform-dependent files as examples. Lv Zheng. 1) Linux kernel: ACPI_USE_STANDARD_HEADERS=n in order not to use system-provided C library. ACPI_USE_SYSTEM_CLIBRARY=y in order not to use ACPICA mini C library. 2) Unix/Windows/BSD applications: ACPI_USE_STANDARD_HEADERS=y in order to use system-provided C library. ACPI_USE_SYSTEM_CLIBRARY=y in order not to use ACPICA mini C library. 3) UEFI applications: ACPI_USE_STANDARD_HEADERS=n in order not to use system-provided C library. ACPI_USE_SYSTEM_CLIBRARY=n in order to use ACPICA mini C library. 4) UEFI applications (EDK2/StdLib): ACPI_USE_STANDARD_HEADERS=y in order to use EDK2 StdLib C library. ACPI_USE_SYSTEM_CLIBRARY=y in order to use EDK2 StdLib C library. AML interpreter: "module-level code" support. Allows for execution of so- called "executable" AML code (math/logical operations, etc.) outside of control methods not just at the module level (top level) but also within any scope declared outside of a control method - Scope{}, Device{}, Processor{}, PowerResource{}, and ThermalZone{}. Lv Zheng. Simplified the configuration of the "maximum AML loops" global option by adding a global public variable, "AcpiGbl_MaxLoopIterations" which can be modified at runtime. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 139.1K Code, 22.9K Data, 162.0K Total Debug Version: 199.0K Code, 81.8K Data, 280.8K Total 2) iASL Compiler/Disassembler and Tools: iASL: Add full support for the RASF ACPI table (RAS Features Table). Includes disassembler, data table compiler, and header support. iASL Expand "module-level code" support. Allows for compilation/disassembly of so-called "executable" AML code (math/logical operations, etc.) outside of control methods not just at the module level (top level) but also within any scope declared outside of a control method - Scope{}, Device{}, Processor{}, PowerResource{}, and ThermalZone{}. AcpiDump: Added support for dumping all SSDTs on newer versions of Windows. These tables are now easily available -- SSDTs are not available through the registry on older versions. ---------------------------------------- 27 May 2016. Summary of changes for version 20160527: 1) ACPICA kernel-resident subsystem: Temporarily reverted the new arbitrary bit length/alignment support in AcpiHwRead/AcpiHwWrite for the Generic Address Structure. There have been a number of regressions with the new code that need to be fully resolved and tested before this support can be finally integrated into ACPICA. Apologies for any inconveniences these issues may have caused. The ACPI message macros are not configurable (ACPI_MSG_ERROR, ACPI_MSG_EXCEPTION, ACPI_MSG_WARNING, ACPI_MSG_INFO, ACPI_MSG_BIOS_ERROR, and ACPI_MSG_BIOS_WARNING). Lv Zheng. Fixed a couple of GCC warnings associated with the use of the -Wcast-qual option. Adds a new return macro, return_STR. Junk-uk Kim. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 136.8K Code, 51.6K Data, 188.4K Total Debug Version: 201.5K Code, 82.2K Data, 283.7K Total Previous Release: Non-Debug Version: 137.4K Code, 52.6K Data, 190.0K Total Debug Version: 200.9K Code, 82.2K Data, 283.1K Total ---------------------------------------- 22 April 2016. Summary of changes for version 20160422: 1) ACPICA kernel-resident subsystem: Fixed a regression in the GAS (generic address structure) arbitrary bit support in AcpiHwRead/AcpiHwWrite. Problem could cause incorrect behavior and incorrect return values. Lv Zheng. ACPICA BZ 1270. ACPI 6.0: Added support for new/renamed resource macros. One new argument was added to each of these macros, and the original name has been deprecated. The AML disassembler will always disassemble to the new names. Support for the new macros was added to iASL, disassembler, resource manager, and the acpihelp utility. ACPICA BZ 1274. I2cSerialBus -> I2cSerialBusV2 SpiSerialBus -> SpiSerialBusV2 UartSerialBus -> UartSerialBusV2 ACPI 6.0: Added support for a new integer field that was appended to the package object returned by the _BIX method. This adds iASL compile-time and AML runtime error checking. ACPICA BZ 1273. ACPI 6.1: Added support for a new PCCT subtable, "HW-Reduced Comm Subspace Type2" (Headers, Disassembler, and data table compiler). Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 137.4K Code, 52.6K Data, 190.0K Total Debug Version: 201.5K Code, 82.2K Data, 283.7K Total Previous Release: Non-Debug Version: 137.1K Code, 51.5K Data, 188.6K Total Debug Version: 201.0K Code, 82.0K Data, 283.0K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented an ASL grammar extension to allow/enable executable "module-level code" to be created and executed under the various operators that create new scopes. This type of AML code is already supported in all known AML interpreters, and the grammar change will appear in the next version of the ACPI specification. Simplifies the conditional runtime creation of named objects under these object types: Device PowerResource Processor Scope ThermalZone iASL: Implemented a new ASL extension, a "For" loop macro to add greater ease-of-use to the ASL language. The syntax is similar to the corresponding C operator, and is implemented with the existing AML While opcode -- thus requiring no changes to existing AML interpreters. For (Initialize, Predicate, Update) {TermList} Grammar: ForTerm := For ( Initializer // Nothing | TermArg => ComputationalData Predicate // Nothing | TermArg => ComputationalData Update // Nothing | TermArg => ComputationalData ) {TermList} iASL: The _HID/_ADR detection and validation has been enhanced to search under conditionals in order to allow these objects to be conditionally created at runtime. iASL: Fixed several issues with the constant folding feature. The improvement allows better detection and resolution of statements that can be folded at compile time. ACPICA BZ 1266. iASL/Disassembler: Fixed a couple issues with the Else{If{}...} conversion to the ASL ElseIf operator where incorrect ASL code could be generated. iASL/Disassembler: Fixed a problem with the ASL+ code disassembly where sometimes an extra (and extraneous) set of parentheses were emitted for some combinations of operators. Although this did not cause any problems with recompilation of the disassembled code, it made the code more difficult to read. David Box. ACPICA BZ 1231. iASL: Changed to ignore the unreferenced detection for predefined names of resource descriptor elements, when the resource descriptor is created/defined within a control method. iASL: Disassembler: Fix a possible fault with externally declared Buffer objects. ---------------------------------------- 18 March 2016. Summary of changes for version 20160318: 1) ACPICA kernel-resident subsystem: Added support for arbitrary bit lengths and bit offsets for registers defined by the Generic Address Structure. Previously, only aligned bit lengths of 8/16/32/64 were supported. This was sufficient for many years, but recently some machines have been seen that require arbitrary bit- level support. ACPICA BZ 1240. Lv Zheng. Fixed an issue where the \_SB._INI method sometimes must be evaluated before any _REG methods are evaluated. Lv Zheng. Implemented several changes related to ACPI table support (Headers/Disassembler/TableCompiler): NFIT: For ACPI 6.1, updated to add some additional new fields and constants. FADT: Updated a warning message and set compliance to ACPI 6.1 (Version 6). DMAR: Added new constants per the 10/2014 DMAR spec. IORT: Added new subtable per the 10/2015 IORT spec. HEST: For ACPI 6.1, added new constants and new subtable. DBG2: Added new constants per the 12/2015 DBG2 spec. FPDT: Fixed several incorrect fields, add the FPDT boot record structure. ACPICA BZ 1249. ERST/EINJ: Updated disassembler with new "Execute Timings" actions. Updated header support for the DMAR table to match the current version of the related spec. Added extensions to the ASL Concatenate operator to allow any ACPI object to be passed as an operand. Any object other than Integer/String/Buffer simply returns a string containing the object type. This extends the usefulness of the Printf macros. Previously, Concatenate would abort the control method if a non-data object was encountered. ACPICA source code: Deployed the C "const" keyword across the source code where appropriate. ACPICA BZ 732. Joerg Sonnenberger (NetBSD). Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 137.1K Code, 51.5K Data, 188.6K Total Debug Version: 201.0K Code, 82.0K Data, 283.0K Total Previous Release: Non-Debug Version: 136.2K Code, 51.5K Data, 187.7K Total Debug Version: 200.4K Code, 82.0K Data, 282.4K Total 2) iASL Compiler/Disassembler and Tools: iASL/Disassembler: Improved the heuristic used to determine the number of arguments for an externally defined control method (a method in another table). Although this is an improvement, there is no deterministic way to "guess" the number of method arguments. Only the ACPI 6.0 External opcode will completely solve this problem as it is deployed (automatically) in newer BIOS code. iASL/Disassembler: Fixed an ordering issue for emitted External() ASL statements that could cause errors when the disassembled file is compiled. ACPICA BZ 1243. David Box. iASL: Fixed a regression caused by the merger of the two versions of the local strtoul64. Because of a dependency on a global variable, strtoul64 could return an error for integers greater than a 32-bit value. ACPICA BZ 1260. iASL: Fixed a regression where a fault could occur for an ASL Return statement if it invokes a control method that is not resolved. ACPICA BZ 1264. AcpiXtract: Improved input file validation: detection of binary files and non-acpidump text files. ---------------------------------------- 12 February 2016. Summary of changes for version 20160212: 1) ACPICA kernel-resident subsystem: Implemented full support for the ACPI 6.1 specification (released in January). This version of the specification is available at: http://www.uefi.org/specifications Only a relatively small number of changes were required in ACPICA to support ACPI 6.1, in these areas: - New predefined names - New _HID values - A new subtable for HEST - A few other header changes for new values Ensure \_SB_._INI is executed before any _REG methods are executed. There appears to be existing BIOS code that relies on this behavior. Lv Zheng. Reverted a change made in version 20151218 which enabled method invocations to be targets of various ASL operators (SuperName and Target grammar elements). While the new behavior is supported by the ACPI specification, other AML interpreters do not support this behavior and never will. The ACPI specification will be updated for ACPI 6.2 to remove this support. Therefore, the change was reverted to the original ACPICA behavior. ACPICA now supports the GCC 6 compiler. Current Release: (Note: build changes increased sizes) Non-Debug Version: 136.2K Code, 51.5K Data, 187.7K Total Debug Version: 200.4K Code, 82.0K Data, 282.4K Total Previous Release: Non-Debug Version: 102.7K Code, 28.4K Data, 131.1K Total Debug Version: 200.4K Code, 81.9K Data, 282.3K Total 2) iASL Compiler/Disassembler and Tools: Completed full support for the ACPI 6.0 External() AML opcode. The compiler emits an external AML opcode for each ASL External statement. This opcode is used by the disassembler to assist with the disassembly of external control methods by specifying the required number of arguments for the method. AML interpreters do not use this opcode. To ensure that interpreters do not even see the opcode, a block of one or more external opcodes is surrounded by an "If(0)" construct. As this feature becomes commonly deployed in BIOS code, the ability of disassemblers to correctly disassemble AML code will be greatly improved. David Box. iASL: Implemented support for an optional cross-reference output file. The -lx option will create a the cross-reference file with the suffix "xrf". Three different types of cross-reference are created in this file: - List of object references made from within each control method - Invocation (caller) list for each user-defined control method - List of references to each non-method object in the namespace iASL: Method invocations as ASL Target operands are now disallowed and flagged as errors in preparation for ACPI 6.2 (see the description of the problem above). ---------------------------------------- 8 January 2016. Summary of changes for version 20160108: 1) ACPICA kernel-resident subsystem: Updated all ACPICA copyrights and signons to 2016: Added the 2016 copyright to all source code module headers and utility/tool signons. This includes the standard Linux dual-license header. This affects virtually every file in the ACPICA core subsystem, iASL compiler, all ACPICA utilities, and the ACPICA test suite. Fixed a regression introduced in version 20151218 concerning the execution of so-called module-level ASL/AML code. Namespace objects created under a module-level If() construct were not properly/fully entered into the namespace and could cause an interpreter fault when accessed. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 102.7K Code, 28.4K Data, 131.1K Total Debug Version: 200.4K Code, 81.9K Data, 282.4K Total Previous Release: Non-Debug Version: 102.6K Code, 28.4K Data, 131.0K Total Debug Version: 200.3K Code, 81.9K Data, 282.3K Total 2) iASL Compiler/Disassembler and Tools: Fixed a problem with the compilation of the GpioIo and GpioInt resource descriptors. The _PIN field name was incorrectly defined to be an array of 32-bit values, but the _PIN values are in fact 16 bits each. This would cause incorrect bit width warnings when using Word (16-bit) fields to access the descriptors. ---------------------------------------- 18 December 2015. Summary of changes for version 20151218: 1) ACPICA kernel-resident subsystem: Implemented per-AML-table execution of "module-level code" as individual ACPI tables are loaded into the namespace during ACPICA initialization. In other words, any module-level code within an AML table is executed immediately after the table is loaded, instead of batched and executed after all of the tables have been loaded. This provides compatibility with other ACPI implementations. ACPICA BZ 1219. Bob Moore, Lv Zheng, David Box. To fully support the feature above, the default operation region handlers for the SystemMemory, SystemIO, and PCI_Config address spaces are now installed before any ACPI tables are loaded. This enables module-level code to access these address spaces during the table load and module- level code execution phase. ACPICA BZ 1220. Bob Moore, Lv Zheng, David Box. Implemented several changes to the internal _REG support in conjunction with the changes above. Also, changes to the AcpiExec/AcpiNames/Examples utilities for the changes above. Although these tools were changed, host operating systems that simply use the default handlers for SystemMemory, SystemIO, and PCI_Config spaces should not require any update. Lv Zheng. For example, in the code below, DEV1 is conditionally added to the namespace by the DSDT via module-level code that accesses an operation region. The SSDT references DEV1 via the Scope operator. DEV1 must be created immediately after the DSDT is loaded in order for the SSDT to successfully reference DEV1. Previously, this code would cause an AE_NOT_EXIST exception during the load of the SSDT. Now, this code is fully supported by ACPICA. DefinitionBlock ("", "DSDT", 2, "Intel", "DSDT1", 1) { OperationRegion (OPR1, SystemMemory, 0x400, 32) Field (OPR1, AnyAcc, NoLock, Preserve) { FLD1, 1 } If (FLD1) { Device (\DEV1) { } } } DefinitionBlock ("", "SSDT", 2, "Intel", "SSDT1", 1) { External (\DEV1, DeviceObj) Scope (\DEV1) { } } Fixed an AML interpreter problem where control method invocations were not handled correctly when the invocation was itself a SuperName argument to another ASL operator. In these cases, the method was not invoked. ACPICA BZ 1002. Affects the following ASL operators that have a SuperName argument: Store Acquire, Wait CondRefOf, RefOf Decrement, Increment Load, Unload Notify Signal, Release, Reset SizeOf Implemented automatic String-to-ObjectReference conversion support for packages returned by predefined names (such as _DEP). A common BIOS error is to add double quotes around an ObjectReference namepath, which turns the reference into an unexpected string object. This support detects the problem and corrects it before the package is returned to the caller that invoked the method. Lv Zheng. Implemented extensions to the Concatenate operator. Concatenate now accepts any type of object, it is not restricted to simply Integer/String/Buffer. For objects other than these 3 basic data types, the argument is treated as a string containing the name of the object type. This expands the utility of Concatenate and the Printf/Fprintf macros. ACPICA BZ 1222. Cleaned up the output of the ASL Debug object. The timer() value is now optional and no longer emitted by default. Also, the basic data types of Integer/String/Buffer are simply emitted as their values, without a data type string -- since the data type is obvious from the output. ACPICA BZ 1221. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 102.6K Code, 28.4K Data, 131.0K Total Debug Version: 200.3K Code, 81.9K Data, 282.3K Total Previous Release: Non-Debug Version: 102.0K Code, 28.3K Data, 130.3K Total Debug Version: 199.6K Code, 81.8K Data, 281.4K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed some issues with the ASL Include() operator. This operator was incorrectly defined in the iASL parser rules, causing a new scope to be opened for the code within the include file. This could lead to several issues, including allowing ASL code that is technically illegal and not supported by AML interpreters. Note, this does not affect the related #include preprocessor operator. ACPICA BZ 1212. iASL/Disassembler: Implemented support for the ASL ElseIf operator. This operator is essentially an ASL macro since there is no AML opcode associated with it. The code emitted by the iASL compiler for ElseIf is an Else opcode followed immediately by an If opcode. The disassembler will now emit an ElseIf if it finds an Else immediately followed by an If. This simplifies the decoded ASL, especially for deeply nested If..Else and large Switch constructs. Thus, the disassembled code more closely follows the original source ASL. ACPICA BZ 1211. Example: Old disassembly: Else { If (Arg0 == 0x02) { Local0 = 0x05 } } New disassembly: ElseIf (Arg0 == 0x02) { Local0 = 0x05 } AcpiExec: Added support for the new module level code behavior and the early region installation. This required a small change to the initialization, since AcpiExec must install its own operation region handlers. AcpiExec: Added support to make the debug object timer optional. Default is timer disabled. This cleans up the debug object output -- the timer data is rarely used. AcpiExec: Multiple ACPI tables are now loaded in the order that they appear on the command line. This can be important when there are interdependencies/references between the tables. iASL/Templates. Add support to generate template files with multiple SSDTs within a single output file. Also added ommand line support to specify the number of SSDTs (in addition to a single DSDT). ACPICA BZ 1223, 1225. ---------------------------------------- 24 November 2015. Summary of changes for version 20151124: 1) ACPICA kernel-resident subsystem: Fixed a possible regression for a previous update to FADT handling. The FADT no longer has a fixed table ID, causing some issues with code that was hardwired to a specific ID. Lv Zheng. Fixed a problem where the method auto-serialization could interfere with the current SyncLevel. This change makes the auto-serialization support transparent to the SyncLevel support and management. Removed support for the _SUB predefined name in AcpiGetObjectInfo. This interface is intended for early access to the namespace during the initial namespace device discovery walk. The _SUB method has been seen to access operation regions in some cases, causing errors because the operation regions are not fully initialized. AML Debugger: Fixed some issues with the terminate/quit/exit commands that can cause faults. Lv Zheng. AML Debugger: Add thread ID support so that single-step mode only applies to the AML Debugger thread. This prevents runtime errors within some kernels. Lv Zheng. Eliminated extraneous warnings from AcpiGetSleepTypeData. Since the _Sx methods that are invoked by this interface are optional, removed warnings emitted for the case where one or more of these methods do not exist. ACPICA BZ 1208, original change by Prarit Bhargava. Made a major pass through the entire ACPICA source code base to standardize formatting that has diverged a bit over time. There are no functional changes, but this will of course cause quite a few code differences from the previous ACPICA release. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 102.0K Code, 28.3K Data, 130.3K Total Debug Version: 199.6K Code, 81.8K Data, 281.4K Total Previous Release: Non-Debug Version: 101.7K Code, 27.9K Data, 129.6K Total Debug Version: 199.3K Code, 81.4K Data, 280.7K Total 2) iASL Compiler/Disassembler and Tools: iASL/acpiexec/acpixtract/disassembler: Added support to allow multiple definition blocks within a single ASL file and the resulting AML file. Support for this type of file was also added to the various tools that use binary AML files: acpiexec, acpixtract, and the AML disassembler. The example code below shows two definition blocks within the same file: DefinitionBlock ("dsdt.aml", "DSDT", 2, "Intel", "Template", 0x12345678) { } DefinitionBlock ("", "SSDT", 2, "Intel", "Template", 0xABCDEF01) { } iASL: Enhanced typechecking for the Name() operator. All expressions for the value of the named object must be reduced/folded to a single constant at compile time, as per the ACPI specification (the AML definition of Name()). iASL: Fixed some code indentation issues for the -ic and -ia options (C and assembly headers). Now all emitted code correctly begins in column 1. iASL: Added an error message for an attempt to open a Scope() on an object defined in an SSDT. The DSDT is always loaded into the namespace first, so any attempt to open a Scope on an SSDT object will fail at runtime. ---------------------------------------- 30 September 2015. Summary of changes for version 20150930: 1) ACPICA kernel-resident subsystem: Debugger: Implemented several changes and bug fixes to assist support for the in-kernel version of the AML debugger. Lv Zheng. - Fix the "predefined" command for in-kernel debugger. - Do not enter debug command loop for the help and version commands. - Disallow "execute" command during execution/single-step of a method. Interpreter: Updated runtime typechecking for all operators that have target operands. The operand is resolved and validated that it is legal. For example, the target cannot be a non-data object such as a Device, Mutex, ThermalZone, etc., as per the ACPI specification. Debugger: Fixed the double-mutex user I/O handshake to work when local deadlock detection is enabled. Debugger: limited display of method locals and arguments (LocalX and ArgX) to only those that have actually been initialized. This prevents lines of extraneous output. Updated the definition of the NFIT table to correct the bit polarity of one flag: ACPI_NFIT_MEM_ARMED --> ACPI_NFIT_MEM_NOT_ARMED Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 101.7K Code, 27.9K Data, 129.6K Total Debug Version: 199.3K Code, 81.4K Data, 280.7K Total Previous Release: Non-Debug Version: 101.3K Code, 27.7K Data, 129.0K Total Debug Version: 198.6K Code, 80.9K Data, 279.5K Total 2) iASL Compiler/Disassembler and Tools: iASL: Improved the compile-time typechecking for operands of many of the ASL operators: -- Added an option to disable compiler operand/operator typechecking (- ot). -- For the following operators, the TermArg operands are now validated when possible to be Integer data objects: BankField, OperationRegion, DataTableRegion, Buffer, and Package. -- Store (Source, Target): Both the source and target operands are resolved and checked that the operands are both legal. For example, neither operand can be a non-data object such as a Device, Mutex, ThermalZone, etc. Note, as per the ACPI specification, the CopyObject operator can be used to store an object to any type of target object. -- Store (Source, Target): If the source is a Package object, the target must be a Package object, LocalX, ArgX, or Debug. Likewise, if the target is a Package, the source must also be a Package. -- Store (Source, Target): A warning is issued if the source and target resolve to the identical named object. -- Store (Source, ): An error is generated for the target method invocation, as this construct is not supported by the AML interpreter. -- For all ASL math and logic operators, the target operand must be a data object (Integer, String, Buffer, LocalX, ArgX, or Debug). This includes the function return value also. -- External declarations are also included in the typechecking where possible. External objects defined using the UnknownObj keyword cannot be typechecked, however. iASL and Disassembler: Added symbolic (ASL+) support for the ASL Index operator: - Legacy code: Index(PKG1, 3) - New ASL+ code: PKG1[3] This completes the ACPI 6.0 ASL+ support as it was the only operator not supported. iASL: Fixed the file suffix for the preprocessor output file (.i). Two spaces were inadvertently appended to the filename, causing file access and deletion problems on some systems. ASL Test Suite (ASLTS): Updated the master makefile to generate all possible compiler output files when building the test suite -- thus exercising these features of the compiler. These files are automatically deleted when the test suite exits. ---------------------------------------- 18 August 2015. Summary of changes for version 20150818: 1) ACPICA kernel-resident subsystem: Fix a regression for AcpiGetTableByIndex interface causing it to fail. Lv Zheng. ACPICA BZ 1186. Completed development to ensure that the ACPICA Disassembler and Debugger are fully standalone components of ACPICA. Removed cross-component dependences. Lv Zheng. The max-number-of-AML-loops is now runtime configurable (previously was compile-time only). This is essentially a loop timeout to force-abort infinite AML loops. ACPCIA BZ 1192. Debugger: Cleanup output to dump ACPI names and namepaths without any trailing underscores. Lv Zheng. ACPICA BZ 1135. Removed unnecessary conditional compilations across the Debugger and Disassembler components where entire modules could be left uncompiled. The aapits test is deprecated and has been removed from the ACPICA git tree. The test has never been completed and has not been maintained, thus becoming rather useless. ACPICA BZ 1015, 794. A batch of small changes to close bugzilla and other reports: - Remove duplicate code for _PLD processing. ACPICA BZ 1176. - Correctly cleanup after a ACPI table load failure. ACPICA BZ 1185. - iASL: Support POSIX yacc again in makefile. Jung-uk Kim. - ACPI table support: general cleanup and simplification. Lv Zheng, Bob Moore. - ACPI table support: fix for a buffer read overrun in AcpiTbFindTable. ACPICA BZ 1184. - Enhance parameter validation for DataTableRegion and LoadTable ASL/AML operators. - Debugger: Split debugger initialization/termination interfaces. Lv Zheng. - AcpiExec: Emit OemTableId for SSDTs during the load phase for table identification. - AcpiExec: Add debug message during _REG method phase during table load/init. - AcpiNames: Fix a regression where some output was missing and no longer emitted. - Debugger: General cleanup and simplification. Lv Zheng. - Disassembler: Cleanup use of several global option variables. Lv Zheng. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 101.3K Code, 27.7K Data, 129.0K Total Debug Version: 198.6K Code, 80.9K Data, 279.5K Total Previous Release: Non-Debug Version: 100.9K Code, 24.5K Data, 125.4K Total Debug Version: 197.8K Code, 81.5K Data, 279.3K Total 2) iASL Compiler/Disassembler and Tools: AcpiExec: Fixed a problem where any more than 32 ACPI tables in the XSDT were not handled properly and caused load errors. Now, properly invoke and use the ACPICA auto-reallocate mechanism for ACPI table data structures. ACPICA BZ 1188 AcpiNames: Add command-line wildcard support for ACPI table files. ACPICA BZ 1190. AcpiExec and AcpiNames: Add -l option to load ACPI tables only. For AcpiExec, this means that no control methods (like _REG/_INI/_STA) are executed during initialization. ACPICA BZ 1187, 1189. iASL/Disassembler: Implemented a prototype "listing" mode that emits AML that corresponds to each disassembled ASL statement, to simplify debugging. ACPICA BZ 1191. Debugger: Add option to the "objects" command to display a summary of the current namespace objects (Object type and count). This is displayed if the command is entered with no arguments. AcpiNames: Add -x option to specify debug level, similar to AcpiExec. ---------------------------------------- 17 July 2015. Summary of changes for version 20150717: 1) ACPICA kernel-resident subsystem: Improved the partitioning between the Debugger and Disassembler components. This allows the Debugger to be used standalone within kernel code without the Disassembler (which is used for single stepping also). This renames and moves one file, dmobject.c to dbobject.c. Lv Zheng. Debugger: Implemented a new command to trace the execution of control methods (Trace). This is especially useful for the in-kernel version of the debugger when file I/O may not be available for method trace output. See the ACPICA reference for more information. Lv Zheng. Moved all C library prototypes (used for the local versions of these functions when requested) to a new header, acclib.h Cleaned up the use of non-ANSI C library functions. These functions are implemented locally in ACPICA. Moved all such functions to a common source file, utnonansi.c Debugger: Fixed a problem with the "!!" command (get last command executed) where the debugger could enter an infinite loop and eventually crash. Removed the use of local macros that were used for some of the standard C library functions to automatically cast input parameters. This mostly affected the is* functions where the input parameter is defined to be an int. This required a few modifications to the main ACPICA source code to provide casting for these functions and eliminate possible compiler warnings for these parameters. Across the source code, added additional status/error checking to resolve issues discovered by static source code analysis tools such as Coverity. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 100.9K Code, 24.5K Data, 125.4K Total Debug Version: 197.8K Code, 81.5K Data, 279.3K Total Previous Release: Non-Debug Version: 100.6K Code, 27.6K Data, 128.2K Total Debug Version: 196.2K Code, 81.0K Data, 277.2K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a regression where the device map file feature no longer worked properly when used in conjunction with the disassembler. It only worked properly with the compiler itself. iASL: Implemented a new warning for method LocalX variables that are set but never used (similar to a C compiler such as gcc). This also applies to ArgX variables that are not defined by the parent method, and are instead (legally) used as local variables. iASL/Preprocessor: Finished the pass-through of line numbers from the preprocessor to the compiler. This ensures that compiler errors/warnings have the correct original line numbers and filenames, regardless of any #include files. iASL/Preprocessor: Fixed a couple of issues with comment handling and the pass-through of comments to the preprocessor output file (which becomes the compiler input file). Also fixed a problem with // comments that appear after a math expression. iASL: Added support for the TCPA server table to the table compiler and template generator. (The client table was already previously supported) iASL/Preprocessor: Added a permanent #define of the symbol "__IASL__" to identify the iASL compiler. Cleaned up the use of the macros NEGATIVE and POSITIVE which were defined multiple times. The new names are ACPI_SIGN_NEGATIVE and ACPI_SIGN_POSITIVE. AcpiHelp: Update to expand help messages for the iASL preprocessor directives. ---------------------------------------- 19 June 2015. Summary of changes for version 20150619: Two regressions in version 20150616 have been addressed: Fixes some problems/issues with the C library macro removal (ACPI_STRLEN, etc.) This update changes ACPICA to only use the standard headers for functions, or the prototypes for the local versions of the C library functions. Across the source code, this required some additional casts for some Clib invocations for portability. Moved all local prototypes to a new file, acclib.h Fixes several problems with recent changes to the handling of the FACS table that could cause some systems not to boot. ---------------------------------------- 16 June 2015. Summary of changes for version 20150616: 1) ACPICA kernel-resident subsystem: Across the entire ACPICA source code base, the various macros for the C library functions (such as ACPI_STRLEN, etc.) have been removed and replaced by the standard C library names (strlen, etc.) The original purpose for these macros is no longer applicable. This simplification reduces the number of macros used in the ACPICA source code significantly, improving readability and maintainability. Implemented support for a new ACPI table, the OSDT. This table, the "override" SDT, can be loaded directly by the host OS at boot time. It enables the replacement of existing namespace objects that were installed via the DSDT and/or SSDTs. The primary purpose for this is to replace buggy or incorrect ASL/AML code obtained via the BIOS. The OSDT is slated for inclusion in a future version of the ACPI Specification. Lv Zheng/Bob Moore. Added support for systems with (improperly) two FACS tables -- a "32-bit" table (via FADT 32-bit legacy field) and a "64-bit" table (via the 64-bit X field). This change will support both automatically. There continues to be systems found with this issue. This support requires a change to the AcpiSetFirmwareWakingVector interface. Also, a public global variable has been added to allow the host to select which FACS is desired (AcpiGbl_Use32BitFacsAddresses). See the ACPICA reference for more details Lv Zheng. Added a new feature to allow for systems that do not contain an FACS. Although this is already supported on hardware-reduced platforms, the feature has been extended for all platforms. The reasoning is that we do not want to abort the entire ACPICA initialization just because the system is seriously buggy and has no FACS. Fixed a problem where the GUID strings for NFIT tables (in acuuid.h) were not correctly transcribed from the ACPI specification in ACPICA version 20150515. Implemented support for the _CLS object in the AcpiGetObjectInfo external interface. Updated the definitions of the TCPA and TPM2 ACPI tables to the more recent TCG ACPI Specification, December 14, 2014. Table disassembler and compiler also updated. Note: The TCPA "server" table is not supported by the disassembler/table-compiler at this time. ACPI 6.0: Added definitions for the new GIC version field in the MADT. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 100.6K Code, 27.6K Data, 128.2K Total Debug Version: 196.2K Code, 81.0K Data, 277.2K Total Previous Release: Non-Debug Version: 99.9K Code, 27.5K Data, 127.4K Total Debug Version: 195.2K Code, 80.8K Data, 276.0K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Fixed a problem with the new symbolic operator disassembler where incorrect ASL code could be emitted in some cases for the "non- commutative" operators -- Subtract, Divide, Modulo, ShiftLeft, and ShiftRight. The actual problem cases seem to be rather unusual in common ASL code, however. David Box. Modified the linux version of acpidump to obtain ACPI tables from not just /dev/mem (which may not exist) and /sys/firmware/acpi/tables. Lv Zheng. iASL: Fixed a problem where the user preprocessor output file (.i) contained extra data that was not expected. The compiler was using this file as a temporary file and passed through #line directives in order to keep compiler error messages in sync with the input file and line number across multiple include files. The (.i) is no longer a temporary file as the compiler uses a new, different file for the original purpose. iASL: Fixed a problem where comments within the original ASL source code file were not passed through to the preprocessor output file, nor any listing files. iASL: Fixed some issues for the handling of the "#include" preprocessor directive and the similar (but not the same) "Include" ASL operator. iASL: Add support for the new OSDT in both the disassembler and compiler. iASL: Fixed a problem with the constant folding support where a Buffer object could be incorrectly generated (incorrectly formed) during a conversion to a Store() operator. AcpiHelp: Updated for new NFIT GUIDs, "External" AML opcode, and new description text for the _REV predefined name. _REV now permanently returns 2, as per the ACPI 6.0 specification. Debugger: Enhanced the output of the Debug ASL object for references produced by the Index operator. For Buffers and strings, only output the actual byte pointed to by the index. For packages, only print the single package element decoded by the index. Previously, the entire buffer/string/package was emitted. iASL/Table-compiler: Fixed a regression where the "generic" data types were no longer recognized, causing errors. ---------------------------------------- 15 May 2015. Summary of changes for version 20150515: This release implements most of ACPI 6.0 as described below. 1) ACPICA kernel-resident subsystem: Implemented runtime argument checking and return value checking for all new ACPI 6.0 predefined names. This includes: _BTH, _CR3, _DSD, _LPI, _MTL, _PRR, _RDI, _RST, _TFP, _TSN. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 99.9K Code, 27.5K Data, 127.4K Total Debug Version: 195.2K Code, 80.8K Data, 276.0K Total Previous Release: Non-Debug Version: 99.1K Code, 27.3K Data, 126.4K Total Debug Version: 192.8K Code, 79.9K Data, 272.7K Total 2) iASL Compiler/Disassembler and Tools: iASL compiler: Added compile-time support for all new ACPI 6.0 predefined names (argument count validation and return value typechecking.) iASL disassembler and table compiler: implemented support for all new ACPI 6.0 tables. This includes: DRTM, IORT, LPIT, NFIT, STAO, WPBT, XENV. iASL disassembler and table compiler: Added ACPI 6.0 changes to existing tables: FADT, MADT. iASL preprocessor: Added a new directive to enable inclusion of binary blobs into ASL code. The new directive is #includebuffer. It takes a binary file as input and emits a named ascii buffer object into the ASL code. AcpiHelp: Added support for all new ACPI 6.0 predefined names. AcpiHelp: Added a new option, -d, to display all iASL preprocessor directives. AcpiHelp: Added a new option, -t, to display all known/supported ACPI tables. ---------------------------------------- 10 April 2015. Summary of changes for version 20150410: Reverted a change introduced in version 20150408 that caused a regression in the disassembler where incorrect operator symbols could be emitted. ---------------------------------------- 08 April 2015. Summary of changes for version 20150408: 1) ACPICA kernel-resident subsystem: Permanently set the return value for the _REV predefined name. It now returns 2 (was 5). This matches other ACPI implementations. _REV will be deprecated in the future, and is now defined to be 1 for ACPI 1.0, and 2 for ACPI 2.0 and later. It should never be used to differentiate or identify operating systems. Added the "Windows 2015" string to the _OSI support. ACPICA will now return TRUE to a query with this string. Fixed several issues with the local version of the printf function. Added the C99 compiler option (-std=c99) to the Unix makefiles. Current Release: Non-Debug Version: 99.9K Code, 27.4K Data, 127.3K Total Debug Version: 195.2K Code, 80.7K Data, 275.9K Total Previous Release: Non-Debug Version: 98.8K Code, 27.3K Data, 126.1K Total Debug Version: 192.1K Code, 79.8K Data, 271.9K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented an enhancement to the constant folding feature to transform the parse tree to a simple Store operation whenever possible: Add (2, 3, X) ==> is converted to: Store (5, X) X = 2 + 3 ==> is converted to: Store (5, X) Updated support for the SLIC table (Software Licensing Description Table) in both the Data Table compiler and the disassembler. The SLIC table support now conforms to "Microsoft Software Licensing Tables (SLIC and MSDM). November 29, 2011. Copyright 2011 Microsoft". Note: Any SLIC data following the ACPI header is now defined to be "Proprietary Data", and as such, can only be entered or displayed as a hex data block. Implemented full support for the MSDM table as described in the document above. Note: The format of MSDM is similar to SLIC. Any MSDM data following the ACPI header is defined to be "Proprietary Data", and can only be entered or displayed as a hex data block. Implemented the -Pn option for the iASL Table Compiler (was only implemented for the ASL compiler). This option disables the iASL preprocessor. Disassembler: For disassembly of Data Tables, added a comment field around the Ascii equivalent data that is emitted as part of the "Raw Table Data" block. This prevents the iASL Preprocessor from possible confusion if/when the table is compiled. Disassembler: Added an option (-df) to force the disassembler to assume that the table being disassembled contains valid AML. This feature is useful for disassembling AML files that contain ACPI signatures other than DSDT or SSDT (such as OEMx or other signatures). Changes for the EFI version of the tools: 1) Fixed a build error/issue 2) Fixed a cast warning iASL: Fixed a path issue with the __FILE__ operator by making the directory prefix optional within the internal SplitInputFilename function. Debugger: Removed some unused global variables. Tests: Updated the makefile for proper generation of the AAPITS suite. ---------------------------------------- 04 February 2015. Summary of changes for version 20150204: ACPICA kernel-resident subsystem: Updated all ACPICA copyrights and signons to 2014. Added the 2014 copyright to all module headers and signons, including the standard Linux header. This affects virtually every file in the ACPICA core subsystem, iASL compiler, all ACPICA utilities, and the test suites. Events: Introduce ACPI_GPE_DISPATCH_RAW_HANDLER to fix GPE storm issues. A raw gpe handling mechanism was created to allow better handling of GPE storms that aren't easily managed by the normal handler. The raw handler allows disabling/renabling of the the GPE so that interrupt storms can be avoided in cases where events cannot be timely serviced. In this scenario, handlers should use the AcpiSetGpe() API to disable/enable the GPE. This API will leave the reference counts undisturbed, thereby preventing unintentional clearing of the GPE when the intent in only to temporarily disable it. Raw handlers allow enabling and disabling of a GPE by removing GPE register locking. As such, raw handlers much provide their own locks while using GPE API's to protect access to GPE data structures. Lv Zheng Events: Always modify GPE registers under the GPE lock. Applies GPE lock around AcpiFinishGpe() to protect access to GPE register values. Reported as bug by joe.liu@apple.com. Unix makefiles: Separate option to disable optimizations and _FORTIFY_SOURCE. This change removes the _FORTIFY_SOURCE flag from the NOOPT disable option and creates a separate flag (NOFORTIFY) for this purpose. Some toolchains may define _FORTIFY_SOURCE which leads redefined errors when building ACPICA. This allows disabling the option without also having to disable optimazations. David Box Current Release: Non-Debug Version: 101.7K Code, 27.9K Data, 129.6K Total Debug Version: 199.2K Code, 82.4K Data, 281.6K Total -- -------------------------------------- 07 November 2014. Summary of changes for version 20141107: This release is available at https://acpica.org/downloads This release introduces and implements language extensions to ASL that provide support for symbolic ("C-style") operators and expressions. These language extensions are known collectively as ASL+. 1) iASL Compiler/Disassembler and Tools: Disassembler: Fixed a problem with disassembly of the UartSerialBus macro. Changed "StopBitsNone" to the correct "StopBitsZero". David E. Box. Disassembler: Fixed the Unicode macro support to add escape sequences. All non-printable ASCII values are emitted as escape sequences, as well as the standard escapes for quote and backslash. Ensures that the disassembled macro can be correctly recompiled. iASL: Added Printf/Fprintf macros for formatted output. These macros are translated to existing AML Concatenate and Store operations. Printf writes to the ASL Debug object. Fprintf allows the specification of an ASL name as the target. Only a single format specifier is required, %o, since the AML interpreter dynamically converts objects to the required type. David E. Box. (old) Store (Concatenate (Concatenate (Concatenate (Concatenate (Concatenate (Concatenate (Concatenate ("", Arg0), ": Unexpected value for "), Arg1), ", "), Arg2), " at line "), Arg3), Debug) (new) Printf ("%o: Unexpected value for %o, %o at line %o", Arg0, Arg1, Arg2, Arg3) (old) Store (Concatenate (Concatenate (Concatenate (Concatenate ("", Arg1), ": "), Arg0), " Successful"), STR1) (new) Fprintf (STR1, "%o: %o Successful", Arg1, Arg0) iASL: Added debug options (-bp, -bt) to dynamically prune levels of the ASL parse tree before the AML code is generated. This allows blocks of ASL code to be removed in order to help locate and identify problem devices and/or code. David E. Box. AcpiExec: Added support (-fi) for an optional namespace object initialization file. This file specifies initial values for namespace objects as necessary for debugging and testing different ASL code paths that may be taken as a result of BIOS options. 2) Overview of symbolic operator support for ASL (ASL+) ------------------------------------------------------- As an extension to the ASL language, iASL implements support for symbolic (C-style) operators for math and logical expressions. This can greatly simplify ASL code as well as improve both readability and maintainability. These language extensions can exist concurrently with all legacy ASL code and expressions. The symbolic extensions are 100% compatible with existing AML interpreters, since no new AML opcodes are created. To implement the extensions, the iASL compiler transforms the symbolic expressions into the legacy ASL/AML equivalents at compile time. Full symbolic expressions are supported, along with the standard C precedence and associativity rules. Full disassembler support for the symbolic expressions is provided, and creates an automatic migration path for existing ASL code to ASL+ code via the disassembly process. By default, the disassembler now emits ASL+ code with symbolic expressions. An option (-dl) is provided to force the disassembler to emit legacy ASL code if desired. Below is the complete list of the currently supported symbolic operators with examples. See the iASL User Guide for additional information. ASL+ Syntax Legacy ASL Equivalent ----------- --------------------- // Math operators Z = X + Y Add (X, Y, Z) Z = X - Y Subtract (X, Y, Z) Z = X * Y Multiply (X, Y, Z) Z = X / Y Divide (X, Y, , Z) Z = X % Y Mod (X, Y, Z) Z = X << Y ShiftLeft (X, Y, Z) Z = X >> Y ShiftRight (X, Y, Z) Z = X & Y And (X, Y, Z) Z = X | Y Or (X, Y, Z) Z = X ^ Y Xor (X, Y, Z) Z = ~X Not (X, Z) X++ Increment (X) X-- Decrement (X) // Logical operators (X == Y) LEqual (X, Y) (X != Y) LNotEqual (X, Y) (X < Y) LLess (X, Y) (X > Y) LGreater (X, Y) (X <= Y) LLessEqual (X, Y) (X >= Y) LGreaterEqual (X, Y) (X && Y) LAnd (X, Y) (X || Y) LOr (X, Y) (!X) LNot (X) // Assignment and compound assignment operations X = Y Store (Y, X) X += Y Add (X, Y, X) X -= Y Subtract (X, Y, X) X *= Y Multiply (X, Y, X) X /= Y Divide (X, Y, , X) X %= Y Mod (X, Y, X) X <<= Y ShiftLeft (X, Y, X) X >>= Y ShiftRight (X, Y, X) X &= Y And (X, Y, X) X |= Y Or (X, Y, X) X ^= Y Xor (X, Y, X) 3) ASL+ Examples: ----------------- Legacy ASL: If (LOr (LOr (LEqual (And (R510, 0x03FB), 0x02E0), LEqual ( And (R520, 0x03FB), 0x02E0)), LOr (LEqual (And (R530, 0x03FB), 0x02E0), LEqual (And (R540, 0x03FB), 0x02E0)))) { And (MEMB, 0xFFFFFFF0, SRMB) Store (MEMB, Local2) Store (PDBM, Local1) And (PDBM, 0xFFFFFFFFFFFFFFF9, PDBM) Store (SRMB, MEMB) Or (PDBM, 0x02, PDBM) } ASL+ version: If (((R510 & 0x03FB) == 0x02E0) || ((R520 & 0x03FB) == 0x02E0) || ((R530 & 0x03FB) == 0x02E0) || ((R540 & 0x03FB) == 0x02E0)) { SRMB = (MEMB & 0xFFFFFFF0) Local2 = MEMB Local1 = PDBM PDBM &= 0xFFFFFFFFFFFFFFF9 MEMB = SRMB PDBM |= 0x02 } Legacy ASL: Store (0x1234, Local1) Multiply (Add (Add (Local1, TEST), 0x20), Local2, Local3) Multiply (Local2, Add (Add (Local1, TEST), 0x20), Local3) Add (Local1, Add (TEST, Multiply (0x20, Local2)), Local3) Store (Index (PKG1, 0x03), Local6) Store (Add (Local3, Local2), Debug) Add (Local1, 0x0F, Local2) Add (Local1, Multiply (Local2, Local3), Local2) Multiply (Add (Add (Local1, TEST), 0x20), ToBCD (Local1), Local3) ASL+ version: Local1 = 0x1234 Local3 = (((Local1 + TEST) + 0x20) * Local2) Local3 = (Local2 * ((Local1 + TEST) + 0x20)) Local3 = (Local1 + (TEST + (0x20 * Local2))) Local6 = Index (PKG1, 0x03) Debug = (Local3 + Local2) Local2 = (Local1 + 0x0F) Local2 = (Local1 + (Local2 * Local3)) Local3 = (((Local1 + TEST) + 0x20) * ToBCD (Local1)) ---------------------------------------- 26 September 2014. Summary of changes for version 20140926: 1) ACPICA kernel-resident subsystem: Updated the GPIO operation region handler interface (GeneralPurposeIo). In order to support GPIO Connection objects with multiple pins, along with the related Field objects, the following changes to the interface have been made: The Address is now defined to be the offset in bits of the field unit from the previous invocation of a Connection. It can be viewed as a "Pin Number Index" into the connection resource descriptor. The BitWidth is the exact bit width of the field. It is usually one bit, but not always. See the ACPICA reference guide (section 8.8.6.2.1) for additional information and examples. GPE support: During ACPICA/GPE initialization, ensure that all GPEs with corresponding _Lxx/_Exx methods are disabled (they may have been enabled by the firmware), so that they cannot fire until they are enabled via AcpiUpdateAllGpes. Rafael J. Wysocki. Added a new return flag for the Event/GPE status interfaces -- AcpiGetEventStatus and AcpiGetGpeStatus. The new ACPI_EVENT_FLAGS_HAS_HANDLER flag is used to indicate that the event or GPE currently has a handler associated with it, and can thus actually affect the system. Lv Zheng. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 99.1K Code, 27.3K Data, 126.4K Total Debug Version: 192.8K Code, 79.9K Data, 272.7K Total Previous Release: Non-Debug Version: 98.8K Code, 27.3K Data, 126.1K Total Debug Version: 192.1K Code, 79.8K Data, 271.9K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a memory allocation/free regression introduced in 20140828 that could cause the compiler to crash. This was introduced inadvertently during the effort to eliminate compiler memory leaks. ACPICA BZ 1111, 1113. iASL: Removed two error messages that have been found to create false positives, until they can be fixed and fully validated (ACPICA BZ 1112): 1) Illegal forward reference within a method 2) Illegal reference across two methods iASL: Implemented a new option (-lm) to create a hardware mapping file that summarizes all GPIO, I2C, SPI, and UART connections. This option works for both the compiler and disassembler. See the iASL compiler user guide for additional information and examples (section 6.4.6). AcpiDump: Added support for the version 1 (ACPI 1.0) RSDP in addition to version 2. This corrects the AE_BAD_HEADER exception seen on systems with a version 1 RSDP. Lv Zheng ACPICA BZ 1097. AcpiExec: For Unix versions, don't attempt to put STDIN into raw mode unless STDIN is actually a terminal. Assists with batch-mode processing. ACPICA BZ 1114. Disassembler/AcpiHelp: Added another large group of recognized _HID values. ---------------------------------------- 28 August 2014. Summary of changes for version 20140828: 1) ACPICA kernel-resident subsystem: Fixed a problem related to the internal use of the Timer() operator where a 64-bit divide could cause an attempted link to a double-precision math library. This divide is not actually necessary, so the code was restructured to eliminate it. Lv Zheng. ACPI 5.1: Added support for the runtime validation of the _DSD package (similar to the iASL support). ACPI 5.1/Headers: Added support for the GICC affinity subtable to the SRAT table. Hanjun Guo . Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 98.8K Code, 27.3K Data, 126.1K Total Debug Version: 192.1K Code, 79.8K Data, 271.9K Total Previous Release: Non-Debug Version: 98.7K Code, 27.3K Data, 126.0K Total1 Debug Version: 192.0K Code, 79.7K Data, 271.7K Total 2) iASL Compiler/Disassembler and Tools: AcpiExec: Fixed a problem on unix systems where the original terminal state was not always properly restored upon exit. Seen when using the -v option. ACPICA BZ 1104. iASL: Fixed a problem with the validation of the ranges/length within the Memory24 resource descriptor. There was a boundary condition when the range was equal to the (length -1) caused by the fact that these values are defined in 256-byte blocks, not bytes. ACPICA BZ 1098 Disassembler: Fixed a problem with the GpioInt descriptor interrupt polarity flags. The flags are actually 2 bits, not 1, and the "ActiveBoth" keyword is now supported properly. ACPI 5.1: Added the GICC affinity subtable to the SRAT table. Supported in the disassembler, data table compiler, and table template generator. iASL: Added a requirement for Device() objects that one of either a _HID or _ADR must exist within the scope of a Device, as per the ACPI specification. Remove a similar requirement that was incorrectly in place for the _DSD object. iASL: Added error detection for illegal named references within control methods that would cause runtime failures. Now trapped as errors are: 1) References to objects within a non-parent control method. 2) Forward references (within a method) -- for control methods, AML interpreters use a one-pass parse of control methods. ACPICA BZ 1008. iASL: Added error checking for dependencies related to the _PSx power methods. ACPICA BZ 1029. 1) For _PS0, one of these must exist within the same scope: _PS1, _PS2, _PS3. 2) For _PS1, _PS2, and PS3: A _PS0 object must exist within the same scope. iASL and table compiler: Cleanup miscellaneous memory leaks by fully deploying the existing object and string caches and adding new caches for the table compiler. iASL: Split the huge parser source file into multiple subfiles to improve manageability. Generation now requires the M4 macro preprocessor, which is part of the Bison distribution on both unix and windows platforms. AcpiSrc: Fixed and removed all extraneous warnings generated during entire ACPICA source code scan and/or conversion. ---------------------------------------- 24 July 2014. Summary of changes for version 20140724: The ACPI 5.1 specification has been released and is available at: http://uefi.org/specs/access 0) ACPI 5.1 support in ACPICA: ACPI 5.1 is fully supported in ACPICA as of this release. New predefined names. Support includes iASL and runtime ACPICA validation. _CCA (Cache Coherency Attribute). _DSD (Device-Specific Data). David Box. Modifications to existing ACPI tables. Support includes headers, iASL Data Table compiler, disassembler, and the template generator. FADT - New fields and flags. Graeme Gregory. GTDT - One new subtable and new fields. Tomasz Nowicki. MADT - Two new subtables. Tomasz Nowicki. PCCT - One new subtable. Miscellaneous. New notification type for System Resource Affinity change events. 1) ACPICA kernel-resident subsystem: Fixed a regression introduced in 20140627 where a fault can happen during the deletion of Alias AML namespace objects. The problem affected both the core ACPICA and the ACPICA tools including iASL and AcpiExec. Implemented a new GPE public interface, AcpiMarkGpeForWake. Provides a simple mechanism to enable wake GPEs that have no associated handler or control method. Rafael Wysocki. Updated the AcpiEnableGpe interface to disallow the enable if there is no handler or control method associated with the particular GPE. This will help avoid meaningless GPEs and even GPE floods. Rafael Wysocki. Updated GPE handling and dispatch by disabling the GPE before clearing the status bit for edge-triggered GPEs. Lv Zheng. Added Timer() support to the AML Debug object. The current timer value is now displayed with each invocation of (Store to) the debug object to enable simple generation of execution times for AML code (method execution for example.) ACPICA BZ 1093. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 98.7K Code, 27.3K Data, 126.0K Total Debug Version: 192.0K Code, 79.7K Data, 271.7K Total Previous Release: Non-Debug Version: 98.7K Code, 27.2K Data, 125.9K Total Debug Version: 191.7K Code, 79.6K Data, 271.3K Total 2) iASL Compiler/Disassembler and Tools: Fixed an issue with the recently added local printf implementation, concerning width/precision specifiers that could cause incorrect output. Lv Zheng. ACPICA BZ 1094. Disassembler: Added support to detect buffers that contain UUIDs and disassemble them to an invocation of the ToUUID operator. Also emit commented descriptions of known ACPI-related UUIDs. AcpiHelp: Added support to display known ACPI-related UUIDs. New option, -u. Adds three new files. iASL: Update table compiler and disassembler for DMAR table changes that were introduced in September 2013. With assistance by David Woodhouse. ---------------------------------------- 27 June 2014. Summary of changes for version 20140627: 1) ACPICA kernel-resident subsystem: Formatted Output: Implemented local versions of standard formatted output utilities such as printf, etc. Over time, it has been discovered that there are in fact many portability issues with printf, and the addition of this feature will fix/prevent these issues once and for all. Some known issues are summarized below: 1) Output of 64-bit values is not portable. For example, UINT64 is %ull for the Linux kernel and is %uI64 for some MSVC versions. 2) Invoking printf consistently in a manner that is portable across both 32-bit and 64-bit platforms is difficult at best in many situations. 3) The output format for pointers varies from system to system (leading zeros especially), and leads to inconsistent output from ACPICA across platforms. 4) Certain platform-specific printf formats may conflict with ACPICA use. 5) If there is no local C library available, ACPICA now has local support for printf. -- To address these printf issues in a complete manner, ACPICA now directly implements a small subset of printf format specifiers, only those that it requires. Adds a new file, utilities/utprint.c. Lv Zheng. Implemented support for ACPICA generation within the EFI environment. Initially, the AcpiDump utility is supported in the UEFI shell environment. Lv Zheng. Added a new external interface, AcpiLogError, to improve ACPICA portability. This allows the host to redirect error messages from the ACPICA utilities. Lv Zheng. Added and deployed new OSL file I/O interfaces to improve ACPICA portability: AcpiOsOpenFile AcpiOsCloseFile AcpiOsReadFile AcpiOsWriteFile AcpiOsGetFileOffset AcpiOsSetFileOffset There are C library implementations of these functions in the new file service_layers/oslibcfs.c -- however, the functions can be implemented by the local host in any way necessary. Lv Zheng. Implemented a mechanism to disable/enable ACPI table checksum validation at runtime. This can be useful when loading tables very early during OS initialization when it may not be possible to map the entire table in order to compute the checksum. Lv Zheng. Fixed a buffer allocation issue for the Generic Serial Bus support. Originally, a fixed buffer length was used. This change allows for variable-length buffers based upon the protocol indicated by the field access attributes. Reported by Lan Tianyu. Lv Zheng. Fixed a problem where an object detached from a namespace node was not properly terminated/cleared and could cause a circular list problem if reattached. ACPICA BZ 1063. David Box. Fixed a possible recursive lock acquisition in hwregs.c. Rakib Mullick. Fixed a possible memory leak in an error return path within the function AcpiUtCopyIobjectToIobject. ACPICA BZ 1087. Colin Ian King. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 98.7K Code, 27.2K Data, 125.9K Total Debug Version: 191.7K Code, 79.6K Data, 271.3K Total Previous Release: Non-Debug Version: 96.8K Code, 27.2K Data, 124.0K Total Debug Version: 189.5K Code, 79.7K Data, 269.2K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Add dump of ASCII equivalent text within a comment at the end of each line of the output for the Buffer() ASL operator. AcpiDump: Miscellaneous changes: Fixed repetitive table dump in -n mode. For older EFI platforms, use the ACPI 1.0 GUID during RSDP search if the ACPI 2.0 GUID fails. iASL: Fixed a problem where the compiler could fault if incorrectly given an acpidump output file as input. ACPICA BZ 1088. David Box. AcpiExec/AcpiNames: Fixed a problem where these utilities could fault if they are invoked without any arguments. Debugger: Fixed a possible memory leak in an error return path. ACPICA BZ 1086. Colin Ian King. Disassembler: Cleaned up a block of code that extracts a parent Op object. Added a comment that explains that the parent is guaranteed to be valid in this case. ACPICA BZ 1069. ---------------------------------------- 24 April 2014. Summary of changes for version 20140424: 1) ACPICA kernel-resident subsystem: Implemented support to skip/ignore NULL address entries in the RSDT/XSDT. Some of these tables are known to contain a trailing NULL entry. Lv Zheng. Removed an extraneous error message for the case where there are a large number of system GPEs (> 124). This was the "32-bit FADT register is too long to convert to GAS struct" message, which is irrelevant for GPEs since the GPEx_BLK_LEN fields of the FADT are always used instead of the (limited capacity) GAS bit length. Also, several changes to ensure proper support for GPE numbers > 255, where some "GPE number" fields were 8-bits internally. Implemented and deployed additional configuration support for the public ACPICA external interfaces. Entire classes of interfaces can now be easily modified or configured out, replaced by stubbed inline functions by default. Lv Zheng. Moved all public ACPICA runtime configuration globals to the public ACPICA external interface file for convenience. Also, removed some obsolete/unused globals. See the file acpixf.h. Lv Zheng. Documentation: Added a new section to the ACPICA reference describing the maximum number of GPEs that can be supported by the FADT-defined GPEs in block zero and one. About 1200 total. See section 4.4.1 of the ACPICA reference. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 96.8K Code, 27.2K Data, 124.0K Total Debug Version: 189.5K Code, 79.7K Data, 269.2K Total Previous Release: Non-Debug Version: 97.0K Code, 27.2K Data, 124.2K Total Debug Version: 189.7K Code, 79.5K Data, 269.2K Total 2) iASL Compiler/Disassembler and Tools: iASL and disassembler: Add full support for the LPIT table (Low Power Idle Table). Includes support in the disassembler, data table compiler, and template generator. AcpiDump utility: 1) Add option to force the use of the RSDT (over the XSDT). 2) Improve validation of the RSDP signature (use 8 chars instead of 4). iASL: Add check for predefined packages that are too large. For predefined names that contain subpackages, check if each subpackage is too large. (Check for too small already exists.) Debugger: Updated the GPE command (which simulates a GPE by executing the GPE code paths in ACPICA). The GPE device is now optional, and defaults to the GPE 0/1 FADT-defined blocks. Unix application OSL: Update line-editing support. Add additional error checking and take care not to reset terminal attributes on exit if they were never set. This should help guarantee that the terminal is always left in the previous state on program exit. ---------------------------------------- 25 March 2014. Summary of changes for version 20140325: 1) ACPICA kernel-resident subsystem: Updated the auto-serialize feature for control methods. This feature automatically serializes all methods that create named objects in order to prevent runtime errors. The update adds support to ignore the currently executing AML SyncLevel when invoking such a method, in order to prevent disruption of any existing SyncLevel priorities that may exist in the AML code. Although the use of SyncLevels is relatively rare, this change fixes a regression where an AE_AML_MUTEX_ORDER exception can appear on some machines starting with the 20140214 release. Added a new external interface to allow the host to install ACPI tables very early, before the namespace is even created. AcpiInstallTable gives the host additional flexibility for ACPI table management. Tables can be installed directly by the host as if they had originally appeared in the XSDT/RSDT. Installed tables can be SSDTs or other ACPI data tables (anything except the DSDT and FACS). Adds a new file, tbdata.c, along with additional internal restructuring and cleanup. See the ACPICA Reference for interface details. Lv Zheng. Added validation of the checksum for all incoming dynamically loaded tables (via external interfaces or via AML Load/LoadTable operators). Lv Zheng. Updated the use of the AcpiOsWaitEventsComplete interface during Notify and GPE handler removal. Restructured calls to eliminate possible race conditions. Lv Zheng. Added a warning for the use/execution of the ASL/AML Unload (table) operator. This will help detect and identify machines that use this operator if and when it is ever used. This operator has never been seen in the field and the usage model and possible side-effects of the drastic runtime action of a full table removal are unknown. Reverted the use of #pragma push/pop which was introduced in the 20140214 release. It appears that push and pop are not implemented by enough compilers to make the use of this feature feasible for ACPICA at this time. However, these operators may be deployed in a future ACPICA release. Added the missing EXPORT_SYMBOL macros for the install and remove SCI handler interfaces. Source code generation: 1) Disabled the use of the "strchr" macro for the gcc-specific generation. For some versions of gcc, this macro can periodically expose a compiler bug which in turn causes compile-time error(s). 2) Added support for PPC64 compilation. Colin Ian King. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 97.0K Code, 27.2K Data, 124.2K Total Debug Version: 189.7K Code, 79.5K Data, 269.2K Total Previous Release: Non-Debug Version: 96.5K Code, 27.2K Data, 123.7K Total Debug Version: 188.6K Code, 79.0K Data, 267.6K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Added several new features to improve the readability of the resulting ASL code. Extra information is emitted within comment fields in the ASL code: 1) Known _HID/_CID values are decoded to descriptive text. 2) Standard values for the Notify() operator are decoded to descriptive text. 3) Target operands are expanded to full pathnames (in a comment) when possible. Disassembler: Miscellaneous updates for extern() handling: 1) Abort compiler if file specified by -fe option does not exist. 2) Silence unnecessary warnings about argument count mismatches. 3) Update warning messages concerning unresolved method externals. 4) Emit "UnknownObj" keyword for externals whose type cannot be determined. AcpiHelp utility: 1) Added the -a option to display both the ASL syntax and the AML encoding for an input ASL operator. This effectively displays all known information about an ASL operator with one AcpiHelp invocation. 2) Added substring match support (similar to a wildcard) for the -i (_HID/PNP IDs) option. iASL/Disassembler: Since this tool does not yet support execution on big- endian machines, added detection of endianness and an error message if execution is attempted on big-endian. Support for big-endian within iASL is a feature that is on the ACPICA to-be-done list. AcpiBin utility: 1) Remove option to extract binary files from an acpidump; this function is made obsolete by the AcpiXtract utility. 2) General cleanup of open files and allocated buffers. ---------------------------------------- 14 February 2014. Summary of changes for version 20140214: 1) ACPICA kernel-resident subsystem: Implemented a new mechanism to proactively prevent problems with ill- behaved reentrant control methods that create named ACPI objects. This behavior is illegal as per the ACPI specification, but is nonetheless frequently seen in the field. Previously, this could lead to an AE_ALREADY_EXISTS exception if the method was actually entered by more than one thread. This new mechanism detects such methods at table load time and marks them "serialized" to prevent reentrancy. A new global option, AcpiGbl_AutoSerializeMethods, has been added to disable this feature if desired. This mechanism and global option obsoletes and supersedes the previous AcpiGbl_SerializeAllMethods option. Added the "Windows 2013" string to the _OSI support. ACPICA will now respond TRUE to _OSI queries with this string. It is the stated policy of ACPICA to add new strings to the _OSI support as soon as possible after they are defined. See the full ACPICA _OSI policy which has been added to the utilities/utosi.c file. Hardened/updated the _PRT return value auto-repair code: 1) Do not abort the repair on a single subpackage failure, continue to check all subpackages. 2) Add check for the minimum subpackage length (4). 3) Properly handle extraneous NULL package elements. Added support to avoid the possibility of infinite loops when traversing object linked lists. Never allow an infinite loop, even in the face of corrupted object lists. ACPICA headers: Deployed the use of #pragma pack(push) and #pragma pack(pop) directives to ensure that the ACPICA headers are independent of compiler settings or other host headers. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 96.5K Code, 27.2K Data, 123.7K Total Debug Version: 188.6K Code, 79.0K Data, 267.6K Total Previous Release: Non-Debug Version: 96.2K Code, 27.0K Data, 123.2K Total Debug Version: 187.5K Code, 78.3K Data, 265.8K Total 2) iASL Compiler/Disassembler and Tools: iASL/Table-compiler: Fixed a problem with support for the SPMI table. The first reserved field was incorrectly forced to have a value of zero. This change correctly forces the field to have a value of one. ACPICA BZ 1081. Debugger: Added missing support for the "Extra" and "Data" subobjects when displaying object data. Debugger: Added support to display entire object linked lists when displaying object data. iASL: Removed the obsolete -g option to obtain ACPI tables from the Windows registry. This feature has been superseded by the acpidump utility. ---------------------------------------- 14 January 2014. Summary of changes for version 20140114: 1) ACPICA kernel-resident subsystem: Updated all ACPICA copyrights and signons to 2014. Added the 2014 copyright to all module headers and signons, including the standard Linux header. This affects virtually every file in the ACPICA core subsystem, iASL compiler, all ACPICA utilities, and the test suites. Improved parameter validation for AcpiInstallGpeBlock. Added the following checks: 1) The incoming device handle refers to type ACPI_TYPE_DEVICE. 2) There is not already a GPE block attached to the device. Likewise, with AcpiRemoveGpeBlock, ensure that the incoming object is a device. Correctly support "references" in the ACPI_OBJECT. This change fixes the support to allow references (namespace nodes) to be passed as arguments to control methods via the evaluate object interface. This is probably most useful for testing purposes, however. Improved support for 32/64 bit physical addresses in printf()-like output. This change improves the support for physical addresses in printf debug statements and other output on both 32-bit and 64-bit hosts. It consistently outputs the appropriate number of bytes for each host. The %p specifier is unsatisfactory since it does not emit uniform output on all hosts/clib implementations (on some, leading zeros are not supported, leading to difficult-to-read output). Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 96.2K Code, 27.0K Data, 123.2K Total Debug Version: 187.5K Code, 78.3K Data, 265.8K Total Previous Release: Non-Debug Version: 96.1K Code, 27.0K Data, 123.1K Total Debug Version: 185.6K Code, 77.3K Data, 262.9K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fix a possible fault when using the Connection() operator. Fixes a problem if the parent Field definition for the Connection operator refers to an operation region that does not exist. ACPICA BZ 1064. AcpiExec: Load of local test tables is now optional. The utility has the capability to load some various tables to test features of ACPICA. However, there are enough of them that the output of the utility became confusing. With this change, only the required local tables are displayed (RSDP, XSDT, etc.) along with the actual tables loaded via the command line specification. This makes the default output simler and easier to understand. The -el command line option restores the original behavior for testing purposes. AcpiExec: Added support for overlapping operation regions. This change expands the simulation of operation regions by supporting regions that overlap within the given address space. Supports SystemMemory and SystemIO. ASLTS test suite updated also. David Box. ACPICA BZ 1031. AcpiExec: Added region handler support for PCI_Config and EC spaces. This allows AcpiExec to simulate these address spaces, similar to the current support for SystemMemory and SystemIO. Debugger: Added new command to read/write/compare all namespace objects. The command "test objects" will exercise the entire namespace by writing new values to each data object, and ensuring that the write was successful. The original value is then restored and verified. Debugger: Added the "test predefined" command. This change makes this test public and puts it under the new "test" command. The test executes each and every predefined name within the current namespace. ---------------------------------------- 18 December 2013. Summary of changes for version 20131218: Global note: The ACPI 5.0A specification was released this month. There are no changes needed for ACPICA since this release of ACPI is an errata/clarification release. The specification is available at acpi.info. 1) ACPICA kernel-resident subsystem: Added validation of the XSDT root table if it is present. Some older platforms contain an XSDT that is ill-formed or otherwise invalid (such as containing some or all entries that are NULL pointers). This change adds a new function to validate the XSDT before actually using it. If the XSDT is found to be invalid, ACPICA will now automatically fall back to using the RSDT instead. Original implementation by Zhao Yakui. Ported to ACPICA and enhanced by Lv Zheng and Bob Moore. Added a runtime option to ignore the XSDT and force the use of the RSDT. This change adds a runtime option that will force ACPICA to use the RSDT instead of the XSDT (AcpiGbl_DoNotUseXsdt). Although the ACPI spec requires that an XSDT be used instead of the RSDT, the XSDT has been found to be corrupt or ill-formed on some machines. Lv Zheng. Added a runtime option to favor 32-bit FADT register addresses over the 64-bit addresses. This change adds an option to favor 32-bit FADT addresses when there is a conflict between the 32-bit and 64-bit versions of the same register. The default behavior is to use the 64-bit version in accordance with the ACPI specification. This can now be overridden via the AcpiGbl_Use32BitFadtAddresses flag. ACPICA BZ 885. Lv Zheng. During the change above, the internal "Convert FADT" and "Verify FADT" functions have been merged to simplify the code, making it easier to understand and maintain. ACPICA BZ 933. Improve exception reporting and handling for GPE block installation. Return an actual status from AcpiEvGetGpeXruptBlock and don't clobber the status when exiting AcpiEvInstallGpeBlock. ACPICA BZ 1019. Added helper macros to extract bus/segment numbers from the HEST table. This change adds two macros to extract the encoded bus and segment numbers from the HEST Bus field - ACPI_HEST_BUS and ACPI_HEST_SEGMENT. Betty Dall Removed the unused ACPI_FREE_BUFFER macro. This macro is no longer used by ACPICA. It is not a public macro, so it should have no effect on existing OSV code. Lv Zheng. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 96.1K Code, 27.0K Data, 123.1K Total Debug Version: 185.6K Code, 77.3K Data, 262.9K Total Previous Release: Non-Debug Version: 95.9K Code, 27.0K Data, 122.9K Total Debug Version: 185.1K Code, 77.2K Data, 262.3K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Improved pathname support for emitted External() statements. This change adds full pathname support for external names that have been resolved internally by the inclusion of additional ACPI tables (via the iASL -e option). Without this change, the disassembler can emit multiple externals for the same object, or it become confused when the Scope() operator is used on an external object. Overall, greatly improves the ability to actually recompile the emitted ASL code when objects a referenced across multiple ACPI tables. Reported by Michael Tsirkin (mst@redhat.com). Tests/ASLTS: Updated functional control suite to execute with no errors. David Box. Fixed several errors related to the testing of the interpreter slack mode. Lv Zheng. iASL: Added support to detect names that are declared within a control method, but are unused (these are temporary names that are only valid during the time the method is executing). A remark is issued for these cases. ACPICA BZ 1022. iASL: Added full support for the DBG2 table. Adds full disassembler, table compiler, and template generator support for the DBG2 table (Debug Port 2 table). iASL: Added full support for the PCCT table, update the table definition. Updates the PCCT table definition in the actbl3.h header and adds table compiler and template generator support. iASL: Added an option to emit only error messages (no warnings/remarks). The -ve option will enable only error messages, warnings and remarks are suppressed. This can simplify debugging when only the errors are important, such as when an ACPI table is disassembled and there are many warnings and remarks -- but only the actual errors are of real interest. Example ACPICA code (source/tools/examples): Updated the example code so that it builds to an actual working program, not just example code. Added ACPI tables and execution of an example control method in the DSDT. Added makefile support for Unix generation. ---------------------------------------- 15 November 2013. Summary of changes for version 20131115: This release is available at https://acpica.org/downloads 1) ACPICA kernel-resident subsystem: Resource Manager: Fixed loop termination for the "get AML length" function. The loop previously had an error termination on a NULL resource pointer, which can never happen since the loop simply increments a valid resource pointer. This fix changes the loop to terminate with an error on an invalid end-of-buffer condition. The problem can be seen as an infinite loop by callers to AcpiSetCurrentResources with an invalid or corrupted resource descriptor, or a resource descriptor that is missing an END_TAG descriptor. Reported by Dan Carpenter . Lv Zheng, Bob Moore. Table unload and ACPICA termination: Delete all attached data objects during namespace node deletion. This fix updates namespace node deletion to delete the entire list of attached objects (attached via AcpiAttachObject) instead of just one of the attached items. ACPICA BZ 1024. Tomasz Nowicki (tomasz.nowicki@linaro.org). ACPICA termination: Added support to delete all objects attached to the root namespace node. This fix deletes any and all objects that have been attached to the root node via AcpiAttachData. Previously, none of these objects were deleted. Reported by Tomasz Nowicki. ACPICA BZ 1026. Debug output: Do not emit the function nesting level for the in-kernel build. The nesting level is really only useful during a single-thread execution. Therefore, only enable this output for the AcpiExec utility. Also, only emit the thread ID when executing under AcpiExec (Context switches are still always detected and a message is emitted). ACPICA BZ 972. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 95.9K Code, 27.0K Data, 122.9K Total Debug Version: 185.1K Code, 77.2K Data, 262.3K Total Previous Release: Non-Debug Version: 95.8K Code, 27.0K Data, 122.8K Total Debug Version: 185.2K Code, 77.2K Data, 262.4K Total 2) iASL Compiler/Disassembler and Tools: AcpiExec/Unix-OSL: Use instead of . This is the correct portable POSIX header for terminal control functions. Disassembler: Fixed control method invocation issues related to the use of the CondRefOf() operator. The problem is seen in the disassembly where control method invocations may not be disassembled properly if the control method name has been used previously as an argument to CondRefOf. The solution is to not attempt to emit an external declaration for the CondRefOf target (it is not necessary in the first place). This prevents disassembler object type confusion. ACPICA BZ 988. Unix Makefiles: Added an option to disable compiler optimizations and the _FORTIFY_SOURCE flag. Some older compilers have problems compiling ACPICA with optimizations (reportedly, gcc 4.4 for example). This change adds a command line option for make (NOOPT) that disables all compiler optimizations and the _FORTIFY_SOURCE compiler flag. The default optimization is -O2 with the _FORTIFY_SOURCE flag specified. ACPICA BZ 1034. Lv Zheng, Bob Moore. Tests/ASLTS: Added options to specify individual test cases and modes. This allows testers running aslts.sh to optionally specify individual test modes and test cases. Also added an option to disable the forced generation of the ACPICA tools from source if desired. Lv Zheng. ---------------------------------------- 27 September 2013. Summary of changes for version 20130927: This release is available at https://acpica.org/downloads 1) ACPICA kernel-resident subsystem: Fixed a problem with store operations to reference objects. This change fixes a problem where a Store operation to an ArgX object that contained a reference to a field object did not complete the automatic dereference and then write to the actual field object. Instead, the object type of the field object was inadvertently changed to match the type of the source operand. The new behavior will actually write to the field object (buffer field or field unit), thus matching the correct ACPI-defined behavior. Implemented support to allow the host to redefine individual OSL prototypes. This change enables the host to redefine OSL prototypes found in the acpiosxf.h file. This allows the host to implement OSL interfaces with a macro or inlined function. Further, it allows the host to add any additional required modifiers such as __iomem, __init, __exit, etc., as necessary on a per-interface basis. Enables maximum flexibility for the OSL interfaces. Lv Zheng. Hardcoded the access width for the FADT-defined reset register. The ACPI specification requires the reset register width to be 8 bits. ACPICA now hardcodes the width to 8 and ignores the FADT width value. This provides compatibility with other ACPI implementations that have allowed BIOS code with bad register width values to go unnoticed. Matthew Garett, Bob Moore, Lv Zheng. Changed the position/use of the ACPI_PRINTF_LIKE macro. This macro is used in the OSL header (acpiosxf). The change modifies the position of this macro in each instance where it is used (AcpiDebugPrint, etc.) to avoid build issues if the OSL defines the implementation of the interface to be an inline stub function. Lv Zheng. Deployed a new macro ACPI_EXPORT_SYMBOL_INIT for the main ACPICA initialization interfaces. This change adds a new macro for the main init and terminate external interfaces in order to support hosts that require additional or different processing for these functions. Changed from ACPI_EXPORT_SYMBOL to ACPI_EXPORT_SYMBOL_INIT for these functions. Lv Zheng, Bob Moore. Cleaned up the memory allocation macros for configurability. In the common case, the ACPI_ALLOCATE and related macros now resolve directly to their respective AcpiOs* OSL interfaces. Two options: 1) The ACPI_ALLOCATE_ZEROED macro uses a simple local implementation by default, unless overridden by the USE_NATIVE_ALLOCATE_ZEROED define. 2) For AcpiExec (and for debugging), the macros can optionally be resolved to the local ACPICA interfaces that track each allocation (local tracking is used to immediately detect memory leaks). Lv Zheng. Simplified the configuration for ACPI_REDUCED_HARDWARE. Allows the kernel to predefine this macro to either TRUE or FALSE during the system build. Replaced __FUNCTION_ with __func__ in the gcc-specific header. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 95.8K Code, 27.0K Data, 122.8K Total Debug Version: 185.2K Code, 77.2K Data, 262.4K Total Previous Release: Non-Debug Version: 96.7K Code, 27.1K Data, 123.9K Total Debug Version: 184.4K Code, 76.8K Data, 261.2K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented wildcard support for the -e option. This simplifies use when there are many SSDTs that must be included to resolve external method declarations. ACPICA BZ 1041. Example: iasl -e ssdt*.dat -d dsdt.dat AcpiExec: Add history/line-editing for Unix/Linux systems. This change adds a portable module that implements full history and limited line editing for Unix and Linux systems. It does not use readline() due to portability issues. Instead it uses the POSIX termio interface to put the terminal in raw input mode so that the various special keys can be trapped (such as up/down-arrow for history support and left/right-arrow for line editing). Uses the existing debugger history mechanism. ACPICA BZ 1036. AcpiXtract: Add support to handle (ignore) "empty" lines containing only one or more spaces. This provides compatible with early or different versions of the AcpiDump utility. ACPICA BZ 1044. AcpiDump: Do not ignore tables that contain only an ACPI table header. Apparently, some BIOSs create SSDTs that contain an ACPI table header but no other data. This change adds support to dump these tables. Any tables shorter than the length of an ACPI table header remain in error (an error message is emitted). Reported by Yi Li. Debugger: Echo actual command along with the "unknown command" message. ---------------------------------------- 23 August 2013. Summary of changes for version 20130823: 1) ACPICA kernel-resident subsystem: Implemented support for host-installed System Control Interrupt (SCI) handlers. Certain ACPI functionality requires the host to handle raw SCIs. For example, the "SCI Doorbell" that is defined for memory power state support requires the host device driver to handle SCIs to examine if the doorbell has been activated. Multiple SCI handlers can be installed to allow for future expansion. New external interfaces are AcpiInstallSciHandler, AcpiRemoveSciHandler; see the ACPICA reference for details. Lv Zheng, Bob Moore. ACPICA BZ 1032. Operation region support: Never locally free the handler "context" pointer. This change removes some dangerous code that attempts to free the handler context pointer in some (rare) circumstances. The owner of the handler owns this pointer and the ACPICA code should never touch it. Although not seen to be an issue in any kernel, it did show up as a problem (fault) under AcpiExec. Also, set the internal storage field for the context pointer to zero when the region is deactivated, simply for sanity. David Box. ACPICA BZ 1039. AcpiRead: On error, do not modify the return value target location. If an error happens in the middle of a split 32/32 64-bit I/O operation, do not modify the target of the return value pointer. Makes the code consistent with the rest of ACPICA. Bjorn Helgaas. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 96.7K Code, 27.1K Data, 123.9K Total Debug Version: 184.4K Code, 76.8K Data, 261.2K Total Previous Release: Non-Debug Version: 96.2K Code, 27.1K Data, 123.3K Total Debug Version: 185.4K Code, 77.1K Data, 262.5K Total 2) iASL Compiler/Disassembler and Tools: AcpiDump: Implemented several new features and fixed some problems: 1) Added support to dump the RSDP, RSDT, and XSDT tables. 2) Added support for multiple table instances (SSDT, UEFI). 3) Added option to dump "customized" (overridden) tables (-c). 4) Fixed a problem where some table filenames were improperly constructed. 5) Improved some error messages, removed some unnecessary messages. iASL: Implemented additional support for disassembly of ACPI tables that contain invocations of external control methods. The -fe option allows the import of a file that specifies the external methods along with the required number of arguments for each -- allowing for the correct disassembly of the table. This is a workaround for a limitation of AML code where the disassembler often cannot determine the number of arguments required for an external control method and generates incorrect ASL code. See the iASL reference for details. ACPICA BZ 1030. Debugger: Implemented a new command (paths) that displays the full pathnames (namepaths) and object types of all objects in the namespace. This is an alternative to the namespace command. Debugger: Implemented a new command (sci) that invokes the SCI dispatch mechanism and any installed handlers. iASL: Fixed a possible segfault for "too many parent prefixes" condition. This can occur if there are too many parent prefixes in a namepath (for example, ^^^^^^PCI0.ECRD). ACPICA BZ 1035. Application OSLs: Set the return value for the PCI read functions. These functions simply return AE_OK, but should set the return value to zero also. This change implements this. ACPICA BZ 1038. Debugger: Prevent possible command line buffer overflow. Increase the size of a couple of the debugger line buffers, and ensure that overflow cannot happen. ACPICA BZ 1037. iASL: Changed to abort immediately on serious errors during the parsing phase. Due to the nature of ASL, there is no point in attempting to compile these types of errors, and they typically end up causing a cascade of hundreds of errors which obscure the original problem. ---------------------------------------- 25 July 2013. Summary of changes for version 20130725: 1) ACPICA kernel-resident subsystem: Fixed a problem with the DerefOf operator where references to FieldUnits and BufferFields incorrectly returned the parent object, not the actual value of the object. After this change, a dereference of a FieldUnit reference results in a read operation on the field to get the value, and likewise, the appropriate BufferField value is extracted from the target buffer. Fixed a problem where the _WAK method could cause a fault under these circumstances: 1) Interpreter slack mode was not enabled, and 2) the _WAK method returned no value. The problem is rarely seen because most kernels run ACPICA in slack mode. For the DerefOf operator, a fatal error now results if an attempt is made to dereference a reference (created by the Index operator) to a NULL package element. Provides compatibility with other ACPI implementations, and this behavior will be added to a future version of the ACPI specification. The ACPI Power Management Timer (defined in the FADT) is now optional. This provides compatibility with other ACPI implementations and will appear in the next version of the ACPI specification. If there is no PM Timer on the platform, AcpiGetTimer returns AE_SUPPORT. An address of zero in the FADT indicates no PM timer. Implemented a new interface for _OSI support, AcpiUpdateInterfaces. This allows the host to globally enable/disable all vendor strings, all feature strings, or both. Intended to be primarily used for debugging purposes only. Lv Zheng. Expose the collected _OSI data to the host via a global variable. This data tracks the highest level vendor ID that has been invoked by the BIOS so that the host (and potentially ACPICA itself) can change behaviors based upon the age of the BIOS. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 96.2K Code, 27.1K Data, 123.3K Total Debug Version: 184.4K Code, 76.8K Data, 261.2K Total Previous Release: Non-Debug Version: 95.9K Code, 26.9K Data, 122.8K Total Debug Version: 184.1K Code, 76.7K Data, 260.8K Total 2) iASL Compiler/Disassembler and Tools: iASL: Created the following enhancements for the -so option (create offset table): 1)Add offsets for the last nameseg in each namepath for every supported object type 2)Add support for Processor, Device, Thermal Zone, and Scope objects 3)Add the actual AML opcode for the parent object of every supported object type 4)Add support for the ZERO/ONE/ONES AML opcodes for integer objects Disassembler: Emit all unresolved external symbols in a single block. These are external references to control methods that could not be resolved, and thus, the disassembler had to make a guess at the number of arguments to parse. iASL: The argument to the -T option (create table template) is now optional. If not specified, the default table is a DSDT, typically the most common case. ---------------------------------------- 26 June 2013. Summary of changes for version 20130626: 1) ACPICA kernel-resident subsystem: Fixed an issue with runtime repair of the _CST object. Null or invalid elements were not always removed properly. Lv Zheng. Removed an arbitrary restriction of 256 GPEs per GPE block (such as the FADT-defined GPE0 and GPE1). For GPE0, GPE1, and each GPE Block Device, the maximum number of GPEs is 1016. Use of multiple GPE block devices makes the system-wide number of GPEs essentially unlimited. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 95.9K Code, 26.9K Data, 122.8K Total Debug Version: 184.1K Code, 76.7K Data, 260.8K Total Previous Release: Non-Debug Version: 96.0K Code, 27.0K Data, 123.0K Total Debug Version: 184.1K Code, 76.8K Data, 260.9K Total 2) iASL Compiler/Disassembler and Tools: Portable AcpiDump: Implemented full support for the Linux and FreeBSD hosts. Now supports Linux, FreeBSD, and Windows. Disassembler: Added some missing types for the HEST and EINJ tables: "Set Error Type With Address", "CMCI", "MCE", and "Flush Cacheline". iASL/Preprocessor: Implemented full support for nested #if/#else/#elif/#endif blocks. Allows arbitrary depth of nested blocks. Disassembler: Expanded maximum output string length to 64K. Was 256 bytes max. The original purpose of this constraint was to limit the amount of debug output. However, the string function in question (UtPrintString) is now used for the disassembler also, where 256 bytes is insufficient. Reported by RehabMan@GitHub. iASL/DataTables: Fixed some problems and issues with compilation of DMAR tables. ACPICA BZ 999. Lv Zheng. iASL: Fixed a couple of error exit issues that could result in a "Could not delete " message during ASL compilation. AcpiDump: Allow "FADT" and "MADT" as valid table signatures, even though the actual signatures for these tables are "FACP" and "APIC", respectively. AcpiDump: Added support for multiple UEFI tables. Only SSDT and UEFI tables are allowed to have multiple instances. ---------------------------------------- 17 May 2013. Summary of changes for version 20130517: 1) ACPICA kernel-resident subsystem: Fixed a regression introduced in version 20130328 for _INI methods. This change fixes a problem introduced in 20130328 where _INI methods are no longer executed properly because of a memory block that was not initialized correctly. ACPICA BZ 1016. Tomasz Nowicki . Fixed a possible problem with the new extended sleep registers in the ACPI 5.0 FADT. Do not use these registers (even if populated) unless the HW- reduced bit is set in the FADT (as per the ACPI specification). ACPICA BZ 1020. Lv Zheng. Implemented return value repair code for _CST predefined objects: Sort the list and detect/remove invalid entries. ACPICA BZ 890. Lv Zheng. Implemented a debug-only option to disable loading of SSDTs from the RSDT/XSDT during ACPICA initialization. This can be useful for debugging ACPI problems on some machines. Set AcpiGbl_DisableSsdtTableLoad in acglobal.h - ACPICA BZ 1005. Lv Zheng. Fixed some issues in the ACPICA initialization and termination code: Tomasz Nowicki 1) Clear events initialized flag upon event component termination. ACPICA BZ 1013. 2) Fixed a possible memory leak in GPE init error path. ACPICA BZ 1018. 3) Delete global lock pending lock during termination. ACPICA BZ 1012. 4) Clear debug buffer global on termination to prevent possible multiple delete. ACPICA BZ 1010. Standardized all switch() blocks across the entire source base. After many years, different formatting for switch() had crept in. This change makes the formatting of every switch block identical. ACPICA BZ 997. Chao Guan. Split some files to enhance ACPICA modularity and configurability: 1) Split buffer dump routines into utilities/utbuffer.c 2) Split internal error message routines into utilities/uterror.c 3) Split table print utilities into tables/tbprint.c 4) Split iASL command-line option processing into asloptions.c Makefile enhancements: 1) Support for all new files above. 2) Abort make on errors from any subcomponent. Chao Guan. 3) Add build support for Apple Mac OS X. Liang Qi. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 96.0K Code, 27.0K Data, 123.0K Total Debug Version: 184.1K Code, 76.8K Data, 260.9K Total Previous Release: Non-Debug Version: 95.6K Code, 26.8K Data, 122.4K Total Debug Version: 183.5K Code, 76.6K Data, 260.1K Total 2) iASL Compiler/Disassembler and Tools: New utility: Implemented an easily portable version of the acpidump utility to extract ACPI tables from the system (or a file) in an ASCII hex dump format. The top-level code implements the various command line options, file I/O, and table dump routines. To port to a new host, only three functions need to be implemented to get tables -- since this functionality is OS-dependent. See the tools/acpidump/apmain.c module and the ACPICA reference for porting instructions. ACPICA BZ 859. Notes: 1) The Windows version obtains the ACPI tables from the Registry. 2) The Linux version is under development. 3) Other hosts - If an OS-dependent module is submitted, it will be distributed with ACPICA. iASL: Fixed a regression for -D preprocessor option (define symbol). A restructuring/change to the initialization sequence caused this option to no longer work properly. iASL: Implemented a mechanism to disable specific warnings and remarks. Adds a new command line option, "-vw as well as "#pragma disable ". ACPICA BZ 989. Chao Guan, Bob Moore. iASL: Fix for too-strict package object validation. The package object validation for return values from the predefined names is a bit too strict, it does not allow names references within the package (which will be resolved at runtime.) These types of references cannot be validated at compile time. This change ignores named references within package objects for names that return or define static packages. Debugger: Fixed the 80-character command line limitation for the History command. Now allows lines of arbitrary length. ACPICA BZ 1000. Chao Guan. iASL: Added control method and package support for the -so option (generates AML offset table for BIOS support.) iASL: issue a remark if a non-serialized method creates named objects. If a thread blocks within the method for any reason, and another thread enters the method, the method will fail because an attempt will be made to create the same (named) object twice. In this case, issue a remark that the method should be marked serialized. NOTE: may become a warning later. ACPICA BZ 909. ---------------------------------------- 18 April 2013. Summary of changes for version 20130418: 1) ACPICA kernel-resident subsystem: Fixed a possible buffer overrun during some rare but specific field unit read operations. This overrun can only happen if the DSDT version is 1 -- meaning that all AML integers are 32 bits -- and the field length is between 33 and 55 bits long. During the read, an internal buffer object is created for the field unit because the field is larger than an integer (32 bits). However, in this case, the buffer will be incorrectly written beyond the end because the buffer length is less than the internal minimum of 64 bits (8 bytes) long. The buffer will be either 5, 6, or 7 bytes long, but a full 8 bytes will be written. Updated the Embedded Controller "orphan" _REG method support. This refers to _REG methods under the EC device that have no corresponding operation region. This is allowed by the ACPI specification. This update removes a dependency on the existence an ECDT table. It will execute an orphan _REG method as long as the operation region handler for the EC is installed at the EC device node and not the namespace root. Rui Zhang (original update), Bob Moore (update/integrate). Implemented run-time argument typechecking for all predefined ACPI names (_STA, _BIF, etc.) This change performs object typechecking on all incoming arguments for all predefined names executed via AcpiEvaluateObject. This ensures that ACPI-related device drivers are passing correct object types as well as the correct number of arguments (therefore identifying any issues immediately). Also, the ASL/namespace definition of the predefined name is checked against the ACPI specification for the proper argument count. Adds one new file, nsarguments.c Changed an exception code for the ASL UnLoad() operator. Changed the exception code for the case where the input DdbHandle is invalid, from AE_BAD_PARAMETER to the more appropriate AE_AML_OPERAND_TYPE. Unix/Linux makefiles: Removed the use of the -O2 optimization flag in the global makefile. The use of this flag causes compiler errors on earlier versions of GCC, so it has been removed for compatibility. Miscellaneous cleanup: 1) Removed some unused/obsolete macros 2) Fixed a possible memory leak in the _OSI support 3) Removed an unused variable in the predefined name support 4) Windows OSL: remove obsolete reference to a memory list field Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Current Release: Non-Debug Version: 95.2K Code, 26.4K Data, 121.6K Total Debug Version: 183.0K Code, 76.0K Data, 259.0K Total Previous Release: Non-Debug Version: 95.6K Code, 26.8K Data, 122.4K Total Debug Version: 183.5K Code, 76.6K Data, 260.1K Total 2) iASL Compiler/Disassembler and Tools: AcpiExec: Added installation of a handler for the SystemCMOS address space. This prevents control method abort if a method accesses this space. AcpiExec: Added support for multiple EC devices, and now install EC operation region handler(s) at the actual EC device instead of the namespace root. This reflects the typical behavior of host operating systems. AcpiExec: Updated to ensure that all operation region handlers are installed before the _REG methods are executed. This prevents a _REG method from aborting if it accesses an address space has no handler. AcpiExec installs a handler for every possible address space. Debugger: Enhanced the "handlers" command to display non-root handlers. This change enhances the handlers command to display handlers associated with individual devices throughout the namespace, in addition to the currently supported display of handlers associated with the root namespace node. ASL Test Suite: Several test suite errors have been identified and resolved, reducing the total error count during execution. Chao Guan. ---------------------------------------- 28 March 2013. Summary of changes for version 20130328: 1) ACPICA kernel-resident subsystem: Fixed several possible race conditions with the internal object reference counting mechanism. Some of the external ACPICA interfaces update object reference counts without holding the interpreter or namespace lock. This change adds a spinlock to protect reference count updates on the internal ACPICA objects. Reported by and with assistance from Andriy Gapon (avg@FreeBSD.org). FADT support: Removed an extraneous warning for very large GPE register sets. This change removes a size mismatch warning if the legacy length field for a GPE register set is larger than the 64-bit GAS structure can accommodate. GPE register sets can be larger than the 255-bit width limitation of the GAS structure. Linn Crosetto (linn@hp.com). _OSI Support: handle any errors from AcpiOsAcquireMutex. Check for error return from this interface. Handles a possible timeout case if ACPI_WAIT_FOREVER is modified by the host to be a value less than "forever". Jung-uk Kim. Predefined name support: Add allowed/required argument type information to the master predefined info table. This change adds the infrastructure to enable typechecking on incoming arguments for all predefined methods/objects. It does not actually contain the code that will fully utilize this information, this is still under development. Also condenses some duplicate code for the predefined names into a new module, utilities/utpredef.c Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 95.0K Code, 25.9K Data, 120.9K Total Debug Version: 182.9K Code, 75.6K Data, 258.5K Total Current Release: Non-Debug Version: 95.2K Code, 26.4K Data, 121.6K Total Debug Version: 183.0K Code, 76.0K Data, 259.0K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented a new option to simplify the development of ACPI- related BIOS code. Adds support for a new "offset table" output file. The -so option will create a C table containing the AML table offsets of various named objects in the namespace so that BIOS code can modify them easily at boot time. This can simplify BIOS runtime code by eliminating expensive searches for "magic values", enhancing boot times and adding greater reliability. With assistance from Lee Hamel. iASL: Allow additional predefined names to return zero-length packages. Now, all predefined names that are defined by the ACPI specification to return a "variable-length package of packages" are allowed to return a zero length top-level package. This allows the BIOS to tell the host that the requested feature is not supported, and supports existing BIOS/ASL code and practices. iASL: Changed the "result not used" warning to an error. This is the case where an ASL operator is effectively a NOOP because the result of the operation is not stored anywhere. For example: Add (4, Local0) There is no target (missing 3rd argument), nor is the function return value used. This is potentially a very serious problem -- since the code was probably intended to do something, but for whatever reason, the value was not stored. Therefore, this issue has been upgraded from a warning to an error. AcpiHelp: Added allowable/required argument types to the predefined names info display. This feature utilizes the recent update to the predefined names table (above). ---------------------------------------- 14 February 2013. Summary of changes for version 20130214: 1) ACPICA Kernel-resident Subsystem: Fixed a possible regression on some hosts: Reinstated the safe return macros (return_ACPI_STATUS, etc.) that ensure that the argument is evaluated only once. Although these macros are not needed for the ACPICA code itself, they are often used by ACPI-related host device drivers where the safe feature may be necessary. Fixed several issues related to the ACPI 5.0 reduced hardware support (SOC): Now ensure that if the platform declares itself as hardware- reduced via the FADT, the following functions become NOOPs (and always return AE_OK) because ACPI is always enabled by definition on these machines: AcpiEnable AcpiDisable AcpiHwGetMode AcpiHwSetMode Dynamic Object Repair: Implemented additional runtime repairs for predefined name return values. Both of these repairs can simplify code in the related device drivers that invoke these methods: 1) For the _STR and _MLS names, automatically repair/convert an ASCII string to a Unicode buffer. 2) For the _CRS, _PRS, and _DMA names, return a resource descriptor with a lone end tag descriptor in the following cases: A Return(0) was executed, a null buffer was returned, or no object at all was returned (non-slack mode only). Adds a new file, nsconvert.c ACPICA BZ 998. Bob Moore, Lv Zheng. Resource Manager: Added additional code to prevent possible infinite loops while traversing corrupted or ill-formed resource template buffers. Check for zero-length resource descriptors in all code that loops through resource templates (the length field is used to index through the template). This change also hardens the external AcpiWalkResources and AcpiWalkResourceBuffer interfaces. Local Cache Manager: Enhanced the main data structure to eliminate an unnecessary mechanism to access the next object in the list. Actually provides a small performance enhancement for hosts that use the local ACPICA cache manager. Jung-uk Kim. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 94.5K Code, 25.4K Data, 119.9K Total Debug Version: 182.3K Code, 75.0K Data, 257.3K Total Current Release: Non-Debug Version: 95.0K Code, 25.9K Data, 120.9K Total Debug Version: 182.9K Code, 75.6K Data, 258.5K Total 2) iASL Compiler/Disassembler and Tools: iASL/Disassembler: Fixed several issues with the definition of the ACPI 5.0 RASF table (RAS Feature Table). This change incorporates late changes that were made to the ACPI 5.0 specification. iASL/Disassembler: Added full support for the following new ACPI tables: 1) The MTMR table (MID Timer Table) 2) The VRTC table (Virtual Real Time Clock Table). Includes header file, disassembler, table compiler, and template support for both tables. iASL: Implemented compile-time validation of package objects returned by predefined names. This new feature validates static package objects returned by the various predefined names defined to return packages. Both object types and package lengths are validated, for both parent packages and sub-packages, if any. The code is similar in structure and behavior to the runtime repair mechanism within the AML interpreter and uses the existing predefined name information table. Adds a new file, aslprepkg.c. ACPICA BZ 938. iASL: Implemented auto-detection of binary ACPI tables for disassembly. This feature detects a binary file with a valid ACPI table header and invokes the disassembler automatically. Eliminates the need to specifically invoke the disassembler with the -d option. ACPICA BZ 862. iASL/Disassembler: Added several warnings for the case where there are unresolved control methods during the disassembly. This can potentially cause errors when the output file is compiled, because the disassembler assumes zero method arguments in these cases (it cannot determine the actual number of arguments without resolution/definition of the method). Debugger: Added support to display all resources with a single command. Invocation of the resources command with no arguments will now display all resources within the current namespace. AcpiHelp: Added descriptive text for each ACPICA exception code displayed via the -e option. ---------------------------------------- 17 January 2013. Summary of changes for version 20130117: 1) ACPICA Kernel-resident Subsystem: Updated the AcpiGetSleepTypeData interface: Allow the \_Sx methods to return either 1 or 2 integers. Although the ACPI spec defines the \_Sx objects to return a package containing one integer, most BIOS code returns two integers and the previous code reflects that. However, we also need to support BIOS code that actually implements to the ACPI spec, and this change reflects this. Fixed two issues with the ACPI_DEBUG_PRINT macros: 1) Added the ACPI_DO_WHILE macro to the main DEBUG_PRINT helper macro for C compilers that require this support. 2) Renamed the internal ACPI_DEBUG macro to ACPI_DO_DEBUG_PRINT since ACPI_DEBUG is already used by many of the various hosts. Updated all ACPICA copyrights and signons to 2013. Added the 2013 copyright to all module headers and signons, including the standard Linux header. This affects virtually every file in the ACPICA core subsystem, iASL compiler, all ACPICA utilities, and the test suites. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 94.5K Code, 25.5K Data, 120.0K Total Debug Version: 182.2K Code, 74.9K Data, 257.1K Total Current Release: Non-Debug Version: 94.5K Code, 25.4K Data, 119.9K Total Debug Version: 182.3K Code, 75.0K Data, 257.3K Total 2) iASL Compiler/Disassembler and Tools: Generic Unix OSL: Use a buffer to eliminate multiple vfprintf()s and prevent a possible fault on some hosts. Some C libraries modify the arg pointer parameter to vfprintf making it difficult to call it twice in the AcpiOsVprintf function. Use a local buffer to workaround this issue. This does not affect the Windows OSL since the Win C library does not modify the arg pointer. Chao Guan, Bob Moore. iASL: Fixed a possible infinite loop when the maximum error count is reached. If an output file other than the .AML file is specified (such as a listing file), and the maximum number of errors is reached, do not attempt to flush data to the output file(s) as the compiler is aborting. This can cause an infinite loop as the max error count code essentially keeps calling itself. iASL/Disassembler: Added an option (-in) to ignore NOOP opcodes/operators. Implemented for both the compiler and the disassembler. Often, the NOOP opcode is used as padding for packages that are changed dynamically by the BIOS. When disassembled and recompiled, these NOOPs will cause syntax errors. This option causes the disassembler to ignore all NOOP opcodes (0xA3), and it also causes the compiler to ignore all ASL source code NOOP statements as well. Debugger: Enhanced the Sleep command to execute all sleep states. This change allows Sleep to be invoked with no arguments and causes the debugger to execute all of the sleep states, 0-5, automatically. ---------------------------------------- 20 December 2012. Summary of changes for version 20121220: 1) ACPICA Kernel-resident Subsystem: Implemented a new interface, AcpiWalkResourceBuffer. This interface is an alternate entry point for AcpiWalkResources and improves the usability of the resource manager by accepting as input a buffer containing the output of either a _CRS, _PRS, or _AEI method. The key functionality is that the input buffer is not deleted by this interface so that it can be used by the host later. See the ACPICA reference for details. Interpreter: Add a warning if a 64-bit constant appears in a 32-bit table (DSDT version < 2). The constant will be truncated and this warning reflects that behavior. Resource Manager: Add support for the new ACPI 5.0 wake bit in the IRQ, ExtendedInterrupt, and GpioInt descriptors. This change adds support to both get and set the new wake bit in these descriptors, separately from the existing share bit. Reported by Aaron Lu. Interpreter: Fix Store() when an implicit conversion is not possible. For example, in the cases such as a store of a string to an existing package object, implement the store as a CopyObject(). This is a small departure from the ACPI specification which states that the control method should be aborted in this case. However, the ASLTS suite depends on this behavior. Performance improvement for the various FUNCTION_TRACE and DEBUG_PRINT macros: check if debug output is currently enabled as soon as possible to minimize performance impact if debug is in fact not enabled. Source code restructuring: Cleanup to improve modularity. The following new files have been added: dbconvert.c, evhandler.c, nsprepkg.c, psopinfo.c, psobject.c, rsdumpinfo.c, utstring.c, and utownerid.c. Associated makefiles and project files have been updated. Changed an exception code for LoadTable operator. For the case where one of the input strings is too long, change the returned exception code from AE_BAD_PARAMETER to AE_AML_STRING_LIMIT. Fixed a possible memory leak in dispatcher error path. On error, delete the mutex object created during method mutex creation. Reported by tim.gardner@canonical.com. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 94.3K Code, 25.3K Data, 119.6K Total Debug Version: 175.5K Code, 74.5K Data, 250.0K Total Current Release: Non-Debug Version: 94.5K Code, 25.5K Data, 120.0K Total Debug Version: 182.2K Code, 74.9K Data, 257.1K Total 2) iASL Compiler/Disassembler and Tools: iASL: Disallow a method call as argument to the ObjectType ASL operator. This change tracks an errata to the ACPI 5.0 document. The AML grammar will not allow the interpreter to differentiate between a method and a method invocation when these are used as an argument to the ObjectType operator. The ACPI specification change is to disallow a method invocation (UserTerm) for the ObjectType operator. Finish support for the TPM2 and CSRT tables in the headers, table compiler, and disassembler. Unix user-space OSL: Fix a problem with WaitSemaphore where the timeout always expires immediately if the semaphore is not available. The original code was using a relative-time timeout, but sem_timedwait requires the use of an absolute time. iASL: Added a remark if the Timer() operator is used within a 32-bit table. This operator returns a 64-bit time value that will be truncated within a 32-bit table. iASL Source code restructuring: Cleanup to improve modularity. The following new files have been added: aslhex.c, aslxref.c, aslnamesp.c, aslmethod.c, and aslfileio.c. Associated makefiles and project files have been updated. ---------------------------------------- 14 November 2012. Summary of changes for version 20121114: 1) ACPICA Kernel-resident Subsystem: Implemented a performance enhancement for ACPI/AML Package objects. This change greatly increases the performance of Package objects within the interpreter. It changes the processing of reference counts for packages by optimizing for the most common case where the package sub-objects are either Integers, Strings, or Buffers. Increases the overall performance of the ASLTS test suite by 1.5X (Increases the Slack Mode performance by 2X.) Chao Guan. ACPICA BZ 943. Implemented and deployed common macros to extract flag bits from resource descriptors. Improves readability and maintainability of the code. Fixes a problem with the UART serial bus descriptor for the number of data bits flags (was incorrectly 2 bits, should be 3). Enhanced the ACPI_GETx and ACPI_SETx macros. Improved the implementation of the macros and changed the SETx macros to the style of (destination, source). Also added ACPI_CASTx companion macros. Lv Zheng. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 93.9K Code, 25.2K Data, 119.1K Total Debug Version: 175.5K Code, 74.5K Data, 250.0K Total Current Release: Non-Debug Version: 94.3K Code, 25.3K Data, 119.6K Total Debug Version: 175.5K Code, 74.5K Data, 250.0K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Added the new ACPI 5.0 interrupt sharing flags. This change adds the ShareAndWake and ExclusiveAndWake flags which were added to the Irq, Interrupt, and Gpio resource descriptors in ACPI 5.0. ACPICA BZ 986. Disassembler: Fixed a problem with external declaration generation. Fixes a problem where an incorrect pathname could be generated for an external declaration if the original reference to the object includes leading carats (^). ACPICA BZ 984. Debugger: Completed a major update for the Disassemble command. This command was out-of-date and did not properly disassemble control methods that had any reasonable complexity. This fix brings the command up to the same level as the rest of the disassembler. Adds one new file, dmdeferred.c, which is existing code that is now common with the main disassembler and the debugger disassemble command. ACPICA MZ 978. iASL: Moved the parser entry prototype to avoid a duplicate declaration. Newer versions of Bison emit this prototype, so moved the prototype out of the iASL header to where it is actually used in order to avoid a duplicate declaration. iASL/Tools: Standardized use of the stream I/O functions: 1) Ensure check for I/O error after every fopen/fread/fwrite 2) Ensure proper order of size/count arguments for fread/fwrite 3) Use test of (Actual != Requested) after all fwrite, and most fread 4) Standardize I/O error messages Improves reliability and maintainability of the code. Bob Moore, Lv Zheng. ACPICA BZ 981. Disassembler: Prevent duplicate External() statements. During generation of external statements, detect similar pathnames that are actually duplicates such as these: External (\ABCD) External (ABCD) Remove all leading '\' characters from pathnames during the external statement generation so that duplicates will be detected and tossed. ACPICA BZ 985. Tools: Replace low-level I/O with stream I/O functions. Replace open/read/write/close with the stream I/O equivalents fopen/fread/fwrite/fclose for portability and performance. Lv Zheng, Bob Moore. AcpiBin: Fix for the dump-to-hex function. Now correctly output the table name header so that AcpiXtract recognizes the output file/table. iASL: Remove obsolete -2 option flag. Originally intended to force the compiler/disassembler into an ACPI 2.0 mode, this was never implemented and the entire concept is now obsolete. ---------------------------------------- 18 October 2012. Summary of changes for version 20121018: 1) ACPICA Kernel-resident Subsystem: Updated support for the ACPI 5.0 MPST table. Fixes some problems introduced by late changes to the table as it was added to the ACPI 5.0 specification. Includes header, disassembler, and data table compiler support as well as a new version of the MPST template. AcpiGetObjectInfo: Enhanced the device object support to include the ACPI 5.0 _SUB method. Now calls _SUB in addition to the other PNP-related ID methods: _HID, _CID, and _UID. Changed ACPI_DEVICE_ID to ACPI_PNP_DEVICE_ID. Also changed ACPI_DEVICE_ID_LIST to ACPI_PNP_DEVICE_ID_LIST. These changes prevent name collisions on hosts that reserve the *_DEVICE_ID (or *DeviceId) names for their various drivers. Affects the AcpiGetObjectInfo external interface, and other internal interfaces as well. Added and deployed a new macro for ACPI_NAME management: ACPI_MOVE_NAME. This macro resolves to a simple 32-bit move of the 4-character ACPI_NAME on machines that support non-aligned transfers. Optimizes for this case rather than using a strncpy. With assistance from Zheng Lv. Resource Manager: Small fix for buffer size calculation. Fixed a one byte error in the output buffer calculation. Feng Tang. ACPICA BZ 849. Added a new debug print message for AML mutex objects that are force- released. At control method termination, any currently acquired mutex objects are force-released. Adds a new debug-only message for each one that is released. Audited/updated all ACPICA return macros and the function debug depth counter: 1) Ensure that all functions that use the various TRACE macros also use the appropriate ACPICA return macros. 2) Ensure that all normal return statements surround the return expression (value) with parens to ensure consistency across the ACPICA code base. Guan Chao, Tang Feng, Zheng Lv, Bob Moore. ACPICA Bugzilla 972. Global source code changes/maintenance: All extra lines at the start and end of each source file have been removed for consistency. Also, within comments, all new sentences start with a single space instead of a double space, again for consistency across the code base. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 93.7K Code, 25.3K Data, 119.0K Total Debug Version: 175.0K Code, 74.4K Data, 249.4K Total Current Release: Non-Debug Version: 93.9K Code, 25.2K Data, 119.1K Total Debug Version: 175.5K Code, 74.5K Data, 250.0K Total 2) iASL Compiler/Disassembler and Tools: AcpiExec: Improved the algorithm used for memory leak/corruption detection. Added some intelligence to the code that maintains the global list of allocated memory. The list is now ordered by allocated memory address, significantly improving performance. When running AcpiExec on the ASLTS test suite, speed improvements of 3X to 5X are seen, depending on the platform and/or the environment. Note, this performance enhancement affects the AcpiExec utility only, not the kernel-resident ACPICA code. Enhanced error reporting for invalid AML opcodes and bad ACPI_NAMEs. For the disassembler, dump the 48 bytes surrounding the invalid opcode. Fix incorrect table offset reported for invalid opcodes. Report the original 32-bit value for bad ACPI_NAMEs (as well as the repaired name.) Disassembler: Enhanced the -vt option to emit the binary table data in hex format to assist with debugging. Fixed a potential filename buffer overflow in osunixdir.c. Increased the size of file structure. Colin Ian King. ---------------------------------------- 13 September 2012. Summary of changes for version 20120913: 1) ACPICA Kernel-resident Subsystem: ACPI 5.0: Added two new notify types for the Hardware Error Notification Structure within the Hardware Error Source Table (HEST) table -- CMCI(5) and MCE(6). Table Manager: Merged/removed duplicate code in the root table resize functions. One function is external, the other is internal. Lv Zheng, ACPICA BZ 846. Makefiles: Completely removed the obsolete "Linux" makefiles under acpica/generate/linux. These makefiles are obsolete and have been replaced by the generic unix makefiles under acpica/generate/unix. Makefiles: Ensure that binary files always copied properly. Minor rule change to ensure that the final binary output files are always copied up to the appropriate binary directory (bin32 or bin64.) Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 93.8K Code, 25.3K Data, 119.1K Total Debug Version: 175.7K Code, 74.8K Data, 250.5K Total Current Release: Non-Debug Version: 93.7K Code, 25.3K Data, 119.0K Total Debug Version: 175.0K Code, 74.4K Data, 249.4K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Fixed a possible fault during the disassembly of resource descriptors when a second parse is required because of the invocation of external control methods within the table. With assistance from adq@lidskialf.net. ACPICA BZ 976. iASL: Fixed a namepath optimization problem. An error can occur if the parse node that contains the namepath to be optimized does not have a parent node that is a named object. This change fixes the problem. iASL: Fixed a regression where the AML file is not deleted on errors. The AML output file should be deleted if there are any errors during the compiler. The only exception is if the -f (force output) option is used. ACPICA BZ 974. iASL: Added a feature to automatically increase internal line buffer sizes. Via realloc(), automatically increase the internal line buffer sizes as necessary to support very long source code lines. The current version of the preprocessor requires a buffer long enough to contain full source code lines. This change increases the line buffer(s) if the input lines go beyond the current buffer size. This eliminates errors that occurred when a source code line was longer than the buffer. iASL: Fixed a problem with constant folding in method declarations. The SyncLevel term is a ByteConstExpr, and incorrect code would be generated if a Type3 opcode was used. Debugger: Improved command help support. For incorrect argument count, display full help for the command. For help command itself, allow an argument to specify a command. Test Suites: Several bug fixes for the ASLTS suite reduces the number of errors during execution of the suite. Guan Chao. ---------------------------------------- 16 August 2012. Summary of changes for version 20120816: 1) ACPICA Kernel-resident Subsystem: Removed all use of the deprecated _GTS and _BFS predefined methods. The _GTS (Going To Sleep) and _BFS (Back From Sleep) methods are essentially deprecated and will probably be removed from the ACPI specification. Windows does not invoke them, and reportedly never will. The final nail in the coffin is that the ACPI specification states that these methods must be run with interrupts off, which is not going to happen in a kernel interpreter. Note: Linux has removed all use of the methods also. It was discovered that invoking these functions caused failures on some machines, probably because they were never tested since Windows does not call them. Affects two external interfaces, AcpiEnterSleepState and AcpiLeaveSleepStatePrep. Tang Feng. ACPICA BZ 969. Implemented support for complex bit-packed buffers returned from the _PLD (Physical Location of Device) predefined method. Adds a new external interface, AcpiDecodePldBuffer that parses the buffer into a more usable C structure. Note: C Bitfields cannot be used for this type of predefined structure since the memory layout of individual bitfields is not defined by the C language. In addition, there are endian concerns where a compiler will change the bitfield ordering based on the machine type. The new ACPICA interface eliminates these issues, and should be called after _PLD is executed. ACPICA BZ 954. Implemented a change to allow a scope change to root (via "Scope (\)") during execution of module-level ASL code (code that is executed at table load time.) Lin Ming. Added the Windows8/Server2012 string for the _OSI method. This change adds a new _OSI string, "Windows 2012" for both Windows 8 and Windows Server 2012. Added header support for the new ACPI tables DBG2 (Debug Port Table Type 2) and CSRT (Core System Resource Table). Added struct header support for the _FDE, _GRT, _GTM, and _SRT predefined names. This simplifies access to the buffers returned by these predefined names. Adds a new file, include/acbuffer.h. ACPICA BZ 956. GPE support: Removed an extraneous parameter from the various low-level internal GPE functions. Tang Feng. Removed the linux makefiles from the unix packages. The generate/linux makefiles are obsolete and have been removed from the unix tarball release packages. The replacement makefiles are under generate/unix, and there is a top-level makefile under the main acpica directory. ACPICA BZ 967, 912. Updates for Unix makefiles: 1) Add -D_FORTIFY_SOURCE=2 for gcc generation. Arjan van de Ven. 2) Update linker flags (move to end of command line) for AcpiExec utility. Guan Chao. Split ACPICA initialization functions to new file, utxfinit.c. Split from utxface.c to improve modularity and reduce file size. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 93.5K Code, 25.3K Data, 118.8K Total Debug Version: 173.7K Code, 74.0K Data, 247.7K Total Current Release: Non-Debug Version: 93.8K Code, 25.3K Data, 119.1K Total Debug Version: 175.7K Code, 74.8K Data, 250.5K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a problem with constant folding for fixed-length constant expressions. The constant-folding code was not being invoked for constant expressions that allow the use of type 3/4/5 opcodes to generate constants for expressions such as ByteConstExpr, WordConstExpr, etc. This could result in the generation of invalid AML bytecode. ACPICA BZ 970. iASL: Fixed a generation issue on newer versions of Bison. Newer versions apparently automatically emit some of the necessary externals. This change handles these versions in order to eliminate generation warnings. Disassembler: Added support to decode the DBG2 and CSRT ACPI tables. Disassembler: Add support to decode _PLD buffers. The decoded buffer appears within comments in the output file. Debugger: Fixed a regression with the "Threads" command where AE_BAD_PARAMETER was always returned. ---------------------------------------- 11 July 2012. Summary of changes for version 20120711: 1) ACPICA Kernel-resident Subsystem: Fixed a possible fault in the return package object repair code. Fixes a problem that can occur when a lone package object is wrapped with an outer package object in order to force conformance to the ACPI specification. Can affect these predefined names: _ALR, _MLS, _PSS, _TRT, _TSS, _PRT, _HPX, _DLM, _CSD, _PSD, _TSD. Removed code to disable/enable bus master arbitration (ARB_DIS bit in the PM2_CNT register) in the ACPICA sleep/wake interfaces. Management of the ARB_DIS bit must be implemented in the host-dependent C3 processor power state support. Note, ARB_DIS is obsolete and only applies to older chipsets, both Intel and other vendors. (for Intel: ICH4-M and earlier) This change removes the code to disable/enable bus master arbitration during suspend/resume. Use of the ARB_DIS bit in the optional PM2_CNT register causes resume problems on some machines. The change has been in use for over seven years within Linux. Implemented two new external interfaces to support host-directed dynamic ACPI table load and unload. They are intended to simplify the host implementation of hot-plug support: AcpiLoadTable: Load an SSDT from a buffer into the namespace. AcpiUnloadParentTable: Unload an SSDT via a named object owned by the table. See the ACPICA reference for additional details. Adds one new file, components/tables/tbxfload.c Implemented and deployed two new interfaces for errors and warnings that are known to be caused by BIOS/firmware issues: AcpiBiosError: Prints "ACPI Firmware Error" message. AcpiBiosWarning: Prints "ACPI Firmware Warning" message. Deployed these new interfaces in the ACPICA Table Manager code for ACPI table and FADT errors. Additional deployment to be completed as appropriate in the future. The associated conditional macros are ACPI_BIOS_ERROR and ACPI_BIOS_WARNING. See the ACPICA reference for additional details. ACPICA BZ 843. Implicit notify support: ensure that no memory allocation occurs within a critical region. This fix moves a memory allocation outside of the time that a spinlock is held. Fixes issues on systems that do not allow this behavior. Jung-uk Kim. Split exception code utilities and tables into a new file, utilities/utexcep.c Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 93.1K Code, 25.1K Data, 118.2K Total Debug Version: 172.9K Code, 73.6K Data, 246.5K Total Current Release: Non-Debug Version: 93.5K Code, 25.3K Data, 118.8K Total Debug Version: 173.7K Code, 74.0K Data, 247.7K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a parser problem for hosts where EOF is defined as -1 instead of 0. Jung-uk Kim. Debugger: Enhanced the "tables" command to emit additional information about the current set of ACPI tables, including the owner ID and flags decode. Debugger: Reimplemented the "unload" command to use the new AcpiUnloadParentTable external interface. This command was disable previously due to need for an unload interface. AcpiHelp: Added a new option to decode ACPICA exception codes. The -e option will decode 16-bit hex status codes (ACPI_STATUS) to name strings. ---------------------------------------- 20 June 2012. Summary of changes for version 20120620: 1) ACPICA Kernel-resident Subsystem: Implemented support to expand the "implicit notify" feature to allow multiple devices to be notified by a single GPE. This feature automatically generates a runtime device notification in the absence of a BIOS-provided GPE control method (_Lxx/_Exx) or a host-installed handler for the GPE. Implicit notify is provided by ACPICA for Windows compatibility, and is a workaround for BIOS AML code errors. See the description of the AcpiSetupGpeForWake interface in the APCICA reference. Bob Moore, Rafael Wysocki. ACPICA BZ 918. Changed some comments and internal function names to simplify and ensure correctness of the Linux code translation. No functional changes. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 93.0K Code, 25.1K Data, 118.1K Total Debug Version: 172.7K Code, 73.6K Data, 246.3K Total Current Release: Non-Debug Version: 93.1K Code, 25.1K Data, 118.2K Total Debug Version: 172.9K Code, 73.6K Data, 246.5K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Added support to emit short, commented descriptions for the ACPI predefined names in order to improve the readability of the disassembled output. ACPICA BZ 959. Changes include: 1) Emit descriptions for all standard predefined names (_INI, _STA, _PRW, etc.) 2) Emit generic descriptions for the special names (_Exx, _Qxx, etc.) 3) Emit descriptions for the resource descriptor names (_MIN, _LEN, etc.) AcpiSrc: Fixed several long-standing Linux code translation issues. Argument descriptions in function headers are now translated properly to lower case and underscores. ACPICA BZ 961. Also fixes translation problems such as these: (old -> new) i_aSL -> iASL 00-7_f -> 00-7F 16_k -> 16K local_fADT -> local_FADT execute_oSI -> execute_OSI iASL: Fixed a problem where null bytes were inadvertently emitted into some listing files. iASL: Added the existing debug options to the standard help screen. There are no longer two different help screens. ACPICA BZ 957. AcpiHelp: Fixed some typos in the various predefined name descriptions. Also expand some of the descriptions where appropriate. iASL: Fixed the -ot option (display compile times/statistics). Was not working properly for standard output; only worked for the debug file case. ---------------------------------------- 18 May 2012. Summary of changes for version 20120518: 1) ACPICA Core Subsystem: Added a new OSL interface, AcpiOsWaitEventsComplete. This interface is defined to block until asynchronous events such as notifies and GPEs have completed. Within ACPICA, it is only called before a notify or GPE handler is removed/uninstalled. It also may be useful for the host OS within related drivers such as the Embedded Controller driver. See the ACPICA reference for additional information. ACPICA BZ 868. ACPI Tables: Added a new error message for a possible overflow failure during the conversion of FADT 32-bit legacy register addresses to internal common 64- bit GAS structure representation. The GAS has a one-byte "bit length" field, thus limiting the register length to 255 bits. ACPICA BZ 953. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 92.9K Code, 25.0K Data, 117.9K Total Debug Version: 172.6K Code, 73.4K Data, 246.0K Total Current Release: Non-Debug Version: 93.0K Code, 25.1K Data, 118.1K Total Debug Version: 172.7K Code, 73.6K Data, 246.3K Total 2) iASL Compiler/Disassembler and Tools: iASL: Added the ACPI 5.0 "PCC" keyword for use in the Register() ASL macro. This keyword was added late in the ACPI 5.0 release cycle and was not implemented until now. Disassembler: Added support for Operation Region externals. Adds missing support for operation regions that are defined in another table, and referenced locally via a Field or BankField ASL operator. Now generates the correct External statement. Disassembler: Several additional fixes for the External() statement generation related to some ASL operators. Also, order the External() statements alphabetically in the disassembler output. Fixes the External() generation for the Create* field, Alias, and Scope operators: 1) Create* buffer field operators - fix type mismatch warning on disassembly 2) Alias - implement missing External support 3) Scope - fix to make sure all necessary externals are emitted. iASL: Improved pathname support. For include files, merge the prefix pathname with the file pathname and eliminate unnecessary components. Convert backslashes in all pathnames to forward slashes, for readability. Include file pathname changes affect both #include and Include() type operators. iASL/DTC/Preprocessor: Gracefully handle early EOF. Handle an EOF at the end of a valid line by inserting a newline and then returning the EOF during the next call to GetNextLine. Prevents the line from being ignored due to EOF condition. iASL: Implemented some changes to enhance the IDE support (-vi option.) Error and Warning messages are now correctly recognized for both the source code browser and the global error and warning counts. ---------------------------------------- 20 April 2012. Summary of changes for version 20120420: 1) ACPICA Core Subsystem: Implemented support for multiple notify handlers. This change adds support to allow multiple system and device notify handlers on Device, Thermal Zone, and Processor objects. This can simplify the host OS notification implementation. Also re-worked and restructured the entire notify support code to simplify handler installation, handler removal, notify event queuing, and notify dispatch to handler(s). Note: there can still only be two global notify handlers - one for system notifies and one for device notifies. There are no changes to the existing handler install/remove interfaces. Lin Ming, Bob Moore, Rafael Wysocki. Fixed a regression in the package repair code where the object reference count was calculated incorrectly. Regression was introduced in the commit "Support to add Package wrappers". Fixed a couple possible memory leaks in the AML parser, in the error recovery path. Jesper Juhl, Lin Ming. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 92.9K Code, 25.0K Data, 117.9K Total Debug Version: 172.5K Code, 73.2K Data, 245.7K Total Current Release: Non-Debug Version: 92.9K Code, 25.0K Data, 117.9K Total Debug Version: 172.6K Code, 73.4K Data, 246.0K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a problem with the resource descriptor support where the length of the StartDependentFn and StartDependentFnNoPrio descriptors were not included in cumulative descriptor offset, resulting in incorrect values for resource tags within resource descriptors appearing after a StartDependent* descriptor. Reported by Petr Vandrovec. ACPICA BZ 949. iASL and Preprocessor: Implemented full support for the #line directive to correctly track original source file line numbers through the .i preprocessor output file - for error and warning messages. iASL: Expand the allowable byte constants for address space IDs. Previously, the allowable range was 0x80-0xFF (user-defined spaces), now the range is 0x0A-0xFF to allow for custom and new IDs without changing the compiler. iASL: Add option to treat all warnings as errors (-we). ACPICA BZ 948. iASL: Add option to completely disable the preprocessor (-Pn). iASL: Now emit all error/warning messages to standard error (stderr) by default (instead of the previous stdout). ASL Test Suite (ASLTS): Reduce iASL warnings due to use of Switch(). Update for resource descriptor offset fix above. Update/cleanup error output routines. Enable and send iASL errors/warnings to an error logfile (error.txt). Send all other iASL output to a logfile (compiler.txt). Fixed several extraneous "unrecognized operator" messages. ---------------------------------------- 20 March 2012. Summary of changes for version 20120320: 1) ACPICA Core Subsystem: Enhanced the sleep/wake interfaces to optionally execute the _GTS method (Going To Sleep) and the _BFS method (Back From Sleep). Windows apparently does not execute these methods, and therefore these methods are often untested. It has been seen on some systems where the execution of these methods causes errors and also prevents the machine from entering S5. It is therefore suggested that host operating systems do not execute these methods by default. In the future, perhaps these methods can be optionally executed based on the age of the system and/or what is the newest version of Windows that the BIOS asks for via _OSI. Changed interfaces: AcpiEnterSleepState and AcpileaveSleepStatePrep. See the ACPICA reference and Linux BZ 13041. Lin Ming. Fixed a problem where the length of the local/common FADT was set too early. The local FADT table length cannot be set to the common length until the original length has been examined. There is code that checks the table length and sets various fields appropriately. This can affect older machines with early FADT versions. For example, this can cause inadvertent writes to the CST_CNT register. Julian Anastasov. Fixed a mapping issue related to a physical table override. Use the deferred mapping mechanism for tables loaded via the physical override OSL interface. This allows for early mapping before the virtual memory manager is available. Thomas Renninger, Bob Moore. Enhanced the automatic return-object repair code: Repair a common problem with predefined methods that are defined to return a variable-length Package of sub-objects. If there is only one sub-object, some BIOS ASL code mistakenly simply returns the single object instead of a Package with one sub- object. This new support will repair this error by wrapping a Package object around the original object, creating the correct and expected Package with one sub- object. Names that can be repaired in this manner include: _ALR, _CSD, _HPX, _MLS, _PLD, _PRT, _PSS, _TRT, _TSS, _BCL, _DOD, _FIX, and _Sx. ACPICA BZ 939. Changed the exception code returned for invalid ACPI paths passed as parameters to external interfaces such as AcpiEvaluateObject. Was AE_BAD_PARAMETER, now is the more sensible AE_BAD_PATHNAME. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 93.0K Code, 25.0K Data, 118.0K Total Debug Version: 172.5K Code, 73.2K Data, 245.7K Total Current Release: Non-Debug Version: 92.9K Code, 25.0K Data, 117.9K Total Debug Version: 172.5K Code, 73.2K Data, 245.7K Total 2) iASL Compiler/Disassembler and Tools: iASL: Added the infrastructure and initial implementation of a integrated C- like preprocessor. This will simplify BIOS development process by eliminating the need for a separate preprocessing step during builds. On Windows, it also eliminates the need to install a separate C compiler. ACPICA BZ 761. Some features including full #define() macro support are still under development. These preprocessor directives are supported: #define #elif #else #endif #error #if #ifdef #ifndef #include #pragma message #undef #warning In addition, these new command line options are supported: -D Define symbol for preprocessor use -li Create preprocessed output file (*.i) -P Preprocess only and create preprocessor output file (*.i) Table Compiler: Fixed a problem where the equals operator within an expression did not work properly. Updated iASL to use the current versions of Bison/Flex. Updated the Windows project file to invoke these tools from the standard location. ACPICA BZ 904. Versions supported: Flex for Windows: V2.5.4 Bison for Windows: V2.4.1 ---------------------------------------- 15 February 2012. Summary of changes for version 20120215: 1) ACPICA Core Subsystem: There have been some major changes to the sleep/wake support code, as described below (a - e). a) The AcpiLeaveSleepState has been split into two interfaces, similar to AcpiEnterSleepStatePrep and AcpiEnterSleepState. The new interface is AcpiLeaveSleepStatePrep. This allows the host to perform actions between the time the _BFS method is called and the _WAK method is called. NOTE: all hosts must update their wake/resume code or else sleep/wake will not work properly. Rafael Wysocki. b) In AcpiLeaveSleepState, now enable all runtime GPEs before calling the _WAK method. Some machines require that the GPEs are enabled before the _WAK method is executed. Thomas Renninger. c) In AcpiLeaveSleepState, now always clear the WAK_STS (wake status) bit. Some BIOS code assumes that WAK_STS will be cleared on resume and use it to determine whether the system is rebooting or resuming. Matthew Garrett. d) Move the invocations of _GTS (Going To Sleep) and _BFS (Back From Sleep) to match the ACPI specification requirement. Rafael Wysocki. e) Implemented full support for the ACPI 5.0 SleepStatus and SleepControl registers within the V5 FADT. This support adds two new files: hardware/hwesleep.c implements the support for the new registers. Moved all sleep/wake external interfaces to hardware/hwxfsleep.c. Added a new OSL interface for ACPI table overrides, AcpiOsPhysicalTableOverride. This interface allows the host to override a table via a physical address, instead of the logical address required by AcpiOsTableOverride. This simplifies the host implementation. Initial implementation by Thomas Renninger. The ACPICA implementation creates a single shared function for table overrides that attempts both a logical and a physical override. Expanded the OSL memory read/write interfaces to 64-bit data (AcpiOsReadMemory, AcpiOsWriteMemory.) This enables full 64-bit memory transfer support for GAS register structures passed to AcpiRead and AcpiWrite. Implemented the ACPI_REDUCED_HARDWARE option to allow the creation of a custom build of ACPICA that supports only the ACPI 5.0 reduced hardware (SoC) model. See the ACPICA reference for details. ACPICA BZ 942. This option removes about 10% of the code and 5% of the static data, and the following hardware ACPI features become unavailable: PM Event and Control registers SCI interrupt (and handler) Fixed Events General Purpose Events (GPEs) Global Lock ACPI PM timer FACS table (Waking vectors and Global Lock) Updated the unix tarball directory structure to match the ACPICA git source tree. This ensures that the generic unix makefiles work properly (in generate/unix). Also updated the Linux makefiles to match. ACPICA BZ 867. Updated the return value of the _REV predefined method to integer value 5 to reflect ACPI 5.0 support. Moved the external ACPI PM timer interface prototypes to the public acpixf.h file where they belong. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 92.8K Code, 24.9K Data, 117.7K Total Debug Version: 171.7K Code, 72.9K Data, 244.5K Total Current Release: Non-Debug Version: 93.0K Code, 25.0K Data, 118.0K Total Debug Version: 172.5K Code, 73.2K Data, 245.7K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Fixed a problem with the new ACPI 5.0 serial resource descriptors (I2C, SPI, UART) where the resource produce/consumer bit was incorrectly displayed. AcpiHelp: Add display of ACPI/PNP device IDs that are defined in the ACPI specification. ---------------------------------------- 11 January 2012. Summary of changes for version 20120111: 1) ACPICA Core Subsystem: Implemented a new mechanism to allow host device drivers to check for address range conflicts with ACPI Operation Regions. Both SystemMemory and SystemIO address spaces are supported. A new external interface, AcpiCheckAddressRange, allows drivers to check an address range against the ACPI namespace. See the ACPICA reference for additional details. Adds one new file, utilities/utaddress.c. Lin Ming, Bob Moore. Fixed several issues with the ACPI 5.0 FADT support: Add the sleep Control and Status registers, update the ACPI 5.0 flags, and update internal data structures to handle an FADT larger than 256 bytes. The size of the ACPI 5.0 FADT is 268 bytes. Updated all ACPICA copyrights and signons to 2012. Added the 2012 copyright to all module headers and signons, including the standard Linux header. This affects virtually every file in the ACPICA core subsystem, iASL compiler, and all ACPICA utilities. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 92.3K Code, 24.9K Data, 117.2K Total Debug Version: 170.8K Code, 72.6K Data, 243.4K Total Current Release: Non-Debug Version: 92.8K Code, 24.9K Data, 117.7K Total Debug Version: 171.7K Code, 72.9K Data, 244.5K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: fixed a problem with the automatic resource tag generation support. Fixes a problem where the resource tags are inadvertently not constructed if the table being disassembled contains external references to control methods. Moved the actual construction of the tags to after the final namespace is constructed (after 2nd parse is invoked due to external control method references.) ACPICA BZ 941. Table Compiler: Make all "generic" operators caseless. These are the operators like UINT8, String, etc. Making these caseless improves ease-of-use. ACPICA BZ 934. ---------------------------------------- 23 November 2011. Summary of changes for version 20111123: 0) ACPI 5.0 Support: This release contains full support for the ACPI 5.0 specification, as summarized below. Reduced Hardware Support: ------------------------- This support allows for ACPI systems without the usual ACPI hardware. This support is enabled by a flag in the revision 5 FADT. If it is set, ACPICA will not attempt to initialize or use any of the usual ACPI hardware. Note, when this flag is set, all of the following ACPI hardware is assumed to be not present and is not initialized or accessed: General Purpose Events (GPEs) Fixed Events (PM1a/PM1b and PM Control) Power Management Timer and Console Buttons (power/sleep) Real-time Clock Alarm Global Lock System Control Interrupt (SCI) The FACS is assumed to be non-existent ACPI Tables: ------------ All new tables and updates to existing tables are fully supported in the ACPICA headers (for use by device drivers), the disassembler, and the iASL Data Table Compiler. ACPI 5.0 defines these new tables: BGRT /* Boot Graphics Resource Table */ DRTM /* Dynamic Root of Trust for Measurement table */ FPDT /* Firmware Performance Data Table */ GTDT /* Generic Timer Description Table */ MPST /* Memory Power State Table */ PCCT /* Platform Communications Channel Table */ PMTT /* Platform Memory Topology Table */ RASF /* RAS Feature table */ Operation Regions/SpaceIDs: --------------------------- All new operation regions are fully supported by the iASL compiler, the disassembler, and the ACPICA runtime code (for dispatch to region handlers.) The new operation region Space IDs are: GeneralPurposeIo GenericSerialBus Resource Descriptors: --------------------- All new ASL resource descriptors are fully supported by the iASL compiler, the ASL/AML disassembler, and the ACPICA runtime Resource Manager code (including all new predefined resource tags). New descriptors are: FixedDma GpioIo GpioInt I2cSerialBus SpiSerialBus UartSerialBus ASL/AML Operators, New and Modified: ------------------------------------ One new operator is added, the Connection operator, which is used to associate a GeneralPurposeIo or GenericSerialBus resource descriptor with individual field objects within an operation region. Several new protocols are associated with the AccessAs operator. All are fully supported by the iASL compiler, disassembler, and runtime ACPICA AML interpreter: Connection // Declare Field Connection attributes AccessAs: AttribBytes (n) // Read/Write N-Bytes Protocol AccessAs: AttribRawBytes (n) // Raw Read/Write N-Bytes Protocol AccessAs: AttribRawProcessBytes (n) // Raw Process Call Protocol RawDataBuffer // Data type for Vendor Data fields Predefined ASL/AML Objects: --------------------------- All new predefined objects/control-methods are supported by the iASL compiler and the ACPICA runtime validation/repair (arguments and return values.) New predefined names include the following: Standard Predefined Names (Objects or Control Methods): _AEI, _CLS, _CPC, _CWS, _DEP, _DLM, _EVT, _GCP, _CRT, _GWS, _HRV, _PRE, _PSE, _SRT, _SUB. Resource Tags (Names used to access individual fields within resource descriptors): _DBT, _DPL, _DRS, _END, _FLC, _IOR, _LIN, _MOD, _PAR, _PHA, _PIN, _PPI, _POL, _RXL, _SLV, _SPE, _STB, _TXL, _VEN. ACPICA External Interfaces: --------------------------- Several new interfaces have been defined for use by ACPI-related device drivers and other host OS services: AcpiAcquireMutex and AcpiReleaseMutex: These interfaces allow the host OS to acquire and release AML mutexes that are defined in the DSDT/SSDT tables provided by the BIOS. They are intended to be used in conjunction with the ACPI 5.0 _DLM (Device Lock Method) in order to provide transaction-level mutual exclusion with the AML code/interpreter. AcpiGetEventResources: Returns the (formatted) resource descriptors as defined by the ACPI 5.0 _AEI object (ACPI Event Information). This object provides resource descriptors associated with hardware-reduced platform events, similar to the AcpiGetCurrentResources interface. Operation Region Handlers: For General Purpose IO and Generic Serial Bus operation regions, information about the Connection() object and any optional length information is passed to the region handler within the Context parameter. AcpiBufferToResource: This interface converts a raw AML buffer containing a resource template or resource descriptor to the ACPI_RESOURCE internal format suitable for use by device drivers. Can be used by an operation region handler to convert the Connection() buffer object into a ACPI_RESOURCE. Miscellaneous/Tools/TestSuites: ------------------------------- Support for extended _HID names (Four alpha characters instead of three). Support for ACPI 5.0 features in the AcpiExec and AcpiHelp utilities. Support for ACPI 5.0 features in the ASLTS test suite. Fully updated documentation (ACPICA and iASL reference documents.) ACPI Table Definition Language: ------------------------------- Support for this language was implemented and released as a subsystem of the iASL compiler in 2010. (See the iASL compiler User Guide.) Non-ACPI 5.0 changes for this release: -------------------------------------- 1) ACPICA Core Subsystem: Fix a problem with operation region declarations where a failure can occur if the region name and an argument that evaluates to an object (such as the region address) are in different namespace scopes. Lin Ming, ACPICA BZ 937. Do not abort an ACPI table load if an invalid space ID is found within. This will be caught later if the offending method is executed. ACPICA BZ 925. Fixed an issue with the FFixedHW space ID where the ID was not always recognized properly (Both ACPICA and iASL). ACPICA BZ 926. Fixed a problem with the 32-bit generation of the unix-specific OSL (osunixxf.c). Lin Ming, ACPICA BZ 936. Several changes made to enable generation with the GCC 4.6 compiler. ACPICA BZ 935. New error messages: Unsupported I/O requests (not 8/16/32 bit), and Index/Bank field registers out-of-range. 2) iASL Compiler/Disassembler and Tools: iASL: Implemented the __PATH__ operator, which returns the full pathname of the current source file. AcpiHelp: Automatically display expanded keyword information for all ASL operators. Debugger: Add "Template" command to disassemble/dump resource template buffers. Added a new master script to generate and execute the ASLTS test suite. Automatically handles 32- and 64-bit generation. See tests/aslts.sh iASL: Fix problem with listing generation during processing of the Switch() operator where AML listing was disabled until the entire Switch block was completed. iASL: Improve support for semicolon statement terminators. Fix "invalid character" message for some cases when the semicolon is used. Semicolons are now allowed after every grammar element. ACPICA BZ 927. iASL: Fixed some possible aliasing warnings during generation. ACPICA BZ 923. Disassembler: Fix problem with disassembly of the DataTableRegion operator where an inadvertent "Unhandled deferred opcode" message could be generated. 3) Example Code and Data Size These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 90.2K Code, 23.9K Data, 114.1K Total Debug Version: 165.6K Code, 68.4K Data, 234.0K Total Current Release: Non-Debug Version: 92.3K Code, 24.9K Data, 117.2K Total Debug Version: 170.8K Code, 72.6K Data, 243.4K Total ---------------------------------------- 22 September 2011. Summary of changes for version 20110922: 0) ACPI 5.0 News: Support for ACPI 5.0 in ACPICA has been underway for several months and will be released at the same time that ACPI 5.0 is officially released. The ACPI 5.0 specification is on track for release in the next few months. 1) ACPICA Core Subsystem: Fixed a problem where the maximum sleep time for the Sleep() operator was intended to be limited to two seconds, but was inadvertently limited to 20 seconds instead. Linux and Unix makefiles: Added header file dependencies to ensure correct generation of ACPICA core code and utilities. Also simplified the makefiles considerably through the use of the vpath variable to specify search paths. ACPICA BZ 924. 2) iASL Compiler/Disassembler and Tools: iASL: Implemented support to check the access length for all fields created to access named Resource Descriptor fields. For example, if a resource field is defined to be two bits, a warning is issued if a CreateXxxxField() is used with an incorrect bit length. This is implemented for all current resource descriptor names. ACPICA BZ 930. Disassembler: Fixed a byte ordering problem with the output of 24-bit and 56- bit integers. iASL: Fixed a couple of issues associated with variable-length package objects. 1) properly handle constants like One, Ones, Zero -- do not make a VAR_PACKAGE when these are used as a package length. 2) Allow the VAR_PACKAGE opcode (in addition to PACKAGE) when validating object types for predefined names. iASL: Emit statistics for all output files (instead of just the ASL input and AML output). Includes listings, hex files, etc. iASL: Added -G option to the table compiler to allow the compilation of custom ACPI tables. The only part of a table that is required is the standard 36- byte ACPI header. AcpiXtract: Ported to the standard ACPICA environment (with ACPICA headers), which also adds correct 64-bit support. Also, now all output filenames are completely lower case. AcpiExec: Ignore any non-AML tables (tables other than DSDT or SSDT) when loading table files. A warning is issued for any such tables. The only exception is an FADT. This also fixes a possible fault when attempting to load non-AML tables. ACPICA BZ 932. AcpiHelp: Added the AccessAs and Offset operators. Fixed a problem where a missing table terminator could cause a fault when using the -p option. AcpiSrc: Fixed a possible divide-by-zero fault when generating file statistics. 3) Example Code and Data Size These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release (VC 9.0): Non-Debug Version: 90.2K Code, 23.9K Data, 114.1K Total Debug Version: 165.6K Code, 68.4K Data, 234.0K Total Current Release (VC 9.0): Non-Debug Version: 90.2K Code, 23.9K Data, 114.1K Total Debug Version: 165.6K Code, 68.4K Data, 234.0K Total ---------------------------------------- 23 June 2011. Summary of changes for version 20110623: 1) ACPI CA Core Subsystem: Updated the predefined name repair mechanism to not attempt repair of a _TSS return object if a _PSS object is present. We can only sort the _TSS return package if there is no _PSS within the same scope. This is because if _PSS is present, the ACPI specification dictates that the _TSS Power Dissipation field is to be ignored, and therefore some BIOSs leave garbage values in the _TSS Power field(s). In this case, it is best to just return the _TSS package as- is. Reported by, and fixed with assistance from Fenghua Yu. Added an option to globally disable the control method return value validation and repair. This runtime option can be used to disable return value repair if this is causing a problem on a particular machine. Also added an option to AcpiExec (-dr) to set this disable flag. All makefiles and project files: Major changes to improve generation of ACPICA tools. ACPICA BZ 912: Reduce default optimization levels to improve compatibility For Linux, add strict-aliasing=0 for gcc 4 Cleanup and simplify use of command line defines Cleanup multithread library support Improve usage messages Linux-specific header: update handling of THREAD_ID and pthread. For the 32- bit case, improve casting to eliminate possible warnings, especially with the acpica tools. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release (VC 9.0): Non-Debug Version: 90.1K Code, 23.9K Data, 114.0K Total Debug Version: 165.6K Code, 68.4K Data, 234.0K Total Current Release (VC 9.0): Non-Debug Version: 90.2K Code, 23.9K Data, 114.1K Total Debug Version: 165.6K Code, 68.4K Data, 234.0K Total 2) iASL Compiler/Disassembler and Tools: With this release, a new utility named "acpihelp" has been added to the ACPICA package. This utility summarizes the ACPI specification chapters for the ASL and AML languages. It generates under Linux/Unix as well as Windows, and provides the following functionality: Find/display ASL operator(s) -- with description and syntax. Find/display ASL keyword(s) -- with exact spelling and descriptions. Find/display ACPI predefined name(s) -- with description, number of arguments, and the return value data type. Find/display AML opcode name(s) -- with opcode, arguments, and grammar. Decode/display AML opcode -- with opcode name, arguments, and grammar. Service Layers: Make multi-thread support configurable. Conditionally compile the multi-thread support so that threading libraries will not be linked if not necessary. The only tool that requires multi-thread support is AcpiExec. iASL: Update yyerrror/AslCompilerError for "const" errors. Newer versions of Bison appear to want the interface to yyerror to be a const char * (or at least this is a problem when generating iASL on some systems.) ACPICA BZ 923 Pierre Lejeune. Tools: Fix for systems where O_BINARY is not defined. Only used for Windows versions of the tools. ---------------------------------------- 27 May 2011. Summary of changes for version 20110527: 1) ACPI CA Core Subsystem: ASL Load() operator: Reinstate most restrictions on the incoming ACPI table signature. Now, only allow SSDT, OEMx, and a null signature. History: 1) Originally, we checked the table signature for "SSDT" or "PSDT". (PSDT is now obsolete.) 2) We added support for OEMx tables, signature "OEM" plus a fourth "don't care" character. 3) Valid tables were encountered with a null signature, so we just gave up on validating the signature, (05/2008). 4) We encountered non-AML tables such as the MADT, which caused interpreter errors and kernel faults. So now, we once again allow only SSDT, OEMx, and now, also a null signature. (05/2011). Added the missing _TDL predefined name to the global name list in order to enable validation. Affects both the core ACPICA code and the iASL compiler. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release (VC 9.0): Non-Debug Version: 90.0K Code, 23.8K Data, 113.8K Total Debug Version: 164.5K Code, 68.0K Data, 232.5K Total Current Release (VC 9.0): Non-Debug Version: 90.1K Code, 23.9K Data, 114.0K Total Debug Version: 165.6K Code, 68.4K Data, 234.0K Total 2) iASL Compiler/Disassembler and Tools: Debugger/AcpiExec: Implemented support for "complex" method arguments on the debugger command line. This adds support beyond simple integers -- including Strings, Buffers, and Packages. Includes support for nested packages. Increased the default command line buffer size to accommodate these arguments. See the ACPICA reference for details and syntax. ACPICA BZ 917. Debugger/AcpiExec: Implemented support for "default" method arguments for the Execute/Debug command. Now, the debugger will always invoke a control method with the required number of arguments -- even if the command line specifies none or insufficient arguments. It uses default integer values for any missing arguments. Also fixes a bug where only six method arguments maximum were supported instead of the required seven. Debugger/AcpiExec: Add a maximum buffer length parameter to AcpiOsGetLine and also return status in order to prevent buffer overruns. See the ACPICA reference for details and syntax. ACPICA BZ 921 iASL: Cleaned up support for Berkeley yacc. A general cleanup of code and makefiles to simplify support for the two different but similar parser generators, bison and yacc. Updated the generic unix makefile for gcc 4. The default gcc version is now expected to be 4 or greater, since options specific to gcc 4 are used. ---------------------------------------- 13 April 2011. Summary of changes for version 20110413: 1) ACPI CA Core Subsystem: Implemented support to execute a so-called "orphan" _REG method under the EC device. This change will force the execution of a _REG method underneath the EC device even if there is no corresponding operation region of type EmbeddedControl. Fixes a problem seen on some machines and apparently is compatible with Windows behavior. ACPICA BZ 875. Added more predefined methods that are eligible for automatic NULL package element removal. This change adds another group of predefined names to the list of names that can be repaired by having NULL package elements dynamically removed. This group are those methods that return a single variable- length package containing simple data types such as integers, buffers, strings. This includes: _ALx, _BCL, _CID,_ DOD, _EDL, _FIX, _PCL, _PLD, _PMD, _PRx, _PSL, _Sx, and _TZD. ACPICA BZ 914. Split and segregated all internal global lock functions to a new file, evglock.c. Updated internal address SpaceID for DataTable regions. Moved this internal space id in preparation for ACPI 5.0 changes that will include some new space IDs. This change should not affect user/host code. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release (VC 9.0): Non-Debug Version: 89.8K Code, 23.8K Data, 113.6K Total Debug Version: 164.2K Code, 67.9K Data, 232.1K Total Current Release (VC 9.0): Non-Debug Version: 90.0K Code, 23.8K Data, 113.8K Total Debug Version: 164.5K Code, 68.0K Data, 232.5K Total 2) iASL Compiler/Disassembler and Tools: iASL/DTC: Major update for new grammar features. Allow generic data types in custom ACPI tables. Field names are now optional. Any line can be split to multiple lines using the continuation char (\). Large buffers now use line- continuation character(s) and no colon on the continuation lines. See the grammar update in the iASL compiler reference. ACPI BZ 910,911. Lin Ming, Bob Moore. iASL: Mark ASL "Return()" and the simple "Return" as "Null" return statements. Since the parser stuffs a "zero" as the return value for these statements (due to the underlying AML grammar), they were seen as "return with value" by the iASL semantic checking. They are now seen correctly as "null" return statements. iASL: Check if a_REG declaration has a corresponding Operation Region. Adds a check for each _REG to ensure that there is in fact a corresponding operation region declaration in the same scope. If not, the _REG method is not very useful since it probably won't be executed. ACPICA BZ 915. iASL/DTC: Finish support for expression evaluation. Added a new expression parser that implements c-style operator precedence and parenthesization. ACPICA bugzilla 908. Disassembler/DTC: Remove support for () and <> style comments in data tables. Now that DTC has full expression support, we don't want to have comment strings that start with a parentheses or a less-than symbol. Now, only the standard /* and // comments are supported, as well as the bracket [] comments. AcpiXtract: Fix for RSDP and dynamic SSDT extraction. These tables have "unusual" headers in the acpidump file. Update the header validation to support these tables. Problem introduced in previous AcpiXtract version in the change to support "wrong checksum" error messages emitted by acpidump utility. iASL: Add a * option to generate all template files (as a synonym for ALL) as in "iasl -T *" or "iasl -T ALL". iASL/DTC: Do not abort compiler on fatal errors. We do not want to completely abort the compiler on "fatal" errors, simply should abort the current compile. This allows multiple compiles with a single (possibly wildcard) compiler invocation. ---------------------------------------- 16 March 2011. Summary of changes for version 20110316: 1) ACPI CA Core Subsystem: Fixed a problem caused by a _PRW method appearing at the namespace root scope during the setup of wake GPEs. A fault could occur if a _PRW directly under the root object was passed to the AcpiSetupGpeForWake interface. Lin Ming. Implemented support for "spurious" Global Lock interrupts. On some systems, a global lock interrupt can occur without the pending flag being set. Upon a GL interrupt, we now ensure that a thread is actually waiting for the lock before signaling GL availability. Rafael Wysocki, Bob Moore. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release (VC 9.0): Non-Debug Version: 89.7K Code, 23.7K Data, 113.4K Total Debug Version: 163.9K Code, 67.5K Data, 231.4K Total Current Release (VC 9.0): Non-Debug Version: 89.8K Code, 23.8K Data, 113.6K Total Debug Version: 164.2K Code, 67.9K Data, 232.1K Total 2) iASL Compiler/Disassembler and Tools: Implemented full support for the "SLIC" ACPI table. Includes support in the header files, disassembler, table compiler, and template generator. Bob Moore, Lin Ming. AcpiXtract: Correctly handle embedded comments and messages from AcpiDump. Apparently some or all versions of acpidump will occasionally emit a comment like "Wrong checksum", etc., into the dump file. This was causing problems for AcpiXtract. ACPICA BZ 905. iASL: Fix the Linux makefile by removing an inadvertent double file inclusion. ACPICA BZ 913. AcpiExec: Update installation of operation region handlers. Install one handler for a user-defined address space. This is used by the ASL test suite (ASLTS). ---------------------------------------- 11 February 2011. Summary of changes for version 20110211: 1) ACPI CA Core Subsystem: Added a mechanism to defer _REG methods for some early-installed handlers. Most user handlers should be installed before call to AcpiEnableSubsystem. However, Event handlers and region handlers should be installed after AcpiInitializeObjects. Override handlers for the "default" regions should be installed early, however. This change executes all _REG methods for the default regions (Memory/IO/PCI/DataTable) simultaneously to prevent any chicken/egg issues between them. ACPICA BZ 848. Implemented an optimization for GPE detection. This optimization will simply ignore GPE registers that contain no enabled GPEs -- there is no need to read the register since this information is available internally. This becomes more important on machines with a large GPE space. ACPICA bugzilla 884. Lin Ming. Suggestion from Joe Liu. Removed all use of the highly unreliable FADT revision field. The revision number in the FADT has been found to be completely unreliable and cannot be trusted. Only the actual table length can be used to infer the version. This change updates the ACPICA core and the disassembler so that both no longer even look at the FADT version and instead depend solely upon the FADT length. Fix an unresolved name issue for the no-debug and no-error-message source generation cases. The _AcpiModuleName was left undefined in these cases, but it is actually needed as a parameter to some interfaces. Define _AcpiModuleName as a null string in these cases. ACPICA Bugzilla 888. Split several large files (makefiles and project files updated) utglobal.c -> utdecode.c dbcomds.c -> dbmethod.c dbnames.c dsopcode.c -> dsargs.c dscontrol.c dsload.c -> dsload2.c aslanalyze.c -> aslbtypes.c aslwalks.c Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release (VC 9.0): Non-Debug Version: 89.7K Code, 23.7K Data, 113.4K Total Debug Version: 163.9K Code, 67.5K Data, 231.4K Total Current Release (VC 9.0): Non-Debug Version: 89.7K Code, 23.7K Data, 113.4K Total Debug Version: 163.9K Code, 67.5K Data, 231.4K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented the predefined macros __LINE__, __FILE__, and __DATE__. These are useful C-style macros with the standard definitions. ACPICA bugzilla 898. iASL/DTC: Added support for integer expressions and labels. Support for full expressions for all integer fields in all ACPI tables. Support for labels in "generic" portions of tables such as UEFI. See the iASL reference manual. Debugger: Added a command to display the status of global handlers. The "handlers" command will display op region, fixed event, and miscellaneous global handlers. installation status -- and for op regions, whether default or user-installed handler will be used. iASL: Warn if reserved method incorrectly returns a value. Many predefined names are defined such that they do not return a value. If implemented as a method, issue a warning if such a name explicitly returns a value. ACPICA Bugzilla 855. iASL: Added detection of GPE method name conflicts. Detects a conflict where there are two GPE methods of the form _Lxy and _Exy in the same scope. (For example, _L1D and _E1D in the same scope.) ACPICA bugzilla 848. iASL/DTC: Fixed a couple input scanner issues with comments and line numbers. Comment remover could get confused and miss a comment ending. Fixed a problem with line counter maintenance. iASL/DTC: Reduced the severity of some errors from fatal to error. There is no need to abort on simple errors within a field definition. Debugger: Simplified the output of the help command. All help output now in a single screen, instead of help subcommands. ACPICA Bugzilla 897. ---------------------------------------- 12 January 2011. Summary of changes for version 20110112: 1) ACPI CA Core Subsystem: Fixed a race condition between method execution and namespace walks that can possibly cause a fault. The problem was apparently introduced in version 20100528 as a result of a performance optimization that reduces the number of namespace walks upon method exit by using the delete_namespace_subtree function instead of the delete_namespace_by_owner function used previously. Bug is a missing namespace lock in the delete_namespace_subtree function. dana.myers@oracle.com Fixed several issues and a possible fault with the automatic "serialized" method support. History: This support changes a method to "serialized" on the fly if the method generates an AE_ALREADY_EXISTS error, indicating the possibility that it cannot handle reentrancy. This fix repairs a couple of issues seen in the field, especially on machines with many cores: 1) Delete method children only upon the exit of the last thread, so as to not delete objects out from under other running threads (and possibly causing a fault.) 2) Set the "serialized" bit for the method only upon the exit of the Last thread, so as to not cause deadlock when running threads attempt to exit. 3) Cleanup the use of the AML "MethodFlags" and internal method flags so that there is no longer any confusion between the two. Lin Ming, Bob Moore. Reported by dana.myers@oracle.com. Debugger: Now lock the namespace for duration of a namespace dump. Prevents issues if the namespace is changing dynamically underneath the debugger. Especially affects temporary namespace nodes, since the debugger displays these also. Updated the ordering of include files. The ACPICA headers should appear before any compiler-specific headers (stdio.h, etc.) so that acenv.h can set any necessary compiler-specific defines, etc. Affects the ACPI-related tools and utilities. Updated all ACPICA copyrights and signons to 2011. Added the 2011 copyright to all module headers and signons, including the Linux header. This affects virtually every file in the ACPICA core subsystem, iASL compiler, and all utilities. Added project files for MS Visual Studio 2008 (VC++ 9.0). The original project files for VC++ 6.0 are now obsolete. New project files can be found under acpica/generate/msvc9. See acpica/generate/msvc9/readme.txt for details. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 9.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release (VC 6.0): Non-Debug Version: 89.8K Code, 18.9K Data, 108.7K Total Debug Version: 166.6K Code, 52.1K Data, 218.7K Total Current Release (VC 9.0): Non-Debug Version: 89.7K Code, 23.7K Data, 113.4K Total Debug Version: 163.9K Code, 67.5K Data, 231.4K Total 2) iASL Compiler/Disassembler and Tools: iASL: Added generic data types to the Data Table compiler. Add "generic" data types such as UINT32, String, Unicode, etc., to simplify the generation of platform-defined tables such as UEFI. Lin Ming. iASL: Added listing support for the Data Table Compiler. Adds listing support (-l) to display actual binary output for each line of input code. ---------------------------------------- 09 December 2010. Summary of changes for version 20101209: 1) ACPI CA Core Subsystem: Completed the major overhaul of the GPE support code that was begun in July 2010. Major features include: removal of _PRW execution in ACPICA (host executes _PRWs anyway), cleanup of "wake" GPE interfaces and processing, changes to existing interfaces, simplification of GPE handler operation, and a handful of new interfaces: AcpiUpdateAllGpes AcpiFinishGpe AcpiSetupGpeForWake AcpiSetGpeWakeMask One new file, evxfgpe.c to consolidate all external GPE interfaces. See the ACPICA Programmer Reference for full details and programming information. See the new section 4.4 "General Purpose Event (GPE) Support" for a full overview, and section 8.7 "ACPI General Purpose Event Management" for programming details. ACPICA BZ 858,870,877. Matthew Garrett, Lin Ming, Bob Moore, Rafael Wysocki. Implemented a new GPE feature for Windows compatibility, the "Implicit Wake GPE Notify". This feature will automatically issue a Notify(2) on a device when a Wake GPE is received if there is no corresponding GPE method or handler. ACPICA BZ 870. Fixed a problem with the Scope() operator during table parse and load phase. During load phase (table load or method execution), the scope operator should not enter the target into the namespace. Instead, it should open a new scope at the target location. Linux BZ 19462, ACPICA BZ 882. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 89.8K Code, 18.9K Data, 108.7K Total Debug Version: 166.6K Code, 52.1K Data, 218.7K Total Current Release: Non-Debug Version: 89.9K Code, 19.0K Data, 108.9K Total Debug Version: 166.3K Code, 52.1K Data, 218.4K Total 2) iASL Compiler/Disassembler and Tools: iASL: Relax the alphanumeric restriction on _CID strings. These strings are "bus-specific" per the ACPI specification, and therefore any characters are acceptable. The only checks that can be performed are for a null string and perhaps for a leading asterisk. ACPICA BZ 886. iASL: Fixed a problem where a syntax error that caused a premature EOF condition on the source file emitted a very confusing error message. The premature EOF is now detected correctly. ACPICA BZ 891. Disassembler: Decode the AccessSize within a Generic Address Structure (byte access, word access, etc.) Note, this field does not allow arbitrary bit access, the size is encoded as 1=byte, 2=word, 3=dword, and 4=qword. New: AcpiNames utility - Example namespace dump utility. Shows an example of ACPICA configuration for a minimal namespace dump utility. Uses table and namespace managers, but no AML interpreter. Does not add any functionality over AcpiExec, it is a subset of AcpiExec. The purpose is to show how to partition and configure ACPICA. ACPICA BZ 883. AML Debugger: Increased the debugger buffer size for method return objects. Was 4K, increased to 16K. Also enhanced error messages for debugger method execution, including the buffer overflow case. ---------------------------------------- 13 October 2010. Summary of changes for version 20101013: 1) ACPI CA Core Subsystem: Added support to clear the PCIEXP_WAKE event. When clearing ACPI events, now clear the PCIEXP_WAKE_STS bit in the ACPI PM1 Status Register, via HwClearAcpiStatus. Original change from Colin King. ACPICA BZ 880. Changed the type of the predefined namespace object _TZ from ThermalZone to Device. This was found to be confusing to the host software that processes the various thermal zones, since _TZ is not really a ThermalZone. However, a Notify() can still be performed on it. ACPICA BZ 876. Suggestion from Rui Zhang. Added Windows Vista SP2 to the list of supported _OSI strings. The actual string is "Windows 2006 SP2". Eliminated duplicate code in AcpiUtExecute* functions. Now that the nsrepair code automatically repairs _HID-related strings, this type of code is no longer needed in Execute_HID, Execute_CID, and Execute_UID. ACPICA BZ 878. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 89.9K Code, 19.0K Data, 108.9K Total Debug Version: 166.3K Code, 52.1K Data, 218.4K Total Current Release: Non-Debug Version: 89.9K Code, 19.0K Data, 108.9K Total Debug Version: 166.3K Code, 52.1K Data, 218.4K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented additional compile-time validation for _HID strings. The non-hex prefix (such as "PNP" or "ACPI") must be uppercase, and the length of the string must be exactly seven or eight characters. For both _HID and _CID strings, all characters must be alphanumeric. ACPICA BZ 874. iASL: Allow certain "null" resource descriptors. Some BIOS code creates descriptors that are mostly or all zeros, with the expectation that they will be filled in at runtime. iASL now allows this as long as there is a "resource tag" (name) associated with the descriptor, which gives the ASL a handle needed to modify the descriptor. ACPICA BZ 873. Added single-thread support to the generic Unix application OSL. Primarily for iASL support, this change removes the use of semaphores in the single- threaded ACPICA tools/applications - increasing performance. The _MULTI_THREADED option was replaced by the (reverse) ACPI_SINGLE_THREADED option. ACPICA BZ 879. AcpiExec: several fixes for the 64-bit version. Adds XSDT support and support for 64-bit DSDT/FACS addresses in the FADT. Lin Ming. iASL: Moved all compiler messages to a new file, aslmessages.h. ---------------------------------------- 15 September 2010. Summary of changes for version 20100915: 1) ACPI CA Core Subsystem: Removed the AcpiOsDerivePciId OSL interface. The various host implementations of this function were not OS-dependent and are now obsolete and can be removed from all host OSLs. This function has been replaced by AcpiHwDerivePciId, which is now part of the ACPICA core code. AcpiHwDerivePciId has been implemented without recursion. Adds one new module, hwpci.c. ACPICA BZ 857. Implemented a dynamic repair for _HID and _CID strings. The following problems are now repaired at runtime: 1) Remove a leading asterisk in the string, and 2) the entire string is uppercased. Both repairs are in accordance with the ACPI specification and will simplify host driver code. ACPICA BZ 871. The ACPI_THREAD_ID type is no longer configurable, internally it is now always UINT64. This simplifies the ACPICA code, especially any printf output. UINT64 is the only common data type for all thread_id types across all operating systems. It is now up to the host OSL to cast the native thread_id type to UINT64 before returning the value to ACPICA (via AcpiOsGetThreadId). Lin Ming, Bob Moore. Added the ACPI_INLINE type to enhance the ACPICA configuration. The "inline" keyword is not standard across compilers, and this type allows inline to be configured on a per-compiler basis. Lin Ming. Made the system global AcpiGbl_SystemAwakeAndRunning publicly available. Added an extern for this boolean in acpixf.h. Some hosts utilize this value during suspend/restore operations. ACPICA BZ 869. All code that implements error/warning messages with the "ACPI:" prefix has been moved to a new module, utxferror.c. The UINT64_OVERLAY was moved to utmath.c, which is the only module where it is used. ACPICA BZ 829. Lin Ming, Bob Moore. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 89.1K Code, 19.0K Data, 108.1K Total Debug Version: 165.1K Code, 51.9K Data, 217.0K Total Current Release: Non-Debug Version: 89.9K Code, 19.0K Data, 108.9K Total Debug Version: 166.3K Code, 52.1K Data, 218.4K Total 2) iASL Compiler/Disassembler and Tools: iASL/Disassembler: Write ACPI errors to stderr instead of the output file. This keeps the output files free of random error messages that may originate from within the namespace/interpreter code. Used this opportunity to merge all ACPI:-style messages into a single new module, utxferror.c. ACPICA BZ 866. Lin Ming, Bob Moore. Tools: update some printfs for ansi warnings on size_t. Handle width change of size_t on 32-bit versus 64-bit generations. Lin Ming. ---------------------------------------- 06 August 2010. Summary of changes for version 20100806: 1) ACPI CA Core Subsystem: Designed and implemented a new host interface to the _OSI support code. This will allow the host to dynamically add or remove multiple _OSI strings, as well as install an optional handler that is called for each _OSI invocation. Also added a new AML debugger command, 'osi' to display and modify the global _OSI string table, and test support in the AcpiExec utility. See the ACPICA reference manual for full details. Lin Ming, Bob Moore. ACPICA BZ 836. New Functions: AcpiInstallInterface - Add an _OSI string. AcpiRemoveInterface - Delete an _OSI string. AcpiInstallInterfaceHandler - Install optional _OSI handler. Obsolete Functions: AcpiOsValidateInterface - no longer used. New Files: source/components/utilities/utosi.c Re-introduced the support to enable multi-byte transfers for Embedded Controller (EC) operation regions. A reported problem was found to be a bug in the host OS, not in the multi-byte support. Previously, the maximum data size passed to the EC operation region handler was a single byte. There are often EC Fields larger than one byte that need to be transferred, and it is useful for the EC driver to lock these as a single transaction. This change enables single transfers larger than 8 bits. This effectively changes the access to the EC space from ByteAcc to AnyAcc, and will probably require changes to the host OS Embedded Controller driver to enable 16/32/64/256- bit transfers in addition to 8-bit transfers. Alexey Starikovskiy, Lin Ming. Fixed a problem with the prototype for AcpiOsReadPciConfiguration. The prototype in acpiosxf.h had the output value pointer as a (void *). It should be a (UINT64 *). This may affect some host OSL code. Fixed a couple problems with the recently modified Linux makefiles for iASL and AcpiExec. These new makefiles place the generated object files in the local directory so that there can be no collisions between the files that are shared between them that are compiled with different options. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 88.3K Code, 18.8K Data, 107.1K Total Debug Version: 164.0K Code, 51.5K Data, 215.5K Total Current Release: Non-Debug Version: 89.1K Code, 19.0K Data, 108.1K Total Debug Version: 165.1K Code, 51.9K Data, 217.0K Total 2) iASL Compiler/Disassembler and Tools: iASL/Disassembler: Added a new option (-da, "disassemble all") to load the namespace from and disassemble an entire group of AML files. Useful for loading all of the AML tables for a given machine (DSDT, SSDT1...SSDTn) and disassembling with one simple command. ACPICA BZ 865. Lin Ming. iASL: Allow multiple invocations of -e option. This change allows multiple uses of -e on the command line: "-e ssdt1.dat -e ssdt2.dat". ACPICA BZ 834. Lin Ming. ---------------------------------------- 02 July 2010. Summary of changes for version 20100702: 1) ACPI CA Core Subsystem: Implemented several updates to the recently added GPE reference count support. The model for "wake" GPEs is changing to give the host OS complete control of these GPEs. Eventually, the ACPICA core will not execute any _PRW methods, since the host already must execute them. Also, additional changes were made to help ensure that the reference counts are kept in proper synchronization with reality. Rafael J. Wysocki. 1) Ensure that GPEs are not enabled twice during initialization. 2) Ensure that GPE enable masks stay in sync with the reference count. 3) Do not inadvertently enable GPEs when writing GPE registers. 4) Remove the internal wake reference counter and add new AcpiGpeWakeup interface. This interface will set or clear individual GPEs for wakeup. 5) Remove GpeType argument from AcpiEnable and AcpiDisable. These interfaces are now used for "runtime" GPEs only. Changed the behavior of the GPE install/remove handler interfaces. The GPE is no longer disabled during this process, as it was found to cause problems on some machines. Rafael J. Wysocki. Reverted a change introduced in version 20100528 to enable Embedded Controller multi-byte transfers. This change was found to cause problems with Index Fields and possibly Bank Fields. It will be reintroduced when these problems have been resolved. Fixed a problem with references to Alias objects within Package Objects. A reference to an Alias within the definition of a Package was not always resolved properly. Aliases to objects like Processors, Thermal zones, etc. were resolved to the actual object instead of a reference to the object as it should be. Package objects are only allowed to contain integer, string, buffer, package, and reference objects. Redhat bugzilla 608648. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 88.3K Code, 18.8K Data, 107.1K Total Debug Version: 164.1K Code, 51.5K Data, 215.6K Total Current Release: Non-Debug Version: 88.3K Code, 18.8K Data, 107.1K Total Debug Version: 164.0K Code, 51.5K Data, 215.5K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented a new compiler subsystem to allow definition and compilation of the non-AML ACPI tables such as FADT, MADT, SRAT, etc. These are called "ACPI Data Tables", and the new compiler is the "Data Table Compiler". This compiler is intended to simplify the existing error-prone process of creating these tables for the BIOS, as well as allowing the disassembly, modification, recompilation, and override of existing ACPI data tables. See the iASL User Guide for detailed information. iASL: Implemented a new Template Generator option in support of the new Data Table Compiler. This option will create examples of all known ACPI tables that can be used as the basis for table development. See the iASL documentation and the -T option. Disassembler and headers: Added support for the WDDT ACPI table (Watchdog Descriptor Table). Updated the Linux makefiles for iASL and AcpiExec to place the generated object files in the local directory so that there can be no collisions between the shared files between them that are generated with different options. Added support for Mac OS X in the Unix OSL used for iASL and AcpiExec. Use the #define __APPLE__ to enable this support. ---------------------------------------- 28 May 2010. Summary of changes for version 20100528: Note: The ACPI 4.0a specification was released on April 5, 2010 and is available at www.acpi.info. This is primarily an errata release. 1) ACPI CA Core Subsystem: Undefined ACPI tables: We are looking for the definitions for the following ACPI tables that have been seen in the field: ATKG, IEIT, GSCI. Implemented support to enable multi-byte transfers for Embedded Controller (EC) operation regions. Previously, the maximum data size passed to the EC operation region handler was a single byte. There are often EC Fields larger than one byte that need to be transferred, and it is useful for the EC driver to lock these as a single transaction. This change enables single transfers larger than 8 bits. This effectively changes the access to the EC space from ByteAcc to AnyAcc, and will probably require changes to the host OS Embedded Controller driver to enable 16/32/64/256-bit transfers in addition to 8- bit transfers. Alexey Starikovskiy, Lin Ming Implemented a performance enhancement for namespace search and access. This change enhances the performance of namespace searches and walks by adding a backpointer to the parent in each namespace node. On large namespaces, this change can improve overall ACPI performance by up to 9X. Adding a pointer to each namespace node increases the overall size of the internal namespace by about 5%, since each namespace entry usually consists of both a namespace node and an ACPI operand object. However, this is the first growth of the namespace in ten years. ACPICA bugzilla 817. Alexey Starikovskiy. Implemented a performance optimization that reduces the number of namespace walks. On control method exit, only walk the namespace if the method is known to have created namespace objects outside of its local scope. Previously, the entire namespace was traversed on each control method exit. This change can improve overall ACPI performance by up to 3X. Alexey Starikovskiy, Bob Moore. Added support to truncate I/O addresses to 16 bits for Windows compatibility. Some ASL code has been seen in the field that inadvertently has bits set above bit 15. This feature is optional and is enabled if the BIOS requests any Windows OSI strings. It can also be enabled by the host OS. Matthew Garrett, Bob Moore. Added support to limit the maximum time for the ASL Sleep() operator. To prevent accidental deep sleeps, limit the maximum time that Sleep() will actually sleep. Configurable, the default maximum is two seconds. ACPICA bugzilla 854. Added run-time validation support for the _WDG and_WED Microsoft predefined methods. These objects are defined by "Windows Instrumentation", and are not part of the ACPI spec. ACPICA BZ 860. Expanded all statistic counters used during namespace and device initialization from 16 to 32 bits in order to support very large namespaces. Replaced all instances of %d in printf format specifiers with %u since nearly all integers in ACPICA are unsigned. Fixed the exception namestring for AE_WAKE_ONLY_GPE. Was incorrectly returned as AE_NO_HANDLER. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 88.4K Code, 18.8K Data, 107.2K Total Debug Version: 164.2K Code, 51.5K Data, 215.7K Total Current Release: Non-Debug Version: 88.3K Code, 18.8K Data, 107.1K Total Debug Version: 164.1K Code, 51.5K Data, 215.6K Total 2) iASL Compiler/Disassembler and Tools: iASL: Added compiler support for the _WDG and_WED Microsoft predefined methods. These objects are defined by "Windows Instrumentation", and are not part of the ACPI spec. ACPICA BZ 860. AcpiExec: added option to disable the memory tracking mechanism. The -dt option will disable the tracking mechanism, which improves performance considerably. AcpiExec: Restructured the command line options into -d (disable) and -e (enable) options. ---------------------------------------- 28 April 2010. Summary of changes for version 20100428: 1) ACPI CA Core Subsystem: Implemented GPE support for dynamically loaded ACPI tables. For all GPEs, including FADT-based and GPE Block Devices, execute any _PRW methods in the new table, and process any _Lxx/_Exx GPE methods in the new table. Any runtime GPE that is referenced by an _Lxx/_Exx method in the new table is immediately enabled. Handles the FADT-defined GPEs as well as GPE Block Devices. Provides compatibility with other ACPI implementations. Two new files added, evgpeinit.c and evgpeutil.c. ACPICA BZ 833. Lin Ming, Bob Moore. Fixed a regression introduced in version 20100331 within the table manager where initial table loading could fail. This was introduced in the fix for AcpiReallocateRootTable. Also, renamed some of fields in the table manager data structures to clarify their meaning and use. Fixed a possible allocation overrun during internal object copy in AcpiUtCopySimpleObject. The original code did not correctly handle the case where the object to be copied was a namespace node. Lin Ming. ACPICA BZ 847. Updated the allocation dump routine, AcpiUtDumpAllocation and fixed a possible access beyond end-of-allocation. Also, now fully validate descriptor (size and type) before output. Lin Ming, Bob Moore. ACPICA BZ 847 Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 87.9K Code, 18.6K Data, 106.5K Total Debug Version: 163.5K Code, 51.3K Data, 214.8K Total Current Release: Non-Debug Version: 88.4K Code, 18.8K Data, 107.2K Total Debug Version: 164.2K Code, 51.5K Data, 215.7K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented Min/Max/Len/Gran validation for address resource descriptors. This change implements validation for the address fields that are common to all address-type resource descriptors. These checks are implemented: Checks for valid Min/Max, length within the Min/Max window, valid granularity, Min/Max a multiple of granularity, and _MIF/_MAF as per table 6-40 in the ACPI 4.0a specification. Also split the large aslrestype1.c and aslrestype2.c files into five new files. ACPICA BZ 840. iASL: Added support for the _Wxx predefined names. This support was missing and these names were not recognized by the compiler as valid predefined names. ACPICA BZ 851. iASL: Added an error for all predefined names that are defined to return no value and thus must be implemented as Control Methods. These include all of the _Lxx, _Exx, _Wxx, and _Qxx names, as well as some other miscellaneous names such as _DIS, _INI, _IRC, _OFF, _ON, and _PSx. ACPICA BZ 850, 856. iASL: Implemented the -ts option to emit hex AML data in ASL format, as an ASL Buffer. Allows ACPI tables to be easily included within ASL files, to be dynamically loaded via the Load() operator. Also cleaned up output for the - ta and -tc options. ACPICA BZ 853. Tests: Added a new file with examples of extended iASL error checking. Demonstrates the advanced error checking ability of the iASL compiler. Available at tests/misc/badcode.asl. ---------------------------------------- 31 March 2010. Summary of changes for version 20100331: 1) ACPI CA Core Subsystem: Completed a major update for the GPE support in order to improve support for shared GPEs and to simplify both host OS and ACPICA code. Added a reference count mechanism to support shared GPEs that require multiple device drivers. Several external interfaces have changed. One external interface has been removed. One new external interface was added. Most of the GPE external interfaces now use the GPE spinlock instead of the events mutex (and the Flags parameter for many GPE interfaces has been removed.) See the updated ACPICA Programmer Reference for details. Matthew Garrett, Bob Moore, Rafael Wysocki. ACPICA BZ 831. Changed: AcpiEnableGpe, AcpiDisableGpe, AcpiClearGpe, AcpiGetGpeStatus Removed: AcpiSetGpeType New: AcpiSetGpe Implemented write support for DataTable operation regions. These regions are defined via the DataTableRegion() operator. Previously, only read support was implemented. The ACPI specification allows DataTableRegions to be read/write, however. Implemented a new subsystem option to force a copy of the DSDT to local memory. Optionally copy the entire DSDT to local memory (instead of simply mapping it.) There are some (albeit very rare) BIOSs that corrupt or replace the original DSDT, creating the need for this option. Default is FALSE, do not copy the DSDT. Implemented detection of a corrupted or replaced DSDT. This change adds support to detect a DSDT that has been corrupted and/or replaced from outside the OS (by firmware). This is typically catastrophic for the system, but has been seen on some machines. Once this problem has been detected, the DSDT copy option can be enabled via system configuration. Lin Ming, Bob Moore. Fixed two problems with AcpiReallocateRootTable during the root table copy. When copying the root table to the new allocation, the length used was incorrect. The new size was used instead of the current table size, meaning too much data was copied. Also, the count of available slots for ACPI tables was not set correctly. Alexey Starikovskiy, Bob Moore. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 87.5K Code, 18.4K Data, 105.9K Total Debug Version: 163.4K Code, 51.1K Data, 214.5K Total Current Release: Non-Debug Version: 87.9K Code, 18.6K Data, 106.5K Total Debug Version: 163.5K Code, 51.3K Data, 214.8K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implement limited typechecking for values returned from predefined control methods. The type of any returned static (unnamed) object is now validated. For example, Return(1). ACPICA BZ 786. iASL: Fixed a predefined name object verification regression. Fixes a problem introduced in version 20100304. An error is incorrectly generated if a predefined name is declared as a static named object with a value defined using the keywords "Zero", "One", or "Ones". Lin Ming. iASL: Added Windows 7 support for the -g option (get local ACPI tables) by reducing the requested registry access rights. ACPICA BZ 842. Disassembler: fixed a possible fault when generating External() statements. Introduced in commit ae7d6fd: Properly handle externals with parent- prefix (carat). Fixes a string length allocation calculation. Lin Ming. ---------------------------------------- 04 March 2010. Summary of changes for version 20100304: 1) ACPI CA Core Subsystem: Fixed a possible problem with the AML Mutex handling function AcpiExReleaseMutex where the function could fault under the very rare condition when the interpreter has blocked, the interpreter lock is released, the interpreter is then reentered via the same thread, and attempts to acquire an AML mutex that was previously acquired. FreeBSD report 140979. Lin Ming. Implemented additional configuration support for the AML "Debug Object". Output from the debug object can now be enabled via a global variable, AcpiGbl_EnableAmlDebugObject. This will assist with remote machine debugging. This debug output is now available in the release version of ACPICA instead of just the debug version. Also, the entire debug output module can now be configured out of the ACPICA build if desired. One new file added, executer/exdebug.c. Lin Ming, Bob Moore. Added header support for the ACPI MCHI table (Management Controller Host Interface Table). This table was added in ACPI 4.0, but the defining document has only recently become available. Standardized output of integer values for ACPICA warnings/errors. Always use 0x prefix for hex output, always use %u for unsigned integer decimal output. Affects ACPI_INFO, ACPI_ERROR, ACPI_EXCEPTION, and ACPI_WARNING (about 400 invocations.) These invocations were converted from the original ACPI_DEBUG_PRINT invocations and were not consistent. ACPICA BZ 835. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 87.1K Code, 18.0K Data, 105.1K Total Debug Version: 163.5K Code, 50.9K Data, 214.4K Total Current Release: Non-Debug Version: 87.5K Code, 18.4K Data, 105.9K Total Debug Version: 163.4K Code, 51.1K Data, 214.5K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented typechecking support for static (non-control method) predefined named objects that are declared with the Name() operator. For example, the type of this object is now validated to be of type Integer: Name(_BBN, 1). This change migrates the compiler to using the core predefined name table instead of maintaining a local version. Added a new file, aslpredef.c. ACPICA BZ 832. Disassembler: Added support for the ACPI 4.0 MCHI table. ---------------------------------------- 21 January 2010. Summary of changes for version 20100121: 1) ACPI CA Core Subsystem: Added the 2010 copyright to all module headers and signons. This affects virtually every file in the ACPICA core subsystem, the iASL compiler, the tools/utilities, and the test suites. Implemented a change to the AcpiGetDevices interface to eliminate unnecessary invocations of the _STA method. In the case where a specific _HID is requested, do not run _STA until a _HID match is found. This eliminates potentially dozens of _STA calls during a search for a particular device/HID, which in turn can improve boot times. ACPICA BZ 828. Lin Ming. Implemented an additional repair for predefined method return values. Attempt to repair unexpected NULL elements within returned Package objects. Create an Integer of value zero, a NULL String, or a zero-length Buffer as appropriate. ACPICA BZ 818. Lin Ming, Bob Moore. Removed the obsolete ACPI_INTEGER data type. This type was introduced as the code was migrated from ACPI 1.0 (with 32-bit AML integers) to ACPI 2.0 (with 64-bit AML integers). It is now obsolete and this change removes it from the ACPICA code base, replaced by UINT64. The original typedef has been retained for now for compatibility with existing device driver code. ACPICA BZ 824. Removed the unused UINT32_STRUCT type, and the obsolete Integer64 field in the parse tree object. Added additional warning options for the gcc-4 generation. Updated the source accordingly. This includes some code restructuring to eliminate unreachable code, elimination of some gotos, elimination of unused return values, some additional casting, and removal of redundant declarations. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 87.0K Code, 18.0K Data, 105.0K Total Debug Version: 163.4K Code, 50.8K Data, 214.2K Total Current Release: Non-Debug Version: 87.1K Code, 18.0K Data, 105.1K Total Debug Version: 163.5K Code, 50.9K Data, 214.4K Total 2) iASL Compiler/Disassembler and Tools: No functional changes for this release. ---------------------------------------- 14 December 2009. Summary of changes for version 20091214: 1) ACPI CA Core Subsystem: Enhanced automatic data type conversions for predefined name repairs. This change expands the automatic repairs/conversions for predefined name return values to make Integers, Strings, and Buffers fully interchangeable. Also, a Buffer can be converted to a Package of Integers if necessary. The nsrepair.c module was completely restructured. Lin Ming, Bob Moore. Implemented automatic removal of null package elements during predefined name repairs. This change will automatically remove embedded and trailing NULL package elements from returned package objects that are defined to contain a variable number of sub-packages. The driver is then presented with a package with no null elements to deal with. ACPICA BZ 819. Implemented a repair for the predefined _FDE and _GTM names. The expected return value for both names is a Buffer of 5 DWORDs. This repair fixes two possible problems (both seen in the field), where a package of integers is returned, or a buffer of BYTEs is returned. With assistance from Jung-uk Kim. Implemented additional module-level code support. This change will properly execute module-level code that is not at the root of the namespace (under a Device object, etc.). Now executes the code within the current scope instead of the root. ACPICA BZ 762. Lin Ming. Fixed possible mutex acquisition errors when running _REG methods. Fixes a problem where mutex errors can occur when running a _REG method that is in the same scope as a method-defined operation region or an operation region under a module-level IF block. This type of code is rare, so the problem has not been seen before. ACPICA BZ 826. Lin Ming, Bob Moore. Fixed a possible memory leak during module-level code execution. An object could be leaked for each block of executed module-level code if the interpreter slack mode is enabled This change deletes any implicitly returned object from the module-level code block. Lin Ming. Removed messages for successful predefined repair(s). The repair mechanism was considered too wordy. Now, messages are only unconditionally emitted if the return object cannot be repaired. Existing messages for successful repairs were converted to ACPI_DEBUG_PRINT messages for now. ACPICA BZ 827. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 86.6K Code, 18.2K Data, 104.8K Total Debug Version: 162.7K Code, 50.8K Data, 213.5K Total Current Release: Non-Debug Version: 87.0K Code, 18.0K Data, 105.0K Total Debug Version: 163.4K Code, 50.8K Data, 214.2K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a regression introduced in 20091112 where intermediate .SRC files were no longer automatically removed at the termination of the compile. acpiexec: Implemented the -f option to specify default region fill value. This option specifies the value used to initialize buffers that simulate operation regions. Default value is zero. Useful for debugging problems that depend on a specific initial value for a region or field. ---------------------------------------- 12 November 2009. Summary of changes for version 20091112: 1) ACPI CA Core Subsystem: Implemented a post-order callback to AcpiWalkNamespace. The existing interface only has a pre-order callback. This change adds an additional parameter for a post-order callback which will be more useful for bus scans. ACPICA BZ 779. Lin Ming. Updated the ACPICA Programmer Reference. Modified the behavior of the operation region memory mapping cache for SystemMemory. Ensure that the memory mappings created for operation regions do not cross 4K page boundaries. Crossing a page boundary while mapping regions can cause kernel warnings on some hosts if the pages have different attributes. Such regions are probably BIOS bugs, and this is the workaround. Linux BZ 14445. Lin Ming. Implemented an automatic repair for predefined methods that must return sorted lists. This change will repair (by sorting) packages returned by _ALR, _PSS, and _TSS. Drivers can now assume that the packages are correctly sorted and do not contain NULL package elements. Adds one new file, namespace/nsrepair2.c. ACPICA BZ 784. Lin Ming, Bob Moore. Fixed a possible fault during predefined name validation if a return Package object contains NULL elements. Also adds a warning if a NULL element is followed by any non-null elements. ACPICA BZ 813, 814. Future enhancement may include repair or removal of all such NULL elements where possible. Implemented additional module-level executable AML code support. This change will execute module-level code that is not at the root of the namespace (under a Device object, etc.) at table load time. Module-level executable AML code has been illegal since ACPI 2.0. ACPICA BZ 762. Lin Ming. Implemented a new internal function to create Integer objects. This function simplifies miscellaneous object creation code. ACPICA BZ 823. Reduced the severity of predefined repair messages, Warning to Info. Since the object was successfully repaired, a warning is too severe. Reduced to an info message for now. These messages may eventually be changed to debug- only. ACPICA BZ 812. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 85.8K Code, 18.0K Data, 103.8K Total Debug Version: 161.8K Code, 50.6K Data, 212.4K Total Current Release: Non-Debug Version: 86.6K Code, 18.2K Data, 104.8K Total Debug Version: 162.7K Code, 50.8K Data, 213.5K Total 2) iASL Compiler/Disassembler and Tools: iASL: Implemented Switch() with While(1) so that Break works correctly. This change correctly implements the Switch operator with a surrounding While(1) so that the Break operator works as expected. ACPICA BZ 461. Lin Ming. iASL: Added a message if a package initializer list is shorter than package length. Adds a new remark for a Package() declaration if an initializer list exists, but is shorter than the declared length of the package. Although technically legal, this is probably a coding error and it is seen in the field. ACPICA BZ 815. Lin Ming, Bob Moore. iASL: Fixed a problem where the compiler could fault after the maximum number of errors was reached (200). acpixtract: Fixed a possible warning for pointer cast if the compiler warning level set very high. ---------------------------------------- 13 October 2009. Summary of changes for version 20091013: 1) ACPI CA Core Subsystem: Fixed a problem where an Operation Region _REG method could be executed more than once. If a custom address space handler is installed by the host before the "initialize operation regions" phase of the ACPICA initialization, any _REG methods for that address space could be executed twice. This change fixes the problem. ACPICA BZ 427. Lin Ming. Fixed a possible memory leak for the Scope() ASL operator. When the exact invocation of "Scope(\)" is executed (change scope to root), one internal operand object was leaked. Lin Ming. Implemented a run-time repair for the _MAT predefined method. If the _MAT return value is defined as a Field object in the AML, and the field size is less than or equal to the default width of an integer (32 or 64),_MAT can incorrectly return an Integer instead of a Buffer. ACPICA now automatically repairs this problem. ACPICA BZ 810. Implemented a run-time repair for the _BIF and _BIX predefined methods. The "OEM Information" field is often incorrectly returned as an Integer with value zero if the field is not supported by the platform. This is due to an ambiguity in the ACPI specification. The field should always be a string. ACPICA now automatically repairs this problem by returning a NULL string within the returned Package. ACPICA BZ 807. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 85.6K Code, 18.0K Data, 103.6K Total Debug Version: 161.7K Code, 50.9K Data, 212.6K Total Current Release: Non-Debug Version: 85.8K Code, 18.0K Data, 103.8K Total Debug Version: 161.8K Code, 50.6K Data, 212.4K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Fixed a problem where references to external symbols that contained one or more parent-prefixes (carats) were not handled correctly, possibly causing a fault. ACPICA BZ 806. Lin Ming. Disassembler: Restructured the code so that all functions that handle external symbols are in a single module. One new file is added, common/dmextern.c. AML Debugger: Added a max count argument for the Batch command (which executes multiple predefined methods within the namespace.) iASL: Updated the compiler documentation (User Reference.) Available at http://www.acpica.org/documentation/. ACPICA BZ 750. AcpiXtract: Updated for Lint and other formatting changes. Close all open files. ---------------------------------------- 03 September 2009. Summary of changes for version 20090903: 1) ACPI CA Core Subsystem: For Windows Vista compatibility, added the automatic execution of an _INI method located at the namespace root (\_INI). This method is executed at table load time. This support is in addition to the automatic execution of \_SB._INI. Lin Ming. Fixed a possible memory leak in the interpreter for AML package objects if the package initializer list is longer than the defined size of the package. This apparently can only happen if the BIOS changes the package size on the fly (seen in a _PSS object), as ASL compilers do not allow this. The interpreter will truncate the package to the defined size (and issue an error message), but previously could leave the extra objects undeleted if they were pre-created during the argument processing (such is the case if the package consists of a number of sub-packages as in the _PSS.) ACPICA BZ 805. Fixed a problem seen when a Buffer or String is stored to itself via ASL. This has been reported in the field. Previously, ACPICA would zero out the buffer/string. Now, the operation is treated as a noop. Provides Windows compatibility. ACPICA BZ 803. Lin Ming. Removed an extraneous error message for ASL constructs of the form Store(LocalX,LocalX) when LocalX is uninitialized. These curious statements are seen in many BIOSs and are once again treated as NOOPs and no error is emitted when they are encountered. ACPICA BZ 785. Fixed an extraneous warning message if a _DSM reserved method returns a Package object. _DSM can return any type of object, so validation on the return type cannot be performed. ACPICA BZ 802. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 85.5K Code, 18.0K Data, 103.5K Total Debug Version: 161.6K Code, 50.9K Data, 212.5K Total Current Release: Non-Debug Version: 85.6K Code, 18.0K Data, 103.6K Total Debug Version: 161.7K Code, 50.9K Data, 212.6K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a problem with the use of the Alias operator and Resource Templates. The correct alias is now constructed and no error is emitted. ACPICA BZ 738. iASL: Implemented the -I option to specify additional search directories for include files. Allows multiple additional search paths for include files. Directories are searched in the order specified on the command line (after the local directory is searched.) ACPICA BZ 800. iASL: Fixed a problem where the full pathname for include files was not emitted for warnings/errors. This caused the IDE support to not work properly. ACPICA BZ 765. iASL: Implemented the -@ option to specify a Windows-style response file containing additional command line options. ACPICA BZ 801. AcpiExec: Added support to load multiple AML files simultaneously (such as a DSDT and multiple SSDTs). Also added support for wildcards within the AML pathname. These features allow all machine tables to be easily loaded and debugged together. ACPICA BZ 804. Disassembler: Added missing support for disassembly of HEST table Error Bank subtables. ---------------------------------------- 30 July 2009. Summary of changes for version 20090730: The ACPI 4.0 implementation for ACPICA is complete with this release. 1) ACPI CA Core Subsystem: ACPI 4.0: Added header file support for all new and changed ACPI tables. Completely new tables are: IBFT, IVRS, MSCT, and WAET. Tables that are new for ACPI 4.0, but have previously been supported in ACPICA are: CPEP, BERT, EINJ, ERST, and HEST. Other newly supported tables are: UEFI and WDAT. There have been some ACPI 4.0 changes to other existing tables. Split the large actbl1.h header into the existing actbl2.h header. ACPICA BZ 774. ACPI 4.0: Implemented predefined name validation for all new names. There are 31 new names in ACPI 4.0. The predefined validation module was split into two files. The new file is namespace/nsrepair.c. ACPICA BZ 770. Implemented support for so-called "module-level executable code". This is executable AML code that exists outside of any control method and is intended to be executed at table load time. Although illegal since ACPI 2.0, this type of code still exists and is apparently still being created. Blocks of this code are now detected and executed as intended. Currently, the code blocks must exist under either an If, Else, or While construct; these are the typical cases seen in the field. ACPICA BZ 762. Lin Ming. Implemented an automatic dynamic repair for predefined names that return nested Package objects. This applies to predefined names that are defined to return a variable-length Package of sub-packages. If the number of sub- packages is one, BIOS code is occasionally seen that creates a simple single package with no sub-packages. This code attempts to fix the problem by wrapping a new package object around the existing package. These methods can be repaired: _ALR, _CSD, _HPX, _MLS, _PRT, _PSS, _TRT, and _TSS. ACPICA BZ 790. Fixed a regression introduced in 20090625 for the AcpiGetDevices interface. The _HID/_CID matching was broken and no longer matched IDs correctly. ACPICA BZ 793. Fixed a problem with AcpiReset where the reset would silently fail if the register was one of the protected I/O ports. AcpiReset now bypasses the port validation mechanism. This may eventually be driven into the AcpiRead/Write interfaces. Fixed a regression related to the recent update of the AcpiRead/Write interfaces. A sleep/suspend could fail if the optional PM2 Control register does not exist during an attempt to write the Bus Master Arbitration bit. (However, some hosts already delete the code that writes this bit, and the code may in fact be obsolete at this date.) ACPICA BZ 799. Fixed a problem where AcpiTerminate could fault if inadvertently called twice in succession. ACPICA BZ 795. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 84.7K Code, 17.8K Data, 102.5K Total Debug Version: 160.5K Code, 50.6K Data, 211.1K Total Current Release: Non-Debug Version: 85.5K Code, 18.0K Data, 103.5K Total Debug Version: 161.6K Code, 50.9K Data, 212.5K Total 2) iASL Compiler/Disassembler and Tools: ACPI 4.0: Implemented disassembler support for all new ACPI tables and changes to existing tables. ACPICA BZ 775. ---------------------------------------- 25 June 2009. Summary of changes for version 20090625: The ACPI 4.0 Specification was released on June 16 and is available at www.acpi.info. ACPICA implementation of ACPI 4.0 is underway and will continue for the next few releases. 1) ACPI CA Core Subsystem: ACPI 4.0: Implemented interpreter support for the IPMI operation region address space. Includes support for bi-directional data buffers and an IPMI address space handler (to be installed by an IPMI device driver.) ACPICA BZ 773. Lin Ming. ACPI 4.0: Added changes for existing ACPI tables - FACS and SRAT. Includes support in both the header files and the disassembler. Completed a major update for the AcpiGetObjectInfo external interface. Changes include: - Support for variable, unlimited length HID, UID, and CID strings. - Support Processor objects the same as Devices (HID,UID,CID,ADR,STA, etc.) - Call the _SxW power methods on behalf of a device object. - Determine if a device is a PCI root bridge. - Change the ACPI_BUFFER parameter to ACPI_DEVICE_INFO. These changes will require an update to all callers of this interface. See the updated ACPICA Programmer Reference for details. One new source file has been added - utilities/utids.c. ACPICA BZ 368, 780. Updated the AcpiRead and AcpiWrite external interfaces to support 64-bit transfers. The Value parameter has been extended from 32 bits to 64 bits in order to support new ACPI 4.0 tables. These changes will require an update to all callers of these interfaces. See the ACPICA Programmer Reference for details. ACPICA BZ 768. Fixed several problems with AcpiAttachData. The handler was not invoked when the host node was deleted. The data sub-object was not automatically deleted when the host node was deleted. The interface to the handler had an unused parameter, this was removed. ACPICA BZ 778. Enhanced the function that dumps ACPI table headers. All non-printable characters in the string fields are now replaced with '?' (Signature, OemId, OemTableId, and CompilerId.) ACPI tables with non-printable characters in these fields are occasionally seen in the field. ACPICA BZ 788. Fixed a problem with predefined method repair code where the code that attempts to repair/convert an object of incorrect type is only executed on the first time the predefined method is called. The mechanism that disables warnings on subsequent calls was interfering with the repair mechanism. ACPICA BZ 781. Fixed a possible memory leak in the predefined validation/repair code when a buffer is automatically converted to an expected string object. Removed obsolete 16-bit files from the distribution and from the current git tree head. ACPICA BZ 776. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 83.4K Code, 17.5K Data, 100.9K Total Debug Version: 158.9K Code, 50.0K Data, 208.9K Total Current Release: Non-Debug Version: 84.7K Code, 17.8K Data, 102.5K Total Debug Version: 160.5K Code, 50.6K Data, 211.1K Total 2) iASL Compiler/Disassembler and Tools: ACPI 4.0: iASL and Disassembler - implemented support for the new IPMI operation region keyword. ACPICA BZ 771, 772. Lin Ming. ACPI 4.0: iASL - implemented compile-time validation support for all new predefined names and control methods (31 total). ACPICA BZ 769. ---------------------------------------- 21 May 2009. Summary of changes for version 20090521: 1) ACPI CA Core Subsystem: Disabled the preservation of the SCI enable bit in the PM1 control register. The SCI enable bit (bit 0, SCI_EN) is defined by the ACPI specification to be a "preserved" bit - "OSPM always preserves this bit position", section 4.7.3.2.1. However, some machines fail if this bit is in fact preserved because the bit needs to be explicitly set by the OS as a workaround. No machines fail if the bit is not preserved. Therefore, ACPICA no longer attempts to preserve this bit. Fixed a problem in AcpiRsGetPciRoutingTableLength where an invalid or incorrectly formed _PRT package could cause a fault. Added validation to ensure that each package element is actually a sub-package. Implemented a new interface to install or override a single control method, AcpiInstallMethod. This interface is useful when debugging in order to repair an existing method or to install a missing method without having to override the entire ACPI table. See the ACPICA Programmer Reference for use and examples. Lin Ming, Bob Moore. Fixed several reference count issues with the DdbHandle object that is created from a Load or LoadTable operator. Prevent premature deletion of the object. Also, mark the object as invalid once the table has been unloaded. This is needed because the handle itself may not be deleted after the table unload, depending on whether it has been stored in a named object by the caller. Lin Ming. Fixed a problem with Mutex Sync Levels. Fixed a problem where if multiple mutexes of the same sync level are acquired but then not released in strict opposite order, the internally maintained Current Sync Level becomes confused and can cause subsequent execution errors. ACPICA BZ 471. Changed the allowable release order for ASL mutex objects. The ACPI 4.0 specification has been changed to make the SyncLevel for mutex objects more useful. When releasing a mutex, the SyncLevel of the mutex must now be the same as the current sync level. This makes more sense than the previous rule (SyncLevel less than or equal). This change updates the code to match the specification. Fixed a problem with the local version of the AcpiOsPurgeCache function. The (local) cache must be locked during all cache object deletions. Andrew Baumann. Updated the Load operator to use operation region interfaces. This replaces direct memory mapping with region access calls. Now, all region accesses go through the installed region handler as they should. Simplified and optimized the NsGetNextNode function. Reduced parameter count and reduced code for this frequently used function. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 82.8K Code, 17.5K Data, 100.3K Total Debug Version: 158.0K Code, 49.9K Data, 207.9K Total Current Release: Non-Debug Version: 83.4K Code, 17.5K Data, 100.9K Total Debug Version: 158.9K Code, 50.0K Data, 208.9K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Fixed some issues with DMAR, HEST, MADT tables. Some problems with sub-table disassembly and handling invalid sub-tables. Attempt recovery after an invalid sub-table ID. ---------------------------------------- 22 April 2009. Summary of changes for version 20090422: 1) ACPI CA Core Subsystem: Fixed a compatibility issue with the recently released I/O port protection mechanism. For windows compatibility, 1) On a port protection violation, simply ignore the request and do not return an exception (allow the control method to continue execution.) 2) If only part of the request overlaps a protected port, read/write the individual ports that are not protected. Linux BZ 13036. Lin Ming Enhanced the execution of the ASL/AML BreakPoint operator so that it actually breaks into the AML debugger if the debugger is present. This matches the ACPI-defined behavior. Fixed several possible warnings related to the use of the configurable ACPI_THREAD_ID. This type can now be configured as either an integer or a pointer with no warnings. Also fixes several warnings in printf-like statements for the 64-bit build when the type is configured as a pointer. ACPICA BZ 766, 767. Fixed a number of possible warnings when compiling with gcc 4+ (depending on warning options.) Examples include printf formats, aliasing, unused globals, missing prototypes, missing switch default statements, use of non-ANSI library functions, use of non-ANSI constructs. See generate/unix/Makefile for a list of warning options used with gcc 3 and 4. ACPICA BZ 735. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 82.6K Code, 17.6K Data, 100.2K Total Debug Version: 157.7K Code, 49.9K Data, 207.6K Total Current Release: Non-Debug Version: 82.8K Code, 17.5K Data, 100.3K Total Debug Version: 158.0K Code, 49.9K Data, 207.9K Total 2) iASL Compiler/Disassembler and Tools: iASL: Fixed a generation warning from Bison 2.3 and fixed several warnings on the 64-bit build. iASL: Fixed a problem where the Unix/Linux versions of the compiler could not correctly digest Windows/DOS formatted files (with CR/LF). iASL: Added a new option for "quiet mode" (-va) that produces only the compilation summary, not individual errors and warnings. Useful for large batch compilations. AcpiExec: Implemented a new option (-z) to enable a forced semaphore/mutex timeout that can be used to detect hang conditions during execution of AML code (includes both internal semaphores and AML-defined mutexes and events.) Added new makefiles for the generation of acpica in a generic unix-like environment. These makefiles are intended to generate the acpica tools and utilities from the original acpica git source tree structure. Test Suites: Updated and cleaned up the documentation files. Updated the copyrights to 2009, affecting all source files. Use the new version of iASL with quiet mode. Increased the number of available semaphores in the Windows OSL, allowing the aslts to execute fully on Windows. For the Unix OSL, added an alternate implementation of the semaphore timeout to allow aslts to execute fully on Cygwin. ---------------------------------------- 20 March 2009. Summary of changes for version 20090320: 1) ACPI CA Core Subsystem: Fixed a possible race condition between AcpiWalkNamespace and dynamic table unloads. Added a reader/writer locking mechanism to allow multiple concurrent namespace walks (readers), but block a dynamic table unload until it can gain exclusive write access to the namespace. This fixes a problem where a table unload could (possibly catastrophically) delete the portion of the namespace that is currently being examined by a walk. Adds a new file, utlock.c, that implements the reader/writer lock mechanism. ACPICA BZ 749. Fixed a regression introduced in version 20090220 where a change to the FADT handling could cause the ACPICA subsystem to access non-existent I/O ports. Modified the handling of FADT register and table (FACS/DSDT) addresses. The FADT can contain both 32-bit and 64-bit versions of these addresses. Previously, the 64-bit versions were favored, meaning that if both 32 and 64 versions were valid, but not equal, the 64-bit version was used. This was found to cause some machines to fail. Now, in this case, the 32-bit version is used instead. This now matches the Windows behavior. Implemented a new mechanism to protect certain I/O ports. Provides Microsoft compatibility and protects the standard PC I/O ports from access via AML code. Adds a new file, hwvalid.c Fixed a possible extraneous warning message from the FADT support. The message warns of a 32/64 length mismatch between the legacy and GAS definitions for a register. Removed the obsolete AcpiOsValidateAddress OSL interface. This interface is made obsolete by the port protection mechanism above. It was previously used to validate the entire address range of an operation region, which could be incorrect if the range included illegal ports, but fields within the operation region did not actually access those ports. Validation is now performed on a per-field basis instead of the entire region. Modified the handling of the PM1 Status Register ignored bit (bit 11.) Ignored bits must be "preserved" according to the ACPI spec. Usually, this means a read/modify/write when writing to the register. However, for status registers, writing a one means clear the event. Writing a zero means preserve the event (do not clear.) This behavior is clarified in the ACPI 4.0 spec, and the ACPICA code now simply always writes a zero to the ignored bit. Modified the handling of ignored bits for the PM1 A/B Control Registers. As per the ACPI specification, for the control registers, preserve (read/modify/write) all bits that are defined as either reserved or ignored. Updated the handling of write-only bits in the PM1 A/B Control Registers. When reading the register, zero the write-only bits as per the ACPI spec. ACPICA BZ 443. Lin Ming. Removed "Linux" from the list of supported _OSI strings. Linux no longer wants to reply true to this request. The Windows strings are the only paths through the AML that are tested and known to work properly. Previous Release: Non-Debug Version: 82.0K Code, 17.5K Data, 99.5K Total Debug Version: 156.9K Code, 49.8K Data, 206.7K Total Current Release: Non-Debug Version: 82.6K Code, 17.6K Data, 100.2K Total Debug Version: 157.7K Code, 49.9K Data, 207.6K Total 2) iASL Compiler/Disassembler and Tools: Acpiexec: Split the large aeexec.c file into two new files, aehandlers.c and aetables.c ---------------------------------------- 20 February 2009. Summary of changes for version 20090220: 1) ACPI CA Core Subsystem: Optimized the ACPI register locking. Removed locking for reads from the ACPI bit registers in PM1 Status, Enable, Control, and PM2 Control. The lock is not required when reading the single-bit registers. The AcpiGetRegisterUnlocked function is no longer needed and has been removed. This will improve performance for reads on these registers. ACPICA BZ 760. Fixed the parameter validation for AcpiRead/Write. Now return AE_BAD_PARAMETER if the input register pointer is null, and AE_BAD_ADDRESS if the register has an address of zero. Previously, these cases simply returned AE_OK. For optional registers such as PM1B status/enable/control, the caller should check for a valid register address before calling. ACPICA BZ 748. Renamed the external ACPI bit register access functions. Renamed AcpiGetRegister and AcpiSetRegister to clarify the purpose of these functions. The new names are AcpiReadBitRegister and AcpiWriteBitRegister. Also, restructured the code for these functions by simplifying the code path and condensing duplicate code to reduce code size. Added new functions to transparently handle the possibly split PM1 A/B registers. AcpiHwReadMultiple and AcpiHwWriteMultiple. These two functions now handle the split registers for PM1 Status, Enable, and Control. ACPICA BZ 746. Added a function to handle the PM1 control registers, AcpiHwWritePm1Control. This function writes both of the PM1 control registers (A/B). These registers are different than the PM1 A/B status and enable registers in that different values can be written to the A/B registers. Most notably, the SLP_TYP bits can be different, as per the values returned from the _Sx predefined methods. Removed an extra register write within AcpiHwClearAcpiStatus. This function was writing an optional PM1B status register twice. The existing call to the low-level AcpiHwRegisterWrite automatically handles a possibly split PM1 A/B register. ACPICA BZ 751. Split out the PM1 Status registers from the FADT. Added new globals for these registers (A/B), similar to the way the PM1 Enable registers are handled. Instead of overloading the FADT Event Register blocks. This makes the code clearer and less prone to error. Fixed the warning message for when the platform contains too many ACPI tables for the default size of the global root table data structure. The calculation for the truncation value was incorrect. Removed the ACPI_GET_OBJECT_TYPE macro. Removed all instances of this obsolete macro, since it is now a simple reference to ->common.type. There were about 150 invocations of the macro across 41 files. ACPICA BZ 755. Removed the redundant ACPI_BITREG_SLEEP_TYPE_B. This type is the same as TYPE_A. Removed this and all related instances. Renamed SLEEP_TYPE_A to simply SLEEP_TYPE. ACPICA BZ 754. Conditionally compile the AcpiSetFirmwareWakingVector64 function. This function is only needed on 64-bit host operating systems and is thus not included for 32-bit hosts. Debug output: print the input and result for invocations of the _OSI reserved control method via the ACPI_LV_INFO debug level. Also, reduced some of the verbosity of this debug level. Len Brown. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 82.3K Code, 17.5K Data, 99.8K Total Debug Version: 157.3K Code, 49.8K Data, 207.1K Total Current Release: Non-Debug Version: 82.0K Code, 17.5K Data, 99.5K Total Debug Version: 156.9K Code, 49.8K Data, 206.7K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Decode the FADT PM_Profile field. Emit ascii names for the various legal performance profiles. ---------------------------------------- 23 January 2009. Summary of changes for version 20090123: 1) ACPI CA Core Subsystem: Added the 2009 copyright to all module headers and signons. This affects virtually every file in the ACPICA core subsystem, the iASL compiler, and the tools/utilities. Implemented a change to allow the host to override any ACPI table, including dynamically loaded tables. Previously, only the DSDT could be replaced by the host. With this change, the AcpiOsTableOverride interface is called for each table found in the RSDT/XSDT during ACPICA initialization, and also whenever a table is dynamically loaded via the AML Load operator. Updated FADT flag definitions, especially the Boot Architecture flags. Debugger: For the Find command, automatically pad the input ACPI name with underscores if the name is shorter than 4 characters. This enables a match with the actual namespace entry which is itself padded with underscores. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 82.3K Code, 17.4K Data, 99.7K Total Debug Version: 157.1K Code, 49.7K Data, 206.8K Total Current Release: Non-Debug Version: 82.3K Code, 17.5K Data, 99.8K Total Debug Version: 157.3K Code, 49.8K Data, 207.1K Total 2) iASL Compiler/Disassembler and Tools: Fix build error under Bison-2.4. -Dissasembler: Enhanced FADT support. Added decoding of the Boot -Architecture +Disassembler: Enhanced FADT support. Added decoding of the Boot +Architecture flags. Now decode all flags, regardless of the FADT version. Flag output includes the FADT version which first defined each flag. The iASL -g option now dumps the RSDT to a file (in addition to the FADT and DSDT). Windows only. ---------------------------------------- 04 December 2008. Summary of changes for version 20081204: 1) ACPI CA Core Subsystem: The ACPICA Programmer Reference has been completely updated and revamped for this release. This includes updates to the external interfaces, OSL interfaces, the overview sections, and the debugger reference. Several new ACPICA interfaces have been implemented and documented in the programmer reference: AcpiReset - Writes the reset value to the FADT-defined reset register. AcpiDisableAllGpes - Disable all available GPEs. AcpiEnableAllRuntimeGpes - Enable all available runtime GPEs. AcpiGetGpeDevice - Get the GPE block device associated with a GPE. AcpiGbl_CurrentGpeCount - Tracks the current number of available GPEs. AcpiRead - Low-level read ACPI register (was HwLowLevelRead.) AcpiWrite - Low-level write ACPI register (was HwLowLevelWrite.) Most of the public ACPI hardware-related interfaces have been moved to a new file, components/hardware/hwxface.c Enhanced the FADT parsing and low-level ACPI register access: The ACPI register lengths within the FADT are now used, and the low level ACPI register access no longer hardcodes the ACPI register lengths. Given that there may be some risk in actually trusting the FADT register lengths, a run- time option was added to fall back to the default hardcoded lengths if the FADT proves to contain incorrect values - UseDefaultRegisterWidths. This option is set to true for now, and a warning is issued if a suspicious FADT register length is overridden with the default value. Fixed a reference count issue in NsRepairObject. This problem was introduced in version 20081031 as part of a fix to repair Buffer objects within Packages. Lin Ming. Added semaphore support to the Linux/Unix application OS-services layer (OSL). ACPICA BZ 448. Lin Ming. Added the ACPI_MUTEX_TYPE configuration option to select whether mutexes will be implemented in the OSL, or will binary semaphores be used instead. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 81.7K Code, 17.3K Data, 99.0K Total Debug Version: 156.4K Code, 49.4K Data, 205.8K Total Current Release: Non-Debug Version: 82.3K Code, 17.4K Data, 99.7K Total Debug Version: 157.1K Code, 49.7K Data, 206.8K Total 2) iASL Compiler/Disassembler and Tools: iASL: Completed the '-e' option to include additional ACPI tables in order to aid with disassembly and External statement generation. ACPICA BZ 742. Lin Ming. iASL: Removed the "named object in while loop" error. The compiler cannot determine how many times a loop will execute. ACPICA BZ 730. Disassembler: Implemented support for FADT revision 2 (MS extension). ACPICA BZ 743. Disassembler: Updates for several ACPI data tables (HEST, EINJ, and MCFG). ---------------------------------------- 31 October 2008. Summary of changes for version 20081031: 1) ACPI CA Core Subsystem: Restructured the ACPICA header files into public/private. acpi.h now includes only the "public" acpica headers. All other acpica headers are "private" and should not be included by acpica users. One new file, accommon.h is used to include the commonly used private headers for acpica code generation. Future plans include moving all private headers to a new subdirectory. Implemented an automatic Buffer->String return value conversion for predefined ACPI methods. For these methods (such as _BIF), added automatic conversion for return objects that are required to be a String, but a Buffer was found instead. This can happen when reading string battery data from an operation region, because it used to be difficult to convert the data from buffer to string from within the ASL. Ensures that the host OS is provided with a valid null-terminated string. Linux BZ 11822. Updated the FACS waking vector interfaces. Split AcpiSetFirmwareWakingVector into two: one for the 32-bit vector, another for the 64-bit vector. This is required because the host OS must setup the wake much differently for each vector (real vs. protected mode, etc.) and the interface itself should not be deciding which vector to use. Also, eliminated the GetFirmwareWakingVector interface, as it served no purpose (only the firmware reads the vector, OS only writes the vector.) ACPICA BZ 731. Implemented a mechanism to escape infinite AML While() loops. Added a loop counter to force exit from AML While loops if the count becomes too large. This can occur in poorly written AML when the hardware does not respond within a while loop and the loop does not implement a timeout. The maximum loop count is configurable. A new exception code is returned when a loop is broken, AE_AML_INFINITE_LOOP. Alexey Starikovskiy, Bob Moore. Optimized the execution of AML While loops. Previously, a control state object was allocated and freed for each execution of the loop. The optimization is to simply reuse the control state for each iteration. This speeds up the raw loop execution time by about 5%. Enhanced the implicit return mechanism. For Windows compatibility, return an implicit integer of value zero for methods that contain no executable code. Such methods are seen in the field as stubs (presumably), and can cause drivers to fail if they expect a return value. Lin Ming. Allow multiple backslashes as root prefixes in namepaths. In a fully qualified namepath, allow multiple backslash prefixes. This can happen (and is seen in the field) because of the use of a double-backslash in strings (since backslash is the escape character) causing confusion. ACPICA BZ 739 Lin Ming. Emit a warning if two different FACS or DSDT tables are discovered in the FADT. Checks if there are two valid but different addresses for the FACS and DSDT within the FADT (mismatch between the 32-bit and 64-bit fields.) Consolidated the method argument count validation code. Merged the code that validates control method argument counts into the predefined validation module. Eliminates possible multiple warnings for incorrect argument counts. Implemented ACPICA example code. Includes code for ACPICA initialization, handler installation, and calling a control method. Available at source/tools/examples. Added a global pointer for FACS table to simplify internal FACS access. Use the global pointer instead of using AcpiGetTableByIndex for each FACS access. This simplifies the code for the Global Lock and the Firmware Waking Vector(s). Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 81.2K Code, 17.0K Data, 98.2K Total Debug Version: 155.8K Code, 49.1K Data, 204.9K Total Current Release: Non-Debug Version: 81.7K Code, 17.3K Data, 99.0K Total Debug Version: 156.4K Code, 49.4K Data, 205.8K Total 2) iASL Compiler/Disassembler and Tools: iASL: Improved disassembly of external method calls. Added the -e option to allow the inclusion of additional ACPI tables to help with the disassembly of method invocations and the generation of external declarations during the disassembly. Certain external method invocations cannot be disassembled properly without the actual declaration of the method. Use the -e option to include the table where the external method(s) are actually declared. Most useful for disassembling SSDTs that make method calls back to the master DSDT. Lin Ming. Example: To disassemble an SSDT with calls to DSDT: iasl -d -e dsdt.aml ssdt1.aml iASL: Fix to allow references to aliases within ASL namepaths. Fixes a problem where the use of an alias within a namepath would result in a not found error or cause the compiler to fault. Also now allows forward references from the Alias operator itself. ACPICA BZ 738. ---------------------------------------- 26 September 2008. Summary of changes for version 20080926: 1) ACPI CA Core Subsystem: Designed and implemented a mechanism to validate predefined ACPI methods and objects. This code validates the predefined ACPI objects (objects whose names start with underscore) that appear in the namespace, at the time they are evaluated. The argument count and the type of the returned object are validated against the ACPI specification. The purpose of this validation is to detect problems with the BIOS-implemented predefined ACPI objects before the results are returned to the ACPI-related drivers. Future enhancements may include actual repair of incorrect return objects where possible. Two new files are nspredef.c and acpredef.h. Fixed a fault in the AML parser if a memory allocation fails during the Op completion routine AcpiPsCompleteThisOp. Lin Ming. ACPICA BZ 492. Fixed an issue with implicit return compatibility. This change improves the implicit return mechanism to be more compatible with the MS interpreter. Lin Ming, ACPICA BZ 349. Implemented support for zero-length buffer-to-string conversions. Allow zero length strings during interpreter buffer-to-string conversions. For example, during the ToDecimalString and ToHexString operators, as well as implicit conversions. Fiodor Suietov, ACPICA BZ 585. Fixed two possible memory leaks in the error exit paths of AcpiUtUpdateObjectReference and AcpiUtWalkPackageTree. These functions are similar in that they use a stack of state objects in order to eliminate recursion. The stack must be fully unwound and deallocated if an error occurs. Lin Ming. ACPICA BZ 383. Removed the unused ACPI_BITREG_WAKE_ENABLE definition and entry in the global ACPI register table. This bit does not exist and is unused. Lin Ming, Bob Moore ACPICA BZ 442. Removed the obsolete version number in module headers. Removed the "$Revision" number that appeared in each module header. This version number was useful under SourceSafe and CVS, but has no meaning under git. It is not only incorrect, it could also be misleading. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.7K Code, 16.4K Data, 96.1K Total Debug Version: 153.7K Code, 48.2K Data, 201.9K Total Current Release: Non-Debug Version: 81.2K Code, 17.0K Data, 98.2K Total Debug Version: 155.8K Code, 49.1K Data, 204.9K Total ---------------------------------------- 29 August 2008. Summary of changes for version 20080829: 1) ACPI CA Core Subsystem: Completed a major cleanup of the internal ACPI_OPERAND_OBJECT of type Reference. Changes include the elimination of cheating on the Object field for the DdbHandle subtype, addition of a reference class field to differentiate the various reference types (instead of an AML opcode), and the cleanup of debug output for this object. Lin Ming, Bob Moore. BZ 723 Reduce an error to a warning for an incorrect method argument count. Previously aborted with an error if too few arguments were passed to a control method via the external ACPICA interface. Now issue a warning instead and continue. Handles the case where the method inadvertently declares too many arguments, but does not actually use the extra ones. Applies mainly to the predefined methods. Lin Ming. Linux BZ 11032. Disallow the evaluation of named object types with no intrinsic value. Return AE_TYPE for objects that have no value and therefore evaluation is undefined: Device, Event, Mutex, Region, Thermal, and Scope. Previously, evaluation of these types were allowed, but an exception would be generated at some point during the evaluation. Now, the error is generated up front. Fixed a possible memory leak in the AcpiNsGetExternalPathname function (nsnames.c). Fixes a leak in the error exit path. Removed the obsolete debug levels ACPI_DB_WARN and ACPI_DB_ERROR. These debug levels were made obsolete by the ACPI_WARNING, ACPI_ERROR, and ACPI_EXCEPTION interfaces. Also added ACPI_DB_EVENTS to correspond with the existing ACPI_LV_EVENTS. Removed obsolete and/or unused exception codes from the acexcep.h header. There is the possibility that certain device drivers may be affected if they use any of these exceptions. The ACPICA documentation has been added to the public git source tree, under acpica/documents. Included are the ACPICA programmer reference, the iASL compiler reference, and the changes.txt release logfile. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.7K Code, 16.4K Data, 96.1K Total Debug Version: 153.9K Code, 48.4K Data, 202.3K Total Current Release: Non-Debug Version: 79.7K Code, 16.4K Data, 96.1K Total Debug Version: 153.7K Code, 48.2K Data, 201.9K Total 2) iASL Compiler/Disassembler and Tools: Allow multiple argument counts for the predefined _SCP method. ACPI 3.0 defines _SCP with 3 arguments. Previous versions defined it with only 1 argument. iASL now allows both definitions. iASL/disassembler: avoid infinite loop on bad ACPI tables. Check for zero- length subtables when disassembling ACPI tables. Also fixed a couple of errors where a full 16-bit table type field was not extracted from the input properly. acpisrc: Improve comment counting mechanism for generating source code statistics. Count first and last lines of multi-line comments as whitespace, not comment lines. Handle Linux legal header in addition to standard acpica header. ---------------------------------------- 29 July 2008. Summary of changes for version 20080729: 1) ACPI CA Core Subsystem: Fix a possible deadlock in the GPE dispatch. Remove call to AcpiHwDisableAllGpes during wake in AcpiEvGpeDispatch. This call will attempt to acquire the GPE lock but can deadlock since the GPE lock is already held at dispatch time. This code was introduced in version 20060831 as a response to Linux BZ 6881 and has since been removed from Linux. Add a function to dereference returned reference objects. Examines the return object from a call to AcpiEvaluateObject. Any Index or RefOf references are automatically dereferenced in an attempt to return something useful (these reference types cannot be converted into an external ACPI_OBJECT.) Provides MS compatibility. Lin Ming, Bob Moore. Linux BZ 11105 x2APIC support: changes for MADT and SRAT ACPI tables. There are 2 new subtables for the MADT and one new subtable for the SRAT. Includes disassembler and AcpiSrc support. Data from the Intel 64 Architecture x2APIC Specification, June 2008. Additional error checking for pathname utilities. Add error check after all calls to AcpiNsGetPathnameLength. Add status return from AcpiNsBuildExternalPath and check after all calls. Add parameter validation to AcpiUtInitializeBuffer. Reported by and initial patch by Ingo Molnar. Return status from the global init function AcpiUtGlobalInitialize. This is used by both the kernel subsystem and the utilities such as iASL compiler. The function could possibly fail when the caches are initialized. Yang Yi. Add a function to decode reference object types to strings. Created for improved error messages. Improve object conversion error messages. Better error messages during object conversion from internal to the external ACPI_OBJECT. Used for external calls to AcpiEvaluateObject. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.6K Code, 16.2K Data, 95.8K Total Debug Version: 153.5K Code, 48.2K Data, 201.7K Total Current Release: Non-Debug Version: 79.7K Code, 16.4K Data, 96.1K Total Debug Version: 153.9K Code, 48.4K Data, 202.3K Total 2) iASL Compiler/Disassembler and Tools: Debugger: fix a possible hang when evaluating non-methods. Fixes a problem introduced in version 20080701. If the object being evaluated (via execute command) is not a method, the debugger can hang while trying to obtain non- existent parameters. iASL: relax error for using reserved "_T_x" identifiers. These names can appear in a disassembled ASL file if they were emitted by the original compiler. Instead of issuing an error or warning and forcing the user to manually change these names, issue a remark instead. iASL: error if named object created in while loop. Emit an error if any named object is created within a While loop. If allowed, this code will generate a run-time error on the second iteration of the loop when an attempt is made to create the same named object twice. ACPICA bugzilla 730. iASL: Support absolute pathnames for include files. Add support for absolute pathnames within the Include operator. previously, only relative pathnames were supported. iASL: Enforce minimum 1 interrupt in interrupt macro and Resource Descriptor. The ACPI spec requires one interrupt minimum. BZ 423 iASL: Handle a missing ResourceSource arg, with a present SourceIndex. Handles the case for the Interrupt Resource Descriptor where the ResourceSource argument is omitted but ResourceSourceIndex is present. Now leave room for the Index. BZ 426 iASL: Prevent error message if CondRefOf target does not exist. Fixes cases where an error message is emitted if the target does not exist. BZ 516 iASL: Fix broken -g option (get Windows ACPI tables). Fixes the -g option (get ACPI tables on Windows). This was apparently broken in version 20070919. AcpiXtract: Handle EOF while extracting data. Correctly handle the case where the EOF happens immediately after the last table in the input file. Print completion message. Previously, no message was displayed in this case. ---------------------------------------- 01 July 2008. Summary of changes for version 20080701: 0) Git source tree / acpica.org Fixed a problem where a git-clone from http would not transfer the entire source tree. 1) ACPI CA Core Subsystem: Implemented a "careful" GPE disable in AcpiEvDisableGpe, only modify one enable bit. Now performs a read-change-write of the enable register instead of simply writing out the cached enable mask. This will prevent inadvertent enabling of GPEs if a rogue GPE is received during initialization (before GPE handlers are installed.) Implemented a copy for dynamically loaded tables. Previously, dynamically loaded tables were simply mapped - but on some machines this memory is corrupted after suspend. Now copy the table to a local buffer. For the OpRegion case, added checksum verify. Use the table length from the table header, not the region length. For the Buffer case, use the table length also. Dennis Noordsij, Bob Moore. BZ 10734 Fixed a problem where the same ACPI table could not be dynamically loaded and unloaded more than once. Without this change, a table cannot be loaded again once it has been loaded/unloaded one time. The current mechanism does not unregister a table upon an unload. During a load, if the same table is found, this no longer returns an exception. BZ 722 Fixed a problem where the wrong descriptor length was calculated for the EndTag descriptor in 64-bit mode. The "minimal" descriptors such as EndTag are calculated as 12 bytes long, but the actual length in the internal descriptor is 16 because of the round-up to 8 on the 64-bit build. Reported by Linn Crosetto. BZ 728 Fixed a possible memory leak in the Unload operator. The DdbHandle returned by Load() did not have its reference count decremented during unload, leading to a memory leak. Lin Ming. BZ 727 Fixed a possible memory leak when deleting thermal/processor objects. Any associated notify handlers (and objects) were not being deleted. Fiodor Suietov. BZ 506 Fixed the ordering of the ASCII names in the global mutex table to match the actual mutex IDs. Used by AcpiUtGetMutexName, a function used for debug only. Vegard Nossum. BZ 726 Enhanced the AcpiGetObjectInfo interface to return the number of required arguments if the object is a control method. Added this call to the debugger so the proper number of default arguments are passed to a method. This prevents a warning when executing methods from AcpiExec. Added a check for an invalid handle in AcpiGetObjectInfo. Return AE_BAD_PARAMETER if input handle is invalid. BZ 474 Fixed an extraneous warning from exconfig.c on the 64-bit build. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.3K Code, 16.2K Data, 95.5K Total Debug Version: 153.0K Code, 48.2K Data, 201.2K Total Current Release: Non-Debug Version: 79.6K Code, 16.2K Data, 95.8K Total Debug Version: 153.5K Code, 48.2K Data, 201.7K Total 2) iASL Compiler/Disassembler and Tools: iASL: Added two missing ACPI reserved names. Added _MTP and _ASZ, both resource descriptor names. iASL: Detect invalid ASCII characters in input (windows version). Removed the "-CF" flag from the flex compile, enables correct detection of non-ASCII characters in the input. BZ 441 iASL: Eliminate warning when result of LoadTable is not used. Eliminate the "result of operation not used" warning when the DDB handle returned from LoadTable is not used. The warning is not needed. BZ 590 AcpiExec: Add support for dynamic table load/unload. Now calls _CFG method to pass address of table to the AML. Added option to disable OpRegion simulation to allow creation of an OpRegion with a real address that was passed to _CFG. All of this allows testing of the Load and Unload operators from AcpiExec. Debugger: update tables command for unloaded tables. Handle unloaded tables and use the standard table header output routine. ---------------------------------------- 09 June 2008. Summary of changes for version 20080609: 1) ACPI CA Core Subsystem: Implemented a workaround for reversed _PRT entries. A significant number of BIOSs erroneously reverse the _PRT SourceName and the SourceIndex. This change dynamically detects and repairs this problem. Provides compatibility with MS ACPI. BZ 6859 Simplified the internal ACPI hardware interfaces to eliminate the locking flag parameter from Register Read/Write. Added a new external interface, AcpiGetRegisterUnlocked. Fixed a problem where the invocation of a GPE control method could hang. This was a regression introduced in 20080514. The new method argument count validation mechanism can enter an infinite loop when a GPE method is dispatched. Problem fixed by removing the obsolete code that passed GPE block information to the notify handler via the control method parameter pointer. Fixed a problem where the _SST execution status was incorrectly returned to the caller of AcpiEnterSleepStatePrep. This was a regression introduced in 20080514. _SST is optional and a NOT_FOUND exception should never be returned. BZ 716 Fixed a problem where a deleted object could be accessed from within the AML parser. This was a regression introduced in version 20080123 as a fix for the Unload operator. Lin Ming. BZ 10669 Cleaned up the debug operand dump mechanism. Eliminated unnecessary operands and eliminated the use of a negative index in a loop. Operands are now displayed in the correct order, not backwards. This also fixes a regression introduced in 20080514 on 64-bit systems where the elimination of ACPI_NATIVE_UINT caused the negative index to go large and positive. BZ 715 Fixed a possible memory leak in EvPciConfigRegionSetup where the error exit path did not delete a locally allocated structure. Updated definitions for the DMAR and SRAT tables to synchronize with the current specifications. Includes disassembler support. Fixed a problem in the mutex debug code (in utmutex.c) where an incorrect loop termination value was used. Loop terminated on iteration early, missing one mutex. Linn Crosetto Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.5K Code, 16.2K Data, 95.7K Total Debug Version: 153.3K Code, 48.3K Data, 201.6K Total Current Release: Non-Debug Version: 79.3K Code, 16.2K Data, 95.5K Total Debug Version: 153.0K Code, 48.2K Data, 201.2K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Implemented support for EisaId() within _CID objects. Now disassemble integer _CID objects back to EisaId invocations, including multiple integers within _CID packages. Includes single-step support for debugger also. Disassembler: Added support for DMAR and SRAT table definition changes. ---------------------------------------- 14 May 2008. Summary of changes for version 20080514: 1) ACPI CA Core Subsystem: Fixed a problem where GPEs were enabled too early during the ACPICA initialization. This could lead to "handler not installed" errors on some machines. Moved GPE enable until after _REG/_STA/_INI methods are run. This ensures that all operation regions and devices throughout the namespace have been initialized before GPEs are enabled. Alexey Starikovskiy, BZ 9916. Implemented a change to the enter sleep code. Moved execution of the _GTS method to just before setting sleep enable bit. The execution was moved from AcpiEnterSleepStatePrep to AcpiEnterSleepState. _GTS is now executed immediately before the SLP_EN bit is set, as per the ACPI specification. Luming Yu, BZ 1653. Implemented a fix to disable unknown GPEs (2nd version). Now always disable the GPE, even if ACPICA thinks that that it is already disabled. It is possible that the AML or some other code has enabled the GPE unbeknownst to the ACPICA code. Fixed a problem with the Field operator where zero-length fields would return an AE_AML_NO_OPERAND exception during table load. Fix enables zero-length ASL field declarations in Field(), BankField(), and IndexField(). BZ 10606. Implemented a fix for the Load operator, now load the table at the namespace root. This reverts a change introduced in version 20071019. The table is now loaded at the namespace root even though this goes against the ACPI specification. This provides compatibility with other ACPI implementations. The ACPI specification will be updated to reflect this in ACPI 4.0. Lin Ming. Fixed a problem where ACPICA would not Load() tables with unusual signatures. Now ignore ACPI table signature for Load() operator. Only "SSDT" is acceptable to the ACPI spec, but tables are seen with OEMx and null sigs. Therefore, signature validation is worthless. Apparently MS ACPI accepts such signatures, ACPICA must be compatible. BZ 10454. Fixed a possible negative array index in AcpiUtValidateException. Added NULL fields to the exception string arrays to eliminate a -1 subtraction on the SubStatus field. Updated the debug tracking macros to reduce overall code and data size. Changed ACPI_MODULE_NAME and ACPI_FUNCTION_NAME to use arrays of strings instead of pointers to static strings. Jan Beulich and Bob Moore. Implemented argument count checking in control method invocation via AcpiEvaluateObject. Now emit an error if too few arguments, warning if too many. This applies only to extern programmatic control method execution, not method-to-method calls within the AML. Lin Ming. Eliminated the ACPI_NATIVE_UINT type across all ACPICA code. This type is no longer needed, especially with the removal of 16-bit support. It was replaced mostly with UINT32, but also ACPI_SIZE where a type that changes 32/64 bit on 32/64-bit platforms is required. Added the C const qualifier for appropriate string constants -- mostly MODULE_NAME and printf format strings. Jan Beulich. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 80.0K Code, 17.4K Data, 97.4K Total Debug Version: 159.4K Code, 64.4K Data, 223.8K Total Current Release: Non-Debug Version: 79.5K Code, 16.2K Data, 95.7K Total Debug Version: 153.3K Code, 48.3K Data, 201.6K Total 2) iASL Compiler/Disassembler and Tools: Implemented ACPI table revision ID validation in the disassembler. Zero is always invalid. For DSDTs, the ID controls the interpreter integer width. 1 means 32-bit and this is unusual. 2 or greater is 64-bit. ---------------------------------------- 21 March 2008. Summary of changes for version 20080321: 1) ACPI CA Core Subsystem: Implemented an additional change to the GPE support in order to suppress spurious or stray GPEs. The AcpiEvDisableGpe function will now permanently disable incoming GPEs that are neither enabled nor disabled -- meaning that the GPE is unknown to the system. This should prevent future interrupt floods from that GPE. BZ 6217 (Zhang Rui) Fixed a problem where NULL package elements were not returned to the AcpiEvaluateObject interface correctly. The element was simply ignored instead of returning a NULL ACPI_OBJECT package element, potentially causing a buffer overflow and/or confusing the caller who expected a fixed number of elements. BZ 10132 (Lin Ming, Bob Moore) Fixed a problem with the CreateField, CreateXXXField (Bit, Byte, Word, Dword, Qword), Field, BankField, and IndexField operators when invoked from inside an executing control method. In this case, these operators created namespace nodes that were incorrectly left marked as permanent nodes instead of temporary nodes. This could cause a problem if there is race condition between an exiting control method and a running namespace walk. (Reported by Linn Crosetto) Fixed a problem where the CreateField and CreateXXXField operators would incorrectly allow duplicate names (the name of the field) with no exception generated. Implemented several changes for Notify handling. Added support for new Notify values (ACPI 2.0+) and improved the Notify debug output. Notify on PowerResource objects is no longer allowed, as per the ACPI specification. (Bob Moore, Zhang Rui) All Reference Objects returned via the AcpiEvaluateObject interface are now marked as type "REFERENCE" instead of "ANY". The type ANY is now reserved for NULL objects - either NULL package elements or unresolved named references. Fixed a problem where an extraneous debug message was produced for package objects (when debugging enabled). The message "Package List length larger than NumElements count" is now produced in the correct case, and is now an error message rather than a debug message. Added a debug message for the opposite case, where NumElements is larger than the Package List (the package will be padded out with NULL elements as per the ACPI spec.) Implemented several improvements for the output of the ASL "Debug" object to clarify and keep all data for a given object on one output line. Fixed two size calculation issues with the variable-length Start Dependent resource descriptor. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.7K Code, 17.3K Data, 97.0K Total Debug Version: 158.9K Code, 64.0K Data, 222.9K Total Current Release: Non-Debug Version: 80.0K Code, 17.4K Data, 97.4K Total Debug Version: 159.4K Code, 64.4K Data, 223.8K Total 2) iASL Compiler/Disassembler and Tools: Fixed a problem with the use of the Switch operator where execution of the containing method by multiple concurrent threads could cause an AE_ALREADY_EXISTS exception. This is caused by the fact that there is no actual Switch opcode, it must be simulated with local named temporary variables and if/else pairs. The solution chosen was to mark any method that uses Switch as Serialized, thus preventing multiple thread entries. BZ 469. ---------------------------------------- 13 February 2008. Summary of changes for version 20080213: 1) ACPI CA Core Subsystem: Implemented another MS compatibility design change for GPE/Notify handling. GPEs are now cleared/enabled asynchronously to allow all pending notifies to complete first. It is expected that the OSL will queue the enable request behind all pending notify requests (may require changes to the local host OSL in AcpiOsExecute). Alexey Starikovskiy. Fixed a problem where buffer and package objects passed as arguments to a control method via the external AcpiEvaluateObject interface could cause an AE_AML_INTERNAL exception depending on the order and type of operators executed by the target control method. Fixed a problem where resource descriptor size optimization could cause a problem when a _CRS resource template is passed to a _SRS method. The _SRS resource template must use the same descriptors (with the same size) as returned from _CRS. This change affects the following resource descriptors: IRQ / IRQNoFlags and StartDependendentFn / StartDependentFnNoPri. (BZ 9487) Fixed a problem where a CopyObject to RegionField, BankField, and IndexField objects did not perform an implicit conversion as it should. These types must retain their initial type permanently as per the ACPI specification. However, a CopyObject to all other object types should not perform an implicit conversion, as per the ACPI specification. (Lin Ming, Bob Moore) BZ 388 Fixed a problem with the AcpiGetDevices interface where the mechanism to match device CIDs did not examine the entire list of available CIDs, but instead aborted on the first non-matching CID. Andrew Patterson. Fixed a regression introduced in version 20071114. The ACPI_HIDWORD macro was inadvertently changed to return a 16-bit value instead of a 32-bit value, truncating the upper dword of a 64-bit value. This macro is only used to display debug output, so no incorrect calculations were made. Also, reimplemented the macro so that a 64-bit shift is not performed by inefficient compilers. Added missing va_end statements that should correspond with each va_start statement. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.5K Code, 17.2K Data, 96.7K Total Debug Version: 159.0K Code, 63.8K Data, 222.8K Total Current Release: Non-Debug Version: 79.7K Code, 17.3K Data, 97.0K Total Debug Version: 158.9K Code, 64.0K Data, 222.9K Total 2) iASL Compiler/Disassembler and Tools: Implemented full disassembler support for the following new ACPI tables: BERT, EINJ, and ERST. Implemented partial disassembler support for the complicated HEST table. These tables support the Windows Hardware Error Architecture (WHEA). ---------------------------------------- 23 January 2008. Summary of changes for version 20080123: 1) ACPI CA Core Subsystem: Added the 2008 copyright to all module headers and signons. This affects virtually every file in the ACPICA core subsystem, the iASL compiler, and the tools/utilities. Fixed a problem with the SizeOf operator when used with Package and Buffer objects. These objects have deferred execution for some arguments, and the execution is now completed before the SizeOf is executed. This problem caused unexpected AE_PACKAGE_LIMIT errors on some systems (Lin Ming, Bob Moore) BZ 9558 Implemented an enhancement to the interpreter "slack mode". In the absence of an explicit return or an implicitly returned object from the last executed opcode, a control method will now implicitly return an integer of value 0 for Microsoft compatibility. (Lin Ming) BZ 392 Fixed a problem with the Load operator where an exception was not returned in the case where the table is already loaded. (Lin Ming) BZ 463 Implemented support for the use of DDBHandles as an Indexed Reference, as per the ACPI spec. (Lin Ming) BZ 486 Implemented support for UserTerm (Method invocation) for the Unload operator as per the ACPI spec. (Lin Ming) BZ 580 Fixed a problem with the LoadTable operator where the OemId and OemTableId input strings could cause unexpected failures if they were shorter than the maximum lengths allowed. (Lin Ming, Bob Moore) BZ 576 Implemented support for UserTerm (Method invocation) for the Unload operator as per the ACPI spec. (Lin Ming) BZ 580 Implemented header file support for new ACPI tables - BERT, ERST, EINJ, HEST, IBFT, UEFI, WDAT. Disassembler support is forthcoming. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.3K Code, 17.2K Data, 96.5K Total Debug Version: 158.6K Code, 63.8K Data, 222.4K Total Current Release: Non-Debug Version: 79.5K Code, 17.2K Data, 96.7K Total Debug Version: 159.0K Code, 63.8K Data, 222.8K Total 2) iASL Compiler/Disassembler and Tools: Implemented support in the disassembler for checksum validation on incoming binary DSDTs and SSDTs. If incorrect, a message is displayed within the table header dump at the start of the disassembly. Implemented additional debugging information in the namespace listing file created during compilation. In addition to the namespace hierarchy, the full pathname to each namespace object is displayed. Fixed a problem with the disassembler where invalid ACPI tables could cause faults or infinite loops. Fixed an unexpected parse error when using the optional "parameter types" list in a control method declaration. (Lin Ming) BZ 397 Fixed a problem where two External declarations with the same name did not cause an error (Lin Ming) BZ 509 Implemented support for full TermArgs (adding Argx, Localx and method invocation) for the ParameterData parameter to the LoadTable operator. (Lin Ming) BZ 583,587 ---------------------------------------- 19 December 2007. Summary of changes for version 20071219: 1) ACPI CA Core Subsystem: Implemented full support for deferred execution for the TermArg string arguments for DataTableRegion. This enables forward references and full operand resolution for the three string arguments. Similar to OperationRegion deferred argument execution.) Lin Ming. BZ 430 Implemented full argument resolution support for the BankValue argument to BankField. Previously, only constants were supported, now any TermArg may be used. Lin Ming BZ 387, 393 Fixed a problem with AcpiGetDevices where the search of a branch of the device tree could be terminated prematurely. In accordance with the ACPI specification, the search down the current branch is terminated if a device is both not present and not functional (instead of just not present.) Yakui Zhao. Fixed a problem where "unknown" GPEs could be allowed to fire repeatedly if the underlying AML code changed the GPE enable registers. Now, any unknown incoming GPE (no _Lxx/_Exx method and not the EC GPE) is immediately disabled instead of simply ignored. Rui Zhang. Fixed a problem with Index Fields where the Index register was incorrectly limited to a maximum of 32 bits. Now any size may be used. Fixed a couple memory leaks associated with "implicit return" objects when the AML Interpreter slack mode is enabled. Lin Ming BZ 349 Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.0K Code, 17.2K Data, 96.2K Total Debug Version: 157.9K Code, 63.6K Data, 221.5K Total Current Release: Non-Debug Version: 79.3K Code, 17.2K Data, 96.5K Total Debug Version: 158.6K Code, 63.8K Data, 222.4K Total ---------------------------------------- 14 November 2007. Summary of changes for version 20071114: 1) ACPI CA Core Subsystem: Implemented event counters for each of the Fixed Events, the ACPI SCI (interrupt) itself, and control methods executed. Named AcpiFixedEventCount[], AcpiSciCount, and AcpiMethodCount respectively. These should be useful for debugging and statistics. Implemented a new external interface, AcpiGetStatistics, to retrieve the contents of the various event counters. Returns the current values for AcpiSciCount, AcpiGpeCount, the AcpiFixedEventCount array, and AcpiMethodCount. The interface can be expanded in the future if new counters are added. Device drivers should use this interface rather than access the counters directly. Fixed a problem with the FromBCD and ToBCD operators. With some compilers, the ShortDivide function worked incorrectly, causing problems with the BCD functions with large input values. A truncation from 64-bit to 32-bit inadvertently occurred. Internal BZ 435. Lin Ming Fixed a problem with Index references passed as method arguments. References passed as arguments to control methods were dereferenced immediately (before control was passed to the called method). The references are now correctly passed directly to the called method. BZ 5389. Lin Ming Fixed a problem with CopyObject used in conjunction with the Index operator. The reference was incorrectly dereferenced before the copy. The reference is now correctly copied. BZ 5391. Lin Ming Fixed a problem with Control Method references within Package objects. These references are now correctly generated. This completes the package construction overhaul that began in version 20071019. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 78.8K Code, 17.2K Data, 96.0K Total Debug Version: 157.2K Code, 63.4K Data, 220.6K Total Current Release: Non-Debug Version: 79.0K Code, 17.2K Data, 96.2K Total Debug Version: 157.9K Code, 63.6K Data, 221.5K Total 2) iASL Compiler/Disassembler and Tools: The AcpiExec utility now installs handlers for all of the predefined Operation Region types. New types supported are: PCI_Config, CMOS, and PCIBARTarget. Fixed a problem with the 64-bit version of AcpiExec where the extended (64- bit) address fields for the DSDT and FACS within the FADT were not being used, causing truncation of the upper 32-bits of these addresses. Lin Ming and Bob Moore ---------------------------------------- 19 October 2007. Summary of changes for version 20071019: 1) ACPI CA Core Subsystem: Fixed a problem with the Alias operator when the target of the alias is a named ASL operator that opens a new scope -- Scope, Device, PowerResource, Processor, and ThermalZone. In these cases, any children of the original operator could not be accessed via the alias, potentially causing unexpected AE_NOT_FOUND exceptions. (BZ 9067) Fixed a problem with the Package operator where all named references were created as object references and left otherwise unresolved. According to the ACPI specification, a Package can only contain Data Objects or references to control methods. The implication is that named references to Data Objects (Integer, Buffer, String, Package, BufferField, Field) should be resolved immediately upon package creation. This is the approach taken with this change. References to all other named objects (Methods, Devices, Scopes, etc.) are all now properly created as reference objects. (BZ 5328) Reverted a change to Notify handling that was introduced in version 20070508. This version changed the Notify handling from asynchronous to fully synchronous (Device driver Notify handling with respect to the Notify ASL operator). It was found that this change caused more problems than it solved and was removed by most users. Fixed a problem with the Increment and Decrement operators where the type of the target object could be unexpectedly and incorrectly changed. (BZ 353) Lin Ming. Fixed a problem with the Load and LoadTable operators where the table location within the namespace was ignored. Instead, the table was always loaded into the root or current scope. Lin Ming. Fixed a problem with the Load operator when loading a table from a buffer object. The input buffer was prematurely zeroed and/or deleted. (BZ 577) Fixed a problem with the Debug object where a store of a DdbHandle reference object to the Debug object could cause a fault. Added a table checksum verification for the Load operator, in the case where the load is from a buffer. (BZ 578). Implemented additional parameter validation for the LoadTable operator. The length of the input strings SignatureString, OemIdString, and OemTableId are now checked for maximum lengths. (BZ 582) Lin Ming. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 78.5K Code, 17.1K Data, 95.6K Total Debug Version: 156.7K Code, 63.2K Data, 219.9K Total Current Release: Non-Debug Version: 78.8K Code, 17.2K Data, 96.0K Total Debug Version: 157.2K Code, 63.4K Data, 220.6K Total 2) iASL Compiler/Disassembler: Fixed a problem where if a single file was specified and the file did not exist, no error message was emitted. (Introduced with wildcard support in version 20070917.) ---------------------------------------- 19 September 2007. Summary of changes for version 20070919: 1) ACPI CA Core Subsystem: Designed and implemented new external interfaces to install and remove handlers for ACPI table-related events. Current events that are defined are LOAD and UNLOAD. These interfaces allow the host to track ACPI tables as they are dynamically loaded and unloaded. See AcpiInstallTableHandler and AcpiRemoveTableHandler. (Lin Ming and Bob Moore) Fixed a problem where the use of the AcpiGbl_AllMethodsSerialized flag (acpi_serialized option on Linux) could cause some systems to hang during initialization. (Bob Moore) BZ 8171 Fixed a problem where objects of certain types (Device, ThermalZone, Processor, PowerResource) can be not found if they are declared and referenced from within the same control method (Lin Ming) BZ 341 Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 78.3K Code, 17.0K Data, 95.3K Total Debug Version: 156.3K Code, 63.1K Data, 219.4K Total Current Release: Non-Debug Version: 78.5K Code, 17.1K Data, 95.6K Total Debug Version: 156.7K Code, 63.2K Data, 219.9K Total 2) iASL Compiler/Disassembler: Implemented support to allow multiple files to be compiled/disassembled in a single invocation. This includes command line wildcard support for both the Windows and Unix versions of the compiler. This feature simplifies the disassembly and compilation of multiple ACPI tables in a single directory. ---------------------------------------- 08 May 2007. Summary of changes for version 20070508: 1) ACPI CA Core Subsystem: Implemented a Microsoft compatibility design change for the handling of the Notify AML operator. Previously, notify handlers were dispatched and executed completely asynchronously in a deferred thread. The new design still executes the notify handlers in a different thread, but the original thread that executed the Notify() now waits at a synchronization point for the notify handler to complete. Some machines depend on a synchronous Notify operator in order to operate correctly. Implemented support to allow Package objects to be passed as method arguments to the external AcpiEvaluateObject interface. Previously, this would return the AE_NOT_IMPLEMENTED exception. This feature had not been implemented since there were no reserved control methods that required it until recently. Fixed a problem with the internal FADT conversion where ACPI 1.0 FADTs that contained invalid non-zero values in reserved fields could cause later failures because these fields have meaning in later revisions of the FADT. For incoming ACPI 1.0 FADTs, these fields are now always zeroed. (The fields are: Preferred_PM_Profile, PSTATE_CNT, CST_CNT, and IAPC_BOOT_FLAGS.) Fixed a problem where the Global Lock handle was not properly updated if a thread that acquired the Global Lock via executing AML code then attempted to acquire the lock via the AcpiAcquireGlobalLock interface. Reported by Joe Liu. Fixed a problem in AcpiEvDeleteGpeXrupt where the global interrupt list could be corrupted if the interrupt being removed was at the head of the list. Reported by Linn Crosetto. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 78.0K Code, 17.1K Data, 95.1K Total Debug Version: 155.9K Code, 63.1K Data, 219.0K Total Current Release: Non-Debug Version: 78.3K Code, 17.0K Data, 95.3K Total Debug Version: 156.3K Code, 63.1K Data, 219.4K Total ---------------------------------------- 20 March 2007. Summary of changes for version 20070320: 1) ACPI CA Core Subsystem: Implemented a change to the order of interpretation and evaluation of AML operand objects within the AML interpreter. The interpreter now evaluates operands in the order that they appear in the AML stream (and the corresponding ASL code), instead of in the reverse order (after the entire operand list has been parsed). The previous behavior caused several subtle incompatibilities with the Microsoft AML interpreter as well as being somewhat non-intuitive. BZ 7871, local BZ 263. Valery Podrezov. Implemented a change to the ACPI Global Lock support. All interfaces to the global lock now allow the same thread to acquire the lock multiple times. This affects the AcpiAcquireGlobalLock external interface to the global lock as well as the internal use of the global lock to support AML fields -- a control method that is holding the global lock can now simultaneously access AML fields that require global lock protection. Previously, in both cases, this would have resulted in an AE_ALREADY_ACQUIRED exception. The change to AcpiAcquireGlobalLock is of special interest to drivers for the Embedded Controller. There is no change to the behavior of the AML Acquire operator, as this can already be used to acquire a mutex multiple times by the same thread. BZ 8066. With assistance from Alexey Starikovskiy. Fixed a problem where invalid objects could be referenced in the AML Interpreter after error conditions. During operand evaluation, ensure that the internal "Return Object" field is cleared on error and only valid pointers are stored there. Caused occasional access to deleted objects that resulted in "large reference count" warning messages. Valery Podrezov. Fixed a problem where an AE_STACK_OVERFLOW internal exception could occur on deeply nested control method invocations. BZ 7873, local BZ 487. Valery Podrezov. Fixed an internal problem with the handling of result objects on the interpreter result stack. BZ 7872. Valery Podrezov. Removed obsolete code that handled the case where AML_NAME_OP is the target of a reference (Reference.Opcode). This code was no longer necessary. BZ 7874. Valery Podrezov. Removed obsolete ACPI_NO_INTEGER64_SUPPORT from two header files. This was a remnant from the previously discontinued 16-bit support. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 78.0K Code, 17.1K Data, 95.1K Total Debug Version: 155.8K Code, 63.3K Data, 219.1K Total Current Release: Non-Debug Version: 78.0K Code, 17.1K Data, 95.1K Total Debug Version: 155.9K Code, 63.1K Data, 219.0K Total ---------------------------------------- 26 January 2007. Summary of changes for version 20070126: 1) ACPI CA Core Subsystem: Added the 2007 copyright to all module headers and signons. This affects virtually every file in the ACPICA core subsystem, the iASL compiler, and the utilities. Implemented a fix for an incorrect parameter passed to AcpiTbDeleteTable during a table load. A bad pointer was passed in the case where the DSDT is overridden, causing a fault in this case. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 78.0K Code, 17.1K Data, 95.1K Total Debug Version: 155.8K Code, 63.3K Data, 219.1K Total Current Release: Non-Debug Version: 78.0K Code, 17.1K Data, 95.1K Total Debug Version: 155.8K Code, 63.3K Data, 219.1K Total ---------------------------------------- 15 December 2006. Summary of changes for version 20061215: 1) ACPI CA Core Subsystem: Support for 16-bit ACPICA has been completely removed since it is no longer necessary and it clutters the code. All 16-bit macros, types, and conditional compiles have been removed, cleaning up and simplifying the code across the entire subsystem. DOS support is no longer needed since the bootable Linux firmware kit is now available. The handler for the Global Lock is now removed during AcpiTerminate to enable a clean subsystem restart, via the implementation of the AcpiEvRemoveGlobalLockHandler function. (With assistance from Joel Bretz, HP) Implemented enhancements to the multithreading support within the debugger to enable improved multithreading debugging and evaluation of the subsystem. (Valery Podrezov) Debugger: Enhanced the Statistics/Memory command to emit the total (maximum) memory used during the execution, as well as the maximum memory consumed by each of the various object types. (Valery Podrezov) Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 77.9K Code, 17.0K Data, 94.9K Total Debug Version: 155.2K Code, 63.1K Data, 218.3K Total Current Release: Non-Debug Version: 78.0K Code, 17.1K Data, 95.1K Total Debug Version: 155.8K Code, 63.3K Data, 219.1K Total 2) iASL Compiler/Disassembler and Tools: AcpiExec: Implemented a new option (-m) to display full memory use statistics upon subsystem/program termination. (Valery Podrezov) ---------------------------------------- 09 November 2006. Summary of changes for version 20061109: 1) ACPI CA Core Subsystem: Optimized the Load ASL operator in the case where the source operand is an operation region. Simply map the operation region memory, instead of performing a bytewise read. (Region must be of type SystemMemory, see below.) Fixed the Load ASL operator for the case where the source operand is a region field. A buffer object is also allowed as the source operand. BZ 480 Fixed a problem where the Load ASL operator allowed the source operand to be an operation region of any type. It is now restricted to regions of type SystemMemory, as per the ACPI specification. BZ 481 Additional cleanup and optimizations for the new Table Manager code. AcpiEnable will now fail if all of the required ACPI tables are not loaded (FADT, FACS, DSDT). BZ 477 Added #pragma pack(8/4) to acobject.h to ensure that the structures in this header are always compiled as aligned. The ACPI_OPERAND_OBJECT has been manually optimized to be aligned and will not work if it is byte-packed. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 78.1K Code, 17.1K Data, 95.2K Total Debug Version: 155.4K Code, 63.1K Data, 218.5K Total Current Release: Non-Debug Version: 77.9K Code, 17.0K Data, 94.9K Total Debug Version: 155.2K Code, 63.1K Data, 218.3K Total 2) iASL Compiler/Disassembler and Tools: Fixed a problem where the presence of the _OSI predefined control method within complex expressions could cause an internal compiler error. AcpiExec: Implemented full region support for multiple address spaces. SpaceId is now part of the REGION object. BZ 429 ---------------------------------------- 11 October 2006. Summary of changes for version 20061011: 1) ACPI CA Core Subsystem: Completed an AML interpreter performance enhancement for control method execution. Previously a 2-pass parse/execution, control methods are now completely parsed and executed in a single pass. This improves overall interpreter performance by ~25%, reduces code size, and reduces CPU stack use. (Valery Podrezov + interpreter changes in version 20051202 that eliminated namespace loading during the pass one parse.) Implemented _CID support for PCI Root Bridge detection. If the _HID does not match the predefined PCI Root Bridge IDs, the _CID list (if present) is now obtained and also checked for an ID match. Implemented additional support for the PCI _ADR execution: upsearch until a device scope is found before executing _ADR. This allows PCI_Config operation regions to be declared locally within control methods underneath PCI device objects. Fixed a problem with a possible race condition between threads executing AcpiWalkNamespace and the AML interpreter. This condition was removed by modifying AcpiWalkNamespace to (by default) ignore all temporary namespace entries created during any concurrent control method execution. An additional namespace race condition is known to exist between AcpiWalkNamespace and the Load/Unload ASL operators and is still under investigation. Restructured the AML ParseLoop function, breaking it into several subfunctions in order to reduce CPU stack use and improve maintainability. (Mikhail Kouzmich) AcpiGetHandle: Fix for parameter validation to detect invalid combinations of prefix handle and pathname. BZ 478 Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 77.9K Code, 17.1K Data, 95.0K Total Debug Version: 154.6K Code, 63.0K Data, 217.6K Total Current Release: Non-Debug Version: 78.1K Code, 17.1K Data, 95.2K Total Debug Version: 155.4K Code, 63.1K Data, 218.5K Total 2) iASL Compiler/Disassembler and Tools: Ported the -g option (get local ACPI tables) to the new ACPICA Table Manager to restore original behavior. ---------------------------------------- 27 September 2006. Summary of changes for version 20060927: 1) ACPI CA Core Subsystem: Removed the "Flags" parameter from AcpiGetRegister and AcpiSetRegister. These functions now use a spinlock for mutual exclusion and the interrupt level indication flag is not needed. Fixed a problem with the Global Lock where the lock could appear to be obtained before it is actually obtained. The global lock semaphore was inadvertently created with one unit instead of zero units. (BZ 464) Fiodor Suietov. Fixed a possible memory leak and fault in AcpiExResolveObjectToValue during a read from a buffer or region field. (BZ 458) Fiodor Suietov. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 77.9K Code, 17.1K Data, 95.0K Total Debug Version: 154.7K Code, 63.0K Data, 217.7K Total Current Release: Non-Debug Version: 77.9K Code, 17.1K Data, 95.0K Total Debug Version: 154.6K Code, 63.0K Data, 217.6K Total 2) iASL Compiler/Disassembler and Tools: Fixed a compilation problem with the pre-defined Resource Descriptor field names where an "object does not exist" error could be incorrectly generated if the parent ResourceTemplate pathname places the template within a different namespace scope than the current scope. (BZ 7212) Fixed a problem where the compiler could hang after syntax errors detected in an ElseIf construct. (BZ 453) Fixed a problem with the AmlFilename parameter to the DefinitionBlock() operator. An incorrect output filename was produced when this parameter was a null string (""). Now, the original input filename is used as the AML output filename, with an ".aml" extension. Implemented a generic batch command mode for the AcpiExec utility (execute any AML debugger command) (Valery Podrezov). ---------------------------------------- 12 September 2006. Summary of changes for version 20060912: 1) ACPI CA Core Subsystem: Enhanced the implementation of the "serialized mode" of the interpreter (enabled via the AcpiGbl_AllMethodsSerialized flag.) When this mode is specified, instead of creating a serialization semaphore per control method, the interpreter lock is simply no longer released before a blocking operation during control method execution. This effectively makes the AML Interpreter single-threaded. The overhead of a semaphore per-method is eliminated. Fixed a regression where an error was no longer emitted if a control method attempts to create 2 objects of the same name. This once again returns AE_ALREADY_EXISTS. When this exception occurs, it invokes the mechanism that will dynamically serialize the control method to possible prevent future errors. (BZ 440) Integrated a fix for a problem with PCI Express HID detection in the PCI Config Space setup procedure. (BZ 7145) Moved all FADT-related functions to a new file, tbfadt.c. Eliminated the AcpiHwInitialize function - the FADT registers are now validated when the table is loaded. Added two new warnings during FADT verification - 1) if the FADT is larger than the largest known FADT version, and 2) if there is a mismatch between a 32-bit block address and the 64-bit X counterpart (when both are non- zero.) Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 77.9K Code, 16.7K Data, 94.6K Total Debug Version: 154.9K Code, 62.6K Data, 217.5K Total Current Release: Non-Debug Version: 77.9K Code, 17.1K Data, 95.0K Total Debug Version: 154.7K Code, 63.0K Data, 217.7K Total 2) iASL Compiler/Disassembler and Tools: Fixed a problem with the implementation of the Switch() operator where the temporary variable was declared too close to the actual Switch, instead of at method level. This could cause a problem if the Switch() operator is within a while loop, causing an error on the second iteration. (BZ 460) Disassembler - fix for error emitted for unknown type for target of scope operator. Now, ignore it and continue. Disassembly of an FADT now verifies the input FADT and reports any errors found. Fix for proper disassembly of full-sized (ACPI 2.0) FADTs. Disassembly of raw data buffers with byte initialization data now prefixes each output line with the current buffer offset. Disassembly of ASF! table now includes all variable-length data fields at the end of some of the subtables. The disassembler now emits a comment if a buffer appears to be a ResourceTemplate, but cannot be disassembled as such because the EndTag does not appear at the very end of the buffer. AcpiExec - Added the "-t" command line option to enable the serialized mode of the AML interpreter. ---------------------------------------- 31 August 2006. Summary of changes for version 20060831: 1) ACPI CA Core Subsystem: Miscellaneous fixes for the Table Manager: - Correctly initialize internal common FADT for all 64-bit "X" fields - Fixed a couple table mapping issues during table load - Fixed a couple alignment issues for IA64 - Initialize input array to zero in AcpiInitializeTables - Additional parameter validation for AcpiGetTable, AcpiGetTableHeader, AcpiGetTableByIndex Change for GPE support: when a "wake" GPE is received, all wake GPEs are now immediately disabled to prevent the waking GPE from firing again and to prevent other wake GPEs from interrupting the wake process. Added the AcpiGpeCount global that tracks the number of processed GPEs, to be used for debugging systems with a large number of ACPI interrupts. Implemented support for the "DMAR" ACPI table (DMA Redirection Table) in both the ACPICA headers and the disassembler. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 77.8K Code, 16.5K Data, 94.3K Total Debug Version: 154.6K Code, 62.3K Data, 216.9K Total Current Release: Non-Debug Version: 77.9K Code, 16.7K Data, 94.6K Total Debug Version: 154.9K Code, 62.6K Data, 217.5K Total 2) iASL Compiler/Disassembler and Tools: Disassembler support for the DMAR ACPI table. ---------------------------------------- 23 August 2006. Summary of changes for version 20060823: 1) ACPI CA Core Subsystem: The Table Manager component has been completely redesigned and reimplemented. The new design is much simpler, and reduces the overall code and data size of the kernel-resident ACPICA by approximately 5%. Also, it is now possible to obtain the ACPI tables very early during kernel initialization, even before dynamic memory management is initialized. (Alexey Starikovskiy, Fiodor Suietov, Bob Moore) Obsolete ACPICA interfaces: - AcpiGetFirmwareTable: Use AcpiGetTable instead (works at early kernel init time). - AcpiLoadTable: Not needed. - AcpiUnloadTable: Not needed. New ACPICA interfaces: - AcpiInitializeTables: Must be called before the table manager can be used. - AcpiReallocateRootTable: Used to transfer the root table to dynamically allocated memory after it becomes available. - AcpiGetTableByIndex: Allows the host to easily enumerate all ACPI tables in the RSDT/XSDT. Other ACPICA changes: - AcpiGetTableHeader returns the actual mapped table header, not a copy. Use AcpiOsUnmapMemory to free this mapping. - AcpiGetTable returns the actual mapped table. The mapping is managed internally and must not be deleted by the caller. Use of this interface causes no additional dynamic memory allocation. - AcpiFindRootPointer: Support for physical addressing has been eliminated, it appeared to be unused. - The interface to AcpiOsMapMemory has changed to be consistent with the other allocation interfaces. - The interface to AcpiOsGetRootPointer has changed to eliminate unnecessary parameters. - ACPI_PHYSICAL_ADDRESS is now 32 bits on 32-bit platforms, 64 bits on 64- bit platforms. Was previously 64 bits on all platforms. - The interface to the ACPI Global Lock acquire/release macros have changed slightly since ACPICA no longer keeps a local copy of the FACS with a constructed pointer to the actual global lock. Porting to the new table manager: - AcpiInitializeTables: Must be called once, and can be called anytime during the OS initialization process. It allows the host to specify an area of memory to be used to store the internal version of the RSDT/XSDT (root table). This allows the host to access ACPI tables before memory management is initialized and running. - AcpiReallocateRootTable: Can be called after memory management is running to copy the root table to a dynamically allocated array, freeing up the scratch memory specified in the call to AcpiInitializeTables. - AcpiSubsystemInitialize: This existing interface is independent of the Table Manager, and does not have to be called before the Table Manager can be used, it only must be called before the rest of ACPICA can be used. - ACPI Tables: Some changes have been made to the names and structure of the actbl.h and actbl1.h header files and may require changes to existing code. For example, bitfields have been completely removed because of their lack of portability across C compilers. - Update interfaces to the Global Lock acquire/release macros if local versions are used. (see acwin.h) Obsolete files: tbconvrt.c, tbget.c, tbgetall.c, tbrsdt.c New files: tbfind.c Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 80.7K Code, 17.9K Data, 98.6K Total Debug Version: 161.0K Code, 65.1K Data, 226.1K Total Current Release: Non-Debug Version: 77.8K Code, 16.5K Data, 94.3K Total Debug Version: 154.6K Code, 62.3K Data, 216.9K Total 2) iASL Compiler/Disassembler and Tools: No changes for this release. ---------------------------------------- 21 July 2006. Summary of changes for version 20060721: 1) ACPI CA Core Subsystem: The full source code for the ASL test suite used to validate the iASL compiler and the ACPICA core subsystem is being released with the ACPICA source for the first time. The source is contained in a separate package and consists of over 1100 files that exercise all ASL/AML operators. The package should appear on the Intel/ACPI web site shortly. (Valery Podrezov, Fiodor Suietov) Completed a new design and implementation for support of the ACPI Global Lock. On the OS side, the global lock is now treated as a standard AML mutex. Previously, multiple OS threads could "acquire" the global lock simultaneously. However, this could cause the BIOS to be starved out of the lock - especially in cases such as the Embedded Controller driver where there is a tight coupling between the OS and the BIOS. Implemented an optimization for the ACPI Global Lock interrupt mechanism. The Global Lock interrupt handler no longer queues the execution of a separate thread to signal the global lock semaphore. Instead, the semaphore is signaled directly from the interrupt handler. Implemented support within the AML interpreter for package objects that contain a larger AML length (package list length) than the package element count. In this case, the length of the package is truncated to match the package element count. Some BIOS code apparently modifies the package length on the fly, and this change supports this behavior. Provides compatibility with the MS AML interpreter. (With assistance from Fiodor Suietov) Implemented a temporary fix for the BankValue parameter of a Bank Field to support all constant values, now including the Zero and One opcodes. Evaluation of this parameter must eventually be converted to a full TermArg evaluation. A not-implemented error is now returned (temporarily) for non- constant values for this parameter. Fixed problem reports (Fiodor Suietov) integrated: - Fix for premature object deletion after CopyObject on Operation Region (BZ 350) Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 80.7K Code, 18.0K Data, 98.7K Total Debug Version: 160.9K Code, 65.1K Data, 226.0K Total Current Release: Non-Debug Version: 80.7K Code, 17.9K Data, 98.6K Total Debug Version: 161.0K Code, 65.1K Data, 226.1K Total 2) iASL Compiler/Disassembler and Tools: No changes for this release. ---------------------------------------- 07 July 2006. Summary of changes for version 20060707: 1) ACPI CA Core Subsystem: Added the ACPI_PACKED_POINTERS_NOT_SUPPORTED macro to support C compilers that do not allow the initialization of address pointers within packed structures - even though the hardware itself may support misaligned transfers. Some of the debug data structures are packed by default to minimize size. Added an error message for the case where AcpiOsGetThreadId() returns zero. A non-zero value is required by the core ACPICA code to ensure the proper operation of AML mutexes and recursive control methods. The DSDT is now the only ACPI table that determines whether the AML interpreter is in 32-bit or 64-bit mode. Not really a functional change, but the hooks for per-table 32/64 switching have been removed from the code. A clarification to the ACPI specification is forthcoming in ACPI 3.0B. Fixed a possible leak of an OwnerID in the error path of AcpiTbInitTableDescriptor (tbinstal.c), and migrated all table OwnerID deletion to a single place in AcpiTbUninstallTable to correct possible leaks when using the AcpiTbDeleteTablesByType interface (with assistance from Lance Ortiz.) Fixed a problem with Serialized control methods where the semaphore associated with the method could be over-signaled after multiple method invocations. Fixed two issues with the locking of the internal namespace data structure. Both the Unload() operator and AcpiUnloadTable interface now lock the namespace during the namespace deletion associated with the table unload (with assistance from Linn Crosetto.) Fixed problem reports (Valery Podrezov) integrated: - Eliminate unnecessary memory allocation for CreateXxxxField (BZ 5426) Fixed problem reports (Fiodor Suietov) integrated: - Incomplete cleanup branches in AcpiTbGetTableRsdt (BZ 369) - On Address Space handler deletion, needless deactivation call (BZ 374) - AcpiRemoveAddressSpaceHandler: validate Device handle parameter (BZ 375) - Possible memory leak, Notify sub-objects of Processor, Power, ThermalZone (BZ 376) - AcpiRemoveAddressSpaceHandler: validate Handler parameter (BZ 378) - Minimum Length of RSDT should be validated (BZ 379) - AcpiRemoveNotifyHandler: return AE_NOT_EXIST if Processor Obj has no Handler (BZ (380) - AcpiUnloadTable: return AE_NOT_EXIST if no table of specified type loaded (BZ 381) Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 80.5K Code, 17.8K Data, 98.3K Total Debug Version: 160.8K Code, 64.8K Data, 225.6K Total Current Release: Non-Debug Version: 80.7K Code, 17.9K Data, 98.6K Total Debug Version: 161.0K Code, 65.1K Data, 226.1K Total 2) iASL Compiler/Disassembler and Tools: Fixed problem reports: Compiler segfault when ASL contains a long (>1024) String declaration (BZ 436) ---------------------------------------- 23 June 2006. Summary of changes for version 20060623: 1) ACPI CA Core Subsystem: Implemented a new ACPI_SPINLOCK type for the OSL lock interfaces. This allows the type to be customized to the host OS for improved efficiency (since a spinlock is usually a very small object.) Implemented support for "ignored" bits in the ACPI registers. According to the ACPI specification, these bits should be preserved when writing the registers via a read/modify/write cycle. There are 3 bits preserved in this manner: PM1_CONTROL[0] (SCI_EN), PM1_CONTROL[9], and PM1_STATUS[11]. Implemented the initial deployment of new OSL mutex interfaces. Since some host operating systems have separate mutex and semaphore objects, this feature was requested. The base code now uses mutexes (and the new mutex interfaces) wherever a binary semaphore was used previously. However, for the current release, the mutex interfaces are defined as macros to map them to the existing semaphore interfaces. Therefore, no OSL changes are required at this time. (See acpiosxf.h) Fixed several problems with the support for the control method SyncLevel parameter. The SyncLevel now works according to the ACPI specification and in concert with the Mutex SyncLevel parameter, since the current SyncLevel is a property of the executing thread. Mutual exclusion for control methods is now implemented with a mutex instead of a semaphore. Fixed three instances of the use of the C shift operator in the bitfield support code (exfldio.c) to avoid the use of a shift value larger than the target data width. The behavior of C compilers is undefined in this case and can cause unpredictable results, and therefore the case must be detected and avoided. (Fiodor Suietov) Added an info message whenever an SSDT or OEM table is loaded dynamically via the Load() or LoadTable() ASL operators. This should improve debugging capability since it will show exactly what tables have been loaded (beyond the tables present in the RSDT/XSDT.) Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 80.0K Code, 17.6K Data, 97.6K Total Debug Version: 160.2K Code, 64.7K Data, 224.9K Total Current Release: Non-Debug Version: 80.5K Code, 17.8K Data, 98.3K Total Debug Version: 160.8K Code, 64.8K Data, 225.6K Total 2) iASL Compiler/Disassembler and Tools: No changes for this release. ---------------------------------------- 08 June 2006. Summary of changes for version 20060608: 1) ACPI CA Core Subsystem: Converted the locking mutex used for the ACPI hardware to a spinlock. This change should eliminate all problems caused by attempting to acquire a semaphore at interrupt level, and it means that all ACPICA external interfaces that directly access the ACPI hardware can be safely called from interrupt level. OSL code that implements the semaphore interfaces should be able to eliminate any workarounds for being called at interrupt level. Fixed a regression introduced in 20060526 where the ACPI device initialization could be prematurely aborted with an AE_NOT_FOUND if a device did not have an optional _INI method. Fixed an IndexField issue where a write to the Data Register should be limited in size to the AccessSize (width) of the IndexField itself. (BZ 433, Fiodor Suietov) Fixed problem reports (Valery Podrezov) integrated: - Allow store of ThermalZone objects to Debug object (BZ 5369/5370) Fixed problem reports (Fiodor Suietov) integrated: - AcpiGetTableHeader doesn't handle multiple instances correctly (BZ 364) Removed four global mutexes that were obsolete and were no longer being used. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 80.0K Code, 17.7K Data, 97.7K Total Debug Version: 160.3K Code, 64.9K Data, 225.2K Total Current Release: Non-Debug Version: 80.0K Code, 17.6K Data, 97.6K Total Debug Version: 160.2K Code, 64.7K Data, 224.9K Total 2) iASL Compiler/Disassembler and Tools: Fixed a fault when using -g option (get tables from registry) on Windows machines. Fixed problem reports integrated: - Generate error if CreateField NumBits parameter is zero. (BZ 405) - Fault if Offset/Length in Field unit is very large (BZ 432, Fiodor Suietov) - Global table revision override (-r) is ignored (BZ 413) ---------------------------------------- 26 May 2006. Summary of changes for version 20060526: 1) ACPI CA Core Subsystem: Restructured, flattened, and simplified the internal interfaces for namespace object evaluation - resulting in smaller code, less CPU stack use, and fewer interfaces. (With assistance from Mikhail Kouzmich) Fixed a problem with the CopyObject operator where the first parameter was not typed correctly for the parser, interpreter, compiler, and disassembler. Caused various errors and unexpected behavior. Fixed a problem where a ShiftLeft or ShiftRight of more than 64 bits produced incorrect results with some C compilers. Since the behavior of C compilers when the shift value is larger than the datatype width is apparently not well defined, the interpreter now detects this condition and simply returns zero as expected in all such cases. (BZ 395) Fixed problem reports (Valery Podrezov) integrated: - Update String-to-Integer conversion to match ACPI 3.0A spec (BZ 5329) - Allow interpreter to handle nested method declarations (BZ 5361) Fixed problem reports (Fiodor Suietov) integrated: - AcpiTerminate doesn't free debug memory allocation list objects (BZ 355) - After Core Subsystem shutdown, AcpiSubsystemStatus returns AE_OK (BZ 356) - AcpiOsUnmapMemory for RSDP can be invoked inconsistently (BZ 357) - Resource Manager should return AE_TYPE for non-device objects (BZ 358) - Incomplete cleanup branch in AcpiNsEvaluateRelative (BZ 359) - Use AcpiOsFree instead of ACPI_FREE in AcpiRsSetSrsMethodData (BZ 360) - Incomplete cleanup branch in AcpiPsParseAml (BZ 361) - Incomplete cleanup branch in AcpiDsDeleteWalkState (BZ 362) - AcpiGetTableHeader returns AE_NO_ACPI_TABLES until DSDT is loaded (BZ 365) - Status of the Global Initialization Handler call not used (BZ 366) - Incorrect object parameter to Global Initialization Handler (BZ 367) Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.8K Code, 17.7K Data, 97.5K Total Debug Version: 160.5K Code, 65.1K Data, 225.6K Total Current Release: Non-Debug Version: 80.0K Code, 17.7K Data, 97.7K Total Debug Version: 160.3K Code, 64.9K Data, 225.2K Total 2) iASL Compiler/Disassembler and Tools: Modified the parser to allow the names IO, DMA, and IRQ to be used as namespace identifiers with no collision with existing resource descriptor macro names. This provides compatibility with other ASL compilers and is most useful for disassembly/recompilation of existing tables without parse errors. (With assistance from Thomas Renninger) Disassembler: fixed an incorrect disassembly problem with the DataTableRegion and CopyObject operators. Fixed a possible fault during disassembly of some Alias operators. ---------------------------------------- 12 May 2006. Summary of changes for version 20060512: 1) ACPI CA Core Subsystem: Replaced the AcpiOsQueueForExecution interface with a new interface named AcpiOsExecute. The major difference is that the new interface does not have a Priority parameter, this appeared to be useless and has been replaced by a Type parameter. The Type tells the host what type of execution is being requested, such as global lock handler, notify handler, GPE handler, etc. This allows the host to queue and execute the request as appropriate for the request type, possibly using different work queues and different priorities for the various request types. This enables fixes for multithreading deadlock problems such as BZ #5534, and will require changes to all existing OS interface layers. (Alexey Starikovskiy and Bob Moore) Fixed a possible memory leak associated with the support for the so- called "implicit return" ACPI extension. Reported by FreeBSD, BZ #6514. (Fiodor Suietov) Fixed a problem with the Load() operator where a table load from an operation region could overwrite an internal table buffer by up to 7 bytes and cause alignment faults on IPF systems. (With assistance from Luming Yu) Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 79.7K Code, 17.7K Data, 97.4K Total Debug Version: 160.1K Code, 65.2K Data, 225.3K Total Current Release: Non-Debug Version: 79.8K Code, 17.7K Data, 97.5K Total Debug Version: 160.5K Code, 65.1K Data, 225.6K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Implemented support to cross reference the internal namespace and automatically generate ASL External() statements for symbols not defined within the current table being disassembled. This will simplify the disassembly and recompilation of interdependent tables such as SSDTs since these statements will no longer have to be added manually. Disassembler: Implemented experimental support to automatically detect invocations of external control methods and generate appropriate External() statements. This is problematic because the AML cannot be correctly parsed until the number of arguments for each control method is known. Currently, standalone method invocations and invocations as the source operand of a Store() statement are supported. Disassembler: Implemented support for the ASL pseudo-operators LNotEqual, LLessEqual, and LGreaterEqual. Previously disassembled as LNot(LEqual()), LNot(LGreater()), and LNot(LLess()), this makes the disassembled ASL code more readable and likely closer to the original ASL source. ---------------------------------------- 21 April 2006. Summary of changes for version 20060421: 1) ACPI CA Core Subsystem: Removed a device initialization optimization introduced in 20051216 where the _STA method was not run unless an _INI was also present for the same device. This optimization could cause problems because it could allow _INI methods to be run within a not-present device subtree. (If a not-present device had no _INI, _STA would not be run, the not-present status would not be discovered, and the children of the device would be incorrectly traversed.) Implemented a new _STA optimization where namespace subtrees that do not contain _INI are identified and ignored during device initialization. Selectively running _STA can significantly improve boot time on large machines (with assistance from Len Brown.) Implemented support for the device initialization case where the returned _STA flags indicate a device not-present but functioning. In this case, _INI is not run, but the device children are examined for presence, as per the ACPI specification. Implemented an additional change to the IndexField support in order to conform to MS behavior. The value written to the Index Register is not simply a byte offset, it is a byte offset in units of the access width of the parent Index Field. (Fiodor Suietov) Defined and deployed a new OSL interface, AcpiOsValidateAddress. This interface is called during the creation of all AML operation regions, and allows the host OS to exert control over what addresses it will allow the AML code to access. Operation Regions whose addresses are disallowed will cause a runtime exception when they are actually accessed (will not affect or abort table loading.) See oswinxf or osunixxf for an example implementation. Defined and deployed a new OSL interface, AcpiOsValidateInterface. This interface allows the host OS to match the various "optional" interface/behavior strings for the _OSI predefined control method as appropriate (with assistance from Bjorn Helgaas.) See oswinxf or osunixxf for an example implementation. Restructured and corrected various problems in the exception handling code paths within DsCallControlMethod and DsTerminateControlMethod in dsmethod (with assistance from Takayoshi Kochi.) Modified the Linux source converter to ignore quoted string literals while converting identifiers from mixed to lower case. This will correct problems with the disassembler and other areas where such strings must not be modified. The ACPI_FUNCTION_* macros no longer require quotes around the function name. This allows the Linux source converter to convert the names, now that the converter ignores quoted strings. Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 81.1K Code, 17.7K Data, 98.8K Total Debug Version: 158.9K Code, 64.9K Data, 223.8K Total Current Release: Non-Debug Version: 79.7K Code, 17.7K Data, 97.4K Total Debug Version: 160.1K Code, 65.2K Data, 225.3K Total 2) iASL Compiler/Disassembler and Tools: Implemented 3 new warnings for iASL, and implemented multiple warning levels (w2 flag). 1) Ignored timeouts: If the TimeoutValue parameter to Wait or Acquire is not WAIT_FOREVER (0xFFFF) and the code does not examine the return value to check for the possible timeout, a warning is issued. 2) Useless operators: If an ASL operator does not specify an optional target operand and it also does not use the function return value from the operator, a warning is issued since the operator effectively does nothing. 3) Unreferenced objects: If a namespace object is created, but never referenced, a warning is issued. This is a warning level 2 since there are cases where this is ok, such as when a secondary table is loaded that uses the unreferenced objects. Even so, care is taken to only flag objects that don't look like they will ever be used. For example, the reserved methods (starting with an underscore) are usually not referenced because it is expected that the OS will invoke them. ---------------------------------------- 31 March 2006. Summary of changes for version 20060331: 1) ACPI CA Core Subsystem: Implemented header file support for the following additional ACPI tables: ASF!, BOOT, CPEP, DBGP, MCFG, SPCR, SPMI, TCPA, and WDRT. With this support, all current and known ACPI tables are now defined in the ACPICA headers and are available for use by device drivers and other software. Implemented support to allow tables that contain ACPI names with invalid characters to be loaded. Previously, this would cause the table load to fail, but since there are several known cases of such tables on existing machines, this change was made to enable ACPI support for them. Also, this matches the behavior of the Microsoft ACPI implementation. Fixed a couple regressions introduced during the memory optimization in the 20060317 release. The namespace node definition required additional reorganization and an internal datatype that had been changed to 8-bit was restored to 32-bit. (Valery Podrezov) Fixed a problem where a null pointer passed to AcpiUtDeleteGenericState could be passed through to AcpiOsReleaseObject which is unexpected. Such null pointers are now trapped and ignored, matching the behavior of the previous implementation before the deployment of AcpiOsReleaseObject. (Valery Podrezov, Fiodor Suietov) Fixed a memory mapping leak during the deletion of a SystemMemory operation region where a cached memory mapping was not deleted. This became a noticeable problem for operation regions that are defined within frequently used control methods. (Dana Meyers) Reorganized the ACPI table header files into two main files: one for the ACPI tables consumed by the ACPICA core, and another for the miscellaneous ACPI tables that are consumed by the drivers and other software. The various FADT definitions were merged into one common section and three different tables (ACPI 1.0, 1.0+, and 2.0) Example Code and Data Size: These are the sizes for the OS-independent acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Previous Release: Non-Debug Version: 80.9K Code, 17.7K Data, 98.6K Total Debug Version: 158.7K Code, 64.8K Data, 223.5K Total Current Release: Non-Debug Version: 81.1K Code, 17.7K Data, 98.8K Total Debug Version: 158.9K Code, 64.9K Data, 223.8K Total 2) iASL Compiler/Disassembler and Tools: Disassembler: Implemented support to decode and format all non-AML ACPI tables (tables other than DSDTs and SSDTs.) This includes the new tables added to the ACPICA headers, therefore all current and known ACPI tables are supported. Disassembler: The change to allow ACPI names with invalid characters also enables the disassembly of such tables. Invalid characters within names are changed to '*' to make the name printable; the iASL compiler will still generate an error for such names, however, since this is an invalid ACPI character. Implemented an option for AcpiXtract (-a) to extract all tables found in the input file. The default invocation extracts only the DSDTs and SSDTs. Fixed a couple of gcc generation issues for iASL and AcpiExec and added a makefile for the AcpiXtract utility. ---------------------------------------- 17 March 2006. Summary of changes for version 20060317: 1) ACPI CA Core Subsystem: Implemented the use of a cache object for all internal namespace nodes. Since there are about 1000 static nodes in a typical system, this will decrease memory use for cache implementations that minimize per- allocation overhead (such as a slab allocator.) Removed the reference count mechanism for internal namespace nodes, since it was deemed unnecessary. This reduces the size of each namespace node by about 5%-10% on all platforms. Nodes are now 20 bytes for the 32-bit case, and 32 bytes for the 64-bit case. Optimized several internal data structures to reduce object size on 64- bit platforms by packing data within the 64-bit alignment. This includes the frequently used ACPI_OPERAND_OBJECT, of which there can be ~1000 static instances corresponding to the namespace objects. Added two new strings for the predefined _OSI method: "Windows 2001.1 SP1" and "Windows 2006". Split the allocation tracking mechanism out to a separate file, from utalloc.c to uttrack.c. This mechanism appears to be only useful for application-level code. Kernels may wish to not include uttrack.c in distributions. Removed all remnants of the obsolete ACPI_REPORT_* macros and the associated code. (These macros have been replaced by the ACPI_ERROR and ACPI_WARNING macros.) Code and Data Size: These are the sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 81.1K Code, 17.8K Data, 98.9K Total Debug Version: 161.6K Code, 65.7K Data, 227.3K Total Current Release: Non-Debug Version: 80.9K Code, 17.7K Data, 98.6K Total Debug Version: 158.7K Code, 64.8K Data, 223.5K Total 2) iASL Compiler/Disassembler and Tools: Implemented an ANSI C version of the acpixtract utility. This version will automatically extract the DSDT and all SSDTs from the input acpidump text file and dump the binary output to separate files. It can also display a summary of the input file including the headers for each table found and will extract any single ACPI table, with any signature. (See source/tools/acpixtract) ---------------------------------------- 10 March 2006. Summary of changes for version 20060310: 1) ACPI CA Core Subsystem: Tagged all external interfaces to the subsystem with the new ACPI_EXPORT_SYMBOL macro. This macro can be defined as necessary to assist kernel integration. For Linux, the macro resolves to the EXPORT_SYMBOL macro. The default definition is NULL. Added the ACPI_THREAD_ID type for the return value from AcpiOsGetThreadId. This allows the host to define this as necessary to simplify kernel integration. The default definition is ACPI_NATIVE_UINT. Fixed two interpreter problems related to error processing, the deletion of objects, and placing invalid pointers onto the internal operator result stack. BZ 6028, 6151 (Valery Podrezov) Increased the reference count threshold where a warning is emitted for large reference counts in order to eliminate unnecessary warnings on systems with large namespaces (especially 64-bit.) Increased the value from 0x400 to 0x800. Due to universal disagreement as to the meaning of the 'c' in the calloc() function, the ACPI_MEM_CALLOCATE macro has been renamed to ACPI_ALLOCATE_ZEROED so that the purpose of the interface is 'clear'. ACPI_MEM_ALLOCATE and ACPI_MEM_FREE are renamed to ACPI_ALLOCATE and ACPI_FREE. Code and Data Size: These are the sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 81.0K Code, 17.8K Data, 98.8K Total Debug Version: 161.4K Code, 65.7K Data, 227.1K Total Current Release: Non-Debug Version: 81.1K Code, 17.8K Data, 98.9K Total Debug Version: 161.6K Code, 65.7K Data, 227.3K Total 2) iASL Compiler/Disassembler: Disassembler: implemented support for symbolic resource descriptor references. If a CreateXxxxField operator references a fixed offset within a resource descriptor, a name is assigned to the descriptor and the offset is translated to the appropriate resource tag and pathname. The addition of this support brings the disassembled code very close to the original ASL source code and helps eliminate run-time errors when the disassembled code is modified (and recompiled) in such a way as to invalidate the original fixed offsets. Implemented support for a Descriptor Name as the last parameter to the ASL Register() macro. This parameter was inadvertently left out of the ACPI specification, and will be added for ACPI 3.0b. Fixed a problem where the use of the "_OSI" string (versus the full path "\_OSI") caused an internal compiler error. ("No back ptr to op") Fixed a problem with the error message that occurs when an invalid string is used for a _HID object (such as one with an embedded asterisk: "*PNP010A".) The correct message is now displayed. ---------------------------------------- 17 February 2006. Summary of changes for version 20060217: 1) ACPI CA Core Subsystem: Implemented a change to the IndexField support to match the behavior of the Microsoft AML interpreter. The value written to the Index register is now a byte offset, no longer an index based upon the width of the Data register. This should fix IndexField problems seen on some machines where the Data register is not exactly one byte wide. The ACPI specification will be clarified on this point. Fixed a problem where several resource descriptor types could overrun the internal descriptor buffer due to size miscalculation: VendorShort, VendorLong, and Interrupt. This was noticed on IA64 machines, but could affect all platforms. Fixed a problem where individual resource descriptors were misaligned within the internal buffer, causing alignment faults on IA64 platforms. Code and Data Size: These are the sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. The values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 81.1K Code, 17.8K Data, 98.9K Total Debug Version: 161.3K Code, 65.6K Data, 226.9K Total Current Release: Non-Debug Version: 81.0K Code, 17.8K Data, 98.8K Total Debug Version: 161.4K Code, 65.7K Data, 227.1K Total 2) iASL Compiler/Disassembler: Implemented support for new reserved names: _WDG and _WED are Microsoft extensions for Windows Instrumentation Management, _TDL is a new ACPI- defined method (Throttling Depth Limit.) Fixed a problem where a zero-length VendorShort or VendorLong resource descriptor was incorrectly emitted as a descriptor of length one. ---------------------------------------- 10 February 2006. Summary of changes for version 20060210: 1) ACPI CA Core Subsystem: Removed a couple of extraneous ACPI_ERROR messages that appeared during normal execution. These became apparent after the conversion from ACPI_DEBUG_PRINT. Fixed a problem where the CreateField operator could hang if the BitIndex or NumBits parameter referred to a named object. (Valery Podrezov, BZ 5359) Fixed a problem where a DeRefOf operation on a buffer object incorrectly failed with an exception. This also fixes a couple of related RefOf and DeRefOf issues. (Valery Podrezov, BZ 5360/5392/5387) Fixed a problem where the AE_BUFFER_LIMIT exception was returned instead of AE_STRING_LIMIT on an out-of-bounds Index() operation. (Valery Podrezov, BZ 5480) Implemented a memory cleanup at the end of the execution of each iteration of an AML While() loop, preventing the accumulation of outstanding objects. (Valery Podrezov, BZ 5427) Eliminated a chunk of duplicate code in the object resolution code. (Valery Podrezov, BZ 5336) Fixed several warnings during the 64-bit code generation. The AcpiSrc source code conversion tool now inserts one line of whitespace after an if() statement that is followed immediately by a comment, improving readability of the Linux code. Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 81.0K Code, 17.9K Data, 98.9K Total Debug Version: 161.3K Code, 65.7K Data, 227.0K Total Current Release: Non-Debug Version: 81.1K Code, 17.8K Data, 98.9K Total Debug Version: 161.3K Code, 65.6K Data, 226.9K Total 2) iASL Compiler/Disassembler: Fixed a problem with the disassembly of a BankField operator with a complex expression for the BankValue parameter. ---------------------------------------- 27 January 2006. Summary of changes for version 20060127: 1) ACPI CA Core Subsystem: Implemented support in the Resource Manager to allow unresolved namestring references within resource package objects for the _PRT method. This support is in addition to the previously implemented unresolved reference support within the AML parser. If the interpreter slack mode is enabled, these unresolved references will be passed through to the caller as a NULL package entry. Implemented and deployed new macros and functions for error and warning messages across the subsystem. These macros are simpler and generate less code than their predecessors. The new macros ACPI_ERROR, ACPI_EXCEPTION, ACPI_WARNING, and ACPI_INFO replace the ACPI_REPORT_* macros. The older macros remain defined to allow ACPI drivers time to migrate to the new macros. Implemented the ACPI_CPU_FLAGS type to simplify host OS integration of the Acquire/Release Lock OSL interfaces. Fixed a problem where Alias ASL operators are sometimes not correctly resolved, in both the interpreter and the iASL compiler. Fixed several problems with the implementation of the ConcatenateResTemplate ASL operator. As per the ACPI specification, zero length buffers are now treated as a single EndTag. One-length buffers always cause a fatal exception. Non-zero length buffers that do not end with a full 2-byte EndTag cause a fatal exception. Fixed a possible structure overwrite in the AcpiGetObjectInfo external interface. (With assistance from Thomas Renninger) Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 83.1K Code, 18.4K Data, 101.5K Total Debug Version: 163.2K Code, 66.2K Data, 229.4K Total Current Release: Non-Debug Version: 81.0K Code, 17.9K Data, 98.9K Total Debug Version: 161.3K Code, 65.7K Data, 227.0K Total 2) iASL Compiler/Disassembler: Fixed an internal error that was generated for any forward references to ASL Alias objects. ---------------------------------------- 13 January 2006. Summary of changes for version 20060113: 1) ACPI CA Core Subsystem: Added 2006 copyright to all module headers and signons. This affects virtually every file in the ACPICA core subsystem, iASL compiler, and the utilities. Enhanced the ACPICA error reporting in order to simplify user migration to the non-debug version of ACPICA. Replaced all instances of the ACPI_DEBUG_PRINT macro invoked at the ACPI_DB_ERROR and ACPI_DB_WARN debug levels with the ACPI_REPORT_ERROR and ACPI_REPORT_WARNING macros, respectively. This preserves all error and warning messages in the non- debug version of the ACPICA code (this has been referred to as the "debug lite" option.) Over 200 cases were converted to create a total of over 380 error/warning messages across the ACPICA code. This increases the code and data size of the default non-debug version of the code somewhat (about 13K), but all error/warning reporting may be disabled if desired (and code eliminated) by specifying the ACPI_NO_ERROR_MESSAGES compile-time configuration option. The size of the debug version of ACPICA remains about the same. Fixed a memory leak within the AML Debugger "Set" command. One object was not properly deleted for every successful invocation of the command. Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 76.6K Code, 12.3K Data, 88.9K Total Debug Version: 163.7K Code, 67.5K Data, 231.2K Total Current Release: Non-Debug Version: 83.1K Code, 18.4K Data, 101.5K Total Debug Version: 163.2K Code, 66.2K Data, 229.4K Total 2) iASL Compiler/Disassembler: The compiler now officially supports the ACPI 3.0a specification that was released on December 30, 2005. (Specification is available at www.acpi.info) ---------------------------------------- 16 December 2005. Summary of changes for version 20051216: 1) ACPI CA Core Subsystem: Implemented optional support to allow unresolved names within ASL Package objects. A null object is inserted in the package when a named reference cannot be located in the current namespace. Enabled via the interpreter slack flag, this should eliminate AE_NOT_FOUND exceptions seen on machines that contain such code. Implemented an optimization to the initialization sequence that can improve boot time. During ACPI device initialization, the _STA method is now run if and only if the _INI method exists. The _STA method is used to determine if the device is present; An _INI can only be run if _STA returns present, but it is a waste of time to run the _STA method if the _INI does not exist. (Prototype and assistance from Dong Wei) Implemented use of the C99 uintptr_t for the pointer casting macros if it is available in the current compiler. Otherwise, the default (void *) cast is used as before. Fixed some possible memory leaks found within the execution path of the Break, Continue, If, and CreateField operators. (Valery Podrezov) Fixed a problem introduced in the 20051202 release where an exception is generated during method execution if a control method attempts to declare another method. Moved resource descriptor string constants that are used by both the AML disassembler and AML debugger to the common utilities directory so that these components are independent. Implemented support in the AcpiExec utility (-e switch) to globally ignore exceptions during control method execution (method is not aborted.) Added the rsinfo.c source file to the AcpiExec makefile for Linux/Unix generation. Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 76.3K Code, 12.3K Data, 88.6K Total Debug Version: 163.2K Code, 67.4K Data, 230.6K Total Current Release: Non-Debug Version: 76.6K Code, 12.3K Data, 88.9K Total Debug Version: 163.7K Code, 67.5K Data, 231.2K Total 2) iASL Compiler/Disassembler: Fixed a problem where a CPU stack overflow fault could occur if a recursive method call was made from within a Return statement. ---------------------------------------- 02 December 2005. Summary of changes for version 20051202: 1) ACPI CA Core Subsystem: Modified the parsing of control methods to no longer create namespace objects during the first pass of the parse. Objects are now created only during the execute phase, at the moment the namespace creation operator is encountered in the AML (Name, OperationRegion, CreateByteField, etc.) This should eliminate ALREADY_EXISTS exceptions seen on some machines where reentrant control methods are protected by an AML mutex. The mutex will now correctly block multiple threads from attempting to create the same object more than once. Increased the number of available Owner Ids for namespace object tracking from 32 to 255. This should eliminate the OWNER_ID_LIMIT exceptions seen on some machines with a large number of ACPI tables (either static or dynamic). Fixed a problem with the AcpiExec utility where a fault could occur when the -b switch (batch mode) is used. Enhanced the namespace dump routine to output the owner ID for each namespace object. Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 76.3K Code, 12.3K Data, 88.6K Total Debug Version: 163.0K Code, 67.4K Data, 230.4K Total Current Release: Non-Debug Version: 76.3K Code, 12.3K Data, 88.6K Total Debug Version: 163.2K Code, 67.4K Data, 230.6K Total 2) iASL Compiler/Disassembler: Fixed a parse error during compilation of certain Switch/Case constructs. To simplify the parse, the grammar now allows for multiple Default statements and this error is now detected and flagged during the analysis phase. Disassembler: The disassembly now includes the contents of the original table header within a comment at the start of the file. This includes the name and version of the original ASL compiler. ---------------------------------------- 17 November 2005. Summary of changes for version 20051117: 1) ACPI CA Core Subsystem: Fixed a problem in the AML parser where the method thread count could be decremented below zero if any errors occurred during the method parse phase. This should eliminate AE_AML_METHOD_LIMIT exceptions seen on some machines. This also fixed a related regression with the mechanism that detects and corrects methods that cannot properly handle reentrancy (related to the deployment of the new OwnerId mechanism.) Eliminated the pre-parsing of control methods (to detect errors) during table load. Related to the problem above, this was causing unwind issues if any errors occurred during the parse, and it seemed to be overkill. A table load should not be aborted if there are problems with any single control method, thus rendering this feature rather pointless. Fixed a problem with the new table-driven resource manager where an internal buffer overflow could occur for small resource templates. Implemented a new external interface, AcpiGetVendorResource. This interface will find and return a vendor-defined resource descriptor within a _CRS or _PRS method via an ACPI 3.0 UUID match. With assistance from Bjorn Helgaas. Removed the length limit (200) on string objects as per the upcoming ACPI 3.0A specification. This affects the following areas of the interpreter: 1) any implicit conversion of a Buffer to a String, 2) a String object result of the ASL Concatenate operator, 3) the String object result of the ASL ToString operator. Fixed a problem in the Windows OS interface layer (OSL) where a WAIT_FOREVER on a semaphore object would incorrectly timeout. This allows the multithreading features of the AcpiExec utility to work properly under Windows. Updated the Linux makefiles for the iASL compiler and AcpiExec to include the recently added file named "utresrc.c". Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 76.2K Code, 12.3K Data, 88.5K Total Debug Version: 163.0K Code, 67.4K Data, 230.4K Total Current Release: Non-Debug Version: 76.3K Code, 12.3K Data, 88.6K Total Debug Version: 163.0K Code, 67.4K Data, 230.4K Total 2) iASL Compiler/Disassembler: Removed the limit (200) on string objects as per the upcoming ACPI 3.0A specification. For the iASL compiler, this means that string literals within the source ASL can be of any length. Enhanced the listing output to dump the AML code for resource descriptors immediately after the ASL code for each descriptor, instead of in a block at the end of the entire resource template. Enhanced the compiler debug output to dump the entire original parse tree constructed during the parse phase, before any transforms are applied to the tree. The transformed tree is dumped also. ---------------------------------------- 02 November 2005. Summary of changes for version 20051102: 1) ACPI CA Core Subsystem: Modified the subsystem initialization sequence to improve GPE support. The GPE initialization has been split into two parts in order to defer execution of the _PRW methods (Power Resources for Wake) until after the hardware is fully initialized and the SCI handler is installed. This allows the _PRW methods to access fields protected by the Global Lock. This will fix systems where a NO_GLOBAL_LOCK exception has been seen during initialization. Converted the ACPI internal object disassemble and display code within the AML debugger to fully table-driven operation, reducing code size and increasing maintainability. Fixed a regression with the ConcatenateResTemplate() ASL operator introduced in the 20051021 release. Implemented support for "local" internal ACPI object types within the debugger "Object" command and the AcpiWalkNamespace external interfaces. These local types include RegionFields, BankFields, IndexFields, Alias, and reference objects. Moved common AML resource handling code into a new file, "utresrc.c". This code is shared by both the Resource Manager and the AML Debugger. Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 76.1K Code, 12.2K Data, 88.3K Total Debug Version: 163.5K Code, 67.0K Data, 230.5K Total Current Release: Non-Debug Version: 76.2K Code, 12.3K Data, 88.5K Total Debug Version: 163.0K Code, 67.4K Data, 230.4K Total 2) iASL Compiler/Disassembler: Fixed a problem with very large initializer lists (more than 4000 elements) for both Buffer and Package objects where the parse stack could overflow. Enhanced the pre-compile source code scan for non-ASCII characters to ignore characters within comment fields. The scan is now always performed and is no longer optional, detecting invalid characters within a source file immediately rather than during the parse phase or later. Enhanced the ASL grammar definition to force early reductions on all list- style grammar elements so that the overall parse stack usage is greatly reduced. This should improve performance and reduce the possibility of parse stack overflow. Eliminated all reduce/reduce conflicts in the iASL parser generation. Also, with the addition of a %expected statement, the compiler generates from source with no warnings. Fixed a possible segment fault in the disassembler if the input filename does not contain a "dot" extension (Thomas Renninger). ---------------------------------------- 21 October 2005. Summary of changes for version 20051021: 1) ACPI CA Core Subsystem: Implemented support for the EM64T and other x86-64 processors. This essentially entails recognizing that these processors support non-aligned memory transfers. Previously, all 64-bit processors were assumed to lack hardware support for non-aligned transfers. Completed conversion of the Resource Manager to nearly full table-driven operation. Specifically, the resource conversion code (convert AML to internal format and the reverse) and the debug code to dump internal resource descriptors are fully table-driven, reducing code and data size and improving maintainability. The OSL interfaces for Acquire and Release Lock now use a 64-bit flag word on 64-bit processors instead of a fixed 32-bit word. (With assistance from Alexey Starikovskiy) Implemented support within the resource conversion code for the Type- Specific byte within the various ACPI 3.0 *WordSpace macros. Fixed some issues within the resource conversion code for the type- specific flags for both Memory and I/O address resource descriptors. For Memory, implemented support for the MTP and TTP flags. For I/O, split the TRS and TTP flags into two separate fields. Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.1K Code, 12.1K Data, 89.2K Total Debug Version: 168.0K Code, 68.3K Data, 236.3K Total Current Release: Non-Debug Version: 76.1K Code, 12.2K Data, 88.3K Total Debug Version: 163.5K Code, 67.0K Data, 230.5K Total 2) iASL Compiler/Disassembler: Relaxed a compiler restriction that disallowed a ResourceIndex byte if the corresponding ResourceSource string was not also present in a resource descriptor declaration. This restriction caused problems with existing AML/ASL code that includes the Index byte without the string. When such AML was disassembled, it could not be compiled without modification. Further, the modified code created a resource template with a different size than the original, breaking code that used fixed offsets into the resource template buffer. Removed a recent feature of the disassembler to ignore a lone ResourceIndex byte. This byte is now emitted if present so that the exact AML can be reproduced when the disassembled code is recompiled. Improved comments and text alignment for the resource descriptor code emitted by the disassembler. Implemented disassembler support for the ACPI 3.0 AccessSize field within a Register() resource descriptor. ---------------------------------------- 30 September 2005. Summary of changes for version 20050930: 1) ACPI CA Core Subsystem: Completed a major overhaul of the Resource Manager code - specifically, optimizations in the area of the AML/internal resource conversion code. The code has been optimized to simplify and eliminate duplicated code, CPU stack use has been decreased by optimizing function parameters and local variables, and naming conventions across the manager have been standardized for clarity and ease of maintenance (this includes function, parameter, variable, and struct/typedef names.) The update may force changes in some driver code, depending on how resources are handled by the host OS. All Resource Manager dispatch and information tables have been moved to a single location for clarity and ease of maintenance. One new file was created, named "rsinfo.c". The ACPI return macros (return_ACPI_STATUS, etc.) have been modified to guarantee that the argument is not evaluated twice, making them less prone to macro side-effects. However, since there exists the possibility of additional stack use if a particular compiler cannot optimize them (such as in the debug generation case), the original macros are optionally available. Note that some invocations of the return_VALUE macro may now cause size mismatch warnings; the return_UINT8 and return_UINT32 macros are provided to eliminate these. (From Randy Dunlap) Implemented a new mechanism to enable debug tracing for individual control methods. A new external interface, AcpiDebugTrace, is provided to enable this mechanism. The intent is to allow the host OS to easily enable and disable tracing for problematic control methods. This interface can be easily exposed to a user or debugger interface if desired. See the file psxface.c for details. AcpiUtCallocate will now return a valid pointer if a length of zero is specified - a length of one is used and a warning is issued. This matches the behavior of AcpiUtAllocate. Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.5K Code, 12.0K Data, 89.5K Total Debug Version: 168.1K Code, 68.4K Data, 236.5K Total Current Release: Non-Debug Version: 77.1K Code, 12.1K Data, 89.2K Total Debug Version: 168.0K Code, 68.3K Data, 236.3K Total 2) iASL Compiler/Disassembler: A remark is issued if the effective compile-time length of a package or buffer is zero. Previously, this was a warning. ---------------------------------------- 16 September 2005. Summary of changes for version 20050916: 1) ACPI CA Core Subsystem: Fixed a problem within the Resource Manager where support for the Generic Register descriptor was not fully implemented. This descriptor is now fully recognized, parsed, disassembled, and displayed. Completely restructured the Resource Manager code to utilize table-driven dispatch and lookup, eliminating many of the large switch() statements. This reduces overall subsystem code size and code complexity. Affects the resource parsing and construction, disassembly, and debug dump output. Cleaned up and restructured the debug dump output for all resource descriptors. Improved readability of the output and reduced code size. Fixed a problem where changes to internal data structures caused the optional ACPI_MUTEX_DEBUG code to fail compilation if specified. Code and Data Size: The current and previous library sizes for the core subsystem are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 32-bit compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.4K Code, 11.8K Data, 90.2K Total Debug Version: 169.6K Code, 69.9K Data, 239.5K Total Current Release: Non-Debug Version: 77.5K Code, 12.0K Data, 89.5K Total Debug Version: 168.1K Code, 68.4K Data, 236.5K Total 2) iASL Compiler/Disassembler: Updated the disassembler to automatically insert an EndDependentFn() macro into the ASL stream if this macro is missing in the original AML code, simplifying compilation of the resulting ASL module. Fixed a problem in the disassembler where a disassembled ResourceSource string (within a large resource descriptor) was not surrounded by quotes and not followed by a comma, causing errors when the resulting ASL module was compiled. Also, escape sequences within a ResourceSource string are now handled correctly (especially "\\") ---------------------------------------- 02 September 2005. Summary of changes for version 20050902: 1) ACPI CA Core Subsystem: Fixed a problem with the internal Owner ID allocation and deallocation mechanisms for control method execution and recursive method invocation. This should eliminate the OWNER_ID_LIMIT exceptions and "Invalid OwnerId" messages seen on some systems. Recursive method invocation depth is currently limited to 255. (Alexey Starikovskiy) Completely eliminated all vestiges of support for the "module-level executable code" until this support is fully implemented and debugged. This should eliminate the NO_RETURN_VALUE exceptions seen during table load on some systems that invoke this support. Fixed a problem within the resource manager code where the transaction flags for a 64-bit address descriptor were handled incorrectly in the type- specific flag byte. Consolidated duplicate code within the address descriptor resource manager code, reducing overall subsystem code size. Fixed a fault when using the AML debugger "disassemble" command to disassemble individual control methods. Removed references to the "release_current" directory within the Unix release package. Code and Data Size: The current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler. These values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.6K Code, 11.7K Data, 90.3K Total Debug Version: 170.0K Code, 69.9K Data, 239.9K Total Current Release: Non-Debug Version: 78.4K Code, 11.8K Data, 90.2K Total Debug Version: 169.6K Code, 69.9K Data, 239.5K Total 2) iASL Compiler/Disassembler: Implemented an error check for illegal duplicate values in the interrupt and dma lists for the following ASL macros: Dma(), Irq(), IrqNoFlags(), and Interrupt(). Implemented error checking for the Irq() and IrqNoFlags() macros to detect too many values in the interrupt list (16 max) and invalid values in the list (range 0 - 15) The maximum length string literal within an ASL file is now restricted to 200 characters as per the ACPI specification. Fixed a fault when using the -ln option (generate namespace listing). Implemented an error check to determine if a DescriptorName within a resource descriptor has already been used within the current scope. ---------------------------------------- 15 August 2005. Summary of changes for version 20050815: 1) ACPI CA Core Subsystem: Implemented a full bytewise compare to determine if a table load request is attempting to load a duplicate table. The compare is performed if the table signatures and table lengths match. This will allow different tables with the same OEM Table ID and revision to be loaded - probably against the ACPI specification, but discovered in the field nonetheless. Added the changes.txt logfile to each of the zipped release packages. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.6K Code, 11.7K Data, 90.3K Total Debug Version: 167.0K Code, 69.9K Data, 236.9K Total Current Release: Non-Debug Version: 78.6K Code, 11.7K Data, 90.3K Total Debug Version: 170.0K Code, 69.9K Data, 239.9K Total 2) iASL Compiler/Disassembler: Fixed a problem where incorrect AML code could be generated for Package objects if optimization is disabled (via the -oa switch). Fixed a problem with where incorrect AML code is generated for variable- length packages when the package length is not specified and the number of initializer values is greater than 255. ---------------------------------------- 29 July 2005. Summary of changes for version 20050729: 1) ACPI CA Core Subsystem: Implemented support to ignore an attempt to install/load a particular ACPI table more than once. Apparently there exists BIOS code that repeatedly attempts to load the same SSDT upon certain events. With assistance from Venkatesh Pallipadi. Restructured the main interface to the AML parser in order to correctly handle all exceptional conditions. This will prevent leakage of the OwnerId resource and should eliminate the AE_OWNER_ID_LIMIT exceptions seen on some machines. With assistance from Alexey Starikovskiy. Support for "module level code" has been disabled in this version due to a number of issues that have appeared on various machines. The support can be enabled by defining ACPI_ENABLE_MODULE_LEVEL_CODE during subsystem compilation. When the issues are fully resolved, the code will be enabled by default again. Modified the internal functions for debug print support to define the FunctionName parameter as a (const char *) for compatibility with compiler built-in macros such as __FUNCTION__, etc. Linted the entire ACPICA source tree for both 32-bit and 64-bit. Implemented support to display an object count summary for the AML Debugger commands Object and Methods. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.6K Code, 11.6K Data, 90.2K Total Debug Version: 170.0K Code, 69.7K Data, 239.7K Total Current Release: Non-Debug Version: 78.6K Code, 11.7K Data, 90.3K Total Debug Version: 167.0K Code, 69.9K Data, 236.9K Total 2) iASL Compiler/Disassembler: Fixed a regression that appeared in the 20050708 version of the compiler where an error message was inadvertently emitted for invocations of the _OSI reserved control method. ---------------------------------------- 08 July 2005. Summary of changes for version 20050708: 1) ACPI CA Core Subsystem: The use of the CPU stack in the debug version of the subsystem has been considerably reduced. Previously, a debug structure was declared in every function that used the debug macros. This structure has been removed in favor of declaring the individual elements as parameters to the debug functions. This reduces the cumulative stack use during nested execution of ACPI function calls at the cost of a small increase in the code size of the debug version of the subsystem. With assistance from Alexey Starikovskiy and Len Brown. Added the ACPI_GET_FUNCTION_NAME macro to enable the compiler-dependent headers to define a macro that will return the current function name at runtime (such as __FUNCTION__ or _func_, etc.) The function name is used by the debug trace output. If ACPI_GET_FUNCTION_NAME is not defined in the compiler-dependent header, the function name is saved on the CPU stack (one pointer per function.) This mechanism is used because apparently there exists no standard ANSI-C defined macro that that returns the function name. Redesigned and reimplemented the "Owner ID" mechanism used to track namespace objects created/deleted by ACPI tables and control method execution. A bitmap is now used to allocate and free the IDs, thus solving the wraparound problem present in the previous implementation. The size of the namespace node descriptor was reduced by 2 bytes as a result (Alexey Starikovskiy). Removed the UINT32_BIT and UINT16_BIT types that were used for the bitfield flag definitions within the headers for the predefined ACPI tables. These have been replaced by UINT8_BIT in order to increase the code portability of the subsystem. If the use of UINT8 remains a problem, we may be forced to eliminate bitfields entirely because of a lack of portability. Enhanced the performance of the AcpiUtUpdateObjectReference procedure. This is a frequently used function and this improvement increases the performance of the entire subsystem (Alexey Starikovskiy). Fixed several possible memory leaks and the inverse - premature object deletion (Alexey Starikovskiy). Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.6K Code, 11.5K Data, 90.1K Total Debug Version: 165.2K Code, 69.6K Data, 234.8K Total Current Release: Non-Debug Version: 78.6K Code, 11.6K Data, 90.2K Total Debug Version: 170.0K Code, 69.7K Data, 239.7K Total ---------------------------------------- 24 June 2005. Summary of changes for version 20050624: 1) ACPI CA Core Subsystem: Modified the new OSL cache interfaces to use ACPI_CACHE_T as the type for the host-defined cache object. This allows the OSL implementation to define and type this object in any manner desired, simplifying the OSL implementation. For example, ACPI_CACHE_T is defined as kmem_cache_t for Linux, and should be defined in the OS-specific header file for other operating systems as required. Changed the interface to AcpiOsAcquireObject to directly return the requested object as the function return (instead of ACPI_STATUS.) This change was made for performance reasons, since this is the purpose of the interface in the first place. AcpiOsAcquireObject is now similar to the AcpiOsAllocate interface. Implemented a new AML debugger command named Businfo. This command displays information about all devices that have an associate _PRT object. The _ADR, _HID, _UID, and _CID are displayed for these devices. Modified the initialization sequence in AcpiInitializeSubsystem to call the OSL interface AcpiOslInitialize first, before any local initialization. This change was required because the global initialization now calls OSL interfaces. Enhanced the Dump command to display the entire contents of Package objects (including all sub-objects and their values.) Restructured the code base to split some files because of size and/or because the code logically belonged in a separate file. New files are listed below. All makefiles and project files included in the ACPI CA release have been updated. utilities/utcache.c /* Local cache interfaces */ utilities/utmutex.c /* Local mutex support */ utilities/utstate.c /* State object support */ interpreter/parser/psloop.c /* Main AML parse loop */ Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.3K Code, 11.6K Data, 89.9K Total Debug Version: 164.0K Code, 69.1K Data, 233.1K Total Current Release: Non-Debug Version: 78.6K Code, 11.5K Data, 90.1K Total Debug Version: 165.2K Code, 69.6K Data, 234.8K Total 2) iASL Compiler/Disassembler: Fixed a regression introduced in version 20050513 where the use of a Package object within a Case() statement caused a compile time exception. The original behavior has been restored (a Match() operator is emitted.) ---------------------------------------- 17 June 2005. Summary of changes for version 20050617: 1) ACPI CA Core Subsystem: Moved the object cache operations into the OS interface layer (OSL) to allow the host OS to handle these operations if desired (for example, the Linux OSL will invoke the slab allocator). This support is optional; the compile time define ACPI_USE_LOCAL_CACHE may be used to utilize the original cache code in the ACPI CA core. The new OSL interfaces are shown below. See utalloc.c for an example implementation, and acpiosxf.h for the exact interface definitions. With assistance from Alexey Starikovskiy. AcpiOsCreateCache AcpiOsDeleteCache AcpiOsPurgeCache AcpiOsAcquireObject AcpiOsReleaseObject Modified the interfaces to AcpiOsAcquireLock and AcpiOsReleaseLock to return and restore a flags parameter. This fits better with many OS lock models. Note: the current execution state (interrupt handler or not) is no longer passed to these interfaces. If necessary, the OSL must determine this state by itself, a simple and fast operation. With assistance from Alexey Starikovskiy. Fixed a problem in the ACPI table handling where a valid XSDT was assumed present if the revision of the RSDP was 2 or greater. According to the ACPI specification, the XSDT is optional in all cases, and the table manager therefore now checks for both an RSDP >=2 and a valid XSDT pointer. Otherwise, the RSDT pointer is used. Some ACPI 2.0 compliant BIOSs contain only the RSDT. Fixed an interpreter problem with the Mid() operator in the case of an input string where the resulting output string is of zero length. It now correctly returns a valid, null terminated string object instead of a string object with a null pointer. Fixed a problem with the control method argument handling to allow a store to an Arg object that already contains an object of type Device. The Device object is now correctly overwritten. Previously, an error was returned. Enhanced the debugger Find command to emit object values in addition to the found object pathnames. The output format is the same as the dump namespace command. Enhanced the debugger Set command. It now has the ability to set the value of any Named integer object in the namespace (Previously, only method locals and args could be set.) Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.1K Code, 11.6K Data, 89.7K Total Debug Version: 164.0K Code, 69.3K Data, 233.3K Total Current Release: Non-Debug Version: 78.3K Code, 11.6K Data, 89.9K Total Debug Version: 164.0K Code, 69.1K Data, 233.1K Total 2) iASL Compiler/Disassembler: Fixed a regression in the disassembler where if/else/while constructs were output incorrectly. This problem was introduced in the previous release (20050526). This problem also affected the single-step disassembly in the debugger. Fixed a problem where compiling the reserved _OSI method would randomly (but rarely) produce compile errors. Enhanced the disassembler to emit compilable code in the face of incorrect AML resource descriptors. If the optional ResourceSourceIndex is present, but the ResourceSource is not, do not emit the ResourceSourceIndex in the disassembly. Otherwise, the resulting code cannot be compiled without errors. ---------------------------------------- 26 May 2005. Summary of changes for version 20050526: 1) ACPI CA Core Subsystem: Implemented support to execute Type 1 and Type 2 AML opcodes appearing at the module level (not within a control method.) These opcodes are executed exactly once at the time the table is loaded. This type of code was legal up until the release of ACPI 2.0B (2002) and is now supported within ACPI CA in order to provide backwards compatibility with earlier BIOS implementations. This eliminates the "Encountered executable code at module level" warning that was previously generated upon detection of such code. Fixed a problem in the interpreter where an AE_NOT_FOUND exception could inadvertently be generated during the lookup of namespace objects in the second pass parse of ACPI tables and control methods. It appears that this problem could occur during the resolution of forward references to namespace objects. Added the ACPI_MUTEX_DEBUG #ifdef to the AcpiUtReleaseMutex function, corresponding to the same #ifdef in the AcpiUtAcquireMutex function. This allows the deadlock detection debug code to be compiled out in the normal case, improving mutex performance (and overall subsystem performance) considerably. Implemented a handful of miscellaneous fixes for possible memory leaks on error conditions and error handling control paths. These fixes were suggested by FreeBSD and the Coverity Prevent source code analysis tool. Added a check for a null RSDT pointer in AcpiGetFirmwareTable (tbxfroot.c) to prevent a fault in this error case. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.2K Code, 11.6K Data, 89.8K Total Debug Version: 163.7K Code, 69.3K Data, 233.0K Total Current Release: Non-Debug Version: 78.1K Code, 11.6K Data, 89.7K Total Debug Version: 164.0K Code, 69.3K Data, 233.3K Total 2) iASL Compiler/Disassembler: Implemented support to allow Type 1 and Type 2 ASL operators to appear at the module level (not within a control method.) These operators will be executed once at the time the table is loaded. This type of code was legal up until the release of ACPI 2.0B (2002) and is now supported by the iASL compiler in order to provide backwards compatibility with earlier BIOS ASL code. The ACPI integer width (specified via the table revision ID or the -r override, 32 or 64 bits) is now used internally during compile-time constant folding to ensure that constants are truncated to 32 bits if necessary. Previously, the revision ID value was only emitted in the AML table header. An error message is now generated for the Mutex and Method operators if the SyncLevel parameter is outside the legal range of 0 through 15. Fixed a problem with the Method operator ParameterTypes list handling (ACPI 3.0). Previously, more than 2 types or 2 arguments generated a syntax error. The actual underlying implementation of method argument typechecking is still under development, however. ---------------------------------------- 13 May 2005. Summary of changes for version 20050513: 1) ACPI CA Core Subsystem: Implemented support for PCI Express root bridges -- added support for device PNP0A08 in the root bridge search within AcpiEvPciConfigRegionSetup. The interpreter now automatically truncates incoming 64-bit constants to 32 bits if currently executing out of a 32-bit ACPI table (Revision < 2). This also affects the iASL compiler constant folding. (Note: as per below, the iASL compiler no longer allows 64-bit constants within 32-bit tables.) Fixed a problem where string and buffer objects with "static" pointers (pointers to initialization data within an ACPI table) were not handled consistently. The internal object copy operation now always copies the data to a newly allocated buffer, regardless of whether the source object is static or not. Fixed a problem with the FromBCD operator where an implicit result conversion was improperly performed while storing the result to the target operand. Since this is an "explicit conversion" operator, the implicit conversion should never be performed on the output. Fixed a problem with the CopyObject operator where a copy to an existing named object did not always completely overwrite the existing object stored at name. Specifically, a buffer-to-buffer copy did not delete the existing buffer. Replaced "InterruptLevel" with "InterruptNumber" in all GPE interfaces and structs for consistency. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.2K Code, 11.6K Data, 89.8K Total Debug Version: 163.7K Code, 69.3K Data, 233.0K Total Current Release: (Same sizes) Non-Debug Version: 78.2K Code, 11.6K Data, 89.8K Total Debug Version: 163.7K Code, 69.3K Data, 233.0K Total 2) iASL Compiler/Disassembler: The compiler now emits a warning if an attempt is made to generate a 64- bit integer constant from within a 32-bit ACPI table (Revision < 2). The integer is truncated to 32 bits. Fixed a problem with large package objects: if the static length of the package is greater than 255, the "variable length package" opcode is emitted. Previously, this caused an error. This requires an update to the ACPI spec, since it currently (incorrectly) states that packages larger than 255 elements are not allowed. The disassembler now correctly handles variable length packages and packages larger than 255 elements. ---------------------------------------- 08 April 2005. Summary of changes for version 20050408: 1) ACPI CA Core Subsystem: Fixed three cases in the interpreter where an "index" argument to an ASL function was still (internally) 32 bits instead of the required 64 bits. This was the Index argument to the Index, Mid, and Match operators. The "strupr" function is now permanently local (AcpiUtStrupr), since this is not a POSIX-defined function and not present in most kernel-level C libraries. All references to the C library strupr function have been removed from the headers. Completed the deployment of static functions/prototypes. All prototypes with the static attribute have been moved from the headers to the owning C file. Implemented an extract option (-e) for the AcpiBin utility (AML binary utility). This option allows the utility to extract individual ACPI tables from the output of AcpiDmp. It provides the same functionality of the acpixtract.pl perl script without the worry of setting the correct perl options. AcpiBin runs on Windows and has not yet been generated/validated in the Linux/Unix environment (but should be soon). Updated and fixed the table dump option for AcpiBin (-d). This option converts a single ACPI table to a hex/ascii file, similar to the output of AcpiDmp. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.0K Code, 11.6K Data, 89.6K Total Debug Version: 163.5K Code, 69.3K Data, 232.8K Total Current Release: Non-Debug Version: 78.2K Code, 11.6K Data, 89.8K Total Debug Version: 163.7K Code, 69.3K Data, 233.0K Total 2) iASL Compiler/Disassembler: Disassembler fix: Added a check to ensure that the table length found in the ACPI table header within the input file is not longer than the actual input file size. This indicates some kind of file or table corruption. ---------------------------------------- 29 March 2005. Summary of changes for version 20050329: 1) ACPI CA Core Subsystem: An error is now generated if an attempt is made to create a Buffer Field of length zero (A CreateField with a length operand of zero.) The interpreter now issues a warning whenever executable code at the module level is detected during ACPI table load. This will give some idea of the prevalence of this type of code. Implemented support for references to named objects (other than control methods) within package objects. Enhanced package object output for the debug object. Package objects are now completely dumped, showing all elements. Enhanced miscellaneous object output for the debug object. Any object can now be written to the debug object (for example, a device object can be written, and the type of the object will be displayed.) The "static" qualifier has been added to all local functions across both the core subsystem and the iASL compiler. The number of "long" lines (> 80 chars) within the source has been significantly reduced, by about 1/3. Cleaned up all header files to ensure that all CA/iASL functions are prototyped (even static functions) and the formatting is consistent. Two new header files have been added, acopcode.h and acnames.h. Removed several obsolete functions that were no longer used. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.3K Code, 11.5K Data, 89.8K Total Debug Version: 165.4K Code, 69.7K Data, 236.1K Total Current Release: Non-Debug Version: 78.0K Code, 11.6K Data, 89.6K Total Debug Version: 163.5K Code, 69.3K Data, 232.8K Total 2) iASL Compiler/Disassembler: Fixed a problem with the resource descriptor generation/support. For the ResourceSourceIndex and the ResourceSource fields, both must be present, or both must be not present - can't have one without the other. The compiler now returns non-zero from the main procedure if any errors have occurred during the compilation. ---------------------------------------- 09 March 2005. Summary of changes for version 20050309: 1) ACPI CA Core Subsystem: The string-to-buffer implicit conversion code has been modified again after a change to the ACPI specification. In order to match the behavior of the other major ACPI implementation, the target buffer is no longer truncated if the source string is smaller than an existing target buffer. This change requires an update to the ACPI spec, and should eliminate the recent AE_AML_BUFFER_LIMIT issues. The "implicit return" support was rewritten to a new algorithm that solves the general case. Rather than attempt to determine when a method is about to exit, the result of every ASL operator is saved momentarily until the very next ASL operator is executed. Therefore, no matter how the method exits, there will always be a saved implicit return value. This feature is only enabled with the AcpiGbl_EnableInterpreterSlack flag, and should eliminate AE_AML_NO_RETURN_VALUE errors when enabled. Implemented implicit conversion support for the predicate (operand) of the If, Else, and While operators. String and Buffer arguments are automatically converted to Integers. Changed the string-to-integer conversion behavior to match the new ACPI errata: "If no integer object exists, a new integer is created. The ASCII string is interpreted as a hexadecimal constant. Each string character is interpreted as a hexadecimal value ('0'-'9', 'A'-'F', 'a', 'f'), starting with the first character as the most significant digit, and ending with the first non-hexadecimal character or end-of-string." This means that the first non-hex character terminates the conversion and this is the code that was changed. Fixed a problem where the ObjectType operator would fail (fault) when used on an Index of a Package which pointed to a null package element. The operator now properly returns zero (Uninitialized) in this case. Fixed a problem where the While operator used excessive memory by not properly popping the result stack during execution. There was no memory leak after execution, however. (Code provided by Valery Podrezov.) Fixed a problem where references to control methods within Package objects caused the method to be invoked, instead of producing a reference object pointing to the method. Restructured and simplified the pswalk.c module (AcpiPsDeleteParseTree) to improve performance and reduce code size. (Code provided by Alexey Starikovskiy.) Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.3K Code, 11.5K Data, 89.8K Total Debug Version: 165.4K Code, 69.6K Data, 236.0K Total Current Release: Non-Debug Version: 78.3K Code, 11.5K Data, 89.8K Total Debug Version: 165.4K Code, 69.7K Data, 236.1K Total 2) iASL Compiler/Disassembler: Fixed a problem with the Return operator with no arguments. Since the AML grammar for the byte encoding requires an operand for the Return opcode, the compiler now emits a Return(Zero) for this case. An ACPI specification update has been written for this case. For tables other than the DSDT, namepath optimization is automatically disabled. This is because SSDTs can be loaded anywhere in the namespace, the compiler has no knowledge of where, and thus cannot optimize namepaths. Added "ProcessorObj" to the ObjectTypeKeyword list. This object type was inadvertently omitted from the ACPI specification, and will require an update to the spec. The source file scan for ASCII characters is now optional (-a). This change was made because some vendors place non-ascii characters within comments. However, the scan is simply a brute-force byte compare to ensure all characters in the file are in the range 0x00 to 0x7F. Fixed a problem with the CondRefOf operator where the compiler was inappropriately checking for the existence of the target. Since the point of the operator is to check for the existence of the target at run-time, the compiler no longer checks for the target existence. Fixed a problem where errors generated from the internal AML interpreter during constant folding were not handled properly, causing a fault. Fixed a problem with overly aggressive range checking for the Stall operator. The valid range (max 255) is now only checked if the operand is of type Integer. All other operand types cannot be statically checked. Fixed a problem where control method references within the RefOf, DeRefOf, and ObjectType operators were not treated properly. They are now treated as actual references, not method invocations. Fixed and enhanced the "list namespace" option (-ln). This option was broken a number of releases ago. Improved error handling for the Field, IndexField, and BankField operators. The compiler now cleanly reports and recovers from errors in the field component (FieldUnit) list. Fixed a disassembler problem where the optional ResourceDescriptor fields TRS and TTP were not always handled correctly. Disassembler - Comments in output now use "//" instead of "/*" ---------------------------------------- 28 February 2005. Summary of changes for version 20050228: 1) ACPI CA Core Subsystem: Fixed a problem where the result of an Index() operator (an object reference) must increment the reference count on the target object for the life of the object reference. Implemented AML Interpreter and Debugger support for the new ACPI 3.0 Extended Address (IO, Memory, Space), QwordSpace, DwordSpace, and WordSpace resource descriptors. Implemented support in the _OSI method for the ACPI 3.0 "Extended Address Space Descriptor" string, indicating interpreter support for the descriptors above. Implemented header support for the new ACPI 3.0 FADT flag bits. Implemented header support for the new ACPI 3.0 PCI Express bits for the PM1 status/enable registers. Updated header support for the MADT processor local Apic struct and MADT platform interrupt source struct for new ACPI 3.0 fields. Implemented header support for the SRAT and SLIT ACPI tables. Implemented the -s switch in AcpiExec to enable the "InterpreterSlack" flag at runtime. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.2K Code, 11.5K Data, 89.7K Total Debug Version: 164.9K Code, 69.2K Data, 234.1K Total Current Release: Non-Debug Version: 78.3K Code, 11.5K Data, 89.8K Total Debug Version: 165.4K Code, 69.6K Data, 236.0K Total 2) iASL Compiler/Disassembler: Fixed a problem with the internal 64-bit String-to-integer conversion with strings less than two characters long. Fixed a problem with constant folding where the result of the Index() operator can not be considered a constant. This means that Index() cannot be a type3 opcode and this will require an update to the ACPI specification. Disassembler: Implemented support for the TTP, MTP, and TRS resource descriptor fields. These fields were inadvertently ignored and not output in the disassembly of the resource descriptor. ---------------------------------------- 11 February 2005. Summary of changes for version 20050211: 1) ACPI CA Core Subsystem: Implemented ACPI 3.0 support for implicit conversion within the Match() operator. MatchObjects can now be of type integer, buffer, or string instead of just type integer. Package elements are implicitly converted to the type of the MatchObject. This change aligns the behavior of Match() with the behavior of the other logical operators (LLess(), etc.) It also requires an errata change to the ACPI specification as this support was intended for ACPI 3.0, but was inadvertently omitted. Fixed a problem with the internal implicit "to buffer" conversion. Strings that are converted to buffers will cause buffer truncation if the string is smaller than the target buffer. Integers that are converted to buffers will not cause buffer truncation, only zero extension (both as per the ACPI spec.) The problem was introduced when code was added to truncate the buffer, but this should not be performed in all cases, only the string case. Fixed a problem with the Buffer and Package operators where the interpreter would get confused if two such operators were used as operands to an ASL operator (such as LLess(Buffer(1){0},Buffer(1){1}). The internal result stack was not being popped after the execution of these operators, resulting in an AE_NO_RETURN_VALUE exception. Fixed a problem with constructs of the form Store(Index(...),...). The reference object returned from Index was inadvertently resolved to an actual value. This problem was introduced in version 20050114 when the behavior of Store() was modified to restrict the object types that can be used as the source operand (to match the ACPI specification.) Reduced excessive stack use within the AcpiGetObjectInfo procedure. Added a fix to aclinux.h to allow generation of AcpiExec on Linux. Updated the AcpiSrc utility to add the FADT_DESCRIPTOR_REV2_MINUS struct. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.1K Code, 11.5K Data, 89.6K Total Debug Version: 164.8K Code, 69.2K Data, 234.0K Total Current Release: Non-Debug Version: 78.2K Code, 11.5K Data, 89.7K Total Debug Version: 164.9K Code, 69.2K Data, 234.1K Total 2) iASL Compiler/Disassembler: Fixed a code generation problem in the constant folding optimization code where incorrect code was generated if a constant was reduced to a buffer object (i.e., a reduced type 5 opcode.) Fixed a typechecking problem for the ToBuffer operator. Caused by an incorrect return type in the internal opcode information table. ---------------------------------------- 25 January 2005. Summary of changes for version 20050125: 1) ACPI CA Core Subsystem: Fixed a recently introduced problem with the Global Lock where the underlying semaphore was not created. This problem was introduced in version 20050114, and caused an AE_AML_NO_OPERAND exception during an Acquire() operation on _GL. The local object cache is now optional, and is disabled by default. Both AcpiExec and the iASL compiler enable the cache because they run in user mode and this enhances their performance. #define ACPI_ENABLE_OBJECT_CACHE to enable the local cache. Fixed an issue in the internal function AcpiUtEvaluateObject concerning the optional "implicit return" support where an error was returned if no return object was expected, but one was implicitly returned. AE_OK is now returned in this case and the implicitly returned object is deleted. AcpiUtEvaluateObject is only occasionally used, and only to execute reserved methods such as _STA and _INI where the return type is known up front. Fixed a few issues with the internal convert-to-integer code. It now returns an error if an attempt is made to convert a null string, a string of only blanks/tabs, or a zero-length buffer. This affects both implicit conversion and explicit conversion via the ToInteger() operator. The internal debug code in AcpiUtAcquireMutex has been commented out. It is not needed for normal operation and should increase the performance of the entire subsystem. The code remains in case it is needed for debug purposes again. The AcpiExec source and makefile are included in the Unix/Linux package for the first time. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.4K Code, 11.5K Data, 89.9K Total Debug Version: 165.4K Code, 69.4K Data, 234.8K Total Current Release: Non-Debug Version: 78.1K Code, 11.5K Data, 89.6K Total Debug Version: 164.8K Code, 69.2K Data, 234.0K Total 2) iASL Compiler/Disassembler: Switch/Case support: A warning is now issued if the type of the Switch value cannot be determined at compile time. For example, Switch(Arg0) will generate the warning, and the type is assumed to be an integer. As per the ACPI spec, use a construct such as Switch(ToInteger(Arg0)) to eliminate the warning. Switch/Case support: Implemented support for buffer and string objects as the switch value. This is an ACPI 3.0 feature, now that LEqual supports buffers and strings. Switch/Case support: The emitted code for the LEqual() comparisons now uses the switch value as the first operand, not the second. The case value is now the second operand, and this allows the case value to be implicitly converted to the type of the switch value, not the other way around. Switch/Case support: Temporary variables are now emitted immediately within the control method, not at the global level. This means that there are now 36 temps available per-method, not 36 temps per-module as was the case with the earlier implementation (_T_0 through _T_9 and _T_A through _T_Z.) ---------------------------------------- 14 January 2005. Summary of changes for version 20050114: Added 2005 copyright to all module headers. This affects every module in the core subsystem, iASL compiler, and the utilities. 1) ACPI CA Core Subsystem: Fixed an issue with the String-to-Buffer conversion code where the string null terminator was not included in the buffer after conversion, but there is existing ASL that assumes the string null terminator is included. This is the root of the ACPI_AML_BUFFER_LIMIT regression. This problem was introduced in the previous version when the code was updated to correctly set the converted buffer size as per the ACPI specification. The ACPI spec is ambiguous and will be updated to specify that the null terminator must be included in the converted buffer. This also affects the ToBuffer() ASL operator. Fixed a problem with the Mid() ASL/AML operator where it did not work correctly on Buffer objects. Newly created sub-buffers were not being marked as initialized. Fixed a problem in AcpiTbFindTable where incorrect string compares were performed on the OemId and OemTableId table header fields. These fields are not null terminated, so strncmp is now used instead of strcmp. Implemented a restriction on the Store() ASL/AML operator to align the behavior with the ACPI specification. Previously, any object could be used as the source operand. Now, the only objects that may be used are Integers, Buffers, Strings, Packages, Object References, and DDB Handles. If necessary, the original behavior can be restored by enabling the EnableInterpreterSlack flag. Enhanced the optional "implicit return" support to allow an implicit return value from methods that are invoked externally via the AcpiEvaluateObject interface. This enables implicit returns from the _STA and _INI methods, for example. Changed the Revision() ASL/AML operator to return the current version of the AML interpreter, in the YYYYMMDD format. Previously, it incorrectly returned the supported ACPI version (This is the function of the _REV method). Updated the _REV predefined method to return the currently supported version of ACPI, now 3. Implemented batch mode option for the AcpiExec utility (-b). Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.3K Code, 11.5K Data, 89.8K Total Debug Version: 165.3K Code, 69.4K Data, 234.7K Total Current Release: Non-Debug Version: 78.4K Code, 11.5K Data, 89.9K Total Debug Version: 165.4K Code, 69.4K Data, 234.8K Total ---------------------------------------- 10 December 2004. Summary of changes for version 20041210: ACPI 3.0 support is nearing completion in both the iASL compiler and the ACPI CA core subsystem. 1) ACPI CA Core Subsystem: Fixed a problem in the ToDecimalString operator where the resulting string length was incorrectly calculated. The length is now calculated exactly, eliminating incorrect AE_STRING_LIMIT exceptions. Fixed a problem in the ToHexString operator to allow a maximum 200 character string to be produced. Fixed a problem in the internal string-to-buffer and buffer-to-buffer copy routine where the length of the resulting buffer was not truncated to the new size (if the target buffer already existed). Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.3K Code, 11.5K Data, 89.8K Total Debug Version: 164.7K Code, 68.5K Data, 233.2K Total Current Release: Non-Debug Version: 78.3K Code, 11.5K Data, 89.8K Total Debug Version: 165.3K Code, 69.4K Data, 234.7K Total 2) iASL Compiler/Disassembler: Implemented the new ACPI 3.0 resource template macros - DWordSpace, ExtendedIO, ExtendedMemory, ExtendedSpace, QWordSpace, and WordSpace. Includes support in the disassembler. Implemented support for the new (ACPI 3.0) parameter to the Register macro, AccessSize. Fixed a problem where the _HE resource name for the Interrupt macro was referencing bit 0 instead of bit 1. Implemented check for maximum 255 interrupts in the Interrupt macro. Fixed a problem with the predefined resource descriptor names where incorrect AML code was generated if the offset within the resource buffer was 0 or 1. The optimizer shortened the AML code to a single byte opcode but did not update the surrounding package lengths. Changes to the Dma macro: All channels within the channel list must be in the range 0-7. Maximum 8 channels can be specified. BusMaster operand is optional (default is BusMaster). Implemented check for maximum 7 data bytes for the VendorShort macro. The ReadWrite parameter is now optional for the Memory32 and similar macros. ---------------------------------------- 03 December 2004. Summary of changes for version 20041203: 1) ACPI CA Core Subsystem: The low-level field insertion/extraction code (exfldio) has been completely rewritten to eliminate unnecessary complexity, bugs, and boundary conditions. Fixed a problem in the ToInteger, ToBuffer, ToHexString, and ToDecimalString operators where the input operand could be inadvertently deleted if no conversion was necessary (e.g., if the input to ToInteger was an Integer object.) Fixed a problem with the ToDecimalString and ToHexString where an incorrect exception code was returned if the resulting string would be > 200 chars. AE_STRING_LIMIT is now returned. Fixed a problem with the Concatenate operator where AE_OK was always returned, even if the operation failed. Fixed a problem in oswinxf (used by AcpiExec and iASL) to allow > 128 semaphores to be allocated. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.5K Code, 11.5K Data, 90.0K Total Debug Version: 165.2K Code, 68.6K Data, 233.8K Total Current Release: Non-Debug Version: 78.3K Code, 11.5K Data, 89.8K Total Debug Version: 164.7K Code, 68.5K Data, 233.2K Total 2) iASL Compiler/Disassembler: Fixed typechecking for the ObjectType and SizeOf operators. Problem was recently introduced in 20041119. Fixed a problem with the ToUUID macro where the upper nybble of each buffer byte was inadvertently set to zero. ---------------------------------------- 19 November 2004. Summary of changes for version 20041119: 1) ACPI CA Core Subsystem: Fixed a problem in the internal ConvertToInteger routine where new integers were not truncated to 32 bits for 32-bit ACPI tables. This routine converts buffers and strings to integers. Implemented support to store a value to an Index() on a String object. This is an ACPI 2.0 feature that had not yet been implemented. Implemented new behavior for storing objects to individual package elements (via the Index() operator). The previous behavior was to invoke the implicit conversion rules if an object was already present at the index. The new behavior is to simply delete any existing object and directly store the new object. Although the ACPI specification seems unclear on this subject, other ACPI implementations behave in this manner. (This is the root of the AE_BAD_HEX_CONSTANT issue.) Modified the RSDP memory scan mechanism to support the extended checksum for ACPI 2.0 (and above) RSDPs. Note that the search continues until a valid RSDP signature is found with a valid checksum. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.5K Code, 11.5K Data, 90.0K Total Debug Version: 165.2K Code, 68.6K Data, 233.8K Total Current Release: Non-Debug Version: 78.5K Code, 11.5K Data, 90.0K Total Debug Version: 165.2K Code, 68.6K Data, 233.8K Total 2) iASL Compiler/Disassembler: Fixed a missing semicolon in the aslcompiler.y file. ---------------------------------------- 05 November 2004. Summary of changes for version 20041105: 1) ACPI CA Core Subsystem: Implemented support for FADT revision 2. This was an interim table (between ACPI 1.0 and ACPI 2.0) that adds support for the FADT reset register. Implemented optional support to allow uninitialized LocalX and ArgX variables in a control method. The variables are initialized to an Integer object with a value of zero. This support is enabled by setting the AcpiGbl_EnableInterpreterSlack flag to TRUE. Implemented support for Integer objects for the SizeOf operator. Either 4 or 8 is returned, depending on the current integer size (32-bit or 64- bit, depending on the parent table revision). Fixed a problem in the implementation of the SizeOf and ObjectType operators where the operand was resolved to a value too early, causing incorrect return values for some objects. Fixed some possible memory leaks during exceptional conditions. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.0K Code, 11.5K Data, 89.5K Total Debug Version: 164.8K Code, 68.6K Data, 233.4K Total Current Release: Non-Debug Version: 78.5K Code, 11.5K Data, 90.0K Total Debug Version: 165.2K Code, 68.6K Data, 233.8K Total 2) iASL Compiler/Disassembler: Implemented support for all ACPI 3.0 reserved names and methods. Implemented all ACPI 3.0 grammar elements in the front-end, including support for semicolons. Implemented the ACPI 3.0 Function() and ToUUID() macros Fixed a problem in the disassembler where a Scope() operator would not be emitted properly if the target of the scope was in another table. ---------------------------------------- 15 October 2004. Summary of changes for version 20041015: Note: ACPI CA is currently undergoing an in-depth and complete formal evaluation to test/verify the following areas. Other suggestions are welcome. This will result in an increase in the frequency of releases and the number of bug fixes in the next few months. - Functional tests for all ASL/AML operators - All implicit/explicit type conversions - Bit fields and operation regions - 64-bit math support and 32-bit-only "truncated" math support - Exceptional conditions, both compiler and interpreter - Dynamic object deletion and memory leaks - ACPI 3.0 support when implemented - External interfaces to the ACPI subsystem 1) ACPI CA Core Subsystem: Fixed two alignment issues on 64-bit platforms - within debug statements in AcpiEvGpeDetect and AcpiEvCreateGpeBlock. Removed references to the Address field within the non-aligned ACPI generic address structure. Fixed a problem in the Increment and Decrement operators where incorrect operand resolution could result in the inadvertent modification of the original integer when the integer is passed into another method as an argument and the arg is then incremented/decremented. Fixed a problem in the FromBCD operator where the upper 32-bits of a 64- bit BCD number were truncated during conversion. Fixed a problem in the ToDecimal operator where the length of the resulting string could be set incorrectly too long if the input operand was a Buffer object. Fixed a problem in the Logical operators (LLess, etc.) where a NULL byte (0) within a buffer would prematurely terminate a compare between buffer objects. Added a check for string overflow (>200 characters as per the ACPI specification) during the Concatenate operator with two string operands. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.8K Code, 11.5K Data, 89.3K Total Debug Version: 164.6K Code, 68.5K Data, 233.1K Total Current Release: Non-Debug Version: 78.0K Code, 11.5K Data, 89.5K Total Debug Version: 164.8K Code, 68.6K Data, 233.4K Total 2) iASL Compiler/Disassembler: Allow the use of the ObjectType operator on uninitialized Locals and Args (returns 0 as per the ACPI specification). Fixed a problem where the compiler would fault if there was a syntax error in the FieldName of all of the various CreateXXXField operators. Disallow the use of lower case letters within the EISAID macro, as per the ACPI specification. All EISAID strings must be of the form "UUUNNNN" Where U is an uppercase letter and N is a hex digit. ---------------------------------------- 06 October 2004. Summary of changes for version 20041006: 1) ACPI CA Core Subsystem: Implemented support for the ACPI 3.0 Timer operator. This ASL function implements a 64-bit timer with 100 nanosecond granularity. Defined a new OSL interface, AcpiOsGetTimer. This interface is used to implement the ACPI 3.0 Timer operator. This allows the host OS to implement the timer with the best clock available. Also, it keeps the core subsystem out of the clock handling business, since the host OS (usually) performs this function. Fixed an alignment issue on 64-bit platforms. The HwLowLevelRead(Write) functions use a 64-bit address which is part of the packed ACPI Generic Address Structure. Since the structure is non-aligned, the alignment macros are now used to extract the address to a local variable before use. Fixed a problem where the ToInteger operator assumed all input strings were hexadecimal. The operator now handles both decimal strings and hex strings (prefixed with "0x"). Fixed a problem where the string length in the string object created as a result of the internal ConvertToString procedure could be incorrect. This potentially affected all implicit conversions and also the ToDecimalString and ToHexString operators. Fixed two problems in the ToString operator. If the length parameter was zero, an incorrect string object was created and the value of the input length parameter was inadvertently changed from zero to Ones. Fixed a problem where the optional ResourceSource string in the ExtendedIRQ resource macro was ignored. Simplified the interfaces to the internal division functions, reducing code size and complexity. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.9K Code, 11.4K Data, 89.3K Total Debug Version: 164.5K Code, 68.3K Data, 232.8K Total Current Release: Non-Debug Version: 77.8K Code, 11.5K Data, 89.3K Total Debug Version: 164.6K Code, 68.5K Data, 233.1K Total 2) iASL Compiler/Disassembler: Implemented support for the ACPI 3.0 Timer operator. Fixed a problem where the Default() operator was inadvertently ignored in a Switch/Case block. This was a problem in the translation of the Switch statement to If...Else pairs. Added support to allow a standalone Return operator, with no parentheses (or operands). Fixed a problem with code generation for the ElseIf operator where the translated Else...If parse tree was improperly constructed leading to the loss of some code. ---------------------------------------- 22 September 2004. Summary of changes for version 20040922: 1) ACPI CA Core Subsystem: Fixed a problem with the implementation of the LNot() operator where "Ones" was not returned for the TRUE case. Changed the code to return Ones instead of (!Arg) which was usually 1. This change affects iASL constant folding for this operator also. Fixed a problem in AcpiUtInitializeBuffer where an existing buffer was not initialized properly -- Now zero the entire buffer in this case where the buffer already exists. Changed the interface to AcpiOsSleep from (UINT32 Seconds, UINT32 Milliseconds) to simply (ACPI_INTEGER Milliseconds). This simplifies all related code considerably. This will require changes/updates to all OS interface layers (OSLs.) Implemented a new external interface, AcpiInstallExceptionHandler, to allow a system exception handler to be installed. This handler is invoked upon any run-time exception that occurs during control method execution. Added support for the DSDT in AcpiTbFindTable. This allows the DataTableRegion() operator to access the local copy of the DSDT. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.8K Code, 11.4K Data, 89.2K Total Debug Version: 164.2K Code, 68.2K Data, 232.4K Total Current Release: Non-Debug Version: 77.9K Code, 11.4K Data, 89.3K Total Debug Version: 164.5K Code, 68.3K Data, 232.8K Total 2) iASL Compiler/Disassembler: Fixed a problem with constant folding and the LNot operator. LNot was returning 1 in the TRUE case, not Ones as per the ACPI specification. This could result in the generation of an incorrect folded/reduced constant. End-Of-File is now allowed within a "//"-style comment. A parse error no longer occurs if such a comment is at the very end of the input ASL source file. Implemented the "-r" option to override the Revision in the table header. The initial use of this option will be to simplify the evaluation of the AML interpreter by allowing a single ASL source module to be compiled for either 32-bit or 64-bit integers. ---------------------------------------- 27 August 2004. Summary of changes for version 20040827: 1) ACPI CA Core Subsystem: - Implemented support for implicit object conversion in the non-numeric logical operators (LEqual, LGreater, LGreaterEqual, LLess, LLessEqual, and LNotEqual.) Any combination of Integers/Strings/Buffers may now be used; the second operand is implicitly converted on the fly to match the type of the first operand. For example: LEqual (Source1, Source2) Source1 and Source2 must each evaluate to an integer, a string, or a buffer. The data type of Source1 dictates the required type of Source2. Source2 is implicitly converted if necessary to match the type of Source1. - Updated and corrected the behavior of the string conversion support. The rules concerning conversion of buffers to strings (according to the ACPI specification) are as follows: ToDecimalString - explicit byte-wise conversion of buffer to string of decimal values (0-255) separated by commas. ToHexString - explicit byte- wise conversion of buffer to string of hex values (0-FF) separated by commas. ToString - explicit byte-wise conversion of buffer to string. Byte-by- byte copy with no transform except NULL terminated. Any other implicit buffer- to- string conversion - byte-wise conversion of buffer to string of hex values (0-FF) separated by spaces. - Fixed typo in definition of AcpiGbl_EnableInterpreterSlack. - Fixed a problem in AcpiNsGetPathnameLength where the returned length was one byte too short in the case of a node in the root scope. This could cause a fault during debug output. - Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.9K Code, 11.5K Data, 89.4K Total Debug Version: 164.1K Code, 68.3K Data, 232.4K Total Current Release: Non-Debug Version: 77.8K Code, 11.4K Data, 89.2K Total Debug Version: 164.2K Code, 68.2K Data, 232.4K Total 2) iASL Compiler/Disassembler: - Fixed a Linux generation error. ---------------------------------------- 16 August 2004. Summary of changes for version 20040816: 1) ACPI CA Core Subsystem: Designed and implemented support within the AML interpreter for the so- called "implicit return". This support returns the result of the last ASL operation within a control method, in the absence of an explicit Return() operator. A few machines depend on this behavior, even though it is not explicitly supported by the ASL language. It is optional support that can be enabled at runtime via the AcpiGbl_EnableInterpreterSlack flag. Removed support for the PCI_Config address space from the internal low level hardware interfaces (AcpiHwLowLevelRead and AcpiHwLowLevelWrite). This support was not used internally, and would not work correctly anyway because the PCI bus number and segment number were not supported. There are separate interfaces for PCI configuration space access because of the unique interface. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 78.0K Code, 11.5K Data, 89.5K Total Debug Version: 164.1K Code, 68.2K Data, 232.3K Total Current Release: Non-Debug Version: 77.9K Code, 11.5K Data, 89.4K Total Debug Version: 164.1K Code, 68.3K Data, 232.4K Total 2) iASL Compiler/Disassembler: Fixed a problem where constants in ASL expressions at the root level (not within a control method) could be inadvertently truncated during code generation. This problem was introduced in the 20040715 release. ---------------------------------------- 15 July 2004. Summary of changes for version 20040715: 1) ACPI CA Core Subsystem: Restructured the internal HW GPE interfaces to pass/track the current state of interrupts (enabled/disabled) in order to avoid possible deadlock and increase flexibility of the interfaces. Implemented a "lexicographical compare" for String and Buffer objects within the logical operators -- LGreater, LLess, LGreaterEqual, and LLessEqual - - as per further clarification to the ACPI specification. Behavior is similar to C library "strcmp". Completed a major reduction in CPU stack use for the AcpiGetFirmwareTable external function. In the 32-bit non-debug case, the stack use has been reduced from 168 bytes to 32 bytes. Deployed a new run-time configuration flag, AcpiGbl_EnableInterpreterSlack, whose purpose is to allow the AML interpreter to forgive certain bad AML constructs. Default setting is FALSE. Implemented the first use of AcpiGbl_EnableInterpreterSlack in the Field IO support code. If enabled, it allows field access to go beyond the end of a region definition if the field is within the region length rounded up to the next access width boundary (a common coding error.) Renamed OSD_HANDLER to ACPI_OSD_HANDLER, and OSD_EXECUTION_CALLBACK to ACPI_OSD_EXEC_CALLBACK for consistency with other ACPI symbols. Also, these symbols are lowercase by the latest version of the AcpiSrc tool. The prototypes for the PCI interfaces in acpiosxf.h have been updated to rename "Register" to simply "Reg" to prevent certain compilers from complaining. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.8K Code, 11.5K Data, 89.3K Total Debug Version: 163.8K Code, 68.2K Data, 232.0K Total Current Release: Non-Debug Version: 78.0K Code, 11.5K Data, 89.5K Total Debug Version: 164.1K Code, 68.2K Data, 232.3K Total 2) iASL Compiler/Disassembler: Implemented full support for Package objects within the Case() operator. Note: The Break() operator is currently not supported within Case blocks (TermLists) as there is some question about backward compatibility with ACPI 1.0 interpreters. Fixed a problem where complex terms were not supported properly within the Switch() operator. Eliminated extraneous warning for compiler-emitted reserved names of the form "_T_x". (Used in Switch/Case operators.) Eliminated optimization messages for "_T_x" objects and small constants within the DefinitionBlock operator. ---------------------------------------- 15 June 2004. Summary of changes for version 20040615: 1) ACPI CA Core Subsystem: Implemented support for Buffer and String objects (as per ACPI 2.0) for the following ASL operators: LEqual, LGreater, LLess, LGreaterEqual, and LLessEqual. All directory names in the entire source package are lower case, as they were in earlier releases. Implemented "Disassemble" command in the AML debugger that will disassemble a single control method. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.7K Code, 11.5K Data, 89.2K Total Debug Version: 163.3K Code, 67.2K Data, 230.5K Total Current Release: Non-Debug Version: 77.8K Code, 11.5K Data, 89.3K Total Debug Version: 163.8K Code, 68.2K Data, 232.0K Total 2) iASL Compiler/Disassembler: Implemented support for Buffer and String objects (as per ACPI 2.0) for the following ASL operators: LEqual, LGreater, LLess, LGreaterEqual, and LLessEqual. All directory names in the entire source package are lower case, as they were in earlier releases. Fixed a fault when using the -g or -d options if the FADT was not found. Fixed an issue with the Windows version of the compiler where later versions of Windows place the FADT in the registry under the name "FADT" and not "FACP" as earlier versions did. This applies when using the -g or - d options. The compiler now looks for both strings as necessary. Fixed a problem with compiler namepath optimization where a namepath within the Scope() operator could not be optimized if the namepath was a subpath of the current scope path. ---------------------------------------- 27 May 2004. Summary of changes for version 20040527: 1) ACPI CA Core Subsystem: Completed a new design and implementation for EBDA (Extended BIOS Data Area) support in the RSDP scan code. The original code improperly scanned for the EBDA by simply scanning from memory location 0 to 0x400. The correct method is to first obtain the EBDA pointer from within the BIOS data area, then scan 1K of memory starting at the EBDA pointer. There appear to be few if any machines that place the RSDP in the EBDA, however. Integrated a fix for a possible fault during evaluation of BufferField arguments. Obsolete code that was causing the problem was removed. Found and fixed a problem in the Field Support Code where data could be corrupted on a bit field read that starts on an aligned boundary but does not end on an aligned boundary. Merged the read/write "datum length" calculation code into a common procedure. Rolled in a couple of changes to the FreeBSD-specific header. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.6K Code, 11.5K Data, 89.1K Total Debug Version: 163.2K Code, 67.2K Data, 230.4K Total Current Release: Non-Debug Version: 77.7K Code, 11.5K Data, 89.2K Total Debug Version: 163.3K Code, 67.2K Data, 230.5K Total 2) iASL Compiler/Disassembler: Fixed a generation warning produced by some overly-verbose compilers for a 64-bit constant. ---------------------------------------- 14 May 2004. Summary of changes for version 20040514: 1) ACPI CA Core Subsystem: Fixed a problem where hardware GPE enable bits sometimes not set properly during and after GPE method execution. Result of 04/27 changes. Removed extra "clear all GPEs" when sleeping/waking. Removed AcpiHwEnableGpe and AcpiHwDisableGpe, replaced by the single AcpiHwWriteGpeEnableReg. Changed a couple of calls to the functions above to the new AcpiEv* calls as appropriate. ACPI_OS_NAME was removed from the OS-specific headers. The default name is now "Microsoft Windows NT" for maximum compatibility. However this can be changed by modifying the acconfig.h file. Allow a single invocation of AcpiInstallNotifyHandler for a handler that traps both types of notifies (System, Device). Use ACPI_ALL_NOTIFY flag. Run _INI methods on ThermalZone objects. This is against the ACPI specification, but there is apparently ASL code in the field that has these _INI methods, and apparently "other" AML interpreters execute them. Performed a full 16/32/64 bit lint that resulted in some small changes. Added a sleep simulation command to the AML debugger to test sleep code. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.6K Code, 11.5K Data, 89.1K Total Debug Version: 162.9K Code, 67.0K Data, 229.9K Total Current Release: Non-Debug Version: 77.6K Code, 11.5K Data, 89.1K Total Debug Version: 163.2K Code, 67.2K Data, 230.4K Total ---------------------------------------- 27 April 2004. Summary of changes for version 20040427: 1) ACPI CA Core Subsystem: Completed a major overhaul of the GPE handling within ACPI CA. There are now three types of GPEs: wake-only, runtime-only, and combination wake/run. The only GPEs allowed to be combination wake/run are for button-style devices such as a control-method power button, control-method sleep button, or a notebook lid switch. GPEs that have an _Lxx or _Exx method and are not referenced by any _PRW methods are marked for "runtime" and hardware enabled. Any GPE that is referenced by a _PRW method is marked for "wake" (and disabled at runtime). However, at sleep time, only those GPEs that have been specifically enabled for wake via the AcpiEnableGpe interface will actually be hardware enabled. A new external interface has been added, AcpiSetGpeType(), that is meant to be used by device drivers to force a GPE to a particular type. It will be especially useful for the drivers for the button devices mentioned above. Completed restructuring of the ACPI CA initialization sequence so that default operation region handlers are installed before GPEs are initialized and the _PRW methods are executed. This will prevent errors when the _PRW methods attempt to access system memory or I/O space. GPE enable/disable no longer reads the GPE enable register. We now keep the enable info for runtime and wake separate and in the GPE_EVENT_INFO. We thus no longer depend on the hardware to maintain these bits. Always clear the wake status and fixed/GPE status bits before sleep, even for state S5. Improved the AML debugger output for displaying the GPE blocks and their current status. Added new strings for the _OSI method, of the form "Windows 2001 SPx" where x = 0,1,2,3,4. Fixed a problem where the physical address was incorrectly calculated when the Load() operator was used to directly load from an Operation Region (vs. loading from a Field object.) Also added check for minimum table length for this case. Fix for multiple mutex acquisition. Restore original thread SyncLevel on mutex release. Added ACPI_VALID_SXDS flag to the AcpiGetObjectInfo interface for consistency with the other fields returned. Shrunk the ACPI_GPE_EVENT_INFO structure by 40%. There is one such structure for each GPE in the system, so the size of this structure is important. CPU stack requirement reduction: Cleaned up the method execution and object evaluation paths so that now a parameter structure is passed, instead of copying the various method parameters over and over again. In evregion.c: Correctly exit and reenter the interpreter region if and only if dispatching an operation region request to a user-installed handler. Do not exit/reenter when dispatching to a default handler (e.g., default system memory or I/O handlers) Notes for updating drivers for the new GPE support. The following changes must be made to ACPI-related device drivers that are attached to one or more GPEs: (This information will be added to the ACPI CA Programmer Reference.) 1) AcpiInstallGpeHandler no longer automatically enables the GPE, you must explicitly call AcpiEnableGpe. 2) There is a new interface called AcpiSetGpeType. This should be called before enabling the GPE. Also, this interface will automatically disable the GPE if it is currently enabled. 3) AcpiEnableGpe no longer supports a GPE type flag. Specific drivers that must be changed: 1) EC driver: AcpiInstallGpeHandler (NULL, GpeNum, ACPI_GPE_EDGE_TRIGGERED, AeGpeHandler, NULL); AcpiSetGpeType (NULL, GpeNum, ACPI_GPE_TYPE_RUNTIME); AcpiEnableGpe (NULL, GpeNum, ACPI_NOT_ISR); 2) Button Drivers (Power, Lid, Sleep): Run _PRW method under parent device If _PRW exists: /* This is a control-method button */ Extract GPE number and possibly GpeDevice AcpiSetGpeType (GpeDevice, GpeNum, ACPI_GPE_TYPE_WAKE_RUN); AcpiEnableGpe (GpeDevice, GpeNum, ACPI_NOT_ISR); For all other devices that have _PRWs, we automatically set the GPE type to ACPI_GPE_TYPE_WAKE, but the GPE is NOT automatically (wake) enabled. This must be done on a selective basis, usually requiring some kind of user app to allow the user to pick the wake devices. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 77.0K Code, 11.4K Data, 88.4K Total Debug Version: 161.0K Code, 66.3K Data, 227.3K Total Current Release: Non-Debug Version: 77.6K Code, 11.5K Data, 89.1K Total Debug Version: 162.9K Code, 67.0K Data, 229.9K Total ---------------------------------------- 02 April 2004. Summary of changes for version 20040402: 1) ACPI CA Core Subsystem: Fixed an interpreter problem where an indirect store through an ArgX parameter was incorrectly applying the "implicit conversion rules" during the store. From the ACPI specification: "If the target is a method local or argument (LocalX or ArgX), no conversion is performed and the result is stored directly to the target". The new behavior is to disable implicit conversion during ALL stores to an ArgX. Changed the behavior of the _PRW method scan to ignore any and all errors returned by a given _PRW. This prevents the scan from aborting from the failure of any single _PRW. Moved the runtime configuration parameters from the global init procedure to static variables in acglobal.h. This will allow the host to override the default values easily. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 76.9K Code, 11.4K Data, 88.3K Total Debug Version: 160.8K Code, 66.1K Data, 226.9K Total Current Release: Non-Debug Version: 77.0K Code, 11.4K Data, 88.4K Total Debug Version: 161.0K Code, 66.3K Data, 227.3K Total 2) iASL Compiler/Disassembler: iASL now fully disassembles SSDTs. However, External() statements are not generated automatically for unresolved symbols at this time. This is a planned feature for future implementation. Fixed a scoping problem in the disassembler that occurs when the type of the target of a Scope() operator is overridden. This problem caused an incorrectly nested internal namespace to be constructed. Any warnings or errors that are emitted during disassembly are now commented out automatically so that the resulting file can be recompiled without any hand editing. ---------------------------------------- 26 March 2004. Summary of changes for version 20040326: 1) ACPI CA Core Subsystem: Implemented support for "wake" GPEs via interaction between GPEs and the _PRW methods. Every GPE that is pointed to by one or more _PRWs is identified as a WAKE GPE and by default will no longer be enabled at runtime. Previously, we were blindly enabling all GPEs with a corresponding _Lxx or _Exx method - but most of these turn out to be WAKE GPEs anyway. We believe this has been the cause of thousands of "spurious" GPEs on some systems. This new GPE behavior is can be reverted to the original behavior (enable ALL GPEs at runtime) via a runtime flag. Fixed a problem where aliased control methods could not access objects properly. The proper scope within the namespace was not initialized (transferred to the target of the aliased method) before executing the target method. Fixed a potential race condition on internal object deletion on the return object in AcpiEvaluateObject. Integrated a fix for resource descriptors where both _MEM and _MTP were being extracted instead of just _MEM. (i.e. bitmask was incorrectly too wide, 0x0F instead of 0x03.) Added a special case for ACPI_ROOT_OBJECT in AcpiUtGetNodeName, preventing a fault in some cases. Updated Notify() values for debug statements in evmisc.c Return proper status from AcpiUtMutexInitialize, not just simply AE_OK. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 76.5K Code, 11.3K Data, 87.8K Total Debug Version: 160.3K Code, 66.0K Data, 226.3K Total Current Release: Non-Debug Version: 76.9K Code, 11.4K Data, 88.3K Total Debug Version: 160.8K Code, 66.1K Data, 226.9K Total ---------------------------------------- 11 March 2004. Summary of changes for version 20040311: 1) ACPI CA Core Subsystem: Fixed a problem where errors occurring during the parse phase of control method execution did not abort cleanly. For example, objects created and installed in the namespace were not deleted. This caused all subsequent invocations of the method to return the AE_ALREADY_EXISTS exception. Implemented a mechanism to force a control method to "Serialized" execution if the method attempts to create namespace objects. (The root of the AE_ALREADY_EXISTS problem.) Implemented support for the predefined _OSI "internal" control method. Initial supported strings are "Linux", "Windows 2000", "Windows 2001", and "Windows 2001.1", and can be easily upgraded for new strings as necessary. This feature will allow "other" operating systems to execute the fully tested, "Windows" code path through the ASL code Global Lock Support: Now allows multiple acquires and releases with any internal thread. Removed concept of "owning thread" for this special mutex. Fixed two functions that were inappropriately declaring large objects on the CPU stack: PsParseLoop, NsEvaluateRelative. Reduces the stack usage during method execution considerably. Fixed a problem in the ACPI 2.0 FACS descriptor (actbl2.h) where the S4Bios_f field was incorrectly defined as UINT32 instead of UINT32_BIT. Fixed a problem where AcpiEvGpeDetect would fault if there were no GPEs defined on the machine. Implemented two runtime options: One to force all control method execution to "Serialized" to mimic Windows behavior, another to disable _OSI support if it causes problems on a given machine. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 74.8K Code, 10.1K Data, 84.9K Total Debug Version: 158.7K Code, 65.1K Data, 223.8K Total Current Release: Non-Debug Version: 76.5K Code, 11.3K Data, 87.8K Total Debug Version: 160.3K Code, 66.0K Data, 226.3K Total 2) iASL Compiler/Disassembler: Fixed an array size problem for FreeBSD that would cause the compiler to fault. ---------------------------------------- 20 February 2004. Summary of changes for version 20040220: 1) ACPI CA Core Subsystem: Implemented execution of _SxD methods for Device objects in the GetObjectInfo interface. Fixed calls to _SST method to pass the correct arguments. Added a call to _SST on wake to restore to "working" state. Check for End-Of-Buffer failure case in the WalkResources interface. Integrated fix for 64-bit alignment issue in acglobal.h by moving two structures to the beginning of the file. After wake, clear GPE status register(s) before enabling GPEs. After wake, clear/enable power button. (Perhaps we should clear/enable all fixed events upon wake.) Fixed a couple of possible memory leaks in the Namespace manager. Integrated latest acnetbsd.h file. ---------------------------------------- 11 February 2004. Summary of changes for version 20040211: 1) ACPI CA Core Subsystem: Completed investigation and implementation of the call-by-reference mechanism for control method arguments. Fixed a problem where a store of an object into an indexed package could fail if the store occurs within a different method than the method that created the package. Fixed a problem where the ToDecimal operator could return incorrect results. Fixed a problem where the CopyObject operator could fail on some of the more obscure objects (e.g., Reference objects.) Improved the output of the Debug object to display buffer, package, and index objects. Fixed a problem where constructs of the form "RefOf (ArgX)" did not return the expected result. Added permanent ACPI_REPORT_ERROR macros for all instances of the ACPI_AML_INTERNAL exception. Integrated latest version of acfreebsd.h ---------------------------------------- 16 January 2004. Summary of changes for version 20040116: The purpose of this release is primarily to update the copyright years in each module, thus causing a huge number of diffs. There are a few small functional changes, however. 1) ACPI CA Core Subsystem: Improved error messages when there is a problem finding one or more of the required base ACPI tables Reintroduced the definition of APIC_HEADER in actbl.h Changed definition of MADT_ADDRESS_OVERRIDE to 64 bits (actbl.h) Removed extraneous reference to NewObj in dsmthdat.c 2) iASL compiler Fixed a problem introduced in December that disabled the correct disassembly of Resource Templates ---------------------------------------- 03 December 2003. Summary of changes for version 20031203: 1) ACPI CA Core Subsystem: Changed the initialization of Operation Regions during subsystem init to perform two entire walks of the ACPI namespace; The first to initialize the regions themselves, the second to execute the _REG methods. This fixed some interdependencies across _REG methods found on some machines. Fixed a problem where a Store(Local0, Local1) could simply update the object reference count, and not create a new copy of the object if the Local1 is uninitialized. Implemented support for the _SST reserved method during sleep transitions. Implemented support to clear the SLP_TYP and SLP_EN bits when waking up, this is apparently required by some machines. When sleeping, clear the wake status only if SleepState is not S5. Fixed a problem in AcpiRsExtendedIrqResource() where an incorrect pointer arithmetic advanced a string pointer too far. Fixed a problem in AcpiTbGetTablePtr() where a garbage pointer could be returned if the requested table has not been loaded. Within the support for IRQ resources, restructured the handling of the active and edge/level bits. Fixed a few problems in AcpiPsxExecute() where memory could be leaked under certain error conditions. Improved error messages for the cases where the ACPI mode could not be entered. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (20031029): Non-Debug Version: 74.4K Code, 10.1K Data, 84.5K Total Debug Version: 158.3K Code, 65.0K Data, 223.3K Total Current Release: Non-Debug Version: 74.8K Code, 10.1K Data, 84.9K Total Debug Version: 158.7K Code, 65.1K Data, 223.8K Total 2) iASL Compiler/Disassembler: Implemented a fix for the iASL disassembler where a bad index was generated. This was most noticeable on 64-bit platforms ---------------------------------------- 29 October 2003. Summary of changes for version 20031029: 1) ACPI CA Core Subsystem: Fixed a problem where a level-triggered GPE with an associated _Lxx control method was incorrectly cleared twice. Fixed a problem with the Field support code where an access can occur beyond the end-of-region if the field is non-aligned but extends to the very end of the parent region (resulted in an AE_AML_REGION_LIMIT exception.) Fixed a problem with ACPI Fixed Events where an RT Clock handler would not get invoked on an RTC event. The RTC event bitmasks for the PM1 registers were not being initialized properly. Implemented support for executing _STA and _INI methods for Processor objects. Although this is currently not part of the ACPI specification, there is existing ASL code that depends on the init-time execution of these methods. Implemented and deployed a GetDescriptorName function to decode the various types of internal descriptors. Guards against null descriptors during debug output also. Implemented and deployed a GetNodeName function to extract the 4- character namespace node name. This function simplifies the debug and error output, as well as guarding against null pointers during output. Implemented and deployed the ACPI_FORMAT_UINT64 helper macro to simplify the debug and error output of 64-bit integers. This macro replaces the HIDWORD and LODWORD macros for dumping these integers. Updated the implementation of the Stall() operator to only call AcpiOsStall(), and also return an error if the operand is larger than 255. This preserves the required behavior of not relinquishing the processor, as would happen if AcpiOsSleep() was called for "long stalls". Constructs of the form "Store(LocalX,LocalX)" where LocalX is not initialized are now treated as NOOPs. Cleaned up a handful of warnings during 64-bit generation. Fixed a reported error where and incorrect GPE number was passed to the GPE dispatch handler. This value is only used for error output, however. Used this opportunity to clean up and streamline the GPE dispatch code. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (20031002): Non-Debug Version: 74.1K Code, 9.7K Data, 83.8K Total Debug Version: 157.9K Code, 64.8K Data, 222.7K Total Current Release: Non-Debug Version: 74.4K Code, 10.1K Data, 84.5K Total Debug Version: 158.3K Code, 65.0K Data, 223.3K Total 2) iASL Compiler/Disassembler: Updated the iASL compiler to return an error if the operand to the Stall() operator is larger than 255. ---------------------------------------- 02 October 2003. Summary of changes for version 20031002: 1) ACPI CA Core Subsystem: Fixed a problem with Index Fields where the index was not incremented for fields that require multiple writes to the index/data registers (Fields that are wider than the data register.) Fixed a problem with all Field objects where a write could go beyond the end-of-field if the field was larger than the access granularity and therefore required multiple writes to complete the request. An extra write beyond the end of the field could happen inadvertently. Fixed a problem with Index Fields where a BUFFER_OVERFLOW error would incorrectly be returned if the width of the Data Register was larger than the specified field access width. Completed fixes for LoadTable() and Unload() and verified their operation. Implemented full support for the "DdbHandle" object throughout the ACPI CA subsystem. Implemented full support for the MADT and ECDT tables in the ACPI CA header files. Even though these tables are not directly consumed by ACPI CA, the header definitions are useful for ACPI device drivers. Integrated resource descriptor fixes posted to the Linux ACPI list. This included checks for minimum descriptor length, and support for trailing NULL strings within descriptors that have optional string elements. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (20030918): Non-Debug Version: 73.9K Code, 9.7K Data, 83.6K Total Debug Version: 157.3K Code, 64.5K Data, 221.8K Total Current Release: Non-Debug Version: 74.1K Code, 9.7K Data, 83.8K Total Debug Version: 157.9K Code, 64.8K Data, 222.7K Total 2) iASL Compiler: Implemented detection of non-ASCII characters within the input source ASL file. This catches attempts to compile binary (AML) files early in the compile, with an informative error message. Fixed a problem where the disassembler would fault if the output filename could not be generated or if the output file could not be opened. ---------------------------------------- 18 September 2003. Summary of changes for version 20030918: 1) ACPI CA Core Subsystem: Found and fixed a longstanding problem with the late execution of the various deferred AML opcodes (such as Operation Regions, Buffer Fields, Buffers, and Packages). If the name string specified for the name of the new object placed the object in a scope other than the current scope, the initialization/execution of the opcode failed. The solution to this problem was to implement a mechanism where the late execution of such opcodes does not attempt to lookup/create the name a second time in an incorrect scope. This fixes the "region size computed incorrectly" problem. Fixed a call to AcpiHwRegisterWrite in hwregs.c that was causing a Global Lock AE_BAD_PARAMETER error. Fixed several 64-bit issues with prototypes, casting and data types. Removed duplicate prototype from acdisasm.h Fixed an issue involving EC Operation Region Detach (Shaohua Li) Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release: Non-Debug Version: 73.7K Code, 9.7K Data, 83.4K Total Debug Version: 156.9K Code, 64.2K Data, 221.1K Total Current Release: Non-Debug Version: 73.9K Code, 9.7K Data, 83.6K Total Debug Version: 157.3K Code, 64.5K Data, 221.8K Total 2) Linux: Fixed the AcpiOsSleep implementation in osunixxf.c to pass the correct sleep time in seconds. ---------------------------------------- 14 July 2003. Summary of changes for version 20030619: 1) ACPI CA Core Subsystem: Parse SSDTs in order discovered, as opposed to reverse order (Hrvoje Habjanic) Fixes from FreeBSD and NetBSD. (Frank van der Linden, Thomas Klausner, Nate Lawson) 2) Linux: Dynamically allocate SDT list (suggested by Andi Kleen) proc function return value cleanups (Andi Kleen) Correctly handle NMI watchdog during long stalls (Andrew Morton) Make it so acpismp=force works (reported by Andrew Morton) ---------------------------------------- 19 June 2003. Summary of changes for version 20030619: 1) ACPI CA Core Subsystem: Fix To/FromBCD, eliminating the need for an arch-specific #define. Do not acquire a semaphore in the S5 shutdown path. Fix ex_digits_needed for 0. (Takayoshi Kochi) Fix sleep/stall code reversal. (Andi Kleen) Revert a change having to do with control method calling semantics. 2) Linux: acpiphp update (Takayoshi Kochi) Export acpi_disabled for sonypi (Stelian Pop) Mention acpismp=force in config help Re-add acpitable.c and acpismp=force. This improves backwards compatibility and also cleans up the code to a significant degree. Add ASUS Value-add driver (Karol Kozimor and Julien Lerouge) ---------------------------------------- 22 May 2003. Summary of changes for version 20030522: 1) ACPI CA Core Subsystem: Found and fixed a reported problem where an AE_NOT_FOUND error occurred occasionally during _BST evaluation. This turned out to be an Owner ID allocation issue where a called method did not get a new ID assigned to it. Eventually, (after 64k calls), the Owner ID UINT16 would wraparound so that the ID would be the same as the caller's and the called method would delete the caller's namespace. Implemented extended error reporting for control methods that are aborted due to a run-time exception. Output includes the exact AML instruction that caused the method abort, a dump of the method locals and arguments at the time of the abort, and a trace of all nested control method calls. Modified the interpreter to allow the creation of buffers of zero length from the AML code. Implemented new code to ensure that no attempt is made to actually allocate a memory buffer (of length zero) - instead, a simple buffer object with a NULL buffer pointer and length zero is created. A warning is no longer issued when the AML attempts to create a zero-length buffer. Implemented a workaround for the "leading asterisk issue" in _HIDs, _UIDs, and _CIDs in the AML interpreter. One leading asterisk is automatically removed if present in any HID, UID, or CID strings. The iASL compiler will still flag this asterisk as an error, however. Implemented full support for _CID methods that return a package of multiple CIDs (Compatible IDs). The AcpiGetObjectInfo() interface now additionally returns a device _CID list if present. This required a change to the external interface in order to pass an ACPI_BUFFER object as a parameter since the _CID list is of variable length. Fixed a problem with the new AE_SAME_HANDLER exception where handler initialization code did not know about this exception. Code and Data Size: Current and previous core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (20030509): Non-Debug Version: 73.4K Code, 9.7K Data, 83.1K Total Debug Version: 156.1K Code, 63.9K Data, 220.0K Total Current Release: Non-Debug Version: 73.7K Code, 9.7K Data, 83.4K Total Debug Version: 156.9K Code, 64.2K Data, 221.1K Total 2) Linux: Fixed a bug in which we would reinitialize the ACPI interrupt after it was already working, thus disabling all ACPI and the IRQs for any other device sharing the interrupt. (Thanks to Stian Jordet) Toshiba driver update (John Belmonte) Return only 0 or 1 for our interrupt handler status (Andrew Morton) 3) iASL Compiler: Fixed a reported problem where multiple (nested) ElseIf() statements were not handled correctly by the compiler, resulting in incorrect warnings and incorrect AML code. This was a problem in both the ASL parser and the code generator. 4) Documentation: Added changes to existing interfaces, new exception codes, and new text concerning reference count object management versus garbage collection. ---------------------------------------- 09 May 2003. Summary of changes for version 20030509. 1) ACPI CA Core Subsystem: Changed the subsystem initialization sequence to hold off installation of address space handlers until the hardware has been initialized and the system has entered ACPI mode. This is because the installation of space handlers can cause _REG methods to be run. Previously, the _REG methods could potentially be run before ACPI mode was enabled. Fixed some memory leak issues related to address space handler and notify handler installation. There were some problems with the reference count mechanism caused by the fact that the handler objects are shared across several namespace objects. Fixed a reported problem where reference counts within the namespace were not properly updated when named objects created by method execution were deleted. Fixed a reported problem where multiple SSDTs caused a deletion issue during subsystem termination. Restructured the table data structures to simplify the linked lists and the related code. Fixed a problem where the table ID associated with secondary tables (SSDTs) was not being propagated into the namespace objects created by those tables. This would only present a problem for tables that are unloaded at run-time, however. Updated AcpiOsReadable and AcpiOsWritable to use the ACPI_SIZE type as the length parameter (instead of UINT32). Solved a long-standing problem where an ALREADY_EXISTS error appears on various systems. This problem could happen when there are multiple PCI_Config operation regions under a single PCI root bus. This doesn't happen very frequently, but there are some systems that do this in the ASL. Fixed a reported problem where the internal DeleteNode function was incorrectly handling the case where a namespace node was the first in the parent's child list, and had additional peers (not the only child, but first in the list of children.) Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 73.7K Code, 9.5K Data, 83.2K Total Debug Version: 156.1K Code, 63.6K Data, 219.7K Total Current Release: Non-Debug Version: 73.4K Code, 9.7K Data, 83.1K Total Debug Version: 156.1K Code, 63.9K Data, 220.0K Total 2) Linux: Allow ":" in OS override string (Ducrot Bruno) Kobject fix (Greg KH) 3 iASL Compiler/Disassembler: Fixed a problem in the generation of the C source code files (AML is emitted in C source statements for BIOS inclusion) where the Ascii dump that appears within a C comment at the end of each line could cause a compile time error if the AML sequence happens to have an open comment or close comment sequence embedded. ---------------------------------------- 24 April 2003. Summary of changes for version 20030424. 1) ACPI CA Core Subsystem: Support for big-endian systems has been implemented. Most of the support has been invisibly added behind big-endian versions of the ACPI_MOVE_* macros. Fixed a problem in AcpiHwDisableGpeBlock() and AcpiHwClearGpeBlock() where an incorrect offset was passed to the low level hardware write routine. The offset parameter was actually eliminated from the low level read/write routines because they had become obsolete. Fixed a problem where a handler object was deleted twice during the removal of a fixed event handler. 2) Linux: A fix for SMP systems with link devices was contributed by Compaq's Dan Zink. (2.5) Return whether we handled the interrupt in our IRQ handler. (Linux ISRs no longer return void, so we can propagate the handler return value from the ACPI CA core back to the OS.) 3) Documentation: The ACPI CA Programmer Reference has been updated to reflect new interfaces and changes to existing interfaces. ---------------------------------------- 28 March 2003. Summary of changes for version 20030328. 1) ACPI CA Core Subsystem: The GPE Block Device support has been completed. New interfaces are AcpiInstallGpeBlock and AcpiRemoveGpeBlock. The Event interfaces (enable, disable, clear, getstatus) have been split into separate interfaces for Fixed Events and General Purpose Events (GPEs) in order to support GPE Block Devices properly. Fixed a problem where the error message "Failed to acquire semaphore" would appear during operations on the embedded controller (EC). Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 72.3K Code, 9.5K Data, 81.8K Total Debug Version: 154.0K Code, 63.4K Data, 217.4K Total Current Release: Non-Debug Version: 73.7K Code, 9.5K Data, 83.2K Total Debug Version: 156.1K Code, 63.6K Data, 219.7K Total ---------------------------------------- 28 February 2003. Summary of changes for version 20030228. 1) ACPI CA Core Subsystem: The GPE handling and dispatch code has been completely overhauled in preparation for support of GPE Block Devices (ID ACPI0006). This affects internal data structures and code only; there should be no differences visible externally. One new file has been added, evgpeblk.c The FADT fields GPE0_BLK_LEN and GPE1_BLK_LEN are now the only fields that are used to determine the GPE block lengths. The REGISTER_BIT_WIDTH field of the X_GPEx_BLK extended address structures are ignored. This is per the ACPI specification but it isn't very clear. The full 256 Block 0/1 GPEs are now supported (the use of REGISTER_BIT_WIDTH limited the number of GPEs to 128). In the SCI interrupt handler, removed the read of the PM1_CONTROL register to look at the SCI_EN bit. On some machines, this read causes an SMI event and greatly slows down SCI events. (This may in fact be the cause of slow battery status response on some systems.) Fixed a problem where a store of a NULL string to a package object could cause the premature deletion of the object. This was seen during execution of the battery _BIF method on some systems, resulting in no battery data being returned. Added AcpiWalkResources interface to simplify parsing of resource lists. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 72.0K Code, 9.5K Data, 81.5K Total Debug Version: 153.0K Code, 62.9K Data, 215.9K Total Current Release: Non-Debug Version: 72.3K Code, 9.5K Data, 81.8K Total Debug Version: 154.0K Code, 63.4K Data, 217.4K Total 2) Linux S3 fixes (Ole Rohne) Update ACPI PHP driver with to use new acpi_walk_resource API (Bjorn Helgaas) Add S4BIOS support (Pavel Machek) Map in entire table before performing checksum (John Stultz) Expand the mem= cmdline to allow the specification of reserved and ACPI DATA blocks (Pavel Machek) Never use ACPI on VISWS Fix derive_pci_id (Ducrot Bruno, Alvaro Lopez) Revert a change that allowed P_BLK lengths to be 4 or 5. This is causing us to think that some systems support C2 when they really don't. Do not count processor objects for non-present CPUs (Thanks to Dominik Brodowski) 3) iASL Compiler: Fixed a problem where ASL include files could not be found and opened. Added support for the _PDC reserved name. ---------------------------------------- 22 January 2003. Summary of changes for version 20030122. 1) ACPI CA Core Subsystem: Added a check for constructs of the form: Store (Local0, Local0) where Local0 is not initialized. Apparently, some BIOS programmers believe that this is a NOOP. Since this store doesn't do anything anyway, the new prototype behavior will ignore this error. This is a case where we can relax the strict checking in the interpreter in the name of compatibility. 2) Linux The AcpiSrc Source Conversion Utility has been released with the Linux package for the first time. This is the utility that is used to convert the ACPI CA base source code to the Linux version. (Both) Handle P_BLK lengths shorter than 6 more gracefully (Both) Move more headers to include/acpi, and delete an unused header. (Both) Move drivers/acpi/include directory to include/acpi (Both) Boot functions don't use cmdline, so don't pass it around (Both) Remove include of unused header (Adrian Bunk) (Both) acpiphp.h includes both linux/acpi.h and acpi_bus.h. Since the former now also includes the latter, acpiphp.h only needs the one, now. (2.5) Make it possible to select method of bios restoring after S3 resume. [=> no more ugly ifdefs] (Pavel Machek) (2.5) Make proc write interfaces work (Pavel Machek) (2.5) Properly init/clean up in cpufreq/acpi (Dominik Brodowski) (2.5) Break out ACPI Perf code into its own module, under cpufreq (Dominik Brodowski) (2.4) S4BIOS support (Ducrot Bruno) (2.4) Fix acpiphp_glue.c for latest ACPI struct changes (Sergio Visinoni) 3) iASL Compiler: Added support to disassemble SSDT and PSDTs. Implemented support to obtain SSDTs from the Windows registry if available. ---------------------------------------- 09 January 2003. Summary of changes for version 20030109. 1) ACPI CA Core Subsystem: Changed the behavior of the internal Buffer-to-String conversion function. The current ACPI specification states that the contents of the buffer are "converted to a string of two-character hexadecimal numbers, each separated by a space". Unfortunately, this definition is not backwards compatible with existing ACPI 1.0 implementations (although the behavior was not defined in the ACPI 1.0 specification). The new behavior simply copies data from the buffer to the string until a null character is found or the end of the buffer is reached. The new String object is always null terminated. This problem was seen during the generation of _BIF battery data where incorrect strings were returned for battery type, etc. This will also require an errata to the ACPI specification. Renamed all instances of NATIVE_UINT and NATIVE_INT to ACPI_NATIVE_UINT and ACPI_NATIVE_INT, respectively. Copyright in all module headers (both Linux and non-Linux) has be updated to 2003. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 72.0K Code, 9.5K Data, 81.5K Total Debug Version: 153.0K Code, 62.9K Data, 215.9K Total Current Release: Non-Debug Version: 72.0K Code, 9.5K Data, 81.5K Total Debug Version: 153.0K Code, 62.9K Data, 215.9K Total 2) Linux Fixed an oops on module insertion/removal (Matthew Tippett) (2.4) Fix to handle dynamic size of mp_irqs (Joerg Prante) (2.5) Replace pr_debug (Randy Dunlap) (2.5) Remove usage of CPUFREQ_ALL_CPUS (Dominik Brodowski) (Both) Eliminate spawning of thread from timer callback, in favor of schedule_work() (Both) Show Lid status in /proc (Zdenek OGAR Skalak) (Both) Added define for Fixed Function HW region (Matthew Wilcox) (Both) Add missing statics to button.c (Pavel Machek) Several changes have been made to the source code translation utility that generates the Linux Code in order to make the code more "Linux-like": All typedefs on structs and unions have been removed in keeping with the Linux coding style. Removed the non-Linux SourceSafe module revision number from each module header. Completed major overhaul of symbols to be lowercase for linux. Doubled the number of symbols that are lowercase. Fixed a problem where identifiers within procedure headers and within quotes were not fully lower cased (they were left with a starting capital.) Some C macros whose only purpose is to allow the generation of 16- bit code are now completely removed in the Linux code, increasing readability and maintainability. ---------------------------------------- 12 December 2002. Summary of changes for version 20021212. 1) ACPI CA Core Subsystem: Fixed a problem where the creation of a zero-length AML Buffer would cause a fault. Fixed a problem where a Buffer object that pointed to a static AML buffer (in an ACPI table) could inadvertently be deleted, causing memory corruption. Fixed a problem where a user buffer (passed in to the external ACPI CA interfaces) could be overwritten if the buffer was too small to complete the operation, causing memory corruption. Fixed a problem in the Buffer-to-String conversion code where a string of length one was always returned, regardless of the size of the input Buffer object. Removed the NATIVE_CHAR data type across the entire source due to lack of need and lack of consistent use. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 72.1K Code, 9.5K Data, 81.6K Total Debug Version: 152.7K Code, 62.7K Data, 215.4K Total Current Release: Non-Debug Version: 72.0K Code, 9.5K Data, 81.5K Total Debug Version: 153.0K Code, 62.9K Data, 215.9K Total ---------------------------------------- 05 December 2002. Summary of changes for version 20021205. 1) ACPI CA Core Subsystem: Fixed a problem where a store to a String or Buffer object could cause corruption of the DSDT if the object type being stored was the same as the target object type and the length of the object being stored was equal to or smaller than the original (existing) target object. This was seen to cause corruption of battery _BIF buffers if the _BIF method modified the buffer on the fly. Fixed a problem where an internal error was generated if a control method invocation was used in an OperationRegion, Buffer, or Package declaration. This was caused by the deferred parsing of the control method and thus the deferred creation of the internal method object. The solution to this problem was to create the internal method object at the moment the method is encountered in the first pass - so that subsequent references to the method will able to obtain the required parameter count and thus properly parse the method invocation. This problem presented itself as an AE_AML_INTERNAL during the pass 1 parse phase during table load. Fixed a problem where the internal String object copy routine did not always allocate sufficient memory for the target String object and caused memory corruption. This problem was seen to cause "Allocation already present in list!" errors as memory allocation became corrupted. Implemented a new function for the evaluation of namespace objects that allows the specification of the allowable return object types. This simplifies a lot of code that checks for a return object of one or more specific objects returned from the evaluation (such as _STA, etc.) This may become and external function if it would be useful to ACPI-related drivers. Completed another round of prefixing #defines with "ACPI_" for clarity. Completed additional code restructuring to allow more modular linking for iASL compiler and AcpiExec. Several files were split creating new files. New files: nsparse.c dsinit.c evgpe.c Implemented an abort mechanism to terminate an executing control method via the AML debugger. This feature is useful for debugging control methods that depend (wait) for specific hardware responses. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 71.4K Code, 9.0K Data, 80.4K Total Debug Version: 152.9K Code, 63.3K Data, 216.2K Total Current Release: Non-Debug Version: 72.1K Code, 9.5K Data, 81.6K Total Debug Version: 152.7K Code, 62.7K Data, 215.4K Total 2) iASL Compiler/Disassembler Fixed a compiler code generation problem for "Interrupt" Resource Descriptors. If specified in the ASL, the optional "Resource Source Index" and "Resource Source" fields were not inserted into the correct location within the AML resource descriptor, creating an invalid descriptor. Fixed a disassembler problem for "Interrupt" resource descriptors. The optional "Resource Source Index" and "Resource Source" fields were ignored. ---------------------------------------- 22 November 2002. Summary of changes for version 20021122. 1) ACPI CA Core Subsystem: Fixed a reported problem where an object stored to a Method Local or Arg was not copied to a new object during the store - the object pointer was simply copied to the Local/Arg. This caused all subsequent operations on the Local/Arg to also affect the original source of the store operation. Fixed a problem where a store operation to a Method Local or Arg was not completed properly if the Local/Arg contained a reference (from RefOf) to a named field. The general-purpose store-to- namespace-node code is now used so that this case is handled automatically. Fixed a problem where the internal object copy routine would cause a protection fault if the object being copied was a Package and contained either 1) a NULL package element or 2) a nested sub- package. Fixed a problem with the GPE initialization that resulted from an ambiguity in the ACPI specification. One section of the specification states that both the address and length of the GPE block must be zero if the block is not supported. Another section implies that only the address need be zero if the block is not supported. The code has been changed so that both the address and the length must be non-zero to indicate a valid GPE block (i.e., if either the address or the length is zero, the GPE block is invalid.) Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 71.3K Code, 9.0K Data, 80.3K Total Debug Version: 152.7K Code, 63.2K Data, 215.5K Total Current Release: Non-Debug Version: 71.4K Code, 9.0K Data, 80.4K Total Debug Version: 152.9K Code, 63.3K Data, 216.2K Total 2) Linux Cleaned up EC driver. Exported an external EC read/write interface. By going through this, other drivers (most notably sonypi) will be able to serialize access to the EC. 3) iASL Compiler/Disassembler Implemented support to optionally generate include files for both ASM and C (the -i switch). This simplifies BIOS development by automatically creating include files that contain external declarations for the symbols that are created within the (optionally generated) ASM and C AML source files. ---------------------------------------- 15 November 2002. Summary of changes for version 20021115. 1) ACPI CA Core Subsystem: Fixed a memory leak problem where an error during resolution of method arguments during a method invocation from another method failed to cleanup properly by deleting all successfully resolved argument objects. Fixed a problem where the target of the Index() operator was not correctly constructed if the source object was a package. This problem has not been detected because the use of a target operand with Index() is very rare. Fixed a problem with the Index() operator where an attempt was made to delete the operand objects twice. Fixed a problem where an attempt was made to delete an operand twice during execution of the CondRefOf() operator if the target did not exist. Implemented the first of perhaps several internal create object functions that create and initialize a specific object type. This consolidates duplicated code wherever the object is created, thus shrinking the size of the subsystem. Implemented improved debug/error messages for errors that occur during nested method invocations. All executing method pathnames are displayed (with the error) as the call stack is unwound - thus simplifying debug. Fixed a problem introduced in the 10/02 release that caused premature deletion of a buffer object if a buffer was used as an ASL operand where an integer operand is required (Thus causing an implicit object conversion from Buffer to Integer.) The change in the 10/02 release was attempting to fix a memory leak (albeit incorrectly.) Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 71.9K Code, 9.1K Data, 81.0K Total Debug Version: 153.1K Code, 63.3K Data, 216.4K Total Current Release: Non-Debug Version: 71.3K Code, 9.0K Data, 80.3K Total Debug Version: 152.7K Code, 63.2K Data, 215.5K Total 2) Linux Changed the implementation of the ACPI semaphores to use down() instead of down_interruptable(). It is important that the execution of ACPI control methods not be interrupted by signals. Methods must run to completion, or the system may be left in an unknown/unstable state. Fixed a compilation error when CONFIG_SOFTWARE_SUSPEND is not set. (Shawn Starr) 3) iASL Compiler/Disassembler Changed the default location of output files. All output files are now placed in the current directory by default instead of in the directory of the source file. This change may affect some existing makefiles, but it brings the behavior of the compiler in line with other similar tools. The location of the output files can be overridden with the -p command line switch. ---------------------------------------- 11 November 2002. Summary of changes for version 20021111. 0) ACPI Specification 2.0B is released and is now available at: http://www.acpi.info/index.html 1) ACPI CA Core Subsystem: Implemented support for the ACPI 2.0 SMBus Operation Regions. This includes the early detection and handoff of the request to the SMBus region handler (avoiding all of the complex field support code), and support for the bidirectional return packet from an SMBus write operation. This paves the way for the development of SMBus drivers in each host operating system. Fixed a problem where the semaphore WAIT_FOREVER constant was defined as 32 bits, but must be 16 bits according to the ACPI specification. This had the side effect of causing ASL Mutex/Event timeouts even though the ASL code requested a wait forever. Changed all internal references to the ACPI timeout parameter to 16 bits to prevent future problems. Changed the name of WAIT_FOREVER to ACPI_WAIT_FOREVER. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 71.4K Code, 9.0K Data, 80.4K Total Debug Version: 152.3K Code, 63.0K Data, 215.3K Total Current Release: Non-Debug Version: 71.9K Code, 9.1K Data, 81.0K Total Debug Version: 153.1K Code, 63.3K Data, 216.4K Total 2) Linux Module loading/unloading fixes (John Cagle) 3) iASL Compiler/Disassembler Added support for the SMBBlockProcessCall keyword (ACPI 2.0) Implemented support for the disassembly of all SMBus protocol keywords (SMBQuick, SMBWord, etc.) ---------------------------------------- 01 November 2002. Summary of changes for version 20021101. 1) ACPI CA Core Subsystem: Fixed a problem where platforms that have a GPE1 block but no GPE0 block were not handled correctly. This resulted in a "GPE overlap" error message. GPE0 is no longer required. Removed code added in the previous release that inserted nodes into the namespace in alphabetical order. This caused some side- effects on various machines. The root cause of the problem is still under investigation since in theory, the internal ordering of the namespace nodes should not matter. Enhanced error reporting for the case where a named object is not found during control method execution. The full ACPI namepath (name reference) of the object that was not found is displayed in this case. Note: as a result of the overhaul of the namespace object types in the previous release, the namespace nodes for the predefined scopes (_TZ, _PR, etc.) are now of the type ACPI_TYPE_LOCAL_SCOPE instead of ACPI_TYPE_ANY. This simplifies the namespace management code but may affect code that walks the namespace tree looking for specific object types. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 70.7K Code, 8.6K Data, 79.3K Total Debug Version: 151.7K Code, 62.4K Data, 214.1K Total Current Release: Non-Debug Version: 71.4K Code, 9.0K Data, 80.4K Total Debug Version: 152.3K Code, 63.0K Data, 215.3K Total 2) Linux Fixed a problem introduced in the previous release where the Processor and Thermal objects were not recognized and installed in /proc. This was related to the scope type change described above. 3) iASL Compiler/Disassembler Implemented the -g option to get all of the required ACPI tables from the registry and save them to files (Windows version of the compiler only.) The required tables are the FADT, FACS, and DSDT. Added ACPI table checksum validation during table disassembly in order to catch corrupted tables. ---------------------------------------- 22 October 2002. Summary of changes for version 20021022. 1) ACPI CA Core Subsystem: Implemented a restriction on the Scope operator that the target must already exist in the namespace at the time the operator is encountered (during table load or method execution). In other words, forward references are not allowed and Scope() cannot create a new object. This changes the previous behavior where the interpreter would create the name if not found. This new behavior correctly enables the search-to-root algorithm during namespace lookup of the target name. Because of this upsearch, this fixes the known Compaq _SB_.OKEC problem and makes both the AML interpreter and iASL compiler compatible with other ACPI implementations. Completed a major overhaul of the internal ACPI object types for the ACPI Namespace and the associated operand objects. Many of these types had become obsolete with the introduction of the two- pass namespace load. This cleanup simplifies the code and makes the entire namespace load mechanism much clearer and easier to understand. Improved debug output for tracking scope opening/closing to help diagnose scoping issues. The old scope name as well as the new scope name are displayed. Also improved error messages for problems with ASL Mutex objects and error messages for GPE problems. Cleaned up the namespace dump code, removed obsolete code. All string output (for all namespace/object dumps) now uses the common ACPI string output procedure which handles escapes properly and does not emit non-printable characters. Fixed some issues with constants in the 64-bit version of the local C library (utclib.c) 2) Linux EC Driver: No longer attempts to acquire the Global Lock at interrupt level. 3) iASL Compiler/Disassembler Implemented ACPI 2.0B grammar change that disallows all Type 1 and 2 opcodes outside of a control method. This means that the "executable" operators (versus the "namespace" operators) cannot be used at the table level; they can only be used within a control method. Implemented the restriction on the Scope() operator where the target must already exist in the namespace at the time the operator is encountered (during ASL compilation). In other words, forward references are not allowed and Scope() cannot create a new object. This makes the iASL compiler compatible with other ACPI implementations and makes the Scope() implementation adhere to the ACPI specification. Fixed a problem where namepath optimization for the Alias operator was optimizing the wrong path (of the two namepaths.) This caused a "Missing alias link" error message. Fixed a problem where an "unknown reserved name" warning could be incorrectly generated for names like "_SB" when the trailing underscore is not used in the original ASL. Fixed a problem where the reserved name check did not handle NamePaths with multiple NameSegs correctly. The first nameseg of the NamePath was examined instead of the last NameSeg. ---------------------------------------- 02 October 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem version 20021002: Fixed a problem where a store/copy of a string to an existing string did not always set the string length properly in the String object. Fixed a reported problem with the ToString operator where the behavior was identical to the ToHexString operator instead of just simply converting a raw buffer to a string data type. Fixed a problem where CopyObject and the other "explicit" conversion operators were not updating the internal namespace node type as part of the store operation. Fixed a memory leak during implicit source operand conversion where the original object was not deleted if it was converted to a new object of a different type. Enhanced error messages for all problems associated with namespace lookups. Common procedure generates and prints the lookup name as well as the formatted status. Completed implementation of a new design for the Alias support within the namespace. The existing design did not handle the case where a new object was assigned to one of the two names due to the use of an explicit conversion operator, resulting in the two names pointing to two different objects. The new design simply points the Alias name to the original name node - not to the object. This results in a level of indirection that must be handled in the name resolution mechanism. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 69.6K Code, 8.3K Data, 77.9K Total Debug Version: 150.0K Code, 61.7K Data, 211.7K Total Current Release: Non-Debug Version: 70.7K Code, 8.6K Data, 79.3K Total Debug Version: 151.7K Code, 62.4K Data, 214.1K Total 2) Linux Initialize thermal driver's timer before it is used. (Knut Neumann) Allow handling negative celsius values. (Kochi Takayoshi) Fix thermal management and make trip points. R/W (Pavel Machek) Fix /proc/acpi/sleep. (P. Christeas) IA64 fixes. (David Mosberger) Fix reversed logic in blacklist code. (Sergio Monteiro Basto) Replace ACPI_DEBUG define with ACPI_DEBUG_OUTPUT. (Dominik Brodowski) 3) iASL Compiler/Disassembler Clarified some warning/error messages. ---------------------------------------- 18 September 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem version 20020918: Fixed a reported problem with reference chaining (via the Index() and RefOf() operators) in the ObjectType() and SizeOf() operators. The definition of these operators includes the dereferencing of all chained references to return information on the base object. Fixed a problem with stores to indexed package elements - the existing code would not complete the store if an "implicit conversion" was not performed. In other words, if the existing object (package element) was to be replaced completely, the code didn't handle this case. Relaxed typechecking on the ASL "Scope" operator to allow the target name to refer to an object of type Integer, String, or Buffer, in addition to the scoping object types (Device, predefined Scopes, Processor, PowerResource, and ThermalZone.) This allows existing AML code that has workarounds for a bug in Windows to function properly. A warning is issued, however. This affects both the AML interpreter and the iASL compiler. Below is an example of this type of ASL code: Name(DEB,0x00) Scope(DEB) { Fixed some reported problems with 64-bit integer support in the local implementation of C library functions (clib.c) 2) Linux Use ACPI fix map region instead of IOAPIC region, since it is undefined in non-SMP. Ensure that the SCI has the proper polarity and trigger, even on systems that do not have an interrupt override entry in the MADT. 2.5 big driver reorganization (Pat Mochel) Use early table mapping code from acpitable.c (Andi Kleen) New blacklist entries (Andi Kleen) Blacklist improvements. Split blacklist code out into a separate file. Move checking the blacklist to very early. Previously, we would use ACPI tables, and then halfway through init, check the blacklist -- too late. Now, it's early enough to completely fall- back to non-ACPI. 3) iASL Compiler/Disassembler version 20020918: Fixed a problem where the typechecking code didn't know that an alias could point to a method. In other words, aliases were not being dereferenced during typechecking. ---------------------------------------- 29 August 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem Version 20020829: If the target of a Scope() operator already exists, it must be an object type that actually opens a scope -- such as a Device, Method, Scope, etc. This is a fatal runtime error. Similar error check has been added to the iASL compiler also. Tightened up the namespace load to disallow multiple names in the same scope. This previously was allowed if both objects were of the same type. (i.e., a lookup was the same as entering a new name). 2) Linux Ensure that the ACPI interrupt has the proper trigger and polarity. local_irq_disable is extraneous. (Matthew Wilcox) Make "acpi=off" actually do what it says, and not use the ACPI interpreter *or* the tables. Added arch-neutral support for parsing SLIT and SRAT tables (Kochi Takayoshi) 3) iASL Compiler/Disassembler Version 20020829: Implemented namepath optimization for name declarations. For example, a declaration like "Method (\_SB_.ABCD)" would get optimized to "Method (ABCD)" if the declaration is within the \_SB_ scope. This optimization is in addition to the named reference path optimization first released in the previous version. This would seem to complete all possible optimizations for namepaths within the ASL/AML. If the target of a Scope() operator already exists, it must be an object type that actually opens a scope -- such as a Device, Method, Scope, etc. Implemented a check and warning for unreachable code in the same block below a Return() statement. Fixed a problem where the listing file was not generated if the compiler aborted if the maximum error count was exceeded (200). Fixed a problem where the typechecking of method return values was broken. This includes the check for a return value when the method is invoked as a TermArg (a return value is expected.) Fixed a reported problem where EOF conditions during a quoted string or comment caused a fault. ---------------------------------------- 15 August 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem Version 20020815: Fixed a reported problem where a Store to a method argument that contains a reference did not perform the indirect store correctly. This problem was created during the conversion to the new reference object model - the indirect store to a method argument code was not updated to reflect the new model. Reworked the ACPI mode change code to better conform to ACPI 2.0, handle corner cases, and improve code legibility (Kochi Takayoshi) Fixed a problem with the pathname parsing for the carat (^) prefix. The heavy use of the carat operator by the new namepath optimization in the iASL compiler uncovered a problem with the AML interpreter handling of this prefix. In the case where one or more carats precede a single nameseg, the nameseg was treated as standalone and the search rule (to root) was inadvertently applied. This could cause both the iASL compiler and the interpreter to find the wrong object or to miss the error that should occur if the object does not exist at that exact pathname. Found and fixed the problem where the HP Pavilion DSDT would not load. This was a relatively minor tweak to the table loading code (a problem caused by the unexpected encounter with a method invocation not within a control method), but it does not solve the overall issue of the execution of AML code at the table level. This investigation is still ongoing. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 69.1K Code, 8.2K Data, 77.3K Total Debug Version: 149.4K Code, 61.6K Data, 211.0K Total Current Release: Non-Debug Version: 69.6K Code, 8.3K Data, 77.9K Total Debug Version: 150.0K Code, 61.7K Data, 211.7K Total 2) Linux Remove redundant slab.h include (Brad Hards) Fix several bugs in thermal.c (Herbert Nachtnebel) Make CONFIG_ACPI_BOOT work properly (Pavel Machek) Change acpi_system_suspend to use updated irq functions (Pavel Machek) Export acpi_get_firmware_table (Matthew Wilcox) Use proper root proc entry for ACPI (Kochi Takayoshi) Fix early-boot table parsing (Bjorn Helgaas) 3) iASL Compiler/Disassembler Reworked the compiler options to make them more consistent and to use two-letter options where appropriate. We were running out of sensible letters. This may break some makefiles, so check the current options list by invoking the compiler with no parameters. Completed the design and implementation of the ASL namepath optimization option for the compiler. This option optimizes all references to named objects to the shortest possible path. The first attempt tries to utilize a single nameseg (4 characters) and the "search-to-root" algorithm used by the interpreter. If that cannot be used (because either the name is not in the search path or there is a conflict with another object with the same name), the pathname is optimized using the carat prefix (usually a shorter string than specifying the entire path from the root.) Implemented support to obtain the DSDT from the Windows registry (when the disassembly option is specified with no input file). Added this code as the implementation for AcpiOsTableOverride in the Windows OSL. Migrated the 16-bit code (used in the AcpiDump utility) to scan memory for the DSDT to the AcpiOsTableOverride function in the DOS OSL to make the disassembler truly OS independent. Implemented a new option to disassemble and compile in one step. When used without an input filename, this option will grab the DSDT from the local machine, disassemble it, and compile it in one step. Added a warning message for invalid escapes (a backslash followed by any character other than the allowable escapes). This catches the quoted string error "\_SB_" (which should be "\\_SB_" ). Also, there are numerous instances in the ACPI specification where this error occurs. Added a compiler option to disable all optimizations. This is basically the "compatibility mode" because by using this option, the AML code will come out exactly the same as other ASL compilers. Added error messages for incorrectly ordered dependent resource functions. This includes: missing EndDependentFn macro at end of dependent resource list, nested dependent function macros (both start and end), and missing StartDependentFn macro. These are common errors that should be caught at compile time. Implemented _OSI support for the disassembler and compiler. _OSI must be included in the namespace for proper disassembly (because the disassembler must know the number of arguments.) Added an "optimization" message type that is optional (off by default). This message is used for all optimizations - including constant folding, integer optimization, and namepath optimization. ---------------------------------------- 25 July 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem Version 20020725: The AML Disassembler has been enhanced to produce compilable ASL code and has been integrated into the iASL compiler (see below) as well as the single-step disassembly for the AML debugger and the disassembler for the AcpiDump utility. All ACPI 2.0A opcodes, resource templates and macros are fully supported. The disassembler has been tested on over 30 different AML files, producing identical AML when the resulting disassembled ASL file is recompiled with the same ASL compiler. Modified the Resource Manager to allow zero interrupts and zero dma channels during the GetCurrentResources call. This was causing problems on some platforms. Added the AcpiOsRedirectOutput interface to the OSL to simplify output redirection for the AcpiOsPrintf and AcpiOsVprintf interfaces. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 68.7K Code, 7.4K Data, 76.1K Total Debug Version: 142.9K Code, 58.7K Data, 201.6K Total Current Release: Non-Debug Version: 69.1K Code, 8.2K Data, 77.3K Total Debug Version: 149.4K Code, 61.6K Data, 211.0K Total 2) Linux Fixed a panic in the EC driver (Dominik Brodowski) Implemented checksum of the R/XSDT itself during Linux table scan (Richard Schaal) 3) iASL compiler The AML disassembler is integrated into the compiler. The "-d" option invokes the disassembler to completely disassemble an input AML file, producing as output a text ASL file with the extension ".dsl" (to avoid name collisions with existing .asl source files.) A future enhancement will allow the disassembler to obtain the BIOS DSDT from the registry under Windows. Fixed a problem with the VendorShort and VendorLong resource descriptors where an invalid AML sequence was created. Implemented a fix for BufferData term in the ASL parser. It was inadvertently defined twice, allowing invalid syntax to pass and causing reduction conflicts. Fixed a problem where the Ones opcode could get converted to a value of zero if "Ones" was used where a byte, word or dword value was expected. The 64-bit value is now truncated to the correct size with the correct value. ---------------------------------------- 02 July 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem Version 20020702: The Table Manager code has been restructured to add several new features. Tables that are not required by the core subsystem (other than the FADT, DSDT, FACS, PSDTs, etc.) are no longer validated in any way and are returned from AcpiGetFirmwareTable if requested. The AcpiOsTableOverride interface is now called for each table that is loaded by the subsystem in order to allow the host to override any table it chooses. Previously, only the DSDT could be overridden. Added one new files, tbrsdt.c and tbgetall.c. Fixed a problem with the conversion of internal package objects to external objects (when a package is returned from a control method.) The return buffer length was set to zero instead of the proper length of the package object. Fixed a reported problem with the use of the RefOf and DeRefOf operators when passing reference arguments to control methods. A new type of Reference object is used internally for references produced by the RefOf operator. Added additional error messages in the Resource Manager to explain AE_BAD_DATA errors when they occur during resource parsing. Split the AcpiEnableSubsystem into two primitives to enable a finer granularity initialization sequence. These two calls should be called in this order: AcpiEnableSubsystem (flags), AcpiInitializeObjects (flags). The flags parameter remains the same. 2) Linux Updated the ACPI utilities module to understand the new style of fully resolved package objects that are now returned from the core subsystem. This eliminates errors of the form: ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PPB_._PRT] acpi_utils-0430 [145] acpi_evaluate_reference: Invalid element in package (not a device reference) The method evaluation utility uses the new buffer allocation scheme instead of calling AcpiEvaluate Object twice. Added support for ECDT. This allows the use of the Embedded Controller before the namespace has been fully initialized, which is necessary for ACPI 2.0 support, and for some laptops to initialize properly. (Laptops using ECDT are still rare, so only limited testing was performed of the added functionality.) Fixed memory leaks in the EC driver. Eliminated a brittle code structure in acpi_bus_init(). Eliminated the acpi_evaluate() helper function in utils.c. It is no longer needed since acpi_evaluate_object can optionally allocate memory for the return object. Implemented fix for keyboard hang when getting battery readings on some systems (Stephen White) PCI IRQ routing update (Dominik Brodowski) Fix an ifdef to allow compilation on UP with LAPIC but no IOAPIC support ---------------------------------------- 11 June 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem Version 20020611: Fixed a reported problem where constants such as Zero and One appearing within _PRT packages were not handled correctly within the resource manager code. Originally reported against the ASL compiler because the code generator now optimizes integers to their minimal AML representation (i.e. AML constants if possible.) The _PRT code now handles all AML constant opcodes correctly (Zero, One, Ones, Revision). Fixed a problem with the Concatenate operator in the AML interpreter where a buffer result object was incorrectly marked as not fully evaluated, causing a run-time error of AE_AML_INTERNAL. All package sub-objects are now fully resolved before they are returned from the external ACPI interfaces. This means that name strings are resolved to object handles, and constant operators (Zero, One, Ones, Revision) are resolved to Integers. Implemented immediate resolution of the AML Constant opcodes (Zero, One, Ones, Revision) to Integer objects upon detection within the AML stream. This has simplified and reduced the generated code size of the subsystem by eliminating about 10 switch statements for these constants (which previously were contained in Reference objects.) The complicating issues are that the Zero opcode is used as a "placeholder" for unspecified optional target operands and stores to constants are defined to be no-ops. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 69.3K Code, 7.4K Data, 76.7K Total Debug Version: 143.8K Code, 58.8K Data, 202.6K Total Current Release: Non-Debug Version: 68.7K Code, 7.4K Data, 76.1K Total Debug Version: 142.9K Code, 58.7K Data, 201.6K Total 2) Linux Added preliminary support for obtaining _TRA data for PCI root bridges (Bjorn Helgaas). 3) iASL Compiler Version X2046: Fixed a problem where the "_DDN" reserved name was defined to be a control method with one argument. There are no arguments, and _DDN does not have to be a control method. Fixed a problem with the Linux version of the compiler where the source lines printed with error messages were the wrong lines. This turned out to be the "LF versus CR/LF" difference between Windows and Unix. This appears to be the longstanding issue concerning listing output and error messages. Fixed a problem with the Linux version of compiler where opcode names within error messages were wrong. This was caused by a slight difference in the output of the Flex tool on Linux versus Windows. Fixed a problem with the Linux compiler where the hex output files contained some garbage data caused by an internal buffer overrun. ---------------------------------------- 17 May 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem Version 20020517: Implemented a workaround to an BIOS bug discovered on the HP OmniBook where the FADT revision number and the table size are inconsistent (ACPI 2.0 revision vs. ACPI 1.0 table size). The new behavior is to fallback to using only the ACPI 1.0 fields of the FADT if the table is too small to be a ACPI 2.0 table as claimed by the revision number. Although this is a BIOS bug, this is a case where the workaround is simple enough and with no side effects, so it seemed prudent to add it. A warning message is issued, however. Implemented minimum size checks for the fixed-length ACPI tables - - the FADT and FACS, as well as consistency checks between the revision number and the table size. Fixed a reported problem in the table override support where the new table pointer was incorrectly treated as a physical address instead of a logical address. Eliminated the use of the AE_AML_ERROR exception and replaced it with more descriptive codes. Fixed a problem where an exception would occur if an ASL Field was defined with no named Field Units underneath it (used by some index fields). Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 68.8K Code, 7.1K Data, 75.9K Total Debug Version: 142.9K Code, 58.4K Data, 201.3K Total Current Release: Non-Debug Version: 69.3K Code, 7.4K Data, 76.7K Total Debug Version: 143.8K Code, 58.8K Data, 202.6K Total 2) Linux Much work done on ACPI init (MADT and PCI IRQ routing support). (Paul D. and Dominik Brodowski) Fix PCI IRQ-related panic on boot (Sam Revitch) Set BM_ARB_DIS when entering a sleep state (Ducrot Bruno) Fix "MHz" typo (Dominik Brodowski) Fix RTC year 2000 issue (Dominik Brodowski) Preclude multiple button proc entries (Eric Brunet) Moved arch-specific code out of include/platform/aclinux.h 3) iASL Compiler Version X2044: Implemented error checking for the string used in the EISAID macro (Usually used in the definition of the _HID object.) The code now strictly enforces the PnP format - exactly 7 characters, 3 uppercase letters and 4 hex digits. If a raw string is used in the definition of the _HID object (instead of the EISAID macro), the string must contain all alphanumeric characters (e.g., "*PNP0011" is not allowed because of the asterisk.) Implemented checking for invalid use of ACPI reserved names for most of the name creation operators (Name, Device, Event, Mutex, OperationRegion, PowerResource, Processor, and ThermalZone.) Previously, this check was only performed for control methods. Implemented an additional check on the Name operator to emit an error if a reserved name that must be implemented in ASL as a control method is used. We know that a reserved name must be a method if it is defined with input arguments. The warning emitted when a namespace object reference is not found during the cross reference phase has been changed into an error. The "External" directive should be used for names defined in other modules. 4) Tools and Utilities The 16-bit tools (adump16 and aexec16) have been regenerated and tested. Fixed a problem with the output of both acpidump and adump16 where the indentation of closing parentheses and brackets was not aligned properly with the parent block. ---------------------------------------- 03 May 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem Version 20020503: Added support a new OSL interface that allows the host operating system software to override the DSDT found in the firmware - AcpiOsTableOverride. With this interface, the OSL can examine the version of the firmware DSDT and replace it with a different one if desired. Added new external interfaces for accessing ACPI registers from device drivers and other system software - AcpiGetRegister and AcpiSetRegister. This was simply an externalization of the existing AcpiHwBitRegister interfaces. Fixed a regression introduced in the previous build where the ASL/AML CreateField operator always returned an error, "destination must be a NS Node". Extended the maximum time (before failure) to successfully enable ACPI mode to 3 seconds. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 68.5K Code, 7.0K Data, 75.5K Total Debug Version: 142.4K Code, 58.3K Data, 200.7K Total Current Release: Non-Debug Version: 68.8K Code, 7.1K Data, 75.9K Total Debug Version: 142.9K Code, 58.4K Data, 201.3K Total 2) Linux Enhanced ACPI init code for SMP. We are now fully MPS and $PIR- free. While 3 out of 4 of our in-house systems work fine, the last one still hangs when testing the LAPIC timer. Renamed many files in 2.5 kernel release to omit "acpi_" from the name. Added warning on boot for Presario 711FR. Sleep improvements (Pavel Machek) ACPI can now be built without CONFIG_PCI enabled. IA64: Fixed memory map functions (JI Lee) 3) iASL Compiler Version X2043: Added support to allow the compiler to be integrated into the MS VC++ development environment for one-button compilation of single files or entire projects -- with error-to-source-line mapping. Implemented support for compile-time constant folding for the Type3, Type4, and Type5 opcodes first defined in the ACPI 2.0 specification. This allows the ASL writer to use expressions instead of Integer/Buffer/String constants in terms that must evaluate to constants at compile time and will also simplify the emitted AML in any such sub-expressions that can be folded (evaluated at compile-time.) This increases the size of the compiler significantly because a portion of the ACPI CA AML interpreter is included within the compiler in order to pre- evaluate constant expressions. Fixed a problem with the "Unicode" ASL macro that caused the compiler to fault. (This macro is used in conjunction with the _STR reserved name.) Implemented an AML opcode optimization to use the Zero, One, and Ones opcodes where possible to further reduce the size of integer constants and thus reduce the overall size of the generated AML code. Implemented error checking for new reserved terms for ACPI version 2.0A. Implemented the -qr option to display the current list of ACPI reserved names known to the compiler. Implemented the -qc option to display the current list of ASL operators that are allowed within constant expressions and can therefore be folded at compile time if the operands are constants. 4) Documentation Updated the Programmer's Reference for new interfaces, data types, and memory allocation model options. Updated the iASL Compiler User Reference to apply new format and add information about new features and options. ---------------------------------------- 19 April 2002. Summary of changes for this release. 1) ACPI CA Core Subsystem Version 20020419: The source code base for the Core Subsystem has been completely cleaned with PC-lint (FlexLint) for both 32-bit and 64-bit versions. The Lint option files used are included in the /acpi/generate/lint directory. Implemented enhanced status/error checking across the entire Hardware manager subsystem. Any hardware errors (reported from the OSL) are now bubbled up and will abort a running control method. Fixed a problem where the per-ACPI-table integer width (32 or 64) was stored only with control method nodes, causing a fault when non-control method code was executed during table loading. The solution implemented uses a global variable to indicate table width across the entire ACPI subsystem. Therefore, ACPI CA does not support mixed integer widths across different ACPI tables (DSDT, SSDT). Fixed a problem where NULL extended fields (X fields) in an ACPI 2.0 ACPI FADT caused the table load to fail. Although the existing ACPI specification is a bit fuzzy on this topic, the new behavior is to fall back on a ACPI 1.0 field if the corresponding ACPI 2.0 X field is zero (even though the table revision indicates a full ACPI 2.0 table.) The ACPI specification will be updated to clarify this issue. Fixed a problem with the SystemMemory operation region handler where memory was always accessed byte-wise even if the AML- specified access width was larger than a byte. This caused problems on systems with memory-mapped I/O. Memory is now accessed with the width specified. On systems that do not support non-aligned transfers, a check is made to guarantee proper address alignment before proceeding in order to avoid an AML-caused alignment fault within the kernel. Fixed a problem with the ExtendedIrq resource where only one byte of the 4-byte Irq field was extracted. Fixed the AcpiExDigitsNeeded() procedure to support _UID. This function was out of date and required a rewrite. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 66.6K Code, 6.5K Data, 73.1K Total Debug Version: 139.8K Code, 57.4K Data, 197.2K Total Current Release: Non-Debug Version: 68.5K Code, 7.0K Data, 75.5K Total Debug Version: 142.4K Code, 58.3K Data, 200.7K Total 2) Linux PCI IRQ routing fixes (Dominik Brodowski) 3) iASL Compiler Version X2042: Implemented an additional compile-time error check for a field unit whose size + minimum access width would cause a run-time access beyond the end-of-region. Previously, only the field size itself was checked. The Core subsystem and iASL compiler now share a common parse object in preparation for compile-time evaluation of the type 3/4/5 ASL operators. ---------------------------------------- Summary of changes for this release: 03_29_02 1) ACPI CA Core Subsystem Version 20020329: Implemented support for late evaluation of TermArg operands to Buffer and Package objects. This allows complex expressions to be used in the declarations of these object types. Fixed an ACPI 1.0 compatibility issue when reading Fields. In ACPI 1.0, if the field was larger than 32 bits, it was returned as a buffer - otherwise it was returned as an integer. In ACPI 2.0, the field is returned as a buffer only if the field is larger than 64 bits. The TableRevision is now considered when making this conversion to avoid incompatibility with existing ASL code. Implemented logical addressing for AcpiOsGetRootPointer. This allows an RSDP with either a logical or physical address. With this support, the host OS can now override all ACPI tables with one logical RSDP. Includes implementation of "typed" pointer support to allow a common data type for both physical and logical pointers internally. This required a change to the AcpiOsGetRootPointer interface. Implemented the use of ACPI 2.0 Generic Address Structures for all GPE, Fixed Event, and PM Timer I/O. This allows the use of memory mapped I/O for these ACPI features. Initialization now ignores not only non-required tables (All tables other than the FADT, FACS, DSDT, and SSDTs), but also does not validate the table headers of unrecognized tables. Fixed a problem where a notify handler could only be installed/removed on an object of type Device. All "notify" objects are now supported -- Devices, Processor, Power, and Thermal. Removed most verbosity from the ACPI_DB_INFO debug level. Only critical information is returned when this debug level is enabled. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release Non-Debug Version: 65.4K Code, 6.2K Data, 71.6K Total Debug Version: 138.0K Code, 56.6K Data, 194.6K Total Current Release: Non-Debug Version: 66.6K Code, 6.5K Data, 73.1K Total Debug Version: 139.8K Code, 57.4K Data, 197.2K Total 2) Linux: The processor driver (acpi_processor.c) now fully supports ACPI 2.0-based processor performance control (e.g. Intel(R) SpeedStep(TM) technology) Note that older laptops that only have the Intel "applet" interface are not supported through this. The 'limit' and 'performance' interface (/proc) are fully functional. [Note that basic policy for controlling performance state transitions will be included in the next version of ospmd.] The idle handler was modified to more aggressively use C2, and PIIX4 errata handling underwent a complete overhaul (big thanks to Dominik Brodowski). Added support for ACPI-PCI device binding (acpi_pci_root.c). _ADR- based devices in the ACPI namespace are now dynamically bound (associated) with their PCI counterparts (e.g. PCI1->01:00.0). This allows, among other things, ACPI to resolve bus numbers for subordinate PCI bridges. Enhanced PCI IRQ routing to get the proper bus number for _PRT entries defined underneath PCI bridges. Added IBM 600E to bad bios list due to invalid _ADR value for PIIX4 PCI-ISA bridge, resulting in improper PCI IRQ routing. In the process of adding full MADT support (e.g. IOAPIC) for IA32 (acpi.c, mpparse.c) -- stay tuned. Added back visual differentiation between fixed-feature and control-method buttons in dmesg. Buttons are also subtyped (e.g. button/power/PWRF) to simplify button identification. We no longer use -Wno-unused when compiling debug. Please ignore any "_THIS_MODULE defined but not used" messages. Can now shut down the system using "magic sysrq" key. 3) iASL Compiler version 2041: Fixed a problem where conversion errors for hex/octal/decimal constants were not reported. Implemented a fix for the General Register template Address field. This field was 8 bits when it should be 64. Fixed a problem where errors/warnings were no longer being emitted within the listing output file. Implemented the ACPI 2.0A restriction on ACPI Table Signatures to exactly 4 characters, alphanumeric only. ---------------------------------------- Summary of changes for this release: 03_08_02 1) ACPI CA Core Subsystem Version 20020308: Fixed a problem with AML Fields where the use of the "AccessAny" keyword could cause an interpreter error due to attempting to read or write beyond the end of the parent Operation Region. Fixed a problem in the SystemMemory Operation Region handler where an attempt was made to map memory beyond the end of the region. This was the root cause of the "AE_ERROR" and "AE_NO_MEMORY" errors on some Linux systems. Fixed a problem where the interpreter/namespace "search to root" algorithm was not functioning for some object types. Relaxed the internal restriction on the search to allow upsearches for all external object types as well as most internal types. 2) Linux: We now use safe_halt() macro versus individual calls to sti | hlt. Writing to the processor limit interface should now work. "echo 1" will increase the limit, 2 will decrease, and 0 will reset to the default. 3) ASL compiler: Fixed segfault on Linux version. ---------------------------------------- Summary of changes for this release: 02_25_02 1) ACPI CA Core Subsystem: Fixed a problem where the GPE bit masks were not initialized properly, causing erratic GPE behavior. Implemented limited support for multiple calling conventions. The code can be generated with either the VPL (variable parameter list, or "C") convention, or the FPL (fixed parameter list, or "Pascal") convention. The core subsystem is about 3.4% smaller when generated with FPL. 2) Linux Re-add some /proc/acpi/event functionality that was lost during the rewrite Resolved issue with /proc events for fixed-feature buttons showing up as the system device. Fixed checks on C2/C3 latencies to be inclusive of maximum values. Replaced AE_ERRORs in acpi_osl.c with more specific error codes. Changed ACPI PRT option from "pci=noacpi-routing" to "pci=noacpi" Fixed limit interface & usage to fix bugs with passive cooling hysterisis. Restructured PRT support. ---------------------------------------- Summary of changes for this label: 02_14_02 1) ACPI CA Core Subsystem: Implemented support in AcpiLoadTable to allow loading of FACS and FADT tables. Support for the now-obsolete interim 0.71 64-bit ACPI tables has been removed. All 64-bit platforms should be migrated to the ACPI 2.0 tables. The actbl71.h header has been removed from the source tree. All C macros defined within the subsystem have been prefixed with "ACPI_" to avoid collision with other system include files. Removed the return value for the two AcpiOsPrint interfaces, since it is never used and causes lint warnings for ignoring the return value. Added error checking to all internal mutex acquire and release calls. Although a failure from one of these interfaces is probably a fatal system error, these checks will cause the immediate abort of the currently executing method or interface. Fixed a problem where the AcpiSetCurrentResources interface could fault. This was a side effect of the deployment of the new memory allocation model. Fixed a couple of problems with the Global Lock support introduced in the last major build. The "common" (1.0/2.0) internal FACS was being overwritten with the FACS signature and clobbering the Global Lock pointer. Also, the actual firmware FACS was being unmapped after construction of the "common" FACS, preventing access to the actual Global Lock field within it. The "common" internal FACS is no longer installed as an actual ACPI table; it is used simply as a global. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (02_07_01) Non-Debug Version: 65.2K Code, 6.2K Data, 71.4K Total Debug Version: 136.9K Code, 56.4K Data, 193.3K Total Current Release: Non-Debug Version: 65.4K Code, 6.2K Data, 71.6K Total Debug Version: 138.0K Code, 56.6K Data, 194.6K Total 2) Linux Updated Linux-specific code for core macro and OSL interface changes described above. Improved /proc/acpi/event. It now can be opened only once and has proper poll functionality. Fixed and restructured power management (acpi_bus). Only create /proc "view by type" when devices of that class exist. Fixed "charging/discharging" bug (and others) in acpi_battery. Improved thermal zone code. 3) ASL Compiler, version X2039: Implemented the new compiler restriction on ASL String hex/octal escapes to non-null, ASCII values. An error results if an invalid value is used. (This will require an ACPI 2.0 specification change.) AML object labels that are output to the optional C and ASM source are now prefixed with both the ACPI table signature and table ID to help guarantee uniqueness within a large BIOS project. ---------------------------------------- Summary of changes for this label: 02_01_02 1) ACPI CA Core Subsystem: ACPI 2.0 support is complete in the entire Core Subsystem and the ASL compiler. All new ACPI 2.0 operators are implemented and all other changes for ACPI 2.0 support are complete. With simultaneous code and data optimizations throughout the subsystem, ACPI 2.0 support has been implemented with almost no additional cost in terms of code and data size. Implemented a new mechanism for allocation of return buffers. If the buffer length is set to ACPI_ALLOCATE_BUFFER, the buffer will be allocated on behalf of the caller. Consolidated all return buffer validation and allocation to a common procedure. Return buffers will be allocated via the primary OSL allocation interface since it appears that a separate pool is not needed by most users. If a separate pool is required for these buffers, the caller can still use the original mechanism and pre-allocate the buffer(s). Implemented support for string operands within the DerefOf operator. Restructured the Hardware and Event managers to be table driven, simplifying the source code and reducing the amount of generated code. Split the common read/write low-level ACPI register bitfield procedure into a separate read and write, simplifying the code considerably. Obsoleted the AcpiOsCallocate OSL interface. This interface was used only a handful of times and didn't have enough critical mass for a separate interface. Replaced with a common calloc procedure in the core. Fixed a reported problem with the GPE number mapping mechanism that allows GPE1 numbers to be non-contiguous with GPE0. Reorganized the GPE information and shrunk a large array that was originally large enough to hold info for all possible GPEs (256) to simply large enough to hold all GPEs up to the largest GPE number on the machine. Fixed a reported problem with resource structure alignment on 64- bit platforms. Changed the AcpiEnableEvent and AcpiDisableEvent external interfaces to not require any flags for the common case of enabling/disabling a GPE. Implemented support to allow a "Notify" on a Processor object. Most TBDs in comments within the source code have been resolved and eliminated. Fixed a problem in the interpreter where a standalone parent prefix (^) was not handled correctly in the interpreter and debugger. Removed obsolete and unnecessary GPE save/restore code. Implemented Field support in the ASL Load operator. This allows a table to be loaded from a named field, in addition to loading a table directly from an Operation Region. Implemented timeout and handle support in the external Global Lock interfaces. Fixed a problem in the AcpiDump utility where pathnames were no longer being generated correctly during the dump of named objects. Modified the AML debugger to give a full display of if/while predicates instead of just one AML opcode at a time. (The predicate can have several nested ASL statements.) The old method was confusing during single stepping. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (12_18_01) Non-Debug Version: 66.1K Code, 5.5K Data, 71.6K Total Debug Version: 138.3K Code, 55.9K Data, 194.2K Total Current Release: Non-Debug Version: 65.2K Code, 6.2K Data, 71.4K Total Debug Version: 136.9K Code, 56.4K Data, 193.3K Total 2) Linux Implemented fix for PIIX reverse throttling errata (Processor driver) Added new Limit interface (Processor and Thermal drivers) New thermal policy (Thermal driver) Many updates to /proc Battery "low" event support (Battery driver) Supports ACPI PCI IRQ routing (PCI Link and PCI root drivers) IA32 - IA64 initialization unification, no longer experimental Menuconfig options redesigned 3) ASL Compiler, version X2037: Implemented several new output features to simplify integration of AML code into firmware: 1) Output the AML in C source code with labels for each named ASL object. The original ASL source code is interleaved as C comments. 2) Output the AML in ASM source code with labels and interleaved ASL source. 3) Output the AML in raw hex table form, in either C or ASM. Implemented support for optional string parameters to the LoadTable operator. Completed support for embedded escape sequences within string literals. The compiler now supports all single character escapes as well as the Octal and Hex escapes. Note: the insertion of a null byte into a string literal (via the hex/octal escape) causes the string to be immediately terminated. A warning is issued. Fixed a problem where incorrect AML was generated for the case where an ASL namepath consists of a single parent prefix ( ) with no trailing name segments. The compiler has been successfully generated with a 64-bit C compiler. ---------------------------------------- Summary of changes for this label: 12_18_01 1) Linux Enhanced blacklist with reason and severity fields. Any table's signature may now be used to identify a blacklisted system. Call _PIC control method to inform the firmware which interrupt model the OS is using. Turn on any disabled link devices. Cleaned up busmgr /proc error handling (Andreas Dilger) 2) ACPI CA Core Subsystem: Implemented ACPI 2.0 semantics for the "Break" operator (Exit from while loop) Completed implementation of the ACPI 2.0 "Continue", "ConcatenateResTemplate", "DataTableRegion", and "LoadTable" operators. All new ACPI 2.0 operators are now implemented in both the ASL compiler and the AML interpreter. The only remaining ACPI 2.0 task is support for the String data type in the DerefOf operator. Fixed a problem with AcquireMutex where the status code was lost if the caller had to actually wait for the mutex. Increased the maximum ASL Field size from 64K bits to 4G bits. Completed implementation of the external Global Lock interfaces -- AcpiAcquireGlobalLock and AcpiReleaseGlobalLock. The Timeout and Handler parameters were added. Completed another pass at removing warnings and issues when compiling with 64-bit compilers. The code now compiles cleanly with the Intel 64-bit C/C++ compiler. Most notably, the pointer add and subtract (diff) macros have changed considerably. Created and deployed a new ACPI_SIZE type that is 64-bits wide on 64-bit platforms, 32-bits on all others. This type is used wherever memory allocation and/or the C sizeof() operator is used, and affects the OSL memory allocation interfaces AcpiOsAllocate and AcpiOsCallocate. Implemented sticky user breakpoints in the AML debugger. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (12_05_01) Non-Debug Version: 64.7K Code, 5.3K Data, 70.0K Total Debug Version: 136.2K Code, 55.6K Data, 191.8K Total Current Release: Non-Debug Version: 66.1K Code, 5.5K Data, 71.6K Total Debug Version: 138.3K Code, 55.9K Data, 194.2K Total 3) ASL Compiler, version X2034: Now checks for (and generates an error if detected) the use of a Break or Continue statement without an enclosing While statement. Successfully generated the compiler with the Intel 64-bit C compiler. ---------------------------------------- Summary of changes for this label: 12_05_01 1) ACPI CA Core Subsystem: The ACPI 2.0 CopyObject operator is fully implemented. This operator creates a new copy of an object (and is also used to bypass the "implicit conversion" mechanism of the Store operator.) The ACPI 2.0 semantics for the SizeOf operator are fully implemented. The change is that performing a SizeOf on a reference object causes an automatic dereference of the object to tha actual value before the size is evaluated. This behavior was undefined in ACPI 1.0. The ACPI 2.0 semantics for the Extended IRQ resource descriptor have been implemented. The interrupt polarity and mode are now independently set. Fixed a problem where ASL Constants (Zero, One, Ones, Revision) appearing in Package objects were not properly converted to integers when the internal Package was converted to an external object (via the AcpiEvaluateObject interface.) Fixed a problem with the namespace object deletion mechanism for objects created by control methods. There were two parts to this problem: 1) Objects created during the initialization phase method parse were not being deleted, and 2) The object owner ID mechanism to track objects was broken. Fixed a problem where the use of the ASL Scope operator within a control method would result in an invalid opcode exception. Fixed a problem introduced in the previous label where the buffer length required for the _PRT structure was not being returned correctly. Code and Data Size: Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (11_20_01) Non-Debug Version: 64.1K Code, 5.3K Data, 69.4K Total Debug Version: 135.1K Code, 55.4K Data, 190.5K Total Current Release: Non-Debug Version: 64.7K Code, 5.3K Data, 70.0K Total Debug Version: 136.2K Code, 55.6K Data, 191.8K Total 2) Linux: Updated all files to apply cleanly against 2.4.16. Added basic PCI Interrupt Routing Table (PRT) support for IA32 (acpi_pci.c), and unified the PRT code for IA32 and IA64. This version supports both static and dynamic PRT entries, but dynamic entries are treated as if they were static (not yet reconfigurable). Architecture- specific code to use this data is absent on IA32 but should be available shortly. Changed the initialization sequence to start the ACPI interpreter (acpi_init) prior to initialization of the PCI driver (pci_init) in init/main.c. This ordering is required to support PRT and facilitate other (future) enhancement. A side effect is that the ACPI bus driver and certain device drivers can no longer be loaded as modules. Modified the 'make menuconfig' options to allow PCI Interrupt Routing support to be included without the ACPI Bus and other device drivers. 3) ASL Compiler, version X2033: Fixed some issues with the use of the new CopyObject and DataTableRegion operators. Both are fully functional. ---------------------------------------- Summary of changes for this label: 11_20_01 20 November 2001. Summary of changes for this release. 1) ACPI CA Core Subsystem: Updated Index support to match ACPI 2.0 semantics. Storing a Integer, String, or Buffer to an Index of a Buffer will store only the least-significant byte of the source to the Indexed buffer byte. Multiple writes are not performed. Fixed a problem where the access type used in an AccessAs ASL operator was not recorded correctly into the field object. Fixed a problem where ASL Event objects were created in a signalled state. Events are now created in an unsignalled state. The internal object cache is now purged after table loading and initialization to reduce the use of dynamic kernel memory -- on the assumption that object use is greatest during the parse phase of the entire table (versus the run-time use of individual control methods.) ACPI 2.0 variable-length packages are now fully operational. Code and Data Size: Code and Data optimizations have permitted new feature development with an actual reduction in the library size. Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (11_09_01): Non-Debug Version: 63.7K Code, 5.2K Data, 68.9K Total Debug Version: 134.5K Code, 55.4K Data, 189.9K Total Current Release: Non-Debug Version: 64.1K Code, 5.3K Data, 69.4K Total Debug Version: 135.1K Code, 55.4K Data, 190.5K Total 2) Linux: Enhanced the ACPI boot-time initialization code to allow the use of Local APIC tables for processor enumeration on IA-32, and to pave the way for a fully MPS-free boot (on SMP systems) in the near future. This functionality replaces arch/i386/kernel/acpitables.c, which was introduced in an earlier 2.4.15-preX release. To enable this feature you must add "acpi_boot=on" to the kernel command line -- see the help entry for CONFIG_ACPI_BOOT for more information. An IA-64 release is in the works... Restructured the configuration options to allow boot-time table parsing support without inclusion of the ACPI Interpreter (and other) code. NOTE: This release does not include fixes for the reported events, power-down, and thermal passive cooling issues (coming soon). 3) ASL Compiler: Added additional typechecking for Fields within restricted access Operation Regions. All fields within EC and CMOS regions must be declared with ByteAcc. All fields within SMBus regions must be declared with the BufferAcc access type. Fixed a problem where the listing file output of control methods no longer interleaved the actual AML code with the ASL source code. ---------------------------------------- Summary of changes for this label: 11_09_01 1) ACPI CA Core Subsystem: Implemented ACPI 2.0-defined support for writes to fields with a Buffer, String, or Integer source operand that is smaller than the target field. In these cases, the source operand is zero-extended to fill the target field. Fixed a problem where a Field starting bit offset (within the parent operation region) was calculated incorrectly if the alignment of the field differed from the access width. This affected CreateWordField, CreateDwordField, CreateQwordField, and possibly other fields that use the "AccessAny" keyword. Fixed a problem introduced in the 11_02_01 release where indirect stores through method arguments did not operate correctly. 2) Linux: Implemented boot-time ACPI table parsing support (CONFIG_ACPI_BOOT) for IA32 and IA64 UP/SMP systems. This code facilitates the use of ACPI tables (e.g. MADT, SRAT) rather than legacy BIOS interfaces (e.g. MPS) for the configuration of system processors, memory, and interrupts during setup_arch(). Note that this patch does not include the required architecture-specific changes required to apply this information -- subsequent patches will be posted for both IA32 and IA64 to achieve this. Added low-level sleep support for IA32 platforms, courtesy of Pat Mochel. This allows IA32 systems to transition to/from various sleeping states (e.g. S1, S3), although the lack of a centralized driver model and power-manageable drivers will prevent its (successful) use on most systems. Revamped the ACPI 'menuconfig' layout: created new "ACPI Support" submenu, unified IA32 and IA64 options, added new "Boot using ACPI tables" option, etc. Increased the default timeout for the EC driver from 1ms to 10ms (1000 cycles of 10us) to try to address AE_TIME errors during EC transactions. ---------------------------------------- Summary of changes for this label: 11_02_01 1) ACPI CA Core Subsystem: ACPI 2.0 Support: Implemented ACPI 2.0 64-bit Field access (QWordAcc keyword). All ACPI 2.0 64-bit support is now implemented. OSL Interfaces: Several of the OSL (AcpiOs*) interfaces required changes to support ACPI 2.0 Qword field access. Read/Write PciConfiguration(), Read/Write Memory(), and Read/Write Port() now accept an ACPI_INTEGER (64 bits) as the value parameter. Also, the value parameter for the address space handler interface is now an ACPI_INTEGER. OSL implementations of these interfaces must now handle the case where the Width parameter is 64. Index Fields: Fixed a problem where unaligned bit assembly and disassembly for IndexFields was not supported correctly. Index and Bank Fields: Nested Index and Bank Fields are now supported. During field access, a check is performed to ensure that the value written to an Index or Bank register is not out of the range of the register. The Index (or Bank) register is written before each access to the field data. Future support will include allowing individual IndexFields to be wider than the DataRegister width. Fields: Fixed a problem where the AML interpreter was incorrectly attempting to write beyond the end of a Field/OpRegion. This was a boundary case that occurred when a DWORD field was written to a BYTE access OpRegion, forcing multiple writes and causing the interpreter to write one datum too many. Fields: Fixed a problem with Field/OpRegion access where the starting bit address of a field was incorrectly calculated if the current access type was wider than a byte (WordAcc, DwordAcc, or QwordAcc). Fields: Fixed a problem where forward references to individual FieldUnits (individual Field names within a Field definition) were not resolved during the AML table load. Fields: Fixed a problem where forward references from a Field definition to the parent Operation Region definition were not resolved during the AML table load. Fields: Duplicate FieldUnit names within a scope are now detected during AML table load. Acpi Interfaces: Fixed a problem where the AcpiGetName() interface returned an incorrect name for the root node. Code and Data Size: Code and Data optimizations have permitted new feature development with an actual reduction in the library size. Current core subsystem library sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the debug output trace mechanism and has a larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Release (10_18_01): Non-Debug Version: 63.9K Code, 5.1K Data, 69.0K Total Debug Version: 136.7K Code, 57.4K Data, 194.2K Total Current Release: Non-Debug Version: 63.7K Code, 5.2K Data, 68.9K Total Debug Version: 134.5K Code, 55.4K Data, 189.9K Total 2) Linux: Improved /proc processor output (Pavel Machek) Re-added MODULE_LICENSE("GPL") to all modules. 3) ASL Compiler version X2030: Duplicate FieldUnit names within a scope are now detected and flagged as errors. 4) Documentation: Programmer Reference updated to reflect OSL and address space handler interface changes described above. ---------------------------------------- Summary of changes for this label: 10_18_01 ACPI CA Core Subsystem: Fixed a problem with the internal object reference count mechanism that occasionally caused premature object deletion. This resolves all of the outstanding problem reports where an object is deleted in the middle of an interpreter evaluation. Although this problem only showed up in rather obscure cases, the solution to the problem involved an adjustment of all reference counts involving objects attached to namespace nodes. Fixed a problem with Field support in the interpreter where writing to an aligned field whose length is an exact multiple (2 or greater) of the field access granularity would cause an attempt to write beyond the end of the field. The top level AML opcode execution functions within the interpreter have been renamed with a more meaningful and consistent naming convention. The modules exmonad.c and exdyadic.c were eliminated. New modules are exoparg1.c, exoparg2.c, exoparg3.c, and exoparg6.c. Support for the ACPI 2.0 "Mid" ASL operator has been implemented. Fixed a problem where the AML debugger was causing some internal objects to not be deleted during subsystem termination. Fixed a problem with the external AcpiEvaluateObject interface where the subsystem would fault if the named object to be evaluated referred to a constant such as Zero, Ones, etc. Fixed a problem with IndexFields and BankFields where the subsystem would fault if the index, data, or bank registers were not defined in the same scope as the field itself. Added printf format string checking for compilers that support this feature. Corrected more than 50 instances of issues with format specifiers within invocations of ACPI_DEBUG_PRINT throughout the core subsystem code. The ASL "Revision" operator now returns the ACPI support level implemented in the core - the value "2" since the ACPI 2.0 support is more than 50% implemented. Enhanced the output of the AML debugger "dump namespace" command to output in a more human-readable form. Current core subsystem library code sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the full debug trace mechanism -- leading to a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Previous Label (09_20_01): Non-Debug Version: 65K Code, 5K Data, 70K Total Debug Version: 138K Code, 58K Data, 196K Total This Label: Non-Debug Version: 63.9K Code, 5.1K Data, 69.0K Total Debug Version: 136.7K Code, 57.4K Data, 194.2K Total Linux: Implemented a "Bad BIOS Blacklist" to track machines that have known ASL/AML problems. Enhanced the /proc interface for the thermal zone driver and added support for _HOT (the critical suspend trip point). The 'info' file now includes threshold/policy information, and allows setting of _SCP (cooling preference) and _TZP (polling frequency) values to the 'info' file. Examples: "echo tzp=5 > info" sets the polling frequency to 5 seconds, and "echo scp=1 > info" sets the cooling preference to the passive/quiet mode (if supported by the ASL). Implemented a workaround for a gcc bug that resuted in an OOPs when loading the control method battery driver. ---------------------------------------- Summary of changes for this label: 09_20_01 ACPI CA Core Subsystem: The AcpiEnableEvent and AcpiDisableEvent interfaces have been modified to allow individual GPE levels to be flagged as wake- enabled (i.e., these GPEs are to remain enabled when the platform sleeps.) The AcpiEnterSleepState and AcpiLeaveSleepState interfaces now support wake-enabled GPEs. This means that upon entering the sleep state, all GPEs that are not wake-enabled are disabled. When leaving the sleep state, these GPEs are re-enabled. A local double-precision divide/modulo module has been added to enhance portability to OS kernels where a 64-bit math library is not available. The new module is "utmath.c". Several optimizations have been made to reduce the use of CPU stack. Originally over 2K, the maximum stack usage is now below 2K at 1860 bytes (1.82k) Fixed a problem with the AcpiGetFirmwareTable interface where the root table pointer was not mapped into a logical address properly. Fixed a problem where a NULL pointer was being dereferenced in the interpreter code for the ASL Notify operator. Fixed a problem where the use of the ASL Revision operator returned an error. This operator now returns the current version of the ACPI CA core subsystem. Fixed a problem where objects passed as control method parameters to AcpiEvaluateObject were always deleted at method termination. However, these objects may end up being stored into the namespace by the called method. The object reference count mechanism was applied to these objects instead of a force delete. Fixed a problem where static strings or buffers (contained in the AML code) that are declared as package elements within the ASL code could cause a fault because the interpreter would attempt to delete them. These objects are now marked with the "static object" flag to prevent any attempt to delete them. Implemented an interpreter optimization to use operands directly from the state object instead of extracting the operands to local variables. This reduces stack use and code size, and improves performance. The module exxface.c was eliminated as it was an unnecessary extra layer of code. Current core subsystem library code sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the full debug trace mechanism -- leading to a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Non-Debug Version: 65K Code, 5K Data, 70K Total (Previously 69K) Debug Version: 138K Code, 58K Data, 196K Total (Previously 195K) Linux: Support for ACPI 2.0 64-bit integers has been added. All ACPI Integer objects are now 64 bits wide All Acpi data types and structures are now in lower case. Only Acpi macros are upper case for differentiation. Documentation: Changes to the external interfaces as described above. ---------------------------------------- Summary of changes for this label: 08_31_01 ACPI CA Core Subsystem: A bug with interpreter implementation of the ASL Divide operator was found and fixed. The implicit function return value (not the explicit store operands) was returning the remainder instead of the quotient. This was a longstanding bug and it fixes several known outstanding issues on various platforms. The ACPI_DEBUG_PRINT and function trace entry/exit macros have been further optimized for size. There are 700 invocations of the DEBUG_PRINT macro alone, so each optimization reduces the size of the debug version of the subsystem significantly. A stack trace mechanism has been implemented. The maximum stack usage is about 2K on 32-bit platforms. The debugger command "stat stack" will display the current maximum stack usage. All public symbols and global variables within the subsystem are now prefixed with the string "Acpi". This keeps all of the symbols grouped together in a kernel map, and avoids conflicts with other kernel subsystems. Most of the internal fixed lookup tables have been moved into the code segment via the const operator. Several enhancements have been made to the interpreter to both reduce the code size and improve performance. Current core subsystem library code sizes are shown below. These are the code and data sizes for the acpica.lib produced by the Microsoft Visual C++ 6.0 compiler, and these values do not include any ACPI driver or OSPM code. The debug version of the code includes the full debug trace mechanism which contains over 700 invocations of the DEBUG_PRINT macro, 500 function entry macro invocations, and over 900 function exit macro invocations -- leading to a much larger code and data size. Note that these values will vary depending on the efficiency of the compiler and the compiler options used during generation. Non-Debug Version: 64K Code, 5K Data, 69K Total Debug Version: 137K Code, 58K Data, 195K Total Linux: Implemented wbinvd() macro, pending a kernel-wide definition. Fixed /proc/acpi/event to handle poll() and short reads. ASL Compiler, version X2026: Fixed a problem introduced in the previous label where the AML code emitted for package objects produced packages with zero length. ---------------------------------------- Summary of changes for this label: 08_16_01 ACPI CA Core Subsystem: The following ACPI 2.0 ASL operators have been implemented in the AML interpreter (These are already supported by the Intel ASL compiler): ToDecimalString, ToHexString, ToString, ToInteger, and ToBuffer. Support for 64-bit AML constants is implemented in the AML parser, debugger, and disassembler. The internal memory tracking mechanism (leak detection code) has been upgraded to reduce the memory overhead (a separate tracking block is no longer allocated for each memory allocation), and now supports all of the internal object caches. The data structures and code for the internal object caches have been coelesced and optimized so that there is a single cache and memory list data structure and a single group of functions that implement generic cache management. This has reduced the code size in both the debug and release versions of the subsystem. The DEBUG_PRINT macro(s) have been optimized for size and replaced by ACPI_DEBUG_PRINT. The syntax for this macro is slightly different, because it generates a single call to an internal function. This results in a savings of about 90 bytes per invocation, resulting in an overall code and data savings of about 16% in the debug version of the subsystem. Linux: Fixed C3 disk corruption problems and re-enabled C3 on supporting machines. Integrated low-level sleep code by Patrick Mochel. Further tweaked source code Linuxization. Other minor fixes. ASL Compiler: Support for ACPI 2.0 variable length packages is fixed/completed. Fixed a problem where the optional length parameter for the ACPI 2.0 ToString operator. Fixed multiple extraneous error messages when a syntax error is detected within the declaration line of a control method. ---------------------------------------- Summary of changes for this label: 07_17_01 ACPI CA Core Subsystem: Added a new interface named AcpiGetFirmwareTable to obtain any ACPI table via the ACPI signature. The interface can be called at any time during kernel initialization, even before the kernel virtual memory manager is initialized and paging is enabled. This allows kernel subsystems to obtain ACPI tables very early, even before the ACPI CA subsystem is initialized. Fixed a problem where Fields defined with the AnyAcc attribute could be resolved to the incorrect address under the following conditions: 1) the field width is larger than 8 bits and 2) the parent operation region is not defined on a DWORD boundary. Fixed a problem where the interpreter is not being locked during namespace initialization (during execution of the _INI control methods), causing an error when an attempt is made to release it later. ACPI 2.0 support in the AML Interpreter has begun and will be ongoing throughout the rest of this year. In this label, The Mod operator is implemented. Added a new data type to contain full PCI addresses named ACPI_PCI_ID. This structure contains the PCI Segment, Bus, Device, and Function values. Linux: Enhanced the Linux version of the source code to change most capitalized ACPI type names to lowercase. For example, all instances of ACPI_STATUS are changed to acpi_status. This will result in a large diff, but the change is strictly cosmetic and aligns the CA code closer to the Linux coding standard. OSL Interfaces: The interfaces to the PCI configuration space have been changed to add the PCI Segment number and to split the single 32-bit combined DeviceFunction field into two 16-bit fields. This was accomplished by moving the four values that define an address in PCI configuration space (segment, bus, device, and function) to the new ACPI_PCI_ID structure. The changes to the PCI configuration space interfaces led to a reexamination of the complete set of address space access interfaces for PCI, I/O, and Memory. The previously existing 18 interfaces have proven difficult to maintain (any small change must be propagated across at least 6 interfaces) and do not easily allow for future expansion to 64 bits if necessary. Also, on some systems, it would not be appropriate to demultiplex the access width (8, 16, 32,or 64) before calling the OSL if the corresponding native OS interfaces contain a similar access width parameter. For these reasons, the 18 address space interfaces have been replaced by these 6 new ones: AcpiOsReadPciConfiguration AcpiOsWritePciConfiguration AcpiOsReadMemory AcpiOsWriteMemory AcpiOsReadPort AcpiOsWritePort Added a new interface named AcpiOsGetRootPointer to allow the OSL to perform the platform and/or OS-specific actions necessary to obtain the ACPI RSDP table pointer. On IA-32 platforms, this interface will simply call down to the CA core to perform the low- memory search for the table. On IA-64, the RSDP is obtained from EFI. Migrating this interface to the OSL allows the CA core to remain OS and platform independent. Added a new interface named AcpiOsSignal to provide a generic "function code and pointer" interface for various miscellaneous signals and notifications that must be made to the host OS. The first such signals are intended to support the ASL Fatal and Breakpoint operators. In the latter case, the AcpiOsBreakpoint interface has been obsoleted. The definition of the AcpiFormatException interface has been changed to simplify its use. The caller no longer must supply a buffer to the call; A pointer to a const string is now returned directly. This allows the call to be easily used in printf statements, etc. since the caller does not have to manage a local buffer. ASL Compiler, Version X2025: The ACPI 2.0 Switch/Case/Default operators have been implemented and are fully functional. They will work with all ACPI 1.0 interpreters, since the operators are simply translated to If/Else pairs. The ACPI 2.0 ElseIf operator is implemented and will also work with 1.0 interpreters, for the same reason. Implemented support for ACPI 2.0 variable-length packages. These packages have a separate opcode, and their size is determined by the interpreter at run-time. Documentation The ACPI CA Programmer Reference has been updated to reflect the new interfaces and changes to existing interfaces. ------------------------------------------ Summary of changes for this label: 06_15_01 ACPI CA Core Subsystem: Fixed a problem where a DWORD-accessed field within a Buffer object would get its byte address inadvertently rounded down to the nearest DWORD. Buffers are always Byte-accessible. ASL Compiler, version X2024: Fixed a problem where the Switch() operator would either fault or hang the compiler. Note however, that the AML code for this ACPI 2.0 operator is not yet implemented. Compiler uses the new AcpiOsGetTimer interface to obtain compile timings. Implementation of the CreateField operator automatically converts a reference to a named field within a resource descriptor from a byte offset to a bit offset if required. Added some missing named fields from the resource descriptor support. These are the names that are automatically created by the compiler to reference fields within a descriptor. They are only valid at compile time and are not passed through to the AML interpreter. Resource descriptor named fields are now typed as Integers and subject to compile-time typechecking when used in expressions. ------------------------------------------ Summary of changes for this label: 05_18_01 ACPI CA Core Subsystem: Fixed a couple of problems in the Field support code where bits from adjacent fields could be returned along with the proper field bits. Restructured the field support code to improve performance, readability and maintainability. New DEBUG_PRINTP macro automatically inserts the procedure name into the output, saving hundreds of copies of procedure name strings within the source, shrinking the memory footprint of the debug version of the core subsystem. Source Code Structure: The source code directory tree was restructured to reflect the current organization of the component architecture. Some files and directories have been moved and/or renamed. Linux: Fixed leaking kacpidpc processes. Fixed queueing event data even when /proc/acpi/event is not opened. ASL Compiler, version X2020: Memory allocation performance enhancement - over 24X compile time improvement on large ASL files. Parse nodes and namestring buffers are now allocated from a large internal compiler buffer. The temporary .SRC file is deleted unless the "-s" option is specified The "-d" debug output option now sends all output to the .DBG file instead of the console. "External" second parameter is now optional "ElseIf" syntax now properly allows the predicate Last operand to "Load" now recognized as a Target operand Debug object can now be used anywhere as a normal object. ResourceTemplate now returns an object of type BUFFER EISAID now returns an object of type INTEGER "Index" now works with a STRING operand "LoadTable" now accepts optional parameters "ToString" length parameter is now optional "Interrupt (ResourceType," parse error fixed. "Register" with a user-defined region space parse error fixed Escaped backslash at the end of a string ("\\") scan/parse error fixed "Revision" is now an object of type INTEGER. ------------------------------------------ Summary of changes for this label: 05_02_01 Linux: /proc/acpi/event now blocks properly. Removed /proc/sys/acpi. You can still dump your DSDT from /proc/acpi/dsdt. ACPI CA Core Subsystem: Fixed a problem introduced in the previous label where some of the "small" resource descriptor types were not recognized. Improved error messages for the case where an ASL Field is outside the range of the parent operation region. ASL Compiler, version X2018: Added error detection for ASL Fields that extend beyond the length of the parent operation region (only if the length of the region is known at compile time.) This includes fields that have a minimum access width that is smaller than the parent region, and individual field units that are partially or entirely beyond the extent of the parent. ------------------------------------------ Summary of changes for this label: 04_27_01 ACPI CA Core Subsystem: Fixed a problem where the namespace mutex could be released at the wrong time during execution of AcpiRemoveAddressSpaceHandler. Added optional thread ID output for debug traces, to simplify debugging of multiple threads. Added context switch notification when the debug code realizes that a different thread is now executing ACPI code. Some additional external data types have been prefixed with the string "ACPI_" for consistency. This may effect existing code. The data types affected are the external callback typedefs - e.g., WALK_CALLBACK becomes ACPI_WALK_CALLBACK. Linux: Fixed an issue with the OSL semaphore implementation where a thread was waking up with an error from receiving a SIGCHLD signal. Linux version of ACPI CA now uses the system C library for string manipulation routines instead of a local implementation. Cleaned up comments and removed TBDs. ASL Compiler, version X2017: Enhanced error detection and reporting for all file I/O operations. Documentation: Programmer Reference updated to version 1.06. ------------------------------------------ Summary of changes for this label: 04_13_01 ACPI CA Core Subsystem: Restructured support for BufferFields and RegionFields. BankFields support is now fully operational. All known 32-bit limitations on field sizes have been removed. Both BufferFields and (Operation) RegionFields are now supported by the same field management code. Resource support now supports QWORD address and IO resources. The 16/32/64 bit address structures and the Extended IRQ structure have been changed to properly handle Source Resource strings. A ThreadId of -1 is now used to indicate a "mutex not acquired" condition internally and must never be returned by AcpiOsThreadId. This reserved value was changed from 0 since Unix systems allow a thread ID of 0. Linux: Driver code reorganized to enhance portability Added a kernel configuration option to control ACPI_DEBUG Fixed the EC driver to honor _GLK. ASL Compiler, version X2016: Fixed support for the "FixedHw" keyword. Previously, the FixedHw address space was set to 0, not 0x7f as it should be. ------------------------------------------ Summary of changes for this label: 03_13_01 ACPI CA Core Subsystem: During ACPI initialization, the _SB_._INI method is now run if present. Notify handler fix - notifies are deferred until the parent method completes execution. This fixes the "mutex already acquired" issue seen occasionally. Part of the "implicit conversion" rules in ACPI 2.0 have been found to cause compatibility problems with existing ASL/AML. The convert "result-to-target-type" implementation has been removed for stores to method Args and Locals. Source operand conversion is still fully implemented. Possible changes to ACPI 2.0 specification pending. Fix to AcpiRsCalculatePciRoutingTableLength to return correct length. Fix for compiler warnings for 64-bit compiles. Linux: /proc output aligned for easier parsing. Release-version compile problem fixed. New kernel configuration options documented in Configure.help. IBM 600E - Fixed Sleep button may generate "Invalid context" message. OSPM: Power resource driver integrated with bus manager. Fixed kernel fault during active cooling for thermal zones. Source Code: The source code tree has been restructured. ------------------------------------------ Summary of changes for this label: 03_02_01 Linux OS Services Layer (OSL): Major revision of all Linux-specific code. Modularized all ACPI-specific drivers. Added new thermal zone and power resource drivers. Revamped /proc interface (new functionality is under /proc/acpi). New kernel configuration options. Linux known issues: New kernel configuration options not documented in Configure.help yet. Module dependencies not currently implemented. If used, they should be loaded in this order: busmgr, power, ec, system, processor, battery, ac_adapter, button, thermal. Modules will not load if CONFIG_MODVERSION is set. IBM 600E - entering S5 may reboot instead of shutting down. IBM 600E - Sleep button may generate "Invalid context" message. Some systems may fail with "execution mutex already acquired" message. ACPI CA Core Subsystem: Added a new OSL Interface, AcpiOsGetThreadId. This was required for the deadlock detection code. Defined to return a non-zero, 32- bit thread ID for the currently executing thread. May be a non- zero constant integer on single-thread systems. Implemented deadlock detection for internal subsystem mutexes. We may add conditional compilation for this code (debug only) later. ASL/AML Mutex object semantics are now fully supported. This includes multiple acquires/releases by owner and support for the Mutex SyncLevel parameter. A new "Force Release" mechanism automatically frees all ASL Mutexes that have been acquired but not released when a thread exits the interpreter. This forces conformance to the ACPI spec ("All mutexes must be released when an invocation exits") and prevents deadlocked ASL threads. This mechanism can be expanded (later) to monitor other resource acquisitions if OEM ASL code continues to misbehave (which it will). Several new ACPI exception codes have been added for the Mutex support. Recursive method calls are now allowed and supported (the ACPI spec does in fact allow recursive method calls.) The number of recursive calls is subject to the restrictions imposed by the SERIALIZED method keyword and SyncLevel (ACPI 2.0) method parameter. Implemented support for the SyncLevel parameter for control methods (ACPI 2.0 feature) Fixed a deadlock problem when multiple threads attempted to use the interpreter. Fixed a problem where the string length of a String package element was not always set in a package returned from AcpiEvaluateObject. Fixed a problem where the length of a String package element was not always included in the length of the overall package returned from AcpiEvaluateObject. Added external interfaces (Acpi*) to the ACPI debug memory manager. This manager keeps a list of all outstanding allocations, and can therefore detect memory leaks and attempts to free memory blocks more than once. Useful for code such as the power manager, etc. May not be appropriate for device drivers. Performance with the debug code enabled is slow. The ACPI Global Lock is now an optional hardware element. ASL Compiler Version X2015: Integrated changes to allow the compiler to be generated on multiple platforms. Linux makefile added to generate the compiler on Linux Source Code: All platform-specific headers have been moved to their own subdirectory, Include/Platform. New source file added, Interpreter/ammutex.c New header file, Include/acstruct.h Documentation: The programmer reference has been updated for the following new interfaces: AcpiOsGetThreadId AcpiAllocate AcpiCallocate AcpiFree ------------------------------------------ Summary of changes for this label: 02_08_01 Core ACPI CA Subsystem: Fixed a problem where an error was incorrectly returned if the return resource buffer was larger than the actual data (in the resource interfaces). References to named objects within packages are resolved to the full pathname string before packages are returned directly (via the AcpiEvaluateObject interface) or indirectly via the resource interfaces. Linux OS Services Layer (OSL): Improved /proc battery interface. Added C-state debugging output and other miscellaneous fixes. ASL Compiler Version X2014: All defined method arguments can now be used as local variables, including the ones that are not actually passed in as parameters. The compiler tracks initialization of the arguments and issues an exception if they are used without prior assignment (just like locals). The -o option now specifies a filename prefix that is used for all output files, including the AML output file. Otherwise, the default behavior is as follows: 1) the AML goes to the file specified in the DSDT. 2) all other output files use the input source filename as the base. ------------------------------------------ Summary of changes for this label: 01_25_01 Core ACPI CA Subsystem: Restructured the implementation of object store support within the interpreter. This includes support for the Store operator as well as any ASL operators that include a target operand. Partially implemented support for Implicit Result-to-Target conversion. This is when a result object is converted on the fly to the type of an existing target object. Completion of this support is pending further analysis of the ACPI specification concerning this matter. CPU-specific code has been removed from the subsystem (hardware directory). New Power Management Timer functions added Linux OS Services Layer (OSL): Moved system state transition code to the core, fixed it, and modified Linux OSL accordingly. Fixed C2 and C3 latency calculations. We no longer use the compilation date for the version message on initialization, but retrieve the version from AcpiGetSystemInfo(). Incorporated for fix Sony VAIO machines. Documentation: The Programmer Reference has been updated and reformatted. ASL Compiler: Version X2013: Fixed a problem where the line numbering and error reporting could get out of sync in the presence of multiple include files. ------------------------------------------ Summary of changes for this label: 01_15_01 Core ACPI CA Subsystem: Implemented support for type conversions in the execution of the ASL Concatenate operator (The second operand is converted to match the type of the first operand before concatenation.) Support for implicit source operand conversion is partially implemented. The ASL source operand types Integer, Buffer, and String are freely interchangeable for most ASL operators and are converted by the interpreter on the fly as required. Implicit Target operand conversion (where the result is converted to the target type before storing) is not yet implemented. Support for 32-bit and 64-bit BCD integers is implemented. Problem fixed where a field read on an aligned field could cause a read past the end of the field. New exception, AE_AML_NO_RETURN_VALUE, is returned when a method does not return a value, but the caller expects one. (The ASL compiler flags this as a warning.) ASL Compiler: Version X2011: 1. Static typechecking of all operands is implemented. This prevents the use of invalid objects (such as using a Package where an Integer is required) at compile time instead of at interpreter run-time. 2. The ASL source line is printed with ALL errors and warnings. 3. Bug fix for source EOF without final linefeed. 4. Debug option is split into a parse trace and a namespace trace. 5. Namespace output option (-n) includes initial values for integers and strings. 6. Parse-only option added for quick syntax checking. 7. Compiler checks for duplicate ACPI name declarations Version X2012: 1. Relaxed typechecking to allow interchangeability between strings, integers, and buffers. These types are now converted by the interpreter at runtime. 2. Compiler reports time taken by each internal subsystem in the debug output file. ------------------------------------------ Summary of changes for this label: 12_14_00 ASL Compiler: This is the first official release of the compiler. Since the compiler requires elements of the Core Subsystem, this label synchronizes everything. ------------------------------------------ Summary of changes for this label: 12_08_00 Fixed a problem where named references within the ASL definition of both OperationRegions and CreateXXXFields did not work properly. The symptom was an AE_AML_OPERAND_TYPE during initialization of the region/field. This is similar (but not related internally) to the problem that was fixed in the last label. Implemented both 32-bit and 64-bit support for the BCD ASL functions ToBCD and FromBCD. Updated all legal headers to include "2000" in the copyright years. ------------------------------------------ Summary of changes for this label: 12_01_00 Fixed a problem where method invocations within the ASL definition of both OperationRegions and CreateXXXFields did not work properly. The symptom was an AE_AML_OPERAND_TYPE during initialization of the region/field: nsinit-0209: AE_AML_OPERAND_TYPE while getting region arguments [DEBG] ammonad-0284: Exec_monadic2_r/Not: bad operand(s) (0x3005) Fixed a problem where operators with more than one nested subexpression would fail. The symptoms were varied, by mostly AE_AML_OPERAND_TYPE errors. This was actually a rather serious problem that has gone unnoticed until now. Subtract (Add (1,2), Multiply (3,4)) Fixed a problem where AcpiGetHandle didn't quite get fixed in the previous build (The prefix part of a relative path was handled incorrectly). Fixed a problem where Operation Region initialization failed if the operation region name was a "namepath" instead of a simple "nameseg". Symptom was an AE_NO_OPERAND error. Fixed a problem where an assignment to a local variable via the indirect RefOf mechanism only worked for the first such assignment. Subsequent assignments were ignored. ------------------------------------------ Summary of changes for this label: 11_15_00 ACPI 2.0 table support with backwards support for ACPI 1.0 and the 0.71 extensions. Note: although we can read ACPI 2.0 BIOS tables, the AML interpreter does NOT have support for the new 2.0 ASL grammar terms at this time. All ACPI hardware access is via the GAS structures in the ACPI 2.0 FADT. All physical memory addresses across all platforms are now 64 bits wide. Logical address width remains dependent on the platform (i.e., "void *"). AcpiOsMapMemory interface changed to a 64-bit physical address. The AML interpreter integer size is now 64 bits, as per the ACPI 2.0 specification. For backwards compatibility with ACPI 1.0, ACPI tables with a revision number less than 2 use 32-bit integers only. Fixed a problem where the evaluation of OpRegion operands did not always resolve them to numbers properly. ------------------------------------------ Summary of changes for this label: 10_20_00 Fix for CBN_._STA issue. This fix will allow correct access to CBN_ OpRegions when the _STA returns 0x8. Support to convert ACPI constants (Ones, Zeros, One) to actual values before a package object is returned Fix for method call as predicate to if/while construct causing incorrect if/while behavior Fix for Else block package lengths sometimes calculated wrong (if block > 63 bytes) Fix for Processor object length field, was always zero Table load abort if FACP sanity check fails Fix for problem with Scope(name) if name already exists Warning emitted if a named object referenced cannot be found (resolved) during method execution. ------------------------------------------ Summary of changes for this label: 9_29_00 New table initialization interfaces: AcpiInitializeSubsystem no longer has any parameters AcpiFindRootPointer - Find the RSDP (if necessary) AcpiLoadTables (RSDP) - load all tables found at RSDP- >RSDT Obsolete Interfaces AcpiLoadFirmwareTables - replaced by AcpiLoadTables Note: These interface changes require changes to all existing OSDs The PCI_Config default address space handler is always installed at the root namespace object. ------------------------------------------- Summary of changes for this label: 09_15_00 The new initialization architecture is implemented. New interfaces are: AcpiInitializeSubsystem (replaces AcpiInitialize) AcpiEnableSubsystem Obsolete Interfaces: AcpiLoadNamespace (Namespace is automatically loaded when a table is loaded) The ACPI_OPERAND_OBJECT has been optimized to shrink its size from 52 bytes to 32 bytes. There is usually one of these for every namespace object, so the memory savings is significant. Implemented just-in-time evaluation of the CreateField operators. Bug fixes for IA-64 support have been integrated. Additional code review comments have been implemented The so-called "third pass parse" has been replaced by a final walk through the namespace to initialize all operation regions (address spaces) and fields that have not yet been initialized during the execution of the various _INI and REG methods. New file - namespace/nsinit.c ------------------------------------------- Summary of changes for this label: 09_01_00 Namespace manager data structures have been reworked to change the primary object from a table to a single object. This has resulted in dynamic memory savings of 3X within the namespace and 2X overall in the ACPI CA subsystem. Fixed problem where the call to AcpiEvFindPciRootBuses was inadvertently left commented out. Reduced the warning count when generating the source with the GCC compiler. Revision numbers added to each module header showing the SourceSafe version of the file. Please refer to this version number when giving us feedback or comments on individual modules. The main object types within the subsystem have been renamed to clarify their purpose: ACPI_INTERNAL_OBJECT -> ACPI_OPERAND_OBJECT ACPI_GENERIC_OP -> ACPI_PARSE_OBJECT ACPI_NAME_TABLE_ENTRY -> ACPI_NAMESPACE_NODE NOTE: no changes to the initialization sequence are included in this label. ------------------------------------------- Summary of changes for this label: 08_23_00 Fixed problem where TerminateControlMethod was being called multiple times per method Fixed debugger problem where single stepping caused a semaphore to be oversignalled Improved performance through additional parse object caching - added ACPI_EXTENDED_OP type ------------------------------------------- Summary of changes for this label: 08_10_00 Parser/Interpreter integration: Eliminated the creation of complete parse trees for ACPI tables and control methods. Instead, parse subtrees are created and then deleted as soon as they are processed (Either entered into the namespace or executed by the interpreter). This reduces the use of dynamic kernel memory significantly. (about 10X) Exception codes broken into classes and renumbered. Be sure to recompile all code that includes acexcep.h. Hopefully we won't have to renumber the codes again now that they are split into classes (environment, programmer, AML code, ACPI table, and internal). Fixed some additional alignment issues in the Resource Manager subcomponent Implemented semaphore tracking in the AcpiExec utility, and fixed several places where mutexes/semaphores were being unlocked without a corresponding lock operation. There are no known semaphore or mutex "leaks" at this time. Fixed the case where an ASL Return operator is used to return an unnamed package. ------------------------------------------- Summary of changes for this label: 07_28_00 Fixed a problem with the way addresses were calculated in AcpiAmlReadFieldData() and AcpiAmlWriteFieldData(). This problem manifested itself when a Field was created with WordAccess or DwordAccess, but the field unit defined within the Field was less than a Word or Dword. Fixed a problem in AmlDumpOperands() module's loop to pull operands off of the operand stack to display information. The problem manifested itself as a TLB error on 64-bit systems when accessing an operand stack with two or more operands. Fixed a problem with the PCI configuration space handlers where context was getting confused between accesses. This required a change to the generic address space handler and address space setup definitions. Handlers now get both a global handler context (this is the one passed in by the user when executing AcpiInstallAddressSpaceHandler() and a specific region context that is unique to each region (For example, the _ADR, _SEG and _BBN values associated with a specific region). The generic function definitions have changed to the following: typedef ACPI_STATUS (*ADDRESS_SPACE_HANDLER) ( UINT32 Function, UINT32 Address, UINT32 BitWidth, UINT32 *Value, void *HandlerContext, // This used to be void *Context void *RegionContext); // This is an additional parameter typedef ACPI_STATUS (*ADDRESS_SPACE_SETUP) ( ACPI_HANDLE RegionHandle, UINT32 Function, void *HandlerContext, void **RegionContext); // This used to be **ReturnContext ------------------------------------------- Summary of changes for this label: 07_21_00 Major file consolidation and rename. All files within the interpreter have been renamed as well as most header files. This was done to prevent collisions with existing files in the host OSs -- filenames such as "config.h" and "global.h" seem to be quite common. The VC project files have been updated. All makefiles will require modification. The parser/interpreter integration continues in Phase 5 with the implementation of a complete 2-pass parse (the AML is parsed twice) for each table; This avoids the construction of a huge parse tree and therefore reduces the amount of dynamic memory required by the subsystem. Greater use of the parse object cache means that performance is unaffected. Many comments from the two code reviews have been rolled in. The 64-bit alignment support is complete. ------------------------------------------- Summary of changes for this label: 06_30_00 With a nod and a tip of the hat to the technology of yesteryear, we've added support in the source code for 80 column output devices. The code is now mostly constrained to 80 columns or less to support environments and editors that 1) cannot display or print more than 80 characters on a single line, and 2) cannot disable line wrapping. A major restructuring of the namespace data structure has been completed. The result is 1) cleaner and more understandable/maintainable code, and 2) a significant reduction in the dynamic memory requirement for each named ACPI object (almost half). ------------------------------------------- Summary of changes for this label: 06_23_00 Linux support has been added. In order to obtain approval to get the ACPI CA subsystem into the Linux kernel, we've had to make quite a few changes to the base subsystem that will affect all users (all the changes are generic and OS- independent). The effects of these global changes have been somewhat far reaching. Files have been merged and/or renamed and interfaces have been renamed. The major changes are described below. Osd* interfaces renamed to AcpiOs* to eliminate namespace pollution/confusion within our target kernels. All OSD interfaces must be modified to match the new naming convention. Files merged across the subsystem. A number of the smaller source and header files have been merged to reduce the file count and increase the density of the existing files. There are too many to list here. In general, makefiles that call out individual files will require rebuilding. Interpreter files renamed. All interpreter files now have the prefix am* instead of ie* and is*. Header files renamed: The acapi.h file is now acpixf.h. The acpiosd.h file is now acpiosxf.h. We are removing references to the acronym "API" since it is somewhat windowsy. The new name is "external interface" or xface or xf in the filenames.j All manifest constants have been forced to upper case (some were mixed case.) Also, the string "ACPI_" has been prepended to many (not all) of the constants, typedefs, and structs. The globals "DebugLevel" and "DebugLayer" have been renamed "AcpiDbgLevel" and "AcpiDbgLayer" respectively. All other globals within the subsystem are now prefixed with "AcpiGbl_" Internal procedures within the subsystem are now prefixed with "Acpi" (with only a few exceptions). The original two-letter abbreviation for the subcomponent remains after "Acpi" - for example, CmCallocate became AcpiCmCallocate. Added a source code translation/conversion utility. Used to generate the Linux source code, it can be modified to generate other types of source as well. Can also be used to cleanup existing source by removing extraneous spaces and blank lines. Found in tools/acpisrc/* OsdUnMapMemory was renamed to OsdUnmapMemory and then AcpiOsUnmapMemory. (UnMap became Unmap). A "MaxUnits" parameter has been added to AcpiOsCreateSemaphore. When set to one, this indicates that the caller wants to use the semaphore as a mutex, not a counting semaphore. ACPI CA uses both types. However, implementers of this call may want to use different OS primitives depending on the type of semaphore requested. For example, some operating systems provide separate "mutex" and "semaphore" interfaces - where the mutex interface is much faster because it doesn't have all the overhead of a full semaphore implementation. Fixed a deadlock problem where a method that accesses the PCI address space can block forever if it is the first access to the space. ------------------------------------------- Summary of changes for this label: 06_02_00 Support for environments that cannot handle unaligned data accesses (e.g. firmware and OS environments devoid of alignment handler technology namely SAL/EFI and the IA-64 Linux kernel) has been added (via configurable macros) in these three areas: - Transfer of data from the raw AML byte stream is done via byte moves instead of word/dword/qword moves. - External objects are aligned within the user buffer, including package elements (sub- objects). - Conversion of name strings to UINT32 Acpi Names is now done byte-wise. The Store operator was modified to mimic Microsoft's implementation when storing to a Buffer Field. Added a check of the BM_STS bit before entering C3. The methods subdirectory has been obsoleted and removed. A new file, cmeval.c subsumes the functionality. A 16-bit (DOS) version of AcpiExec has been developed. The makefile is under the acpiexec directory. Index: projects/clang1000-import/sys/contrib/dev/acpica/common/adisasm.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/common/adisasm.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/common/adisasm.c (revision 357966) @@ -1,804 +1,819 @@ /****************************************************************************** * * Module Name: adisasm - Application-level disassembler routines * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #include #include #include #include #include +#include #define _COMPONENT ACPI_TOOLS ACPI_MODULE_NAME ("adisasm") /* Local prototypes */ static ACPI_STATUS AdDoExternalFileList ( char *Filename); static ACPI_STATUS AdDisassembleOneTable ( ACPI_TABLE_HEADER *Table, FILE *File, char *Filename, char *DisasmFilename); static ACPI_STATUS AdReparseOneTable ( ACPI_TABLE_HEADER *Table, FILE *File, ACPI_OWNER_ID OwnerId); ACPI_TABLE_DESC LocalTables[1]; ACPI_PARSE_OBJECT *AcpiGbl_ParseOpRoot; /* Stubs for everything except ASL compiler */ #ifndef ACPI_ASL_COMPILER BOOLEAN AcpiDsIsResultUsed ( ACPI_PARSE_OBJECT *Op, ACPI_WALK_STATE *WalkState) { return (TRUE); } ACPI_STATUS AcpiDsMethodError ( ACPI_STATUS Status, ACPI_WALK_STATE *WalkState) { return (Status); } #endif /******************************************************************************* * * FUNCTION: AdInitialize * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: ACPICA and local initialization * ******************************************************************************/ ACPI_STATUS AdInitialize ( void) { ACPI_STATUS Status; /* ACPICA subsystem initialization */ Status = AcpiOsInitialize (); if (ACPI_FAILURE (Status)) { fprintf (stderr, "Could not initialize ACPICA subsystem: %s\n", AcpiFormatException (Status)); return (Status); } Status = AcpiUtInitGlobals (); if (ACPI_FAILURE (Status)) { fprintf (stderr, "Could not initialize ACPICA globals: %s\n", AcpiFormatException (Status)); return (Status); } Status = AcpiUtMutexInitialize (); if (ACPI_FAILURE (Status)) { fprintf (stderr, "Could not initialize ACPICA mutex objects: %s\n", AcpiFormatException (Status)); return (Status); } Status = AcpiNsRootInitialize (); if (ACPI_FAILURE (Status)) { fprintf (stderr, "Could not initialize ACPICA namespace: %s\n", AcpiFormatException (Status)); return (Status); } /* Setup the Table Manager (cheat - there is no RSDT) */ AcpiGbl_RootTableList.MaxTableCount = 1; AcpiGbl_RootTableList.CurrentTableCount = 0; AcpiGbl_RootTableList.Tables = LocalTables; return (AE_OK); } /****************************************************************************** * * FUNCTION: AdAmlDisassemble * * PARAMETERS: Filename - AML input filename * OutToFile - TRUE if output should go to a file * Prefix - Path prefix for output * OutFilename - where the filename is returned * * RETURN: Status * * DESCRIPTION: Disassembler entry point. Disassemble an entire ACPI table. * *****************************************************************************/ ACPI_STATUS AdAmlDisassemble ( BOOLEAN OutToFile, char *Filename, char *Prefix, char **OutFilename) { ACPI_STATUS Status; char *DisasmFilename = NULL; FILE *File = NULL; ACPI_TABLE_HEADER *Table = NULL; ACPI_NEW_TABLE_DESC *ListHead = NULL; /* * Input: AML code from either a file or via GetTables (memory or * registry) */ if (Filename) { /* Get the list of all AML tables in the file */ Status = AcGetAllTablesFromFile (Filename, ACPI_GET_ALL_TABLES, &ListHead); if (ACPI_FAILURE (Status)) { AcpiOsPrintf ("Could not get ACPI tables from %s, %s\n", Filename, AcpiFormatException (Status)); return (Status); } /* Process any user-specified files for external objects */ Status = AdDoExternalFileList (Filename); if (ACPI_FAILURE (Status)) { return (Status); } } else { Status = AdGetLocalTables (); if (ACPI_FAILURE (Status)) { AcpiOsPrintf ("Could not get ACPI tables, %s\n", AcpiFormatException (Status)); return (Status); } if (!AcpiGbl_DmOpt_Disasm) { return (AE_OK); } /* Obtained the local tables, just disassemble the DSDT */ Status = AcpiGetTable (ACPI_SIG_DSDT, 0, &Table); if (ACPI_FAILURE (Status)) { AcpiOsPrintf ("Could not get DSDT, %s\n", AcpiFormatException (Status)); return (Status); } AcpiOsPrintf ("\nDisassembly of DSDT\n"); Prefix = AdGenerateFilename ("dsdt", Table->OemTableId); } /* * Output: ASL code. Redirect to a file if requested */ if (OutToFile) { /* Create/Open a disassembly output file */ DisasmFilename = FlGenerateFilename (Prefix, FILE_SUFFIX_DISASSEMBLY); if (!DisasmFilename) { fprintf (stderr, "Could not generate output filename\n"); Status = AE_ERROR; goto Cleanup; } File = fopen (DisasmFilename, "w+"); if (!File) { fprintf (stderr, "Could not open output file %s\n", DisasmFilename); Status = AE_ERROR; goto Cleanup; } - - AcpiOsRedirectOutput (File); } *OutFilename = DisasmFilename; /* Disassemble all AML tables within the file */ while (ListHead) { Status = AdDisassembleOneTable (ListHead->Table, File, Filename, DisasmFilename); if (ACPI_FAILURE (Status)) { break; } ListHead = ListHead->Next; } Cleanup: if (Table && !AcpiGbl_ForceAmlDisassembly && !AcpiUtIsAmlTable (Table)) { ACPI_FREE (Table); } AcDeleteTableList (ListHead); if (File) { fclose (File); AcpiOsRedirectOutput (stdout); } AcpiPsDeleteParseTree (AcpiGbl_ParseOpRoot); AcpiGbl_ParseOpRoot = NULL; return (Status); } /****************************************************************************** * * FUNCTION: AdDisassembleOneTable * * PARAMETERS: Table - Raw AML table * File - Pointer for the input file * Filename - AML input filename * DisasmFilename - Output filename * * RETURN: Status * * DESCRIPTION: Disassemble a single ACPI table. AML or data table. * *****************************************************************************/ static ACPI_STATUS AdDisassembleOneTable ( ACPI_TABLE_HEADER *Table, FILE *File, char *Filename, char *DisasmFilename) { ACPI_STATUS Status; ACPI_OWNER_ID OwnerId; #ifdef ACPI_ASL_COMPILER /* * For ASL-/ASL+ converter: replace the temporary "XXXX" * table signature with the original. This "XXXX" makes * it harder for the AML interpreter to run the badaml * (.xxx) file produced from the converter in case if * it fails to get deleted. */ if (AcpiGbl_CaptureComments) { strncpy (Table->Signature, AcpiGbl_TableSig, ACPI_NAMESEG_SIZE); } #endif /* ForceAmlDisassembly means to assume the table contains valid AML */ if (!AcpiGbl_ForceAmlDisassembly && !AcpiUtIsAmlTable (Table)) { + if (File) + { + AcpiOsRedirectOutput (File); + } + AdDisassemblerHeader (Filename, ACPI_IS_DATA_TABLE); /* This is a "Data Table" (non-AML table) */ AcpiOsPrintf (" * ACPI Data Table [%4.4s]\n *\n", Table->Signature); AcpiOsPrintf (" * Format: [HexOffset DecimalOffset ByteLength] " "FieldName : FieldValue\n */\n\n"); AcpiDmDumpDataTable (Table); fprintf (stderr, "Acpi Data Table [%4.4s] decoded\n", Table->Signature); if (File) { fprintf (stderr, "Formatted output: %s - %u bytes\n", DisasmFilename, CmGetFileSize (File)); } return (AE_OK); } + /* Initialize the converter output file */ + + ASL_CV_INIT_FILETREE(Table, File); + /* * This is an AML table (DSDT or SSDT). * Always parse the tables, only option is what to display */ Status = AdParseTable (Table, &OwnerId, TRUE, FALSE); if (ACPI_FAILURE (Status)) { AcpiOsPrintf ("Could not parse ACPI tables, %s\n", AcpiFormatException (Status)); return (Status); + } + + /* Redirect output for code generation and debugging output */ + + if (File) + { + AcpiOsRedirectOutput (File); } /* Debug output, namespace and parse tree */ if (AslCompilerdebug && File) { AcpiOsPrintf ("/**** Before second load\n"); NsSetupNamespaceListing (File); NsDisplayNamespace (); AcpiOsPrintf ("*****/\n"); } /* Load namespace from names created within control methods */ AcpiDmFinishNamespaceLoad (AcpiGbl_ParseOpRoot, AcpiGbl_RootNode, OwnerId); /* * Cross reference the namespace here, in order to * generate External() statements */ AcpiDmCrossReferenceNamespace (AcpiGbl_ParseOpRoot, AcpiGbl_RootNode, OwnerId); if (AslCompilerdebug) { AcpiDmDumpTree (AcpiGbl_ParseOpRoot); } /* Find possible calls to external control methods */ AcpiDmFindOrphanMethods (AcpiGbl_ParseOpRoot); /* * If we found any external control methods, we must reparse * the entire tree with the new information (namely, the * number of arguments per method) */ if (AcpiDmGetUnresolvedExternalMethodCount ()) { Status = AdReparseOneTable (Table, File, OwnerId); if (ACPI_FAILURE (Status)) { return (Status); } } /* * Now that the namespace is finalized, we can perform namespace * transforms. * * 1) Convert fixed-offset references to resource descriptors * to symbolic references (Note: modifies namespace) */ AcpiDmConvertParseObjects (AcpiGbl_ParseOpRoot, AcpiGbl_RootNode); /* Optional displays */ if (AcpiGbl_DmOpt_Disasm) { /* This is the real disassembly */ AdDisplayTables (Filename, Table); /* Dump hex table if requested (-vt) */ AcpiDmDumpDataTable (Table); fprintf (stderr, "Disassembly completed\n"); if (File) { fprintf (stderr, "ASL Output: %s - %u bytes\n", DisasmFilename, CmGetFileSize (File)); } if (AslGbl_MapfileFlag) { fprintf (stderr, "%14s %s - %u bytes\n", AslGbl_FileDescs[ASL_FILE_MAP_OUTPUT].ShortDescription, AslGbl_Files[ASL_FILE_MAP_OUTPUT].Filename, FlGetFileSize (ASL_FILE_MAP_OUTPUT)); } } return (AE_OK); } /****************************************************************************** * * FUNCTION: AdReparseOneTable * * PARAMETERS: Table - Raw AML table * File - Pointer for the input file * OwnerId - ID for this table * * RETURN: Status * * DESCRIPTION: Reparse a table that has already been loaded. Used to * integrate information about external control methods. * These methods may have been previously parsed incorrectly. * *****************************************************************************/ static ACPI_STATUS AdReparseOneTable ( ACPI_TABLE_HEADER *Table, FILE *File, ACPI_OWNER_ID OwnerId) { ACPI_STATUS Status; ACPI_COMMENT_ADDR_NODE *AddrListHead; fprintf (stderr, "\nFound %u external control methods, " "reparsing with new information\n", AcpiDmGetUnresolvedExternalMethodCount ()); /* Reparse, rebuild namespace */ AcpiPsDeleteParseTree (AcpiGbl_ParseOpRoot); AcpiGbl_ParseOpRoot = NULL; AcpiNsDeleteNamespaceSubtree (AcpiGbl_RootNode); AcpiGbl_RootNode = NULL; AcpiGbl_RootNodeStruct.Name.Integer = ACPI_ROOT_NAME; AcpiGbl_RootNodeStruct.DescriptorType = ACPI_DESC_TYPE_NAMED; AcpiGbl_RootNodeStruct.Type = ACPI_TYPE_DEVICE; AcpiGbl_RootNodeStruct.Parent = NULL; AcpiGbl_RootNodeStruct.Child = NULL; AcpiGbl_RootNodeStruct.Peer = NULL; AcpiGbl_RootNodeStruct.Object = NULL; AcpiGbl_RootNodeStruct.Flags = 0; Status = AcpiNsRootInitialize (); if (ACPI_FAILURE (Status)) { return (Status); } /* New namespace, add the external definitions first */ AcpiDmAddExternalListToNamespace (); /* For -ca option: clear the list of comment addresses. */ while (AcpiGbl_CommentAddrListHead) { AddrListHead= AcpiGbl_CommentAddrListHead; AcpiGbl_CommentAddrListHead = AcpiGbl_CommentAddrListHead->Next; AcpiOsFree(AddrListHead); } /* Parse the table again. No need to reload it, however */ Status = AdParseTable (Table, NULL, FALSE, FALSE); if (ACPI_FAILURE (Status)) { AcpiOsPrintf ("Could not parse ACPI tables, %s\n", AcpiFormatException (Status)); return (Status); } /* Cross reference the namespace again */ AcpiDmFinishNamespaceLoad (AcpiGbl_ParseOpRoot, AcpiGbl_RootNode, OwnerId); AcpiDmCrossReferenceNamespace (AcpiGbl_ParseOpRoot, AcpiGbl_RootNode, OwnerId); /* Debug output - namespace and parse tree */ if (AslCompilerdebug) { AcpiOsPrintf ("/**** After second load and resource conversion\n"); if (File) { NsSetupNamespaceListing (File); NsDisplayNamespace (); } AcpiOsPrintf ("*****/\n"); AcpiDmDumpTree (AcpiGbl_ParseOpRoot); } return (AE_OK); } /****************************************************************************** * * FUNCTION: AdDoExternalFileList * * PARAMETERS: Filename - Input file for the table * * RETURN: Status * * DESCRIPTION: Process all tables found in the -e external files list * *****************************************************************************/ static ACPI_STATUS AdDoExternalFileList ( char *Filename) { ACPI_EXTERNAL_FILE *ExternalFileList; char *ExternalFilename; ACPI_NEW_TABLE_DESC *ExternalListHead = NULL; ACPI_STATUS Status; ACPI_STATUS GlobalStatus = AE_OK; ACPI_OWNER_ID OwnerId; /* * External filenames are specified on the command line like this: * Example: iasl -e file1,file2,file3 -d xxx.aml */ ExternalFileList = AcpiGbl_ExternalFileList; /* Process each external file */ while (ExternalFileList) { ExternalFilename = ExternalFileList->Path; if (!strcmp (ExternalFilename, Filename)) { /* Next external file */ ExternalFileList = ExternalFileList->Next; continue; } AcpiOsPrintf ("External object resolution file %16s\n", ExternalFilename); Status = AcGetAllTablesFromFile ( ExternalFilename, ACPI_GET_ONLY_AML_TABLES, &ExternalListHead); if (ACPI_FAILURE (Status)) { if (Status == AE_TYPE) { ExternalFileList = ExternalFileList->Next; GlobalStatus = AE_TYPE; continue; } AcDeleteTableList (ExternalListHead); return (Status); } /* Load external tables for symbol resolution */ while (ExternalListHead) { Status = AdParseTable ( ExternalListHead->Table, &OwnerId, TRUE, TRUE); if (ACPI_FAILURE (Status)) { AcpiOsPrintf ("Could not parse external ACPI tables, %s\n", AcpiFormatException (Status)); AcDeleteTableList (ExternalListHead); return (Status); } /* * Load namespace from names created within control methods * Set owner id of nodes in external table */ AcpiDmFinishNamespaceLoad (AcpiGbl_ParseOpRoot, AcpiGbl_RootNode, OwnerId); AcpiPsDeleteParseTree (AcpiGbl_ParseOpRoot); ExternalListHead = ExternalListHead->Next; } /* Next external file */ ExternalFileList = ExternalFileList->Next; } AcDeleteTableList (ExternalListHead); if (ACPI_FAILURE (GlobalStatus)) { return (GlobalStatus); } /* Clear external list generated by Scope in external tables */ if (AcpiGbl_ExternalFileList) { AcpiDmClearExternalList (); } /* Load any externals defined in the optional external ref file */ AcpiDmGetExternalsFromFile (); return (AE_OK); } Index: projects/clang1000-import/sys/contrib/dev/acpica/common/dmtables.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/common/dmtables.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/common/dmtables.c (revision 357966) @@ -1,614 +1,613 @@ /****************************************************************************** * * Module Name: dmtables - disassembler ACPI table support * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #include #include #include #include #include #include #define _COMPONENT ACPI_TOOLS ACPI_MODULE_NAME ("dmtables") /* Local prototypes */ static void AdCreateTableHeader ( char *Filename, ACPI_TABLE_HEADER *Table); static ACPI_STATUS AdStoreTable ( ACPI_TABLE_HEADER *Table, UINT32 *TableIndex); extern ACPI_TABLE_DESC LocalTables[1]; extern ACPI_PARSE_OBJECT *AcpiGbl_ParseOpRoot; /****************************************************************************** * * FUNCTION: AdDisassemblerHeader * * PARAMETERS: Filename - Input file for the table * TableType - Either AML or DataTable * * RETURN: None * * DESCRIPTION: Create the disassembler header, including ACPICA signon with * current time and date. * *****************************************************************************/ void AdDisassemblerHeader ( char *Filename, UINT8 TableType) { time_t Timer; time (&Timer); /* Header and input table info */ AcpiOsPrintf ("/*\n"); AcpiOsPrintf (ACPI_COMMON_HEADER (AML_DISASSEMBLER_NAME, " * ")); if (TableType == ACPI_IS_AML_TABLE) { if (AcpiGbl_CstyleDisassembly) { AcpiOsPrintf ( " * Disassembling to symbolic ASL+ operators\n" " *\n"); } else { AcpiOsPrintf ( " * Disassembling to non-symbolic legacy ASL operators\n" " *\n"); } } AcpiOsPrintf (" * Disassembly of %s, %s", Filename, ctime (&Timer)); AcpiOsPrintf (" *\n"); } /****************************************************************************** * * FUNCTION: AdCreateTableHeader * * PARAMETERS: Filename - Input file for the table * Table - Pointer to the raw table * * RETURN: None * * DESCRIPTION: Create the ASL table header, including ACPICA signon with * current time and date. * *****************************************************************************/ static void AdCreateTableHeader ( char *Filename, ACPI_TABLE_HEADER *Table) { UINT8 Checksum; /* Reset globals for External statements */ AcpiGbl_NumExternalMethods = 0; AcpiGbl_ResolvedExternalMethods = 0; /* * Print file header and dump original table header */ AdDisassemblerHeader (Filename, ACPI_IS_AML_TABLE); AcpiOsPrintf (" * Original Table Header:\n"); AcpiOsPrintf (" * Signature \"%4.4s\"\n", Table->Signature); AcpiOsPrintf (" * Length 0x%8.8X (%u)\n", Table->Length, Table->Length); /* Print and validate the revision */ AcpiOsPrintf (" * Revision 0x%2.2X", Table->Revision); switch (Table->Revision) { case 0: AcpiOsPrintf (" **** Invalid Revision"); break; case 1: /* Revision of DSDT controls the ACPI integer width */ if (ACPI_COMPARE_NAMESEG (Table->Signature, ACPI_SIG_DSDT)) { AcpiOsPrintf (" **** 32-bit table (V1), no 64-bit math support"); } break; default: break; } /* Print and validate the table checksum */ AcpiOsPrintf ("\n * Checksum 0x%2.2X", Table->Checksum); Checksum = AcpiTbChecksum (ACPI_CAST_PTR (UINT8, Table), Table->Length); if (Checksum) { AcpiOsPrintf (" **** Incorrect checksum, should be 0x%2.2X", (UINT8) (Table->Checksum - Checksum)); } AcpiOsPrintf ("\n"); AcpiOsPrintf (" * OEM ID \"%.6s\"\n", Table->OemId); AcpiOsPrintf (" * OEM Table ID \"%.8s\"\n", Table->OemTableId); AcpiOsPrintf (" * OEM Revision 0x%8.8X (%u)\n", Table->OemRevision, Table->OemRevision); AcpiOsPrintf (" * Compiler ID \"%.4s\"\n", Table->AslCompilerId); AcpiOsPrintf (" * Compiler Version 0x%8.8X (%u)\n", Table->AslCompilerRevision, Table->AslCompilerRevision); AcpiOsPrintf (" */\n"); /* * Print comments that come before this definition block. */ if (AcpiGbl_CaptureComments) { ASL_CV_PRINT_ONE_COMMENT(AcpiGbl_ParseOpRoot,AML_COMMENT_STANDARD, NULL, 0); } /* * Open the ASL definition block. * * Note: the AMLFilename string is left zero-length in order to just let * the compiler create it when the disassembled file is compiled. This * makes it easier to rename the disassembled ASL file if needed. */ AcpiOsPrintf ( "DefinitionBlock (\"\", \"%4.4s\", %u, \"%.6s\", \"%.8s\", 0x%8.8X)\n", Table->Signature, Table->Revision, Table->OemId, Table->OemTableId, Table->OemRevision); } /****************************************************************************** * * FUNCTION: AdDisplayTables * * PARAMETERS: Filename - Input file for the table * Table - Pointer to the raw table * * RETURN: Status * * DESCRIPTION: Display (disassemble) loaded tables and dump raw tables * *****************************************************************************/ ACPI_STATUS AdDisplayTables ( char *Filename, ACPI_TABLE_HEADER *Table) { if (!AcpiGbl_ParseOpRoot) { return (AE_NOT_EXIST); } if (!AcpiGbl_DmOpt_Listing) { AdCreateTableHeader (Filename, Table); } AcpiDmDisassemble (NULL, AcpiGbl_ParseOpRoot, ACPI_UINT32_MAX); MpEmitMappingInfo (); if (AcpiGbl_DmOpt_Listing) { AcpiOsPrintf ("\n\nTable Header:\n"); AcpiUtDebugDumpBuffer ((UINT8 *) Table, sizeof (ACPI_TABLE_HEADER), DB_BYTE_DISPLAY, ACPI_UINT32_MAX); AcpiOsPrintf ("Table Body (Length 0x%X)\n", Table->Length); AcpiUtDebugDumpBuffer (((UINT8 *) Table + sizeof (ACPI_TABLE_HEADER)), Table->Length, DB_BYTE_DISPLAY, ACPI_UINT32_MAX); } return (AE_OK); } /******************************************************************************* * * FUNCTION: AdStoreTable * * PARAMETERS: Table - Table header * TableIndex - Where the table index is returned * * RETURN: Status and table index. * * DESCRIPTION: Add an ACPI table to the global table list * ******************************************************************************/ static ACPI_STATUS AdStoreTable ( ACPI_TABLE_HEADER *Table, UINT32 *TableIndex) { ACPI_STATUS Status; ACPI_TABLE_DESC *TableDesc; Status = AcpiTbGetNextTableDescriptor (TableIndex, &TableDesc); if (ACPI_FAILURE (Status)) { return (Status); } /* Initialize added table */ AcpiTbInitTableDescriptor (TableDesc, ACPI_PTR_TO_PHYSADDR (Table), ACPI_TABLE_ORIGIN_INTERNAL_VIRTUAL, Table); Status = AcpiTbValidateTable (TableDesc); return (Status); } /****************************************************************************** * * FUNCTION: AdGetLocalTables * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Get the ACPI tables from either memory or a file * *****************************************************************************/ ACPI_STATUS AdGetLocalTables ( void) { ACPI_STATUS Status; ACPI_TABLE_HEADER TableHeader; ACPI_TABLE_HEADER *NewTable; UINT32 TableIndex; /* Get the DSDT via table override */ ACPI_MOVE_32_TO_32 (TableHeader.Signature, ACPI_SIG_DSDT); Status = AcpiOsTableOverride (&TableHeader, &NewTable); if (ACPI_FAILURE (Status) || !NewTable) { fprintf (stderr, "Could not obtain DSDT\n"); return (AE_NO_ACPI_TABLES); } AdWriteTable (NewTable, NewTable->Length, ACPI_SIG_DSDT, NewTable->OemTableId); /* Store DSDT in the Table Manager */ Status = AdStoreTable (NewTable, &TableIndex); if (ACPI_FAILURE (Status)) { fprintf (stderr, "Could not store DSDT\n"); return (AE_NO_ACPI_TABLES); } return (AE_OK); } /****************************************************************************** * * FUNCTION: AdParseTable * * PARAMETERS: Table - Pointer to the raw table * OwnerId - Returned OwnerId of the table * LoadTable - If add table to the global table list * External - If this is an external table * * RETURN: Status * * DESCRIPTION: Parse an ACPI AML table * *****************************************************************************/ ACPI_STATUS AdParseTable ( ACPI_TABLE_HEADER *Table, ACPI_OWNER_ID *OwnerId, BOOLEAN LoadTable, BOOLEAN External) { ACPI_STATUS Status = AE_OK; ACPI_WALK_STATE *WalkState; UINT8 *AmlStart; UINT32 AmlLength; UINT32 TableIndex; if (!Table) { return (AE_NOT_EXIST); } /* Pass 1: Parse everything except control method bodies */ fprintf (stderr, "Pass 1 parse of [%4.4s]\n", (char *) Table->Signature); AmlLength = Table->Length - sizeof (ACPI_TABLE_HEADER); AmlStart = ((UINT8 *) Table + sizeof (ACPI_TABLE_HEADER)); - ASL_CV_INIT_FILETREE(Table, AmlStart, AmlLength); AcpiUtSetIntegerWidth (Table->Revision); /* Create the root object */ AcpiGbl_ParseOpRoot = AcpiPsCreateScopeOp (AmlStart); if (!AcpiGbl_ParseOpRoot) { return (AE_NO_MEMORY); } #ifdef ACPI_ASL_COMPILER if (AcpiGbl_CaptureComments) { AcpiGbl_ParseOpRoot->Common.CvFilename = AcpiGbl_FileTreeRoot->Filename; } else { AcpiGbl_ParseOpRoot->Common.CvFilename = NULL; } #endif /* Create and initialize a new walk state */ WalkState = AcpiDsCreateWalkState (0, AcpiGbl_ParseOpRoot, NULL, NULL); if (!WalkState) { return (AE_NO_MEMORY); } Status = AcpiDsInitAmlWalk (WalkState, AcpiGbl_ParseOpRoot, NULL, AmlStart, AmlLength, NULL, ACPI_IMODE_LOAD_PASS1); if (ACPI_FAILURE (Status)) { return (Status); } WalkState->ParseFlags &= ~ACPI_PARSE_DELETE_TREE; Status = AcpiPsParseAml (WalkState); if (ACPI_FAILURE (Status)) { return (Status); } /* If LoadTable is FALSE, we are parsing the last loaded table */ TableIndex = AcpiGbl_RootTableList.CurrentTableCount - 1; /* Pass 2 */ if (LoadTable) { Status = AdStoreTable (Table, &TableIndex); if (ACPI_FAILURE (Status)) { return (Status); } Status = AcpiTbAllocateOwnerId (TableIndex); if (ACPI_FAILURE (Status)) { return (Status); } if (OwnerId) { Status = AcpiTbGetOwnerId (TableIndex, OwnerId); if (ACPI_FAILURE (Status)) { return (Status); } } } fprintf (stderr, "Pass 2 parse of [%4.4s]\n", (char *) Table->Signature); Status = AcpiNsOneCompleteParse (ACPI_IMODE_LOAD_PASS2, TableIndex, NULL); if (ACPI_FAILURE (Status)) { return (Status); } /* No need to parse control methods of external table */ if (External) { return (AE_OK); } /* * Pass 3: Parse control methods and link their parse trees * into the main parse tree */ fprintf (stderr, "Parsing Deferred Opcodes (Methods/Buffers/Packages/Regions)\n"); (void) AcpiDmParseDeferredOps (AcpiGbl_ParseOpRoot); fprintf (stderr, "\n"); /* Process Resource Templates */ AcpiDmFindResources (AcpiGbl_ParseOpRoot); fprintf (stderr, "Parsing completed\n"); return (AE_OK); } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslanalyze.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslanalyze.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslanalyze.c (revision 357966) @@ -1,844 +1,852 @@ /****************************************************************************** * * Module Name: aslanalyze.c - Support functions for parse tree walks * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include "aslcompiler.y.h" +#include #include #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("aslanalyze") /* Local Prototypes */ static ACPI_STATUS ApDeviceSubtreeWalk ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); /******************************************************************************* * * FUNCTION: AnIsInternalMethod * * PARAMETERS: Op - Current op * * RETURN: Boolean * * DESCRIPTION: Check for an internal control method. * ******************************************************************************/ BOOLEAN AnIsInternalMethod ( ACPI_PARSE_OBJECT *Op) { if ((!strcmp (Op->Asl.ExternalName, "\\_OSI")) || (!strcmp (Op->Asl.ExternalName, "_OSI"))) { return (TRUE); } return (FALSE); } /******************************************************************************* * * FUNCTION: AnGetInternalMethodReturnType * * PARAMETERS: Op - Current op * * RETURN: Btype * * DESCRIPTION: Get the return type of an internal method * ******************************************************************************/ UINT32 AnGetInternalMethodReturnType ( ACPI_PARSE_OBJECT *Op) { if ((!strcmp (Op->Asl.ExternalName, "\\_OSI")) || (!strcmp (Op->Asl.ExternalName, "_OSI"))) { return (ACPI_BTYPE_STRING); } return (0); } /******************************************************************************* * * FUNCTION: AnCheckId * * PARAMETERS: Op - Current parse op * Type - HID or CID * * RETURN: None * * DESCRIPTION: Perform various checks on _HID and _CID strings. Only limited * checks can be performed on _CID strings. * ******************************************************************************/ void AnCheckId ( ACPI_PARSE_OBJECT *Op, ACPI_NAME Type) { UINT32 i; ACPI_SIZE Length; /* Only care about string versions of _HID/_CID (integers are legal) */ if (Op->Asl.ParseOpcode != PARSEOP_STRING_LITERAL) { return; } /* For both _HID and _CID, the string must be non-null */ Length = strlen (Op->Asl.Value.String); if (!Length) { AslError (ASL_ERROR, ASL_MSG_NULL_STRING, Op, NULL); return; } /* * One of the things we want to catch here is the use of a leading * asterisk in the string -- an odd construct that certain platform * manufacturers are fond of. Technically, a leading asterisk is OK * for _CID, but a valid use of this has not been seen. */ if (*Op->Asl.Value.String == '*') { AslError (ASL_ERROR, ASL_MSG_LEADING_ASTERISK, Op, Op->Asl.Value.String); return; } /* _CID strings are bus-specific, no more checks can be performed */ if (Type == ASL_TYPE_CID) { return; } /* For _HID, all characters must be alphanumeric */ for (i = 0; Op->Asl.Value.String[i]; i++) { if (!isalnum ((int) Op->Asl.Value.String[i])) { AslError (ASL_ERROR, ASL_MSG_ALPHANUMERIC_STRING, Op, Op->Asl.Value.String); return; } } /* * _HID String must be one of these forms: * * "AAA####" A is an uppercase letter and # is a hex digit * "ACPI####" # is a hex digit * "NNNN####" N is an uppercase letter or decimal digit (0-9) * # is a hex digit (ACPI 5.0) */ if ((Length < 7) || (Length > 8)) { AslError (ASL_ERROR, ASL_MSG_HID_LENGTH, Op, Op->Asl.Value.String); return; } /* _HID Length is valid (7 or 8), now check prefix (first 3 or 4 chars) */ if (Length == 7) { /* AAA####: Ensure the alphabetic prefix is all uppercase */ for (i = 0; i < 3; i++) { if (!isupper ((int) Op->Asl.Value.String[i])) { AslError (ASL_ERROR, ASL_MSG_UPPER_CASE, Op, &Op->Asl.Value.String[i]); return; } } } else /* Length == 8 */ { /* * ACPI#### or NNNN####: * Ensure the prefix contains only uppercase alpha or decimal digits */ for (i = 0; i < 4; i++) { if (!isupper ((int) Op->Asl.Value.String[i]) && !isdigit ((int) Op->Asl.Value.String[i])) { AslError (ASL_ERROR, ASL_MSG_HID_PREFIX, Op, &Op->Asl.Value.String[i]); return; } } } /* Remaining characters (suffix) must be hex digits */ for (; i < Length; i++) { if (!isxdigit ((int) Op->Asl.Value.String[i])) { AslError (ASL_ERROR, ASL_MSG_HID_SUFFIX, Op, &Op->Asl.Value.String[i]); break; } } } /******************************************************************************* * * FUNCTION: AnLastStatementIsReturn * * PARAMETERS: Op - A method parse node * * RETURN: TRUE if last statement is an ASL RETURN. False otherwise * * DESCRIPTION: Walk down the list of top level statements within a method * to find the last one. Check if that last statement is in * fact a RETURN statement. * ******************************************************************************/ BOOLEAN AnLastStatementIsReturn ( ACPI_PARSE_OBJECT *Op) { ACPI_PARSE_OBJECT *Next; /* Check if last statement is a return */ Next = ASL_GET_CHILD_NODE (Op); while (Next) { if ((!Next->Asl.Next) && (Next->Asl.ParseOpcode == PARSEOP_RETURN)) { return (TRUE); } Next = ASL_GET_PEER_NODE (Next); } return (FALSE); } /******************************************************************************* * * FUNCTION: AnCheckMethodReturnValue * * PARAMETERS: Op - Parent * OpInfo - Parent info * ArgOp - Method invocation op * RequiredBtypes - What caller requires * ThisNodeBtype - What this node returns (if anything) * * RETURN: None * * DESCRIPTION: Check a method invocation for 1) A return value and if it does * in fact return a value, 2) check the type of the return value. * ******************************************************************************/ void AnCheckMethodReturnValue ( ACPI_PARSE_OBJECT *Op, const ACPI_OPCODE_INFO *OpInfo, ACPI_PARSE_OBJECT *ArgOp, UINT32 RequiredBtypes, UINT32 ThisNodeBtype) { ACPI_PARSE_OBJECT *OwningOp; ACPI_NAMESPACE_NODE *Node; + char *ExternalPath; Node = ArgOp->Asl.Node; if (!Node) { /* No error message, this can happen and is OK */ return; } /* Examine the parent op of this method */ OwningOp = Node->Op; + ExternalPath = AcpiNsGetNormalizedPathname (Node, TRUE); + if (OwningOp->Asl.CompileFlags & OP_METHOD_NO_RETVAL) { /* Method NEVER returns a value */ - AslError (ASL_ERROR, ASL_MSG_NO_RETVAL, Op, Op->Asl.ExternalName); + AslError (ASL_ERROR, ASL_MSG_NO_RETVAL, Op, ExternalPath); } else if (OwningOp->Asl.CompileFlags & OP_METHOD_SOME_NO_RETVAL) { /* Method SOMETIMES returns a value, SOMETIMES not */ - AslError (ASL_WARNING, ASL_MSG_SOME_NO_RETVAL, - Op, Op->Asl.ExternalName); + AslError (ASL_WARNING, ASL_MSG_SOME_NO_RETVAL, Op, ExternalPath); } else if (!(ThisNodeBtype & RequiredBtypes)) { /* Method returns a value, but the type is wrong */ AnFormatBtype (AslGbl_StringBuffer, ThisNodeBtype); AnFormatBtype (AslGbl_StringBuffer2, RequiredBtypes); /* * The case where the method does not return any value at all * was already handled in the namespace cross reference * -- Only issue an error if the method in fact returns a value, * but it is of the wrong type */ if (ThisNodeBtype != 0) { sprintf (AslGbl_MsgBuffer, "Method returns [%s], %s operator requires [%s]", AslGbl_StringBuffer, OpInfo->Name, AslGbl_StringBuffer2); AslError (ASL_ERROR, ASL_MSG_INVALID_TYPE, ArgOp, AslGbl_MsgBuffer); } + } + + if (ExternalPath) + { + ACPI_FREE (ExternalPath); } } /******************************************************************************* * * FUNCTION: AnIsResultUsed * * PARAMETERS: Op - Parent op for the operator * * RETURN: TRUE if result from this operation is actually consumed * * DESCRIPTION: Determine if the function result value from an operator is * used. * ******************************************************************************/ BOOLEAN AnIsResultUsed ( ACPI_PARSE_OBJECT *Op) { ACPI_PARSE_OBJECT *Parent; switch (Op->Asl.ParseOpcode) { case PARSEOP_INCREMENT: case PARSEOP_DECREMENT: /* These are standalone operators, no return value */ return (TRUE); default: break; } /* Examine parent to determine if the return value is used */ Parent = Op->Asl.Parent; switch (Parent->Asl.ParseOpcode) { /* If/While - check if the operator is the predicate */ case PARSEOP_IF: case PARSEOP_WHILE: /* First child is the predicate */ if (Parent->Asl.Child == Op) { return (TRUE); } return (FALSE); /* Not used if one of these is the parent */ case PARSEOP_METHOD: case PARSEOP_DEFINITION_BLOCK: case PARSEOP_ELSE: return (FALSE); default: /* Any other type of parent means that the result is used */ return (TRUE); } } /******************************************************************************* * * FUNCTION: ApCheckForGpeNameConflict * * PARAMETERS: Op - Current parse op * * RETURN: None * * DESCRIPTION: Check for a conflict between GPE names within this scope. * Conflict means two GPE names with the same GPE number, but * different types -- such as _L1C and _E1C. * ******************************************************************************/ void ApCheckForGpeNameConflict ( ACPI_PARSE_OBJECT *Op) { ACPI_PARSE_OBJECT *NextOp; UINT32 GpeNumber; char Name[ACPI_NAMESEG_SIZE + 1]; char Target[ACPI_NAMESEG_SIZE]; /* Need a null-terminated string version of NameSeg */ ACPI_MOVE_32_TO_32 (Name, Op->Asl.NameSeg); Name[ACPI_NAMESEG_SIZE] = 0; /* * For a GPE method: * 1st char must be underscore * 2nd char must be L or E * 3rd/4th chars must be a hex number */ if ((Name[0] != '_') || ((Name[1] != 'L') && (Name[1] != 'E'))) { return; } /* Verify 3rd/4th chars are a valid hex value */ GpeNumber = strtoul (&Name[2], NULL, 16); if (GpeNumber == ACPI_UINT32_MAX) { return; } /* * We are now sure we have an _Lxx or _Exx. * Create the target name that would cause collision (Flip E/L) */ ACPI_MOVE_32_TO_32 (Target, Name); /* Inject opposite letter ("L" versus "E") */ if (Name[1] == 'L') { Target[1] = 'E'; } else /* Name[1] == 'E' */ { Target[1] = 'L'; } /* Search all peers (objects within this scope) for target match */ NextOp = Op->Asl.Next; while (NextOp) { /* * We mostly care about methods, but check Name() constructs also, * even though they will get another error for not being a method. * All GPE names must be defined as control methods. */ if ((NextOp->Asl.ParseOpcode == PARSEOP_METHOD) || (NextOp->Asl.ParseOpcode == PARSEOP_NAME)) { if (ACPI_COMPARE_NAMESEG (Target, NextOp->Asl.NameSeg)) { /* Found both _Exy and _Lxy in the same scope, error */ AslError (ASL_ERROR, ASL_MSG_GPE_NAME_CONFLICT, NextOp, Name); return; } } NextOp = NextOp->Asl.Next; } /* OK, no conflict found */ return; } /******************************************************************************* * * FUNCTION: ApCheckRegMethod * * PARAMETERS: Op - Current parse op * * RETURN: None * * DESCRIPTION: Ensure that a _REG method has a corresponding Operation * Region declaration within the same scope. Note: _REG is defined * to have two arguments and must therefore be defined as a * control method. * ******************************************************************************/ void ApCheckRegMethod ( ACPI_PARSE_OBJECT *Op) { ACPI_PARSE_OBJECT *Next; ACPI_PARSE_OBJECT *Parent; /* We are only interested in _REG methods */ if (!ACPI_COMPARE_NAMESEG (METHOD_NAME__REG, &Op->Asl.NameSeg)) { return; } /* Get the start of the current scope */ Parent = Op->Asl.Parent; Next = Parent->Asl.Child; /* Search entire scope for an operation region declaration */ while (Next) { if (Next->Asl.ParseOpcode == PARSEOP_OPERATIONREGION) { return; /* Found region, OK */ } Next = Next->Asl.Next; } /* No region found, issue warning */ AslError (ASL_WARNING, ASL_MSG_NO_REGION, Op, NULL); } /******************************************************************************* * * FUNCTION: ApFindNameInDeviceTree * * PARAMETERS: Name - Name to search for * Op - Current parse op * * RETURN: TRUE if name found in the same scope as Op. * * DESCRIPTION: Determine if a name appears in the same scope as Op, as either * a Method() or a Name(). "Same scope" can mean under an If or * Else statement. * * NOTE: Detects _HID/_ADR in this type of construct (legal in ACPI 6.1+) * * Scope (\_SB.PCI0) * { * Device (I2C0) * { * If (SMD0 != 4) { * Name (_HID, "INT3442") * } Else { * Name (_ADR, 0x400) * } * } * } ******************************************************************************/ BOOLEAN ApFindNameInDeviceTree ( char *Name, ACPI_PARSE_OBJECT *Op) { ACPI_STATUS Status; Status = TrWalkParseTree (Op, ASL_WALK_VISIT_DOWNWARD, ApDeviceSubtreeWalk, NULL, Name); if (Status == AE_CTRL_TRUE) { return (TRUE); /* Found a match */ } return (FALSE); } /* Callback function for interface above */ static ACPI_STATUS ApDeviceSubtreeWalk ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { char *Name = ACPI_CAST_PTR (char, Context); switch (Op->Asl.ParseOpcode) { case PARSEOP_DEVICE: /* Level 0 is the starting device, ignore it */ if (Level > 0) { /* Ignore sub-devices */ return (AE_CTRL_DEPTH); } break; case PARSEOP_NAME: case PARSEOP_METHOD: /* These are what we are looking for */ if (ACPI_COMPARE_NAMESEG (Name, Op->Asl.NameSeg)) { return (AE_CTRL_TRUE); } return (AE_CTRL_DEPTH); case PARSEOP_SCOPE: case PARSEOP_FIELD: case PARSEOP_OPERATIONREGION: /* * We want to ignore these, because either they can be large * subtrees or open a scope to somewhere else. */ return (AE_CTRL_DEPTH); default: break; } return (AE_OK); } /******************************************************************************* * * FUNCTION: ApFindNameInScope * * PARAMETERS: Name - Name to search for * Op - Current parse op * * RETURN: TRUE if name found in the same scope as Op. * * DESCRIPTION: Determine if a name appears in the same scope as Op, as either * a Method() or a Name(). * ******************************************************************************/ BOOLEAN ApFindNameInScope ( char *Name, ACPI_PARSE_OBJECT *Op) { ACPI_PARSE_OBJECT *Next; ACPI_PARSE_OBJECT *Parent; /* Get the start of the current scope */ Parent = Op->Asl.Parent; Next = Parent->Asl.Child; /* Search entire scope for a match to the name */ while (Next) { if ((Next->Asl.ParseOpcode == PARSEOP_METHOD) || (Next->Asl.ParseOpcode == PARSEOP_NAME)) { if (ACPI_COMPARE_NAMESEG (Name, Next->Asl.NameSeg)) { return (TRUE); } } Next = Next->Asl.Next; } return (FALSE); } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslcompile.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslcompile.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslcompile.c (revision 357966) @@ -1,1057 +1,1061 @@ /****************************************************************************** * * Module Name: aslcompile - top level compile module * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #include #include #include #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("aslcompile") /* * Main parser entry * External is here in case the parser emits the same external in the * generated header. (Newer versions of Bison) */ int AslCompilerparse( void); /* Local prototypes */ static void CmFlushSourceCode ( void); static void CmDumpAllEvents ( void); static void CmFinishFiles( BOOLEAN DeleteAmlFile); /******************************************************************************* * * FUNCTION: CmDoCompile * * PARAMETERS: None * * RETURN: Status (0 = OK) * * DESCRIPTION: This procedure performs the entire compile * ******************************************************************************/ ACPI_STATUS CmDoCompile ( void) { UINT8 FullCompile; UINT8 Event; ASL_GLOBAL_FILE_NODE *FileNode; FullCompile = UtBeginEvent ("*** Total Compile time ***"); Event = UtBeginEvent ("Open input and output files"); UtEndEvent (Event); Event = UtBeginEvent ("Preprocess input file"); if (AslGbl_PreprocessFlag) { /* Enter compiler name as a #define */ PrAddDefine (ASL_DEFINE, "", FALSE); /* Preprocessor */ PrDoPreprocess (); AslGbl_CurrentLineNumber = 1; AslGbl_LogicalLineNumber = 1; AslGbl_CurrentLineOffset = 0; if (AslGbl_PreprocessOnly) { UtEndEvent (Event); return (AE_OK); } } UtEndEvent (Event); /* Build the parse tree */ Event = UtBeginEvent ("Parse source code and build parse tree"); AslCompilerparse(); UtEndEvent (Event); /* Check for parser-detected syntax errors */ if (AslGbl_SyntaxError) { fprintf (stderr, "Compiler aborting due to parser-detected syntax error(s)\n"); /* Flag this error in the FileNode for compilation summary */ FileNode = FlGetCurrentFileNode (); FileNode->ParserErrorDetected = TRUE; AslGbl_ParserErrorDetected = TRUE; LsDumpParseTree (); goto ErrorExit; } /* Did the parse tree get successfully constructed? */ if (!AslGbl_ParseTreeRoot) { /* * If there are no errors, then we have some sort of * internal problem. */ AslError (ASL_ERROR, ASL_MSG_COMPILER_INTERNAL, NULL, "- Could not resolve parse tree root node"); goto ErrorExit; } /* Flush out any remaining source after parse tree is complete */ Event = UtBeginEvent ("Flush source input"); CmFlushSourceCode (); /* Prune the parse tree if requested (debug purposes only) */ if (AslGbl_PruneParseTree) { AslPruneParseTree (AslGbl_PruneDepth, AslGbl_PruneType); } /* Optional parse tree dump, compiler debug output only */ LsDumpParseTree (); UtEndEvent (FullCompile); return (AE_OK); ErrorExit: UtEndEvent (FullCompile); return (AE_ERROR); } /******************************************************************************* * * FUNCTION: CmDoAslMiddleAndBackEnd * * PARAMETERS: None * * RETURN: Status of middle-end and back-end * * DESCRIPTION: Perform compiler middle-end (type checking and semantic * analysis) and back-end (code generation) * ******************************************************************************/ int CmDoAslMiddleAndBackEnd ( void) { UINT8 Event; ACPI_STATUS Status; OpcGetIntegerWidth (AslGbl_ParseTreeRoot->Asl.Child); /* Pre-process parse tree for any operator transforms */ Event = UtBeginEvent ("Parse tree transforms"); DbgPrint (ASL_DEBUG_OUTPUT, "\nParse tree transforms\n\n"); TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_TWICE, TrAmlTransformWalkBegin, TrAmlTransformWalkEnd, NULL); UtEndEvent (Event); /* Generate AML opcodes corresponding to the parse tokens */ Event = UtBeginEvent ("Generate AML opcodes"); DbgPrint (ASL_DEBUG_OUTPUT, "Generating AML opcodes\n\n"); TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_UPWARD, NULL, OpcAmlOpcodeWalk, NULL); UtEndEvent (Event); /* Interpret and generate all compile-time constants */ Event = UtBeginEvent ("Constant folding via AML interpreter"); DbgPrint (ASL_DEBUG_OUTPUT, "Interpreting compile-time constant expressions\n\n"); if (AslGbl_FoldConstants) { TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_UPWARD, NULL, OpcAmlConstantWalk, NULL); } else { DbgPrint (ASL_PARSE_OUTPUT, " Optional folding disabled\n"); } UtEndEvent (Event); /* Update AML opcodes if necessary, after constant folding */ Event = UtBeginEvent ("Updating AML opcodes after constant folding"); DbgPrint (ASL_DEBUG_OUTPUT, "Updating AML opcodes after constant folding\n\n"); TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_UPWARD, NULL, OpcAmlOpcodeUpdateWalk, NULL); UtEndEvent (Event); /* Calculate all AML package lengths */ Event = UtBeginEvent ("Generate AML package lengths"); DbgPrint (ASL_DEBUG_OUTPUT, "Generating Package lengths\n\n"); TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_UPWARD, NULL, LnPackageLengthWalk, NULL); UtEndEvent (Event); if (AslGbl_ParseOnlyFlag) { AePrintErrorLog (ASL_FILE_STDERR); UtDisplaySummary (ASL_FILE_STDERR); if (AslGbl_DebugFlag) { /* Print error summary to the stdout also */ AePrintErrorLog (ASL_FILE_STDOUT); UtDisplaySummary (ASL_FILE_STDOUT); } return (0); } /* * Create an internal namespace and use it as a symbol table */ /* Namespace loading */ Event = UtBeginEvent ("Create ACPI Namespace"); DbgPrint (ASL_DEBUG_OUTPUT, "Creating ACPI Namespace\n\n"); Status = LdLoadNamespace (AslGbl_ParseTreeRoot); UtEndEvent (Event); if (ACPI_FAILURE (Status)) { return (-1); } /* Namespace cross-reference */ AslGbl_NamespaceEvent = UtBeginEvent ( "Cross reference parse tree and Namespace"); DbgPrint (ASL_DEBUG_OUTPUT, "Cross referencing namespace\n\n"); Status = XfCrossReferenceNamespace (); if (ACPI_FAILURE (Status)) { return (-1); } /* Namespace - Check for non-referenced objects */ LkFindUnreferencedObjects (); UtEndEvent (AslGbl_NamespaceEvent); /* Resolve External Declarations */ Event = UtBeginEvent ("Resolve all Externals"); DbgPrint (ASL_DEBUG_OUTPUT, "\nResolve Externals\n\n"); if (AslGbl_DoExternalsInPlace) { TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_DOWNWARD, ExAmlExternalWalkBegin, NULL, NULL); } else { TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_TWICE, ExAmlExternalWalkBegin, ExAmlExternalWalkEnd, NULL); } UtEndEvent (Event); /* * Semantic analysis. This can happen only after the * namespace has been loaded and cross-referenced. * * part one - check control methods */ Event = UtBeginEvent ("Analyze control method return types"); AslGbl_AnalysisWalkInfo.MethodStack = NULL; DbgPrint (ASL_DEBUG_OUTPUT, "Semantic analysis - Method analysis\n\n"); if (AslGbl_CrossReferenceOutput) { OtPrintHeaders ("Part 1: Object Reference Map " "(Object references from within each control method)"); } TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_TWICE, MtMethodAnalysisWalkBegin, MtMethodAnalysisWalkEnd, &AslGbl_AnalysisWalkInfo); UtEndEvent (Event); /* Generate the object cross-reference file if requested */ Event = UtBeginEvent ("Generate cross-reference file"); OtCreateXrefFile (); UtEndEvent (Event); /* Semantic error checking part two - typing of method returns */ Event = UtBeginEvent ("Determine object types returned by methods"); DbgPrint (ASL_DEBUG_OUTPUT, "Semantic analysis - Method typing\n\n"); TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_UPWARD, NULL, AnMethodTypingWalkEnd, NULL); UtEndEvent (Event); /* Semantic error checking part three - operand type checking */ Event = UtBeginEvent ("Analyze AML operand types"); DbgPrint (ASL_DEBUG_OUTPUT, "Semantic analysis - Operand type checking\n\n"); if (AslGbl_DoTypechecking) { TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_UPWARD, NULL, AnOperandTypecheckWalkEnd, &AslGbl_AnalysisWalkInfo); } UtEndEvent (Event); /* Semantic error checking part four - other miscellaneous checks */ Event = UtBeginEvent ("Miscellaneous analysis"); DbgPrint (ASL_DEBUG_OUTPUT, "Semantic analysis - miscellaneous\n\n"); TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_DOWNWARD, AnOtherSemanticAnalysisWalkBegin, NULL, &AslGbl_AnalysisWalkInfo); UtEndEvent (Event); /* * ASL-/ASL+ converter: Gbl_ParseTreeRoot->CommentList contains the * very last comment of a given ASL file because it's the last constructed * node during compilation. We take the very last comment and save it in a * global for it to be used by the disassembler. */ if (AcpiGbl_CaptureComments) { AcpiGbl_LastListHead = AslGbl_ParseTreeRoot->Asl.CommentList; AslGbl_ParseTreeRoot->Asl.CommentList = NULL; } /* Calculate all AML package lengths */ Event = UtBeginEvent ("Finish AML package length generation"); DbgPrint (ASL_DEBUG_OUTPUT, "Generating Package lengths\n\n"); TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_UPWARD, NULL, LnInitLengthsWalk, NULL); TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_UPWARD, NULL, LnPackageLengthWalk, NULL); UtEndEvent (Event); /* Code generation - emit the AML */ Event = UtBeginEvent ("Generate AML code and write output files"); DbgPrint (ASL_DEBUG_OUTPUT, "Writing AML byte code\n\n"); AslGbl_CurrentDB = AslGbl_ParseTreeRoot->Asl.Child; while (AslGbl_CurrentDB) { switch (FlSwitchFileSet(AslGbl_CurrentDB->Asl.Filename)) { case SWITCH_TO_DIFFERENT_FILE: /* * Reset these parameters when definition blocks belong in * different files. If they belong in the same file, there is * no need to reset these parameters */ FlSeekFile (ASL_FILE_SOURCE_OUTPUT, 0); AslGbl_SourceLine = 0; AslGbl_NextError = AslGbl_ErrorLog; /* fall-through */ case SWITCH_TO_SAME_FILE: CgGenerateAmlOutput (); CmDoOutputFiles (); AslGbl_CurrentDB = AslGbl_CurrentDB->Asl.Next; break; default: /* FILE_NOT_FOUND */ /* The requested file could not be found. Get out of here */ AslGbl_CurrentDB = NULL; break; } } UtEndEvent (Event); Event = UtBeginEvent ("Write optional output files"); UtEndEvent (Event); return (0); } /******************************************************************************* * * FUNCTION: AslCompilerSignon * * PARAMETERS: FileId - ID of the output file * * RETURN: None * * DESCRIPTION: Display compiler signon * ******************************************************************************/ void AslCompilerSignon ( UINT32 FileId) { char *Prefix = ""; char *UtilityName; /* Set line prefix depending on the destination file type */ switch (FileId) { case ASL_FILE_ASM_SOURCE_OUTPUT: case ASL_FILE_ASM_INCLUDE_OUTPUT: Prefix = "; "; break; case ASL_FILE_HEX_OUTPUT: if (AslGbl_HexOutputFlag == HEX_OUTPUT_ASM) { Prefix = "; "; } else if ((AslGbl_HexOutputFlag == HEX_OUTPUT_C) || (AslGbl_HexOutputFlag == HEX_OUTPUT_ASL)) { FlPrintFile (ASL_FILE_HEX_OUTPUT, "/*\n"); Prefix = " * "; } break; case ASL_FILE_C_SOURCE_OUTPUT: case ASL_FILE_C_OFFSET_OUTPUT: case ASL_FILE_C_INCLUDE_OUTPUT: Prefix = " * "; break; default: /* No other output types supported */ break; } /* Running compiler or disassembler? */ if (AcpiGbl_DisasmFlag) { UtilityName = AML_DISASSEMBLER_NAME; } else { UtilityName = ASL_COMPILER_NAME; } /* Compiler signon with copyright */ FlPrintFile (FileId, "%s\n", Prefix); FlPrintFile (FileId, ACPI_COMMON_HEADER (UtilityName, Prefix)); } /******************************************************************************* * * FUNCTION: AslCompilerFileHeader * * PARAMETERS: FileId - ID of the output file * * RETURN: None * * DESCRIPTION: Header used at the beginning of output files * ******************************************************************************/ void AslCompilerFileHeader ( UINT32 FileId) { - struct tm *NewTime; + char *NewTime; time_t Aclock; char *Prefix = ""; /* Set line prefix depending on the destination file type */ switch (FileId) { case ASL_FILE_ASM_SOURCE_OUTPUT: case ASL_FILE_ASM_INCLUDE_OUTPUT: Prefix = "; "; break; case ASL_FILE_HEX_OUTPUT: if (AslGbl_HexOutputFlag == HEX_OUTPUT_ASM) { Prefix = "; "; } else if ((AslGbl_HexOutputFlag == HEX_OUTPUT_C) || (AslGbl_HexOutputFlag == HEX_OUTPUT_ASL)) { Prefix = " * "; } break; case ASL_FILE_C_SOURCE_OUTPUT: case ASL_FILE_C_OFFSET_OUTPUT: case ASL_FILE_C_INCLUDE_OUTPUT: Prefix = " * "; break; default: /* No other output types supported */ break; } /* Compilation header with timestamp */ - (void) time (&Aclock); - NewTime = localtime (&Aclock); + Aclock = time (NULL); + NewTime = ctime (&Aclock); FlPrintFile (FileId, - "%sCompilation of \"%s\" - %s%s\n", - Prefix, AslGbl_Files[ASL_FILE_INPUT].Filename, asctime (NewTime), - Prefix); + "%sCompilation of \"%s\" -", + Prefix, AslGbl_Files[ASL_FILE_INPUT].Filename); + + if (NewTime) + { + FlPrintFile (FileId, " %s%s\n", NewTime, Prefix); + } switch (FileId) { case ASL_FILE_C_SOURCE_OUTPUT: case ASL_FILE_C_OFFSET_OUTPUT: case ASL_FILE_C_INCLUDE_OUTPUT: FlPrintFile (FileId, " */\n"); break; default: /* Nothing to do for other output types */ break; } } /******************************************************************************* * * FUNCTION: CmFlushSourceCode * * PARAMETERS: None * * RETURN: None * * DESCRIPTION: Read in any remaining source code after the parse tree * has been constructed. * ******************************************************************************/ static void CmFlushSourceCode ( void) { char Buffer; while (FlReadFile (ASL_FILE_INPUT, &Buffer, 1) != AE_ERROR) { AslInsertLineBuffer ((int) Buffer); } AslResetCurrentLineBuffer (); } /******************************************************************************* * * FUNCTION: CmDoOutputFiles * * PARAMETERS: None * * RETURN: None. * * DESCRIPTION: Create all "listing" type files * ******************************************************************************/ void CmDoOutputFiles ( void) { /* Create listings and hex files */ LsDoListings (); HxDoHexOutput (); /* Dump the namespace to the .nsp file if requested */ (void) NsDisplayNamespace (); /* Dump the device mapping file */ MpEmitMappingInfo (); } /******************************************************************************* * * FUNCTION: CmDumpAllEvents * * PARAMETERS: None * * RETURN: None. * * DESCRIPTION: Dump all compiler events * ******************************************************************************/ static void CmDumpAllEvents ( void) { ASL_EVENT_INFO *Event; UINT32 Delta; UINT32 MicroSeconds; UINT32 MilliSeconds; UINT32 i; Event = AslGbl_Events; DbgPrint (ASL_DEBUG_OUTPUT, "\n\nElapsed time for major events\n\n"); if (AslGbl_CompileTimesFlag) { printf ("\nElapsed time for major events\n\n"); } for (i = 0; i < AslGbl_NextEvent; i++) { if (Event->Valid) { /* Delta will be in 100-nanosecond units */ Delta = (UINT32) (Event->EndTime - Event->StartTime); MicroSeconds = Delta / ACPI_100NSEC_PER_USEC; MilliSeconds = Delta / ACPI_100NSEC_PER_MSEC; /* Round milliseconds up */ if ((MicroSeconds - (MilliSeconds * ACPI_USEC_PER_MSEC)) >= 500) { MilliSeconds++; } DbgPrint (ASL_DEBUG_OUTPUT, "%8u usec %8u msec - %s\n", MicroSeconds, MilliSeconds, Event->EventName); if (AslGbl_CompileTimesFlag) { printf ("%8u usec %8u msec - %s\n", MicroSeconds, MilliSeconds, Event->EventName); } } Event++; } } /******************************************************************************* * * FUNCTION: CmCleanupAndExit * * PARAMETERS: None * * RETURN: None. * * DESCRIPTION: Close all open files and exit the compiler * ******************************************************************************/ int CmCleanupAndExit ( void) { int Status = 0; BOOLEAN DeleteAmlFile = FALSE; ASL_GLOBAL_FILE_NODE *CurrentFileNode = AslGbl_FilesList; /* Check if any errors occurred during compile */ (void) AslCheckForErrorExit (); AePrintErrorLog (ASL_FILE_STDERR); if (AslGbl_DebugFlag) { /* Print error summary to stdout also */ AePrintErrorLog (ASL_FILE_STDOUT); } /* Emit compile times if enabled */ CmDumpAllEvents (); if (AslGbl_CompileTimesFlag) { printf ("\nMiscellaneous compile statistics\n\n"); printf ("%11u : %s\n", AslGbl_TotalParseNodes, "Parse nodes"); printf ("%11u : %s\n", AslGbl_NsLookupCount, "Namespace searches"); printf ("%11u : %s\n", AslGbl_TotalNamedObjects, "Named objects"); printf ("%11u : %s\n", AslGbl_TotalMethods, "Control methods"); printf ("%11u : %s\n", AslGbl_TotalAllocations, "Memory Allocations"); printf ("%11u : %s\n", AslGbl_TotalAllocated, "Total allocated memory"); printf ("%11u : %s\n", AslGbl_TotalFolds, "Constant subtrees folded"); printf ("\n"); } if (AslGbl_NsLookupCount) { DbgPrint (ASL_DEBUG_OUTPUT, "\n\nMiscellaneous compile statistics\n\n"); DbgPrint (ASL_DEBUG_OUTPUT, "%32s : %u\n", "Total Namespace searches", AslGbl_NsLookupCount); DbgPrint (ASL_DEBUG_OUTPUT, "%32s : %u usec\n", "Time per search", ((UINT32) (AslGbl_Events[AslGbl_NamespaceEvent].EndTime - AslGbl_Events[AslGbl_NamespaceEvent].StartTime) / 10) / AslGbl_NsLookupCount); } if (AslGbl_ExceptionCount[ASL_ERROR] > ASL_MAX_ERROR_COUNT) { printf ("\nMaximum error count (%d) exceeded\n", ASL_MAX_ERROR_COUNT); } UtDisplaySummary (ASL_FILE_STDOUT); /* * Delete the AML file if there are errors and the force AML output option * (-f) has not been used. * * Return -1 as a status of the compiler if no AML files are generated. If * the AML file is generated in the presence of errors, return 0. In the * latter case, the errors were ignored by the user so the compilation is * considered successful. */ if (AslGbl_ParserErrorDetected || AslGbl_PreprocessOnly || ((AslGbl_ExceptionCount[ASL_ERROR] > 0) && (!AslGbl_IgnoreErrors) && AslGbl_Files[ASL_FILE_AML_OUTPUT].Handle)) { DeleteAmlFile = TRUE; Status = -1; } /* Close all open files */ while (CurrentFileNode) { /* * Set the program return status based on file errors. If there are any * errors and during compilation, the command is not considered * successful. */ if (Status != -1 && !AslGbl_IgnoreErrors && CurrentFileNode->ParserErrorDetected) { Status = -1; } switch (FlSwitchFileSet (CurrentFileNode->Files[ASL_FILE_INPUT].Filename)) { case SWITCH_TO_SAME_FILE: case SWITCH_TO_DIFFERENT_FILE: CmFinishFiles (DeleteAmlFile); CurrentFileNode = CurrentFileNode->Next; break; case FILE_NOT_FOUND: default: CurrentFileNode = NULL; break; } } /* Final cleanup after compiling one file */ if (!AslGbl_DoAslConversion) { UtDeleteLocalCaches (); } return (Status); } /******************************************************************************* * * FUNCTION: CmFinishFiles * * PARAMETERS: DeleteAmlFile * * RETURN: None. * * DESCRIPTION: Close all open files, delete AML files depending on the * function parameter is true. * ******************************************************************************/ static void CmFinishFiles( BOOLEAN DeleteAmlFile) { UINT32 i; /* * Take care with the preprocessor file (.pre), it might be the same * as the "input" file, depending on where the compiler has terminated * or aborted. Prevent attempt to close the same file twice in * loop below. */ if (AslGbl_Files[ASL_FILE_PREPROCESSOR].Handle == AslGbl_Files[ASL_FILE_INPUT].Handle) { AslGbl_Files[ASL_FILE_PREPROCESSOR].Handle = NULL; } /* Close the standard I/O files */ for (i = ASL_FILE_INPUT; i < ASL_MAX_FILE_TYPE; i++) { /* * Some files such as debug output files could be pointing to * stderr or stdout. Leave these alone. */ if (AslGbl_Files[i].Handle != stderr && AslGbl_Files[i].Handle != stdout) { FlCloseFile (i); } } /* Delete AML file if there are errors */ if (DeleteAmlFile) { FlDeleteFile (ASL_FILE_AML_OUTPUT); } /* Delete the preprocessor temp file unless full debug was specified */ if (AslGbl_PreprocessFlag && !AslGbl_KeepPreprocessorTempFile) { FlDeleteFile (ASL_FILE_PREPROCESSOR); } /* * Delete intermediate ("combined") source file (if -ls flag not set) * This file is created during normal ASL/AML compiles. It is not * created by the data table compiler. * * If the -ls flag is set, then the .SRC file should not be deleted. * In this case, Gbl_SourceOutputFlag is set to TRUE. * * Note: Handles are cleared by FlCloseFile above, so we look at the * filename instead, to determine if the .SRC file was actually * created. */ if (!AslGbl_SourceOutputFlag) { FlDeleteFile (ASL_FILE_SOURCE_OUTPUT); } } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslcompiler.h =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslcompiler.h (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslcompiler.h (revision 357966) @@ -1,1629 +1,1642 @@ /****************************************************************************** * * Module Name: aslcompiler.h - common include file for iASL * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #ifndef __ASLCOMPILER_H #define __ASLCOMPILER_H #include #include #include #include /* Microsoft-specific */ #if (defined WIN32 || defined WIN64) /* warn : used #pragma pack */ #pragma warning(disable:4103) /* warn : named type definition in parentheses */ #pragma warning(disable:4115) #endif #include #include #include #include #include /* Compiler headers */ #include #include #include #include #include #include /******************************************************************************* * * Compiler prototypes * ******************************************************************************/ /* * Main ASL parser - generated from flex/bison, lex/yacc, etc. */ ACPI_PARSE_OBJECT * AslDoError ( void); int AslCompilerlex( void); void AslResetCurrentLineBuffer ( void); void AslInsertLineBuffer ( int SourceChar); int AslPopInputFileStack ( void); void AslPushInputFileStack ( FILE *InputFile, char *Filename); void AslParserCleanup ( void); /* * aslstartup - entered from main() */ void AslInitializeGlobals ( void); typedef ACPI_STATUS (*ASL_PATHNAME_CALLBACK) ( char *); ACPI_STATUS AslDoOneFile ( char *Filename); ACPI_STATUS AslCheckForErrorExit ( void); /* * aslcompile - compile mainline */ void AslCompilerSignon ( UINT32 FileId); void AslCompilerFileHeader ( UINT32 FileId); ACPI_STATUS CmDoCompile ( void); int CmDoAslMiddleAndBackEnd ( void); void CmDoOutputFiles ( void); int CmCleanupAndExit ( void); ACPI_STATUS AslDoDisassembly ( void); /* * aslallocate - memory allocation */ void * UtLocalCalloc ( UINT32 Size); void UtExpandLineBuffers ( void); void UtReallocLineBuffers ( char **Buffer, UINT32 OldSize, UINT32 NewSize); void UtFreeLineBuffers ( void); /* * aslcache - local cache support */ char * UtLocalCacheCalloc ( UINT32 Length); ACPI_PARSE_OBJECT * UtParseOpCacheCalloc ( void); DT_SUBTABLE * UtSubtableCacheCalloc ( void); DT_FIELD * UtFieldCacheCalloc ( void); void UtDeleteLocalCaches ( void); /* * aslascii - ascii support */ ACPI_STATUS FlIsFileAsciiSource ( char *Filename, BOOLEAN DisplayErrors); /* * aslwalks - semantic analysis and parse tree walks */ ACPI_STATUS AnOtherSemanticAnalysisWalkBegin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); ACPI_STATUS AnOtherSemanticAnalysisWalkEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); ACPI_STATUS AnOperandTypecheckWalkEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); ACPI_STATUS AnMethodTypingWalkEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); /* * aslmethod - Control method analysis walk */ ACPI_STATUS MtMethodAnalysisWalkBegin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); ACPI_STATUS MtMethodAnalysisWalkEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); +UINT32 +MtProcessTypeOp ( + ACPI_PARSE_OBJECT *TypeOp); +UINT8 +MtProcessParameterTypeList ( + ACPI_PARSE_OBJECT *ParamTypeOp, + UINT32 *TypeList); + + /* * aslbtypes - bitfield data types */ UINT32 AnMapObjTypeToBtype ( ACPI_PARSE_OBJECT *Op); UINT32 AnMapArgTypeToBtype ( UINT32 ArgType); UINT32 AnGetBtype ( ACPI_PARSE_OBJECT *Op); void AnFormatBtype ( char *Buffer, UINT32 Btype); /* * aslanalyze - Support functions for parse tree walks */ void AnCheckId ( ACPI_PARSE_OBJECT *Op, ACPI_NAME Type); /* Values for Type argument above */ #define ASL_TYPE_HID 0 #define ASL_TYPE_CID 1 BOOLEAN AnIsInternalMethod ( ACPI_PARSE_OBJECT *Op); UINT32 AnGetInternalMethodReturnType ( ACPI_PARSE_OBJECT *Op); BOOLEAN AnLastStatementIsReturn ( ACPI_PARSE_OBJECT *Op); void AnCheckMethodReturnValue ( ACPI_PARSE_OBJECT *Op, const ACPI_OPCODE_INFO *OpInfo, ACPI_PARSE_OBJECT *ArgOp, UINT32 RequiredBtypes, UINT32 ThisNodeBtype); BOOLEAN AnIsResultUsed ( ACPI_PARSE_OBJECT *Op); void ApCheckForGpeNameConflict ( ACPI_PARSE_OBJECT *Op); void ApCheckRegMethod ( ACPI_PARSE_OBJECT *Op); BOOLEAN ApFindNameInScope ( char *Name, ACPI_PARSE_OBJECT *Op); BOOLEAN ApFindNameInDeviceTree ( char *Name, ACPI_PARSE_OBJECT *Op); /* * aslerror - error handling/reporting */ void AslAbort ( void); void AslDualParseOpError ( UINT8 Level, UINT16 MainMessageId, ACPI_PARSE_OBJECT *MainOp, char *MainMessage, UINT16 SecondMessageId, ACPI_PARSE_OBJECT *SecondOp, char *SecondaryMessage); void AslError ( UINT8 Level, UINT16 MessageId, ACPI_PARSE_OBJECT *Op, char *ExtraMessage); void AslCheckExpectedExceptions ( void); ACPI_STATUS AslExpectException ( char *MessageIdString); ACPI_STATUS AslElevateException ( char *MessageIdString); ACPI_STATUS AslDisableException ( char *MessageIdString); BOOLEAN AslIsExceptionIgnored ( UINT8 Level, UINT16 MessageId); void AslCoreSubsystemError ( ACPI_PARSE_OBJECT *Op, ACPI_STATUS Status, char *ExtraMessage, BOOLEAN Abort); int AslCompilererror( const char *s); void AslCommonError ( UINT8 Level, UINT16 MessageId, UINT32 CurrentLineNumber, UINT32 LogicalLineNumber, UINT32 LogicalByteOffset, UINT32 Column, char *Filename, char *ExtraMessage); void AslCommonError2 ( UINT8 Level, UINT16 MessageId, UINT32 LineNumber, UINT32 Column, char *SourceLine, char *Filename, char *ExtraMessage); void AePrintException ( UINT32 FileId, ASL_ERROR_MSG *Enode, char *Header); void AePrintErrorLog ( UINT32 FileId); void AeClearErrorLog ( void); /* * asllisting - generate all "listing" type files */ void LsDoListings ( void); void LsWriteNodeToAsmListing ( ACPI_PARSE_OBJECT *Op); void LsWriteNode ( ACPI_PARSE_OBJECT *Op, UINT32 FileId); void LsDumpParseTree ( void); /* * asllistsup - Listing file support utilities */ void LsDumpAscii ( UINT32 FileId, UINT32 Count, UINT8 *Buffer); void LsDumpAsciiInComment ( UINT32 FileId, UINT32 Count, UINT8 *Buffer); void LsCheckException ( UINT32 LineNumber, UINT32 FileId); void LsFlushListingBuffer ( UINT32 FileId); void LsWriteListingHexBytes ( UINT8 *Buffer, UINT32 Length, UINT32 FileId); void LsWriteSourceLines ( UINT32 ToLineNumber, UINT32 ToLogicalLineNumber, UINT32 FileId); UINT32 LsWriteOneSourceLine ( UINT32 FileId); void LsPushNode ( char *Filename); ASL_LISTING_NODE * LsPopNode ( void); /* * aslhex - generate all "hex" output files (C, ASM, ASL) */ void HxDoHexOutput ( void); /* * aslfold - constant folding */ ACPI_STATUS OpcAmlConstantWalk ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); /* * aslmessages - exception strings */ const char * AeDecodeMessageId ( UINT16 MessageId); const char * AeDecodeExceptionLevel ( UINT8 Level); UINT16 AeBuildFullExceptionCode ( UINT8 Level, UINT16 MessageId); /* * asloffset - generate C offset file for BIOS support */ ACPI_STATUS LsAmlOffsetWalk ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); void LsDoOffsetTableHeader ( UINT32 FileId); void LsDoOffsetTableFooter ( UINT32 FileId); /* * aslopcodes - generate AML opcodes */ ACPI_STATUS OpcAmlOpcodeWalk ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); ACPI_STATUS OpcAmlOpcodeUpdateWalk ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); void OpcGenerateAmlOpcode ( ACPI_PARSE_OBJECT *Op); UINT32 OpcSetOptimalIntegerSize ( ACPI_PARSE_OBJECT *Op); void OpcGetIntegerWidth ( ACPI_PARSE_OBJECT *Op); /* * asloperands - generate AML operands for the AML opcodes */ ACPI_PARSE_OBJECT * UtGetArg ( ACPI_PARSE_OBJECT *Op, UINT32 Argn); void OpnGenerateAmlOperands ( ACPI_PARSE_OBJECT *Op); void OpnDoPackage ( ACPI_PARSE_OBJECT *Op); /* * aslopt - optimization */ void OptOptimizeNamePath ( ACPI_PARSE_OBJECT *Op, UINT32 Flags, ACPI_WALK_STATE *WalkState, char *AmlNameString, ACPI_NAMESPACE_NODE *TargetNode); /* * aslpld - ToPLD macro support */ void OpcDoPld ( ACPI_PARSE_OBJECT *Op); /* * aslprintf - Printf/Fprintf macros */ void OpcDoPrintf ( ACPI_PARSE_OBJECT *Op); void OpcDoFprintf ( ACPI_PARSE_OBJECT *Op); /* * aslprune - parse tree pruner */ void AslPruneParseTree ( UINT32 PruneDepth, UINT32 Type); /* * aslcodegen - code generation */ void CgGenerateAmlOutput ( void); void CgLocalWriteAmlData ( ACPI_PARSE_OBJECT *Op, void *Buffer, UINT32 Length); /* * aslfile */ void FlOpenFile ( UINT32 FileId, char *Filename, char *Mode); /* * asllength - calculate/adjust AML package lengths */ ACPI_STATUS LnPackageLengthWalk ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); ACPI_STATUS LnInitLengthsWalk ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); void CgGenerateAmlLengths ( ACPI_PARSE_OBJECT *Op); /* * aslmap - opcode mappings and reserved method names */ ACPI_OBJECT_TYPE AslMapNamedOpcodeToDataType ( UINT16 Opcode); /* * aslpredef - ACPI predefined names support */ BOOLEAN ApCheckForPredefinedMethod ( ACPI_PARSE_OBJECT *Op, ASL_METHOD_INFO *MethodInfo); void ApCheckPredefinedReturnValue ( ACPI_PARSE_OBJECT *Op, ASL_METHOD_INFO *MethodInfo); UINT32 ApCheckForPredefinedName ( ACPI_PARSE_OBJECT *Op, char *Name); void ApCheckForPredefinedObject ( ACPI_PARSE_OBJECT *Op, char *Name); ACPI_STATUS ApCheckObjectType ( const char *PredefinedName, ACPI_PARSE_OBJECT *Op, UINT32 ExpectedBtypes, UINT32 PackageIndex); void ApDisplayReservedNames ( void); /* * aslprepkg - ACPI predefined names support for packages */ void ApCheckPackage ( ACPI_PARSE_OBJECT *ParentOp, const ACPI_PREDEFINED_INFO *Predefined); /* * asltransform - parse tree transformations */ ACPI_STATUS TrAmlTransformWalkBegin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); ACPI_STATUS TrAmlTransformWalkEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); /* * aslexternal - External opcode support */ ACPI_STATUS ExAmlExternalWalkBegin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); ACPI_STATUS ExAmlExternalWalkEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); void ExDoExternal ( ACPI_PARSE_OBJECT *Op); /* Values for "Visitation" parameter above */ #define ASL_WALK_VISIT_DOWNWARD 0x01 #define ASL_WALK_VISIT_UPWARD 0x02 #define ASL_WALK_VISIT_DB_SEPARATELY 0x04 #define ASL_WALK_VISIT_TWICE (ASL_WALK_VISIT_DOWNWARD | ASL_WALK_VISIT_UPWARD) /* * aslparseop.c - Parse op create/allocate/cache */ ACPI_PARSE_OBJECT * TrCreateOp ( UINT32 ParseOpcode, UINT32 NumChildren, ...); ACPI_PARSE_OBJECT * TrCreateLeafOp ( UINT32 ParseOpcode); ACPI_PARSE_OBJECT * TrCreateNullTargetOp ( void); ACPI_PARSE_OBJECT * TrCreateAssignmentOp ( ACPI_PARSE_OBJECT *Target, ACPI_PARSE_OBJECT *Source); ACPI_PARSE_OBJECT * TrCreateTargetOp ( ACPI_PARSE_OBJECT *OriginalOp, ACPI_PARSE_OBJECT *ParentOp); ACPI_PARSE_OBJECT * TrCreateValuedLeafOp ( UINT32 ParseOpcode, UINT64 Value); ACPI_PARSE_OBJECT * TrCreateConstantLeafOp ( UINT32 ParseOpcode); ACPI_PARSE_OBJECT * TrAllocateOp ( UINT32 ParseOpcode); void TrPrintOpFlags ( UINT32 Flags, UINT32 OutputLevel); /* * asltree.c - Parse tree management */ void TrSetOpParent ( ACPI_PARSE_OBJECT *Op, ACPI_PARSE_OBJECT *ParentOp); ACPI_PARSE_OBJECT * TrSetOpIntegerValue ( UINT32 ParseOpcode, ACPI_PARSE_OBJECT *Op); void TrSetOpEndLineNumber ( ACPI_PARSE_OBJECT *Op); void TrSetOpCurrentFilename ( ACPI_PARSE_OBJECT *Op); void TrSetOpIntegerWidth ( ACPI_PARSE_OBJECT *TableSignature, ACPI_PARSE_OBJECT *Revision); ACPI_PARSE_OBJECT * TrLinkOpChildren ( ACPI_PARSE_OBJECT *Op, UINT32 NumChildren, ...); ACPI_PARSE_OBJECT * TrLinkPeerOp ( ACPI_PARSE_OBJECT *Op1, ACPI_PARSE_OBJECT *Op2); ACPI_PARSE_OBJECT * TrLinkChildOp ( ACPI_PARSE_OBJECT *Op1, ACPI_PARSE_OBJECT *Op2); ACPI_PARSE_OBJECT * TrSetOpFlags ( ACPI_PARSE_OBJECT *Op, UINT32 Flags); ACPI_PARSE_OBJECT * TrSetOpAmlLength ( ACPI_PARSE_OBJECT *Op, UINT32 Length); ACPI_PARSE_OBJECT * TrLinkPeerOps ( UINT32 NumPeers, ...); ACPI_STATUS TrWalkParseTree ( ACPI_PARSE_OBJECT *Op, UINT32 Visitation, ASL_WALK_CALLBACK DescendingCallback, ASL_WALK_CALLBACK AscendingCallback, void *Context); /* * aslfiles - File I/O support */ void FlAddIncludeDirectory ( char *Dir); char * FlMergePathnames ( char *PrefixDir, char *FilePathname); void FlOpenIncludeFile ( ACPI_PARSE_OBJECT *Op); void FlFileError ( UINT32 FileId, UINT8 ErrorId); UINT32 FlGetFileSize ( UINT32 FileId); ACPI_STATUS FlReadFile ( UINT32 FileId, void *Buffer, UINT32 Length); void FlWriteFile ( UINT32 FileId, void *Buffer, UINT32 Length); void FlSeekFile ( UINT32 FileId, long Offset); void FlSeekFileSet ( UINT32 FileId, long Offset); void FlCloseFile ( UINT32 FileId); ACPI_PRINTF_LIKE (2) void FlPrintFile ( UINT32 FileId, char *Format, ...); void FlDeleteFile ( UINT32 FileId); void FlSetLineNumber ( UINT32 LineNumber); void FlSetFilename ( char *Filename); ACPI_STATUS FlOpenInputFile ( char *InputFilename); ACPI_STATUS FlOpenAmlOutputFile ( char *InputFilename); ACPI_STATUS FlOpenMiscOutputFiles ( char *InputFilename); ACPI_STATUS FlInitOneFile ( char *InputFilename); ASL_FILE_SWITCH_STATUS FlSwitchFileSet ( char *InputFilename); FILE * FlGetFileHandle ( UINT32 OutFileId, UINT32 InFileId, char *Filename); ASL_GLOBAL_FILE_NODE * FlGetFileNode ( UINT32 FileId, char *Filename); ASL_GLOBAL_FILE_NODE * FlGetCurrentFileNode ( void); /* * aslhwmap - hardware map summary */ void MpEmitMappingInfo ( void); /* * asload - load namespace in prep for cross reference */ ACPI_STATUS LdLoadNamespace ( ACPI_PARSE_OBJECT *RootOp); /* * asllookup - namespace lookup functions */ void LkFindUnreferencedObjects ( void); /* * aslhelp - help screens */ void Usage ( void); void AslFilenameHelp ( void); void AslDisassemblyHelp ( void); /* * aslnamesp - namespace output file generation */ ACPI_STATUS NsDisplayNamespace ( void); void NsSetupNamespaceListing ( void *Handle); /* * asloptions - command line processing */ int AslCommandLine ( int argc, char **argv); /* * aslxref - namespace cross reference */ ACPI_STATUS XfCrossReferenceNamespace ( void); /* * aslxrefout */ void OtPrintHeaders ( char *Message); void OtCreateXrefFile ( void); void OtXrefWalkPart1 ( ACPI_PARSE_OBJECT *Op, UINT32 Level, ASL_METHOD_INFO *MethodInfo); /* * aslutils - common compiler utilities */ ACPI_PRINTF_LIKE(2) void DbgPrint ( UINT32 Type, char *Format, ...); /* Type values for above */ #define ASL_DEBUG_OUTPUT 0 #define ASL_PARSE_OUTPUT 1 #define ASL_TREE_OUTPUT 2 UINT8 UtIsBigEndianMachine ( void); BOOLEAN UtQueryForOverwrite ( char *Pathname); void UtDumpStringOp ( ACPI_PARSE_OBJECT *Op, UINT32 Level); void UtDumpIntegerOp ( ACPI_PARSE_OBJECT *Op, UINT32 Level, UINT32 IntegerLength); void UtDumpBasicOp ( ACPI_PARSE_OBJECT *Op, UINT32 Level); -void * -UtGetParentMethod ( +ACPI_NAMESPACE_NODE * +UtGetParentMethodNode ( ACPI_NAMESPACE_NODE *Node); + +ACPI_PARSE_OBJECT * +UtGetParentMethodOp ( + ACPI_PARSE_OBJECT *Op); BOOLEAN UtNodeIsDescendantOf ( ACPI_NAMESPACE_NODE *Node1, ACPI_NAMESPACE_NODE *Node2); void UtDisplaySupportedTables ( void); void UtDisplayConstantOpcodes ( void); UINT8 UtBeginEvent ( char *Name); void UtEndEvent ( UINT8 Event); void UtDisplaySummary ( UINT32 FileId); void UtDisplayOneSummary ( UINT32 FileId, BOOLEAN DisplayErrorSummary); void UtConvertByteToHex ( UINT8 RawByte, UINT8 *Buffer); void UtConvertByteToAsmHex ( UINT8 RawByte, UINT8 *Buffer); char * UtGetOpName ( UINT32 ParseOpcode); void UtSetParseOpName ( ACPI_PARSE_OBJECT *Op); ACPI_STATUS UtInternalizeName ( char *ExternalName, char **ConvertedName); BOOLEAN UtNameContainsAllPrefix ( ACPI_PARSE_OBJECT *Op); void UtAttachNamepathToOwner ( ACPI_PARSE_OBJECT *Op, ACPI_PARSE_OBJECT *NameNode); ACPI_PARSE_OBJECT * UtCheckIntegerRange ( ACPI_PARSE_OBJECT *Op, UINT32 LowValue, UINT32 HighValue); UINT64 UtDoConstant ( char *String); char * AcpiUtStrdup ( char *String); char * AcpiUtStrcat ( char *String1, char *String2); /* * asluuid - UUID support */ ACPI_STATUS AuValidateUuid ( char *InString); ACPI_STATUS AuConvertUuidToString ( char *UuIdBuffer, char *OutString); /* * aslresource - Resource template generation utilities */ void RsSmallAddressCheck ( UINT8 Type, UINT32 Minimum, UINT32 Maximum, UINT32 Length, UINT32 Alignment, ACPI_PARSE_OBJECT *MinOp, ACPI_PARSE_OBJECT *MaxOp, ACPI_PARSE_OBJECT *LengthOp, ACPI_PARSE_OBJECT *AlignOp, ACPI_PARSE_OBJECT *Op); void RsLargeAddressCheck ( UINT64 Minimum, UINT64 Maximum, UINT64 Length, UINT64 Granularity, UINT8 Flags, ACPI_PARSE_OBJECT *MinOp, ACPI_PARSE_OBJECT *MaxOp, ACPI_PARSE_OBJECT *LengthOp, ACPI_PARSE_OBJECT *GranOp, ACPI_PARSE_OBJECT *Op); UINT16 RsGetStringDataLength ( ACPI_PARSE_OBJECT *InitializerOp); ASL_RESOURCE_NODE * RsAllocateResourceNode ( UINT32 Size); void RsCreateResourceField ( ACPI_PARSE_OBJECT *Op, char *Name, UINT32 ByteOffset, UINT32 BitOffset, UINT32 BitLength); void RsSetFlagBits ( UINT8 *Flags, ACPI_PARSE_OBJECT *Op, UINT8 Position, UINT8 DefaultBit); void RsSetFlagBits16 ( UINT16 *Flags, ACPI_PARSE_OBJECT *Op, UINT8 Position, UINT8 DefaultBit); ACPI_PARSE_OBJECT * RsCompleteNodeAndGetNext ( ACPI_PARSE_OBJECT *Op); void RsCheckListForDuplicates ( ACPI_PARSE_OBJECT *Op); ASL_RESOURCE_NODE * RsDoOneResourceDescriptor ( ASL_RESOURCE_INFO *Info, UINT8 *State); /* Values for State above */ #define ACPI_RSTATE_NORMAL 0 #define ACPI_RSTATE_START_DEPENDENT 1 #define ACPI_RSTATE_DEPENDENT_LIST 2 UINT32 RsLinkDescriptorChain ( ASL_RESOURCE_NODE **PreviousRnode, ASL_RESOURCE_NODE *Rnode); void RsDoResourceTemplate ( ACPI_PARSE_OBJECT *Op); /* * aslrestype1 - Miscellaneous Small descriptors */ ASL_RESOURCE_NODE * RsDoEndTagDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoEndDependentDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoMemory24Descriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoMemory32Descriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoMemory32FixedDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoStartDependentDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoStartDependentNoPriDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoVendorSmallDescriptor ( ASL_RESOURCE_INFO *Info); /* * aslrestype1i - I/O-related Small descriptors */ ASL_RESOURCE_NODE * RsDoDmaDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoFixedDmaDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoFixedIoDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoIoDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoIrqDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoIrqNoFlagsDescriptor ( ASL_RESOURCE_INFO *Info); /* * aslrestype2 - Large resource descriptors */ ASL_RESOURCE_NODE * RsDoInterruptDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoVendorLargeDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoGeneralRegisterDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoGpioIntDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoGpioIoDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoI2cSerialBusDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoSpiSerialBusDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoUartSerialBusDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoPinFunctionDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoPinConfigDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoPinGroupDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoPinGroupFunctionDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoPinGroupConfigDescriptor ( ASL_RESOURCE_INFO *Info); /* * aslrestype2d - DWord address descriptors */ ASL_RESOURCE_NODE * RsDoDwordIoDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoDwordMemoryDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoDwordSpaceDescriptor ( ASL_RESOURCE_INFO *Info); /* * aslrestype2e - Extended address descriptors */ ASL_RESOURCE_NODE * RsDoExtendedIoDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoExtendedMemoryDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoExtendedSpaceDescriptor ( ASL_RESOURCE_INFO *Info); /* * aslrestype2q - QWord address descriptors */ ASL_RESOURCE_NODE * RsDoQwordIoDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoQwordMemoryDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoQwordSpaceDescriptor ( ASL_RESOURCE_INFO *Info); /* * aslrestype2w - Word address descriptors */ ASL_RESOURCE_NODE * RsDoWordIoDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoWordSpaceDescriptor ( ASL_RESOURCE_INFO *Info); ASL_RESOURCE_NODE * RsDoWordBusNumberDescriptor ( ASL_RESOURCE_INFO *Info); /* * Entry to data table compiler subsystem */ ACPI_STATUS DtDoCompile( void); ACPI_STATUS DtCreateTemplates ( char **argv); /* * ASL/ASL+ converter debug */ ACPI_PRINTF_LIKE (1) void CvDbgPrint ( char *Fmt, ...); #endif /* __ASLCOMPILER_H */ Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslexternal.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslexternal.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslexternal.c (revision 357966) @@ -1,630 +1,672 @@ /****************************************************************************** * * Module Name: aslexternal - ASL External opcode compiler support * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include "aslcompiler.y.h" #include #include #include #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("aslexternal") /* Local prototypes */ static void ExInsertArgCount ( ACPI_PARSE_OBJECT *Op); static void ExMoveExternals ( ACPI_PARSE_OBJECT *DefinitionBlockOp); /******************************************************************************* * * FUNCTION: ExDoExternal * * PARAMETERS: Op - Current Parse node * * RETURN: None * * DESCRIPTION: Add an External() definition to the global list. This list * is used to generate External opcodes. * ******************************************************************************/ void ExDoExternal ( ACPI_PARSE_OBJECT *Op) { ACPI_PARSE_OBJECT *ListOp; ACPI_PARSE_OBJECT *Prev; ACPI_PARSE_OBJECT *Next; ACPI_PARSE_OBJECT *ArgCountOp; + ACPI_PARSE_OBJECT *TypeOp; + ACPI_PARSE_OBJECT *ExternTypeOp = Op->Asl.Child->Asl.Next; + UINT32 ExternType; + UINT8 ParamCount = ASL_EXTERNAL_METHOD_UNKNOWN_PARAMS; + UINT32 ParamTypes[ACPI_METHOD_NUM_ARGS]; + ExternType = AnMapObjTypeToBtype (ExternTypeOp); + + /* + * The parser allows optional parameter return types regardless of the + * type. Check object type keyword emit error if optional parameter/return + * types exist. + * + * Check the parameter return type + */ + TypeOp = ExternTypeOp->Asl.Next; + if (TypeOp->Asl.Child) + { + /* Ignore the return type for now. */ + + (void) MtProcessTypeOp (TypeOp->Asl.Child); + if (ExternType != ACPI_BTYPE_METHOD) + { + sprintf (AslGbl_MsgBuffer, "Found type [%s]", AcpiUtGetTypeName(ExternType)); + AslError (ASL_ERROR, ASL_MSG_EXTERN_INVALID_RET_TYPE, TypeOp, + AslGbl_MsgBuffer); + } + } + + /* Check the parameter types */ + + TypeOp = TypeOp->Asl.Next; + if (TypeOp->Asl.Child) + { + ParamCount = MtProcessParameterTypeList (TypeOp->Asl.Child, ParamTypes); + if (ExternType != ACPI_BTYPE_METHOD) + { + sprintf (AslGbl_MsgBuffer, "Found type [%s]", AcpiUtGetTypeName(ExternType)); + AslError (ASL_ERROR, ASL_MSG_EXTERN_INVALID_PARAM_TYPE, TypeOp, + AslGbl_MsgBuffer); + } + } + ArgCountOp = Op->Asl.Child->Asl.Next->Asl.Next; ArgCountOp->Asl.AmlOpcode = AML_RAW_DATA_BYTE; ArgCountOp->Asl.ParseOpcode = PARSEOP_BYTECONST; - ArgCountOp->Asl.Value.Integer = 0; + ArgCountOp->Asl.Value.Integer = ParamCount; UtSetParseOpName (ArgCountOp); /* Create new list node of arbitrary type */ ListOp = TrAllocateOp (PARSEOP_DEFAULT_ARG); /* Store External node as child */ ListOp->Asl.Child = Op; ListOp->Asl.Next = NULL; if (AslGbl_ExternalsListHead) { /* Link new External to end of list */ Prev = AslGbl_ExternalsListHead; Next = Prev; while (Next) { Prev = Next; Next = Next->Asl.Next; } Prev->Asl.Next = ListOp; } else { AslGbl_ExternalsListHead = ListOp; } } /******************************************************************************* * * FUNCTION: ExInsertArgCount * * PARAMETERS: Op - Op for a method invocation * * RETURN: None * * DESCRIPTION: Obtain the number of arguments for a control method -- from * the actual invocation. * ******************************************************************************/ static void ExInsertArgCount ( ACPI_PARSE_OBJECT *Op) { ACPI_PARSE_OBJECT *Next; ACPI_PARSE_OBJECT *NameOp; ACPI_PARSE_OBJECT *Child; ACPI_PARSE_OBJECT *ArgCountOp; char * ExternalName; char * CallName; UINT16 ArgCount = 0; ACPI_STATUS Status; CallName = AcpiNsGetNormalizedPathname (Op->Asl.Node, TRUE); Next = AslGbl_ExternalsListHead; while (Next) { ArgCount = 0; /* Skip if External node already handled */ if (Next->Asl.Child->Asl.CompileFlags & OP_VISITED) { Next = Next->Asl.Next; continue; } NameOp = Next->Asl.Child->Asl.Child; ExternalName = AcpiNsGetNormalizedPathname (NameOp->Asl.Node, TRUE); if (strcmp (CallName, ExternalName)) { ACPI_FREE (ExternalName); Next = Next->Asl.Next; continue; } Next->Asl.Child->Asl.CompileFlags |= OP_VISITED; /* * Since we will reposition Externals to the Root, set Namepath * to the fully qualified name and recalculate the aml length */ Status = UtInternalizeName (ExternalName, &NameOp->Asl.Value.String); ACPI_FREE (ExternalName); if (ACPI_FAILURE (Status)) { AslError (ASL_ERROR, ASL_MSG_COMPILER_INTERNAL, NULL, "- Could not Internalize External"); break; } NameOp->Asl.AmlLength = strlen (NameOp->Asl.Value.String); /* Get argument count */ Child = Op->Asl.Child; while (Child) { ArgCount++; Child = Child->Asl.Next; } /* Setup ArgCount operand */ ArgCountOp = Next->Asl.Child->Asl.Child->Asl.Next->Asl.Next; ArgCountOp->Asl.Value.Integer = ArgCount; break; } ACPI_FREE (CallName); } /******************************************************************************* * * FUNCTION: ExAmlExternalWalkBegin * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: None * * DESCRIPTION: Parse tree walk to create external opcode list for methods. * ******************************************************************************/ ACPI_STATUS ExAmlExternalWalkBegin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { /* External list head saved in the definition block op */ if (Op->Asl.ParseOpcode == PARSEOP_DEFINITION_BLOCK) { AslGbl_ExternalsListHead = Op->Asl.Value.Arg; } if (!AslGbl_ExternalsListHead) { return (AE_OK); } if (Op->Asl.ParseOpcode != PARSEOP_METHODCALL) { return (AE_OK); } /* * The NameOp child under an ExternalOp gets turned into PARSE_METHODCALL * by XfNamespaceLocateBegin(). Ignore these. */ if (Op->Asl.Parent && Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_EXTERNAL) { return (AE_OK); } ExInsertArgCount (Op); return (AE_OK); } /******************************************************************************* * * FUNCTION: ExAmlExternalWalkEnd * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: None * * DESCRIPTION: Parse tree walk to create external opcode list for methods. * Here, we just want to catch the case where a definition block * has been completed. Then we move all of the externals into * a single block in the parse tree and thus the AML code. * ******************************************************************************/ ACPI_STATUS ExAmlExternalWalkEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { if (Op->Asl.ParseOpcode == PARSEOP_DEFINITION_BLOCK) { /* * Process any existing external list. (Support for * multiple definition blocks in a single file/compile) */ ExMoveExternals (Op); AslGbl_ExternalsListHead = NULL; } return (AE_OK); } /******************************************************************************* * * FUNCTION: ExMoveExternals * * PARAMETERS: DefinitionBlockOp - Op for current definition block * * RETURN: None * * DESCRIPTION: Move all externals present in the source file into a single * block of AML code, surrounded by an "If (0)" to prevent * AML interpreters from attempting to execute the External * opcodes. * ******************************************************************************/ static void ExMoveExternals ( ACPI_PARSE_OBJECT *DefinitionBlockOp) { ACPI_PARSE_OBJECT *ParentOp; ACPI_PARSE_OBJECT *ExternalOp; ACPI_PARSE_OBJECT *PredicateOp; ACPI_PARSE_OBJECT *NextOp; ACPI_PARSE_OBJECT *Prev; ACPI_PARSE_OBJECT *Next; char *ExternalName; ACPI_OBJECT_TYPE ObjType; ACPI_STATUS Status; UINT32 i; if (!AslGbl_ExternalsListHead) { return; } /* Remove the External nodes from the tree */ NextOp = AslGbl_ExternalsListHead; while (NextOp) { /* * The External is stored in child pointer of each node in the * list */ ExternalOp = NextOp->Asl.Child; /* Get/set the fully qualified name */ ExternalName = AcpiNsGetNormalizedPathname (ExternalOp->Asl.Node, TRUE); ExternalOp->Asl.ExternalName = ExternalName; ExternalOp->Asl.Namepath = ExternalName; /* Set line numbers (for listings, etc.) */ ExternalOp->Asl.LineNumber = 0; ExternalOp->Asl.LogicalLineNumber = 0; Next = ExternalOp->Asl.Child; Next->Asl.LineNumber = 0; Next->Asl.LogicalLineNumber = 0; if (Next->Asl.ParseOpcode == PARSEOP_NAMESEG) { Next->Asl.ParseOpcode = PARSEOP_NAMESTRING; } Next->Asl.ExternalName = ExternalName; Status = UtInternalizeName (ExternalName, &Next->Asl.Value.String); if (ACPI_FAILURE (Status)) { AslError (ASL_ERROR, ASL_MSG_COMPILER_INTERNAL, Next, "Could not internalize namestring"); return; } Next->Asl.AmlLength = strlen (Next->Asl.Value.String); Next = Next->Asl.Next; Next->Asl.LineNumber = 0; Next->Asl.LogicalLineNumber = 0; Next = Next->Asl.Next; Next->Asl.LineNumber = 0; Next->Asl.LogicalLineNumber = 0; Next = Next->Asl.Next; Next->Asl.LineNumber = 0; Next->Asl.LogicalLineNumber = 0; ParentOp = ExternalOp->Asl.Parent; Prev = Next = ParentOp->Asl.Child; /* Now find the External node's position in parse tree */ while (Next != ExternalOp) { Prev = Next; Next = Next->Asl.Next; } /* Remove the External from the parse tree */ if (Prev == ExternalOp) { /* External was the first child node */ ParentOp->Asl.Child = ExternalOp->Asl.Next; } Prev->Asl.Next = ExternalOp->Asl.Next; ExternalOp->Asl.Next = NULL; ExternalOp->Asl.Parent = AslGbl_ExternalsListHead; /* Point the External to the next in the list */ if (NextOp->Asl.Next) { ExternalOp->Asl.Next = NextOp->Asl.Next->Asl.Child; } NextOp = NextOp->Asl.Next; } /* * Loop again to remove MethodObj Externals for which * a MethodCall was not found (dead external reference) */ Prev = AslGbl_ExternalsListHead->Asl.Child; Next = Prev; while (Next) { ObjType = (ACPI_OBJECT_TYPE) Next->Asl.Child->Asl.Next->Asl.Value.Integer; if (ObjType == ACPI_TYPE_METHOD && !(Next->Asl.CompileFlags & OP_VISITED)) { if (Next == Prev) { AslGbl_ExternalsListHead->Asl.Child = Next->Asl.Next; Next->Asl.Next = NULL; Prev = AslGbl_ExternalsListHead->Asl.Child; Next = Prev; continue; } else { Prev->Asl.Next = Next->Asl.Next; Next->Asl.Next = NULL; Next = Prev->Asl.Next; continue; } } Prev = Next; Next = Next->Asl.Next; } /* If list is now empty, don't bother to make If (0) block */ if (!AslGbl_ExternalsListHead->Asl.Child) { return; } /* Convert Gbl_ExternalsListHead parent to If(). */ AslGbl_ExternalsListHead->Asl.ParseOpcode = PARSEOP_IF; AslGbl_ExternalsListHead->Asl.AmlOpcode = AML_IF_OP; AslGbl_ExternalsListHead->Asl.CompileFlags = OP_AML_PACKAGE; UtSetParseOpName (AslGbl_ExternalsListHead); /* Create a Zero op for the If predicate */ PredicateOp = TrAllocateOp (PARSEOP_ZERO); PredicateOp->Asl.AmlOpcode = AML_ZERO_OP; PredicateOp->Asl.Parent = AslGbl_ExternalsListHead; PredicateOp->Asl.Child = NULL; PredicateOp->Asl.Next = AslGbl_ExternalsListHead->Asl.Child; AslGbl_ExternalsListHead->Asl.Child = PredicateOp; /* Set line numbers (for listings, etc.) */ AslGbl_ExternalsListHead->Asl.LineNumber = 0; AslGbl_ExternalsListHead->Asl.LogicalLineNumber = 0; PredicateOp->Asl.LineNumber = 0; PredicateOp->Asl.LogicalLineNumber = 0; /* Insert block back in the list */ Prev = DefinitionBlockOp->Asl.Child; Next = Prev; /* Find last default arg */ for (i = 0; i < 6; i++) { Prev = Next; Next = Prev->Asl.Next; } if (Next) { /* Definition Block is not empty */ AslGbl_ExternalsListHead->Asl.Next = Next; } else { /* Definition Block is empty. */ AslGbl_ExternalsListHead->Asl.Next = NULL; } Prev->Asl.Next = AslGbl_ExternalsListHead; AslGbl_ExternalsListHead->Asl.Parent = Prev->Asl.Parent; } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslload.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslload.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslload.c (revision 357966) @@ -1,1307 +1,1427 @@ /****************************************************************************** * * Module Name: dswload - Dispatcher namespace load callbacks * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #include #include #include #include "aslcompiler.y.h" #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("aslload") /* Local prototypes */ static ACPI_STATUS LdLoadFieldElements ( UINT32 AmlType, ACPI_PARSE_OBJECT *Op, ACPI_WALK_STATE *WalkState); static ACPI_STATUS LdLoadResourceElements ( ACPI_PARSE_OBJECT *Op, ACPI_WALK_STATE *WalkState); static ACPI_STATUS LdNamespace1Begin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); static ACPI_STATUS LdNamespace2Begin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); static ACPI_STATUS LdCommonNamespaceEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); static void LdCheckSpecialNames ( ACPI_NAMESPACE_NODE *Node, ACPI_PARSE_OBJECT *Op); +static ACPI_STATUS +LdAnalyzeExternals ( + ACPI_NAMESPACE_NODE *Node, + ACPI_PARSE_OBJECT *Op, + ACPI_OBJECT_TYPE ExternalOpType, + ACPI_OBJECT_TYPE ObjectType, + ACPI_WALK_STATE *WalkState); + + /******************************************************************************* * * FUNCTION: LdLoadNamespace * * PARAMETERS: RootOp - Root of the parse tree * * RETURN: Status * * DESCRIPTION: Perform a walk of the parse tree that in turn loads all of the * named ASL/AML objects into the namespace. The namespace is * constructed in order to resolve named references and references * to named fields within resource templates/descriptors. * ******************************************************************************/ ACPI_STATUS LdLoadNamespace ( ACPI_PARSE_OBJECT *RootOp) { ACPI_WALK_STATE *WalkState; /* Create a new walk state */ WalkState = AcpiDsCreateWalkState (0, NULL, NULL, NULL); if (!WalkState) { return (AE_NO_MEMORY); } /* Walk the entire parse tree, first pass */ TrWalkParseTree (RootOp, ASL_WALK_VISIT_TWICE, LdNamespace1Begin, LdCommonNamespaceEnd, WalkState); /* Second pass to handle forward references */ TrWalkParseTree (RootOp, ASL_WALK_VISIT_TWICE, LdNamespace2Begin, LdCommonNamespaceEnd, WalkState); /* Dump the namespace if debug is enabled */ if (AcpiDbgLevel & ACPI_LV_TABLES) { AcpiNsDumpTables (ACPI_NS_ALL, ACPI_UINT32_MAX); } ACPI_FREE (WalkState); return (AE_OK); } /******************************************************************************* * * FUNCTION: LdLoadFieldElements * * PARAMETERS: AmlType - Type to search * Op - Parent node (Field) * WalkState - Current walk state * * RETURN: Status * * DESCRIPTION: Enter the named elements of the field (children of the parent) * into the namespace. * ******************************************************************************/ static ACPI_STATUS LdLoadFieldElements ( UINT32 AmlType, ACPI_PARSE_OBJECT *Op, ACPI_WALK_STATE *WalkState) { ACPI_PARSE_OBJECT *Child = NULL; ACPI_PARSE_OBJECT *SourceRegion; ACPI_NAMESPACE_NODE *Node; ACPI_STATUS Status; char *ExternalPath; SourceRegion = UtGetArg (Op, 0); if (SourceRegion) { Status = AcpiNsLookup (WalkState->ScopeInfo, SourceRegion->Asl.Value.String, AmlType, ACPI_IMODE_EXECUTE, ACPI_NS_SEARCH_PARENT | ACPI_NS_DONT_OPEN_SCOPE, NULL, &Node); if (Status == AE_NOT_FOUND) { /* * If the named object is not found, it means that it is either a * forward reference or the named object does not exist. */ SourceRegion->Asl.CompileFlags |= OP_NOT_FOUND_DURING_LOAD; } } /* Get the first named field element */ switch (Op->Asl.AmlOpcode) { case AML_BANK_FIELD_OP: Child = UtGetArg (Op, 6); break; case AML_INDEX_FIELD_OP: Child = UtGetArg (Op, 5); break; case AML_FIELD_OP: Child = UtGetArg (Op, 4); break; default: /* No other opcodes should arrive here */ return (AE_BAD_PARAMETER); } /* Enter all elements into the namespace */ while (Child) { switch (Child->Asl.AmlOpcode) { case AML_INT_RESERVEDFIELD_OP: case AML_INT_ACCESSFIELD_OP: case AML_INT_CONNECTION_OP: break; default: Status = AcpiNsLookup (WalkState->ScopeInfo, Child->Asl.Value.String, ACPI_TYPE_LOCAL_REGION_FIELD, ACPI_IMODE_LOAD_PASS1, ACPI_NS_NO_UPSEARCH | ACPI_NS_DONT_OPEN_SCOPE | ACPI_NS_ERROR_IF_FOUND, NULL, &Node); if (ACPI_FAILURE (Status)) { if (Status != AE_ALREADY_EXISTS) { AslError (ASL_ERROR, ASL_MSG_CORE_EXCEPTION, Child, Child->Asl.Value.String); return (Status); } else if (Status == AE_ALREADY_EXISTS && (Node->Flags & ANOBJ_IS_EXTERNAL)) { Node->Type = (UINT8) ACPI_TYPE_LOCAL_REGION_FIELD; Node->Flags &= ~ANOBJ_IS_EXTERNAL; } else { /* * The name already exists in this scope * But continue processing the elements */ ExternalPath = AcpiNsGetNormalizedPathname (Node, TRUE); AslDualParseOpError (ASL_ERROR, ASL_MSG_NAME_EXISTS, Child, ExternalPath, ASL_MSG_FOUND_HERE, Node->Op, ExternalPath); if (ExternalPath) { ACPI_FREE (ExternalPath); } } } else { Child->Asl.Node = Node; Node->Op = Child; } break; } Child = Child->Asl.Next; } return (AE_OK); } /******************************************************************************* * * FUNCTION: LdLoadResourceElements * * PARAMETERS: Op - Parent node (Resource Descriptor) * WalkState - Current walk state * * RETURN: Status * * DESCRIPTION: Enter the named elements of the resource descriptor (children * of the parent) into the namespace. * * NOTE: In the real AML namespace, these named elements never exist. But * we simply use the namespace here as a symbol table so we can look * them up as they are referenced. * ******************************************************************************/ static ACPI_STATUS LdLoadResourceElements ( ACPI_PARSE_OBJECT *Op, ACPI_WALK_STATE *WalkState) { ACPI_PARSE_OBJECT *InitializerOp = NULL; ACPI_NAMESPACE_NODE *Node; ACPI_STATUS Status; char *ExternalPath; /* * Enter the resource name into the namespace. Name must not already exist. * This opens a scope, so later field names are guaranteed to be new/unique. */ Status = AcpiNsLookup (WalkState->ScopeInfo, Op->Asl.Namepath, ACPI_TYPE_LOCAL_RESOURCE, ACPI_IMODE_LOAD_PASS1, ACPI_NS_NO_UPSEARCH | ACPI_NS_ERROR_IF_FOUND, WalkState, &Node); if (ACPI_FAILURE (Status)) { if (Status == AE_ALREADY_EXISTS) { /* Actual node causing the error was saved in ParentMethod */ ExternalPath = AcpiNsGetNormalizedPathname (Node, TRUE); AslDualParseOpError (ASL_ERROR, ASL_MSG_NAME_EXISTS, (ACPI_PARSE_OBJECT *) Op->Asl.ParentMethod, ExternalPath, ASL_MSG_FOUND_HERE, Node->Op, ExternalPath); if (ExternalPath) { ACPI_FREE (ExternalPath); } return (AE_OK); } return (Status); } Node->Value = (UINT32) Op->Asl.Value.Integer; Node->Op = Op; Op->Asl.Node = Node; /* * Now enter the predefined fields, for easy lookup when referenced * by the source ASL */ InitializerOp = ASL_GET_CHILD_NODE (Op); while (InitializerOp) { if (InitializerOp->Asl.ExternalName) { Status = AcpiNsLookup (WalkState->ScopeInfo, InitializerOp->Asl.ExternalName, ACPI_TYPE_LOCAL_RESOURCE_FIELD, ACPI_IMODE_LOAD_PASS1, ACPI_NS_NO_UPSEARCH | ACPI_NS_DONT_OPEN_SCOPE, NULL, &Node); if (ACPI_FAILURE (Status)) { return (Status); } /* * Store the field offset and length in the namespace node * so it can be used when the field is referenced */ Node->Value = InitializerOp->Asl.Value.Tag.BitOffset; Node->Length = InitializerOp->Asl.Value.Tag.BitLength; InitializerOp->Asl.Node = Node; Node->Op = InitializerOp; } InitializerOp = ASL_GET_PEER_NODE (InitializerOp); } return (AE_OK); } /******************************************************************************* * * FUNCTION: LdNamespace1Begin * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: Status * * DESCRIPTION: Descending callback used during the parse tree walk. If this * is a named AML opcode, enter into the namespace * ******************************************************************************/ static ACPI_STATUS LdNamespace1Begin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { ACPI_WALK_STATE *WalkState = (ACPI_WALK_STATE *) Context; ACPI_NAMESPACE_NODE *Node; ACPI_PARSE_OBJECT *MethodOp; ACPI_STATUS Status; ACPI_OBJECT_TYPE ObjectType; ACPI_OBJECT_TYPE ActualObjectType = ACPI_TYPE_ANY; char *Path; UINT32 Flags = ACPI_NS_NO_UPSEARCH; ACPI_PARSE_OBJECT *Arg; UINT32 i; BOOLEAN ForceNewScope = FALSE; const ACPI_OPCODE_INFO *OpInfo; ACPI_PARSE_OBJECT *ParentOp; char *ExternalPath; ACPI_FUNCTION_NAME (LdNamespace1Begin); ACPI_DEBUG_PRINT ((ACPI_DB_DISPATCH, "Op %p [%s]\n", Op, Op->Asl.ParseOpName)); /* * We are only interested in opcodes that have an associated name * (or multiple names) */ switch (Op->Asl.AmlOpcode) { case AML_INDEX_FIELD_OP: Status = LdLoadFieldElements (ACPI_TYPE_LOCAL_REGION_FIELD, Op, WalkState); return (Status); case AML_BANK_FIELD_OP: case AML_FIELD_OP: Status = LdLoadFieldElements (ACPI_TYPE_REGION, Op, WalkState); return (Status); case AML_INT_CONNECTION_OP: if (Op->Asl.Child->Asl.AmlOpcode != AML_INT_NAMEPATH_OP) { break; } Arg = Op->Asl.Child; Status = AcpiNsLookup (WalkState->ScopeInfo, Arg->Asl.ExternalName, ACPI_TYPE_ANY, ACPI_IMODE_EXECUTE, ACPI_NS_SEARCH_PARENT, WalkState, &Node); if (ACPI_FAILURE (Status)) { break; } break; default: /* All other opcodes go below */ break; } /* Check if this object has already been installed in the namespace */ if (Op->Asl.Node) { return (AE_OK); } /* Check for a possible illegal forward reference */ if ((Op->Asl.ParseOpcode == PARSEOP_NAMESEG) || - (Op->Asl.ParseOpcode == PARSEOP_NAMESTRING)) + (Op->Asl.ParseOpcode == PARSEOP_NAMESTRING) || + (Op->Asl.ParseOpcode == PARSEOP_METHODCALL)) { /* * Op->Asl.Namepath will be NULL for these opcodes. * These opcodes are guaranteed to have a parent. * Examine the parent opcode. */ ParentOp = Op->Asl.Parent; OpInfo = AcpiPsGetOpcodeInfo (ParentOp->Asl.AmlOpcode); /* * Exclude all operators that actually declare a new name: * Name (ABCD, 1) -> Ignore (AML_CLASS_NAMED_OBJECT) * We only want references to named objects: * Store (2, WXYZ) -> Attempt to resolve the name */ - if (OpInfo->Class == AML_CLASS_NAMED_OBJECT) + if ((Op->Asl.ParseOpcode != PARSEOP_METHODCALL) && + (OpInfo->Class == AML_CLASS_NAMED_OBJECT)) { return (AE_OK); } /* * Check if the referenced object exists at this point during * the load: * 1) If it exists, then this cannot be a forward reference. * 2) If it does not exist, it could be a forward reference or * it truly does not exist (and no external declaration). */ Status = AcpiNsLookup (WalkState->ScopeInfo, Op->Asl.Value.Name, ACPI_TYPE_ANY, ACPI_IMODE_EXECUTE, ACPI_NS_SEARCH_PARENT | ACPI_NS_DONT_OPEN_SCOPE, WalkState, &Node); if (Status == AE_NOT_FOUND) { /* * This is either a forward reference or the object truly * does not exist. The two cases can only be differentiated * during the cross-reference stage later. Mark the Op/Name * as not-found for now to indicate the need for further * processing. * * Special case: Allow forward references from elements of * Package objects. This provides compatibility with other * ACPI implementations. To correctly implement this, the * ACPICA table load defers package resolution until the entire * namespace has been loaded. */ if ((ParentOp->Asl.ParseOpcode != PARSEOP_PACKAGE) && (ParentOp->Asl.ParseOpcode != PARSEOP_VAR_PACKAGE)) { Op->Asl.CompileFlags |= OP_NOT_FOUND_DURING_LOAD; } return (AE_OK); } return (Status); } Path = Op->Asl.Namepath; if (!Path) { return (AE_OK); } /* Map the raw opcode into an internal object type */ switch (Op->Asl.ParseOpcode) { case PARSEOP_NAME: Arg = Op->Asl.Child; /* Get the NameSeg/NameString node */ Arg = Arg->Asl.Next; /* First peer is the object to be associated with the name */ /* * If this name refers to a ResourceTemplate, we will need to open * a new scope so that the resource subfield names can be entered into * the namespace underneath this name */ if (Op->Asl.CompileFlags & OP_IS_RESOURCE_DESC) { ForceNewScope = TRUE; } /* Get the data type associated with the named object, not the name itself */ /* Log2 loop to convert from Btype (binary) to Etype (encoded) */ ObjectType = 1; for (i = 1; i < Arg->Asl.AcpiBtype; i *= 2) { ObjectType++; } break; case PARSEOP_EXTERNAL: /* * "External" simply enters a name and type into the namespace. * We must be careful to not open a new scope, however, no matter * what type the external name refers to (e.g., a method) * * first child is name, next child is ObjectType */ ActualObjectType = (UINT8) Op->Asl.Child->Asl.Next->Asl.Value.Integer; ObjectType = ACPI_TYPE_ANY; /* * We will mark every new node along the path as "External". This * allows some or all of the nodes to be created later in the ASL * code. Handles cases like this: * * External (\_SB_.PCI0.ABCD, IntObj) * Scope (_SB_) * { * Device (PCI0) * { * } * } * Method (X) * { * Store (\_SB_.PCI0.ABCD, Local0) * } */ Flags |= ACPI_NS_EXTERNAL; break; case PARSEOP_DEFAULT_ARG: if (Op->Asl.CompileFlags == OP_IS_RESOURCE_DESC) { Status = LdLoadResourceElements (Op, WalkState); return_ACPI_STATUS (Status); } ObjectType = AslMapNamedOpcodeToDataType (Op->Asl.AmlOpcode); break; case PARSEOP_SCOPE: /* * The name referenced by Scope(Name) must already exist at this point. * In other words, forward references for Scope() are not supported. * The only real reason for this is that the MS interpreter cannot * handle this case. Perhaps someday this case can go away. */ Status = AcpiNsLookup (WalkState->ScopeInfo, Path, ACPI_TYPE_ANY, ACPI_IMODE_EXECUTE, ACPI_NS_SEARCH_PARENT, WalkState, &Node); if (ACPI_FAILURE (Status)) { if (Status == AE_NOT_FOUND) { /* The name was not found, go ahead and create it */ Status = AcpiNsLookup (WalkState->ScopeInfo, Path, ACPI_TYPE_LOCAL_SCOPE, ACPI_IMODE_LOAD_PASS1, Flags, WalkState, &Node); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* However, this is an error -- operand to Scope must exist */ if (strlen (Op->Asl.ExternalName) == ACPI_NAMESEG_SIZE) { AslError (ASL_ERROR, ASL_MSG_NOT_FOUND, Op, Op->Asl.ExternalName); } else { AslError (ASL_ERROR, ASL_MSG_NAMEPATH_NOT_EXIST, Op, Op->Asl.ExternalName); } goto FinishNode; } AslCoreSubsystemError (Op, Status, "Failure from namespace lookup", FALSE); return_ACPI_STATUS (Status); } else /* Status AE_OK */ { /* * Do not allow references to external scopes from the DSDT. * This is because the DSDT is always loaded first, and the * external reference cannot be resolved -- causing a runtime * error because Scope() must be resolved immediately. * 10/2015. */ if ((Node->Flags & ANOBJ_IS_EXTERNAL) && (ACPI_COMPARE_NAMESEG (AslGbl_TableSignature, "DSDT"))) { /* However, allowed if the reference is within a method */ MethodOp = Op->Asl.Parent; while (MethodOp && (MethodOp->Asl.ParseOpcode != PARSEOP_METHOD)) { MethodOp = MethodOp->Asl.Parent; } if (!MethodOp) { /* Not in a control method, error */ AslError (ASL_ERROR, ASL_MSG_CROSS_TABLE_SCOPE, Op, NULL); } } } /* We found a node with this name, now check the type */ switch (Node->Type) { case ACPI_TYPE_LOCAL_SCOPE: case ACPI_TYPE_DEVICE: case ACPI_TYPE_POWER: case ACPI_TYPE_PROCESSOR: case ACPI_TYPE_THERMAL: /* These are acceptable types - they all open a new scope */ break; case ACPI_TYPE_INTEGER: case ACPI_TYPE_STRING: case ACPI_TYPE_BUFFER: /* * These types we will allow, but we will change the type. * This enables some existing code of the form: * * Name (DEB, 0) * Scope (DEB) { ... } * * Which is used to workaround the fact that the MS interpreter * does not allow Scope() forward references. */ sprintf (AslGbl_MsgBuffer, "%s [%s], changing type to [Scope]", Op->Asl.ExternalName, AcpiUtGetTypeName (Node->Type)); AslError (ASL_REMARK, ASL_MSG_SCOPE_TYPE, Op, AslGbl_MsgBuffer); /* Switch the type to scope, open the new scope */ Node->Type = ACPI_TYPE_LOCAL_SCOPE; Status = AcpiDsScopeStackPush (Node, ACPI_TYPE_LOCAL_SCOPE, WalkState); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } break; default: /* All other types are an error */ sprintf (AslGbl_MsgBuffer, "%s [%s]", Op->Asl.ExternalName, AcpiUtGetTypeName (Node->Type)); AslError (ASL_ERROR, ASL_MSG_SCOPE_TYPE, Op, AslGbl_MsgBuffer); /* * However, switch the type to be an actual scope so * that compilation can continue without generating a whole * cascade of additional errors. Open the new scope. */ Node->Type = ACPI_TYPE_LOCAL_SCOPE; Status = AcpiDsScopeStackPush (Node, ACPI_TYPE_LOCAL_SCOPE, WalkState); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } break; } Status = AE_OK; goto FinishNode; default: ObjectType = AslMapNamedOpcodeToDataType (Op->Asl.AmlOpcode); break; } ACPI_DEBUG_PRINT ((ACPI_DB_DISPATCH, "Loading name: %s, (%s)\n", Op->Asl.ExternalName, AcpiUtGetTypeName (ObjectType))); /* The name must not already exist */ Flags |= ACPI_NS_ERROR_IF_FOUND; /* * For opcodes that enter new names into the namespace, * all prefix NameSegs must exist. */ WalkState->OpInfo = AcpiPsGetOpcodeInfo (Op->Asl.AmlOpcode); if (((WalkState->OpInfo->Flags & AML_NAMED) || (WalkState->OpInfo->Flags & AML_CREATE)) && (Op->Asl.AmlOpcode != AML_EXTERNAL_OP)) { Flags |= ACPI_NS_PREFIX_MUST_EXIST; } /* * Enter the named type into the internal namespace. We enter the name * as we go downward in the parse tree. Any necessary subobjects that * involve arguments to the opcode must be created as we go back up the * parse tree later. */ Status = AcpiNsLookup (WalkState->ScopeInfo, Path, ObjectType, ACPI_IMODE_LOAD_PASS1, Flags, WalkState, &Node); if (ACPI_FAILURE (Status)) { if (Status == AE_ALREADY_EXISTS) { /* The name already exists in this scope */ if (Node->Type == ACPI_TYPE_LOCAL_SCOPE) { /* Allow multiple references to the same scope */ Node->Type = (UINT8) ObjectType; Status = AE_OK; } - else if ((Node->Flags & ANOBJ_IS_EXTERNAL) && - (Op->Asl.ParseOpcode != PARSEOP_EXTERNAL)) + else if ((Node->Flags & ANOBJ_IS_EXTERNAL) || + (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL)) { - /* - * Allow one create on an object or segment that was - * previously declared External - */ - Node->Flags &= ~ANOBJ_IS_EXTERNAL; - Node->Type = (UINT8) ObjectType; - - /* Just retyped a node, probably will need to open a scope */ - - if (AcpiNsOpensScope (ObjectType)) + Status = LdAnalyzeExternals (Node, Op, ActualObjectType, + ObjectType, WalkState); + if (ACPI_FAILURE (Status)) { - Status = AcpiDsScopeStackPush (Node, ObjectType, WalkState); - if (ACPI_FAILURE (Status)) + if (Status == AE_ERROR) { - return_ACPI_STATUS (Status); + /* + * The use of AE_ERROR here indicates that there was a + * compiler error emitted in LdAnalyzeExternals which + * means that the caller should proceed to the next Op + * for analysis of subsequent parse objects. + */ + Status = AE_OK; } + return_ACPI_STATUS (Status); } - - Status = AE_OK; } - else if (!(Node->Flags & ANOBJ_IS_EXTERNAL) && - (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL)) - { - /* - * Allow externals in same scope as the definition of the - * actual object. Similar to C. Allows multiple definition - * blocks that refer to each other in the same file. - */ - Status = AE_OK; - } - else if ((Node->Flags & ANOBJ_IS_EXTERNAL) && - (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL) && - (ObjectType == ACPI_TYPE_ANY)) - { - /* Allow update of externals of unknown type. */ - - if (AcpiNsOpensScope (ActualObjectType)) - { - Node->Type = (UINT8) ActualObjectType; - Status = AE_OK; - } - else - { - sprintf (AslGbl_MsgBuffer, "%s [%s]", Op->Asl.ExternalName, - AcpiUtGetTypeName (Node->Type)); - AslError (ASL_ERROR, ASL_MSG_SCOPE_TYPE, Op, AslGbl_MsgBuffer); - return_ACPI_STATUS (AE_OK); - } - } else { /* Valid error, object already exists */ ExternalPath = AcpiNsGetNormalizedPathname (Node, TRUE); AslDualParseOpError (ASL_ERROR, ASL_MSG_NAME_EXISTS, Op, ExternalPath, ASL_MSG_FOUND_HERE, Node->Op, ExternalPath); if (ExternalPath) { ACPI_FREE (ExternalPath); } return_ACPI_STATUS (AE_OK); } } else if (AE_NOT_FOUND) { /* * One or more prefix NameSegs of the NamePath do not exist * (all of them must exist). Attempt to continue compilation * by setting the current scope to the root. */ Node = AcpiGbl_RootNode; Status = AE_OK; } else { /* Flag all other errors as coming from the ACPICA core */ AslCoreSubsystemError (Op, Status, "Failure from namespace lookup", FALSE); return_ACPI_STATUS (Status); } } /* Check special names like _WAK and _PTS */ LdCheckSpecialNames (Node, Op); if (ForceNewScope) { Status = AcpiDsScopeStackPush (Node, ObjectType, WalkState); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } } FinishNode: /* * Point the parse node to the new namespace node, and point * the Node back to the original Parse node */ Op->Asl.Node = Node; Node->Op = Op; /* * Set the actual data type if appropriate (EXTERNAL term only) * As of 11/19/2019, ASL External() does not support parameter * counts. When an External method is loaded, the parameter count is - * unknown setting Node->Value to ASL_EXTERNAL_METHOD_UNKNOWN_PARAMS - * indicates that the parameter count for this method is unknown. - * This information is used in ASL cross reference to help determine the - * parameter count through method calls. + * recorded in the external's arg count parameter. The parameter count may + * or may not be known in the declaration. If the value of this node turns + * out to be ASL_EXTERNAL_METHOD_UNKNOWN_PARAMS, it indicates that + * we do not know the parameter count and that we must look at the usage of + * the External method call to get this information. */ if (ActualObjectType != ACPI_TYPE_ANY) { Node->Type = (UINT8) ActualObjectType; - Node->Value = ASL_EXTERNAL_METHOD_UNKNOWN_PARAMS; + Node->Value = (UINT32) + Op->Asl.Child->Asl.Next->Asl.Next->Asl.Value.Integer; } if (Op->Asl.ParseOpcode == PARSEOP_METHOD) { /* * Get the method argument count from "Extra" and save * it in the namespace node */ Node->Value = (UINT32) Op->Asl.Extra; } return_ACPI_STATUS (Status); +} + + +/******************************************************************************* + * + * FUNCTION: LdAnalyzeExternals + * + * PARAMETERS: Node - Node that represents the named object + * Op - Named object declaring this named object + * ExternalOpType - Type of ExternalOp + * ObjectType - Type of Declared object + * WalkState - Current WalkState + * + * RETURN: Status + * + * DESCRIPTION: Node and Op represents an identically named object declaration + * that is either declared by the ASL external keyword or declared + * by operators that declare named objects (i.e. Name, Device, + * OperationRegion, and etc.). This function ensures that the + * declarations do not contradict each other. + * + ******************************************************************************/ + +static ACPI_STATUS +LdAnalyzeExternals ( + ACPI_NAMESPACE_NODE *Node, + ACPI_PARSE_OBJECT *Op, + ACPI_OBJECT_TYPE ExternalOpType, + ACPI_OBJECT_TYPE ObjectType, + ACPI_WALK_STATE *WalkState) +{ + ACPI_STATUS Status = AE_OK; + ACPI_OBJECT_TYPE ActualExternalOpType; + ACPI_OBJECT_TYPE ActualOpType; + ACPI_PARSE_OBJECT *ExternalOp; + ACPI_PARSE_OBJECT *ActualOp; + + + /* + * The declaration represented by Node and Op must have the same type. + * The type of the external Op is represented by ExternalOpType. However, + * the type of the pre-existing declaration depends on whether if Op + * is an external declaration or an actual declaration. + */ + if (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL) + { + ActualExternalOpType = ExternalOpType; + ActualOpType = Node->Type; + } + else + { + ActualExternalOpType = Node->Type; + ActualOpType = ObjectType; + } + + if ((ActualOpType != ACPI_TYPE_ANY) && + (ActualExternalOpType != ACPI_TYPE_ANY) && + (ActualExternalOpType != ActualOpType)) + { + if (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL && + Node->Op->Asl.ParseOpcode == PARSEOP_EXTERNAL) + { + AslDualParseOpError (ASL_ERROR, + ASL_MSG_DUPLICATE_EXTERN_MISMATCH, Op, NULL, + ASL_MSG_DUPLICATE_EXTERN_FOUND_HERE, Node->Op, NULL); + } + else + { + if (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL && + Node->Op->Asl.ParseOpcode != PARSEOP_EXTERNAL) + { + ExternalOp = Op; + ActualOp = Node->Op; + } + else + { + ExternalOp = Node->Op; + ActualOp = Op; + } + AslDualParseOpError (ASL_ERROR, + ASL_MSG_DECLARATION_TYPE_MISMATCH, ExternalOp, NULL, + ASL_MSG_TYPE_MISMATCH_FOUND_HERE, ActualOp, NULL); + } + } + + if ((Node->Flags & ANOBJ_IS_EXTERNAL) && + (Op->Asl.ParseOpcode != PARSEOP_EXTERNAL)) + { + /* + * Allow one create on an object or segment that was + * previously declared External + */ + Node->Flags &= ~ANOBJ_IS_EXTERNAL; + Node->Type = (UINT8) ObjectType; + + /* Just retyped a node, probably will need to open a scope */ + + if (AcpiNsOpensScope (ObjectType)) + { + Status = AcpiDsScopeStackPush (Node, ObjectType, WalkState); + if (ACPI_FAILURE (Status)) + { + return (Status); + } + } + + Status = AE_OK; + } + else if (!(Node->Flags & ANOBJ_IS_EXTERNAL) && + (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL)) + { + /* + * Allow externals in same scope as the definition of the + * actual object. Similar to C. Allows multiple definition + * blocks that refer to each other in the same file. + */ + Status = AE_OK; + } + else if ((Node->Flags & ANOBJ_IS_EXTERNAL) && + (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL) && + (ObjectType == ACPI_TYPE_ANY)) + { + /* Allow update of externals of unknown type. */ + + if (AcpiNsOpensScope (ExternalOpType)) + { + Node->Type = (UINT8) ExternalOpType; + Status = AE_OK; + } + else + { + sprintf (AslGbl_MsgBuffer, "%s [%s]", Op->Asl.ExternalName, + AcpiUtGetTypeName (Node->Type)); + AslError (ASL_ERROR, ASL_MSG_SCOPE_TYPE, Op, AslGbl_MsgBuffer); + Status = AE_ERROR; + } + } + + return (Status); } /******************************************************************************* * * FUNCTION: LdCheckSpecialNames * * PARAMETERS: Node - Node that represents the named object * Op - Named object declaring this named object * * RETURN: None * * DESCRIPTION: Check if certain named objects are declared in the incorrect * scope. Special named objects are listed in * AslGbl_SpecialNamedObjects and can only be declared at the root * scope. _UID inside of a processor declaration must not be a * string. * ******************************************************************************/ static void LdCheckSpecialNames ( ACPI_NAMESPACE_NODE *Node, ACPI_PARSE_OBJECT *Op) { UINT32 i; for (i = 0; i < MAX_SPECIAL_NAMES; i++) { if (ACPI_COMPARE_NAMESEG(Node->Name.Ascii, AslGbl_SpecialNamedObjects[i]) && Node->Parent != AcpiGbl_RootNode) { AslError (ASL_ERROR, ASL_MSG_INVALID_SPECIAL_NAME, Op, Op->Asl.ExternalName); return; } } if (ACPI_COMPARE_NAMESEG (Node->Name.Ascii, "_UID") && Node->Parent->Type == ACPI_TYPE_PROCESSOR && Node->Type == ACPI_TYPE_STRING) { AslError (ASL_ERROR, ASL_MSG_INVALID_PROCESSOR_UID , Op, "found a string"); } } /******************************************************************************* * * FUNCTION: LdNamespace2Begin * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: Status * * DESCRIPTION: Descending callback used during the pass 2 parse tree walk. * Second pass resolves some forward references. * * Notes: * Currently only needs to handle the Alias operator. * Could be used to allow forward references from the Scope() operator, but * the MS interpreter does not allow this, so this compiler does not either. * ******************************************************************************/ static ACPI_STATUS LdNamespace2Begin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { ACPI_WALK_STATE *WalkState = (ACPI_WALK_STATE *) Context; ACPI_STATUS Status; ACPI_NAMESPACE_NODE *Node; ACPI_OBJECT_TYPE ObjectType; BOOLEAN ForceNewScope = FALSE; ACPI_PARSE_OBJECT *Arg; char *Path; ACPI_NAMESPACE_NODE *TargetNode; ACPI_FUNCTION_NAME (LdNamespace2Begin); ACPI_DEBUG_PRINT ((ACPI_DB_DISPATCH, "Op %p [%s]\n", Op, Op->Asl.ParseOpName)); /* Ignore Ops with no namespace node */ Node = Op->Asl.Node; if (!Node) { return (AE_OK); } /* Get the type to determine if we should push the scope */ if ((Op->Asl.ParseOpcode == PARSEOP_DEFAULT_ARG) && (Op->Asl.CompileFlags == OP_IS_RESOURCE_DESC)) { ObjectType = ACPI_TYPE_LOCAL_RESOURCE; } else { ObjectType = AslMapNamedOpcodeToDataType (Op->Asl.AmlOpcode); } /* Push scope for Resource Templates */ if (Op->Asl.ParseOpcode == PARSEOP_NAME) { if (Op->Asl.CompileFlags & OP_IS_RESOURCE_DESC) { ForceNewScope = TRUE; } } /* Push the scope stack */ if (ForceNewScope || AcpiNsOpensScope (ObjectType)) { Status = AcpiDsScopeStackPush (Node, ObjectType, WalkState); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } } if (Op->Asl.ParseOpcode == PARSEOP_ALIAS) { /* * Complete the alias node by getting and saving the target node. * First child is the alias target */ Arg = Op->Asl.Child; /* Get the target pathname */ Path = Arg->Asl.Namepath; if (!Path) { Status = UtInternalizeName (Arg->Asl.ExternalName, &Path); if (ACPI_FAILURE (Status)) { return (Status); } } /* Get the NS node associated with the target. It must exist. */ Status = AcpiNsLookup (WalkState->ScopeInfo, Path, ACPI_TYPE_ANY, ACPI_IMODE_EXECUTE, ACPI_NS_SEARCH_PARENT | ACPI_NS_DONT_OPEN_SCOPE, WalkState, &TargetNode); if (ACPI_FAILURE (Status)) { if (Status == AE_NOT_FOUND) { /* Standalone NameSeg vs. NamePath */ if (strlen (Arg->Asl.ExternalName) == ACPI_NAMESEG_SIZE) { AslError (ASL_ERROR, ASL_MSG_NOT_FOUND, Op, Arg->Asl.ExternalName); } else { AslError (ASL_ERROR, ASL_MSG_NAMEPATH_NOT_EXIST, Op, Arg->Asl.ExternalName); } #if 0 /* * NOTE: Removed 10/2018 to enhance compiler error reporting. No * regressions seen. */ /* * The name was not found, go ahead and create it. * This prevents more errors later. */ Status = AcpiNsLookup (WalkState->ScopeInfo, Path, ACPI_TYPE_ANY, ACPI_IMODE_LOAD_PASS1, ACPI_NS_NO_UPSEARCH, WalkState, &Node); #endif return (Status); /* Removed: return (AE_OK)*/ } AslCoreSubsystemError (Op, Status, "Failure from namespace lookup", FALSE); return (AE_OK); } /* Save the target node within the alias node */ Node->Object = ACPI_CAST_PTR (ACPI_OPERAND_OBJECT, TargetNode); } return (AE_OK); } /******************************************************************************* * * FUNCTION: LdCommonNamespaceEnd * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: Status * * DESCRIPTION: Ascending callback used during the loading of the namespace, * We only need to worry about managing the scope stack here. * ******************************************************************************/ static ACPI_STATUS LdCommonNamespaceEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { ACPI_WALK_STATE *WalkState = (ACPI_WALK_STATE *) Context; ACPI_OBJECT_TYPE ObjectType; BOOLEAN ForceNewScope = FALSE; ACPI_FUNCTION_NAME (LdCommonNamespaceEnd); /* We are only interested in opcodes that have an associated name */ if (!Op->Asl.Namepath) { return (AE_OK); } /* Get the type to determine if we should pop the scope */ if ((Op->Asl.ParseOpcode == PARSEOP_DEFAULT_ARG) && (Op->Asl.CompileFlags == OP_IS_RESOURCE_DESC)) { /* TBD: Merge into AcpiDsMapNamedOpcodeToDataType */ ObjectType = ACPI_TYPE_LOCAL_RESOURCE; } else { ObjectType = AslMapNamedOpcodeToDataType (Op->Asl.AmlOpcode); } /* Pop scope that was pushed for Resource Templates */ if (Op->Asl.ParseOpcode == PARSEOP_NAME) { if (Op->Asl.CompileFlags & OP_IS_RESOURCE_DESC) { ForceNewScope = TRUE; } } /* Pop the scope stack */ if (ForceNewScope || AcpiNsOpensScope (ObjectType)) { ACPI_DEBUG_PRINT ((ACPI_DB_DISPATCH, "(%s): Popping scope for Op [%s] %p\n", AcpiUtGetTypeName (ObjectType), Op->Asl.ParseOpName, Op)); (void) AcpiDsScopeStackPop (WalkState); } return (AE_OK); } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslmessages.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslmessages.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslmessages.c (revision 357966) @@ -1,548 +1,557 @@ /****************************************************************************** * * Module Name: aslmessages.c - Compiler error/warning message strings * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("aslmessages") /* * Strings for message reporting levels, must match error * type string tables in aslmessages.c */ const char *AslErrorLevel [ASL_NUM_REPORT_LEVELS] = { "Optimize", "Remark ", "Warning ", "Warning ", "Warning ", "Error " }; /* All lowercase versions for IDEs */ const char *AslErrorLevelIde [ASL_NUM_REPORT_LEVELS] = { "optimize", "remark ", "warning ", "warning ", "warning ", "error " }; /* * Actual message strings for each compiler message ID. There are currently * three distinct blocks of error messages (so that they can be expanded * individually): * Main ASL compiler * Data Table compiler * Preprocessor * * NOTE1: These tables must match the enum list of message IDs in the file * aslmessages.h exactly. * * NOTE2: With the introduction of the -vw option to disable specific messages, * new messages should only be added to the end of this list, so that values * for existing messages are not disturbed. */ /* ASL compiler */ const char *AslCompilerMsgs [] = { /* The zeroth message is reserved */ "", /* ASL_MSG_ALIGNMENT */ "Must be a multiple of alignment/granularity value", /* ASL_MSG_ALPHANUMERIC_STRING */ "String must be entirely alphanumeric", /* ASL_MSG_AML_NOT_IMPLEMENTED */ "Opcode is not implemented in compiler AML code generator", /* ASL_MSG_ARG_COUNT_HI */ "Too many arguments", /* ASL_MSG_ARG_COUNT_LO */ "Too few arguments", /* ASL_MSG_ARG_INIT */ "Method argument is not initialized", /* ASL_MSG_BACKWARDS_OFFSET */ "Invalid backwards offset", /* ASL_MSG_BUFFER_LENGTH */ "Effective AML buffer length is zero", /* ASL_MSG_CLOSE */ "Could not close file", /* ASL_MSG_COMPILER_INTERNAL */ "Internal compiler error", /* ASL_MSG_COMPILER_RESERVED */ "Use of compiler reserved name", /* ASL_MSG_CONNECTION_MISSING */ "A Connection operator is required for this field SpaceId", /* ASL_MSG_CONNECTION_INVALID */ "Invalid OpRegion SpaceId for use of Connection operator", /* ASL_MSG_CONSTANT_EVALUATION */ "Could not evaluate constant expression", /* ASL_MSG_CONSTANT_FOLDED */ "Constant expression evaluated and reduced", /* ASL_MSG_CORE_EXCEPTION */ "From ACPICA Subsystem", /* ASL_MSG_DEBUG_FILE_OPEN */ "Could not open debug file", /* ASL_MSG_DEBUG_FILENAME */ "Could not create debug filename", /* ASL_MSG_DEPENDENT_NESTING */ "Dependent function macros cannot be nested", /* ASL_MSG_DMA_CHANNEL */ "Invalid DMA channel (must be 0-7)", /* ASL_MSG_DMA_LIST */ "Too many DMA channels (8 max)", /* ASL_MSG_DUPLICATE_CASE */ "Case value already specified", /* ASL_MSG_DUPLICATE_ITEM */ "Duplicate value in list", /* ASL_MSG_EARLY_EOF */ "Premature end-of-file reached", /* ASL_MSG_ENCODING_LENGTH */ "Package length too long to encode", /* ASL_MSG_EX_INTERRUPT_LIST */ "Too many interrupts (255 max)", /* ASL_MSG_EX_INTERRUPT_LIST_MIN */ "Too few interrupts (1 minimum required)", /* ASL_MSG_EX_INTERRUPT_NUMBER */ "Invalid interrupt number (must be 32 bits)", /* ASL_MSG_FIELD_ACCESS_WIDTH */ "Access width is greater than region size", /* ASL_MSG_FIELD_UNIT_ACCESS_WIDTH */ "Access width of Field Unit extends beyond region limit", /* ASL_MSG_FIELD_UNIT_OFFSET */ "Field Unit extends beyond region limit", /* ASL_MSG_GPE_NAME_CONFLICT */ "Name conflicts with a previous GPE method", /* ASL_MSG_HID_LENGTH */ "_HID string must be exactly 7 or 8 characters", /* ASL_MSG_HID_PREFIX */ "_HID prefix must be all uppercase or decimal digits", /* ASL_MSG_HID_SUFFIX */ "_HID suffix must be all hex digits", /* ASL_MSG_INCLUDE_FILE_OPEN */ "Could not open include file", /* ASL_MSG_INPUT_FILE_OPEN */ "Could not open input file", /* ASL_MSG_INTEGER_LENGTH */ "Truncating 64-bit constant found in 32-bit table", /* ASL_MSG_INTEGER_OPTIMIZATION */ "Integer optimized to single-byte AML opcode", /* ASL_MSG_INTERRUPT_LIST */ "Too many interrupts (16 max)", /* ASL_MSG_INTERRUPT_NUMBER */ "Invalid interrupt number (must be 0-15)", /* ASL_MSG_INVALID_ACCESS_SIZE */ "Invalid AccessSize (Maximum is 4 - QWord access)", /* ASL_MSG_INVALID_ADDR_FLAGS */ "Invalid combination of Length and Min/Max fixed flags", /* ASL_MSG_INVALID_CONSTANT_OP */ "Invalid operator in constant expression (not type 3/4/5)", /* ASL_MSG_INVALID_EISAID */ "EISAID string must be of the form \"UUUXXXX\" (3 uppercase, 4 hex digits)", /* ASL_MSG_INVALID_ESCAPE */ "Invalid or unknown escape sequence", /* ASL_MSG_INVALID_GRAN_FIXED */ "Granularity must be zero for fixed Min/Max", /* ASL_MSG_INVALID_GRANULARITY */ "Granularity must be zero or a power of two minus one", /* ASL_MSG_INVALID_LENGTH */ "Length is larger than Min/Max window", /* ASL_MSG_INVALID_LENGTH_FIXED */ "Length is not equal to fixed Min/Max window", /* ASL_MSG_INVALID_MIN_MAX */ "Address Min is greater than Address Max", /* ASL_MSG_INVALID_OPERAND */ "Invalid operand", /* ASL_MSG_INVALID_PERFORMANCE */ "Invalid performance/robustness value", /* ASL_MSG_INVALID_PRIORITY */ "Invalid priority value", /* ASL_MSG_INVALID_STRING */ "Invalid Hex/Octal Escape - Non-ASCII or NULL", /* ASL_MSG_INVALID_TARGET */ "Target operand not allowed in constant expression", /* ASL_MSG_INVALID_TIME */ "Time parameter too long (255 max)", /* ASL_MSG_INVALID_TYPE */ "Invalid type", /* ASL_MSG_INVALID_UUID */ "UUID string must be of the form \"aabbccdd-eeff-gghh-iijj-kkllmmnnoopp\"", /* ASL_MSG_ISA_ADDRESS */ "Maximum 10-bit ISA address (0x3FF)", /* ASL_MSG_LEADING_ASTERISK */ "Invalid leading asterisk", /* ASL_MSG_LIST_LENGTH_LONG */ "Initializer list longer than declared package length", /* ASL_MSG_LIST_LENGTH_SHORT */ "Initializer list shorter than declared package length", /* ASL_MSG_LISTING_FILE_OPEN */ "Could not open listing file", /* ASL_MSG_LISTING_FILENAME */ "Could not create listing filename", /* ASL_MSG_LOCAL_INIT */ "Method local variable is not initialized", /* ASL_MSG_LOCAL_OUTSIDE_METHOD */ "Local or Arg used outside a control method", /* ASL_MSG_LONG_LINE */ "Splitting long input line", /* ASL_MSG_MEMORY_ALLOCATION */ "Memory allocation failure", /* ASL_MSG_MISSING_ENDDEPENDENT */ "Missing EndDependentFn() macro in dependent resource list", /* ASL_MSG_MISSING_STARTDEPENDENT */ "Missing StartDependentFn() macro in dependent resource list", /* ASL_MSG_MULTIPLE_DEFAULT */ "More than one Default statement within Switch construct", /* ASL_MSG_MULTIPLE_TYPES */ "Multiple types", /* ASL_MSG_NAME_EXISTS */ "Name already exists in scope", /* ASL_MSG_NAME_OPTIMIZATION */ "NamePath optimized", /* ASL_MSG_NAMED_OBJECT_IN_WHILE */ "Creating a named object in a While loop", /* ASL_MSG_NESTED_COMMENT */ "Nested comment found", /* ASL_MSG_NO_CASES */ "No Case statements under Switch", /* ASL_MSG_NO_REGION */ "_REG has no corresponding Operation Region", /* ASL_MSG_NO_RETVAL */ "Called method returns no value", /* ASL_MSG_NO_WHILE */ "No enclosing While statement", /* ASL_MSG_NON_ASCII */ "Invalid characters found in file", /* ASL_MSG_BUFFER_FIELD_LENGTH */ "Field length must be non-zero", /* ASL_MSG_NOT_EXIST */ "Object does not exist", /* ASL_MSG_NOT_FOUND */ "Object not found or not accessible from current scope", /* ASL_MSG_NOT_METHOD */ "Not a control method, cannot invoke", /* ASL_MSG_NOT_PARAMETER */ "Not a parameter, used as local only", /* ASL_MSG_NOT_REACHABLE */ "Object is not accessible from this scope", /* ASL_MSG_NOT_REFERENCED */ "Object is not referenced", /* ASL_MSG_NULL_DESCRIPTOR */ "Min/Max/Length/Gran are all zero, but no resource tag", /* ASL_MSG_NULL_STRING */ "Invalid zero-length (null) string", /* ASL_MSG_OPEN */ "Could not open file", /* ASL_MSG_OUTPUT_FILE_OPEN */ "Could not open output AML file", /* ASL_MSG_OUTPUT_FILENAME */ "Could not create output filename", /* ASL_MSG_PACKAGE_LENGTH */ "Effective AML package length is zero", /* ASL_MSG_PREPROCESSOR_FILENAME */ "Could not create preprocessor filename", /* ASL_MSG_READ */ "Could not read file", /* ASL_MSG_RECURSION */ "Recursive method call", /* ASL_MSG_REGION_BUFFER_ACCESS */ "Host Operation Region requires BufferAcc access", /* ASL_MSG_REGION_BYTE_ACCESS */ "Host Operation Region requires ByteAcc access", /* ASL_MSG_RESERVED_ARG_COUNT_HI */ "Reserved method has too many arguments", /* ASL_MSG_RESERVED_ARG_COUNT_LO */ "Reserved method has too few arguments", /* ASL_MSG_RESERVED_METHOD */ "Reserved name must be a control method", /* ASL_MSG_RESERVED_NO_RETURN_VAL */ "Reserved method should not return a value", /* ASL_MSG_RESERVED_OPERAND_TYPE */ "Invalid object type for reserved name", /* ASL_MSG_RESERVED_PACKAGE_LENGTH */ "Invalid package length for reserved name", /* ASL_MSG_RESERVED_RETURN_VALUE */ "Reserved method must return a value", /* ASL_MSG_RESERVED_USE */ "Invalid use of reserved name", /* ASL_MSG_RESERVED_WORD */ "Use of reserved name", /* ASL_MSG_RESOURCE_FIELD */ "Resource field name cannot be used as a target", /* ASL_MSG_RESOURCE_INDEX */ "Missing ResourceSourceIndex (required)", /* ASL_MSG_RESOURCE_LIST */ "Too many resource items (internal error)", /* ASL_MSG_RESOURCE_SOURCE */ "Missing ResourceSource string (required)", /* ASL_MSG_RESULT_NOT_USED */ "Result is not used, operator has no effect", /* ASL_MSG_RETURN_TYPES */ "Not all control paths return a value", /* ASL_MSG_SCOPE_FWD_REF */ "Forward references from Scope operator not allowed", /* ASL_MSG_SCOPE_TYPE */ "Existing object has invalid type for Scope operator", /* ASL_MSG_SEEK */ "Could not seek file", /* ASL_MSG_SERIALIZED */ "Control Method marked Serialized", -/* ASL_MSG_SERIALIZED_REQUIRED */ "Control Method should be made Serialized", +/* ASL_MSG_SERIALIZED_REQUIRED */ "Control Method should be made Serialized due to creation of named objects within", /* ASL_MSG_SINGLE_NAME_OPTIMIZATION */ "NamePath optimized to NameSeg (uses run-time search path)", /* ASL_MSG_SOME_NO_RETVAL */ "Called method may not always return a value", /* ASL_MSG_STRING_LENGTH */ "String literal too long", /* ASL_MSG_SWITCH_TYPE */ "Switch expression is not a static Integer/Buffer/String data type, defaulting to Integer", /* ASL_MSG_SYNC_LEVEL */ "SyncLevel must be in the range 0-15", /* ASL_MSG_SYNTAX */ "", /* ASL_MSG_TABLE_SIGNATURE */ "Invalid Table Signature", /* ASL_MSG_TAG_LARGER */ "ResourceTag larger than Field", /* ASL_MSG_TAG_SMALLER */ "ResourceTag smaller than Field", /* ASL_MSG_TIMEOUT */ "Result is not used, possible operator timeout will be missed", /* ASL_MSG_TOO_MANY_TEMPS */ "Method requires too many temporary variables (_T_x)", /* ASL_MSG_TRUNCATION */ "64-bit return value will be truncated to 32 bits (DSDT or SSDT version < 2)", /* ASL_MSG_UNKNOWN_RESERVED_NAME */ "Unknown reserved name", /* ASL_MSG_UNREACHABLE_CODE */ "Statement is unreachable", /* ASL_MSG_UNSUPPORTED */ "Unsupported feature", /* ASL_MSG_UPPER_CASE */ "Non-hex letters must be upper case", /* ASL_MSG_VENDOR_LIST */ "Too many vendor data bytes (7 max)", /* ASL_MSG_WRITE */ "Could not write file", /* ASL_MSG_RANGE */ "Constant out of range", /* ASL_MSG_BUFFER_ALLOCATION */ "Could not allocate line buffer", /* ASL_MSG_MISSING_DEPENDENCY */ "Missing dependency", /* ASL_MSG_ILLEGAL_FORWARD_REF */ "Illegal forward reference", /* ASL_MSG_ILLEGAL_METHOD_REF */ "Object is declared in a different method", /* ASL_MSG_LOCAL_NOT_USED */ "Method Local is set but never used", /* ASL_MSG_ARG_AS_LOCAL_NOT_USED */ "Method Argument (as a local) is set but never used", /* ASL_MSG_ARG_NOT_USED */ "Method Argument is never used", /* ASL_MSG_CONSTANT_REQUIRED */ "Non-reducible expression", /* ASL_MSG_CROSS_TABLE_SCOPE */ "Illegal open scope on external object from within DSDT", /* ASL_MSG_EXCEPTION_NOT_RECEIVED */ "Expected remark, warning, or error did not occur. Message ID:", /* ASL_MSG_NULL_RESOURCE_TEMPLATE */ "Empty Resource Template (END_TAG only)", /* ASL_MSG_FOUND_HERE */ "Original name creation/declaration below: ", /* ASL_MSG_ILLEGAL_RECURSION */ "Illegal recursive call to method that creates named objects", /* ASL_MSG_DUPLICATE_INPUT_FILE */ "Duplicate input files detected:", /* ASL_MSG_WARNING_AS_ERROR */ "Warnings detected during compilation", /* ASL_MSG_OEM_TABLE_ID */ "Invalid OEM Table ID", /* ASL_MSG_OEM_ID */ "Invalid OEM ID", /* ASL_MSG_UNLOAD */ "Unload is not supported by all operating systems", /* ASL_MSG_OFFSET */ "Unnecessary/redundant use of Offset operator", /* ASL_MSG_LONG_SLEEP */ "Very long Sleep, greater than 1 second", /* ASL_MSG_PREFIX_NOT_EXIST */ "One or more prefix Scopes do not exist", /* ASL_MSG_NAMEPATH_NOT_EXIST */ "One or more objects within the Pathname do not exist", /* ASL_MSG_REGION_LENGTH */ "Operation Region declared with zero length", /* ASL_MSG_TEMPORARY_OBJECT */ "Object is created temporarily in another method and cannot be accessed", /* ASL_MSG_UNDEFINED_EXTERNAL */ "Named object was declared external but the actual definition does not exist", /* ASL_MSG_BUFFER_FIELD_OVERFLOW */ "Buffer field extends beyond end of target buffer", /* ASL_MSG_INVALID_SPECIAL_NAME */ "declaration of this named object outside root scope is illegal", /* ASL_MSG_INVALID_PROCESSOR_UID */ "_UID inside processor declaration must be an integer", /* ASL_MSG_LEGACY_PROCESSOR_OP */ "Legacy Processor() keyword detected. Use Device() keyword instead.", /* ASL_MSG_NAMESTRING_LENGTH */ "NameString contains too many NameSegs (>255)", -/* ASL_MSG_CASE_FOUND_HERE */ "Original Case value below:" +/* ASL_MSG_CASE_FOUND_HERE */ "Original Case value below:", +/* ASL_MSG_EXTERN_INVALID_RET_TYPE */ "Return type is only allowed for Externals declared as MethodObj", +/* ASL_MSG_EXTERN_INVALID_PARAM_TYPE */ "Parameter type is only allowed for Externals declared as MethodObj", +/* ASL_MSG_NAMED_OBJECT_CREATION */ "Creation of named objects within a method is highly inefficient, use globals or method local variables instead", +/* ASL_MSG_ARG_COUNT_MISMATCH */ "Method NumArgs count does not match length of ParameterTypes list", +/* ASL_MSG_STATIC_OPREGION_IN_METHOD */ "Static OperationRegion should be declared outside control method", +/* ASL_MSG_DECLARATION_TYPE_MISMATCH */ "Type mismatch between external declaration and actual object declaration detected", +/* ASL_MSG_TYPE_MISMATCH_FOUND_HERE */ "Actual object declaration:", +/* ASL_MSG_DUPLICATE_EXTERN_MISMATCH */ "Type mismatch between multiple external declarations detected", +/* ASL_MSG_DUPLICATE_EXTERN_FOUND_HERE */"Duplicate external declaration:", }; /* Table compiler */ const char *AslTableCompilerMsgs [] = { /* ASL_MSG_BUFFER_ELEMENT */ "Invalid element in buffer initializer list", /* ASL_MSG_DIVIDE_BY_ZERO */ "Expression contains divide-by-zero", /* ASL_MSG_FLAG_VALUE */ "Flag value is too large", /* ASL_MSG_INTEGER_SIZE */ "Integer too large for target", /* ASL_MSG_INVALID_EXPRESSION */ "Invalid expression", /* ASL_MSG_INVALID_FIELD_NAME */ "Invalid Field Name", /* ASL_MSG_INVALID_HEX_INTEGER */ "Invalid hex integer constant", /* ASL_MSG_OEM_TABLE */ "OEM table - unknown contents", /* ASL_MSG_RESERVED_VALUE */ "Reserved field", /* ASL_MSG_UNKNOWN_LABEL */ "Label is undefined", /* ASL_MSG_UNKNOWN_SUBTABLE */ "Unknown subtable type", /* ASL_MSG_UNKNOWN_TABLE */ "Unknown ACPI table signature", /* ASL_MSG_ZERO_VALUE */ "Value must be non-zero", /* ASL_MSG_INVALID_LABEL */ "Invalid field label detected", /* ASL_MSG_BUFFER_LIST */ "Invalid buffer initializer list", /* ASL_MSG_ENTRY_LIST */ "Invalid entry initializer list" }; /* Preprocessor */ const char *AslPreprocessorMsgs [] = { /* ASL_MSG_DIRECTIVE_SYNTAX */ "Invalid directive syntax", /* ASL_MSG_ENDIF_MISMATCH */ "Mismatched #endif", /* ASL_MSG_ERROR_DIRECTIVE */ "#error", /* ASL_MSG_EXISTING_NAME */ "Name is already defined", /* ASL_MSG_INVALID_INVOCATION */ "Invalid macro invocation", /* ASL_MSG_MACRO_SYNTAX */ "Invalid macro syntax", /* ASL_MSG_TOO_MANY_ARGUMENTS */ "Too many macro arguments", /* ASL_MSG_UNKNOWN_DIRECTIVE */ "Unknown directive", /* ASL_MSG_UNKNOWN_PRAGMA */ "Unknown pragma", /* ASL_MSG_WARNING_DIRECTIVE */ "#warning", /* ASL_MSG_INCLUDE_FILE */ "Found a # preprocessor directive in ASL Include() file" }; /******************************************************************************* * * FUNCTION: AeDecodeMessageId * * PARAMETERS: MessageId - ASL message ID (exception code) to be * formatted. Possibly fully encoded. * * RETURN: A string containing the exception message text. * * DESCRIPTION: This function validates and translates an ASL message ID into * an ASCII string. * ******************************************************************************/ const char * AeDecodeMessageId ( UINT16 MessageId) { UINT32 Index; const char **MessageTable; /* Main ASL Compiler messages */ if (MessageId <= ASL_MSG_MAIN_COMPILER_END) { MessageTable = AslCompilerMsgs; Index = MessageId; if (Index >= ACPI_ARRAY_LENGTH (AslCompilerMsgs)) { return ("[Unknown ASL Compiler exception ID]"); } } /* Data Table Compiler messages */ else if (MessageId <= ASL_MSG_TABLE_COMPILER_END) { MessageTable = AslTableCompilerMsgs; Index = MessageId - ASL_MSG_TABLE_COMPILER; if (Index >= ACPI_ARRAY_LENGTH (AslTableCompilerMsgs)) { return ("[Unknown Table Compiler exception ID]"); } } /* Preprocessor messages */ else if (MessageId <= ASL_MSG_PREPROCESSOR_END) { MessageTable = AslPreprocessorMsgs; Index = MessageId - ASL_MSG_PREPROCESSOR; if (Index >= ACPI_ARRAY_LENGTH (AslPreprocessorMsgs)) { return ("[Unknown Preprocessor exception ID]"); } } /* Everything else is unknown */ else { return ("[Unknown exception/component ID]"); } return (MessageTable[Index]); } /******************************************************************************* * * FUNCTION: AeDecodeExceptionLevel * * PARAMETERS: Level - The ASL error level to be decoded * * RETURN: A string containing the error level text * * DESCRIPTION: This function validates and translates an ASL error level into * an ASCII string. * ******************************************************************************/ const char * AeDecodeExceptionLevel ( UINT8 Level) { /* Range check on Level */ if (Level >= ACPI_ARRAY_LENGTH (AslErrorLevel)) { return ("Unknown exception level"); } /* Differentiate the string type to be used (IDE is all lower case) */ if (AslGbl_VerboseErrors) { return (AslErrorLevel[Level]); } return (AslErrorLevelIde[Level]); } /******************************************************************************* * * FUNCTION: AeBuildFullExceptionCode * * PARAMETERS: Level - ASL error level * MessageId - ASL exception code to be formatted * * RETURN: Fully encoded exception code * * DESCRIPTION: Build the full exception code from the error level and the * actual message ID. * ******************************************************************************/ UINT16 AeBuildFullExceptionCode ( UINT8 Level, UINT16 MessageId) { /* * Error level is in the thousands slot (error/warning/remark, etc.) * Error codes are 0 - 999 */ return (((Level + 1) * 1000) + MessageId); } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslmessages.h =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslmessages.h (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslmessages.h (revision 357966) @@ -1,413 +1,423 @@ /****************************************************************************** * * Module Name: aslmessages.h - Compiler error/warning messages * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #ifndef __ASLMESSAGES_H #define __ASLMESSAGES_H /* These values must match error type string tables in aslmessages.c */ typedef enum { ASL_OPTIMIZATION = 0, ASL_REMARK, ASL_WARNING, ASL_WARNING2, ASL_WARNING3, ASL_ERROR, ASL_NUM_REPORT_LEVELS } ASL_MESSAGE_TYPES; #define ASL_ERROR_LEVEL_LENGTH 8 /* Length of strings for types above */ /* * Exception code blocks, 0 - 999 * Available for new exception blocks: 600 - 999 */ #define ASL_MSG_MAIN_COMPILER 0 /* 0 - 299 */ #define ASL_MSG_MAIN_COMPILER_END 299 #define ASL_MSG_TABLE_COMPILER 300 /* 300 - 499 */ #define ASL_MSG_TABLE_COMPILER_END 499 #define ASL_MSG_PREPROCESSOR 500 /* 500 - 599 */ #define ASL_MSG_PREPROCESSOR_END 599 /* * Values (message IDs) for all compiler messages. There are currently * three distinct blocks of error messages (so that they can be expanded * individually): * Main ASL compiler * Data Table compiler * Preprocessor * * NOTE1: This list must match the tables of message strings in the file * aslmessages.c exactly. * * NOTE2: With the introduction of the -vw option to disable specific * messages, new messages should only be added to the end of these * lists, so that values for existing messages are not disturbed. */ typedef enum { ASL_MSG_RESERVED = ASL_MSG_MAIN_COMPILER, ASL_MSG_ALIGNMENT, ASL_MSG_ALPHANUMERIC_STRING, ASL_MSG_AML_NOT_IMPLEMENTED, ASL_MSG_ARG_COUNT_HI, ASL_MSG_ARG_COUNT_LO, ASL_MSG_ARG_INIT, ASL_MSG_BACKWARDS_OFFSET, ASL_MSG_BUFFER_LENGTH, ASL_MSG_CLOSE, ASL_MSG_COMPILER_INTERNAL, ASL_MSG_COMPILER_RESERVED, ASL_MSG_CONNECTION_MISSING, ASL_MSG_CONNECTION_INVALID, ASL_MSG_CONSTANT_EVALUATION, ASL_MSG_CONSTANT_FOLDED, ASL_MSG_CORE_EXCEPTION, ASL_MSG_DEBUG_FILE_OPEN, ASL_MSG_DEBUG_FILENAME, ASL_MSG_DEPENDENT_NESTING, ASL_MSG_DMA_CHANNEL, ASL_MSG_DMA_LIST, ASL_MSG_DUPLICATE_CASE, ASL_MSG_DUPLICATE_ITEM, ASL_MSG_EARLY_EOF, ASL_MSG_ENCODING_LENGTH, ASL_MSG_EX_INTERRUPT_LIST, ASL_MSG_EX_INTERRUPT_LIST_MIN, ASL_MSG_EX_INTERRUPT_NUMBER, ASL_MSG_FIELD_ACCESS_WIDTH, ASL_MSG_FIELD_UNIT_ACCESS_WIDTH, ASL_MSG_FIELD_UNIT_OFFSET, ASL_MSG_GPE_NAME_CONFLICT, ASL_MSG_HID_LENGTH, ASL_MSG_HID_PREFIX, ASL_MSG_HID_SUFFIX, ASL_MSG_INCLUDE_FILE_OPEN, ASL_MSG_INPUT_FILE_OPEN, ASL_MSG_INTEGER_LENGTH, ASL_MSG_INTEGER_OPTIMIZATION, ASL_MSG_INTERRUPT_LIST, ASL_MSG_INTERRUPT_NUMBER, ASL_MSG_INVALID_ACCESS_SIZE, ASL_MSG_INVALID_ADDR_FLAGS, ASL_MSG_INVALID_CONSTANT_OP, ASL_MSG_INVALID_EISAID, ASL_MSG_INVALID_ESCAPE, ASL_MSG_INVALID_GRAN_FIXED, ASL_MSG_INVALID_GRANULARITY, ASL_MSG_INVALID_LENGTH, ASL_MSG_INVALID_LENGTH_FIXED, ASL_MSG_INVALID_MIN_MAX, ASL_MSG_INVALID_OPERAND, ASL_MSG_INVALID_PERFORMANCE, ASL_MSG_INVALID_PRIORITY, ASL_MSG_INVALID_STRING, ASL_MSG_INVALID_TARGET, ASL_MSG_INVALID_TIME, ASL_MSG_INVALID_TYPE, ASL_MSG_INVALID_UUID, ASL_MSG_ISA_ADDRESS, ASL_MSG_LEADING_ASTERISK, ASL_MSG_LIST_LENGTH_LONG, ASL_MSG_LIST_LENGTH_SHORT, ASL_MSG_LISTING_FILE_OPEN, ASL_MSG_LISTING_FILENAME, ASL_MSG_LOCAL_INIT, ASL_MSG_LOCAL_OUTSIDE_METHOD, ASL_MSG_LONG_LINE, ASL_MSG_MEMORY_ALLOCATION, ASL_MSG_MISSING_ENDDEPENDENT, ASL_MSG_MISSING_STARTDEPENDENT, ASL_MSG_MULTIPLE_DEFAULT, ASL_MSG_MULTIPLE_TYPES, ASL_MSG_NAME_EXISTS, ASL_MSG_NAME_OPTIMIZATION, ASL_MSG_NAMED_OBJECT_IN_WHILE, ASL_MSG_NESTED_COMMENT, ASL_MSG_NO_CASES, ASL_MSG_NO_REGION, ASL_MSG_NO_RETVAL, ASL_MSG_NO_WHILE, ASL_MSG_NON_ASCII, ASL_MSG_BUFFER_FIELD_LENGTH, ASL_MSG_NOT_EXIST, ASL_MSG_NOT_FOUND, ASL_MSG_NOT_METHOD, ASL_MSG_NOT_PARAMETER, ASL_MSG_NOT_REACHABLE, ASL_MSG_NOT_REFERENCED, ASL_MSG_NULL_DESCRIPTOR, ASL_MSG_NULL_STRING, ASL_MSG_OPEN, ASL_MSG_OUTPUT_FILE_OPEN, ASL_MSG_OUTPUT_FILENAME, ASL_MSG_PACKAGE_LENGTH, ASL_MSG_PREPROCESSOR_FILENAME, ASL_MSG_READ, ASL_MSG_RECURSION, ASL_MSG_REGION_BUFFER_ACCESS, ASL_MSG_REGION_BYTE_ACCESS, ASL_MSG_RESERVED_ARG_COUNT_HI, ASL_MSG_RESERVED_ARG_COUNT_LO, ASL_MSG_RESERVED_METHOD, ASL_MSG_RESERVED_NO_RETURN_VAL, ASL_MSG_RESERVED_OPERAND_TYPE, ASL_MSG_RESERVED_PACKAGE_LENGTH, ASL_MSG_RESERVED_RETURN_VALUE, ASL_MSG_RESERVED_USE, ASL_MSG_RESERVED_WORD, ASL_MSG_RESOURCE_FIELD, ASL_MSG_RESOURCE_INDEX, ASL_MSG_RESOURCE_LIST, ASL_MSG_RESOURCE_SOURCE, ASL_MSG_RESULT_NOT_USED, ASL_MSG_RETURN_TYPES, ASL_MSG_SCOPE_FWD_REF, ASL_MSG_SCOPE_TYPE, ASL_MSG_SEEK, ASL_MSG_SERIALIZED, ASL_MSG_SERIALIZED_REQUIRED, ASL_MSG_SINGLE_NAME_OPTIMIZATION, ASL_MSG_SOME_NO_RETVAL, ASL_MSG_STRING_LENGTH, ASL_MSG_SWITCH_TYPE, ASL_MSG_SYNC_LEVEL, ASL_MSG_SYNTAX, ASL_MSG_TABLE_SIGNATURE, ASL_MSG_TAG_LARGER, ASL_MSG_TAG_SMALLER, ASL_MSG_TIMEOUT, ASL_MSG_TOO_MANY_TEMPS, ASL_MSG_TRUNCATION, ASL_MSG_UNKNOWN_RESERVED_NAME, ASL_MSG_UNREACHABLE_CODE, ASL_MSG_UNSUPPORTED, ASL_MSG_UPPER_CASE, ASL_MSG_VENDOR_LIST, ASL_MSG_WRITE, ASL_MSG_RANGE, ASL_MSG_BUFFER_ALLOCATION, ASL_MSG_MISSING_DEPENDENCY, ASL_MSG_ILLEGAL_FORWARD_REF, ASL_MSG_ILLEGAL_METHOD_REF, ASL_MSG_LOCAL_NOT_USED, ASL_MSG_ARG_AS_LOCAL_NOT_USED, ASL_MSG_ARG_NOT_USED, ASL_MSG_CONSTANT_REQUIRED, ASL_MSG_CROSS_TABLE_SCOPE, ASL_MSG_EXCEPTION_NOT_RECEIVED, ASL_MSG_NULL_RESOURCE_TEMPLATE, ASL_MSG_FOUND_HERE, ASL_MSG_ILLEGAL_RECURSION, ASL_MSG_DUPLICATE_INPUT_FILE, ASL_MSG_WARNING_AS_ERROR, ASL_MSG_OEM_TABLE_ID, ASL_MSG_OEM_ID, ASL_MSG_UNLOAD, ASL_MSG_OFFSET, ASL_MSG_LONG_SLEEP, ASL_MSG_PREFIX_NOT_EXIST, ASL_MSG_NAMEPATH_NOT_EXIST, ASL_MSG_REGION_LENGTH, ASL_MSG_TEMPORARY_OBJECT, ASL_MSG_UNDEFINED_EXTERNAL, ASL_MSG_BUFFER_FIELD_OVERFLOW, ASL_MSG_INVALID_SPECIAL_NAME, ASL_MSG_INVALID_PROCESSOR_UID, ASL_MSG_LEGACY_PROCESSOR_OP, ASL_MSG_NAMESTRING_LENGTH, ASL_MSG_CASE_FOUND_HERE, + ASL_MSG_EXTERN_INVALID_RET_TYPE, + ASL_MSG_EXTERN_INVALID_PARAM_TYPE, + ASL_MSG_NAMED_OBJECT_CREATION, + ASL_MSG_ARG_COUNT_MISMATCH, + ASL_MSG_STATIC_OPREGION_IN_METHOD, + ASL_MSG_DECLARATION_TYPE_MISMATCH, + ASL_MSG_TYPE_MISMATCH_FOUND_HERE, + ASL_MSG_DUPLICATE_EXTERN_MISMATCH, + ASL_MSG_DUPLICATE_EXTERN_FOUND_HERE, + /* These messages are used by the Data Table compiler only */ ASL_MSG_BUFFER_ELEMENT = ASL_MSG_TABLE_COMPILER, ASL_MSG_DIVIDE_BY_ZERO, ASL_MSG_FLAG_VALUE, ASL_MSG_INTEGER_SIZE, ASL_MSG_INVALID_EXPRESSION, ASL_MSG_INVALID_FIELD_NAME, ASL_MSG_INVALID_HEX_INTEGER, ASL_MSG_OEM_TABLE, ASL_MSG_RESERVED_VALUE, ASL_MSG_UNKNOWN_LABEL, ASL_MSG_UNKNOWN_SUBTABLE, ASL_MSG_UNKNOWN_TABLE, ASL_MSG_ZERO_VALUE, ASL_MSG_INVALID_LABEL, ASL_MSG_BUFFER_LIST, ASL_MSG_ENTRY_LIST, /* These messages are used by the Preprocessor only */ ASL_MSG_DIRECTIVE_SYNTAX = ASL_MSG_PREPROCESSOR, ASL_MSG_ENDIF_MISMATCH, ASL_MSG_ERROR_DIRECTIVE, ASL_MSG_EXISTING_NAME, ASL_MSG_INVALID_INVOCATION, ASL_MSG_MACRO_SYNTAX, ASL_MSG_TOO_MANY_ARGUMENTS, ASL_MSG_UNKNOWN_DIRECTIVE, ASL_MSG_UNKNOWN_PRAGMA, ASL_MSG_WARNING_DIRECTIVE, ASL_MSG_INCLUDE_FILE } ASL_MESSAGE_IDS; #endif /* __ASLMESSAGES_H */ Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslmethod.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslmethod.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslmethod.c (revision 357966) @@ -1,892 +1,1027 @@ /****************************************************************************** * * Module Name: aslmethod.c - Control method analysis walk * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include "aslcompiler.y.h" +#include #include #include #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("aslmethod") /* Local prototypes */ static void MtCheckNamedObjectInMethod ( ACPI_PARSE_OBJECT *Op, ASL_METHOD_INFO *MethodInfo); +static void +MtCheckStaticOperationRegionInMethod ( + ACPI_PARSE_OBJECT *Op); + /******************************************************************************* * * FUNCTION: MtMethodAnalysisWalkBegin * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: Status * * DESCRIPTION: Descending callback for the analysis walk. Check methods for: * 1) Initialized local variables * 2) Valid arguments * 3) Return types * ******************************************************************************/ ACPI_STATUS MtMethodAnalysisWalkBegin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { ASL_ANALYSIS_WALK_INFO *WalkInfo = (ASL_ANALYSIS_WALK_INFO *) Context; ASL_METHOD_INFO *MethodInfo = WalkInfo->MethodStack; ACPI_PARSE_OBJECT *Next; UINT32 RegisterNumber; UINT32 i; char LocalName[] = "Local0"; char ArgName[] = "Arg0"; ACPI_PARSE_OBJECT *ArgNode; ACPI_PARSE_OBJECT *NextType; - ACPI_PARSE_OBJECT *NextParamType; UINT8 ActualArgs = 0; BOOLEAN HidExists; BOOLEAN AdrExists; /* Build cross-reference output file if requested */ if (AslGbl_CrossReferenceOutput) { OtXrefWalkPart1 (Op, Level, MethodInfo); } switch (Op->Asl.ParseOpcode) { case PARSEOP_METHOD: AslGbl_TotalMethods++; /* Create and init method info */ MethodInfo = UtLocalCalloc (sizeof (ASL_METHOD_INFO)); MethodInfo->Next = WalkInfo->MethodStack; MethodInfo->Op = Op; WalkInfo->MethodStack = MethodInfo; /* * Special handling for _PSx methods. Dependency rules (same scope): * * 1) _PS0 - One of these must exist: _PS1, _PS2, _PS3 * 2) _PS1/_PS2/_PS3: A _PS0 must exist */ if (ACPI_COMPARE_NAMESEG (METHOD_NAME__PS0, Op->Asl.NameSeg)) { /* For _PS0, one of _PS1/_PS2/_PS3 must exist */ if ((!ApFindNameInScope (METHOD_NAME__PS1, Op)) && (!ApFindNameInScope (METHOD_NAME__PS2, Op)) && (!ApFindNameInScope (METHOD_NAME__PS3, Op))) { AslError (ASL_WARNING, ASL_MSG_MISSING_DEPENDENCY, Op, "_PS0 requires one of _PS1/_PS2/_PS3 in same scope"); } } else if ( ACPI_COMPARE_NAMESEG (METHOD_NAME__PS1, Op->Asl.NameSeg) || ACPI_COMPARE_NAMESEG (METHOD_NAME__PS2, Op->Asl.NameSeg) || ACPI_COMPARE_NAMESEG (METHOD_NAME__PS3, Op->Asl.NameSeg)) { /* For _PS1/_PS2/_PS3, a _PS0 must exist */ if (!ApFindNameInScope (METHOD_NAME__PS0, Op)) { sprintf (AslGbl_MsgBuffer, "%4.4s requires _PS0 in same scope", Op->Asl.NameSeg); AslError (ASL_WARNING, ASL_MSG_MISSING_DEPENDENCY, Op, AslGbl_MsgBuffer); } } /* Get the name node */ Next = Op->Asl.Child; /* Get the NumArguments node */ Next = Next->Asl.Next; MethodInfo->NumArguments = (UINT8) (((UINT8) Next->Asl.Value.Integer) & 0x07); /* Get the SerializeRule and SyncLevel nodes, ignored here */ Next = Next->Asl.Next; MethodInfo->ShouldBeSerialized = (UINT8) Next->Asl.Value.Integer; Next = Next->Asl.Next; ArgNode = Next; /* Get the ReturnType node */ Next = Next->Asl.Next; NextType = Next->Asl.Child; - while (NextType) - { - /* Get and map each of the ReturnTypes */ - MethodInfo->ValidReturnTypes |= AnMapObjTypeToBtype (NextType); - NextType->Asl.ParseOpcode = PARSEOP_DEFAULT_ARG; - NextType = NextType->Asl.Next; - } + MethodInfo->ValidReturnTypes = MtProcessTypeOp (NextType); /* Get the ParameterType node */ Next = Next->Asl.Next; NextType = Next->Asl.Child; - while (NextType) + if (!NextType) { - if (NextType->Asl.ParseOpcode == PARSEOP_DEFAULT_ARG) - { - NextParamType = NextType->Asl.Child; - while (NextParamType) - { - MethodInfo->ValidArgTypes[ActualArgs] |= - AnMapObjTypeToBtype (NextParamType); - - NextParamType->Asl.ParseOpcode = PARSEOP_DEFAULT_ARG; - NextParamType = NextParamType->Asl.Next; - } - } - else - { - MethodInfo->ValidArgTypes[ActualArgs] = - AnMapObjTypeToBtype (NextType); - - NextType->Asl.ParseOpcode = PARSEOP_DEFAULT_ARG; - ActualArgs++; - } - - NextType = NextType->Asl.Next; + /* + * The optional parameter types list was omitted at the source + * level. Use the Argument count parameter instead. + */ + ActualArgs = MethodInfo->NumArguments; } + else + { + ActualArgs = MtProcessParameterTypeList (NextType, + MethodInfo->ValidArgTypes); + } if ((MethodInfo->NumArguments) && (MethodInfo->NumArguments != ActualArgs)) { - /* error: Param list did not match number of args */ + sprintf (AslGbl_MsgBuffer, + "Length = %u", ActualArgs); + AslError (ASL_ERROR, ASL_MSG_ARG_COUNT_MISMATCH, + Op->Asl.Child->Asl.Next, AslGbl_MsgBuffer); } /* Allow numarguments == 0 for Function() */ if ((!MethodInfo->NumArguments) && (ActualArgs)) { MethodInfo->NumArguments = ActualArgs; ArgNode->Asl.Value.Integer |= ActualArgs; } /* * Actual arguments are initialized at method entry. * All other ArgX "registers" can be used as locals, so we * track their initialization. */ for (i = 0; i < MethodInfo->NumArguments; i++) { MethodInfo->ArgInitialized[i] = TRUE; } break; case PARSEOP_METHODCALL: /* Check for a recursive method call */ if (MethodInfo && (Op->Asl.Node == MethodInfo->Op->Asl.Node)) { if (MethodInfo->CreatesNamedObjects) { /* * This is an error, as it will fail at runtime on all ACPI * implementations. Any named object declarations will be * executed twice, causing failure the second time. Note, * this is independent of whether the method is declared * Serialized, because the same thread is attempting to * reenter the method, and this will always succeed. */ AslDualParseOpError (ASL_ERROR, ASL_MSG_ILLEGAL_RECURSION, Op, Op->Asl.Value.String, ASL_MSG_FOUND_HERE, MethodInfo->Op, MethodInfo->Op->Asl.ExternalName); } else { /* Method does not create objects, issue a remark */ AslError (ASL_REMARK, ASL_MSG_RECURSION, Op, Op->Asl.ExternalName); } } break; case PARSEOP_LOCAL0: case PARSEOP_LOCAL1: case PARSEOP_LOCAL2: case PARSEOP_LOCAL3: case PARSEOP_LOCAL4: case PARSEOP_LOCAL5: case PARSEOP_LOCAL6: case PARSEOP_LOCAL7: if (!MethodInfo) { /* * Local was used outside a control method, or there was an error * in the method declaration. */ AslError (ASL_REMARK, ASL_MSG_LOCAL_OUTSIDE_METHOD, Op, Op->Asl.ExternalName); return (AE_ERROR); } RegisterNumber = (Op->Asl.AmlOpcode & 0x0007); /* * If the local is being used as a target, mark the local * initialized */ if (Op->Asl.CompileFlags & OP_IS_TARGET) { MethodInfo->LocalInitialized[RegisterNumber] = TRUE; } /* * Otherwise, this is a reference, check if the local * has been previously initialized. * * The only operator that accepts an uninitialized value is ObjectType() */ else if ((!MethodInfo->LocalInitialized[RegisterNumber]) && (Op->Asl.Parent->Asl.ParseOpcode != PARSEOP_OBJECTTYPE)) { LocalName[strlen (LocalName) -1] = (char) (RegisterNumber + 0x30); AslError (ASL_ERROR, ASL_MSG_LOCAL_INIT, Op, LocalName); } break; case PARSEOP_ARG0: case PARSEOP_ARG1: case PARSEOP_ARG2: case PARSEOP_ARG3: case PARSEOP_ARG4: case PARSEOP_ARG5: case PARSEOP_ARG6: if (!MethodInfo) { /* * Arg was used outside a control method, or there was an error * in the method declaration. */ AslError (ASL_REMARK, ASL_MSG_LOCAL_OUTSIDE_METHOD, Op, Op->Asl.ExternalName); return (AE_ERROR); } RegisterNumber = (Op->Asl.AmlOpcode & 0x000F) - 8; ArgName[strlen (ArgName) -1] = (char) (RegisterNumber + 0x30); /* * If the Arg is being used as a target, mark the local * initialized */ if (Op->Asl.CompileFlags & OP_IS_TARGET) { MethodInfo->ArgInitialized[RegisterNumber] = TRUE; } /* * Otherwise, this is a reference, check if the Arg * has been previously initialized. * * The only operator that accepts an uninitialized value is ObjectType() */ else if ((!MethodInfo->ArgInitialized[RegisterNumber]) && (Op->Asl.Parent->Asl.ParseOpcode != PARSEOP_OBJECTTYPE)) { AslError (ASL_ERROR, ASL_MSG_ARG_INIT, Op, ArgName); } /* Flag this arg if it is not a "real" argument to the method */ if (RegisterNumber >= MethodInfo->NumArguments) { AslError (ASL_REMARK, ASL_MSG_NOT_PARAMETER, Op, ArgName); } break; case PARSEOP_RETURN: if (!MethodInfo) { /* * Probably was an error in the method declaration, * no additional error here */ ACPI_WARNING ((AE_INFO, "%p, No parent method", Op)); return (AE_ERROR); } /* * A child indicates a possible return value. A simple Return or * Return() is marked with OP_IS_NULL_RETURN by the parser so * that it is not counted as a "real" return-with-value, although * the AML code that is actually emitted is Return(0). The AML * definition of Return has a required parameter, so we are * forced to convert a null return to Return(0). */ if ((Op->Asl.Child) && (Op->Asl.Child->Asl.ParseOpcode != PARSEOP_DEFAULT_ARG) && (!(Op->Asl.Child->Asl.CompileFlags & OP_IS_NULL_RETURN))) { MethodInfo->NumReturnWithValue++; } else { MethodInfo->NumReturnNoValue++; } break; case PARSEOP_BREAK: case PARSEOP_CONTINUE: Next = Op->Asl.Parent; while (Next) { if (Next->Asl.ParseOpcode == PARSEOP_WHILE) { break; } Next = Next->Asl.Parent; } if (!Next) { AslError (ASL_ERROR, ASL_MSG_NO_WHILE, Op, NULL); } break; case PARSEOP_STALL: /* We can range check if the argument is an integer */ if ((Op->Asl.Child->Asl.ParseOpcode == PARSEOP_INTEGER) && (Op->Asl.Child->Asl.Value.Integer > ACPI_UINT8_MAX)) { AslError (ASL_ERROR, ASL_MSG_INVALID_TIME, Op, NULL); } break; case PARSEOP_DEVICE: /* Check usage of _HID and _ADR objects */ HidExists = ApFindNameInDeviceTree (METHOD_NAME__HID, Op); AdrExists = ApFindNameInDeviceTree (METHOD_NAME__ADR, Op); if (!HidExists && !AdrExists) { AslError (ASL_WARNING, ASL_MSG_MISSING_DEPENDENCY, Op, "Device object requires a _HID or _ADR in same scope"); } else if (HidExists && AdrExists) { /* * According to the ACPI spec, "A device object must contain * either an _HID object or an _ADR object, but should not contain * both". */ AslError (ASL_WARNING, ASL_MSG_MULTIPLE_TYPES, Op, "Device object requires either a _HID or _ADR, but not both"); } break; case PARSEOP_EVENT: case PARSEOP_MUTEX: case PARSEOP_OPERATIONREGION: case PARSEOP_POWERRESOURCE: case PARSEOP_PROCESSOR: case PARSEOP_THERMALZONE: /* * The first operand is a name to be created in the namespace. * Check against the reserved list. */ i = ApCheckForPredefinedName (Op, Op->Asl.NameSeg); if (i < ACPI_VALID_RESERVED_NAME_MAX) { AslError (ASL_ERROR, ASL_MSG_RESERVED_USE, Op, Op->Asl.ExternalName); } + + MtCheckStaticOperationRegionInMethod (Op); break; case PARSEOP_NAME: /* Typecheck any predefined names statically defined with Name() */ ApCheckForPredefinedObject (Op, Op->Asl.NameSeg); /* Special typechecking for _HID */ if (ACPI_COMPARE_NAMESEG (METHOD_NAME__HID, Op->Asl.NameSeg)) { Next = Op->Asl.Child->Asl.Next; AnCheckId (Next, ASL_TYPE_HID); } /* Special typechecking for _CID */ else if (ACPI_COMPARE_NAMESEG (METHOD_NAME__CID, Op->Asl.NameSeg)) { Next = Op->Asl.Child->Asl.Next; if ((Next->Asl.ParseOpcode == PARSEOP_PACKAGE) || (Next->Asl.ParseOpcode == PARSEOP_VAR_PACKAGE)) { Next = Next->Asl.Child; while (Next) { AnCheckId (Next, ASL_TYPE_CID); Next = Next->Asl.Next; } } else { AnCheckId (Next, ASL_TYPE_CID); } } break; default: break; } /* Check for named object creation within a non-serialized method */ MtCheckNamedObjectInMethod (Op, MethodInfo); return (AE_OK); } /******************************************************************************* * + * FUNCTION: MtProcessTypeOp + * + * PARAMETERS: Op - Op representing a btype + * + * RETURN: Btype represented by Op + * + * DESCRIPTION: Process a parse object that represents single parameter type or + * a return type in method, function, and external declarations. + * + ******************************************************************************/ + +UINT32 +MtProcessTypeOp ( + ACPI_PARSE_OBJECT *TypeOp) +{ + UINT32 Btype = ACPI_BTYPE_ANY; + + + while (TypeOp) + { + Btype |= AnMapObjTypeToBtype (TypeOp); + TypeOp->Asl.ParseOpcode = PARSEOP_DEFAULT_ARG; + TypeOp = TypeOp->Asl.Next; + } + + return (Btype); +} + + +/******************************************************************************* + * + * FUNCTION: MtProcessParameterTypeList + * + * PARAMETERS: Op - Op representing a btype + * + * RETURN: Btype represented by Op + * + * DESCRIPTION: Process a parse object that represents a parameter type list in + * method, function, and external declarations. + * + ******************************************************************************/ + +UINT8 +MtProcessParameterTypeList ( + ACPI_PARSE_OBJECT *ParamTypeOp, + UINT32 *TypeList) +{ + UINT8 ParameterCount = 0; + + + while (ParamTypeOp) + { + TypeList[ParameterCount] = + MtProcessTypeOp (ParamTypeOp->Asl.Child); + + ParameterCount++; + ParamTypeOp = ParamTypeOp->Asl.Next; + } + + return (ParameterCount); +} + + +/******************************************************************************* + * * FUNCTION: MtCheckNamedObjectInMethod * * PARAMETERS: Op - Current parser op * MethodInfo - Info for method being parsed * * RETURN: None * * DESCRIPTION: Detect if a non-serialized method is creating a named object, * which could possibly cause problems if two threads execute * the method concurrently. Emit a remark in this case. * ******************************************************************************/ static void MtCheckNamedObjectInMethod ( ACPI_PARSE_OBJECT *Op, ASL_METHOD_INFO *MethodInfo) { const ACPI_OPCODE_INFO *OpInfo; + char *ExternalPath; /* We don't care about actual method declarations or scopes */ if ((Op->Asl.AmlOpcode == AML_METHOD_OP) || (Op->Asl.AmlOpcode == AML_SCOPE_OP)) { return; } /* Determine if we are creating a named object within a method */ if (!MethodInfo) { return; } OpInfo = AcpiPsGetOpcodeInfo (Op->Asl.AmlOpcode); if (OpInfo->Class == AML_CLASS_NAMED_OBJECT) { /* * 1) Mark the method as a method that creates named objects. * - * 2) If the method is non-serialized, emit a remark that the method + * 2) Issue a remark indicating the inefficiency of creating named + * objects within a method (Except for compiler-emitted temporary + * variables). + * + * 3) If the method is non-serialized, emit a remark that the method * should be serialized. * * Reason: If a thread blocks within the method for any reason, and * another thread enters the method, the method will fail because * an attempt will be made to create the same object twice. */ + ExternalPath = AcpiNsGetNormalizedPathname (MethodInfo->Op->Asl.Node, TRUE); + + /* No error for compiler temp variables (name starts with "_T_") */ + + if ((Op->Asl.NameSeg[0] != '_') && + (Op->Asl.NameSeg[1] != 'T') && + (Op->Asl.NameSeg[2] != '_')) + { + AslError (ASL_REMARK, ASL_MSG_NAMED_OBJECT_CREATION, Op, + ExternalPath); + } + MethodInfo->CreatesNamedObjects = TRUE; if (!MethodInfo->ShouldBeSerialized) { AslError (ASL_REMARK, ASL_MSG_SERIALIZED_REQUIRED, MethodInfo->Op, - "due to creation of named objects within"); + ExternalPath); /* Emit message only ONCE per method */ MethodInfo->ShouldBeSerialized = TRUE; } + + if (ExternalPath) + { + ACPI_FREE (ExternalPath); + } } } /******************************************************************************* * + * FUNCTION: MtCheckStaticOperationRegionInMethod + * + * PARAMETERS: Op - Current parser op + * + * RETURN: None + * + * DESCRIPTION: Warns if an Operation Region with static address or length + * is declared inside a control method + * + ******************************************************************************/ + +static void +MtCheckStaticOperationRegionInMethod( + ACPI_PARSE_OBJECT* Op) +{ + ACPI_PARSE_OBJECT* AddressOp; + ACPI_PARSE_OBJECT* LengthOp; + + + if (Op->Asl.ParseOpcode != PARSEOP_OPERATIONREGION) + { + return; + } + + /* + * OperationRegion should have 4 arguments defined. At this point, we + * assume that the parse tree is well-formed. + */ + AddressOp = Op->Asl.Child->Asl.Next->Asl.Next; + LengthOp = Op->Asl.Child->Asl.Next->Asl.Next->Asl.Next; + + if (UtGetParentMethodOp (Op) && + AddressOp->Asl.ParseOpcode == PARSEOP_INTEGER && + LengthOp->Asl.ParseOpcode == PARSEOP_INTEGER) + { + /* + * At this point, a static operation region declared inside of a + * control method has been found. Throw a warning because this is + * highly inefficient. + */ + AslError(ASL_WARNING, ASL_MSG_STATIC_OPREGION_IN_METHOD, Op, NULL); + } + + return; +} + + +/******************************************************************************* + * * FUNCTION: MtMethodAnalysisWalkEnd * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: Status * * DESCRIPTION: Ascending callback for analysis walk. Complete method * return analysis. * ******************************************************************************/ ACPI_STATUS MtMethodAnalysisWalkEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { ASL_ANALYSIS_WALK_INFO *WalkInfo = (ASL_ANALYSIS_WALK_INFO *) Context; ASL_METHOD_INFO *MethodInfo = WalkInfo->MethodStack; + char *ExternalPath; switch (Op->Asl.ParseOpcode) { case PARSEOP_METHOD: case PARSEOP_RETURN: if (!MethodInfo) { printf ("No method info for method! [%s]\n", Op->Asl.Namepath); AslError (ASL_ERROR, ASL_MSG_COMPILER_INTERNAL, Op, "No method info for this method"); CmCleanupAndExit (); return (AE_AML_INTERNAL); } break; default: break; } switch (Op->Asl.ParseOpcode) { case PARSEOP_METHOD: WalkInfo->MethodStack = MethodInfo->Next; /* * Check if there is no return statement at the end of the * method AND we can actually get there -- i.e., the execution * of the method can possibly terminate without a return statement. */ if ((!AnLastStatementIsReturn (Op)) && (!(Op->Asl.CompileFlags & OP_HAS_NO_EXIT))) { /* * No return statement, and execution can possibly exit * via this path. This is equivalent to Return () */ MethodInfo->NumReturnNoValue++; } /* * Check for case where some return statements have a return value * and some do not. Exit without a return statement is a return with * no value */ if (MethodInfo->NumReturnNoValue && MethodInfo->NumReturnWithValue) { + ExternalPath = AcpiNsGetNormalizedPathname (Op->Asl.Node, TRUE); + AslError (ASL_WARNING, ASL_MSG_RETURN_TYPES, Op, - Op->Asl.ExternalName); + ExternalPath); + + if (ExternalPath) + { + ACPI_FREE (ExternalPath); + } } /* * If there are any RETURN() statements with no value, or there is a * control path that allows the method to exit without a return value, * we mark the method as a method that does not return a value. This * knowledge can be used to check method invocations that expect a * returned value. */ if (MethodInfo->NumReturnNoValue) { if (MethodInfo->NumReturnWithValue) { Op->Asl.CompileFlags |= OP_METHOD_SOME_NO_RETVAL; } else { Op->Asl.CompileFlags |= OP_METHOD_NO_RETVAL; } } /* * Check predefined method names for correct return behavior * and correct number of arguments. Also, some special checks * For GPE and _REG methods. */ if (ApCheckForPredefinedMethod (Op, MethodInfo)) { /* Special check for two names like _L01 and _E01 in same scope */ ApCheckForGpeNameConflict (Op); /* * Special check for _REG: Must have an operation region definition * within the same scope! */ ApCheckRegMethod (Op); } ACPI_FREE (MethodInfo); break; case PARSEOP_NAME: /* Special check for two names like _L01 and _E01 in same scope */ ApCheckForGpeNameConflict (Op); break; case PARSEOP_RETURN: /* * If the parent is a predefined method name, attempt to typecheck * the return value. Only static types can be validated. */ ApCheckPredefinedReturnValue (Op, MethodInfo); /* * The parent block does not "exit" and continue execution -- the * method is terminated here with the Return() statement. */ Op->Asl.Parent->Asl.CompileFlags |= OP_HAS_NO_EXIT; /* Used in the "typing" pass later */ Op->Asl.ParentMethod = MethodInfo->Op; /* * If there is a peer node after the return statement, then this * node is unreachable code -- i.e., it won't be executed because of * the preceding Return() statement. */ if (Op->Asl.Next) { AslError (ASL_WARNING, ASL_MSG_UNREACHABLE_CODE, Op->Asl.Next, NULL); } break; case PARSEOP_IF: if ((Op->Asl.CompileFlags & OP_HAS_NO_EXIT) && (Op->Asl.Next) && (Op->Asl.Next->Asl.ParseOpcode == PARSEOP_ELSE)) { /* * This IF has a corresponding ELSE. The IF block has no exit, * (it contains an unconditional Return) * mark the ELSE block to remember this fact. */ Op->Asl.Next->Asl.CompileFlags |= OP_IF_HAS_NO_EXIT; } break; case PARSEOP_ELSE: if ((Op->Asl.CompileFlags & OP_HAS_NO_EXIT) && (Op->Asl.CompileFlags & OP_IF_HAS_NO_EXIT)) { /* * This ELSE block has no exit and the corresponding IF block * has no exit either. Therefore, the parent node has no exit. */ Op->Asl.Parent->Asl.CompileFlags |= OP_HAS_NO_EXIT; } break; default: if ((Op->Asl.CompileFlags & OP_HAS_NO_EXIT) && (Op->Asl.Parent)) { /* If this node has no exit, then the parent has no exit either */ Op->Asl.Parent->Asl.CompileFlags |= OP_HAS_NO_EXIT; } break; } return (AE_OK); } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslparseop.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslparseop.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslparseop.c (revision 357966) @@ -1,908 +1,913 @@ /****************************************************************************** * * Module Name: aslparseop - Parse op create/allocate/cache interfaces * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include "aslcompiler.y.h" #include #include #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("aslparseop") /******************************************************************************* * * FUNCTION: TrCreateOp * * PARAMETERS: ParseOpcode - Opcode to be assigned to the op * NumChildren - Number of children to follow * ... - A list of child ops to link to the new * op. NumChildren long. * * RETURN: Pointer to the new op. Aborts on allocation failure * * DESCRIPTION: Create a new parse op and link together a list of child * ops underneath the new op. * ******************************************************************************/ ACPI_PARSE_OBJECT * TrCreateOp ( UINT32 ParseOpcode, UINT32 NumChildren, ...) { ACPI_PARSE_OBJECT *Op; ACPI_PARSE_OBJECT *Child; ACPI_PARSE_OBJECT *PrevChild; va_list ap; UINT32 i; BOOLEAN FirstChild; va_start (ap, NumChildren); /* Allocate one new op */ Op = TrAllocateOp (ParseOpcode); DbgPrint (ASL_PARSE_OUTPUT, "\nCreateOp Ln/Col %u/%u NewParent %p Child %u Op %s ", Op->Asl.LineNumber, Op->Asl.Column, Op, NumChildren, UtGetOpName(ParseOpcode)); /* Some extra debug output based on the parse opcode */ switch (ParseOpcode) { case PARSEOP_ASL_CODE: AslGbl_ParseTreeRoot = Op; Op->Asl.ParseOpcode = PARSEOP_DEFAULT_ARG; DbgPrint (ASL_PARSE_OUTPUT, "ASLCODE (Tree Completed)->"); break; case PARSEOP_DEFINITION_BLOCK: DbgPrint (ASL_PARSE_OUTPUT, "DEFINITION_BLOCK (Tree Completed)->"); break; case PARSEOP_OPERATIONREGION: DbgPrint (ASL_PARSE_OUTPUT, "OPREGION->"); break; case PARSEOP_OR: DbgPrint (ASL_PARSE_OUTPUT, "OR->"); break; default: /* Nothing to do for other opcodes */ break; } /* Link the new op to its children */ PrevChild = NULL; FirstChild = TRUE; for (i = 0; i < NumChildren; i++) { /* Get the next child */ Child = va_arg (ap, ACPI_PARSE_OBJECT *); DbgPrint (ASL_PARSE_OUTPUT, "%p, ", Child); /* * If child is NULL, this means that an optional argument * was omitted. We must create a placeholder with a special * opcode (DEFAULT_ARG) so that the code generator will know * that it must emit the correct default for this argument */ if (!Child) { Child = TrAllocateOp (PARSEOP_DEFAULT_ARG); } /* Link first child to parent */ if (FirstChild) { FirstChild = FALSE; Op->Asl.Child = Child; /* * For the ASL-/ASL+ converter: if the ParseOp is a Connection, * External, Offset or AccessAs, it means that the comments in the * FirstChild belongs to their parent due to the parsing order in * the .y files. To correct this, take the comments in the * FirstChild place it in the parent. This also means that * legitimate comments for the child gets put to the parent. */ if (AcpiGbl_CaptureComments && ((ParseOpcode == PARSEOP_CONNECTION) || (ParseOpcode == PARSEOP_EXTERNAL) || (ParseOpcode == PARSEOP_OFFSET) || (ParseOpcode == PARSEOP_ACCESSAS))) { Op->Asl.CommentList = Child->Asl.CommentList; Op->Asl.EndBlkComment = Child->Asl.EndBlkComment; Op->Asl.InlineComment = Child->Asl.InlineComment; Op->Asl.FileChanged = Child->Asl.FileChanged; Child->Asl.CommentList = NULL; Child->Asl.EndBlkComment = NULL; Child->Asl.InlineComment = NULL; Child->Asl.FileChanged = FALSE; /* * These do not need to be "passed off". They can be copied * because the code for these opcodes should be printed in the * same file. */ Op->Asl.Filename = Child->Asl.Filename; Op->Asl.ParentFilename = Child->Asl.ParentFilename; } } /* Point all children to parent */ Child->Asl.Parent = Op; /* Link children in a peer list */ if (PrevChild) { PrevChild->Asl.Next = Child; }; /* Get the comment from last child in the resource template call */ if (AcpiGbl_CaptureComments && (Op->Asl.ParseOpcode == PARSEOP_RESOURCETEMPLATE)) { CvDbgPrint ("Transferred current comment list to this op.\n"); Op->Asl.CommentList = Child->Asl.CommentList; Child->Asl.CommentList = NULL; Op->Asl.InlineComment = Child->Asl.InlineComment; Child->Asl.InlineComment = NULL; } /* * This child might be a list, point all ops in the list * to the same parent */ while (Child->Asl.Next) { Child = Child->Asl.Next; Child->Asl.Parent = Op; } PrevChild = Child; } va_end(ap); DbgPrint (ASL_PARSE_OUTPUT, "\n"); return (Op); } /******************************************************************************* * * FUNCTION: TrCreateLeafOp * * PARAMETERS: ParseOpcode - New opcode to be assigned to the op * * RETURN: Pointer to the new op. Aborts on allocation failure * * DESCRIPTION: Create a simple leaf op (no children or peers, and no value * assigned to the op) * ******************************************************************************/ ACPI_PARSE_OBJECT * TrCreateLeafOp ( UINT32 ParseOpcode) { ACPI_PARSE_OBJECT *Op; Op = TrAllocateOp (ParseOpcode); DbgPrint (ASL_PARSE_OUTPUT, "\nCreateLeafOp Ln/Col %u/%u NewOp %p Op %s\n\n", Op->Asl.LineNumber, Op->Asl.Column, Op, UtGetOpName (ParseOpcode)); return (Op); } /******************************************************************************* * * FUNCTION: TrCreateValuedLeafOp * * PARAMETERS: ParseOpcode - New opcode to be assigned to the op * Value - Value to be assigned to the op * * RETURN: Pointer to the new op. Aborts on allocation failure * * DESCRIPTION: Create a leaf op (no children or peers) with a value * assigned to it * ******************************************************************************/ ACPI_PARSE_OBJECT * TrCreateValuedLeafOp ( UINT32 ParseOpcode, UINT64 Value) { ACPI_PARSE_OBJECT *Op; Op = TrAllocateOp (ParseOpcode); Op->Asl.Value.Integer = Value; DbgPrint (ASL_PARSE_OUTPUT, "\nCreateValuedLeafOp Ln/Col %u/%u NewOp %p " "Op %s Value %8.8X%8.8X ", Op->Asl.LineNumber, Op->Asl.Column, Op, UtGetOpName(ParseOpcode), ACPI_FORMAT_UINT64 (Value)); switch (ParseOpcode) { case PARSEOP_STRING_LITERAL: DbgPrint (ASL_PARSE_OUTPUT, "STRING->%s", Op->Asl.Value.String); break; case PARSEOP_NAMESEG: DbgPrint (ASL_PARSE_OUTPUT, "NAMESEG->%s", Op->Asl.Value.String); break; case PARSEOP_NAMESTRING: DbgPrint (ASL_PARSE_OUTPUT, "NAMESTRING->%s", Op->Asl.Value.String); break; case PARSEOP_EISAID: DbgPrint (ASL_PARSE_OUTPUT, "EISAID->%s", Op->Asl.Value.String); break; case PARSEOP_METHOD: DbgPrint (ASL_PARSE_OUTPUT, "METHOD"); break; case PARSEOP_INTEGER: DbgPrint (ASL_PARSE_OUTPUT, "INTEGER->%8.8X%8.8X", ACPI_FORMAT_UINT64 (Value)); break; default: break; } DbgPrint (ASL_PARSE_OUTPUT, "\n\n"); return (Op); } /******************************************************************************* * * FUNCTION: TrCreateTargetOp * * PARAMETERS: OriginalOp - Op to be copied * * RETURN: Pointer to the new op. Aborts on allocation failure * * DESCRIPTION: Copy an existing op (and subtree). Used in ASL+ (C-style) * expressions where the target is the same as one of the * operands. A new op and subtree must be created from the * original so that the parse tree can be linked properly. * * NOTE: This code is specific to target operands that are the last * operand in an ASL/AML operator. Meaning that the top-level * parse Op in a possible subtree has a NULL Next pointer. * This simplifies the recursion. * * Subtree example: * DeRefOf (Local1) += 32 * * This gets converted to: * Add (DeRefOf (Local1), 32, DeRefOf (Local1)) * * Each DeRefOf has a single child, Local1. Even more complex * subtrees can be created via the Index and DeRefOf operators. * ******************************************************************************/ ACPI_PARSE_OBJECT * TrCreateTargetOp ( ACPI_PARSE_OBJECT *OriginalOp, ACPI_PARSE_OBJECT *ParentOp) { ACPI_PARSE_OBJECT *Op; if (!OriginalOp) { return (NULL); } Op = UtParseOpCacheCalloc (); /* Copy the pertinent values (omit link pointer fields) */ Op->Asl.Value = OriginalOp->Asl.Value; Op->Asl.Filename = OriginalOp->Asl.Filename; Op->Asl.LineNumber = OriginalOp->Asl.LineNumber; Op->Asl.LogicalLineNumber = OriginalOp->Asl.LogicalLineNumber; Op->Asl.LogicalByteOffset = OriginalOp->Asl.LogicalByteOffset; Op->Asl.Column = OriginalOp->Asl.Column; Op->Asl.Flags = OriginalOp->Asl.Flags; Op->Asl.CompileFlags = OriginalOp->Asl.CompileFlags; Op->Asl.AmlOpcode = OriginalOp->Asl.AmlOpcode; Op->Asl.ParseOpcode = OriginalOp->Asl.ParseOpcode; Op->Asl.Parent = ParentOp; UtSetParseOpName (Op); /* Copy a possible subtree below this op */ if (OriginalOp->Asl.Child) { Op->Asl.Child = TrCreateTargetOp (OriginalOp->Asl.Child, Op); } if (OriginalOp->Asl.Next) /* Null for top-level op */ { Op->Asl.Next = TrCreateTargetOp (OriginalOp->Asl.Next, ParentOp); } return (Op); } /******************************************************************************* * * FUNCTION: TrCreateAssignmentOp * * PARAMETERS: Target - Assignment target * Source - Assignment source * * RETURN: Pointer to the new op. Aborts on allocation failure * * DESCRIPTION: Implements the C-style '=' operator. It changes the parse * tree if possible to utilize the last argument of the math * operators which is a target operand -- thus saving invocation * of and additional Store() operator. An optimization. * ******************************************************************************/ ACPI_PARSE_OBJECT * TrCreateAssignmentOp ( ACPI_PARSE_OBJECT *Target, ACPI_PARSE_OBJECT *Source) { ACPI_PARSE_OBJECT *TargetOp; ACPI_PARSE_OBJECT *SourceOp1; ACPI_PARSE_OBJECT *SourceOp2; ACPI_PARSE_OBJECT *Operator; DbgPrint (ASL_PARSE_OUTPUT, "\nTrCreateAssignmentOp Line [%u to %u] Source %s Target %s\n", Source->Asl.LineNumber, Source->Asl.EndLine, UtGetOpName (Source->Asl.ParseOpcode), UtGetOpName (Target->Asl.ParseOpcode)); TrSetOpFlags (Target, OP_IS_TARGET); switch (Source->Asl.ParseOpcode) { /* * Only these operators can be optimized because they have * a target operand */ case PARSEOP_ADD: case PARSEOP_AND: case PARSEOP_DIVIDE: case PARSEOP_INDEX: case PARSEOP_MOD: case PARSEOP_MULTIPLY: case PARSEOP_NOT: case PARSEOP_OR: case PARSEOP_SHIFTLEFT: case PARSEOP_SHIFTRIGHT: case PARSEOP_SUBTRACT: case PARSEOP_XOR: break; /* Otherwise, just create a normal Store operator */ default: goto CannotOptimize; } /* * Transform the parse tree such that the target is moved to the * last operand of the operator */ SourceOp1 = Source->Asl.Child; SourceOp2 = SourceOp1->Asl.Next; /* NOT only has one operand, but has a target */ if (Source->Asl.ParseOpcode == PARSEOP_NOT) { SourceOp2 = SourceOp1; } /* DIVIDE has an extra target operand (remainder) */ if (Source->Asl.ParseOpcode == PARSEOP_DIVIDE) { SourceOp2 = SourceOp2->Asl.Next; } TargetOp = SourceOp2->Asl.Next; /* * Can't perform this optimization if there already is a target * for the operator (ZERO is a "no target" placeholder). */ if (TargetOp->Asl.ParseOpcode != PARSEOP_ZERO) { goto CannotOptimize; } /* Link in the target as the final operand */ SourceOp2->Asl.Next = Target; Target->Asl.Parent = Source; return (Source); CannotOptimize: Operator = TrAllocateOp (PARSEOP_STORE); TrLinkOpChildren (Operator, 2, Source, Target); /* Set the appropriate line numbers for the new op */ Operator->Asl.LineNumber = Target->Asl.LineNumber; Operator->Asl.LogicalLineNumber = Target->Asl.LogicalLineNumber; Operator->Asl.LogicalByteOffset = Target->Asl.LogicalByteOffset; Operator->Asl.Column = Target->Asl.Column; return (Operator); } /******************************************************************************* * * FUNCTION: TrCreateNullTargetOp * * PARAMETERS: None * * RETURN: Pointer to the new op. Aborts on allocation failure * * DESCRIPTION: Create a "null" target op. This is defined by the ACPI * specification to be a zero AML opcode, and indicates that * no target has been specified for the parent operation * ******************************************************************************/ ACPI_PARSE_OBJECT * TrCreateNullTargetOp ( void) { ACPI_PARSE_OBJECT *Op; Op = TrAllocateOp (PARSEOP_ZERO); Op->Asl.CompileFlags |= (OP_IS_TARGET | OP_COMPILE_TIME_CONST); DbgPrint (ASL_PARSE_OUTPUT, "\nCreateNullTargetOp Ln/Col %u/%u NewOp %p Op %s\n", Op->Asl.LineNumber, Op->Asl.Column, Op, UtGetOpName (Op->Asl.ParseOpcode)); return (Op); } /******************************************************************************* * * FUNCTION: TrCreateConstantLeafOp * * PARAMETERS: ParseOpcode - The constant opcode * * RETURN: Pointer to the new op. Aborts on allocation failure * * DESCRIPTION: Create a leaf op (no children or peers) for one of the * special constants - __LINE__, __FILE__, and __DATE__. * * Note: The fullimplemenation of __METHOD__ cannot happen here because we * don't have a full parse tree at this time and cannot find the parent * control method. __METHOD__ must be implemented later, after the parse * tree has been fully constructed. * ******************************************************************************/ ACPI_PARSE_OBJECT * TrCreateConstantLeafOp ( UINT32 ParseOpcode) { ACPI_PARSE_OBJECT *Op = NULL; time_t CurrentTime; char *StaticTimeString; char *TimeString; char *Filename = NULL; ACPI_STATUS Status; switch (ParseOpcode) { case PARSEOP___LINE__: Op = TrAllocateOp (PARSEOP_INTEGER); Op->Asl.Value.Integer = Op->Asl.LineNumber; break; case PARSEOP___METHOD__: /* Will become a string literal later */ Op = TrAllocateOp (PARSEOP___METHOD__); Op->Asl.Value.String = NULL; break; case PARSEOP___PATH__: Op = TrAllocateOp (PARSEOP_STRING_LITERAL); /* Op.Asl.Filename contains the full pathname to the file */ Op->Asl.Value.String = Op->Asl.Filename; break; case PARSEOP___FILE__: Op = TrAllocateOp (PARSEOP_STRING_LITERAL); /* Get the simple filename from the full path */ Status = FlSplitInputPathname (Op->Asl.Filename, NULL, &Filename); if (ACPI_FAILURE (Status)) { return (NULL); } Op->Asl.Value.String = Filename; break; case PARSEOP___DATE__: Op = TrAllocateOp (PARSEOP_STRING_LITERAL); /* Get a copy of the current time */ + Op->Asl.Value.String = ""; CurrentTime = time (NULL); + StaticTimeString = ctime (&CurrentTime); - TimeString = UtLocalCalloc (strlen (StaticTimeString) + 1); - strcpy (TimeString, StaticTimeString); + if (StaticTimeString) + { + TimeString = UtLocalCalloc (strlen (StaticTimeString) + 1); + strcpy (TimeString, StaticTimeString); - TimeString[strlen(TimeString) -1] = 0; /* Remove trailing newline */ - Op->Asl.Value.String = TimeString; + TimeString[strlen(TimeString) -1] = 0; /* Remove trailing newline */ + Op->Asl.Value.String = TimeString; + } break; default: /* This would be an internal error */ return (NULL); } DbgPrint (ASL_PARSE_OUTPUT, "\nCreateConstantLeafOp Ln/Col %u/%u NewOp %p " "Op %s Value %8.8X%8.8X \n", Op->Asl.LineNumber, Op->Asl.Column, Op, UtGetOpName (ParseOpcode), ACPI_FORMAT_UINT64 (Op->Asl.Value.Integer)); return (Op); } /******************************************************************************* * * FUNCTION: TrAllocateOp * * PARAMETERS: ParseOpcode - Opcode to be assigned to the op * * RETURN: New parse op. Aborts on allocation failure * * DESCRIPTION: Allocate and initialize a new parse op for the parse tree * ******************************************************************************/ ACPI_PARSE_OBJECT * TrAllocateOp ( UINT32 ParseOpcode) { ACPI_PARSE_OBJECT *Op; ACPI_PARSE_OBJECT *LatestOp; Op = UtParseOpCacheCalloc (); Op->Asl.ParseOpcode = (UINT16) ParseOpcode; Op->Asl.Filename = AslGbl_Files[ASL_FILE_INPUT].Filename; Op->Asl.LineNumber = AslGbl_CurrentLineNumber; Op->Asl.LogicalLineNumber = AslGbl_LogicalLineNumber; Op->Asl.LogicalByteOffset = AslGbl_CurrentLineOffset; Op->Asl.Column = AslGbl_CurrentColumn; UtSetParseOpName (Op); /* The following is for capturing comments */ if (AcpiGbl_CaptureComments) { LatestOp = AslGbl_CommentState.LatestParseOp; Op->Asl.InlineComment = NULL; Op->Asl.EndNodeComment = NULL; Op->Asl.CommentList = NULL; Op->Asl.FileChanged = FALSE; /* * Check to see if the file name has changed before resetting the * latest parse op. */ if (LatestOp && (ParseOpcode != PARSEOP_INCLUDE) && (ParseOpcode != PARSEOP_INCLUDE_END) && strcmp (LatestOp->Asl.Filename, Op->Asl.Filename)) { CvDbgPrint ("latest op: %s\n", LatestOp->Asl.ParseOpName); Op->Asl.FileChanged = TRUE; if (AslGbl_IncludeFileStack) { Op->Asl.ParentFilename = AslGbl_IncludeFileStack->Filename; } else { Op->Asl.ParentFilename = NULL; } } AslGbl_CommentState.LatestParseOp = Op; CvDbgPrint ("TrAllocateOp=Set latest parse op to this op.\n"); CvDbgPrint (" Op->Asl.ParseOpName = %s\n", AslGbl_CommentState.LatestParseOp->Asl.ParseOpName); CvDbgPrint (" Op->Asl.ParseOpcode = 0x%x\n", ParseOpcode); if (Op->Asl.FileChanged) { CvDbgPrint(" file has been changed!\n"); } /* * if this parse op's syntax uses () and {} (i.e. Package(1){0x00}) then * set a flag in the comment state. This facilitates paring comments for * these types of opcodes. */ if ((CvParseOpBlockType(Op) == (BLOCK_PAREN | BLOCK_BRACE)) && (ParseOpcode != PARSEOP_DEFINITION_BLOCK)) { CvDbgPrint ("Parsing paren/Brace op now!\n"); AslGbl_CommentState.ParsingParenBraceNode = Op; } if (AslGbl_CommentListHead) { CvDbgPrint ("Transferring...\n"); Op->Asl.CommentList = AslGbl_CommentListHead; AslGbl_CommentListHead = NULL; AslGbl_CommentListTail = NULL; CvDbgPrint (" Transferred current comment list to this op.\n"); CvDbgPrint (" %s\n", Op->Asl.CommentList->Comment); } if (AslGbl_InlineCommentBuffer) { Op->Asl.InlineComment = AslGbl_InlineCommentBuffer; AslGbl_InlineCommentBuffer = NULL; CvDbgPrint ("Transferred current inline comment list to this op.\n"); } } return (Op); } /******************************************************************************* * * FUNCTION: TrPrintOpFlags * * PARAMETERS: Flags - Flags word to be decoded * OutputLevel - Debug output level: ASL_TREE_OUTPUT etc. * * RETURN: None * * DESCRIPTION: Decode a flags word to text. Displays all flags that are set. * ******************************************************************************/ void TrPrintOpFlags ( UINT32 Flags, UINT32 OutputLevel) { UINT32 FlagBit = 1; UINT32 i; for (i = 0; i < ACPI_NUM_OP_FLAGS; i++) { if (Flags & FlagBit) { DbgPrint (OutputLevel, " %s", AslGbl_OpFlagNames[i]); } FlagBit <<= 1; } } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslprimaries.y =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslprimaries.y (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslprimaries.y (revision 357966) @@ -1,1474 +1,1474 @@ NoEcho(' /****************************************************************************** * * Module Name: aslprimaries.y - Rules for primary ASL operators * - Keep this file synched with the * CvParseOpBlockType function in cvcompiler.c * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ ') /******************************************************************************* * * ASL Primary Terms * ******************************************************************************/ AccessAsTerm : PARSEOP_ACCESSAS PARSEOP_OPEN_PAREN AccessTypeKeyword OptionalAccessAttribTerm PARSEOP_CLOSE_PAREN {$$ = TrCreateOp (PARSEOP_ACCESSAS,2,$3,$4);} | PARSEOP_ACCESSAS PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; AcquireTerm : PARSEOP_ACQUIRE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp(PARSEOP_ACQUIRE);} SuperName ',' WordConstExpr PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$6);} | PARSEOP_ACQUIRE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; AddTerm : PARSEOP_ADD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_ADD);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_ADD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; AliasTerm : PARSEOP_ALIAS PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_ALIAS);} NameString NameStringItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4, TrSetOpFlags ($5, OP_IS_NAME_DECLARATION));} | PARSEOP_ALIAS PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; AndTerm : PARSEOP_AND PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_AND);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_AND PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ArgTerm : PARSEOP_ARG0 {$$ = TrCreateLeafOp (PARSEOP_ARG0);} | PARSEOP_ARG1 {$$ = TrCreateLeafOp (PARSEOP_ARG1);} | PARSEOP_ARG2 {$$ = TrCreateLeafOp (PARSEOP_ARG2);} | PARSEOP_ARG3 {$$ = TrCreateLeafOp (PARSEOP_ARG3);} | PARSEOP_ARG4 {$$ = TrCreateLeafOp (PARSEOP_ARG4);} | PARSEOP_ARG5 {$$ = TrCreateLeafOp (PARSEOP_ARG5);} | PARSEOP_ARG6 {$$ = TrCreateLeafOp (PARSEOP_ARG6);} ; BankFieldTerm : PARSEOP_BANKFIELD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_BANKFIELD);} NameString NameStringItem TermArgItem OptionalAccessTypeKeyword OptionalLockRuleKeyword OptionalUpdateRuleKeyword PARSEOP_CLOSE_PAREN '{' FieldUnitList '}' {$$ = TrLinkOpChildren ($3,7, $4,$5,$6,$7,$8,$9,$12);} | PARSEOP_BANKFIELD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN '{' error '}' {$$ = AslDoError(); yyclearin;} ; BreakTerm : PARSEOP_BREAK {$$ = TrCreateOp (PARSEOP_BREAK, 0);} ; BreakPointTerm : PARSEOP_BREAKPOINT {$$ = TrCreateOp (PARSEOP_BREAKPOINT, 0);} ; BufferTerm : PARSEOP_BUFFER {$$ = TrCreateLeafOp (PARSEOP_BUFFER); COMMENT_CAPTURE_OFF; } OptionalDataCount '{' BufferTermData '}' {$$ = TrLinkOpChildren ($2,2,$3,$5); COMMENT_CAPTURE_ON;} ; BufferTermData : ByteList {} | StringData {} ; CaseTerm : PARSEOP_CASE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CASE);} DataObject PARSEOP_CLOSE_PAREN '{' TermList '}' {$$ = TrLinkOpChildren ($3,2,$4,$7);} | PARSEOP_CASE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ConcatTerm : PARSEOP_CONCATENATE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CONCATENATE);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_CONCATENATE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ConcatResTerm : PARSEOP_CONCATENATERESTEMPLATE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp ( PARSEOP_CONCATENATERESTEMPLATE);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_CONCATENATERESTEMPLATE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; CondRefOfTerm : PARSEOP_CONDREFOF PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CONDREFOF);} CondRefOfSource Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_CONDREFOF PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ConnectionTerm : PARSEOP_CONNECTION PARSEOP_OPEN_PAREN NameString PARSEOP_CLOSE_PAREN {$$ = TrCreateOp (PARSEOP_CONNECTION,1,$3);} | PARSEOP_CONNECTION PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CONNECTION);} ResourceMacroTerm PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3, 1, TrLinkOpChildren ( TrCreateLeafOp (PARSEOP_RESOURCETEMPLATE), 3, TrCreateLeafOp (PARSEOP_DEFAULT_ARG), TrCreateLeafOp (PARSEOP_DEFAULT_ARG), $4));} | PARSEOP_CONNECTION PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ContinueTerm : PARSEOP_CONTINUE {$$ = TrCreateOp (PARSEOP_CONTINUE, 0);} ; CopyObjectTerm : PARSEOP_COPYOBJECT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_COPYOBJECT);} TermArg ',' SimpleName PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4, TrSetOpFlags ($6, OP_IS_TARGET));} | PARSEOP_COPYOBJECT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; CreateBitFieldTerm : PARSEOP_CREATEBITFIELD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CREATEBITFIELD);} TermArg TermArgItem NameStringItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5, TrSetOpFlags ($6, OP_IS_NAME_DECLARATION));} | PARSEOP_CREATEBITFIELD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; CreateByteFieldTerm : PARSEOP_CREATEBYTEFIELD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CREATEBYTEFIELD);} TermArg TermArgItem NameStringItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5, TrSetOpFlags ($6, OP_IS_NAME_DECLARATION));} | PARSEOP_CREATEBYTEFIELD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; CreateDWordFieldTerm : PARSEOP_CREATEDWORDFIELD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CREATEDWORDFIELD);} TermArg TermArgItem NameStringItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5, TrSetOpFlags ($6, OP_IS_NAME_DECLARATION));} | PARSEOP_CREATEDWORDFIELD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; CreateFieldTerm : PARSEOP_CREATEFIELD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CREATEFIELD);} TermArg TermArgItem TermArgItem NameStringItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,4,$4,$5,$6, TrSetOpFlags ($7, OP_IS_NAME_DECLARATION));} | PARSEOP_CREATEFIELD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; CreateQWordFieldTerm : PARSEOP_CREATEQWORDFIELD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CREATEQWORDFIELD);} TermArg TermArgItem NameStringItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5, TrSetOpFlags ($6, OP_IS_NAME_DECLARATION));} | PARSEOP_CREATEQWORDFIELD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; CreateWordFieldTerm : PARSEOP_CREATEWORDFIELD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_CREATEWORDFIELD);} TermArg TermArgItem NameStringItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5, TrSetOpFlags ($6, OP_IS_NAME_DECLARATION));} | PARSEOP_CREATEWORDFIELD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; DataRegionTerm : PARSEOP_DATATABLEREGION PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_DATATABLEREGION);} NameString TermArgItem TermArgItem TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,4, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION),$5,$6,$7);} | PARSEOP_DATATABLEREGION PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; DebugTerm : PARSEOP_DEBUG {$$ = TrCreateLeafOp (PARSEOP_DEBUG);} ; DecTerm : PARSEOP_DECREMENT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_DECREMENT);} SuperName PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_DECREMENT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; DefaultTerm : PARSEOP_DEFAULT '{' {$$ = TrCreateLeafOp (PARSEOP_DEFAULT);} TermList '}' {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_DEFAULT '{' error '}' {$$ = AslDoError(); yyclearin;} ; DerefOfTerm : PARSEOP_DEREFOF PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_DEREFOF);} DerefOfSource PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_DEREFOF PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; DeviceTerm : PARSEOP_DEVICE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_DEVICE);} NameString PARSEOP_CLOSE_PAREN '{' TermList '}' {$$ = TrLinkOpChildren ($3,2, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION),$7);} | PARSEOP_DEVICE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; DivideTerm : PARSEOP_DIVIDE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_DIVIDE);} TermArg TermArgItem Target Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,4,$4,$5,$6,$7);} | PARSEOP_DIVIDE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; EISAIDTerm : PARSEOP_EISAID PARSEOP_OPEN_PAREN StringData PARSEOP_CLOSE_PAREN {$$ = TrSetOpIntegerValue (PARSEOP_EISAID, $3);} | PARSEOP_EISAID PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ElseIfTerm : IfTerm ElseTerm {$$ = TrLinkPeerOp ($1,$2);} ; ElseTerm : {$$ = NULL;} | PARSEOP_ELSE '{' TermList {$$ = TrCreateLeafOp (PARSEOP_ELSE);} '}' {$$ = TrLinkOpChildren ($4,1,$3);} | PARSEOP_ELSE '{' error '}' {$$ = AslDoError(); yyclearin;} | PARSEOP_ELSE error {$$ = AslDoError(); yyclearin;} | PARSEOP_ELSEIF PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_ELSE);} TermArg {$$ = TrCreateLeafOp (PARSEOP_IF);} PARSEOP_CLOSE_PAREN '{' TermList '}' {TrLinkOpChildren ($5,2,$4,$8);} ElseTerm {TrLinkPeerOp ($5,$11);} {$$ = TrLinkOpChildren ($3,1,$5);} | PARSEOP_ELSEIF PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} | PARSEOP_ELSEIF error {$$ = AslDoError(); yyclearin;} ; EventTerm : PARSEOP_EVENT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_EVENT);} NameString PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION));} | PARSEOP_EVENT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ExternalTerm : PARSEOP_EXTERNAL PARSEOP_OPEN_PAREN NameString OptionalObjectTypeKeyword OptionalParameterTypePackage OptionalParameterTypesPackage PARSEOP_CLOSE_PAREN {$$ = TrCreateOp (PARSEOP_EXTERNAL,4,$3,$4,$5,$6);} | PARSEOP_EXTERNAL PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; FatalTerm : PARSEOP_FATAL PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_FATAL);} ByteConstExpr ',' DWordConstExpr TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$6,$7);} | PARSEOP_FATAL PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; FieldTerm : PARSEOP_FIELD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_FIELD);} NameString OptionalAccessTypeKeyword OptionalLockRuleKeyword OptionalUpdateRuleKeyword PARSEOP_CLOSE_PAREN '{' FieldUnitList '}' {$$ = TrLinkOpChildren ($3,5,$4,$5,$6,$7,$10);} | PARSEOP_FIELD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN '{' error '}' {$$ = AslDoError(); yyclearin;} ; FindSetLeftBitTerm : PARSEOP_FINDSETLEFTBIT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_FINDSETLEFTBIT);} TermArg Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_FINDSETLEFTBIT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; FindSetRightBitTerm : PARSEOP_FINDSETRIGHTBIT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_FINDSETRIGHTBIT);} TermArg Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_FINDSETRIGHTBIT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; /* Convert a For() loop to a While() loop */ ForTerm : PARSEOP_FOR PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_WHILE);} OptionalTermArg ',' {} OptionalPredicate ',' OptionalTermArg {$$ = TrLinkPeerOp ($4,$3); TrSetOpParent ($9,$3);} /* New parent is WHILE */ PARSEOP_CLOSE_PAREN '{' TermList '}' {$$ = TrLinkOpChildren ($3,2,$7,$13);} {$$ = TrLinkPeerOp ($13,$9); $$ = $10;} ; OptionalPredicate : {$$ = TrCreateValuedLeafOp (PARSEOP_INTEGER, 1);} | TermArg {$$ = $1;} ; FprintfTerm : PARSEOP_FPRINTF PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_FPRINTF);} TermArg ',' StringData PrintfArgList PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$6,$7);} | PARSEOP_FPRINTF PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; FromBCDTerm : PARSEOP_FROMBCD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_FROMBCD);} TermArg Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_FROMBCD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; FunctionTerm : PARSEOP_FUNCTION PARSEOP_OPEN_PAREN {COMMENT_CAPTURE_OFF; $$ = TrCreateLeafOp (PARSEOP_METHOD); } NameString OptionalParameterTypePackage OptionalParameterTypesPackage PARSEOP_CLOSE_PAREN '{' {COMMENT_CAPTURE_ON; } TermList '}' {$$ = TrLinkOpChildren ($3,7, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION), - TrCreateValuedLeafOp (PARSEOP_BYTECONST, 0), + TrCreateLeafOp (PARSEOP_DEFAULT_ARG), TrCreateLeafOp (PARSEOP_SERIALIZERULE_NOTSERIAL), TrCreateValuedLeafOp (PARSEOP_BYTECONST, 0),$5,$6,$10);} | PARSEOP_FUNCTION PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; IfTerm : PARSEOP_IF PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_IF);} TermArg PARSEOP_CLOSE_PAREN '{' TermList '}' {$$ = TrLinkOpChildren ($3,2,$4,$7);} | PARSEOP_IF PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; IncludeTerm : PARSEOP_INCLUDE PARSEOP_OPEN_PAREN String PARSEOP_CLOSE_PAREN {$$ = TrSetOpIntegerValue (PARSEOP_INCLUDE, $3); FlOpenIncludeFile ($3);} ; IncludeEndTerm : PARSEOP_INCLUDE_END {$$ = TrCreateLeafOp (PARSEOP_INCLUDE_END); TrSetOpCurrentFilename ($$);} ; IncTerm : PARSEOP_INCREMENT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_INCREMENT);} SuperName PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_INCREMENT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; IndexFieldTerm : PARSEOP_INDEXFIELD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_INDEXFIELD);} NameString NameStringItem OptionalAccessTypeKeyword OptionalLockRuleKeyword OptionalUpdateRuleKeyword PARSEOP_CLOSE_PAREN '{' FieldUnitList '}' {$$ = TrLinkOpChildren ($3,6,$4,$5,$6,$7,$8,$11);} | PARSEOP_INDEXFIELD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN '{' error '}' {$$ = AslDoError(); yyclearin;} ; IndexTerm : PARSEOP_INDEX PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_INDEX);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_INDEX PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LAndTerm : PARSEOP_LAND PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LAND);} TermArg TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_LAND PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LEqualTerm : PARSEOP_LEQUAL PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LEQUAL);} TermArg TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_LEQUAL PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LGreaterEqualTerm : PARSEOP_LGREATEREQUAL PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LLESS);} TermArg TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrCreateOp (PARSEOP_LNOT, 1, TrLinkOpChildren ($3,2,$4,$5));} | PARSEOP_LGREATEREQUAL PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LGreaterTerm : PARSEOP_LGREATER PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LGREATER);} TermArg TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_LGREATER PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LLessEqualTerm : PARSEOP_LLESSEQUAL PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LGREATER);} TermArg TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrCreateOp (PARSEOP_LNOT, 1, TrLinkOpChildren ($3,2,$4,$5));} | PARSEOP_LLESSEQUAL PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LLessTerm : PARSEOP_LLESS PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LLESS);} TermArg TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_LLESS PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LNotEqualTerm : PARSEOP_LNOTEQUAL PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LEQUAL);} TermArg TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrCreateOp (PARSEOP_LNOT, 1, TrLinkOpChildren ($3,2,$4,$5));} | PARSEOP_LNOTEQUAL PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LNotTerm : PARSEOP_LNOT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LNOT);} TermArg PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_LNOT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LoadTableTerm : PARSEOP_LOADTABLE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LOADTABLE);} TermArg TermArgItem TermArgItem OptionalListString OptionalListString OptionalReference PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,6,$4,$5,$6,$7,$8,$9);} | PARSEOP_LOADTABLE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LoadTerm : PARSEOP_LOAD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LOAD);} NameString RequiredTarget PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_LOAD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; LocalTerm : PARSEOP_LOCAL0 {$$ = TrCreateLeafOp (PARSEOP_LOCAL0);} | PARSEOP_LOCAL1 {$$ = TrCreateLeafOp (PARSEOP_LOCAL1);} | PARSEOP_LOCAL2 {$$ = TrCreateLeafOp (PARSEOP_LOCAL2);} | PARSEOP_LOCAL3 {$$ = TrCreateLeafOp (PARSEOP_LOCAL3);} | PARSEOP_LOCAL4 {$$ = TrCreateLeafOp (PARSEOP_LOCAL4);} | PARSEOP_LOCAL5 {$$ = TrCreateLeafOp (PARSEOP_LOCAL5);} | PARSEOP_LOCAL6 {$$ = TrCreateLeafOp (PARSEOP_LOCAL6);} | PARSEOP_LOCAL7 {$$ = TrCreateLeafOp (PARSEOP_LOCAL7);} ; LOrTerm : PARSEOP_LOR PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_LOR);} TermArg TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_LOR PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; MatchTerm : PARSEOP_MATCH PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_MATCH);} TermArg ',' MatchOpKeyword TermArgItem ',' MatchOpKeyword TermArgItem TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,6,$4,$6,$7,$9,$10,$11);} | PARSEOP_MATCH PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; MethodTerm : PARSEOP_METHOD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_METHOD); COMMENT_CAPTURE_OFF;} NameString OptionalByteConstExpr {UtCheckIntegerRange ($5, 0, 7);} OptionalSerializeRuleKeyword OptionalByteConstExpr OptionalParameterTypePackage OptionalParameterTypesPackage PARSEOP_CLOSE_PAREN '{' {COMMENT_CAPTURE_ON;} TermList '}' {$$ = TrLinkOpChildren ($3,7, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION), $5,$7,$8,$9,$10,$14);} | PARSEOP_METHOD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; MidTerm : PARSEOP_MID PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_MID);} TermArg TermArgItem TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,4,$4,$5,$6,$7);} | PARSEOP_MID PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ModTerm : PARSEOP_MOD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_MOD);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_MOD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; MultiplyTerm : PARSEOP_MULTIPLY PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_MULTIPLY);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_MULTIPLY PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; MutexTerm : PARSEOP_MUTEX PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_MUTEX);} NameString OptionalSyncLevel PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION),$5);} | PARSEOP_MUTEX PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; NameTerm : PARSEOP_NAME PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_NAME);} NameString ',' DataObject PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION),$6);} | PARSEOP_NAME PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; NAndTerm : PARSEOP_NAND PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_NAND);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_NAND PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; NoOpTerm : PARSEOP_NOOP {$$ = TrCreateOp (PARSEOP_NOOP, 0);} ; NOrTerm : PARSEOP_NOR PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_NOR);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_NOR PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; NotifyTerm : PARSEOP_NOTIFY PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_NOTIFY);} SuperName TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_NOTIFY PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; NotTerm : PARSEOP_NOT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_NOT);} TermArg Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_NOT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ObjectTypeTerm : PARSEOP_OBJECTTYPE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_OBJECTTYPE);} ObjectTypeSource PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_OBJECTTYPE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; OffsetTerm : PARSEOP_OFFSET PARSEOP_OPEN_PAREN AmlPackageLengthTerm PARSEOP_CLOSE_PAREN {$$ = TrCreateOp (PARSEOP_OFFSET,1,$3);} | PARSEOP_OFFSET PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; OpRegionTerm : PARSEOP_OPERATIONREGION PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_OPERATIONREGION);} NameString ',' OpRegionSpaceIdTerm TermArgItem TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,4, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION), $6,$7,$8);} | PARSEOP_OPERATIONREGION PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; OpRegionSpaceIdTerm : RegionSpaceKeyword {} | ByteConst {$$ = UtCheckIntegerRange ($1, 0x80, 0xFF);} ; OrTerm : PARSEOP_OR PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_OR);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_OR PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; PackageTerm : PARSEOP_PACKAGE {$$ = TrCreateLeafOp (PARSEOP_VAR_PACKAGE);} OptionalDataCount '{' PackageList '}' {$$ = TrLinkOpChildren ($2,2,$3,$5);} PowerResTerm : PARSEOP_POWERRESOURCE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_POWERRESOURCE);} NameString ',' ByteConstExpr ',' WordConstExpr PARSEOP_CLOSE_PAREN '{' TermList '}' {$$ = TrLinkOpChildren ($3,4, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION), $6,$8,$11);} | PARSEOP_POWERRESOURCE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; PrintfTerm : PARSEOP_PRINTF PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_PRINTF);} StringData PrintfArgList PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_PRINTF PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; PrintfArgList : {$$ = NULL;} | TermArg {$$ = $1;} | PrintfArgList ',' TermArg {$$ = TrLinkPeerOp ($1, $3);} ; ProcessorTerm : PARSEOP_PROCESSOR PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_PROCESSOR);} NameString ',' ByteConstExpr OptionalDWordConstExpr OptionalByteConstExpr PARSEOP_CLOSE_PAREN '{' TermList '}' {$$ = TrLinkOpChildren ($3,5, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION), $6,$7,$8,$11);} | PARSEOP_PROCESSOR PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; RawDataBufferTerm : PARSEOP_DATABUFFER PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_DATABUFFER);} OptionalWordConst PARSEOP_CLOSE_PAREN '{' ByteList '}' {$$ = TrLinkOpChildren ($3,2,$4,$7);} | PARSEOP_DATABUFFER PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; /* * In RefOf, the node isn't really a target, but we can't keep track of it after * we've taken a pointer to it. (hard to tell if a local becomes initialized this way.) */ RefOfTerm : PARSEOP_REFOF PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_REFOF);} RefOfSource PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1, TrSetOpFlags ($4, OP_IS_TARGET));} | PARSEOP_REFOF PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ReleaseTerm : PARSEOP_RELEASE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_RELEASE);} SuperName PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_RELEASE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ResetTerm : PARSEOP_RESET PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_RESET);} SuperName PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_RESET PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ReturnTerm : PARSEOP_RETURN PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_RETURN);} OptionalReturnArg PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_RETURN {$$ = TrLinkOpChildren ( TrCreateLeafOp (PARSEOP_RETURN),1, TrSetOpFlags (TrCreateLeafOp (PARSEOP_ZERO), OP_IS_NULL_RETURN));} | PARSEOP_RETURN PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ScopeTerm : PARSEOP_SCOPE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_SCOPE);} NameString PARSEOP_CLOSE_PAREN '{' TermList '}' {$$ = TrLinkOpChildren ($3,2, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION),$7);} | PARSEOP_SCOPE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ShiftLeftTerm : PARSEOP_SHIFTLEFT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_SHIFTLEFT);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_SHIFTLEFT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ShiftRightTerm : PARSEOP_SHIFTRIGHT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_SHIFTRIGHT);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_SHIFTRIGHT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; SignalTerm : PARSEOP_SIGNAL PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_SIGNAL);} SuperName PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_SIGNAL PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; SizeOfTerm : PARSEOP_SIZEOF PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_SIZEOF);} SuperName PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_SIZEOF PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; SleepTerm : PARSEOP_SLEEP PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_SLEEP);} TermArg PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_SLEEP PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; StallTerm : PARSEOP_STALL PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_STALL);} TermArg PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_STALL PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; StoreTerm : PARSEOP_STORE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_STORE);} TermArg ',' SuperName PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4, TrSetOpFlags ($6, OP_IS_TARGET));} | PARSEOP_STORE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; SubtractTerm : PARSEOP_SUBTRACT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_SUBTRACT);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_SUBTRACT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; SwitchTerm : PARSEOP_SWITCH PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_SWITCH);} TermArg PARSEOP_CLOSE_PAREN '{' CaseDefaultTermList '}' {$$ = TrLinkOpChildren ($3,2,$4,$7);} | PARSEOP_SWITCH PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ThermalZoneTerm : PARSEOP_THERMALZONE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_THERMALZONE);} NameString PARSEOP_CLOSE_PAREN '{' TermList '}' {$$ = TrLinkOpChildren ($3,2, TrSetOpFlags ($4, OP_IS_NAME_DECLARATION),$7);} | PARSEOP_THERMALZONE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; TimerTerm : PARSEOP_TIMER PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_TIMER);} PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,0);} | PARSEOP_TIMER {$$ = TrLinkOpChildren ( TrCreateLeafOp (PARSEOP_TIMER),0);} | PARSEOP_TIMER PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ToBCDTerm : PARSEOP_TOBCD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_TOBCD);} TermArg Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_TOBCD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ToBufferTerm : PARSEOP_TOBUFFER PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_TOBUFFER);} TermArg Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_TOBUFFER PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ToDecimalStringTerm : PARSEOP_TODECIMALSTRING PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_TODECIMALSTRING);} TermArg Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_TODECIMALSTRING PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ToHexStringTerm : PARSEOP_TOHEXSTRING PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_TOHEXSTRING);} TermArg Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_TOHEXSTRING PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ToIntegerTerm : PARSEOP_TOINTEGER PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_TOINTEGER);} TermArg Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_TOINTEGER PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ToPLDTerm : PARSEOP_TOPLD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_TOPLD);} PldKeywordList PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_TOPLD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; PldKeywordList : {$$ = NULL;} | PldKeyword PARSEOP_EXP_EQUALS Integer {$$ = TrLinkOpChildren ($1,1,$3);} | PldKeyword PARSEOP_EXP_EQUALS String {$$ = TrLinkOpChildren ($1,1,$3);} | PldKeywordList ',' /* Allows a trailing comma at list end */ | PldKeywordList ',' PldKeyword PARSEOP_EXP_EQUALS Integer {$$ = TrLinkPeerOp ($1,TrLinkOpChildren ($3,1,$5));} | PldKeywordList ',' PldKeyword PARSEOP_EXP_EQUALS String {$$ = TrLinkPeerOp ($1,TrLinkOpChildren ($3,1,$5));} ; ToStringTerm : PARSEOP_TOSTRING PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_TOSTRING);} TermArg OptionalCount Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_TOSTRING PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; ToUUIDTerm : PARSEOP_TOUUID PARSEOP_OPEN_PAREN StringData PARSEOP_CLOSE_PAREN {$$ = TrSetOpIntegerValue (PARSEOP_TOUUID, $3);} | PARSEOP_TOUUID PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; UnicodeTerm : PARSEOP_UNICODE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_UNICODE);} StringData PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,0,$4);} | PARSEOP_UNICODE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; UnloadTerm : PARSEOP_UNLOAD PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_UNLOAD);} SuperName PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,1,$4);} | PARSEOP_UNLOAD PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; WaitTerm : PARSEOP_WAIT PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_WAIT);} SuperName TermArgItem PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,2,$4,$5);} | PARSEOP_WAIT PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; XOrTerm : PARSEOP_XOR PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_XOR);} TermArg TermArgItem Target PARSEOP_CLOSE_PAREN {$$ = TrLinkOpChildren ($3,3,$4,$5,$6);} | PARSEOP_XOR PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; WhileTerm : PARSEOP_WHILE PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_WHILE);} TermArg PARSEOP_CLOSE_PAREN '{' TermList '}' {$$ = TrLinkOpChildren ($3,2,$4,$7);} | PARSEOP_WHILE PARSEOP_OPEN_PAREN error PARSEOP_CLOSE_PAREN {$$ = AslDoError(); yyclearin;} ; Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslrules.y =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslrules.y (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslrules.y (revision 357966) @@ -1,883 +1,886 @@ NoEcho(' /****************************************************************************** * * Module Name: aslrules.y - Main Bison/Yacc production rules * - Keep this file synched with the * CvParseOpBlockType function in cvcompiler.c * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ ') /******************************************************************************* * * ASL Root and Secondary Terms * ******************************************************************************/ /* * Root term. Allow multiple #line directives before the definition block * to handle output from preprocessors */ AslCode : DefinitionBlockList {$$ = TrLinkOpChildren ( TrCreateLeafOp (PARSEOP_ASL_CODE),1, $1);} | error {YYABORT; $$ = NULL;} ; /* * Note concerning support for "module-level code". * * ACPI 1.0 allowed Type1 and Type2 executable opcodes outside of control * methods (the so-called module-level code.) This support was explicitly * removed in ACPI 2.0, but this type of code continues to be created by * BIOS vendors. In order to support the disassembly and recompilation of * such code (and the porting of ASL code to iASL), iASL supports this * code in violation of the current ACPI specification. * * The grammar change to support module-level code is to revert the * {ObjectList} portion of the DefinitionBlockTerm in ACPI 2.0 to the * original use of {TermList} instead (see below.) This allows the use * of Type1 and Type2 opcodes at module level. * * 04/2016: The module-level code is now allowed in the following terms: * DeviceTerm, PowerResTerm, ProcessorTerm, ScopeTerm, ThermalZoneTerm. * The ObjectList term is obsolete and has been removed. */ DefinitionBlockTerm : PARSEOP_DEFINITION_BLOCK PARSEOP_OPEN_PAREN {$$ = TrCreateLeafOp (PARSEOP_DEFINITION_BLOCK); COMMENT_CAPTURE_OFF;} String ',' String ',' ByteConst ',' String ',' String ',' DWordConst PARSEOP_CLOSE_PAREN {TrSetOpIntegerWidth ($6,$8); TrSetOpEndLineNumber ($3); COMMENT_CAPTURE_ON;} '{' TermList '}' {$$ = TrLinkOpChildren ($3,7, $4,$6,$8,$10,$12,$14,$18);} ; DefinitionBlockList : DefinitionBlockTerm | DefinitionBlockTerm DefinitionBlockList {$$ = TrLinkPeerOps (2, $1,$2);} ; /******* Basic ASCII identifiers **************************************************/ /* Allow IO, DMA, IRQ Resource macro and FOR macro names to also be used as identifiers */ NameString : NameSeg {} | PARSEOP_NAMESTRING {$$ = TrCreateValuedLeafOp (PARSEOP_NAMESTRING, (ACPI_NATIVE_INT) $1);} | PARSEOP_IO {$$ = TrCreateValuedLeafOp (PARSEOP_NAMESTRING, (ACPI_NATIVE_INT) "IO");} | PARSEOP_DMA {$$ = TrCreateValuedLeafOp (PARSEOP_NAMESTRING, (ACPI_NATIVE_INT) "DMA");} | PARSEOP_IRQ {$$ = TrCreateValuedLeafOp (PARSEOP_NAMESTRING, (ACPI_NATIVE_INT) "IRQ");} | PARSEOP_FOR {$$ = TrCreateValuedLeafOp (PARSEOP_NAMESTRING, (ACPI_NATIVE_INT) "FOR");} ; /* NameSeg : PARSEOP_NAMESEG {$$ = TrCreateValuedLeafOp (PARSEOP_NAMESEG, (ACPI_NATIVE_INT) TrNormalizeNameSeg ($1));} ; */ NameSeg : PARSEOP_NAMESEG {$$ = TrCreateValuedLeafOp (PARSEOP_NAMESEG, (ACPI_NATIVE_INT) AslCompilerlval.s);} ; /******* Fundamental argument/statement types ***********************************/ Term : Object {} | Type1Opcode {} | Type2Opcode {} | Type2IntegerOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | Type2StringOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | Type2BufferOpcode {} | Type2BufferOrStringOpcode {} | error {$$ = AslDoError(); yyclearin;} ; SuperName : SimpleName {} | DebugTerm {} | Type6Opcode {} ; Target : {$$ = TrCreateNullTargetOp ();} /* Placeholder is a ZeroOp object */ | ',' {$$ = TrCreateNullTargetOp ();} /* Placeholder is a ZeroOp object */ | ',' SuperName {$$ = TrSetOpFlags ($2, OP_IS_TARGET);} ; RequiredTarget : ',' SuperName {$$ = TrSetOpFlags ($2, OP_IS_TARGET);} ; TermArg : SimpleName {$$ = TrSetOpFlags ($1, OP_IS_TERM_ARG);} | Type2Opcode {$$ = TrSetOpFlags ($1, OP_IS_TERM_ARG);} | DataObject {$$ = TrSetOpFlags ($1, OP_IS_TERM_ARG);} | PARSEOP_OPEN_PAREN TermArg PARSEOP_CLOSE_PAREN {$$ = TrSetOpFlags ($2, OP_IS_TERM_ARG);} ; /* NOTE: Removed from TermArg due to reduce/reduce conflicts: | Type2IntegerOpcode {$$ = TrSetOpFlags ($1, OP_IS_TERM_ARG);} | Type2StringOpcode {$$ = TrSetOpFlags ($1, OP_IS_TERM_ARG);} | Type2BufferOpcode {$$ = TrSetOpFlags ($1, OP_IS_TERM_ARG);} | Type2BufferOrStringOpcode {$$ = TrSetOpFlags ($1, OP_IS_TERM_ARG);} */ MethodInvocationTerm : NameString PARSEOP_OPEN_PAREN {TrSetOpIntegerValue (PARSEOP_METHODCALL, $1); COMMENT_CAPTURE_OFF;} ArgList PARSEOP_CLOSE_PAREN {$$ = TrLinkChildOp ($1,$4); COMMENT_CAPTURE_ON;} ; /* OptionalCount must appear before ByteList or an incorrect reduction will result */ OptionalCount : {$$ = TrCreateLeafOp (PARSEOP_ONES);} /* Placeholder is a OnesOp object */ | ',' {$$ = TrCreateLeafOp (PARSEOP_ONES);} /* Placeholder is a OnesOp object */ | ',' TermArg {$$ = $2;} ; /* * Data count for buffers and packages (byte count for buffers, * element count for packages). */ OptionalDataCount /* Legacy ASL */ : {$$ = NULL;} | PARSEOP_OPEN_PAREN TermArg PARSEOP_CLOSE_PAREN {$$ = $2;} | PARSEOP_OPEN_PAREN PARSEOP_CLOSE_PAREN {$$ = NULL;} /* C-style (ASL+) -- adds equals term */ | PARSEOP_EXP_EQUALS {$$ = NULL;} | PARSEOP_OPEN_PAREN TermArg PARSEOP_CLOSE_PAREN PARSEOP_EXP_EQUALS {$$ = $2;} | PARSEOP_OPEN_PAREN PARSEOP_CLOSE_PAREN String PARSEOP_EXP_EQUALS {$$ = NULL;} ; /******* List Terms **************************************************/ /* ACPI 3.0 -- allow semicolons between terms */ TermList : {$$ = NULL;} | TermList Term {$$ = TrLinkPeerOp ( TrSetOpFlags ($1, OP_RESULT_NOT_USED),$2);} | TermList Term ';' {$$ = TrLinkPeerOp ( TrSetOpFlags ($1, OP_RESULT_NOT_USED),$2);} | TermList ';' Term {$$ = TrLinkPeerOp ( TrSetOpFlags ($1, OP_RESULT_NOT_USED),$3);} | TermList ';' Term ';' {$$ = TrLinkPeerOp ( TrSetOpFlags ($1, OP_RESULT_NOT_USED),$3);} ; ArgList : {$$ = NULL;} | TermArg | ArgList ',' /* Allows a trailing comma at list end */ | ArgList ',' TermArg {$$ = TrLinkPeerOp ($1,$3);} ; ByteList : {$$ = NULL;} | ByteConstExpr | ByteList ',' /* Allows a trailing comma at list end */ | ByteList ',' ByteConstExpr {$$ = TrLinkPeerOp ($1,$3);} ; DWordList : {$$ = NULL;} | DWordConstExpr | DWordList ',' /* Allows a trailing comma at list end */ | DWordList ',' DWordConstExpr {$$ = TrLinkPeerOp ($1,$3);} ; FieldUnitList : {$$ = NULL;} | FieldUnit | FieldUnitList ',' /* Allows a trailing comma at list end */ | FieldUnitList ',' FieldUnit {$$ = TrLinkPeerOp ($1,$3);} ; FieldUnit : FieldUnitEntry {} | OffsetTerm {} | AccessAsTerm {} | ConnectionTerm {} ; FieldUnitEntry : ',' AmlPackageLengthTerm {$$ = TrCreateOp (PARSEOP_RESERVED_BYTES,1,$2);} | NameSeg ',' AmlPackageLengthTerm {$$ = TrLinkChildOp ($1,$3);} ; Object : CompilerDirective {} | NamedObject {} | NameSpaceModifier {} /* | StructureTerm {} */ ; PackageList : {$$ = NULL;} | PackageElement | PackageList ',' /* Allows a trailing comma at list end */ | PackageList ',' PackageElement {$$ = TrLinkPeerOp ($1,$3);} ; PackageElement : DataObject {} | NameString {} ; /* Rules for specifying the type of one method argument or return value */ ParameterTypePackage : {$$ = NULL;} | ObjectTypeKeyword {$$ = $1;} | ParameterTypePackage ',' ObjectTypeKeyword {$$ = TrLinkPeerOps (2,$1,$3);} ; ParameterTypePackageList : {$$ = NULL;} - | ObjectTypeKeyword {$$ = $1;} - | '{' ParameterTypePackage '}' {$$ = $2;} + | ObjectTypeKeyword {$$ = TrLinkOpChildren ( + TrCreateLeafOp (PARSEOP_DEFAULT_ARG),1,$1);} + | '{' ParameterTypePackage '}' {$$ = TrLinkOpChildren ( + TrCreateLeafOp (PARSEOP_DEFAULT_ARG),1,$2);} ; + OptionalParameterTypePackage - : {$$ = TrCreateLeafOp (PARSEOP_DEFAULT_ARG);} - | ',' ParameterTypePackageList {$$ = TrLinkOpChildren ( - TrCreateLeafOp (PARSEOP_DEFAULT_ARG),1,$2);} + : {$$ = NULL;} + | ',' ParameterTypePackageList {$$ = $2;} ; /* Rules for specifying the types for method arguments */ ParameterTypesPackage : ParameterTypePackageList {$$ = $1;} | ParameterTypesPackage ',' ParameterTypePackageList {$$ = TrLinkPeerOps (2,$1,$3);} ; ParameterTypesPackageList : {$$ = NULL;} - | ObjectTypeKeyword {$$ = $1;} - | '{' ParameterTypesPackage '}' {$$ = $2;} + | ObjectTypeKeyword {$$ = TrLinkOpChildren ( + TrCreateLeafOp (PARSEOP_DEFAULT_ARG),1,$1);} + | '{' ParameterTypesPackage '}' {$$ = TrLinkOpChildren ( + TrCreateLeafOp (PARSEOP_DEFAULT_ARG),1,$2);} ; OptionalParameterTypesPackage - : {$$ = TrCreateLeafOp (PARSEOP_DEFAULT_ARG);} - | ',' ParameterTypesPackageList {$$ = TrLinkOpChildren ( - TrCreateLeafOp (PARSEOP_DEFAULT_ARG),1,$2);} + : {$$ = NULL;} + | ',' ParameterTypesPackageList {$$ = $2;} ; /* * Case-Default list; allow only one Default term and unlimited Case terms */ CaseDefaultTermList : {$$ = NULL;} | CaseTerm {} | DefaultTerm {} | CaseDefaultTermList CaseTerm {$$ = TrLinkPeerOp ($1,$2);} | CaseDefaultTermList DefaultTerm {$$ = TrLinkPeerOp ($1,$2);} /* Original - attempts to force zero or one default term within the switch */ /* CaseDefaultTermList : {$$ = NULL;} | CaseTermList DefaultTerm CaseTermList {$$ = TrLinkPeerOp ($1,TrLinkPeerOp ($2, $3));} | CaseTermList CaseTerm {$$ = TrLinkPeerOp ($1,$2);} ; CaseTermList : {$$ = NULL;} | CaseTerm {} | CaseTermList CaseTerm {$$ = TrLinkPeerOp ($1,$2);} ; */ /******************************************************************************* * * ASL Data and Constant Terms * ******************************************************************************/ DataObject : BufferData {} | PackageData {} | IntegerData {} | StringData {} ; BufferData : Type5Opcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | Type2BufferOrStringOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | Type2BufferOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | BufferTerm {} ; PackageData : PackageTerm {} ; IntegerData : Type2IntegerOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | Type3Opcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | Integer {} | ConstTerm {} ; StringData : Type2StringOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | String {} ; ByteConst : Integer {$$ = TrSetOpIntegerValue (PARSEOP_BYTECONST, $1);} ; WordConst : Integer {$$ = TrSetOpIntegerValue (PARSEOP_WORDCONST, $1);} ; DWordConst : Integer {$$ = TrSetOpIntegerValue (PARSEOP_DWORDCONST, $1);} ; QWordConst : Integer {$$ = TrSetOpIntegerValue (PARSEOP_QWORDCONST, $1);} ; /* * The OP_COMPILE_TIME_CONST flag in the following constant expressions * enables compile-time constant folding to reduce the Type3Opcodes/Type2IntegerOpcodes * to simple integers. It is an error if these types of expressions cannot be * reduced, since the AML grammar for ****ConstExpr requires a simple constant. * Note: The required byte length of the constant is passed through to the * constant folding code in the node AmlLength field. */ ByteConstExpr : Type3Opcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST); TrSetOpAmlLength ($1, 1);} | Type2IntegerOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST); TrSetOpAmlLength ($1, 1);} | ConstExprTerm {$$ = TrSetOpIntegerValue (PARSEOP_BYTECONST, $1);} | ByteConst {} ; WordConstExpr : Type3Opcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST); TrSetOpAmlLength ($1, 2);} | Type2IntegerOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST); TrSetOpAmlLength ($1, 2);} | ConstExprTerm {$$ = TrSetOpIntegerValue (PARSEOP_WORDCONST, $1);} | WordConst {} ; DWordConstExpr : Type3Opcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST); TrSetOpAmlLength ($1, 4);} | Type2IntegerOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST); TrSetOpAmlLength ($1, 4);} | ConstExprTerm {$$ = TrSetOpIntegerValue (PARSEOP_DWORDCONST, $1);} | DWordConst {} ; QWordConstExpr : Type3Opcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST); TrSetOpAmlLength ($1, 8);} | Type2IntegerOpcode {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST); TrSetOpAmlLength ($1, 8);} | ConstExprTerm {$$ = TrSetOpIntegerValue (PARSEOP_QWORDCONST, $1);} | QWordConst {} ; ConstTerm : ConstExprTerm {} | PARSEOP_REVISION {$$ = TrCreateLeafOp (PARSEOP_REVISION);} ; ConstExprTerm : PARSEOP_ZERO {$$ = TrCreateValuedLeafOp (PARSEOP_ZERO, 0);} | PARSEOP_ONE {$$ = TrCreateValuedLeafOp (PARSEOP_ONE, 1);} | PARSEOP_ONES {$$ = TrCreateValuedLeafOp (PARSEOP_ONES, ACPI_UINT64_MAX);} | PARSEOP___DATE__ {$$ = TrCreateConstantLeafOp (PARSEOP___DATE__);} | PARSEOP___FILE__ {$$ = TrCreateConstantLeafOp (PARSEOP___FILE__);} | PARSEOP___LINE__ {$$ = TrCreateConstantLeafOp (PARSEOP___LINE__);} | PARSEOP___PATH__ {$$ = TrCreateConstantLeafOp (PARSEOP___PATH__);} | PARSEOP___METHOD__ {$$ = TrCreateConstantLeafOp (PARSEOP___METHOD__);} ; Integer : PARSEOP_INTEGER {$$ = TrCreateValuedLeafOp (PARSEOP_INTEGER, AslCompilerlval.i);} ; String : PARSEOP_STRING_LITERAL {$$ = TrCreateValuedLeafOp (PARSEOP_STRING_LITERAL, (ACPI_NATIVE_INT) AslCompilerlval.s);} ; /******************************************************************************* * * ASL Opcode Terms * ******************************************************************************/ CompilerDirective : IncludeTerm {} | IncludeEndTerm {} | ExternalTerm {} ; NamedObject : BankFieldTerm {} | CreateBitFieldTerm {} | CreateByteFieldTerm {} | CreateDWordFieldTerm {} | CreateFieldTerm {} | CreateQWordFieldTerm {} | CreateWordFieldTerm {} | DataRegionTerm {} | DeviceTerm {} | EventTerm {} | FieldTerm {} | FunctionTerm {} | IndexFieldTerm {} | MethodTerm {} | MutexTerm {} | OpRegionTerm {} | PowerResTerm {} | ProcessorTerm {} | ThermalZoneTerm {} ; NameSpaceModifier : AliasTerm {} | NameTerm {} /* | NameTermAslPlus {} */ | ScopeTerm {} ; SimpleName : NameString {} | LocalTerm {} | ArgTerm {} ; /* For ObjectType(), SuperName except for MethodInvocationTerm */ ObjectTypeSource : SimpleName {} | DebugTerm {} | RefOfTerm {} | DerefOfTerm {} | IndexTerm {} | IndexExpTerm {} ; /* For DeRefOf(), SuperName except for DerefOf and Debug */ DerefOfSource : SimpleName {} | RefOfTerm {} | DerefOfTerm {} | IndexTerm {} | IndexExpTerm {} | StoreTerm {} | EqualsTerm {} | MethodInvocationTerm {} ; /* For RefOf(), SuperName except for RefOf and MethodInvocationTerm */ RefOfSource : SimpleName {} | DebugTerm {} | DerefOfTerm {} | IndexTerm {} | IndexExpTerm {} ; /* For CondRefOf(), SuperName except for RefOf and MethodInvocationTerm */ CondRefOfSource : SimpleName {} | DebugTerm {} | DerefOfTerm {} | IndexTerm {} | IndexExpTerm {} ; /* * Opcode types, as defined in the ACPI specification */ Type1Opcode : BreakTerm {} | BreakPointTerm {} | ContinueTerm {} | FatalTerm {} | ForTerm {} | ElseIfTerm {} | LoadTerm {} | NoOpTerm {} | NotifyTerm {} | ReleaseTerm {} | ResetTerm {} | ReturnTerm {} | SignalTerm {} | SleepTerm {} | StallTerm {} | SwitchTerm {} | UnloadTerm {} | WhileTerm {} ; Type2Opcode : AcquireTerm {} | CondRefOfTerm {} | CopyObjectTerm {} | DerefOfTerm {} | ObjectTypeTerm {} | RefOfTerm {} | SizeOfTerm {} | StoreTerm {} | EqualsTerm {} | TimerTerm {} | WaitTerm {} | MethodInvocationTerm {} ; /* * Type 3/4/5 opcodes */ Type2IntegerOpcode /* "Type3" opcodes */ : Expression {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | AddTerm {} | AndTerm {} | DecTerm {} | DivideTerm {} | FindSetLeftBitTerm {} | FindSetRightBitTerm {} | FromBCDTerm {} | IncTerm {} | IndexTerm {} /* | StructureIndexTerm {} */ /* | StructurePointerTerm {} */ | LAndTerm {} | LEqualTerm {} | LGreaterTerm {} | LGreaterEqualTerm {} | LLessTerm {} | LLessEqualTerm {} | LNotTerm {} | LNotEqualTerm {} | LoadTableTerm {} | LOrTerm {} | MatchTerm {} | ModTerm {} | MultiplyTerm {} | NAndTerm {} | NOrTerm {} | NotTerm {} | OrTerm {} | ShiftLeftTerm {} | ShiftRightTerm {} | SubtractTerm {} | ToBCDTerm {} | ToIntegerTerm {} | XOrTerm {} ; Type2StringOpcode /* "Type4" Opcodes */ : ToDecimalStringTerm {} | ToHexStringTerm {} | ToStringTerm {} ; Type2BufferOpcode /* "Type5" Opcodes */ : ToBufferTerm {} | ConcatResTerm {} ; Type2BufferOrStringOpcode : ConcatTerm {$$ = TrSetOpFlags ($1, OP_COMPILE_TIME_CONST);} | PrintfTerm {} | FprintfTerm {} | MidTerm {} ; /* * A type 3 opcode evaluates to an Integer and cannot have a destination operand */ Type3Opcode : EISAIDTerm {} ; /* Obsolete Type4Opcode : ConcatTerm {} | ToDecimalStringTerm {} | ToHexStringTerm {} | MidTerm {} | ToStringTerm {} ; */ /* Type 5 opcodes are a subset of Type2 opcodes, and return a constant */ Type5Opcode : ResourceTemplateTerm {} | UnicodeTerm {} | ToPLDTerm {} | ToUUIDTerm {} ; Type6Opcode : RefOfTerm {} | DerefOfTerm {} | IndexTerm {} | IndexExpTerm {} /* | StructureIndexTerm {} */ /* | StructurePointerTerm {} */ | MethodInvocationTerm {} ; /******************************************************************************* * * ASL Helper Terms * ******************************************************************************/ AmlPackageLengthTerm : Integer {$$ = TrSetOpIntegerValue (PARSEOP_PACKAGE_LENGTH, (ACPI_PARSE_OBJECT *) $1);} ; NameStringItem : ',' NameString {$$ = $2;} | ',' error {$$ = AslDoError (); yyclearin;} ; TermArgItem : ',' TermArg {$$ = $2;} | ',' error {$$ = AslDoError (); yyclearin;} ; OptionalReference : {$$ = TrCreateLeafOp (PARSEOP_ZERO);} /* Placeholder is a ZeroOp object */ | ',' {$$ = TrCreateLeafOp (PARSEOP_ZERO);} /* Placeholder is a ZeroOp object */ | ',' TermArg {$$ = $2;} ; OptionalReturnArg : {$$ = TrSetOpFlags (TrCreateLeafOp (PARSEOP_ZERO), OP_IS_NULL_RETURN);} /* Placeholder is a ZeroOp object */ | TermArg {$$ = $1;} ; OptionalSerializeRuleKeyword : {$$ = NULL;} | ',' {$$ = NULL;} | ',' SerializeRuleKeyword {$$ = $2;} ; OptionalTermArg : {$$ = TrCreateLeafOp (PARSEOP_DEFAULT_ARG);} | TermArg {$$ = $1;} ; OptionalWordConst : {$$ = NULL;} | WordConst {$$ = $1;} ; Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/asltransform.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/asltransform.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/asltransform.c (revision 357966) @@ -1,1242 +1,1316 @@ /****************************************************************************** * * Module Name: asltransform - Parse tree transforms * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include "aslcompiler.y.h" #include #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("asltransform") /* Local prototypes */ static void TrTransformSubtree ( ACPI_PARSE_OBJECT *Op); static char * TrAmlGetNextTempName ( ACPI_PARSE_OBJECT *Op, UINT8 *TempCount); static void TrAmlInitLineNumbers ( ACPI_PARSE_OBJECT *Op, ACPI_PARSE_OBJECT *Neighbor); static void TrAmlInitNode ( ACPI_PARSE_OBJECT *Op, UINT16 ParseOpcode); static void TrAmlSetSubtreeParent ( ACPI_PARSE_OBJECT *Op, ACPI_PARSE_OBJECT *Parent); static void TrAmlInsertPeer ( ACPI_PARSE_OBJECT *Op, ACPI_PARSE_OBJECT *NewPeer); static void TrDoDefinitionBlock ( ACPI_PARSE_OBJECT *Op); static void TrDoSwitch ( ACPI_PARSE_OBJECT *StartNode); static void TrCheckForDuplicateCase ( ACPI_PARSE_OBJECT *CaseOp, ACPI_PARSE_OBJECT *Predicate1); static BOOLEAN TrCheckForBufferMatch ( ACPI_PARSE_OBJECT *Next1, ACPI_PARSE_OBJECT *Next2); +static void +TrDoMethod ( + ACPI_PARSE_OBJECT *Op); + /******************************************************************************* * * FUNCTION: TrAmlGetNextTempName * * PARAMETERS: Op - Current parse op * TempCount - Current temporary counter. Was originally * per-module; Currently per method, could be * expanded to per-scope. * * RETURN: A pointer to name (allocated here). * * DESCRIPTION: Generate an ACPI name of the form _T_x. These names are * reserved for use by the ASL compiler. (_T_0 through _T_Z) * ******************************************************************************/ static char * TrAmlGetNextTempName ( ACPI_PARSE_OBJECT *Op, UINT8 *TempCount) { char *TempName; if (*TempCount >= (10 + 26)) /* 0-35 valid: 0-9 and A-Z for TempName[3] */ { /* Too many temps */ AslError (ASL_ERROR, ASL_MSG_TOO_MANY_TEMPS, Op, NULL); return (NULL); } TempName = UtLocalCalloc (5); if (*TempCount < 10) /* 0-9 */ { TempName[3] = (char) (*TempCount + '0'); } else /* 10-35: A-Z */ { TempName[3] = (char) (*TempCount + ('A' - 10)); } (*TempCount)++; /* First three characters are always "_T_" */ TempName[0] = '_'; TempName[1] = 'T'; TempName[2] = '_'; return (TempName); } /******************************************************************************* * * FUNCTION: TrAmlInitLineNumbers * * PARAMETERS: Op - Op to be initialized * Neighbor - Op used for initialization values * * RETURN: None * * DESCRIPTION: Initialized the various line numbers for a parse node. * ******************************************************************************/ static void TrAmlInitLineNumbers ( ACPI_PARSE_OBJECT *Op, ACPI_PARSE_OBJECT *Neighbor) { Op->Asl.EndLine = Neighbor->Asl.EndLine; Op->Asl.EndLogicalLine = Neighbor->Asl.EndLogicalLine; Op->Asl.LineNumber = Neighbor->Asl.LineNumber; Op->Asl.LogicalByteOffset = Neighbor->Asl.LogicalByteOffset; Op->Asl.LogicalLineNumber = Neighbor->Asl.LogicalLineNumber; } /******************************************************************************* * * FUNCTION: TrAmlInitNode * * PARAMETERS: Op - Op to be initialized * ParseOpcode - Opcode for this node * * RETURN: None * * DESCRIPTION: Initialize a node with the parse opcode and opcode name. * ******************************************************************************/ static void TrAmlInitNode ( ACPI_PARSE_OBJECT *Op, UINT16 ParseOpcode) { Op->Asl.ParseOpcode = ParseOpcode; UtSetParseOpName (Op); } /******************************************************************************* * * FUNCTION: TrAmlSetSubtreeParent * * PARAMETERS: Op - First node in a list of peer nodes * Parent - Parent of the subtree * * RETURN: None * * DESCRIPTION: Set the parent for all peer nodes in a subtree * ******************************************************************************/ static void TrAmlSetSubtreeParent ( ACPI_PARSE_OBJECT *Op, ACPI_PARSE_OBJECT *Parent) { ACPI_PARSE_OBJECT *Next; Next = Op; while (Next) { Next->Asl.Parent = Parent; Next = Next->Asl.Next; } } /******************************************************************************* * * FUNCTION: TrAmlInsertPeer * * PARAMETERS: Op - First node in a list of peer nodes * NewPeer - Peer node to insert * * RETURN: None * * DESCRIPTION: Insert a new peer node into a list of peers. * ******************************************************************************/ static void TrAmlInsertPeer ( ACPI_PARSE_OBJECT *Op, ACPI_PARSE_OBJECT *NewPeer) { NewPeer->Asl.Next = Op->Asl.Next; Op->Asl.Next = NewPeer; } /******************************************************************************* * * FUNCTION: TrAmlTransformWalkBegin * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: None * * DESCRIPTION: Parse tree walk to generate both the AML opcodes and the AML * operands. * ******************************************************************************/ ACPI_STATUS TrAmlTransformWalkBegin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { TrTransformSubtree (Op); return (AE_OK); } /******************************************************************************* * * FUNCTION: TrAmlTransformWalkEnd * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: None * * DESCRIPTION: Parse tree walk to generate both the AML opcodes and the AML * operands. * ******************************************************************************/ ACPI_STATUS TrAmlTransformWalkEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { /* Save possible Externals list in the DefintionBlock Op */ if (Op->Asl.ParseOpcode == PARSEOP_DEFINITION_BLOCK) { Op->Asl.Value.Arg = AslGbl_ExternalsListHead; AslGbl_ExternalsListHead = NULL; } return (AE_OK); } /******************************************************************************* * * FUNCTION: TrTransformSubtree * * PARAMETERS: Op - The parent parse node * * RETURN: None * * DESCRIPTION: Prepare nodes to be output as AML data and operands. The more * complex AML opcodes require processing of the child nodes * (arguments/operands). * ******************************************************************************/ static void TrTransformSubtree ( ACPI_PARSE_OBJECT *Op) { ACPI_PARSE_OBJECT *MethodOp; ACPI_NAMESTRING_INFO Info; if (Op->Asl.AmlOpcode == AML_RAW_DATA_BYTE) { return; } switch (Op->Asl.ParseOpcode) { case PARSEOP_DEFINITION_BLOCK: TrDoDefinitionBlock (Op); break; case PARSEOP_SWITCH: TrDoSwitch (Op); break; case PARSEOP_METHOD: - /* - * TBD: Zero the tempname (_T_x) count. Probably shouldn't be a global, - * however - */ - AslGbl_TempCount = 0; + + TrDoMethod (Op); break; case PARSEOP_EXTERNAL: ExDoExternal (Op); break; case PARSEOP___METHOD__: /* Transform to a string op containing the parent method name */ Op->Asl.ParseOpcode = PARSEOP_STRING_LITERAL; UtSetParseOpName (Op); /* Find the parent control method op */ MethodOp = Op; while (MethodOp) { if (MethodOp->Asl.ParseOpcode == PARSEOP_METHOD) { /* First child contains the method name */ MethodOp = MethodOp->Asl.Child; Op->Asl.Value.String = MethodOp->Asl.Value.String; return; } MethodOp = MethodOp->Asl.Parent; } /* At the root, invocation not within a control method */ Op->Asl.Value.String = "\\"; break; case PARSEOP_NAMESTRING: /* * A NameString can be up to 255 (0xFF) individual NameSegs maximum * (with 254 dot separators) - as per the ACPI specification. Note: * Cannot check for NumSegments == 0 because things like * Scope(\) are legal and OK. */ Info.ExternalName = Op->Asl.Value.String; AcpiNsGetInternalNameLength (&Info); if (Info.NumSegments > 255) { AslError (ASL_ERROR, ASL_MSG_NAMESTRING_LENGTH, Op, NULL); } break; case PARSEOP_UNLOAD: AslError (ASL_WARNING, ASL_MSG_UNLOAD, Op, NULL); break; case PARSEOP_SLEEP: /* Remark for very long sleep values */ if (Op->Asl.Child->Asl.Value.Integer > 1000) { AslError (ASL_REMARK, ASL_MSG_LONG_SLEEP, Op, NULL); } break; case PARSEOP_PROCESSOR: AslError (ASL_WARNING, ASL_MSG_LEGACY_PROCESSOR_OP, Op, Op->Asl.ExternalName); break; default: /* Nothing to do here for other opcodes */ break; } } /******************************************************************************* * * FUNCTION: TrDoDefinitionBlock * * PARAMETERS: Op - Parse node * * RETURN: None * * DESCRIPTION: Find the end of the definition block and set a global to this * node. It is used by the compiler to insert compiler-generated * names at the root level of the namespace. * ******************************************************************************/ static void TrDoDefinitionBlock ( ACPI_PARSE_OBJECT *Op) { ACPI_PARSE_OBJECT *Next; UINT32 i; /* Reset external list when starting a definition block */ AslGbl_ExternalsListHead = NULL; Next = Op->Asl.Child; for (i = 0; i < 5; i++) { Next = Next->Asl.Next; if (i == 0) { /* * This is the table signature. Only the DSDT can be assumed * to be at the root of the namespace; Therefore, namepath * optimization can only be performed on the DSDT. */ if (!ACPI_COMPARE_NAMESEG (Next->Asl.Value.String, ACPI_SIG_DSDT)) { AslGbl_ReferenceOptimizationFlag = FALSE; } } } AslGbl_FirstLevelInsertionNode = Next; } /******************************************************************************* * * FUNCTION: TrDoSwitch * * PARAMETERS: StartNode - Parse node for SWITCH * * RETURN: None * * DESCRIPTION: Translate ASL SWITCH statement to if/else pairs. There is * no actual AML opcode for SWITCH -- it must be simulated. * ******************************************************************************/ static void TrDoSwitch ( ACPI_PARSE_OBJECT *StartNode) { ACPI_PARSE_OBJECT *Next; ACPI_PARSE_OBJECT *CaseOp = NULL; ACPI_PARSE_OBJECT *CaseBlock = NULL; ACPI_PARSE_OBJECT *DefaultOp = NULL; ACPI_PARSE_OBJECT *CurrentParentNode; ACPI_PARSE_OBJECT *Conditional = NULL; ACPI_PARSE_OBJECT *Predicate; ACPI_PARSE_OBJECT *Peer; ACPI_PARSE_OBJECT *NewOp; ACPI_PARSE_OBJECT *NewOp2; ACPI_PARSE_OBJECT *MethodOp; ACPI_PARSE_OBJECT *StoreOp; ACPI_PARSE_OBJECT *BreakOp; ACPI_PARSE_OBJECT *BufferOp; char *PredicateValueName; UINT16 Index; UINT32 Btype; /* Start node is the Switch() node */ CurrentParentNode = StartNode; /* Create a new temp name of the form _T_x */ PredicateValueName = TrAmlGetNextTempName (StartNode, &AslGbl_TempCount); if (!PredicateValueName) { return; } /* First child is the Switch() predicate */ Next = StartNode->Asl.Child; /* * Examine the return type of the Switch Value - * must be Integer/Buffer/String */ Index = (UINT16) (Next->Asl.ParseOpcode - ASL_PARSE_OPCODE_BASE); Btype = AslKeywordMapping[Index].AcpiBtype; if ((Btype != ACPI_BTYPE_INTEGER) && (Btype != ACPI_BTYPE_STRING) && (Btype != ACPI_BTYPE_BUFFER)) { AslError (ASL_WARNING, ASL_MSG_SWITCH_TYPE, Next, NULL); Btype = ACPI_BTYPE_INTEGER; } /* CASE statements start at next child */ Peer = Next->Asl.Next; while (Peer) { Next = Peer; Peer = Next->Asl.Next; if (Next->Asl.ParseOpcode == PARSEOP_CASE) { TrCheckForDuplicateCase (Next, Next->Asl.Child); if (CaseOp) { /* Add an ELSE to complete the previous CASE */ NewOp = TrCreateLeafOp (PARSEOP_ELSE); NewOp->Asl.Parent = Conditional->Asl.Parent; TrAmlInitLineNumbers (NewOp, NewOp->Asl.Parent); /* Link ELSE node as a peer to the previous IF */ TrAmlInsertPeer (Conditional, NewOp); CurrentParentNode = NewOp; } CaseOp = Next; Conditional = CaseOp; CaseBlock = CaseOp->Asl.Child->Asl.Next; Conditional->Asl.Child->Asl.Next = NULL; Predicate = CaseOp->Asl.Child; if ((Predicate->Asl.ParseOpcode == PARSEOP_PACKAGE) || (Predicate->Asl.ParseOpcode == PARSEOP_VAR_PACKAGE)) { /* * Convert the package declaration to this form: * * If (LNotEqual (Match (Package(){}, * MEQ, _T_x, MTR, Zero, Zero), Ones)) */ NewOp2 = TrCreateLeafOp (PARSEOP_MATCHTYPE_MEQ); Predicate->Asl.Next = NewOp2; TrAmlInitLineNumbers (NewOp2, Conditional); NewOp = NewOp2; NewOp2 = TrCreateValuedLeafOp (PARSEOP_NAMESTRING, (UINT64) ACPI_TO_INTEGER (PredicateValueName)); NewOp->Asl.Next = NewOp2; TrAmlInitLineNumbers (NewOp2, Predicate); NewOp = NewOp2; NewOp2 = TrCreateLeafOp (PARSEOP_MATCHTYPE_MTR); NewOp->Asl.Next = NewOp2; TrAmlInitLineNumbers (NewOp2, Predicate); NewOp = NewOp2; NewOp2 = TrCreateLeafOp (PARSEOP_ZERO); NewOp->Asl.Next = NewOp2; TrAmlInitLineNumbers (NewOp2, Predicate); NewOp = NewOp2; NewOp2 = TrCreateLeafOp (PARSEOP_ZERO); NewOp->Asl.Next = NewOp2; TrAmlInitLineNumbers (NewOp2, Predicate); NewOp2 = TrCreateLeafOp (PARSEOP_MATCH); NewOp2->Asl.Child = Predicate; /* PARSEOP_PACKAGE */ TrAmlInitLineNumbers (NewOp2, Conditional); TrAmlSetSubtreeParent (Predicate, NewOp2); NewOp = NewOp2; NewOp2 = TrCreateLeafOp (PARSEOP_ONES); NewOp->Asl.Next = NewOp2; TrAmlInitLineNumbers (NewOp2, Conditional); NewOp2 = TrCreateLeafOp (PARSEOP_LEQUAL); NewOp2->Asl.Child = NewOp; NewOp->Asl.Parent = NewOp2; TrAmlInitLineNumbers (NewOp2, Conditional); TrAmlSetSubtreeParent (NewOp, NewOp2); NewOp = NewOp2; NewOp2 = TrCreateLeafOp (PARSEOP_LNOT); NewOp2->Asl.Child = NewOp; NewOp2->Asl.Parent = Conditional; NewOp->Asl.Parent = NewOp2; TrAmlInitLineNumbers (NewOp2, Conditional); Conditional->Asl.Child = NewOp2; NewOp2->Asl.Next = CaseBlock; } else { /* * Integer and Buffer case. * * Change CaseOp() to: If (LEqual (SwitchValue, CaseValue)) {...} * Note: SwitchValue is first to allow the CaseValue to be implicitly * converted to the type of SwitchValue if necessary. * * CaseOp->Child is the case value * CaseOp->Child->Peer is the beginning of the case block */ NewOp = TrCreateValuedLeafOp (PARSEOP_NAMESTRING, (UINT64) ACPI_TO_INTEGER (PredicateValueName)); NewOp->Asl.Next = Predicate; TrAmlInitLineNumbers (NewOp, Predicate); NewOp2 = TrCreateLeafOp (PARSEOP_LEQUAL); NewOp2->Asl.Parent = Conditional; NewOp2->Asl.Child = NewOp; TrAmlInitLineNumbers (NewOp2, Conditional); TrAmlSetSubtreeParent (NewOp, NewOp2); Predicate = NewOp2; Predicate->Asl.Next = CaseBlock; TrAmlSetSubtreeParent (Predicate, Conditional); Conditional->Asl.Child = Predicate; } /* Reinitialize the CASE node to an IF node */ TrAmlInitNode (Conditional, PARSEOP_IF); /* * The first CASE(IF) is not nested under an ELSE. * All other CASEs are children of a parent ELSE. */ if (CurrentParentNode == StartNode) { Conditional->Asl.Next = NULL; } else { /* * The IF is a child of previous IF/ELSE. It * is therefore without peer. */ CurrentParentNode->Asl.Child = Conditional; Conditional->Asl.Parent = CurrentParentNode; Conditional->Asl.Next = NULL; } } else if (Next->Asl.ParseOpcode == PARSEOP_DEFAULT) { if (DefaultOp) { /* * More than one Default * (Parser does not catch this, must check here) */ AslError (ASL_ERROR, ASL_MSG_MULTIPLE_DEFAULT, Next, NULL); } else { /* Save the DEFAULT node for later, after CASEs */ DefaultOp = Next; } } else { /* Unknown peer opcode */ AcpiOsPrintf ("Unknown parse opcode for switch statement: %s (%u)\n", Next->Asl.ParseOpName, Next->Asl.ParseOpcode); } } /* Add the default case at the end of the if/else construct */ if (DefaultOp) { /* If no CASE statements, this is an error - see below */ if (CaseOp) { /* Convert the DEFAULT node to an ELSE */ TrAmlInitNode (DefaultOp, PARSEOP_ELSE); DefaultOp->Asl.Parent = Conditional->Asl.Parent; /* Link ELSE node as a peer to the previous IF */ TrAmlInsertPeer (Conditional, DefaultOp); } } if (!CaseOp) { AslError (ASL_ERROR, ASL_MSG_NO_CASES, StartNode, NULL); } /* * Create a Name(_T_x, ...) statement. This statement must appear at the * method level, in case a loop surrounds the switch statement and could * cause the name to be created twice (error). */ /* Create the Name node */ Predicate = StartNode->Asl.Child; NewOp = TrCreateLeafOp (PARSEOP_NAME); TrAmlInitLineNumbers (NewOp, StartNode); /* Find the parent method */ Next = StartNode; while ((Next->Asl.ParseOpcode != PARSEOP_METHOD) && (Next->Asl.ParseOpcode != PARSEOP_DEFINITION_BLOCK)) { Next = Next->Asl.Parent; } MethodOp = Next; NewOp->Asl.CompileFlags |= OP_COMPILER_EMITTED; NewOp->Asl.Parent = Next; /* Insert name after the method name and arguments */ Next = Next->Asl.Child; /* Name */ Next = Next->Asl.Next; /* NumArgs */ Next = Next->Asl.Next; /* SerializeRule */ /* * If method is not Serialized, we must make is so, because of the way * that Switch() must be implemented -- we cannot allow multiple threads * to execute this method concurrently since we need to create local * temporary name(s). */ if (Next->Asl.ParseOpcode != PARSEOP_SERIALIZERULE_SERIAL) { AslError (ASL_REMARK, ASL_MSG_SERIALIZED, MethodOp, "Due to use of Switch operator"); Next->Asl.ParseOpcode = PARSEOP_SERIALIZERULE_SERIAL; } Next = Next->Asl.Next; /* SyncLevel */ Next = Next->Asl.Next; /* ReturnType */ Next = Next->Asl.Next; /* ParameterTypes */ TrAmlInsertPeer (Next, NewOp); TrAmlInitLineNumbers (NewOp, Next); /* Create the NameSeg child for the Name node */ NewOp2 = TrCreateValuedLeafOp (PARSEOP_NAMESEG, (UINT64) ACPI_TO_INTEGER (PredicateValueName)); TrAmlInitLineNumbers (NewOp2, NewOp); NewOp2->Asl.CompileFlags |= OP_IS_NAME_DECLARATION; NewOp->Asl.Child = NewOp2; /* Create the initial value for the Name. Btype was already validated above */ switch (Btype) { case ACPI_BTYPE_INTEGER: NewOp2->Asl.Next = TrCreateValuedLeafOp (PARSEOP_ZERO, (UINT64) 0); TrAmlInitLineNumbers (NewOp2->Asl.Next, NewOp); break; case ACPI_BTYPE_STRING: NewOp2->Asl.Next = TrCreateValuedLeafOp (PARSEOP_STRING_LITERAL, (UINT64) ACPI_TO_INTEGER ("")); TrAmlInitLineNumbers (NewOp2->Asl.Next, NewOp); break; case ACPI_BTYPE_BUFFER: (void) TrLinkPeerOp (NewOp2, TrCreateValuedLeafOp (PARSEOP_BUFFER, (UINT64) 0)); Next = NewOp2->Asl.Next; TrAmlInitLineNumbers (Next, NewOp2); (void) TrLinkOpChildren (Next, 1, TrCreateValuedLeafOp (PARSEOP_ZERO, (UINT64) 1)); TrAmlInitLineNumbers (Next->Asl.Child, Next); BufferOp = TrCreateValuedLeafOp (PARSEOP_DEFAULT_ARG, (UINT64) 0); TrAmlInitLineNumbers (BufferOp, Next->Asl.Child); (void) TrLinkPeerOp (Next->Asl.Child, BufferOp); TrAmlSetSubtreeParent (Next->Asl.Child, Next); break; default: break; } TrAmlSetSubtreeParent (NewOp2, NewOp); /* * Transform the Switch() into a While(One)-Break node. * And create a Store() node which will be used to save the * Switch() value. The store is of the form: Store (Value, _T_x) * where _T_x is the temp variable. */ TrAmlInitNode (StartNode, PARSEOP_WHILE); NewOp = TrCreateLeafOp (PARSEOP_ONE); TrAmlInitLineNumbers (NewOp, StartNode); NewOp->Asl.Next = Predicate->Asl.Next; NewOp->Asl.Parent = StartNode; StartNode->Asl.Child = NewOp; /* Create a Store() node */ StoreOp = TrCreateLeafOp (PARSEOP_STORE); TrAmlInitLineNumbers (StoreOp, NewOp); StoreOp->Asl.Parent = StartNode; TrAmlInsertPeer (NewOp, StoreOp); /* Complete the Store subtree */ StoreOp->Asl.Child = Predicate; Predicate->Asl.Parent = StoreOp; NewOp = TrCreateValuedLeafOp (PARSEOP_NAMESEG, (UINT64) ACPI_TO_INTEGER (PredicateValueName)); TrAmlInitLineNumbers (NewOp, StoreOp); NewOp->Asl.Parent = StoreOp; Predicate->Asl.Next = NewOp; /* Create a Break() node and insert it into the end of While() */ Conditional = StartNode->Asl.Child; while (Conditional->Asl.Next) { Conditional = Conditional->Asl.Next; } BreakOp = TrCreateLeafOp (PARSEOP_BREAK); TrAmlInitLineNumbers (BreakOp, NewOp); BreakOp->Asl.Parent = StartNode; TrAmlInsertPeer (Conditional, BreakOp); } /******************************************************************************* * * FUNCTION: TrCheckForDuplicateCase * * PARAMETERS: CaseOp - Parse node for first Case statement in list * Predicate1 - Case value for the input CaseOp * * RETURN: None * * DESCRIPTION: Check for duplicate case values. Currently, only handles * Integers, Strings and Buffers. No support for Package objects. * ******************************************************************************/ static void TrCheckForDuplicateCase ( ACPI_PARSE_OBJECT *CaseOp, ACPI_PARSE_OBJECT *Predicate1) { ACPI_PARSE_OBJECT *Next; ACPI_PARSE_OBJECT *Predicate2; /* Walk the list of CASE opcodes */ Next = CaseOp->Asl.Next; while (Next) { if (Next->Asl.ParseOpcode == PARSEOP_CASE) { /* Emit error only once */ if (Next->Asl.CompileFlags & OP_IS_DUPLICATE) { goto NextCase; } /* Check for a duplicate plain integer */ Predicate2 = Next->Asl.Child; if ((Predicate1->Asl.ParseOpcode == PARSEOP_INTEGER) && (Predicate2->Asl.ParseOpcode == PARSEOP_INTEGER)) { if (Predicate1->Asl.Value.Integer == Predicate2->Asl.Value.Integer) { goto FoundDuplicate; } } /* Check for pairs of the constants ZERO, ONE, ONES */ else if (((Predicate1->Asl.ParseOpcode == PARSEOP_ZERO) && (Predicate2->Asl.ParseOpcode == PARSEOP_ZERO)) || ((Predicate1->Asl.ParseOpcode == PARSEOP_ONE) && (Predicate2->Asl.ParseOpcode == PARSEOP_ONE)) || ((Predicate1->Asl.ParseOpcode == PARSEOP_ONES) && (Predicate2->Asl.ParseOpcode == PARSEOP_ONES))) { goto FoundDuplicate; } /* Check for a duplicate string constant (literal) */ else if ((Predicate1->Asl.ParseOpcode == PARSEOP_STRING_LITERAL) && (Predicate2->Asl.ParseOpcode == PARSEOP_STRING_LITERAL)) { if (!strcmp (Predicate1->Asl.Value.String, Predicate2->Asl.Value.String)) { goto FoundDuplicate; } } /* Check for a duplicate buffer constant */ else if ((Predicate1->Asl.ParseOpcode == PARSEOP_BUFFER) && (Predicate2->Asl.ParseOpcode == PARSEOP_BUFFER)) { if (TrCheckForBufferMatch (Predicate1->Asl.Child, Predicate2->Asl.Child)) { goto FoundDuplicate; } } } goto NextCase; FoundDuplicate: /* Emit error message only once */ Next->Asl.CompileFlags |= OP_IS_DUPLICATE; AslDualParseOpError (ASL_ERROR, ASL_MSG_DUPLICATE_CASE, Next, Next->Asl.Value.String, ASL_MSG_CASE_FOUND_HERE, CaseOp, CaseOp->Asl.ExternalName); NextCase: Next = Next->Asl.Next; } } /******************************************************************************* * * FUNCTION: TrBufferIsAllZero * * PARAMETERS: Op - Parse node for first opcode in buffer initializer * list * * RETURN: TRUE if buffer contains all zeros or a DEFAULT_ARG * * DESCRIPTION: Check for duplicate Buffer case values. * ******************************************************************************/ static BOOLEAN TrBufferIsAllZero ( ACPI_PARSE_OBJECT *Op) { while (Op) { if (Op->Asl.ParseOpcode == PARSEOP_DEFAULT_ARG) { return (TRUE); } else if (Op->Asl.Value.Integer != 0) { return (FALSE); } Op = Op->Asl.Next; } return (TRUE); } /******************************************************************************* * * FUNCTION: TrCheckForBufferMatch * * PARAMETERS: Next1 - Parse node for first opcode in first buffer list * (The DEFAULT_ARG or INTEGER node) * Next2 - Parse node for first opcode in second buffer list * (The DEFAULT_ARG or INTEGER node) * * RETURN: TRUE if buffers match, FALSE otherwise * * DESCRIPTION: Check for duplicate Buffer case values. * ******************************************************************************/ static BOOLEAN TrCheckForBufferMatch ( ACPI_PARSE_OBJECT *NextOp1, ACPI_PARSE_OBJECT *NextOp2) { /* * The buffer length can be a DEFAULT_ARG or INTEGER. If any of the nodes * are DEFAULT_ARG, it means that the length has yet to be computed. * However, the initializer list can be compared to determine if these two * buffers match. */ if ((NextOp1->Asl.ParseOpcode == PARSEOP_INTEGER && NextOp2->Asl.ParseOpcode == PARSEOP_INTEGER) && NextOp1->Asl.Value.Integer != NextOp2->Asl.Value.Integer) { return (FALSE); } /* * Buffers that have explicit lengths but no initializer lists are * filled with zeros at runtime. This is equivalent to buffers that have the * same length that are filled with zeros. * * In other words, the following buffers are equivalent: * * Buffer(0x4) {} * Buffer() {0x0, 0x0, 0x0, 0x0} * * This statement checks for matches where one buffer does not have an * initializer list and another buffer contains all zeros. */ if (NextOp1->Asl.ParseOpcode != NextOp2->Asl.ParseOpcode && TrBufferIsAllZero (NextOp1->Asl.Next) && TrBufferIsAllZero (NextOp2->Asl.Next)) { return (TRUE); } /* Start at the BYTECONST initializer node list */ NextOp1 = NextOp1->Asl.Next; NextOp2 = NextOp2->Asl.Next; /* * Walk both lists until either a mismatch is found, or one or more * end-of-lists are found */ while (NextOp1 && NextOp2) { if ((NextOp1->Asl.ParseOpcode == PARSEOP_STRING_LITERAL) && (NextOp2->Asl.ParseOpcode == PARSEOP_STRING_LITERAL)) { if (!strcmp (NextOp1->Asl.Value.String, NextOp2->Asl.Value.String)) { return (TRUE); } else { return (FALSE); } } if ((UINT8) NextOp1->Asl.Value.Integer != (UINT8) NextOp2->Asl.Value.Integer) { return (FALSE); } NextOp1 = NextOp1->Asl.Next; NextOp2 = NextOp2->Asl.Next; } /* Not a match if one of the lists is not at end-of-list */ if (NextOp1 || NextOp2) { return (FALSE); } /* Otherwise, the buffers match */ return (TRUE); +} + + +/******************************************************************************* + * + * FUNCTION: TrDoMethod + * + * PARAMETERS: Op - Parse node for SWITCH + * + * RETURN: None + * + * DESCRIPTION: Determine that parameter count of an ASL method node by + * translating the parameter count parse node from + * PARSEOP_DEFAULT_ARG to PARSEOP_BYTECONST. + * + ******************************************************************************/ + +static void +TrDoMethod ( + ACPI_PARSE_OBJECT *Op) +{ + ACPI_PARSE_OBJECT *ArgCountOp; + UINT8 ArgCount; + ACPI_PARSE_OBJECT *ParameterOp; + + + /* + * TBD: Zero the tempname (_T_x) count. Probably shouldn't be a global, + * however + */ + AslGbl_TempCount = 0; + + ArgCountOp = Op->Asl.Child->Asl.Next; + if (ArgCountOp->Asl.ParseOpcode == PARSEOP_BYTECONST) + { + /* + * Parameter count for this method has already been recorded in the + * method declaration. + */ + return; + } + + /* + * Parameter count has been omitted in the method declaration. + * Count the amount of arguments here. + */ + ParameterOp = ArgCountOp->Asl.Next->Asl.Next->Asl.Next->Asl.Next; + if (ParameterOp->Asl.ParseOpcode == PARSEOP_DEFAULT_ARG) + { + ArgCount = 0; + ParameterOp = ParameterOp->Asl.Child; + + while (ParameterOp) + { + ParameterOp = ParameterOp->Asl.Next; + ArgCount++; + } + + ArgCountOp->Asl.Value.Integer = ArgCount; + ArgCountOp->Asl.ParseOpcode = PARSEOP_BYTECONST; + } + else + { + /* + * Method parameters can be counted by analyzing the Parameter type + * list. If the Parameter list contains more than 1 parameter, it + * is nested under PARSEOP_DEFAULT_ARG. When there is only 1 + * parameter, the parse tree contains a single node representing + * that type. + */ + ArgCountOp->Asl.Value.Integer = 1; + ArgCountOp->Asl.ParseOpcode = PARSEOP_BYTECONST; + } } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslutils.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslutils.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslutils.c (revision 357966) @@ -1,1110 +1,1145 @@ /****************************************************************************** * * Module Name: aslutils -- compiler utilities * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include "aslcompiler.y.h" #include #include #include #include #include #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("aslutils") /* Local prototypes */ static void UtPadNameWithUnderscores ( char *NameSeg, char *PaddedNameSeg); static void UtAttachNameseg ( ACPI_PARSE_OBJECT *Op, char *Name); static void UtDisplayErrorSummary ( UINT32 FileId); /******************************************************************************* * * FUNCTION: UtIsBigEndianMachine * * PARAMETERS: None * * RETURN: TRUE if machine is big endian * FALSE if machine is little endian * * DESCRIPTION: Detect whether machine is little endian or big endian. * ******************************************************************************/ UINT8 UtIsBigEndianMachine ( void) { union { UINT32 Integer; UINT8 Bytes[4]; } Overlay = {0xFF000000}; return (Overlay.Bytes[0]); /* Returns 0xFF (TRUE) for big endian */ } /****************************************************************************** * * FUNCTION: UtQueryForOverwrite * * PARAMETERS: Pathname - Output filename * * RETURN: TRUE if file does not exist or overwrite is authorized * * DESCRIPTION: Query for file overwrite if it already exists. * ******************************************************************************/ BOOLEAN UtQueryForOverwrite ( char *Pathname) { struct stat StatInfo; int InChar; if (!stat (Pathname, &StatInfo)) { fprintf (stderr, "Target file \"%s\" already exists, overwrite? [y|n] ", Pathname); InChar = fgetc (stdin); if (InChar == '\n') { InChar = fgetc (stdin); } if ((InChar != 'y') && (InChar != 'Y')) { return (FALSE); } } return (TRUE); } /******************************************************************************* * * FUNCTION: UtNodeIsDescendantOf * * PARAMETERS: Node1 - Child node * Node2 - Possible parent node * * RETURN: Boolean * * DESCRIPTION: Returns TRUE if Node1 is a descendant of Node2. Otherwise, * return FALSE. Note, we assume a NULL Node2 element to be the * topmost (root) scope. All nodes are descendants of the root. * Note: Nodes at the same level (siblings) are not considered * descendants. * ******************************************************************************/ BOOLEAN UtNodeIsDescendantOf ( ACPI_NAMESPACE_NODE *Node1, ACPI_NAMESPACE_NODE *Node2) { if (Node1 == Node2) { return (FALSE); } if (!Node2) { return (TRUE); /* All nodes descend from the root */ } /* Walk upward until the root is reached or parent is found */ while (Node1) { if (Node1 == Node2) { return (TRUE); } Node1 = Node1->Parent; } return (FALSE); } /******************************************************************************* * - * FUNCTION: UtGetParentMethod + * FUNCTION: UtGetParentMethodNode * * PARAMETERS: Node - Namespace node for any object * * RETURN: Namespace node for the parent method * NULL - object is not within a method * * DESCRIPTION: Find the parent (owning) method node for a namespace object * ******************************************************************************/ -void * -UtGetParentMethod ( +ACPI_NAMESPACE_NODE * +UtGetParentMethodNode ( ACPI_NAMESPACE_NODE *Node) { ACPI_NAMESPACE_NODE *ParentNode; if (!Node) { return (NULL); } /* Walk upward until a method is found, or the root is reached */ ParentNode = Node->Parent; while (ParentNode) { if (ParentNode->Type == ACPI_TYPE_METHOD) { return (ParentNode); } ParentNode = ParentNode->Parent; } return (NULL); /* Object is not within a control method */ +} + + +/******************************************************************************* + * + * FUNCTION: UtGetParentMethodOp + * + * PARAMETERS: Op - Parse Op to be checked + * + * RETURN: Control method Op if found. NULL otherwise + * + * DESCRIPTION: Find the control method parent of a parse op. Returns NULL if + * the input Op is not within a control method. + * + ******************************************************************************/ + +ACPI_PARSE_OBJECT * +UtGetParentMethodOp ( + ACPI_PARSE_OBJECT *Op) +{ + ACPI_PARSE_OBJECT *NextOp; + + + NextOp = Op->Asl.Parent; + while (NextOp) + { + if (NextOp->Asl.AmlOpcode == AML_METHOD_OP) + { + return (NextOp); + } + + NextOp = NextOp->Asl.Parent; + } + + return (NULL); /* No parent method found */ } /******************************************************************************* * * FUNCTION: UtDisplaySupportedTables * * PARAMETERS: None * * RETURN: None * * DESCRIPTION: Print all supported ACPI table names. * ******************************************************************************/ void UtDisplaySupportedTables ( void) { const AH_TABLE *TableData; UINT32 i; printf ("\nACPI tables supported by iASL version %8.8X:\n" " (Compiler, Disassembler, Template Generator)\n\n", ACPI_CA_VERSION); /* All ACPI tables with the common table header */ printf ("\n Supported ACPI tables:\n"); for (TableData = AcpiGbl_SupportedTables, i = 1; TableData->Signature; TableData++, i++) { printf ("%8u) %s %s\n", i, TableData->Signature, TableData->Description); } } /******************************************************************************* * * FUNCTION: UtDisplayConstantOpcodes * * PARAMETERS: None * * RETURN: None * * DESCRIPTION: Print AML opcodes that can be used in constant expressions. * ******************************************************************************/ void UtDisplayConstantOpcodes ( void) { UINT32 i; printf ("Constant expression opcode information\n\n"); for (i = 0; i < sizeof (AcpiGbl_AmlOpInfo) / sizeof (ACPI_OPCODE_INFO); i++) { if (AcpiGbl_AmlOpInfo[i].Flags & AML_CONSTANT) { printf ("%s\n", AcpiGbl_AmlOpInfo[i].Name); } } } /******************************************************************************* * * FUNCTION: UtBeginEvent * * PARAMETERS: Name - Ascii name of this event * * RETURN: Event number (integer index) * * DESCRIPTION: Saves the current time with this event * ******************************************************************************/ UINT8 UtBeginEvent ( char *Name) { if (AslGbl_NextEvent >= ASL_NUM_EVENTS) { AcpiOsPrintf ("Ran out of compiler event structs!\n"); return (AslGbl_NextEvent); } /* Init event with current (start) time */ AslGbl_Events[AslGbl_NextEvent].StartTime = AcpiOsGetTimer (); AslGbl_Events[AslGbl_NextEvent].EventName = Name; AslGbl_Events[AslGbl_NextEvent].Valid = TRUE; return (AslGbl_NextEvent++); } /******************************************************************************* * * FUNCTION: UtEndEvent * * PARAMETERS: Event - Event number (integer index) * * RETURN: None * * DESCRIPTION: Saves the current time (end time) with this event * ******************************************************************************/ void UtEndEvent ( UINT8 Event) { if (Event >= ASL_NUM_EVENTS) { return; } /* Insert end time for event */ AslGbl_Events[Event].EndTime = AcpiOsGetTimer (); } /******************************************************************************* * * FUNCTION: DbgPrint * * PARAMETERS: Type - Type of output * Fmt - Printf format string * ... - variable printf list * * RETURN: None * * DESCRIPTION: Conditional print statement. Prints to stderr only if the * debug flag is set. * ******************************************************************************/ void DbgPrint ( UINT32 Type, char *Fmt, ...) { va_list Args; if (!AslGbl_DebugFlag) { return; } if ((Type == ASL_PARSE_OUTPUT) && (!(AslCompilerdebug))) { return; } va_start (Args, Fmt); (void) vfprintf (stderr, Fmt, Args); va_end (Args); return; } /******************************************************************************* * * FUNCTION: UtSetParseOpName * * PARAMETERS: Op - Parse op to be named. * * RETURN: None * * DESCRIPTION: Insert the ascii name of the parse opcode * ******************************************************************************/ void UtSetParseOpName ( ACPI_PARSE_OBJECT *Op) { AcpiUtSafeStrncpy (Op->Asl.ParseOpName, UtGetOpName (Op->Asl.ParseOpcode), ACPI_MAX_PARSEOP_NAME); } /******************************************************************************* * * FUNCTION: UtDisplayOneSummary * * PARAMETERS: FileID - ID of outpout file * * RETURN: None * * DESCRIPTION: Display compilation statistics for one input file * ******************************************************************************/ void UtDisplayOneSummary ( UINT32 FileId, BOOLEAN DisplayErrorSummary) { UINT32 i; ASL_GLOBAL_FILE_NODE *FileNode; BOOLEAN DisplayAMLSummary; DisplayAMLSummary = !AslGbl_PreprocessOnly && !AslGbl_ParserErrorDetected && ((AslGbl_ExceptionCount[ASL_ERROR] == 0) || AslGbl_IgnoreErrors) && AslGbl_Files[ASL_FILE_AML_OUTPUT].Handle; if (FileId != ASL_FILE_STDOUT) { /* Compiler name and version number */ FlPrintFile (FileId, "%s version %X\n\n", ASL_COMPILER_NAME, (UINT32) ACPI_CA_VERSION); } /* Summary of main input and output files */ FileNode = FlGetCurrentFileNode (); if (FileNode->ParserErrorDetected) { FlPrintFile (FileId, "%-14s %s - Compilation aborted due to parser-detected syntax error(s)\n", "Input file:", AslGbl_Files[ASL_FILE_INPUT].Filename); } else if (FileNode->FileType == ASL_INPUT_TYPE_ASCII_DATA) { FlPrintFile (FileId, "%-14s %s - %7u bytes %6u fields %8u source lines\n", "Table Input:", AslGbl_Files[ASL_FILE_INPUT].Filename, FileNode->OriginalInputFileSize, FileNode->TotalFields, FileNode->TotalLineCount); FlPrintFile (FileId, "%-14s %s - %7u bytes\n", "Binary Output:", AslGbl_Files[ASL_FILE_AML_OUTPUT].Filename, FileNode->OutputByteLength); } else if (FileNode->FileType == ASL_INPUT_TYPE_ASCII_ASL) { FlPrintFile (FileId, "%-14s %s - %7u bytes %6u keywords %6u source lines\n", "ASL Input:", AslGbl_Files[ASL_FILE_INPUT].Filename, FileNode->OriginalInputFileSize, FileNode->TotalKeywords, FileNode->TotalLineCount); /* AML summary */ if (DisplayAMLSummary) { FlPrintFile (FileId, "%-14s %s - %7u bytes %6u opcodes %6u named objects\n", "AML Output:", AslGbl_Files[ASL_FILE_AML_OUTPUT].Filename, FlGetFileSize (ASL_FILE_AML_OUTPUT), FileNode->TotalExecutableOpcodes, FileNode->TotalNamedObjects); } } /* Display summary of any optional files */ for (i = ASL_FILE_SOURCE_OUTPUT; i <= ASL_MAX_FILE_TYPE; i++) { if (!AslGbl_Files[i].Filename || !AslGbl_Files[i].Handle) { continue; } /* .SRC is a temp file unless specifically requested */ if ((i == ASL_FILE_SOURCE_OUTPUT) && (!AslGbl_SourceOutputFlag)) { continue; } /* .PRE is the preprocessor intermediate file */ if ((i == ASL_FILE_PREPROCESSOR) && (!AslGbl_KeepPreprocessorTempFile)) { continue; } FlPrintFile (FileId, "%-14s %s - %7u bytes\n", AslGbl_FileDescs[i].ShortDescription, AslGbl_Files[i].Filename, FlGetFileSize (i)); } /* * Optionally emit an error summary for a file. This is used to enhance the * appearance of listing files. */ if (DisplayErrorSummary) { UtDisplayErrorSummary (FileId); } } /******************************************************************************* * * FUNCTION: UtDisplayErrorSummary * * PARAMETERS: FileID - ID of outpout file * * RETURN: None * * DESCRIPTION: Display compilation statistics for all input files * ******************************************************************************/ static void UtDisplayErrorSummary ( UINT32 FileId) { BOOLEAN ErrorDetected; ErrorDetected = AslGbl_ParserErrorDetected || ((AslGbl_ExceptionCount[ASL_ERROR] > 0) && !AslGbl_IgnoreErrors); if (ErrorDetected) { FlPrintFile (FileId, "\nCompilation failed. "); } else { FlPrintFile (FileId, "\nCompilation successful. "); } FlPrintFile (FileId, "%u Errors, %u Warnings, %u Remarks", AslGbl_ExceptionCount[ASL_ERROR], AslGbl_ExceptionCount[ASL_WARNING] + AslGbl_ExceptionCount[ASL_WARNING2] + AslGbl_ExceptionCount[ASL_WARNING3], AslGbl_ExceptionCount[ASL_REMARK]); if (AslGbl_FileType != ASL_INPUT_TYPE_ASCII_DATA) { if (AslGbl_ParserErrorDetected) { FlPrintFile (FileId, "\nNo AML files were generated due to syntax error(s)\n"); return; } else if (ErrorDetected) { FlPrintFile (FileId, "\nNo AML files were generated due to compiler error(s)\n"); return; } FlPrintFile (FileId, ", %u Optimizations", AslGbl_ExceptionCount[ASL_OPTIMIZATION]); if (AslGbl_TotalFolds) { FlPrintFile (FileId, ", %u Constants Folded", AslGbl_TotalFolds); } } FlPrintFile (FileId, "\n"); } /******************************************************************************* * * FUNCTION: UtDisplaySummary * * PARAMETERS: FileID - ID of outpout file * * RETURN: None * * DESCRIPTION: Display compilation statistics for all input files * ******************************************************************************/ void UtDisplaySummary ( UINT32 FileId) { ASL_GLOBAL_FILE_NODE *Current = AslGbl_FilesList; while (Current) { switch (FlSwitchFileSet(Current->Files[ASL_FILE_INPUT].Filename)) { case SWITCH_TO_SAME_FILE: case SWITCH_TO_DIFFERENT_FILE: UtDisplayOneSummary (FileId, FALSE); Current = Current->Next; break; case FILE_NOT_FOUND: default: Current = NULL; break; } } UtDisplayErrorSummary (FileId); } /******************************************************************************* * * FUNCTION: UtCheckIntegerRange * * PARAMETERS: Op - Integer parse node * LowValue - Smallest allowed value * HighValue - Largest allowed value * * RETURN: Op if OK, otherwise NULL * * DESCRIPTION: Check integer for an allowable range * ******************************************************************************/ ACPI_PARSE_OBJECT * UtCheckIntegerRange ( ACPI_PARSE_OBJECT *Op, UINT32 LowValue, UINT32 HighValue) { if (!Op) { return (NULL); } if ((Op->Asl.Value.Integer < LowValue) || (Op->Asl.Value.Integer > HighValue)) { sprintf (AslGbl_MsgBuffer, "0x%X, allowable: 0x%X-0x%X", (UINT32) Op->Asl.Value.Integer, LowValue, HighValue); AslError (ASL_ERROR, ASL_MSG_RANGE, Op, AslGbl_MsgBuffer); return (NULL); } return (Op); } /******************************************************************************* * * FUNCTION: UtInternalizeName * * PARAMETERS: ExternalName - Name to convert * ConvertedName - Where the converted name is returned * * RETURN: Status * * DESCRIPTION: Convert an external (ASL) name to an internal (AML) name * ******************************************************************************/ ACPI_STATUS UtInternalizeName ( char *ExternalName, char **ConvertedName) { ACPI_NAMESTRING_INFO Info; ACPI_STATUS Status; if (!ExternalName) { return (AE_OK); } /* Get the length of the new internal name */ Info.ExternalName = ExternalName; AcpiNsGetInternalNameLength (&Info); /* We need a segment to store the internal name */ Info.InternalName = UtLocalCacheCalloc (Info.Length); /* Build the name */ Status = AcpiNsBuildInternalName (&Info); if (ACPI_FAILURE (Status)) { return (Status); } *ConvertedName = Info.InternalName; return (AE_OK); } /******************************************************************************* * * FUNCTION: UtPadNameWithUnderscores * * PARAMETERS: NameSeg - Input nameseg * PaddedNameSeg - Output padded nameseg * * RETURN: Padded nameseg. * * DESCRIPTION: Pads a NameSeg with underscores if necessary to form a full * ACPI_NAME. * ******************************************************************************/ static void UtPadNameWithUnderscores ( char *NameSeg, char *PaddedNameSeg) { UINT32 i; for (i = 0; (i < ACPI_NAMESEG_SIZE); i++) { if (*NameSeg) { *PaddedNameSeg = *NameSeg; NameSeg++; } else { *PaddedNameSeg = '_'; } PaddedNameSeg++; } } /******************************************************************************* * * FUNCTION: UtAttachNameseg * * PARAMETERS: Op - Parent parse node * Name - Full ExternalName * * RETURN: None; Sets the NameSeg field in parent node * * DESCRIPTION: Extract the last nameseg of the ExternalName and store it * in the NameSeg field of the Op. * ******************************************************************************/ static void UtAttachNameseg ( ACPI_PARSE_OBJECT *Op, char *Name) { char *NameSeg; char PaddedNameSeg[4]; if (!Name) { return; } /* Look for the last dot in the namepath */ NameSeg = strrchr (Name, '.'); if (NameSeg) { /* Found last dot, we have also found the final nameseg */ NameSeg++; UtPadNameWithUnderscores (NameSeg, PaddedNameSeg); } else { /* No dots in the namepath, there is only a single nameseg. */ /* Handle prefixes */ while (ACPI_IS_ROOT_PREFIX (*Name) || ACPI_IS_PARENT_PREFIX (*Name)) { Name++; } /* Remaining string should be one single nameseg */ UtPadNameWithUnderscores (Name, PaddedNameSeg); } ACPI_COPY_NAMESEG (Op->Asl.NameSeg, PaddedNameSeg); } /******************************************************************************* * * FUNCTION: UtAttachNamepathToOwner * * PARAMETERS: Op - Parent parse node * NameOp - Node that contains the name * * RETURN: Sets the ExternalName and Namepath in the parent node * * DESCRIPTION: Store the name in two forms in the parent node: The original * (external) name, and the internalized name that is used within * the ACPI namespace manager. * ******************************************************************************/ void UtAttachNamepathToOwner ( ACPI_PARSE_OBJECT *Op, ACPI_PARSE_OBJECT *NameOp) { ACPI_STATUS Status; /* Full external path */ Op->Asl.ExternalName = NameOp->Asl.Value.String; /* Save the NameOp for possible error reporting later */ Op->Asl.ParentMethod = (void *) NameOp; /* Last nameseg of the path */ UtAttachNameseg (Op, Op->Asl.ExternalName); /* Create internalized path */ Status = UtInternalizeName (NameOp->Asl.Value.String, &Op->Asl.Namepath); if (ACPI_FAILURE (Status)) { /* TBD: abort on no memory */ } } /******************************************************************************* * * FUNCTION: UtNameContainsAllPrefix * * PARAMETERS: Op - Op containing NameString * * RETURN: NameString consists of all ^ characters * * DESCRIPTION: Determine if this Op contains a name segment that consists of * all '^' characters. * ******************************************************************************/ BOOLEAN UtNameContainsAllPrefix ( ACPI_PARSE_OBJECT *Op) { UINT32 Length = Op->Asl.AmlLength; UINT32 i; for (i = 0; i < Length; i++) { if (Op->Asl.Value.String[i] != '^') { return (FALSE); } } return (TRUE); } /******************************************************************************* * * FUNCTION: UtDoConstant * * PARAMETERS: String - Hex/Decimal/Octal * * RETURN: Converted Integer * * DESCRIPTION: Convert a string to an integer, with overflow/error checking. * ******************************************************************************/ UINT64 UtDoConstant ( char *String) { ACPI_STATUS Status; UINT64 ConvertedInteger; char ErrBuf[64]; Status = AcpiUtStrtoul64 (String, &ConvertedInteger); if (ACPI_FAILURE (Status)) { sprintf (ErrBuf, "While creating 64-bit constant: %s\n", AcpiFormatException (Status)); AslCommonError (ASL_ERROR, ASL_MSG_SYNTAX, AslGbl_CurrentLineNumber, AslGbl_LogicalLineNumber, AslGbl_CurrentLineOffset, AslGbl_CurrentColumn, AslGbl_Files[ASL_FILE_INPUT].Filename, ErrBuf); } return (ConvertedInteger); } /****************************************************************************** * * FUNCTION: AcpiUtStrdup * * PARAMETERS: String1 - string to duplicate * * RETURN: int that signifies string relationship. Zero means strings * are equal. * * DESCRIPTION: Duplicate the string using UtCacheAlloc to avoid manual memory * reclamation. * ******************************************************************************/ char * AcpiUtStrdup ( char *String) { char *NewString = (char *) UtLocalCalloc (strlen (String) + 1); strcpy (NewString, String); return (NewString); } /****************************************************************************** * * FUNCTION: AcpiUtStrcat * * PARAMETERS: String1 * String2 * * RETURN: New string with String1 concatenated with String2 * * DESCRIPTION: Concatenate string1 and string2 * ******************************************************************************/ char * AcpiUtStrcat ( char *String1, char *String2) { UINT32 String1Length = strlen (String1); char *NewString = (char *) UtLocalCalloc (strlen (String1) + strlen (String2) + 1); strcpy (NewString, String1); strcpy (NewString + String1Length, String2); return (NewString); } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslxref.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslxref.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/aslxref.c (revision 357966) @@ -1,1418 +1,1379 @@ /****************************************************************************** * * Module Name: aslxref - Namespace cross-reference * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include "aslcompiler.y.h" #include #include #include #include #define _COMPONENT ACPI_COMPILER ACPI_MODULE_NAME ("aslxref") /* Local prototypes */ static ACPI_STATUS XfNamespaceLocateBegin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); static ACPI_STATUS XfNamespaceLocateEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context); static BOOLEAN XfValidateCrossReference ( ACPI_PARSE_OBJECT *Op, const ACPI_OPCODE_INFO *OpInfo, ACPI_NAMESPACE_NODE *Node); -static ACPI_PARSE_OBJECT * -XfGetParentMethod ( - ACPI_PARSE_OBJECT *Op); - static BOOLEAN XfObjectExists ( char *Name); static ACPI_STATUS XfCompareOneNamespaceObject ( ACPI_HANDLE ObjHandle, UINT32 Level, void *Context, void **ReturnValue); static void XfCheckFieldRange ( ACPI_PARSE_OBJECT *Op, UINT32 RegionBitLength, UINT32 FieldBitOffset, UINT32 FieldBitLength, UINT32 AccessBitWidth); /******************************************************************************* * * FUNCTION: XfCrossReferenceNamespace * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Perform a cross reference check of the parse tree against the * namespace. Every named referenced within the parse tree * should be get resolved with a namespace lookup. If not, the * original reference in the ASL code is invalid -- i.e., refers * to a non-existent object. * * NOTE: The ASL "External" operator causes the name to be inserted into the * namespace so that references to the external name will be resolved * correctly here. * ******************************************************************************/ ACPI_STATUS XfCrossReferenceNamespace ( void) { ACPI_WALK_STATE *WalkState; /* * Create a new walk state for use when looking up names * within the namespace (Passed as context to the callbacks) */ WalkState = AcpiDsCreateWalkState (0, NULL, NULL, NULL); if (!WalkState) { return (AE_NO_MEMORY); } /* Walk the entire parse tree */ TrWalkParseTree (AslGbl_ParseTreeRoot, ASL_WALK_VISIT_TWICE, XfNamespaceLocateBegin, XfNamespaceLocateEnd, WalkState); ACPI_FREE (WalkState); return (AE_OK); } /******************************************************************************* * * FUNCTION: XfObjectExists * * PARAMETERS: Name - 4 char ACPI name * * RETURN: TRUE if name exists in namespace * * DESCRIPTION: Walk the namespace to find an object * ******************************************************************************/ static BOOLEAN XfObjectExists ( char *Name) { ACPI_STATUS Status; /* Walk entire namespace from the supplied root */ Status = AcpiNsWalkNamespace (ACPI_TYPE_ANY, ACPI_ROOT_OBJECT, ACPI_UINT32_MAX, FALSE, XfCompareOneNamespaceObject, NULL, Name, NULL); if (Status == AE_CTRL_TRUE) { /* At least one instance of the name was found */ return (TRUE); } return (FALSE); } /******************************************************************************* * * FUNCTION: XfCompareOneNamespaceObject * * PARAMETERS: ACPI_WALK_CALLBACK * * RETURN: Status * * DESCRIPTION: Compare name of one object. * ******************************************************************************/ static ACPI_STATUS XfCompareOneNamespaceObject ( ACPI_HANDLE ObjHandle, UINT32 Level, void *Context, void **ReturnValue) { ACPI_NAMESPACE_NODE *Node = (ACPI_NAMESPACE_NODE *) ObjHandle; /* Simply check the name */ if (*((UINT32 *) (Context)) == Node->Name.Integer) { /* Abort walk if we found one instance */ return (AE_CTRL_TRUE); } return (AE_OK); } /******************************************************************************* * * FUNCTION: XfCheckFieldRange * * PARAMETERS: RegionBitLength - Length of entire parent region * FieldBitOffset - Start of the field unit (within region) * FieldBitLength - Entire length of field unit * AccessBitWidth - Access width of the field unit * * RETURN: None * * DESCRIPTION: Check one field unit to make sure it fits in the parent * op region. * * Note: AccessBitWidth must be either 8,16,32, or 64 * ******************************************************************************/ static void XfCheckFieldRange ( ACPI_PARSE_OBJECT *Op, UINT32 RegionBitLength, UINT32 FieldBitOffset, UINT32 FieldBitLength, UINT32 AccessBitWidth) { UINT32 FieldEndBitOffset; /* * Check each field unit against the region size. The entire * field unit (start offset plus length) must fit within the * region. */ FieldEndBitOffset = FieldBitOffset + FieldBitLength; if (FieldEndBitOffset > RegionBitLength) { /* Field definition itself is beyond the end-of-region */ AslError (ASL_ERROR, ASL_MSG_FIELD_UNIT_OFFSET, Op, NULL); return; } /* * Now check that the field plus AccessWidth doesn't go beyond * the end-of-region. Assumes AccessBitWidth is a power of 2 */ FieldEndBitOffset = ACPI_ROUND_UP (FieldEndBitOffset, AccessBitWidth); if (FieldEndBitOffset > RegionBitLength) { /* Field definition combined with the access is beyond EOR */ AslError (ASL_ERROR, ASL_MSG_FIELD_UNIT_ACCESS_WIDTH, Op, NULL); } } /******************************************************************************* * - * FUNCTION: XfGetParentMethod - * - * PARAMETERS: Op - Parse Op to be checked - * - * RETURN: Control method Op if found. NULL otherwise - * - * DESCRIPTION: Find the control method parent of a parse op. Returns NULL if - * the input Op is not within a control method. - * - ******************************************************************************/ - -static ACPI_PARSE_OBJECT * -XfGetParentMethod ( - ACPI_PARSE_OBJECT *Op) -{ - ACPI_PARSE_OBJECT *NextOp; - - - NextOp = Op->Asl.Parent; - while (NextOp) - { - if (NextOp->Asl.AmlOpcode == AML_METHOD_OP) - { - return (NextOp); - } - - NextOp = NextOp->Asl.Parent; - } - - return (NULL); /* No parent method found */ -} - - -/******************************************************************************* - * * FUNCTION: XfNamespaceLocateBegin * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: Status * * DESCRIPTION: Descending callback used during cross-reference. For named * object references, attempt to locate the name in the * namespace. * * NOTE: ASL references to named fields within resource descriptors are * resolved to integer values here. Therefore, this step is an * important part of the code generation. We don't know that the * name refers to a resource descriptor until now. * ******************************************************************************/ static ACPI_STATUS XfNamespaceLocateBegin ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { ACPI_WALK_STATE *WalkState = (ACPI_WALK_STATE *) Context; ACPI_NAMESPACE_NODE *Node; ACPI_STATUS Status; ACPI_OBJECT_TYPE ObjectType; char *Path; UINT8 PassedArgs; ACPI_PARSE_OBJECT *NextOp; ACPI_PARSE_OBJECT *OwningOp; ACPI_PARSE_OBJECT *SpaceIdOp; UINT32 MinimumLength; UINT32 Offset; UINT32 FieldBitLength; UINT32 TagBitLength; UINT8 Message = 0; const ACPI_OPCODE_INFO *OpInfo; UINT32 Flags; ASL_METHOD_LOCAL *MethodLocals = NULL; ASL_METHOD_LOCAL *MethodArgs = NULL; int RegisterNumber; UINT32 i; ACPI_NAMESPACE_NODE *DeclarationParentMethod; ACPI_PARSE_OBJECT *ReferenceParentMethod; ACPI_FUNCTION_TRACE_PTR (XfNamespaceLocateBegin, Op); if ((Op->Asl.AmlOpcode == AML_METHOD_OP) && Op->Asl.Node) { Node = Op->Asl.Node; /* Support for method LocalX/ArgX analysis */ if (!Node->MethodLocals) { /* Create local/arg info blocks */ MethodLocals = UtLocalCalloc ( sizeof (ASL_METHOD_LOCAL) * ACPI_METHOD_NUM_LOCALS); Node->MethodLocals = MethodLocals; MethodArgs = UtLocalCalloc ( sizeof (ASL_METHOD_LOCAL) * ACPI_METHOD_NUM_ARGS); Node->MethodArgs = MethodArgs; /* * Get the method argument count * First, get the name node */ NextOp = Op->Asl.Child; /* Get the NumArguments node */ NextOp = NextOp->Asl.Next; Node->ArgCount = (UINT8) (((UINT8) NextOp->Asl.Value.Integer) & 0x07); /* We will track all possible ArgXs */ for (i = 0; i < ACPI_METHOD_NUM_ARGS; i++) { if (i < Node->ArgCount) { /* Real Args are always "initialized" */ MethodArgs[i].Flags = ASL_ARG_INITIALIZED; } else { /* Other ArgXs can be used as locals */ MethodArgs[i].Flags = ASL_ARG_IS_LOCAL; } MethodArgs[i].Op = Op; } } } /* * If this node is the actual declaration of a name * [such as the XXXX name in "Method (XXXX)"], * we are not interested in it here. We only care about names that are * references to other objects within the namespace and the parent objects * of name declarations */ if (Op->Asl.CompileFlags & OP_IS_NAME_DECLARATION) { return_ACPI_STATUS (AE_OK); } OpInfo = AcpiPsGetOpcodeInfo (Op->Asl.AmlOpcode); /* Check method LocalX variables */ if (OpInfo->Type == AML_TYPE_LOCAL_VARIABLE) { /* Find parent method Op */ - NextOp = XfGetParentMethod (Op); + NextOp = UtGetParentMethodOp (Op); if (!NextOp) { return_ACPI_STATUS (AE_OK); } /* Get method node */ Node = NextOp->Asl.Node; RegisterNumber = Op->Asl.AmlOpcode & 0x0007; /* 0x60 through 0x67 */ MethodLocals = Node->MethodLocals; if (Op->Asl.CompileFlags & OP_IS_TARGET) { /* Local is being initialized */ MethodLocals[RegisterNumber].Flags |= ASL_LOCAL_INITIALIZED; MethodLocals[RegisterNumber].Op = Op; return_ACPI_STATUS (AE_OK); } /* Mark this Local as referenced */ MethodLocals[RegisterNumber].Flags |= ASL_LOCAL_REFERENCED; MethodLocals[RegisterNumber].Op = Op; return_ACPI_STATUS (AE_OK); } /* Check method ArgX variables */ if (OpInfo->Type == AML_TYPE_METHOD_ARGUMENT) { /* Find parent method Op */ - NextOp = XfGetParentMethod (Op); + NextOp = UtGetParentMethodOp (Op); if (!NextOp) { return_ACPI_STATUS (AE_OK); } /* Get method node */ Node = NextOp->Asl.Node; /* Get Arg # */ RegisterNumber = Op->Asl.AmlOpcode - AML_ARG0; /* 0x68 through 0x6F */ MethodArgs = Node->MethodArgs; /* Mark this Arg as referenced */ MethodArgs[RegisterNumber].Flags |= ASL_ARG_REFERENCED; MethodArgs[RegisterNumber].Op = Op; if (Op->Asl.CompileFlags & OP_IS_TARGET) { /* Arg is being initialized */ MethodArgs[RegisterNumber].Flags |= ASL_ARG_INITIALIZED; } return_ACPI_STATUS (AE_OK); } /* * After method ArgX and LocalX, we are only interested in opcodes * that have an associated name */ if ((!(OpInfo->Flags & AML_NAMED)) && (!(OpInfo->Flags & AML_CREATE)) && (Op->Asl.ParseOpcode != PARSEOP_NAMESTRING) && (Op->Asl.ParseOpcode != PARSEOP_NAMESEG) && (Op->Asl.ParseOpcode != PARSEOP_METHODCALL) && (Op->Asl.ParseOpcode != PARSEOP_EXTERNAL)) { return_ACPI_STATUS (AE_OK); } /* * One special case: CondRefOf operator - we don't care if the name exists * or not at this point, just ignore it, the point of the operator is to * determine if the name exists at runtime. */ if ((Op->Asl.Parent) && (Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_CONDREFOF)) { return_ACPI_STATUS (AE_OK); } /* * We must enable the "search-to-root" for single NameSegs, but * we have to be very careful about opening up scopes */ Flags = ACPI_NS_SEARCH_PARENT; if ((Op->Asl.ParseOpcode == PARSEOP_NAMESTRING) || (Op->Asl.ParseOpcode == PARSEOP_NAMESEG) || (Op->Asl.ParseOpcode == PARSEOP_METHODCALL) || (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL)) { /* * These are name references, do not push the scope stack * for them. */ Flags |= ACPI_NS_DONT_OPEN_SCOPE; } /* Get the NamePath from the appropriate place */ if (OpInfo->Flags & AML_NAMED) { /* For nearly all NAMED operators, the name reference is the first child */ Path = Op->Asl.Child->Asl.Value.String; if (Op->Asl.AmlOpcode == AML_ALIAS_OP) { /* * ALIAS is the only oddball opcode, the name declaration * (alias name) is the second operand */ Path = Op->Asl.Child->Asl.Next->Asl.Value.String; } } else if (OpInfo->Flags & AML_CREATE) { /* Name must appear as the last parameter */ NextOp = Op->Asl.Child; while (!(NextOp->Asl.CompileFlags & OP_IS_NAME_DECLARATION)) { NextOp = NextOp->Asl.Next; } Path = NextOp->Asl.Value.String; } else { Path = Op->Asl.Value.String; } ObjectType = AslMapNamedOpcodeToDataType (Op->Asl.AmlOpcode); ACPI_DEBUG_PRINT ((ACPI_DB_DISPATCH, "Type=%s\n", AcpiUtGetTypeName (ObjectType))); /* * Lookup the name in the namespace. Name must exist at this point, or it * is an invalid reference. * * The namespace is also used as a lookup table for references to resource * descriptors and the fields within them. */ AslGbl_NsLookupCount++; Status = AcpiNsLookup (WalkState->ScopeInfo, Path, ObjectType, ACPI_IMODE_EXECUTE, Flags, WalkState, &Node); if (ACPI_FAILURE (Status)) { if (Status == AE_NOT_FOUND) { /* * We didn't find the name reference by path -- we can qualify this * a little better before we print an error message */ if (strlen (Path) == ACPI_NAMESEG_SIZE) { /* A simple, one-segment ACPI name */ if (XfObjectExists (Path)) { /* * There exists such a name, but we couldn't get to it * from this scope */ AslError (ASL_ERROR, ASL_MSG_NOT_REACHABLE, Op, Op->Asl.ExternalName); } else { /* The name doesn't exist, period */ AslError (ASL_ERROR, ASL_MSG_NOT_EXIST, Op, Op->Asl.ExternalName); } } else { /* The NamePath contains multiple NameSegs */ if ((OpInfo->Flags & AML_CREATE) || (OpInfo->ObjectType == ACPI_TYPE_LOCAL_ALIAS)) { /* * The new name is the last parameter. For the * CreateXXXXField and Alias operators */ NextOp = Op->Asl.Child; while (!(NextOp->Asl.CompileFlags & OP_IS_NAME_DECLARATION)) { NextOp = NextOp->Asl.Next; } AslError (ASL_ERROR, ASL_MSG_PREFIX_NOT_EXIST, NextOp, NextOp->Asl.ExternalName); } else if (OpInfo->Flags & AML_NAMED) { /* The new name is the first parameter */ AslError (ASL_ERROR, ASL_MSG_PREFIX_NOT_EXIST, Op, Op->Asl.ExternalName); } else if (Path[0] == AML_ROOT_PREFIX) { /* Full namepath from root, the object does not exist */ AslError (ASL_ERROR, ASL_MSG_NOT_EXIST, Op, Op->Asl.ExternalName); } else { /* * Generic "not found" error. Cannot determine whether it * doesn't exist or just can't be reached. However, we * can differentiate between a NameSeg vs. NamePath. */ if (strlen (Op->Asl.ExternalName) == ACPI_NAMESEG_SIZE) { AslError (ASL_ERROR, ASL_MSG_NOT_FOUND, Op, Op->Asl.ExternalName); } else { AslError (ASL_ERROR, ASL_MSG_NAMEPATH_NOT_EXIST, Op, Op->Asl.ExternalName); } } } Status = AE_OK; } return_ACPI_STATUS (Status); } /* Check for an attempt to access an object in another method */ if (!XfValidateCrossReference (Op, OpInfo, Node)) { AslError (ASL_ERROR, ASL_MSG_TEMPORARY_OBJECT, Op, Op->Asl.ExternalName); return_ACPI_STATUS (Status); } /* Object was found above, check for an illegal forward reference */ if (Op->Asl.CompileFlags & OP_NOT_FOUND_DURING_LOAD) { /* * During the load phase, this Op was flagged as a possible * illegal forward reference. In other words, Op is a name path or * name segment that refers to a named object declared after the * reference. In this scinario, Node refers to the actual declaration * and Op is a parse node that references the named object. * * Note: * * Object references inside of control methods are allowed to * refer to objects declared outside of control methods. * * If the declaration and reference are both contained inside of the * same method or outside of any method, this is a forward reference * and should be reported as a compiler error. */ - DeclarationParentMethod = UtGetParentMethod (Node); - ReferenceParentMethod = XfGetParentMethod (Op); + DeclarationParentMethod = UtGetParentMethodNode (Node); + ReferenceParentMethod = UtGetParentMethodOp (Op); - /* case 1: declaration and refrence are both outside of method */ + /* case 1: declaration and reference are both outside of method */ if (!ReferenceParentMethod && !DeclarationParentMethod) { AslError (ASL_ERROR, ASL_MSG_ILLEGAL_FORWARD_REF, Op, Op->Asl.ExternalName); } /* case 2: declaration and reference are both inside of the same method */ else if (ReferenceParentMethod && DeclarationParentMethod && ReferenceParentMethod == DeclarationParentMethod->Op) { AslError (ASL_ERROR, ASL_MSG_ILLEGAL_FORWARD_REF, Op, Op->Asl.ExternalName); } } /* Check for a reference vs. name declaration */ if (!(OpInfo->Flags & AML_NAMED) && !(OpInfo->Flags & AML_CREATE)) { /* This node has been referenced, mark it for reference check */ Node->Flags |= ANOBJ_IS_REFERENCED; } /* Attempt to optimize the NamePath */ OptOptimizeNamePath (Op, OpInfo->Flags, WalkState, Path, Node); /* * 1) Dereference an alias (A name reference that is an alias) * Aliases are not nested, the alias always points to the final object */ if ((Op->Asl.ParseOpcode != PARSEOP_ALIAS) && (Node->Type == ACPI_TYPE_LOCAL_ALIAS)) { /* This node points back to the original PARSEOP_ALIAS */ NextOp = Node->Op; /* The first child is the alias target op */ NextOp = NextOp->Asl.Child; /* That in turn points back to original target alias node */ if (NextOp->Asl.Node) { Node = NextOp->Asl.Node; } /* Else - forward reference to alias, will be resolved later */ } /* 2) Check for a reference to a resource descriptor */ if ((Node->Type == ACPI_TYPE_LOCAL_RESOURCE_FIELD) || (Node->Type == ACPI_TYPE_LOCAL_RESOURCE)) { /* * This was a reference to a field within a resource descriptor. * Extract the associated field offset (either a bit or byte * offset depending on the field type) and change the named * reference into an integer for AML code generation */ Offset = Node->Value; TagBitLength = Node->Length; /* * If a field is being created, generate the length (in bits) of * the field. Note: Opcodes other than CreateXxxField and Index * can come through here. For other opcodes, we just need to * convert the resource tag reference to an integer offset. */ switch (Op->Asl.Parent->Asl.AmlOpcode) { case AML_CREATE_FIELD_OP: /* Variable "Length" field, in bits */ /* * We know the length operand is an integer constant because * we know that it contains a reference to a resource * descriptor tag. */ FieldBitLength = (UINT32) Op->Asl.Next->Asl.Value.Integer; break; case AML_CREATE_BIT_FIELD_OP: FieldBitLength = 1; break; case AML_CREATE_BYTE_FIELD_OP: case AML_INDEX_OP: FieldBitLength = 8; break; case AML_CREATE_WORD_FIELD_OP: FieldBitLength = 16; break; case AML_CREATE_DWORD_FIELD_OP: FieldBitLength = 32; break; case AML_CREATE_QWORD_FIELD_OP: FieldBitLength = 64; break; default: FieldBitLength = 0; break; } /* Check the field length against the length of the resource tag */ if (FieldBitLength) { if (TagBitLength < FieldBitLength) { Message = ASL_MSG_TAG_SMALLER; } else if (TagBitLength > FieldBitLength) { Message = ASL_MSG_TAG_LARGER; } if (Message) { sprintf (AslGbl_MsgBuffer, "Size mismatch, Tag: %u bit%s, Field: %u bit%s", TagBitLength, (TagBitLength > 1) ? "s" : "", FieldBitLength, (FieldBitLength > 1) ? "s" : ""); AslError (ASL_WARNING, Message, Op, AslGbl_MsgBuffer); } } /* Convert the BitOffset to a ByteOffset for certain opcodes */ switch (Op->Asl.Parent->Asl.AmlOpcode) { case AML_CREATE_BYTE_FIELD_OP: case AML_CREATE_WORD_FIELD_OP: case AML_CREATE_DWORD_FIELD_OP: case AML_CREATE_QWORD_FIELD_OP: case AML_INDEX_OP: Offset = ACPI_DIV_8 (Offset); break; default: break; } /* Now convert this node to an integer whose value is the field offset */ Op->Asl.AmlLength = 0; Op->Asl.ParseOpcode = PARSEOP_INTEGER; Op->Asl.Value.Integer = (UINT64) Offset; Op->Asl.CompileFlags |= OP_IS_RESOURCE_FIELD; OpcGenerateAmlOpcode (Op); } /* 3) Check for a method invocation */ else if ((((Op->Asl.ParseOpcode == PARSEOP_NAMESTRING) || (Op->Asl.ParseOpcode == PARSEOP_NAMESEG)) && (Node->Type == ACPI_TYPE_METHOD) && (Op->Asl.Parent) && (Op->Asl.Parent->Asl.ParseOpcode != PARSEOP_METHOD)) || (Op->Asl.ParseOpcode == PARSEOP_METHODCALL)) { /* * A reference to a method within one of these opcodes is not an * invocation of the method, it is simply a reference to the method. * * September 2016: Removed DeRefOf from this list */ if ((Op->Asl.Parent) && ((Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_REFOF) || (Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_PACKAGE) || (Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_VAR_PACKAGE)|| (Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_OBJECTTYPE))) { return_ACPI_STATUS (AE_OK); } /* * There are two types of method invocation: * 1) Invocation with arguments -- the parser recognizes this * as a METHODCALL. * 2) Invocation with no arguments --the parser cannot determine that * this is a method invocation, therefore we have to figure it out * here. */ if (Node->Type != ACPI_TYPE_METHOD) { sprintf (AslGbl_MsgBuffer, "%s is a %s", Op->Asl.ExternalName, AcpiUtGetTypeName (Node->Type)); AslError (ASL_ERROR, ASL_MSG_NOT_METHOD, Op, AslGbl_MsgBuffer); return_ACPI_STATUS (AE_OK); } /* Save the method node in the caller's op */ Op->Asl.Node = Node; if (Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_CONDREFOF) { return_ACPI_STATUS (AE_OK); } /* * This is a method invocation, with or without arguments. * Count the number of arguments, each appears as a child * under the parent node */ Op->Asl.ParseOpcode = PARSEOP_METHODCALL; UtSetParseOpName (Op); PassedArgs = 0; NextOp = Op->Asl.Child; while (NextOp) { PassedArgs++; NextOp = NextOp->Asl.Next; } if (Node->Value != ASL_EXTERNAL_METHOD_UNKNOWN_PARAMS && Op->Asl.Parent->Asl.ParseOpcode != PARSEOP_EXTERNAL) { /* * Check the parsed arguments with the number expected by the * method declaration itself */ if (PassedArgs != Node->Value) { if (Node->Flags & ANOBJ_IS_EXTERNAL) { sprintf (AslGbl_MsgBuffer, "according to previous use, %s requires %u", Op->Asl.ExternalName, Node->Value); } else { sprintf (AslGbl_MsgBuffer, "%s requires %u", Op->Asl.ExternalName, Node->Value); } if (PassedArgs < Node->Value) { AslError (ASL_ERROR, ASL_MSG_ARG_COUNT_LO, Op, AslGbl_MsgBuffer); } else { AslError (ASL_ERROR, ASL_MSG_ARG_COUNT_HI, Op, AslGbl_MsgBuffer); } } } /* * At this point, a method call to an external method has been * detected. As of 11/19/2019, iASL does not support parameter counts * for methods declared as external. Therefore, save the parameter * count of the first method call and use this count check other * method calls to ensure that the methods are being called with the * same amount of parameters. */ else if (Node->Type == ACPI_TYPE_METHOD && (Node->Flags & ANOBJ_IS_EXTERNAL) && Node->Value == ASL_EXTERNAL_METHOD_UNKNOWN_PARAMS && Op->Asl.Parent->Asl.ParseOpcode != PARSEOP_EXTERNAL) { Node->Value = PassedArgs; } } /* 4) Check for an ASL Field definition */ else if ((Op->Asl.Parent) && ((Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_FIELD) || (Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_BANKFIELD))) { /* * Offset checking for fields. If the parent operation region has a * constant length (known at compile time), we can check fields * defined in that region against the region length. This will catch * fields and field units that cannot possibly fit within the region. * * Note: Index fields do not directly reference an operation region, * thus they are not included in this check. */ if (Op == Op->Asl.Parent->Asl.Child) { /* * This is the first child of the field node, which is * the name of the region. Get the parse node for the * region -- which contains the length of the region. */ OwningOp = Node->Op; Op->Asl.Parent->Asl.ExtraValue = ACPI_MUL_8 ((UINT32) OwningOp->Asl.Value.Integer); /* Examine the field access width */ switch ((UINT8) Op->Asl.Parent->Asl.Value.Integer) { case AML_FIELD_ACCESS_ANY: case AML_FIELD_ACCESS_BYTE: case AML_FIELD_ACCESS_BUFFER: default: MinimumLength = 1; break; case AML_FIELD_ACCESS_WORD: MinimumLength = 2; break; case AML_FIELD_ACCESS_DWORD: MinimumLength = 4; break; case AML_FIELD_ACCESS_QWORD: MinimumLength = 8; break; } /* * Is the region at least as big as the access width? * Note: DataTableRegions have 0 length */ if (((UINT32) OwningOp->Asl.Value.Integer) && ((UINT32) OwningOp->Asl.Value.Integer < MinimumLength)) { AslError (ASL_ERROR, ASL_MSG_FIELD_ACCESS_WIDTH, Op, NULL); } /* * Check EC/CMOS/SMBUS fields to make sure that the correct * access type is used (BYTE for EC/CMOS, BUFFER for SMBUS) */ SpaceIdOp = OwningOp->Asl.Child->Asl.Next; switch ((UINT32) SpaceIdOp->Asl.Value.Integer) { case ACPI_ADR_SPACE_EC: case ACPI_ADR_SPACE_CMOS: case ACPI_ADR_SPACE_GPIO: if ((UINT8) Op->Asl.Parent->Asl.Value.Integer != AML_FIELD_ACCESS_BYTE) { AslError (ASL_ERROR, ASL_MSG_REGION_BYTE_ACCESS, Op, NULL); } break; case ACPI_ADR_SPACE_SMBUS: case ACPI_ADR_SPACE_IPMI: case ACPI_ADR_SPACE_GSBUS: if ((UINT8) Op->Asl.Parent->Asl.Value.Integer != AML_FIELD_ACCESS_BUFFER) { AslError (ASL_ERROR, ASL_MSG_REGION_BUFFER_ACCESS, Op, NULL); } break; default: /* Nothing to do for other address spaces */ break; } } else { /* * This is one element of the field list. Check to make sure * that it does not go beyond the end of the parent operation region. * * In the code below: * Op->Asl.Parent->Asl.ExtraValue - Region Length (bits) * Op->Asl.ExtraValue - Field start offset (bits) * Op->Asl.Child->Asl.Value.Integer32 - Field length (bits) * Op->Asl.Child->Asl.ExtraValue - Field access width (bits) */ if (Op->Asl.Parent->Asl.ExtraValue && Op->Asl.Child) { XfCheckFieldRange (Op, Op->Asl.Parent->Asl.ExtraValue, Op->Asl.ExtraValue, (UINT32) Op->Asl.Child->Asl.Value.Integer, Op->Asl.Child->Asl.ExtraValue); } } } /* * 5) Check for external resolution * By this point, everything should be loaded in the namespace. If a * namespace lookup results in a namespace node that is an external, it * means that this named object was not defined in the input ASL. This * causes issues because there are plenty of incidents where developers * use the external keyword to suppress compiler errors about undefined * objects. Note: this only applies when compiling multiple definition * blocks. */ if (AslGbl_ParseTreeRoot->Asl.Child && AslGbl_ParseTreeRoot->Asl.Child->Asl.Next && (Op->Asl.ParseOpcode != PARSEOP_EXTERNAL && Op->Asl.Parent->Asl.ParseOpcode != PARSEOP_EXTERNAL) && (Node->Flags & ANOBJ_IS_EXTERNAL)) { AslError (ASL_ERROR, ASL_MSG_UNDEFINED_EXTERNAL, Op, NULL); } /* 5) Check for a connection object */ #if 0 else if (Op->Asl.Parent->Asl.ParseOpcode == PARSEOP_CONNECTION) { return_ACPI_STATUS (Status); } #endif Op->Asl.Node = Node; return_ACPI_STATUS (Status); } /******************************************************************************* * * FUNCTION: XfNamespaceLocateEnd * * PARAMETERS: ASL_WALK_CALLBACK * * RETURN: Status * * DESCRIPTION: Ascending callback used during cross reference. We only * need to worry about scope management here. * ******************************************************************************/ static ACPI_STATUS XfNamespaceLocateEnd ( ACPI_PARSE_OBJECT *Op, UINT32 Level, void *Context) { ACPI_WALK_STATE *WalkState = (ACPI_WALK_STATE *) Context; const ACPI_OPCODE_INFO *OpInfo; ACPI_FUNCTION_TRACE (XfNamespaceLocateEnd); /* We are only interested in opcodes that have an associated name */ OpInfo = AcpiPsGetOpcodeInfo (Op->Asl.AmlOpcode); if (!(OpInfo->Flags & AML_NAMED)) { return_ACPI_STATUS (AE_OK); } /* Not interested in name references, we did not open a scope for them */ if ((Op->Asl.ParseOpcode == PARSEOP_NAMESTRING) || (Op->Asl.ParseOpcode == PARSEOP_NAMESEG) || (Op->Asl.ParseOpcode == PARSEOP_METHODCALL) || (Op->Asl.ParseOpcode == PARSEOP_EXTERNAL)) { return_ACPI_STATUS (AE_OK); } /* Pop the scope stack if necessary */ if (AcpiNsOpensScope (AslMapNamedOpcodeToDataType (Op->Asl.AmlOpcode))) { ACPI_DEBUG_PRINT ((ACPI_DB_DISPATCH, "%s: Popping scope for Op %p\n", AcpiUtGetTypeName (OpInfo->ObjectType), Op)); (void) AcpiDsScopeStackPop (WalkState); } return_ACPI_STATUS (AE_OK); } /******************************************************************************* * * FUNCTION: XfValidateCrossReference * * PARAMETERS: Op - Parse Op that references the object * OpInfo - Parse Op info struct * Node - Node for the referenced object * * RETURN: TRUE if the reference is legal, FALSE otherwise * * DESCRIPTION: Determine if a reference to another object is allowed. * * EXAMPLE: * Method (A) {Name (INT1, 1)} Declaration of object INT1 * Method (B) (Store (2, \A.INT1)} Illegal reference to object INT1 * (INT1 is temporary, valid only during * execution of A) * * NOTES: - * A null pointer returned by either XfGetParentMethod or - * UtGetParentMethod indicates that the parameter object is not + * A null pointer returned by either UtGetParentMethodOp or + * UtGetParentMethodNode indicates that the parameter object is not * within a control method. * * Five cases are handled: Case(Op, Node) * 1) Case(0,0): Op is not within a method, Node is not --> OK * 2) Case(0,1): Op is not within a method, but Node is --> Illegal * 3) Case(1,0): Op is within a method, Node is not --> OK * 4) Case(1,1): Both are within the same method --> OK * 5) Case(1,1): Both are in methods, but not same method --> Illegal * ******************************************************************************/ static BOOLEAN XfValidateCrossReference ( ACPI_PARSE_OBJECT *Op, const ACPI_OPCODE_INFO *OpInfo, ACPI_NAMESPACE_NODE *Node) { ACPI_PARSE_OBJECT *ReferencingMethodOp; ACPI_NAMESPACE_NODE *ReferencedMethodNode; /* Ignore actual named (and related) object declarations */ if (OpInfo->Flags & (AML_NAMED | AML_CREATE | AML_DEFER | AML_HAS_ARGS)) { return (TRUE); } /* * 1) Search upwards in parse tree for owner of the referencing object * 2) Search upwards in namespace to find the owner of the referenced object */ - ReferencingMethodOp = XfGetParentMethod (Op); - ReferencedMethodNode = UtGetParentMethod (Node); + ReferencingMethodOp = UtGetParentMethodOp (Op); + ReferencedMethodNode = UtGetParentMethodNode (Node); if (!ReferencingMethodOp && !ReferencedMethodNode) { /* * 1) Case (0,0): Both Op and Node are not within methods * --> OK */ return (TRUE); } if (!ReferencingMethodOp && ReferencedMethodNode) { /* * 2) Case (0,1): Op is not in a method, but Node is within a * method --> illegal */ return (FALSE); } else if (ReferencingMethodOp && !ReferencedMethodNode) { /* * 3) Case (1,0): Op is within a method, but Node is not * --> OK */ return (TRUE); } else if (ReferencingMethodOp->Asl.Node == ReferencedMethodNode) { /* * 4) Case (1,1): Both Op and Node are within the same method * --> OK */ return (TRUE); } else { /* * 5) Case (1,1), Op and Node are in different methods * --> Illegal */ return (FALSE); } } Index: projects/clang1000-import/sys/contrib/dev/acpica/compiler/cvparser.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/compiler/cvparser.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/compiler/cvparser.c (revision 357966) @@ -1,1035 +1,1039 @@ /****************************************************************************** * * Module Name: cvparser - Converter functions that are called from the AML * parser. * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #include #include #include #include #include /* local prototypes */ static BOOLEAN CvCommentExists ( UINT8 *Address); static BOOLEAN CvIsFilename ( char *Filename); static ACPI_FILE_NODE* CvFileAddressLookup( char *Address, ACPI_FILE_NODE *Head); static void CvAddToFileTree ( char *Filename, char *PreviousFilename); static void CvSetFileParent ( char *ChildFile, char *ParentFile); /******************************************************************************* * * FUNCTION: CvIsFilename * * PARAMETERS: filename - input filename * * RETURN: BOOLEAN - TRUE if all characters are between 0x20 and 0x7f * * DESCRIPTION: Take a given char * and see if it contains all printable * characters. If all characters have hexvalues 20-7f and ends with * .dsl, we will assume that it is a proper filename. * ******************************************************************************/ static BOOLEAN CvIsFilename ( char *Filename) { UINT64 Length = strlen(Filename); char *FileExt = Filename + Length - 4; UINT64 i; if ((Length > 4) && AcpiUtStricmp (FileExt, ".dsl")) { return (FALSE); } for(i = 0; iLength - sizeof (ACPI_TABLE_HEADER); + AmlStart = ((UINT8 *) Table + sizeof (ACPI_TABLE_HEADER)); + CvDbgPrint ("AmlLength: %x\n", AmlLength); CvDbgPrint ("AmlStart: %p\n", AmlStart); - CvDbgPrint ("AmlEnd?: %p\n", AmlStart+AmlLength); + CvDbgPrint ("AmlEnd: %p\n", AmlStart+AmlLength); AcpiGbl_FileTreeRoot = AcpiOsAcquireObject (AcpiGbl_FileCache); AcpiGbl_FileTreeRoot->FileStart = (char *)(AmlStart); AcpiGbl_FileTreeRoot->FileEnd = (char *)(AmlStart + Table->Length); AcpiGbl_FileTreeRoot->Next = NULL; AcpiGbl_FileTreeRoot->Parent = NULL; AcpiGbl_FileTreeRoot->Filename = (char *)(AmlStart+2); /* Set the root file to the current open file */ - AcpiGbl_FileTreeRoot->File = AcpiGbl_OutputFile; + AcpiGbl_FileTreeRoot->File = RootFile; /* * Set this to true because we don't need to output * an include statement for the topmost file */ AcpiGbl_FileTreeRoot->IncludeWritten = TRUE; Filename = NULL; AcpiGbl_CurrentFilename = (char *)(AmlStart+2); AcpiGbl_RootFilename = (char *)(AmlStart+2); TreeAml = AmlStart; FileEnd = AmlStart + AmlLength; while (TreeAml <= FileEnd) { /* * Make sure that this filename contains all printable characters * and a .dsl extension at the end. If not, then it must be some * raw data that doesn't outline a filename. */ if ((*TreeAml == AML_COMMENT_OP) && (*(TreeAml +1) == FILENAME_COMMENT) && (CvIsFilename ((char *)(TreeAml +2)))) { CvDbgPrint ("A9 and a 08 file\n"); PreviousFilename = Filename; Filename = (char *) (TreeAml +2); CvAddToFileTree (Filename, PreviousFilename); ChildFilename = Filename; CvDbgPrint ("%s\n", Filename); } else if ((*TreeAml == AML_COMMENT_OP) && (*(TreeAml +1) == PARENTFILENAME_COMMENT) && (CvIsFilename ((char *)(TreeAml +2)))) { CvDbgPrint ("A9 and a 09 file\n"); ParentFilename = (char *)(TreeAml +2); CvSetFileParent (ChildFilename, ParentFilename); CvDbgPrint ("%s\n", ParentFilename); } ++TreeAml; } } /******************************************************************************* * * FUNCTION: CvClearOpComments * * PARAMETERS: Op -- clear all comments within this Op * * RETURN: None * * DESCRIPTION: Clear all converter-related fields of the given Op. * This is referred as ASL_CV_CLEAR_OP_COMMENTS. * ******************************************************************************/ void CvClearOpComments ( ACPI_PARSE_OBJECT *Op) { Op->Common.InlineComment = NULL; Op->Common.EndNodeComment = NULL; Op->Common.NameComment = NULL; Op->Common.CommentList = NULL; Op->Common.EndBlkComment = NULL; Op->Common.CloseBraceComment = NULL; Op->Common.CvFilename = NULL; Op->Common.CvParentFilename = NULL; } /******************************************************************************* * * FUNCTION: CvCommentExists * * PARAMETERS: Address - check if this address appears in the list * * RETURN: BOOLEAN - TRUE if the address exists. * * DESCRIPTION: Look at the pointer address and check if this appears in the * list of all addresses. If it exists in the list, return TRUE * if it exists. Otherwise add to the list and return FALSE. * ******************************************************************************/ static BOOLEAN CvCommentExists ( UINT8 *Address) { ACPI_COMMENT_ADDR_NODE *Current = AcpiGbl_CommentAddrListHead; UINT8 Option; if (!Address) { return (FALSE); } Option = *(Address + 1); /* * FILENAME_COMMENT and PARENTFILENAME_COMMENT are not treated as * comments. They serve as markers for where the file starts and ends. */ if ((Option == FILENAME_COMMENT) || (Option == PARENTFILENAME_COMMENT)) { return (FALSE); } if (!Current) { AcpiGbl_CommentAddrListHead = AcpiOsAcquireObject (AcpiGbl_RegCommentCache); AcpiGbl_CommentAddrListHead->Addr = Address; AcpiGbl_CommentAddrListHead->Next = NULL; return (FALSE); } else { while (Current) { if (Current->Addr != Address) { Current = Current->Next; } else { return (TRUE); } } /* * If the execution gets to this point, it means that this * address does not exists in the list. Add this address to the * beginning of the list. */ Current = AcpiGbl_CommentAddrListHead; AcpiGbl_CommentAddrListHead = AcpiOsAcquireObject (AcpiGbl_RegCommentCache); AcpiGbl_CommentAddrListHead->Addr = Address; AcpiGbl_CommentAddrListHead->Next = Current; return (FALSE); } } /******************************************************************************* * * FUNCTION: CvFilenameExists * * PARAMETERS: Filename - filename to search * * RETURN: ACPI_FILE_NODE - a pointer to a file node * * DESCRIPTION: Look for the given filename in the file dependency tree. * Returns the file node if it exists, returns NULL if it does not. * ******************************************************************************/ ACPI_FILE_NODE* CvFilenameExists( char *Filename, ACPI_FILE_NODE *Head) { ACPI_FILE_NODE *Current = Head; if (!Filename) { return (NULL); } while (Current) { if (!AcpiUtStricmp (Current->Filename, Filename)) { return (Current); } Current = Current->Next; } return (NULL); } /******************************************************************************* * * FUNCTION: CvFileAddressLookup * * PARAMETERS: Address - address to look up * Head - file dependency tree * * RETURN: ACPI_FILE_NODE - pointer to a file node containing the address * * DESCRIPTION: Look for the given address in the file dependency tree. * Returns the first file node where the given address is within * the file node's starting and ending address. * ******************************************************************************/ static ACPI_FILE_NODE * CvFileAddressLookup( char *Address, ACPI_FILE_NODE *Head) { ACPI_FILE_NODE *Current = Head; while (Current) { if ((Address >= Current->FileStart) && (Address < Current->FileEnd || !Current->FileEnd)) { return (Current); } Current = Current->Next; } return (NULL); } /******************************************************************************* * * FUNCTION: CvLabelFileNode * * PARAMETERS: Op * * RETURN: None * * DESCRIPTION: Takes a given parse op, looks up its Op->Common.Aml field * within the file tree and fills in appropriate file information * from a matching node within the tree. * This is referred as ASL_CV_LABEL_FILENODE. * ******************************************************************************/ void CvLabelFileNode( ACPI_PARSE_OBJECT *Op) { ACPI_FILE_NODE *Node; if (!Op) { return; } Node = CvFileAddressLookup ((char *) Op->Common.Aml, AcpiGbl_FileTreeRoot); if (!Node) { return; } Op->Common.CvFilename = Node->Filename; if (Node->Parent) { Op->Common.CvParentFilename = Node->Parent->Filename; } else { Op->Common.CvParentFilename = Node->Filename; } } /******************************************************************************* * * FUNCTION: CvAddToFileTree * * PARAMETERS: Filename - Address containing the name of the current * filename * PreviousFilename - Address containing the name of the previous * filename * * RETURN: None * * DESCRIPTION: Add this filename to the AcpiGbl_FileTree if it does not exist. * ******************************************************************************/ static void CvAddToFileTree ( char *Filename, char *PreviousFilename) { ACPI_FILE_NODE *Node; if (!AcpiUtStricmp(Filename, AcpiGbl_RootFilename) && PreviousFilename) { Node = CvFilenameExists (PreviousFilename, AcpiGbl_FileTreeRoot); if (Node) { /* * Set the end point of the PreviousFilename to the address * of Filename. */ Node->FileEnd = Filename; } } else if (!AcpiUtStricmp(Filename, AcpiGbl_RootFilename) && !PreviousFilename) { return; } Node = CvFilenameExists (Filename, AcpiGbl_FileTreeRoot); if (Node && PreviousFilename) { /* * Update the end of the previous file and all of their parents' * ending addresses. This is done to ensure that parent file * ranges extend to the end of their childrens' files. */ Node = CvFilenameExists (PreviousFilename, AcpiGbl_FileTreeRoot); if (Node && (Node->FileEnd < Filename)) { Node->FileEnd = Filename; Node = Node->Parent; while (Node) { if (Node->FileEnd < Filename) { Node->FileEnd = Filename; } Node = Node->Parent; } } } else { Node = AcpiGbl_FileTreeRoot; AcpiGbl_FileTreeRoot = AcpiOsAcquireObject (AcpiGbl_FileCache); AcpiGbl_FileTreeRoot->Next = Node; AcpiGbl_FileTreeRoot->Parent = NULL; AcpiGbl_FileTreeRoot->Filename = Filename; AcpiGbl_FileTreeRoot->FileStart = Filename; AcpiGbl_FileTreeRoot->IncludeWritten = FALSE; AcpiGbl_FileTreeRoot->File = fopen(Filename, "w+"); /* * If we can't open the file, we need to abort here before we * accidentally write to a NULL file. */ if (!AcpiGbl_FileTreeRoot->File) { /* delete the .xxx file */ FlDeleteFile (ASL_FILE_AML_OUTPUT); sprintf (AslGbl_MsgBuffer, "\"%s\" - %s", Filename, strerror (errno)); AslCommonError (ASL_ERROR, ASL_MSG_OPEN, 0, 0, 0, 0, NULL, AslGbl_MsgBuffer); AslAbort (); } } } /******************************************************************************* * * FUNCTION: CvSetFileParent * * PARAMETERS: ChildFile - contains the filename of the child file * ParentFile - contains the filename of the parent file. * * RETURN: None * * DESCRIPTION: Point the parent pointer of the Child to the node that * corresponds with the parent file node. * ******************************************************************************/ static void CvSetFileParent ( char *ChildFile, char *ParentFile) { ACPI_FILE_NODE *Child; ACPI_FILE_NODE *Parent; Child = CvFilenameExists (ChildFile, AcpiGbl_FileTreeRoot); Parent = CvFilenameExists (ParentFile, AcpiGbl_FileTreeRoot); if (Child && Parent) { Child->Parent = Parent; while (Child->Parent) { if (Child->Parent->FileEnd < Child->FileStart) { Child->Parent->FileEnd = Child->FileStart; } Child = Child->Parent; } } } /******************************************************************************* * * FUNCTION: CvCaptureCommentsOnly * * PARAMETERS: ParserState - A parser state object * * RETURN: None * * DESCRIPTION: Look at the aml that the parser state is pointing to, * capture any AML_COMMENT_OP and it's arguments and increment the * aml pointer past the comment. Comments are transferred to parse * nodes through CvTransferComments() as well as * AcpiPsBuildNamedOp(). * This is referred as ASL_CV_CAPTURE_COMMENTS_ONLY. * ******************************************************************************/ void CvCaptureCommentsOnly ( ACPI_PARSE_STATE *ParserState) { UINT8 *Aml = ParserState->Aml; UINT16 Opcode = (UINT16) ACPI_GET8 (Aml); UINT32 Length = 0; UINT8 CommentOption; BOOLEAN StdDefBlockFlag = FALSE; ACPI_COMMENT_NODE *CommentNode; ACPI_FILE_NODE *FileNode; if (!AcpiGbl_CaptureComments || Opcode != AML_COMMENT_OP) { return; } while (Opcode == AML_COMMENT_OP) { CvDbgPrint ("comment aml address: %p\n", Aml); if (CvCommentExists(ParserState->Aml)) { CvDbgPrint ("Avoiding capturing an existing comment.\n"); } else { CommentOption = *(Aml +1); /* * Increment past the comment option and point the * appropriate char pointers */ Aml += 2; /* Found a comment. Now, set pointers to these comments. */ switch (CommentOption) { case STD_DEFBLK_COMMENT: StdDefBlockFlag = TRUE; /* * Add to a linked list of nodes. This list will be * taken by the parse node created next. */ CommentNode = AcpiOsAcquireObject ( AcpiGbl_RegCommentCache); CommentNode->Comment = ACPI_CAST_PTR (char, Aml); CommentNode->Next = NULL; if (!AcpiGbl_DefBlkCommentListHead) { AcpiGbl_DefBlkCommentListHead = CommentNode; AcpiGbl_DefBlkCommentListTail = CommentNode; } else { AcpiGbl_DefBlkCommentListTail->Next = CommentNode; AcpiGbl_DefBlkCommentListTail = AcpiGbl_DefBlkCommentListTail->Next; } break; case STANDARD_COMMENT: CvDbgPrint ("found regular comment.\n"); /* * Add to a linked list of nodes. This list will be * taken by the parse node created next. */ CommentNode = AcpiOsAcquireObject ( AcpiGbl_RegCommentCache); CommentNode->Comment = ACPI_CAST_PTR (char, Aml); CommentNode->Next = NULL; if (!AcpiGbl_RegCommentListHead) { AcpiGbl_RegCommentListHead = CommentNode; AcpiGbl_RegCommentListTail = CommentNode; } else { AcpiGbl_RegCommentListTail->Next = CommentNode; AcpiGbl_RegCommentListTail = AcpiGbl_RegCommentListTail->Next; } break; case ENDBLK_COMMENT: CvDbgPrint ("found endblk comment.\n"); /* Add to a linked list of nodes. This will be * taken by the next created parse node. */ CommentNode = AcpiOsAcquireObject ( AcpiGbl_RegCommentCache); CommentNode->Comment = ACPI_CAST_PTR (char, Aml); CommentNode->Next = NULL; if (!AcpiGbl_EndBlkCommentListHead) { AcpiGbl_EndBlkCommentListHead = CommentNode; AcpiGbl_EndBlkCommentListTail = CommentNode; } else { AcpiGbl_EndBlkCommentListTail->Next = CommentNode; AcpiGbl_EndBlkCommentListTail = AcpiGbl_EndBlkCommentListTail->Next; } break; case INLINE_COMMENT: CvDbgPrint ("found inline comment.\n"); AcpiGbl_CurrentInlineComment = ACPI_CAST_PTR (char, Aml); break; case ENDNODE_COMMENT: CvDbgPrint ("found EndNode comment.\n"); AcpiGbl_CurrentEndNodeComment = ACPI_CAST_PTR (char, Aml); break; case CLOSE_BRACE_COMMENT: CvDbgPrint ("found close brace comment.\n"); AcpiGbl_CurrentCloseBraceComment = ACPI_CAST_PTR (char, Aml); break; case END_DEFBLK_COMMENT: CvDbgPrint ("Found comment that belongs after" " the } for a definition block.\n"); AcpiGbl_CurrentScope->Common.CloseBraceComment = ACPI_CAST_PTR (char, Aml); break; case FILENAME_COMMENT: CvDbgPrint ("Found a filename: %s\n", ACPI_CAST_PTR (char, Aml)); FileNode = CvFilenameExists ( ACPI_CAST_PTR (char, Aml), AcpiGbl_FileTreeRoot); /* * If there is an INCLUDE_COMMENT followed by a * FILENAME_COMMENT, then the INCLUDE_COMMENT is a comment * that is emitted before the #include for the file. * We will save the IncludeComment within the FileNode * associated with this FILENAME_COMMENT. */ if (FileNode && AcpiGbl_IncCommentListHead) { FileNode->IncludeComment = AcpiGbl_IncCommentListHead; AcpiGbl_IncCommentListHead = NULL; AcpiGbl_IncCommentListTail = NULL; } break; case PARENTFILENAME_COMMENT: CvDbgPrint (" Found a parent filename.\n"); break; case INCLUDE_COMMENT: /* * Add to a linked list. This list will be taken by the * parse node created next. See the FILENAME_COMMENT case * for more details */ CommentNode = AcpiOsAcquireObject ( AcpiGbl_RegCommentCache); CommentNode->Comment = ACPI_CAST_PTR (char, Aml); CommentNode->Next = NULL; if (!AcpiGbl_IncCommentListHead) { AcpiGbl_IncCommentListHead = CommentNode; AcpiGbl_IncCommentListTail = CommentNode; } else { AcpiGbl_IncCommentListTail->Next = CommentNode; AcpiGbl_IncCommentListTail = AcpiGbl_IncCommentListTail->Next; } CvDbgPrint ("Found a include comment: %s\n", CommentNode->Comment); break; default: /* Not a valid comment option. Revert the AML */ goto DefBlock; } /* End switch statement */ } /* End else */ /* Determine the length and move forward that amount */ Length = 0; while (ParserState->Aml[Length]) { Length++; } ParserState->Aml += Length + 1; /* Peek at the next Opcode. */ Aml = ParserState->Aml; Opcode = (UINT16) ACPI_GET8 (Aml); } DefBlock: if (StdDefBlockFlag) { /* * Give all of its comments to the current scope, which is known as * the definition block, since STD_DEFBLK_COMMENT only appears after * definition block headers. */ AcpiGbl_CurrentScope->Common.CommentList = AcpiGbl_DefBlkCommentListHead; AcpiGbl_DefBlkCommentListHead = NULL; AcpiGbl_DefBlkCommentListTail = NULL; } } /******************************************************************************* * * FUNCTION: CvCaptureComments * * PARAMETERS: ParserState - A parser state object * * RETURN: None * * DESCRIPTION: Wrapper function for CvCaptureCommentsOnly * This is referred as ASL_CV_CAPTURE_COMMENTS. * ******************************************************************************/ void CvCaptureComments ( ACPI_WALK_STATE *WalkState) { UINT8 *Aml; UINT16 Opcode; const ACPI_OPCODE_INFO *OpInfo; if (!AcpiGbl_CaptureComments) { return; } /* * Before parsing, check to see that comments that come directly * after deferred opcodes aren't being processed. */ Aml = WalkState->ParserState.Aml; Opcode = (UINT16) ACPI_GET8 (Aml); OpInfo = AcpiPsGetOpcodeInfo (Opcode); if (!(OpInfo->Flags & AML_DEFER) || ((OpInfo->Flags & AML_DEFER) && (WalkState->PassNumber != ACPI_IMODE_LOAD_PASS1))) { CvCaptureCommentsOnly (&WalkState->ParserState); WalkState->Aml = WalkState->ParserState.Aml; } } /******************************************************************************* * * FUNCTION: CvTransferComments * * PARAMETERS: Op - Transfer comments to this Op * * RETURN: None * * DESCRIPTION: Transfer all of the comments stored in global containers to the * given Op. This will be invoked shortly after the parser creates * a ParseOp. * This is referred as ASL_CV_TRANSFER_COMMENTS. * ******************************************************************************/ void CvTransferComments ( ACPI_PARSE_OBJECT *Op) { Op->Common.InlineComment = AcpiGbl_CurrentInlineComment; AcpiGbl_CurrentInlineComment = NULL; Op->Common.EndNodeComment = AcpiGbl_CurrentEndNodeComment; AcpiGbl_CurrentEndNodeComment = NULL; Op->Common.CloseBraceComment = AcpiGbl_CurrentCloseBraceComment; AcpiGbl_CurrentCloseBraceComment = NULL; Op->Common.CommentList = AcpiGbl_RegCommentListHead; AcpiGbl_RegCommentListHead = NULL; AcpiGbl_RegCommentListTail = NULL; Op->Common.EndBlkComment = AcpiGbl_EndBlkCommentListHead; AcpiGbl_EndBlkCommentListHead = NULL; AcpiGbl_EndBlkCommentListTail = NULL; } Index: projects/clang1000-import/sys/contrib/dev/acpica/components/events/evevent.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/components/events/evevent.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/components/events/evevent.c (revision 357966) @@ -1,449 +1,449 @@ /****************************************************************************** * * Module Name: evevent - Fixed Event handling and dispatch * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #include #define _COMPONENT ACPI_EVENTS ACPI_MODULE_NAME ("evevent") #if (!ACPI_REDUCED_HARDWARE) /* Entire module */ /* Local prototypes */ static ACPI_STATUS AcpiEvFixedEventInitialize ( void); static UINT32 AcpiEvFixedEventDispatch ( UINT32 Event); /******************************************************************************* * * FUNCTION: AcpiEvInitializeEvents * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Initialize global data structures for ACPI events (Fixed, GPE) * ******************************************************************************/ ACPI_STATUS AcpiEvInitializeEvents ( void) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (EvInitializeEvents); /* If Hardware Reduced flag is set, there are no fixed events */ if (AcpiGbl_ReducedHardware) { return_ACPI_STATUS (AE_OK); } /* * Initialize the Fixed and General Purpose Events. This is done prior to * enabling SCIs to prevent interrupts from occurring before the handlers * are installed. */ Status = AcpiEvFixedEventInitialize (); if (ACPI_FAILURE (Status)) { ACPI_EXCEPTION ((AE_INFO, Status, "Unable to initialize fixed events")); return_ACPI_STATUS (Status); } Status = AcpiEvGpeInitialize (); if (ACPI_FAILURE (Status)) { ACPI_EXCEPTION ((AE_INFO, Status, "Unable to initialize general purpose events")); return_ACPI_STATUS (Status); } return_ACPI_STATUS (Status); } /******************************************************************************* * * FUNCTION: AcpiEvInstallXruptHandlers * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Install interrupt handlers for the SCI and Global Lock * ******************************************************************************/ ACPI_STATUS AcpiEvInstallXruptHandlers ( void) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (EvInstallXruptHandlers); /* If Hardware Reduced flag is set, there is no ACPI h/w */ if (AcpiGbl_ReducedHardware) { return_ACPI_STATUS (AE_OK); } /* Install the SCI handler */ Status = AcpiEvInstallSciHandler (); if (ACPI_FAILURE (Status)) { ACPI_EXCEPTION ((AE_INFO, Status, "Unable to install System Control Interrupt handler")); return_ACPI_STATUS (Status); } /* Install the handler for the Global Lock */ Status = AcpiEvInitGlobalLockHandler (); if (ACPI_FAILURE (Status)) { ACPI_EXCEPTION ((AE_INFO, Status, "Unable to initialize Global Lock handler")); return_ACPI_STATUS (Status); } AcpiGbl_EventsInitialized = TRUE; return_ACPI_STATUS (Status); } /******************************************************************************* * * FUNCTION: AcpiEvFixedEventInitialize * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Install the fixed event handlers and disable all fixed events. * ******************************************************************************/ static ACPI_STATUS AcpiEvFixedEventInitialize ( void) { UINT32 i; ACPI_STATUS Status; /* * Initialize the structure that keeps track of fixed event handlers and - * enable the fixed events. + * disable all of the fixed events. */ for (i = 0; i < ACPI_NUM_FIXED_EVENTS; i++) { AcpiGbl_FixedEventHandlers[i].Handler = NULL; AcpiGbl_FixedEventHandlers[i].Context = NULL; /* Disable the fixed event */ if (AcpiGbl_FixedEventInfo[i].EnableRegisterId != 0xFF) { Status = AcpiWriteBitRegister ( AcpiGbl_FixedEventInfo[i].EnableRegisterId, ACPI_DISABLE_EVENT); if (ACPI_FAILURE (Status)) { return (Status); } } } return (AE_OK); } /******************************************************************************* * * FUNCTION: AcpiEvFixedEventDetect * * PARAMETERS: None * * RETURN: INTERRUPT_HANDLED or INTERRUPT_NOT_HANDLED * * DESCRIPTION: Checks the PM status register for active fixed events * ******************************************************************************/ UINT32 AcpiEvFixedEventDetect ( void) { UINT32 IntStatus = ACPI_INTERRUPT_NOT_HANDLED; UINT32 FixedStatus; UINT32 FixedEnable; UINT32 i; ACPI_STATUS Status; ACPI_FUNCTION_NAME (EvFixedEventDetect); /* * Read the fixed feature status and enable registers, as all the cases * depend on their values. Ignore errors here. */ Status = AcpiHwRegisterRead (ACPI_REGISTER_PM1_STATUS, &FixedStatus); Status |= AcpiHwRegisterRead (ACPI_REGISTER_PM1_ENABLE, &FixedEnable); if (ACPI_FAILURE (Status)) { return (IntStatus); } ACPI_DEBUG_PRINT ((ACPI_DB_INTERRUPTS, "Fixed Event Block: Enable %08X Status %08X\n", FixedEnable, FixedStatus)); /* * Check for all possible Fixed Events and dispatch those that are active */ for (i = 0; i < ACPI_NUM_FIXED_EVENTS; i++) { /* Both the status and enable bits must be on for this event */ if ((FixedStatus & AcpiGbl_FixedEventInfo[i].StatusBitMask) && (FixedEnable & AcpiGbl_FixedEventInfo[i].EnableBitMask)) { /* * Found an active (signalled) event. Invoke global event * handler if present. */ AcpiFixedEventCount[i]++; if (AcpiGbl_GlobalEventHandler) { AcpiGbl_GlobalEventHandler (ACPI_EVENT_TYPE_FIXED, NULL, i, AcpiGbl_GlobalEventHandlerContext); } IntStatus |= AcpiEvFixedEventDispatch (i); } } return (IntStatus); } /******************************************************************************* * * FUNCTION: AcpiEvFixedEventDispatch * * PARAMETERS: Event - Event type * * RETURN: INTERRUPT_HANDLED or INTERRUPT_NOT_HANDLED * * DESCRIPTION: Clears the status bit for the requested event, calls the * handler that previously registered for the event. * NOTE: If there is no handler for the event, the event is * disabled to prevent further interrupts. * ******************************************************************************/ static UINT32 AcpiEvFixedEventDispatch ( UINT32 Event) { ACPI_FUNCTION_ENTRY (); /* Clear the status bit */ (void) AcpiWriteBitRegister ( AcpiGbl_FixedEventInfo[Event].StatusRegisterId, ACPI_CLEAR_STATUS); /* * Make sure that a handler exists. If not, report an error * and disable the event to prevent further interrupts. */ if (!AcpiGbl_FixedEventHandlers[Event].Handler) { (void) AcpiWriteBitRegister ( AcpiGbl_FixedEventInfo[Event].EnableRegisterId, ACPI_DISABLE_EVENT); ACPI_ERROR ((AE_INFO, "No installed handler for fixed event - %s (%u), disabling", AcpiUtGetEventName (Event), Event)); return (ACPI_INTERRUPT_NOT_HANDLED); } /* Invoke the Fixed Event handler */ return ((AcpiGbl_FixedEventHandlers[Event].Handler)( AcpiGbl_FixedEventHandlers[Event].Context)); } #endif /* !ACPI_REDUCED_HARDWARE */ Index: projects/clang1000-import/sys/contrib/dev/acpica/components/events/evxfgpe.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/components/events/evxfgpe.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/components/events/evxfgpe.c (revision 357966) @@ -1,1317 +1,1355 @@ /****************************************************************************** * * Module Name: evxfgpe - External Interfaces for General Purpose Events (GPEs) * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #define EXPORT_ACPI_INTERFACES #include #include #include #include #define _COMPONENT ACPI_EVENTS ACPI_MODULE_NAME ("evxfgpe") #if (!ACPI_REDUCED_HARDWARE) /* Entire module */ /******************************************************************************* * * FUNCTION: AcpiUpdateAllGpes * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Complete GPE initialization and enable all GPEs that have * associated _Lxx or _Exx methods and are not pointed to by any * device _PRW methods (this indicates that these GPEs are * generally intended for system or device wakeup. Such GPEs * have to be enabled directly when the devices whose _PRW * methods point to them are set up for wakeup signaling.) * * NOTE: Should be called after any GPEs are added to the system. Primarily, * after the system _PRW methods have been run, but also after a GPE Block * Device has been added or if any new GPE methods have been added via a * dynamic table load. * ******************************************************************************/ ACPI_STATUS AcpiUpdateAllGpes ( void) { ACPI_STATUS Status; BOOLEAN IsPollingNeeded = FALSE; ACPI_FUNCTION_TRACE (AcpiUpdateAllGpes); Status = AcpiUtAcquireMutex (ACPI_MTX_EVENTS); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } if (AcpiGbl_AllGpesInitialized) { goto UnlockAndExit; } Status = AcpiEvWalkGpeList (AcpiEvInitializeGpeBlock, &IsPollingNeeded); if (ACPI_SUCCESS (Status)) { AcpiGbl_AllGpesInitialized = TRUE; } UnlockAndExit: (void) AcpiUtReleaseMutex (ACPI_MTX_EVENTS); if (IsPollingNeeded && AcpiGbl_AllGpesInitialized) { /* Poll GPEs to handle already triggered events */ AcpiEvGpeDetect (AcpiGbl_GpeXruptListHead); } return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiUpdateAllGpes) /******************************************************************************* * * FUNCTION: AcpiEnableGpe * * PARAMETERS: GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * * RETURN: Status * * DESCRIPTION: Add a reference to a GPE. On the first reference, the GPE is * hardware-enabled. * ******************************************************************************/ ACPI_STATUS AcpiEnableGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber) { ACPI_STATUS Status = AE_BAD_PARAMETER; ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_CPU_FLAGS Flags; ACPI_FUNCTION_TRACE (AcpiEnableGpe); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* * Ensure that we have a valid GPE number and that there is some way * of handling the GPE (handler or a GPE method). In other words, we * won't allow a valid GPE to be enabled if there is no way to handle it. */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (GpeEventInfo) { if (ACPI_GPE_DISPATCH_TYPE (GpeEventInfo->Flags) != ACPI_GPE_DISPATCH_NONE) { Status = AcpiEvAddGpeReference (GpeEventInfo, TRUE); if (ACPI_SUCCESS (Status) && ACPI_GPE_IS_POLLING_NEEDED (GpeEventInfo)) { /* Poll edge-triggered GPEs to handle existing events */ AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); (void) AcpiEvDetectGpe ( GpeDevice, GpeEventInfo, GpeNumber); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); } } else { Status = AE_NO_HANDLER; } } AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiEnableGpe) /******************************************************************************* * * FUNCTION: AcpiDisableGpe * * PARAMETERS: GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * * RETURN: Status * * DESCRIPTION: Remove a reference to a GPE. When the last reference is * removed, only then is the GPE disabled (for runtime GPEs), or * the GPE mask bit disabled (for wake GPEs) * ******************************************************************************/ ACPI_STATUS AcpiDisableGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber) { ACPI_STATUS Status = AE_BAD_PARAMETER; ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_CPU_FLAGS Flags; ACPI_FUNCTION_TRACE (AcpiDisableGpe); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* Ensure that we have a valid GPE number */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (GpeEventInfo) { Status = AcpiEvRemoveGpeReference (GpeEventInfo); } AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiDisableGpe) /******************************************************************************* * * FUNCTION: AcpiSetGpe * * PARAMETERS: GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * Action - ACPI_GPE_ENABLE or ACPI_GPE_DISABLE * * RETURN: Status * * DESCRIPTION: Enable or disable an individual GPE. This function bypasses * the reference count mechanism used in the AcpiEnableGpe(), * AcpiDisableGpe() interfaces. * This API is typically used by the GPE raw handler mode driver * to switch between the polling mode and the interrupt mode after * the driver has enabled the GPE. * The APIs should be invoked in this order: * AcpiEnableGpe() <- Ensure the reference count > 0 * AcpiSetGpe(ACPI_GPE_DISABLE) <- Enter polling mode * AcpiSetGpe(ACPI_GPE_ENABLE) <- Leave polling mode * AcpiDisableGpe() <- Decrease the reference count * * Note: If a GPE is shared by 2 silicon components, then both the drivers * should support GPE polling mode or disabling the GPE for long period * for one driver may break the other. So use it with care since all * firmware _Lxx/_Exx handlers currently rely on the GPE interrupt mode. * ******************************************************************************/ ACPI_STATUS AcpiSetGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, UINT8 Action) { ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_STATUS Status; ACPI_CPU_FLAGS Flags; ACPI_FUNCTION_TRACE (AcpiSetGpe); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* Ensure that we have a valid GPE number */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (!GpeEventInfo) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } /* Perform the action */ switch (Action) { case ACPI_GPE_ENABLE: Status = AcpiHwLowSetGpe (GpeEventInfo, ACPI_GPE_ENABLE); GpeEventInfo->DisableForDispatch = FALSE; break; case ACPI_GPE_DISABLE: Status = AcpiHwLowSetGpe (GpeEventInfo, ACPI_GPE_DISABLE); GpeEventInfo->DisableForDispatch = TRUE; break; default: Status = AE_BAD_PARAMETER; break; } UnlockAndExit: AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiSetGpe) /******************************************************************************* * * FUNCTION: AcpiMaskGpe * * PARAMETERS: GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * IsMasked - Whether the GPE is masked or not * * RETURN: Status * * DESCRIPTION: Unconditionally mask/unmask the an individual GPE, ex., to * prevent a GPE flooding. * ******************************************************************************/ ACPI_STATUS AcpiMaskGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, BOOLEAN IsMasked) { ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_STATUS Status; ACPI_CPU_FLAGS Flags; ACPI_FUNCTION_TRACE (AcpiMaskGpe); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* Ensure that we have a valid GPE number */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (!GpeEventInfo) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } Status = AcpiEvMaskGpe (GpeEventInfo, IsMasked); UnlockAndExit: AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiMaskGpe) /******************************************************************************* * * FUNCTION: AcpiMarkGpeForWake * * PARAMETERS: GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * * RETURN: Status * * DESCRIPTION: Mark a GPE as having the ability to wake the system. Simply * sets the ACPI_GPE_CAN_WAKE flag. * * Some potential callers of AcpiSetupGpeForWake may know in advance that * there won't be any notify handlers installed for device wake notifications * from the given GPE (one example is a button GPE in Linux). For these cases, * AcpiMarkGpeForWake should be used instead of AcpiSetupGpeForWake. * This will set the ACPI_GPE_CAN_WAKE flag for the GPE without trying to * setup implicit wake notification for it (since there's no handler method). * ******************************************************************************/ ACPI_STATUS AcpiMarkGpeForWake ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber) { ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_STATUS Status = AE_BAD_PARAMETER; ACPI_CPU_FLAGS Flags; ACPI_FUNCTION_TRACE (AcpiMarkGpeForWake); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* Ensure that we have a valid GPE number */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (GpeEventInfo) { /* Mark the GPE as a possible wake event */ GpeEventInfo->Flags |= ACPI_GPE_CAN_WAKE; Status = AE_OK; } AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiMarkGpeForWake) /******************************************************************************* * * FUNCTION: AcpiSetupGpeForWake * * PARAMETERS: WakeDevice - Device associated with the GPE (via _PRW) * GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * * RETURN: Status * * DESCRIPTION: Mark a GPE as having the ability to wake the system. This * interface is intended to be used as the host executes the * _PRW methods (Power Resources for Wake) in the system tables. * Each _PRW appears under a Device Object (The WakeDevice), and * contains the info for the wake GPE associated with the * WakeDevice. * ******************************************************************************/ ACPI_STATUS AcpiSetupGpeForWake ( ACPI_HANDLE WakeDevice, ACPI_HANDLE GpeDevice, UINT32 GpeNumber) { ACPI_STATUS Status; ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_NAMESPACE_NODE *DeviceNode; ACPI_GPE_NOTIFY_INFO *Notify; ACPI_GPE_NOTIFY_INFO *NewNotify; ACPI_CPU_FLAGS Flags; ACPI_FUNCTION_TRACE (AcpiSetupGpeForWake); /* Parameter Validation */ if (!WakeDevice) { /* * By forcing WakeDevice to be valid, we automatically enable the * implicit notify feature on all hosts. */ return_ACPI_STATUS (AE_BAD_PARAMETER); } /* Handle root object case */ if (WakeDevice == ACPI_ROOT_OBJECT) { DeviceNode = AcpiGbl_RootNode; } else { DeviceNode = ACPI_CAST_PTR (ACPI_NAMESPACE_NODE, WakeDevice); } /* Validate WakeDevice is of type Device */ if (DeviceNode->Type != ACPI_TYPE_DEVICE) { return_ACPI_STATUS (AE_BAD_PARAMETER); } /* * Allocate a new notify object up front, in case it is needed. * Memory allocation while holding a spinlock is a big no-no * on some hosts. */ NewNotify = ACPI_ALLOCATE_ZEROED (sizeof (ACPI_GPE_NOTIFY_INFO)); if (!NewNotify) { return_ACPI_STATUS (AE_NO_MEMORY); } Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* Ensure that we have a valid GPE number */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (!GpeEventInfo) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } /* * If there is no method or handler for this GPE, then the * WakeDevice will be notified whenever this GPE fires. This is * known as an "implicit notify". Note: The GPE is assumed to be * level-triggered (for windows compatibility). */ if (ACPI_GPE_DISPATCH_TYPE (GpeEventInfo->Flags) == ACPI_GPE_DISPATCH_NONE) { /* * This is the first device for implicit notify on this GPE. * Just set the flags here, and enter the NOTIFY block below. */ GpeEventInfo->Flags = (ACPI_GPE_DISPATCH_NOTIFY | ACPI_GPE_LEVEL_TRIGGERED); } else if (GpeEventInfo->Flags & ACPI_GPE_AUTO_ENABLED) { /* * A reference to this GPE has been added during the GPE block * initialization, so drop it now to prevent the GPE from being * permanently enabled and clear its ACPI_GPE_AUTO_ENABLED flag. */ (void) AcpiEvRemoveGpeReference (GpeEventInfo); GpeEventInfo->Flags &= ~~ACPI_GPE_AUTO_ENABLED; } /* * If we already have an implicit notify on this GPE, add * this device to the notify list. */ if (ACPI_GPE_DISPATCH_TYPE (GpeEventInfo->Flags) == ACPI_GPE_DISPATCH_NOTIFY) { /* Ensure that the device is not already in the list */ Notify = GpeEventInfo->Dispatch.NotifyList; while (Notify) { if (Notify->DeviceNode == DeviceNode) { Status = AE_ALREADY_EXISTS; goto UnlockAndExit; } Notify = Notify->Next; } /* Add this device to the notify list for this GPE */ NewNotify->DeviceNode = DeviceNode; NewNotify->Next = GpeEventInfo->Dispatch.NotifyList; GpeEventInfo->Dispatch.NotifyList = NewNotify; NewNotify = NULL; } /* Mark the GPE as a possible wake event */ GpeEventInfo->Flags |= ACPI_GPE_CAN_WAKE; Status = AE_OK; UnlockAndExit: AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); /* Delete the notify object if it was not used above */ if (NewNotify) { ACPI_FREE (NewNotify); } return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiSetupGpeForWake) /******************************************************************************* * * FUNCTION: AcpiSetGpeWakeMask * * PARAMETERS: GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * Action - Enable or Disable * * RETURN: Status * * DESCRIPTION: Set or clear the GPE's wakeup enable mask bit. The GPE must * already be marked as a WAKE GPE. * ******************************************************************************/ ACPI_STATUS AcpiSetGpeWakeMask ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, UINT8 Action) { ACPI_STATUS Status = AE_OK; ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_GPE_REGISTER_INFO *GpeRegisterInfo; ACPI_CPU_FLAGS Flags; UINT32 RegisterBit; ACPI_FUNCTION_TRACE (AcpiSetGpeWakeMask); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* * Ensure that we have a valid GPE number and that this GPE is in * fact a wake GPE */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (!GpeEventInfo) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } if (!(GpeEventInfo->Flags & ACPI_GPE_CAN_WAKE)) { Status = AE_TYPE; goto UnlockAndExit; } GpeRegisterInfo = GpeEventInfo->RegisterInfo; if (!GpeRegisterInfo) { Status = AE_NOT_EXIST; goto UnlockAndExit; } RegisterBit = AcpiHwGetGpeRegisterBit (GpeEventInfo); /* Perform the action */ switch (Action) { case ACPI_GPE_ENABLE: ACPI_SET_BIT (GpeRegisterInfo->EnableForWake, (UINT8) RegisterBit); break; case ACPI_GPE_DISABLE: ACPI_CLEAR_BIT (GpeRegisterInfo->EnableForWake, (UINT8) RegisterBit); break; default: ACPI_ERROR ((AE_INFO, "%u, Invalid action", Action)); Status = AE_BAD_PARAMETER; break; } UnlockAndExit: AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiSetGpeWakeMask) /******************************************************************************* * * FUNCTION: AcpiClearGpe * * PARAMETERS: GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * * RETURN: Status * * DESCRIPTION: Clear an ACPI event (general purpose) * ******************************************************************************/ ACPI_STATUS AcpiClearGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber) { ACPI_STATUS Status = AE_OK; ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_CPU_FLAGS Flags; ACPI_FUNCTION_TRACE (AcpiClearGpe); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* Ensure that we have a valid GPE number */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (!GpeEventInfo) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } Status = AcpiHwClearGpe (GpeEventInfo); UnlockAndExit: AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiClearGpe) /******************************************************************************* * * FUNCTION: AcpiGetGpeStatus * * PARAMETERS: GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * EventStatus - Where the current status of the event * will be returned * * RETURN: Status * * DESCRIPTION: Get the current status of a GPE (signalled/not_signalled) * ******************************************************************************/ ACPI_STATUS AcpiGetGpeStatus ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, ACPI_EVENT_STATUS *EventStatus) { ACPI_STATUS Status = AE_OK; ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_CPU_FLAGS Flags; ACPI_FUNCTION_TRACE (AcpiGetGpeStatus); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* Ensure that we have a valid GPE number */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (!GpeEventInfo) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } /* Obtain status on the requested GPE number */ Status = AcpiHwGetGpeStatus (GpeEventInfo, EventStatus); UnlockAndExit: AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiGetGpeStatus) /******************************************************************************* * * FUNCTION: AcpiDispatchGpe * * PARAMETERS: GpeDevice - Parent GPE Device. NULL for GPE0/GPE1 * GpeNumber - GPE level within the GPE block * * RETURN: INTERRUPT_HANDLED or INTERRUPT_NOT_HANDLED * * DESCRIPTION: Detect and dispatch a General Purpose Event to either a function * (e.g. EC) or method (e.g. _Lxx/_Exx) handler. * ******************************************************************************/ UINT32 AcpiDispatchGpe( ACPI_HANDLE GpeDevice, UINT32 GpeNumber) { ACPI_FUNCTION_TRACE(acpi_dispatch_gpe); return (AcpiEvDetectGpe (GpeDevice, NULL, GpeNumber)); } ACPI_EXPORT_SYMBOL (AcpiDispatchGpe) /******************************************************************************* * * FUNCTION: AcpiFinishGpe * * PARAMETERS: GpeDevice - Namespace node for the GPE Block * (NULL for FADT defined GPEs) * GpeNumber - GPE level within the GPE block * * RETURN: Status * * DESCRIPTION: Clear and conditionally re-enable a GPE. This completes the GPE * processing. Intended for use by asynchronous host-installed * GPE handlers. The GPE is only re-enabled if the EnableForRun bit * is set in the GPE info. * ******************************************************************************/ ACPI_STATUS AcpiFinishGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber) { ACPI_GPE_EVENT_INFO *GpeEventInfo; ACPI_STATUS Status; ACPI_CPU_FLAGS Flags; ACPI_FUNCTION_TRACE (AcpiFinishGpe); Flags = AcpiOsAcquireLock (AcpiGbl_GpeLock); /* Ensure that we have a valid GPE number */ GpeEventInfo = AcpiEvGetGpeEventInfo (GpeDevice, GpeNumber); if (!GpeEventInfo) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } Status = AcpiEvFinishGpe (GpeEventInfo); UnlockAndExit: AcpiOsReleaseLock (AcpiGbl_GpeLock, Flags); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiFinishGpe) /****************************************************************************** * * FUNCTION: AcpiDisableAllGpes * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Disable and clear all GPEs in all GPE blocks * ******************************************************************************/ ACPI_STATUS AcpiDisableAllGpes ( void) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (AcpiDisableAllGpes); Status = AcpiUtAcquireMutex (ACPI_MTX_EVENTS); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } Status = AcpiHwDisableAllGpes (); (void) AcpiUtReleaseMutex (ACPI_MTX_EVENTS); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiDisableAllGpes) /****************************************************************************** * * FUNCTION: AcpiEnableAllRuntimeGpes * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Enable all "runtime" GPEs, in all GPE blocks * ******************************************************************************/ ACPI_STATUS AcpiEnableAllRuntimeGpes ( void) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (AcpiEnableAllRuntimeGpes); Status = AcpiUtAcquireMutex (ACPI_MTX_EVENTS); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } Status = AcpiHwEnableAllRuntimeGpes (); (void) AcpiUtReleaseMutex (ACPI_MTX_EVENTS); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiEnableAllRuntimeGpes) /****************************************************************************** * * FUNCTION: AcpiEnableAllWakeupGpes * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Enable all "wakeup" GPEs and disable all of the other GPEs, in * all GPE blocks. * ******************************************************************************/ ACPI_STATUS AcpiEnableAllWakeupGpes ( void) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (AcpiEnableAllWakeupGpes); Status = AcpiUtAcquireMutex (ACPI_MTX_EVENTS); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } Status = AcpiHwEnableAllWakeupGpes (); (void) AcpiUtReleaseMutex (ACPI_MTX_EVENTS); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiEnableAllWakeupGpes) +/****************************************************************************** + * + * FUNCTION: AcpiAnyGpeStatusSet + * + * PARAMETERS: None + * + * RETURN: Whether or not the status bit is set for any GPE + * + * DESCRIPTION: Check the status bits of all enabled GPEs and return TRUE if any + * of them is set or FALSE otherwise. + * + ******************************************************************************/ + +UINT32 +AcpiAnyGpeStatusSet ( + void) +{ + ACPI_STATUS Status; + UINT8 Ret; + + + ACPI_FUNCTION_TRACE (AcpiAnyGpeStatusSet); + + Status = AcpiUtAcquireMutex (ACPI_MTX_EVENTS); + if (ACPI_FAILURE (Status)) + { + return (FALSE); + } + + Ret = AcpiHwCheckAllGpes (); + (void) AcpiUtReleaseMutex (ACPI_MTX_EVENTS); + + return (Ret); +} + +ACPI_EXPORT_SYMBOL(AcpiAnyGpeStatusSet) + + /******************************************************************************* * * FUNCTION: AcpiInstallGpeBlock * * PARAMETERS: GpeDevice - Handle to the parent GPE Block Device * GpeBlockAddress - Address and SpaceID * RegisterCount - Number of GPE register pairs in the block * InterruptNumber - H/W interrupt for the block * * RETURN: Status * * DESCRIPTION: Create and Install a block of GPE registers. The GPEs are not * enabled here. * ******************************************************************************/ ACPI_STATUS AcpiInstallGpeBlock ( ACPI_HANDLE GpeDevice, ACPI_GENERIC_ADDRESS *GpeBlockAddress, UINT32 RegisterCount, UINT32 InterruptNumber) { ACPI_STATUS Status; ACPI_OPERAND_OBJECT *ObjDesc; ACPI_NAMESPACE_NODE *Node; ACPI_GPE_BLOCK_INFO *GpeBlock; ACPI_FUNCTION_TRACE (AcpiInstallGpeBlock); if ((!GpeDevice) || (!GpeBlockAddress) || (!RegisterCount)) { return_ACPI_STATUS (AE_BAD_PARAMETER); } Status = AcpiUtAcquireMutex (ACPI_MTX_NAMESPACE); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } Node = AcpiNsValidateHandle (GpeDevice); if (!Node) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } /* Validate the parent device */ if (Node->Type != ACPI_TYPE_DEVICE) { Status = AE_TYPE; goto UnlockAndExit; } if (Node->Object) { Status = AE_ALREADY_EXISTS; goto UnlockAndExit; } /* * For user-installed GPE Block Devices, the GpeBlockBaseNumber * is always zero */ Status = AcpiEvCreateGpeBlock (Node, GpeBlockAddress->Address, GpeBlockAddress->SpaceId, RegisterCount, 0, InterruptNumber, &GpeBlock); if (ACPI_FAILURE (Status)) { goto UnlockAndExit; } /* Install block in the DeviceObject attached to the node */ ObjDesc = AcpiNsGetAttachedObject (Node); if (!ObjDesc) { /* * No object, create a new one (Device nodes do not always have * an attached object) */ ObjDesc = AcpiUtCreateInternalObject (ACPI_TYPE_DEVICE); if (!ObjDesc) { Status = AE_NO_MEMORY; goto UnlockAndExit; } Status = AcpiNsAttachObject (Node, ObjDesc, ACPI_TYPE_DEVICE); /* Remove local reference to the object */ AcpiUtRemoveReference (ObjDesc); if (ACPI_FAILURE (Status)) { goto UnlockAndExit; } } /* Now install the GPE block in the DeviceObject */ ObjDesc->Device.GpeBlock = GpeBlock; UnlockAndExit: (void) AcpiUtReleaseMutex (ACPI_MTX_NAMESPACE); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiInstallGpeBlock) /******************************************************************************* * * FUNCTION: AcpiRemoveGpeBlock * * PARAMETERS: GpeDevice - Handle to the parent GPE Block Device * * RETURN: Status * * DESCRIPTION: Remove a previously installed block of GPE registers * ******************************************************************************/ ACPI_STATUS AcpiRemoveGpeBlock ( ACPI_HANDLE GpeDevice) { ACPI_OPERAND_OBJECT *ObjDesc; ACPI_STATUS Status; ACPI_NAMESPACE_NODE *Node; ACPI_FUNCTION_TRACE (AcpiRemoveGpeBlock); if (!GpeDevice) { return_ACPI_STATUS (AE_BAD_PARAMETER); } Status = AcpiUtAcquireMutex (ACPI_MTX_NAMESPACE); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } Node = AcpiNsValidateHandle (GpeDevice); if (!Node) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } /* Validate the parent device */ if (Node->Type != ACPI_TYPE_DEVICE) { Status = AE_TYPE; goto UnlockAndExit; } /* Get the DeviceObject attached to the node */ ObjDesc = AcpiNsGetAttachedObject (Node); if (!ObjDesc || !ObjDesc->Device.GpeBlock) { return_ACPI_STATUS (AE_NULL_OBJECT); } /* Delete the GPE block (but not the DeviceObject) */ Status = AcpiEvDeleteGpeBlock (ObjDesc->Device.GpeBlock); if (ACPI_SUCCESS (Status)) { ObjDesc->Device.GpeBlock = NULL; } UnlockAndExit: (void) AcpiUtReleaseMutex (ACPI_MTX_NAMESPACE); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiRemoveGpeBlock) /******************************************************************************* * * FUNCTION: AcpiGetGpeDevice * * PARAMETERS: Index - System GPE index (0-CurrentGpeCount) * GpeDevice - Where the parent GPE Device is returned * * RETURN: Status * * DESCRIPTION: Obtain the GPE device associated with the input index. A NULL * gpe device indicates that the gpe number is contained in one of * the FADT-defined gpe blocks. Otherwise, the GPE block device. * ******************************************************************************/ ACPI_STATUS AcpiGetGpeDevice ( UINT32 Index, ACPI_HANDLE *GpeDevice) { ACPI_GPE_DEVICE_INFO Info; ACPI_STATUS Status; ACPI_FUNCTION_TRACE (AcpiGetGpeDevice); if (!GpeDevice) { return_ACPI_STATUS (AE_BAD_PARAMETER); } if (Index >= AcpiCurrentGpeCount) { return_ACPI_STATUS (AE_NOT_EXIST); } /* Setup and walk the GPE list */ Info.Index = Index; Info.Status = AE_NOT_EXIST; Info.GpeDevice = NULL; Info.NextBlockBaseIndex = 0; Status = AcpiEvWalkGpeList (AcpiEvGetGpeDevice, &Info); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } *GpeDevice = ACPI_CAST_PTR (ACPI_HANDLE, Info.GpeDevice); return_ACPI_STATUS (Info.Status); } ACPI_EXPORT_SYMBOL (AcpiGetGpeDevice) #endif /* !ACPI_REDUCED_HARDWARE */ Index: projects/clang1000-import/sys/contrib/dev/acpica/components/hardware/hwgpe.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/components/hardware/hwgpe.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/components/hardware/hwgpe.c (revision 357966) @@ -1,718 +1,797 @@ /****************************************************************************** * * Module Name: hwgpe - Low level GPE enable/disable/clear functions * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #include #define _COMPONENT ACPI_HARDWARE ACPI_MODULE_NAME ("hwgpe") #if (!ACPI_REDUCED_HARDWARE) /* Entire module */ /* Local prototypes */ static ACPI_STATUS AcpiHwEnableWakeupGpeBlock ( ACPI_GPE_XRUPT_INFO *GpeXruptInfo, ACPI_GPE_BLOCK_INFO *GpeBlock, void *Context); static ACPI_STATUS AcpiHwGpeEnableWrite ( UINT8 EnableMask, ACPI_GPE_REGISTER_INFO *GpeRegisterInfo); /****************************************************************************** * * FUNCTION: AcpiHwGetGpeRegisterBit * * PARAMETERS: GpeEventInfo - Info block for the GPE * * RETURN: Register mask with a one in the GPE bit position * * DESCRIPTION: Compute the register mask for this GPE. One bit is set in the * correct position for the input GPE. * ******************************************************************************/ UINT32 AcpiHwGetGpeRegisterBit ( ACPI_GPE_EVENT_INFO *GpeEventInfo) { return ((UINT32) 1 << (GpeEventInfo->GpeNumber - GpeEventInfo->RegisterInfo->BaseGpeNumber)); } /****************************************************************************** * * FUNCTION: AcpiHwLowSetGpe * * PARAMETERS: GpeEventInfo - Info block for the GPE to be disabled * Action - Enable or disable * * RETURN: Status * * DESCRIPTION: Enable or disable a single GPE in the parent enable register. * The EnableMask field of the involved GPE register must be * updated by the caller if necessary. * ******************************************************************************/ ACPI_STATUS AcpiHwLowSetGpe ( ACPI_GPE_EVENT_INFO *GpeEventInfo, UINT32 Action) { ACPI_GPE_REGISTER_INFO *GpeRegisterInfo; ACPI_STATUS Status = AE_OK; UINT64 EnableMask; UINT32 RegisterBit; ACPI_FUNCTION_ENTRY (); /* Get the info block for the entire GPE register */ GpeRegisterInfo = GpeEventInfo->RegisterInfo; if (!GpeRegisterInfo) { return (AE_NOT_EXIST); } /* Get current value of the enable register that contains this GPE */ Status = AcpiHwRead (&EnableMask, &GpeRegisterInfo->EnableAddress); if (ACPI_FAILURE (Status)) { return (Status); } /* Set or clear just the bit that corresponds to this GPE */ RegisterBit = AcpiHwGetGpeRegisterBit (GpeEventInfo); switch (Action) { case ACPI_GPE_CONDITIONAL_ENABLE: /* Only enable if the corresponding EnableMask bit is set */ if (!(RegisterBit & GpeRegisterInfo->EnableMask)) { return (AE_BAD_PARAMETER); } /*lint -fallthrough */ case ACPI_GPE_ENABLE: ACPI_SET_BIT (EnableMask, RegisterBit); break; case ACPI_GPE_DISABLE: ACPI_CLEAR_BIT (EnableMask, RegisterBit); break; default: ACPI_ERROR ((AE_INFO, "Invalid GPE Action, %u", Action)); return (AE_BAD_PARAMETER); } if (!(RegisterBit & GpeRegisterInfo->MaskForRun)) { /* Write the updated enable mask */ Status = AcpiHwWrite (EnableMask, &GpeRegisterInfo->EnableAddress); } return (Status); } /****************************************************************************** * * FUNCTION: AcpiHwClearGpe * * PARAMETERS: GpeEventInfo - Info block for the GPE to be cleared * * RETURN: Status * * DESCRIPTION: Clear the status bit for a single GPE. * ******************************************************************************/ ACPI_STATUS AcpiHwClearGpe ( ACPI_GPE_EVENT_INFO *GpeEventInfo) { ACPI_GPE_REGISTER_INFO *GpeRegisterInfo; ACPI_STATUS Status; UINT32 RegisterBit; ACPI_FUNCTION_ENTRY (); /* Get the info block for the entire GPE register */ GpeRegisterInfo = GpeEventInfo->RegisterInfo; if (!GpeRegisterInfo) { return (AE_NOT_EXIST); } /* * Write a one to the appropriate bit in the status register to * clear this GPE. */ RegisterBit = AcpiHwGetGpeRegisterBit (GpeEventInfo); Status = AcpiHwWrite (RegisterBit, &GpeRegisterInfo->StatusAddress); return (Status); } /****************************************************************************** * * FUNCTION: AcpiHwGetGpeStatus * * PARAMETERS: GpeEventInfo - Info block for the GPE to queried * EventStatus - Where the GPE status is returned * * RETURN: Status * * DESCRIPTION: Return the status of a single GPE. * ******************************************************************************/ ACPI_STATUS AcpiHwGetGpeStatus ( ACPI_GPE_EVENT_INFO *GpeEventInfo, ACPI_EVENT_STATUS *EventStatus) { UINT64 InByte; UINT32 RegisterBit; ACPI_GPE_REGISTER_INFO *GpeRegisterInfo; ACPI_EVENT_STATUS LocalEventStatus = 0; ACPI_STATUS Status; ACPI_FUNCTION_ENTRY (); if (!EventStatus) { return (AE_BAD_PARAMETER); } /* GPE currently handled? */ if (ACPI_GPE_DISPATCH_TYPE (GpeEventInfo->Flags) != ACPI_GPE_DISPATCH_NONE) { LocalEventStatus |= ACPI_EVENT_FLAG_HAS_HANDLER; } /* Get the info block for the entire GPE register */ GpeRegisterInfo = GpeEventInfo->RegisterInfo; /* Get the register bitmask for this GPE */ RegisterBit = AcpiHwGetGpeRegisterBit (GpeEventInfo); /* GPE currently enabled? (enabled for runtime?) */ if (RegisterBit & GpeRegisterInfo->EnableForRun) { LocalEventStatus |= ACPI_EVENT_FLAG_ENABLED; } /* GPE currently masked? (masked for runtime?) */ if (RegisterBit & GpeRegisterInfo->MaskForRun) { LocalEventStatus |= ACPI_EVENT_FLAG_MASKED; } /* GPE enabled for wake? */ if (RegisterBit & GpeRegisterInfo->EnableForWake) { LocalEventStatus |= ACPI_EVENT_FLAG_WAKE_ENABLED; } /* GPE currently enabled (enable bit == 1)? */ Status = AcpiHwRead (&InByte, &GpeRegisterInfo->EnableAddress); if (ACPI_FAILURE (Status)) { return (Status); } if (RegisterBit & InByte) { LocalEventStatus |= ACPI_EVENT_FLAG_ENABLE_SET; } /* GPE currently active (status bit == 1)? */ Status = AcpiHwRead (&InByte, &GpeRegisterInfo->StatusAddress); if (ACPI_FAILURE (Status)) { return (Status); } if (RegisterBit & InByte) { LocalEventStatus |= ACPI_EVENT_FLAG_STATUS_SET; } /* Set return value */ (*EventStatus) = LocalEventStatus; return (AE_OK); } /****************************************************************************** * * FUNCTION: AcpiHwGpeEnableWrite * * PARAMETERS: EnableMask - Bit mask to write to the GPE register * GpeRegisterInfo - Gpe Register info * * RETURN: Status * * DESCRIPTION: Write the enable mask byte to the given GPE register. * ******************************************************************************/ static ACPI_STATUS AcpiHwGpeEnableWrite ( UINT8 EnableMask, ACPI_GPE_REGISTER_INFO *GpeRegisterInfo) { ACPI_STATUS Status; GpeRegisterInfo->EnableMask = EnableMask; Status = AcpiHwWrite (EnableMask, &GpeRegisterInfo->EnableAddress); return (Status); } /****************************************************************************** * * FUNCTION: AcpiHwDisableGpeBlock * * PARAMETERS: GpeXruptInfo - GPE Interrupt info * GpeBlock - Gpe Block info * * RETURN: Status * * DESCRIPTION: Disable all GPEs within a single GPE block * ******************************************************************************/ ACPI_STATUS AcpiHwDisableGpeBlock ( ACPI_GPE_XRUPT_INFO *GpeXruptInfo, ACPI_GPE_BLOCK_INFO *GpeBlock, void *Context) { UINT32 i; ACPI_STATUS Status; /* Examine each GPE Register within the block */ for (i = 0; i < GpeBlock->RegisterCount; i++) { /* Disable all GPEs in this register */ Status = AcpiHwGpeEnableWrite (0x00, &GpeBlock->RegisterInfo[i]); if (ACPI_FAILURE (Status)) { return (Status); } } return (AE_OK); } /****************************************************************************** * * FUNCTION: AcpiHwClearGpeBlock * * PARAMETERS: GpeXruptInfo - GPE Interrupt info * GpeBlock - Gpe Block info * * RETURN: Status * * DESCRIPTION: Clear status bits for all GPEs within a single GPE block * ******************************************************************************/ ACPI_STATUS AcpiHwClearGpeBlock ( ACPI_GPE_XRUPT_INFO *GpeXruptInfo, ACPI_GPE_BLOCK_INFO *GpeBlock, void *Context) { UINT32 i; ACPI_STATUS Status; /* Examine each GPE Register within the block */ for (i = 0; i < GpeBlock->RegisterCount; i++) { /* Clear status on all GPEs in this register */ Status = AcpiHwWrite (0xFF, &GpeBlock->RegisterInfo[i].StatusAddress); if (ACPI_FAILURE (Status)) { return (Status); } } return (AE_OK); } /****************************************************************************** * * FUNCTION: AcpiHwEnableRuntimeGpeBlock * * PARAMETERS: GpeXruptInfo - GPE Interrupt info * GpeBlock - Gpe Block info * * RETURN: Status * * DESCRIPTION: Enable all "runtime" GPEs within a single GPE block. Includes * combination wake/run GPEs. * ******************************************************************************/ ACPI_STATUS AcpiHwEnableRuntimeGpeBlock ( ACPI_GPE_XRUPT_INFO *GpeXruptInfo, ACPI_GPE_BLOCK_INFO *GpeBlock, void *Context) { UINT32 i; ACPI_STATUS Status; ACPI_GPE_REGISTER_INFO *GpeRegisterInfo; UINT8 EnableMask; /* NOTE: assumes that all GPEs are currently disabled */ /* Examine each GPE Register within the block */ for (i = 0; i < GpeBlock->RegisterCount; i++) { GpeRegisterInfo = &GpeBlock->RegisterInfo[i]; if (!GpeRegisterInfo->EnableForRun) { continue; } /* Enable all "runtime" GPEs in this register */ EnableMask = GpeRegisterInfo->EnableForRun & ~GpeRegisterInfo->MaskForRun; Status = AcpiHwGpeEnableWrite (EnableMask, GpeRegisterInfo); if (ACPI_FAILURE (Status)) { return (Status); } } return (AE_OK); } /****************************************************************************** * * FUNCTION: AcpiHwEnableWakeupGpeBlock * * PARAMETERS: GpeXruptInfo - GPE Interrupt info * GpeBlock - Gpe Block info * * RETURN: Status * * DESCRIPTION: Enable all "wake" GPEs within a single GPE block. Includes * combination wake/run GPEs. * ******************************************************************************/ static ACPI_STATUS AcpiHwEnableWakeupGpeBlock ( ACPI_GPE_XRUPT_INFO *GpeXruptInfo, ACPI_GPE_BLOCK_INFO *GpeBlock, void *Context) { UINT32 i; ACPI_STATUS Status; ACPI_GPE_REGISTER_INFO *GpeRegisterInfo; /* Examine each GPE Register within the block */ for (i = 0; i < GpeBlock->RegisterCount; i++) { GpeRegisterInfo = &GpeBlock->RegisterInfo[i]; /* * Enable all "wake" GPEs in this register and disable the * remaining ones. */ Status = AcpiHwGpeEnableWrite (GpeRegisterInfo->EnableForWake, GpeRegisterInfo); if (ACPI_FAILURE (Status)) { return (Status); } } return (AE_OK); } /****************************************************************************** * + * FUNCTION: AcpiHwGetGpeBlockStatus + * + * PARAMETERS: GpeXruptInfo - GPE Interrupt info + * GpeBlock - Gpe Block info + * + * RETURN: Success + * + * DESCRIPTION: Produce a combined GPE status bits mask for the given block. + * + ******************************************************************************/ + +static ACPI_STATUS +AcpiHwGetGpeBlockStatus( + ACPI_GPE_XRUPT_INFO *GpeXruptInfo, + ACPI_GPE_BLOCK_INFO *GpeBlock, + void *RetPtr) +{ + ACPI_GPE_REGISTER_INFO *GpeRegisterInfo; + UINT64 InEnable; + UINT64 InStatus; + ACPI_STATUS Status; + UINT8 *Ret = RetPtr; + UINT32 i; + + + /* Examine each GPE Register within the block */ + + for (i = 0; i < GpeBlock->RegisterCount; i++) + { + GpeRegisterInfo = &GpeBlock->RegisterInfo[i]; + + Status = AcpiHwRead (&InEnable, &GpeRegisterInfo->EnableAddress); + if (ACPI_FAILURE (Status)) + { + continue; + } + + Status = AcpiHwRead (&InStatus, &GpeRegisterInfo->StatusAddress); + if (ACPI_FAILURE (Status)) + { + continue; + } + + *Ret |= InEnable & InStatus; + } + + return (AE_OK); +} + + +/****************************************************************************** + * * FUNCTION: AcpiHwDisableAllGpes * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Disable and clear all GPEs in all GPE blocks * ******************************************************************************/ ACPI_STATUS AcpiHwDisableAllGpes ( void) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (HwDisableAllGpes); Status = AcpiEvWalkGpeList (AcpiHwDisableGpeBlock, NULL); return_ACPI_STATUS (Status); } /****************************************************************************** * * FUNCTION: AcpiHwEnableAllRuntimeGpes * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Enable all "runtime" GPEs, in all GPE blocks * ******************************************************************************/ ACPI_STATUS AcpiHwEnableAllRuntimeGpes ( void) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (HwEnableAllRuntimeGpes); Status = AcpiEvWalkGpeList (AcpiHwEnableRuntimeGpeBlock, NULL); return_ACPI_STATUS (Status); } /****************************************************************************** * * FUNCTION: AcpiHwEnableAllWakeupGpes * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Enable all "wakeup" GPEs, in all GPE blocks * ******************************************************************************/ ACPI_STATUS AcpiHwEnableAllWakeupGpes ( void) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (HwEnableAllWakeupGpes); Status = AcpiEvWalkGpeList (AcpiHwEnableWakeupGpeBlock, NULL); return_ACPI_STATUS (Status); +} + + +/****************************************************************************** + * + * FUNCTION: AcpiHwCheckAllGpes + * + * PARAMETERS: None + * + * RETURN: Combined status of all GPEs + * + * DESCRIPTION: Check all enabled GPEs in all GPE blocks and return TRUE if the + * status bit is set for at least one of them of FALSE otherwise. + * + ******************************************************************************/ + +UINT8 +AcpiHwCheckAllGpes ( + void) +{ + UINT8 Ret = 0; + + + ACPI_FUNCTION_TRACE (AcpiHwCheckAllGpes); + + (void) AcpiEvWalkGpeList (AcpiHwGetGpeBlockStatus, &Ret); + return (Ret != 0); } #endif /* !ACPI_REDUCED_HARDWARE */ Index: projects/clang1000-import/sys/contrib/dev/acpica/components/hardware/hwsleep.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/components/hardware/hwsleep.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/components/hardware/hwsleep.c (revision 357966) @@ -1,471 +1,481 @@ /****************************************************************************** * * Name: hwsleep.c - ACPI Hardware Sleep/Wake Support functions for the * original/legacy sleep/PM registers. * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #define _COMPONENT ACPI_HARDWARE ACPI_MODULE_NAME ("hwsleep") #if (!ACPI_REDUCED_HARDWARE) /* Entire module */ /******************************************************************************* * * FUNCTION: AcpiHwLegacySleep * * PARAMETERS: SleepState - Which sleep state to enter * * RETURN: Status * * DESCRIPTION: Enter a system sleep state via the legacy FADT PM registers * THIS FUNCTION MUST BE CALLED WITH INTERRUPTS DISABLED * ******************************************************************************/ ACPI_STATUS AcpiHwLegacySleep ( UINT8 SleepState) { ACPI_BIT_REGISTER_INFO *SleepTypeRegInfo; ACPI_BIT_REGISTER_INFO *SleepEnableRegInfo; UINT32 Pm1aControl; UINT32 Pm1bControl; UINT32 InValue; ACPI_STATUS Status; ACPI_FUNCTION_TRACE (HwLegacySleep); SleepTypeRegInfo = AcpiHwGetBitRegisterInfo (ACPI_BITREG_SLEEP_TYPE); SleepEnableRegInfo = AcpiHwGetBitRegisterInfo (ACPI_BITREG_SLEEP_ENABLE); /* Clear wake status */ Status = AcpiWriteBitRegister (ACPI_BITREG_WAKE_STATUS, ACPI_CLEAR_STATUS); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* Disable all GPEs */ Status = AcpiHwDisableAllGpes (); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } Status = AcpiHwClearAcpiStatus(); if (ACPI_FAILURE(Status)) { return_ACPI_STATUS(Status); } AcpiGbl_SystemAwakeAndRunning = FALSE; /* Enable all wakeup GPEs */ Status = AcpiHwEnableAllWakeupGpes (); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* Get current value of PM1A control */ Status = AcpiHwRegisterRead (ACPI_REGISTER_PM1_CONTROL, &Pm1aControl); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } ACPI_DEBUG_PRINT ((ACPI_DB_INIT, "Entering sleep state [S%u]\n", SleepState)); /* Clear the SLP_EN and SLP_TYP fields */ Pm1aControl &= ~(SleepTypeRegInfo->AccessBitMask | SleepEnableRegInfo->AccessBitMask); Pm1bControl = Pm1aControl; /* Insert the SLP_TYP bits */ Pm1aControl |= (AcpiGbl_SleepTypeA << SleepTypeRegInfo->BitPosition); Pm1bControl |= (AcpiGbl_SleepTypeB << SleepTypeRegInfo->BitPosition); /* * We split the writes of SLP_TYP and SLP_EN to workaround * poorly implemented hardware. */ /* Write #1: write the SLP_TYP data to the PM1 Control registers */ Status = AcpiHwWritePm1Control (Pm1aControl, Pm1bControl); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* Insert the sleep enable (SLP_EN) bit */ Pm1aControl |= SleepEnableRegInfo->AccessBitMask; Pm1bControl |= SleepEnableRegInfo->AccessBitMask; /* Flush caches, as per ACPI specification */ ACPI_FLUSH_CPU_CACHE (); Status = AcpiOsEnterSleep (SleepState, Pm1aControl, Pm1bControl); if (Status == AE_CTRL_TERMINATE) { return_ACPI_STATUS (AE_OK); } if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* Write #2: Write both SLP_TYP + SLP_EN */ Status = AcpiHwWritePm1Control (Pm1aControl, Pm1bControl); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } if (SleepState > ACPI_STATE_S3) { /* * We wanted to sleep > S3, but it didn't happen (by virtue of the * fact that we are still executing!) * * Wait ten seconds, then try again. This is to get S4/S5 to work on * all machines. * * We wait so long to allow chipsets that poll this reg very slowly * to still read the right value. Ideally, this block would go * away entirely. */ AcpiOsStall (10 * ACPI_USEC_PER_SEC); Status = AcpiHwRegisterWrite (ACPI_REGISTER_PM1_CONTROL, SleepEnableRegInfo->AccessBitMask); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } } /* Wait for transition back to Working State */ do { Status = AcpiReadBitRegister (ACPI_BITREG_WAKE_STATUS, &InValue); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } } while (!InValue); return_ACPI_STATUS (AE_OK); } /******************************************************************************* * * FUNCTION: AcpiHwLegacyWakePrep * * PARAMETERS: SleepState - Which sleep state we just exited * * RETURN: Status * * DESCRIPTION: Perform the first state of OS-independent ACPI cleanup after a * sleep. * Called with interrupts ENABLED. * ******************************************************************************/ ACPI_STATUS AcpiHwLegacyWakePrep ( UINT8 SleepState) { ACPI_STATUS Status; ACPI_BIT_REGISTER_INFO *SleepTypeRegInfo; ACPI_BIT_REGISTER_INFO *SleepEnableRegInfo; UINT32 Pm1aControl; UINT32 Pm1bControl; ACPI_FUNCTION_TRACE (HwLegacyWakePrep); /* * Set SLP_TYPE and SLP_EN to state S0. * This is unclear from the ACPI Spec, but it is required * by some machines. */ Status = AcpiGetSleepTypeData (ACPI_STATE_S0, &AcpiGbl_SleepTypeA, &AcpiGbl_SleepTypeB); if (ACPI_SUCCESS (Status)) { SleepTypeRegInfo = AcpiHwGetBitRegisterInfo (ACPI_BITREG_SLEEP_TYPE); SleepEnableRegInfo = AcpiHwGetBitRegisterInfo (ACPI_BITREG_SLEEP_ENABLE); /* Get current value of PM1A control */ Status = AcpiHwRegisterRead (ACPI_REGISTER_PM1_CONTROL, &Pm1aControl); if (ACPI_SUCCESS (Status)) { /* Clear the SLP_EN and SLP_TYP fields */ Pm1aControl &= ~(SleepTypeRegInfo->AccessBitMask | SleepEnableRegInfo->AccessBitMask); Pm1bControl = Pm1aControl; /* Insert the SLP_TYP bits */ Pm1aControl |= (AcpiGbl_SleepTypeA << SleepTypeRegInfo->BitPosition); Pm1bControl |= (AcpiGbl_SleepTypeB << SleepTypeRegInfo->BitPosition); /* Write the control registers and ignore any errors */ (void) AcpiHwWritePm1Control (Pm1aControl, Pm1bControl); } } return_ACPI_STATUS (Status); } /******************************************************************************* * * FUNCTION: AcpiHwLegacyWake * * PARAMETERS: SleepState - Which sleep state we just exited * * RETURN: Status * * DESCRIPTION: Perform OS-independent ACPI cleanup after a sleep * Called with interrupts ENABLED. * ******************************************************************************/ ACPI_STATUS AcpiHwLegacyWake ( UINT8 SleepState) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (HwLegacyWake); /* Ensure EnterSleepStatePrep -> EnterSleepState ordering */ AcpiGbl_SleepTypeA = ACPI_SLEEP_TYPE_INVALID; AcpiHwExecuteSleepMethod (METHOD_PATHNAME__SST, ACPI_SST_WAKING); /* * GPEs must be enabled before _WAK is called as GPEs * might get fired there * * Restore the GPEs: * 1) Disable all GPEs * 2) Enable all runtime GPEs */ Status = AcpiHwDisableAllGpes (); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } Status = AcpiHwEnableAllRuntimeGpes (); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* * Now we can execute _WAK, etc. Some machines require that the GPEs * are enabled before the wake methods are executed. */ AcpiHwExecuteSleepMethod (METHOD_PATHNAME__WAK, SleepState); /* * Some BIOS code assumes that WAK_STS will be cleared on resume * and use it to determine whether the system is rebooting or * resuming. Clear WAK_STS for compatibility. */ (void) AcpiWriteBitRegister (ACPI_BITREG_WAKE_STATUS, ACPI_CLEAR_STATUS); AcpiGbl_SystemAwakeAndRunning = TRUE; /* Enable power button */ (void) AcpiWriteBitRegister( AcpiGbl_FixedEventInfo[ACPI_EVENT_POWER_BUTTON].EnableRegisterId, ACPI_ENABLE_EVENT); (void) AcpiWriteBitRegister( AcpiGbl_FixedEventInfo[ACPI_EVENT_POWER_BUTTON].StatusRegisterId, ACPI_CLEAR_STATUS); + /* Enable sleep button */ + + (void) AcpiWriteBitRegister ( + AcpiGbl_FixedEventInfo[ACPI_EVENT_SLEEP_BUTTON].EnableRegisterId, + ACPI_ENABLE_EVENT); + + (void) AcpiWriteBitRegister ( + AcpiGbl_FixedEventInfo[ACPI_EVENT_SLEEP_BUTTON].StatusRegisterId, + ACPI_CLEAR_STATUS); + AcpiHwExecuteSleepMethod (METHOD_PATHNAME__SST, ACPI_SST_WORKING); return_ACPI_STATUS (Status); } #endif /* !ACPI_REDUCED_HARDWARE */ Index: projects/clang1000-import/sys/contrib/dev/acpica/components/namespace/nsnames.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/components/namespace/nsnames.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/components/namespace/nsnames.c (revision 357966) @@ -1,684 +1,684 @@ /******************************************************************************* * * Module Name: nsnames - Name manipulation and search * ******************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #include #include #define _COMPONENT ACPI_NAMESPACE ACPI_MODULE_NAME ("nsnames") /* Local Prototypes */ static void AcpiNsNormalizePathname ( char *OriginalPath); /******************************************************************************* * * FUNCTION: AcpiNsGetExternalPathname * * PARAMETERS: Node - Namespace node whose pathname is needed * * RETURN: Pointer to storage containing the fully qualified name of * the node, In external format (name segments separated by path * separators.) * * DESCRIPTION: Used to obtain the full pathname to a namespace node, usually * for error and debug statements. * ******************************************************************************/ char * AcpiNsGetExternalPathname ( ACPI_NAMESPACE_NODE *Node) { char *NameBuffer; ACPI_FUNCTION_TRACE_PTR (NsGetExternalPathname, Node); NameBuffer = AcpiNsGetNormalizedPathname (Node, FALSE); return_PTR (NameBuffer); } /******************************************************************************* * * FUNCTION: AcpiNsGetPathnameLength * * PARAMETERS: Node - Namespace node * * RETURN: Length of path, including prefix * * DESCRIPTION: Get the length of the pathname string for this node * ******************************************************************************/ ACPI_SIZE AcpiNsGetPathnameLength ( ACPI_NAMESPACE_NODE *Node) { ACPI_SIZE Size; /* Validate the Node */ if (ACPI_GET_DESCRIPTOR_TYPE (Node) != ACPI_DESC_TYPE_NAMED) { ACPI_ERROR ((AE_INFO, "Invalid/cached reference target node: %p, descriptor type %d", Node, ACPI_GET_DESCRIPTOR_TYPE (Node))); return (0); } Size = AcpiNsBuildNormalizedPath (Node, NULL, 0, FALSE); return (Size); } /******************************************************************************* * * FUNCTION: AcpiNsHandleToName * * PARAMETERS: TargetHandle - Handle of named object whose name is * to be found * Buffer - Where the name is returned * * RETURN: Status, Buffer is filled with name if status is AE_OK * * DESCRIPTION: Build and return a full namespace name * ******************************************************************************/ ACPI_STATUS AcpiNsHandleToName ( ACPI_HANDLE TargetHandle, ACPI_BUFFER *Buffer) { ACPI_STATUS Status; ACPI_NAMESPACE_NODE *Node; const char *NodeName; ACPI_FUNCTION_TRACE_PTR (NsHandleToName, TargetHandle); Node = AcpiNsValidateHandle (TargetHandle); if (!Node) { return_ACPI_STATUS (AE_BAD_PARAMETER); } /* Validate/Allocate/Clear caller buffer */ Status = AcpiUtInitializeBuffer (Buffer, ACPI_PATH_SEGMENT_LENGTH); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* Just copy the ACPI name from the Node and zero terminate it */ NodeName = AcpiUtGetNodeName (Node); ACPI_COPY_NAMESEG (Buffer->Pointer, NodeName); ((char *) Buffer->Pointer) [ACPI_NAMESEG_SIZE] = 0; ACPI_DEBUG_PRINT ((ACPI_DB_EXEC, "%4.4s\n", (char *) Buffer->Pointer)); return_ACPI_STATUS (AE_OK); } /******************************************************************************* * * FUNCTION: AcpiNsHandleToPathname * * PARAMETERS: TargetHandle - Handle of named object whose name is * to be found * Buffer - Where the pathname is returned * NoTrailing - Remove trailing '_' for each name * segment * * RETURN: Status, Buffer is filled with pathname if status is AE_OK * * DESCRIPTION: Build and return a full namespace pathname * ******************************************************************************/ ACPI_STATUS AcpiNsHandleToPathname ( ACPI_HANDLE TargetHandle, ACPI_BUFFER *Buffer, BOOLEAN NoTrailing) { ACPI_STATUS Status; ACPI_NAMESPACE_NODE *Node; ACPI_SIZE RequiredSize; ACPI_FUNCTION_TRACE_PTR (NsHandleToPathname, TargetHandle); Node = AcpiNsValidateHandle (TargetHandle); if (!Node) { return_ACPI_STATUS (AE_BAD_PARAMETER); } /* Determine size required for the caller buffer */ RequiredSize = AcpiNsBuildNormalizedPath (Node, NULL, 0, NoTrailing); if (!RequiredSize) { return_ACPI_STATUS (AE_BAD_PARAMETER); } /* Validate/Allocate/Clear caller buffer */ Status = AcpiUtInitializeBuffer (Buffer, RequiredSize); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* Build the path in the caller buffer */ (void) AcpiNsBuildNormalizedPath (Node, Buffer->Pointer, - RequiredSize, NoTrailing); + (UINT32) RequiredSize, NoTrailing); ACPI_DEBUG_PRINT ((ACPI_DB_EXEC, "%s [%X]\n", (char *) Buffer->Pointer, (UINT32) RequiredSize)); return_ACPI_STATUS (AE_OK); } /******************************************************************************* * * FUNCTION: AcpiNsBuildNormalizedPath * * PARAMETERS: Node - Namespace node * FullPath - Where the path name is returned * PathSize - Size of returned path name buffer * NoTrailing - Remove trailing '_' from each name segment * * RETURN: Return 1 if the AML path is empty, otherwise returning (length * of pathname + 1) which means the 'FullPath' contains a trailing * null. * * DESCRIPTION: Build and return a full namespace pathname. * Note that if the size of 'FullPath' isn't large enough to * contain the namespace node's path name, the actual required * buffer length is returned, and it should be greater than * 'PathSize'. So callers are able to check the returning value * to determine the buffer size of 'FullPath'. * ******************************************************************************/ UINT32 AcpiNsBuildNormalizedPath ( ACPI_NAMESPACE_NODE *Node, char *FullPath, UINT32 PathSize, BOOLEAN NoTrailing) { UINT32 Length = 0, i; char Name[ACPI_NAMESEG_SIZE]; BOOLEAN DoNoTrailing; char c, *Left, *Right; ACPI_NAMESPACE_NODE *NextNode; ACPI_FUNCTION_TRACE_PTR (NsBuildNormalizedPath, Node); #define ACPI_PATH_PUT8(Path, Size, Byte, Length) \ do { \ if ((Length) < (Size)) \ { \ (Path)[(Length)] = (Byte); \ } \ (Length)++; \ } while (0) /* * Make sure the PathSize is correct, so that we don't need to * validate both FullPath and PathSize. */ if (!FullPath) { PathSize = 0; } if (!Node) { goto BuildTrailingNull; } NextNode = Node; while (NextNode && NextNode != AcpiGbl_RootNode) { if (NextNode != Node) { ACPI_PATH_PUT8(FullPath, PathSize, AML_DUAL_NAME_PREFIX, Length); } ACPI_MOVE_32_TO_32 (Name, &NextNode->Name); DoNoTrailing = NoTrailing; for (i = 0; i < 4; i++) { c = Name[4-i-1]; if (DoNoTrailing && c != '_') { DoNoTrailing = FALSE; } if (!DoNoTrailing) { ACPI_PATH_PUT8(FullPath, PathSize, c, Length); } } NextNode = NextNode->Parent; } ACPI_PATH_PUT8(FullPath, PathSize, AML_ROOT_PREFIX, Length); /* Reverse the path string */ if (Length <= PathSize) { Left = FullPath; Right = FullPath+Length - 1; while (Left < Right) { c = *Left; *Left++ = *Right; *Right-- = c; } } /* Append the trailing null */ BuildTrailingNull: ACPI_PATH_PUT8 (FullPath, PathSize, '\0', Length); #undef ACPI_PATH_PUT8 return_UINT32 (Length); } /******************************************************************************* * * FUNCTION: AcpiNsGetNormalizedPathname * * PARAMETERS: Node - Namespace node whose pathname is needed * NoTrailing - Remove trailing '_' from each name segment * * RETURN: Pointer to storage containing the fully qualified name of * the node, In external format (name segments separated by path * separators.) * * DESCRIPTION: Used to obtain the full pathname to a namespace node, usually * for error and debug statements. All trailing '_' will be * removed from the full pathname if 'NoTrailing' is specified.. * ******************************************************************************/ char * AcpiNsGetNormalizedPathname ( ACPI_NAMESPACE_NODE *Node, BOOLEAN NoTrailing) { char *NameBuffer; ACPI_SIZE Size; ACPI_FUNCTION_TRACE_PTR (NsGetNormalizedPathname, Node); /* Calculate required buffer size based on depth below root */ Size = AcpiNsBuildNormalizedPath (Node, NULL, 0, NoTrailing); if (!Size) { return_PTR (NULL); } /* Allocate a buffer to be returned to caller */ NameBuffer = ACPI_ALLOCATE_ZEROED (Size); if (!NameBuffer) { ACPI_ERROR ((AE_INFO, "Could not allocate %u bytes", (UINT32) Size)); return_PTR (NULL); } /* Build the path in the allocated buffer */ - (void) AcpiNsBuildNormalizedPath (Node, NameBuffer, Size, NoTrailing); + (void) AcpiNsBuildNormalizedPath (Node, NameBuffer, (UINT32) Size, NoTrailing); ACPI_DEBUG_PRINT_RAW ((ACPI_DB_NAMES, "%s: Path \"%s\"\n", ACPI_GET_FUNCTION_NAME, NameBuffer)); return_PTR (NameBuffer); } /******************************************************************************* * * FUNCTION: AcpiNsBuildPrefixedPathname * * PARAMETERS: PrefixScope - Scope/Path that prefixes the internal path * InternalPath - Name or path of the namespace node * * RETURN: None * * DESCRIPTION: Construct a fully qualified pathname from a concatenation of: * 1) Path associated with the PrefixScope namespace node * 2) External path representation of the Internal path * ******************************************************************************/ char * AcpiNsBuildPrefixedPathname ( ACPI_GENERIC_STATE *PrefixScope, const char *InternalPath) { ACPI_STATUS Status; char *FullPath = NULL; char *ExternalPath = NULL; char *PrefixPath = NULL; - UINT32 PrefixPathLength = 0; + ACPI_SIZE PrefixPathLength = 0; /* If there is a prefix, get the pathname to it */ if (PrefixScope && PrefixScope->Scope.Node) { PrefixPath = AcpiNsGetNormalizedPathname (PrefixScope->Scope.Node, TRUE); if (PrefixPath) { PrefixPathLength = strlen (PrefixPath); } } Status = AcpiNsExternalizeName (ACPI_UINT32_MAX, InternalPath, NULL, &ExternalPath); if (ACPI_FAILURE (Status)) { goto Cleanup; } /* Merge the prefix path and the path. 2 is for one dot and trailing null */ FullPath = ACPI_ALLOCATE_ZEROED ( PrefixPathLength + strlen (ExternalPath) + 2); if (!FullPath) { goto Cleanup; } /* Don't merge if the External path is already fully qualified */ if (PrefixPath && (*ExternalPath != '\\') && (*ExternalPath != '^')) { strcat (FullPath, PrefixPath); if (PrefixPath[1]) { strcat (FullPath, "."); } } AcpiNsNormalizePathname (ExternalPath); strcat (FullPath, ExternalPath); Cleanup: if (PrefixPath) { ACPI_FREE (PrefixPath); } if (ExternalPath) { ACPI_FREE (ExternalPath); } return (FullPath); } /******************************************************************************* * * FUNCTION: AcpiNsNormalizePathname * * PARAMETERS: OriginalPath - Path to be normalized, in External format * * RETURN: The original path is processed in-place * * DESCRIPTION: Remove trailing underscores from each element of a path. * * For example: \A___.B___.C___ becomes \A.B.C * ******************************************************************************/ static void AcpiNsNormalizePathname ( char *OriginalPath) { char *InputPath = OriginalPath; char *NewPathBuffer; char *NewPath; UINT32 i; /* Allocate a temp buffer in which to construct the new path */ NewPathBuffer = ACPI_ALLOCATE_ZEROED (strlen (InputPath) + 1); NewPath = NewPathBuffer; if (!NewPathBuffer) { return; } /* Special characters may appear at the beginning of the path */ if (*InputPath == '\\') { *NewPath = *InputPath; NewPath++; InputPath++; } while (*InputPath == '^') { *NewPath = *InputPath; NewPath++; InputPath++; } /* Remainder of the path */ while (*InputPath) { /* Do one nameseg at a time */ for (i = 0; (i < ACPI_NAMESEG_SIZE) && *InputPath; i++) { if ((i == 0) || (*InputPath != '_')) /* First char is allowed to be underscore */ { *NewPath = *InputPath; NewPath++; } InputPath++; } /* Dot means that there are more namesegs to come */ if (*InputPath == '.') { *NewPath = *InputPath; NewPath++; InputPath++; } } *NewPath = 0; strcpy (OriginalPath, NewPathBuffer); ACPI_FREE (NewPathBuffer); } Index: projects/clang1000-import/sys/contrib/dev/acpica/components/namespace/nsxfname.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/components/namespace/nsxfname.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/components/namespace/nsxfname.c (revision 357966) @@ -1,812 +1,812 @@ /****************************************************************************** * * Module Name: nsxfname - Public interfaces to the ACPI subsystem * ACPI Namespace oriented interfaces * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #define EXPORT_ACPI_INTERFACES #include #include #include #include #include #define _COMPONENT ACPI_NAMESPACE ACPI_MODULE_NAME ("nsxfname") /* Local prototypes */ static char * AcpiNsCopyDeviceId ( ACPI_PNP_DEVICE_ID *Dest, ACPI_PNP_DEVICE_ID *Source, char *StringArea); /****************************************************************************** * * FUNCTION: AcpiGetHandle * * PARAMETERS: Parent - Object to search under (search scope). * Pathname - Pointer to an asciiz string containing the * name * RetHandle - Where the return handle is returned * * RETURN: Status * * DESCRIPTION: This routine will search for a caller specified name in the * name space. The caller can restrict the search region by * specifying a non NULL parent. The parent value is itself a * namespace handle. * ******************************************************************************/ ACPI_STATUS AcpiGetHandle ( ACPI_HANDLE Parent, ACPI_STRING Pathname, ACPI_HANDLE *RetHandle) { ACPI_STATUS Status; ACPI_NAMESPACE_NODE *Node = NULL; ACPI_NAMESPACE_NODE *PrefixNode = NULL; ACPI_FUNCTION_ENTRY (); /* Parameter Validation */ if (!RetHandle || !Pathname) { return (AE_BAD_PARAMETER); } /* Convert a parent handle to a prefix node */ if (Parent) { PrefixNode = AcpiNsValidateHandle (Parent); if (!PrefixNode) { return (AE_BAD_PARAMETER); } } /* * Valid cases are: * 1) Fully qualified pathname * 2) Parent + Relative pathname * * Error for */ if (ACPI_IS_ROOT_PREFIX (Pathname[0])) { /* Pathname is fully qualified (starts with '\') */ /* Special case for root-only, since we can't search for it */ if (!strcmp (Pathname, ACPI_NS_ROOT_PATH)) { *RetHandle = ACPI_CAST_PTR (ACPI_HANDLE, AcpiGbl_RootNode); return (AE_OK); } } else if (!PrefixNode) { /* Relative path with null prefix is disallowed */ return (AE_BAD_PARAMETER); } /* Find the Node and convert to a handle */ Status = AcpiNsGetNode (PrefixNode, Pathname, ACPI_NS_NO_UPSEARCH, &Node); if (ACPI_SUCCESS (Status)) { *RetHandle = ACPI_CAST_PTR (ACPI_HANDLE, Node); } return (Status); } ACPI_EXPORT_SYMBOL (AcpiGetHandle) /****************************************************************************** * * FUNCTION: AcpiGetName * * PARAMETERS: Handle - Handle to be converted to a pathname * NameType - Full pathname or single segment * Buffer - Buffer for returned path * * RETURN: Pointer to a string containing the fully qualified Name. * * DESCRIPTION: This routine returns the fully qualified name associated with * the Handle parameter. This and the AcpiPathnameToHandle are * complementary functions. * ******************************************************************************/ ACPI_STATUS AcpiGetName ( ACPI_HANDLE Handle, UINT32 NameType, ACPI_BUFFER *Buffer) { ACPI_STATUS Status; /* Parameter validation */ if (NameType > ACPI_NAME_TYPE_MAX) { return (AE_BAD_PARAMETER); } Status = AcpiUtValidateBuffer (Buffer); if (ACPI_FAILURE (Status)) { return (Status); } /* * Wants the single segment ACPI name. * Validate handle and convert to a namespace Node */ Status = AcpiUtAcquireMutex (ACPI_MTX_NAMESPACE); if (ACPI_FAILURE (Status)) { return (Status); } if (NameType == ACPI_FULL_PATHNAME || NameType == ACPI_FULL_PATHNAME_NO_TRAILING) { /* Get the full pathname (From the namespace root) */ Status = AcpiNsHandleToPathname (Handle, Buffer, NameType == ACPI_FULL_PATHNAME ? FALSE : TRUE); } else { /* Get the single name */ Status = AcpiNsHandleToName (Handle, Buffer); } (void) AcpiUtReleaseMutex (ACPI_MTX_NAMESPACE); return (Status); } ACPI_EXPORT_SYMBOL (AcpiGetName) /****************************************************************************** * * FUNCTION: AcpiNsCopyDeviceId * * PARAMETERS: Dest - Pointer to the destination PNP_DEVICE_ID * Source - Pointer to the source PNP_DEVICE_ID * StringArea - Pointer to where to copy the dest string * * RETURN: Pointer to the next string area * * DESCRIPTION: Copy a single PNP_DEVICE_ID, including the string data. * ******************************************************************************/ static char * AcpiNsCopyDeviceId ( ACPI_PNP_DEVICE_ID *Dest, ACPI_PNP_DEVICE_ID *Source, char *StringArea) { /* Create the destination PNP_DEVICE_ID */ Dest->String = StringArea; Dest->Length = Source->Length; /* Copy actual string and return a pointer to the next string area */ memcpy (StringArea, Source->String, Source->Length); return (StringArea + Source->Length); } /****************************************************************************** * * FUNCTION: AcpiGetObjectInfo * * PARAMETERS: Handle - Object Handle * ReturnBuffer - Where the info is returned * * RETURN: Status * * DESCRIPTION: Returns information about an object as gleaned from the * namespace node and possibly by running several standard * control methods (Such as in the case of a device.) * * For Device and Processor objects, run the Device _HID, _UID, _CID, * _CLS, _ADR, _SxW, and _SxD methods. * * Note: Allocates the return buffer, must be freed by the caller. * * Note: This interface is intended to be used during the initial device * discovery namespace traversal. Therefore, no complex methods can be * executed, especially those that access operation regions. Therefore, do * not add any additional methods that could cause problems in this area. * Because of this reason support for the following methods has been removed: * 1) _SUB method was removed (11/2015) * 2) _STA method was removed (02/2018) * ******************************************************************************/ ACPI_STATUS AcpiGetObjectInfo ( ACPI_HANDLE Handle, ACPI_DEVICE_INFO **ReturnBuffer) { ACPI_NAMESPACE_NODE *Node; ACPI_DEVICE_INFO *Info; ACPI_PNP_DEVICE_ID_LIST *CidList = NULL; ACPI_PNP_DEVICE_ID *Hid = NULL; ACPI_PNP_DEVICE_ID *Uid = NULL; ACPI_PNP_DEVICE_ID *Cls = NULL; char *NextIdString; ACPI_OBJECT_TYPE Type; ACPI_NAME Name; UINT8 ParamCount= 0; UINT16 Valid = 0; UINT32 InfoSize; UINT32 i; ACPI_STATUS Status; /* Parameter validation */ if (!Handle || !ReturnBuffer) { return (AE_BAD_PARAMETER); } Status = AcpiUtAcquireMutex (ACPI_MTX_NAMESPACE); if (ACPI_FAILURE (Status)) { return (Status); } Node = AcpiNsValidateHandle (Handle); if (!Node) { (void) AcpiUtReleaseMutex (ACPI_MTX_NAMESPACE); return (AE_BAD_PARAMETER); } /* Get the namespace node data while the namespace is locked */ InfoSize = sizeof (ACPI_DEVICE_INFO); Type = Node->Type; Name = Node->Name.Integer; if (Node->Type == ACPI_TYPE_METHOD) { ParamCount = Node->Object->Method.ParamCount; } Status = AcpiUtReleaseMutex (ACPI_MTX_NAMESPACE); if (ACPI_FAILURE (Status)) { return (Status); } if ((Type == ACPI_TYPE_DEVICE) || (Type == ACPI_TYPE_PROCESSOR)) { /* * Get extra info for ACPI Device/Processor objects only: * Run the Device _HID, _UID, _CLS, and _CID methods. * * Note: none of these methods are required, so they may or may * not be present for this device. The Info->Valid bitfield is used * to indicate which methods were found and run successfully. */ /* Execute the Device._HID method */ Status = AcpiUtExecute_HID (Node, &Hid); if (ACPI_SUCCESS (Status)) { InfoSize += Hid->Length; Valid |= ACPI_VALID_HID; } /* Execute the Device._UID method */ Status = AcpiUtExecute_UID (Node, &Uid); if (ACPI_SUCCESS (Status)) { InfoSize += Uid->Length; Valid |= ACPI_VALID_UID; } /* Execute the Device._CID method */ Status = AcpiUtExecute_CID (Node, &CidList); if (ACPI_SUCCESS (Status)) { /* Add size of CID strings and CID pointer array */ InfoSize += (CidList->ListSize - sizeof (ACPI_PNP_DEVICE_ID_LIST)); Valid |= ACPI_VALID_CID; } /* Execute the Device._CLS method */ Status = AcpiUtExecute_CLS (Node, &Cls); if (ACPI_SUCCESS (Status)) { InfoSize += Cls->Length; Valid |= ACPI_VALID_CLS; } } /* * Now that we have the variable-length data, we can allocate the * return buffer */ Info = ACPI_ALLOCATE_ZEROED (InfoSize); if (!Info) { Status = AE_NO_MEMORY; goto Cleanup; } /* Get the fixed-length data */ if ((Type == ACPI_TYPE_DEVICE) || (Type == ACPI_TYPE_PROCESSOR)) { /* * Get extra info for ACPI Device/Processor objects only: * Run the _ADR and, SxW, and _SxD methods. * * Notes: none of these methods are required, so they may or may * not be present for this device. The Info->Valid bitfield is used * to indicate which methods were found and run successfully. */ /* Execute the Device._ADR method */ Status = AcpiUtEvaluateNumericObject (METHOD_NAME__ADR, Node, &Info->Address); if (ACPI_SUCCESS (Status)) { Valid |= ACPI_VALID_ADR; } /* Execute the Device._SxW methods */ Status = AcpiUtExecutePowerMethods (Node, AcpiGbl_LowestDstateNames, ACPI_NUM_SxW_METHODS, Info->LowestDstates); if (ACPI_SUCCESS (Status)) { Valid |= ACPI_VALID_SXWS; } /* Execute the Device._SxD methods */ Status = AcpiUtExecutePowerMethods (Node, AcpiGbl_HighestDstateNames, ACPI_NUM_SxD_METHODS, Info->HighestDstates); if (ACPI_SUCCESS (Status)) { Valid |= ACPI_VALID_SXDS; } } /* * Create a pointer to the string area of the return buffer. * Point to the end of the base ACPI_DEVICE_INFO structure. */ NextIdString = ACPI_CAST_PTR (char, Info->CompatibleIdList.Ids); if (CidList) { /* Point past the CID PNP_DEVICE_ID array */ NextIdString += ((ACPI_SIZE) CidList->Count * sizeof (ACPI_PNP_DEVICE_ID)); } /* * Copy the HID, UID, and CIDs to the return buffer. The variable-length * strings are copied to the reserved area at the end of the buffer. * * For HID and CID, check if the ID is a PCI Root Bridge. */ if (Hid) { NextIdString = AcpiNsCopyDeviceId (&Info->HardwareId, Hid, NextIdString); if (AcpiUtIsPciRootBridge (Hid->String)) { Info->Flags |= ACPI_PCI_ROOT_BRIDGE; } } if (Uid) { NextIdString = AcpiNsCopyDeviceId (&Info->UniqueId, Uid, NextIdString); } if (CidList) { Info->CompatibleIdList.Count = CidList->Count; Info->CompatibleIdList.ListSize = CidList->ListSize; /* Copy each CID */ for (i = 0; i < CidList->Count; i++) { NextIdString = AcpiNsCopyDeviceId (&Info->CompatibleIdList.Ids[i], &CidList->Ids[i], NextIdString); if (AcpiUtIsPciRootBridge (CidList->Ids[i].String)) { Info->Flags |= ACPI_PCI_ROOT_BRIDGE; } } } if (Cls) { (void) AcpiNsCopyDeviceId (&Info->ClassCode, Cls, NextIdString); } /* Copy the fixed-length data */ Info->InfoSize = InfoSize; Info->Type = Type; Info->Name = Name; Info->ParamCount = ParamCount; Info->Valid = Valid; *ReturnBuffer = Info; Status = AE_OK; Cleanup: if (Hid) { ACPI_FREE (Hid); } if (Uid) { ACPI_FREE (Uid); } if (CidList) { ACPI_FREE (CidList); } if (Cls) { ACPI_FREE (Cls); } return (Status); } ACPI_EXPORT_SYMBOL (AcpiGetObjectInfo) /****************************************************************************** * * FUNCTION: AcpiInstallMethod * * PARAMETERS: Buffer - An ACPI table containing one control method * * RETURN: Status * * DESCRIPTION: Install a control method into the namespace. If the method * name already exists in the namespace, it is overwritten. The * input buffer must contain a valid DSDT or SSDT containing a * single control method. * ******************************************************************************/ ACPI_STATUS AcpiInstallMethod ( UINT8 *Buffer) { ACPI_TABLE_HEADER *Table = ACPI_CAST_PTR (ACPI_TABLE_HEADER, Buffer); UINT8 *AmlBuffer; UINT8 *AmlStart; char *Path; ACPI_NAMESPACE_NODE *Node; ACPI_OPERAND_OBJECT *MethodObj; ACPI_PARSE_STATE ParserState; UINT32 AmlLength; UINT16 Opcode; UINT8 MethodFlags; ACPI_STATUS Status; /* Parameter validation */ if (!Buffer) { return (AE_BAD_PARAMETER); } /* Table must be a DSDT or SSDT */ if (!ACPI_COMPARE_NAMESEG (Table->Signature, ACPI_SIG_DSDT) && !ACPI_COMPARE_NAMESEG (Table->Signature, ACPI_SIG_SSDT)) { return (AE_BAD_HEADER); } /* First AML opcode in the table must be a control method */ ParserState.Aml = Buffer + sizeof (ACPI_TABLE_HEADER); Opcode = AcpiPsPeekOpcode (&ParserState); if (Opcode != AML_METHOD_OP) { return (AE_BAD_PARAMETER); } /* Extract method information from the raw AML */ ParserState.Aml += AcpiPsGetOpcodeSize (Opcode); ParserState.PkgEnd = AcpiPsGetNextPackageEnd (&ParserState); Path = AcpiPsGetNextNamestring (&ParserState); MethodFlags = *ParserState.Aml++; AmlStart = ParserState.Aml; - AmlLength = ACPI_PTR_DIFF (ParserState.PkgEnd, AmlStart); + AmlLength = (UINT32) ACPI_PTR_DIFF (ParserState.PkgEnd, AmlStart); /* * Allocate resources up-front. We don't want to have to delete a new * node from the namespace if we cannot allocate memory. */ AmlBuffer = ACPI_ALLOCATE (AmlLength); if (!AmlBuffer) { return (AE_NO_MEMORY); } MethodObj = AcpiUtCreateInternalObject (ACPI_TYPE_METHOD); if (!MethodObj) { ACPI_FREE (AmlBuffer); return (AE_NO_MEMORY); } /* Lock namespace for AcpiNsLookup, we may be creating a new node */ Status = AcpiUtAcquireMutex (ACPI_MTX_NAMESPACE); if (ACPI_FAILURE (Status)) { goto ErrorExit; } /* The lookup either returns an existing node or creates a new one */ Status = AcpiNsLookup (NULL, Path, ACPI_TYPE_METHOD, ACPI_IMODE_LOAD_PASS1, ACPI_NS_DONT_OPEN_SCOPE | ACPI_NS_ERROR_IF_FOUND, NULL, &Node); (void) AcpiUtReleaseMutex (ACPI_MTX_NAMESPACE); if (ACPI_FAILURE (Status)) /* NsLookup */ { if (Status != AE_ALREADY_EXISTS) { goto ErrorExit; } /* Node existed previously, make sure it is a method node */ if (Node->Type != ACPI_TYPE_METHOD) { Status = AE_TYPE; goto ErrorExit; } } /* Copy the method AML to the local buffer */ memcpy (AmlBuffer, AmlStart, AmlLength); /* Initialize the method object with the new method's information */ MethodObj->Method.AmlStart = AmlBuffer; MethodObj->Method.AmlLength = AmlLength; MethodObj->Method.ParamCount = (UINT8) (MethodFlags & AML_METHOD_ARG_COUNT); if (MethodFlags & AML_METHOD_SERIALIZED) { MethodObj->Method.InfoFlags = ACPI_METHOD_SERIALIZED; MethodObj->Method.SyncLevel = (UINT8) ((MethodFlags & AML_METHOD_SYNC_LEVEL) >> 4); } /* * Now that it is complete, we can attach the new method object to * the method Node (detaches/deletes any existing object) */ Status = AcpiNsAttachObject (Node, MethodObj, ACPI_TYPE_METHOD); /* * Flag indicates AML buffer is dynamic, must be deleted later. * Must be set only after attach above. */ Node->Flags |= ANOBJ_ALLOCATED_BUFFER; /* Remove local reference to the method object */ AcpiUtRemoveReference (MethodObj); return (Status); ErrorExit: ACPI_FREE (AmlBuffer); ACPI_FREE (MethodObj); return (Status); } ACPI_EXPORT_SYMBOL (AcpiInstallMethod) Index: projects/clang1000-import/sys/contrib/dev/acpica/components/tables/tbxface.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/components/tables/tbxface.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/components/tables/tbxface.c (revision 357966) @@ -1,747 +1,747 @@ /****************************************************************************** * * Module Name: tbxface - ACPI table-oriented external interfaces * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #define EXPORT_ACPI_INTERFACES #include #include #include #define _COMPONENT ACPI_TABLES ACPI_MODULE_NAME ("tbxface") /******************************************************************************* * * FUNCTION: AcpiAllocateRootTable * * PARAMETERS: InitialTableCount - Size of InitialTableArray, in number of * ACPI_TABLE_DESC structures * * RETURN: Status * * DESCRIPTION: Allocate a root table array. Used by iASL compiler and * AcpiInitializeTables. * ******************************************************************************/ ACPI_STATUS AcpiAllocateRootTable ( UINT32 InitialTableCount) { AcpiGbl_RootTableList.MaxTableCount = InitialTableCount; AcpiGbl_RootTableList.Flags = ACPI_ROOT_ALLOW_RESIZE; return (AcpiTbResizeRootTableList ()); } /******************************************************************************* * * FUNCTION: AcpiInitializeTables * * PARAMETERS: InitialTableArray - Pointer to an array of pre-allocated * ACPI_TABLE_DESC structures. If NULL, the * array is dynamically allocated. * InitialTableCount - Size of InitialTableArray, in number of * ACPI_TABLE_DESC structures * AllowResize - Flag to tell Table Manager if resize of * pre-allocated array is allowed. Ignored * if InitialTableArray is NULL. * * RETURN: Status * * DESCRIPTION: Initialize the table manager, get the RSDP and RSDT/XSDT. * * NOTE: Allows static allocation of the initial table array in order * to avoid the use of dynamic memory in confined environments * such as the kernel boot sequence where it may not be available. * * If the host OS memory managers are initialized, use NULL for * InitialTableArray, and the table will be dynamically allocated. * ******************************************************************************/ ACPI_STATUS ACPI_INIT_FUNCTION AcpiInitializeTables ( ACPI_TABLE_DESC *InitialTableArray, UINT32 InitialTableCount, BOOLEAN AllowResize) { ACPI_PHYSICAL_ADDRESS RsdpAddress; ACPI_STATUS Status; ACPI_FUNCTION_TRACE (AcpiInitializeTables); /* * Setup the Root Table Array and allocate the table array * if requested */ if (!InitialTableArray) { Status = AcpiAllocateRootTable (InitialTableCount); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } } else { /* Root Table Array has been statically allocated by the host */ memset (InitialTableArray, 0, (ACPI_SIZE) InitialTableCount * sizeof (ACPI_TABLE_DESC)); AcpiGbl_RootTableList.Tables = InitialTableArray; AcpiGbl_RootTableList.MaxTableCount = InitialTableCount; AcpiGbl_RootTableList.Flags = ACPI_ROOT_ORIGIN_UNKNOWN; if (AllowResize) { AcpiGbl_RootTableList.Flags |= ACPI_ROOT_ALLOW_RESIZE; } } /* Get the address of the RSDP */ RsdpAddress = AcpiOsGetRootPointer (); if (!RsdpAddress) { return_ACPI_STATUS (AE_NOT_FOUND); } /* * Get the root table (RSDT or XSDT) and extract all entries to the local * Root Table Array. This array contains the information of the RSDT/XSDT * in a common, more usable format. */ Status = AcpiTbParseRootTable (RsdpAddress); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL_INIT (AcpiInitializeTables) /******************************************************************************* * * FUNCTION: AcpiReallocateRootTable * * PARAMETERS: None * * RETURN: Status * * DESCRIPTION: Reallocate Root Table List into dynamic memory. Copies the * root list from the previously provided scratch area. Should * be called once dynamic memory allocation is available in the * kernel. * ******************************************************************************/ ACPI_STATUS ACPI_INIT_FUNCTION AcpiReallocateRootTable ( void) { ACPI_STATUS Status; ACPI_TABLE_DESC *TableDesc; UINT32 i, j; ACPI_FUNCTION_TRACE (AcpiReallocateRootTable); /* * If there are tables unverified, it is required to reallocate the * root table list to clean up invalid table entries. Otherwise only * reallocate the root table list if the host provided a static buffer * for the table array in the call to AcpiInitializeTables(). */ if ((AcpiGbl_RootTableList.Flags & ACPI_ROOT_ORIGIN_ALLOCATED) && AcpiGbl_EnableTableValidation) { return_ACPI_STATUS (AE_SUPPORT); } (void) AcpiUtAcquireMutex (ACPI_MTX_TABLES); /* * Ensure OS early boot logic, which is required by some hosts. If the * table state is reported to be wrong, developers should fix the * issue by invoking AcpiPutTable() for the reported table during the * early stage. */ for (i = 0; i < AcpiGbl_RootTableList.CurrentTableCount; ++i) { TableDesc = &AcpiGbl_RootTableList.Tables[i]; if (TableDesc->Pointer) { ACPI_ERROR ((AE_INFO, "Table [%4.4s] is not invalidated during early boot stage", TableDesc->Signature.Ascii)); } } if (!AcpiGbl_EnableTableValidation) { /* * Now it's safe to do full table validation. We can do deferred * table initialization here once the flag is set. */ AcpiGbl_EnableTableValidation = TRUE; for (i = 0; i < AcpiGbl_RootTableList.CurrentTableCount; ++i) { TableDesc = &AcpiGbl_RootTableList.Tables[i]; if (!(TableDesc->Flags & ACPI_TABLE_IS_VERIFIED)) { Status = AcpiTbVerifyTempTable (TableDesc, NULL, &j); if (ACPI_FAILURE (Status)) { AcpiTbUninstallTable (TableDesc); } } } } AcpiGbl_RootTableList.Flags |= ACPI_ROOT_ALLOW_RESIZE; Status = AcpiTbResizeRootTableList (); AcpiGbl_RootTableList.Flags |= ACPI_ROOT_ORIGIN_ALLOCATED; (void) AcpiUtReleaseMutex (ACPI_MTX_TABLES); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL_INIT (AcpiReallocateRootTable) /******************************************************************************* * * FUNCTION: AcpiGetTableHeader * * PARAMETERS: Signature - ACPI signature of needed table * Instance - Which instance (for SSDTs) - * OutTableHeader - The pointer to the table header to fill + * OutTableHeader - The pointer to the where the table header + * is returned * - * RETURN: Status and pointer to mapped table header + * RETURN: Status and a copy of the table header * - * DESCRIPTION: Finds an ACPI table header. - * - * NOTE: Caller is responsible in unmapping the header with - * AcpiOsUnmapMemory + * DESCRIPTION: Finds and returns an ACPI table header. Caller provides the + * memory where a copy of the header is to be returned + * (fixed length). * ******************************************************************************/ ACPI_STATUS AcpiGetTableHeader ( char *Signature, UINT32 Instance, ACPI_TABLE_HEADER *OutTableHeader) { UINT32 i; UINT32 j; ACPI_TABLE_HEADER *Header; /* Parameter validation */ if (!Signature || !OutTableHeader) { return (AE_BAD_PARAMETER); } /* Walk the root table list */ for (i = 0, j = 0; i < AcpiGbl_RootTableList.CurrentTableCount; i++) { if (!ACPI_COMPARE_NAMESEG ( &(AcpiGbl_RootTableList.Tables[i].Signature), Signature)) { continue; } if (++j < Instance) { continue; } if (!AcpiGbl_RootTableList.Tables[i].Pointer) { if ((AcpiGbl_RootTableList.Tables[i].Flags & ACPI_TABLE_ORIGIN_MASK) == ACPI_TABLE_ORIGIN_INTERNAL_PHYSICAL) { Header = AcpiOsMapMemory ( AcpiGbl_RootTableList.Tables[i].Address, sizeof (ACPI_TABLE_HEADER)); if (!Header) { return (AE_NO_MEMORY); } memcpy (OutTableHeader, Header, sizeof (ACPI_TABLE_HEADER)); AcpiOsUnmapMemory (Header, sizeof (ACPI_TABLE_HEADER)); } else { return (AE_NOT_FOUND); } } else { memcpy (OutTableHeader, AcpiGbl_RootTableList.Tables[i].Pointer, sizeof (ACPI_TABLE_HEADER)); } return (AE_OK); } return (AE_NOT_FOUND); } ACPI_EXPORT_SYMBOL (AcpiGetTableHeader) /******************************************************************************* * * FUNCTION: AcpiGetTable * * PARAMETERS: Signature - ACPI signature of needed table * Instance - Which instance (for SSDTs) * OutTable - Where the pointer to the table is returned * * RETURN: Status and pointer to the requested table * * DESCRIPTION: Finds and verifies an ACPI table. Table must be in the * RSDT/XSDT. * Note that an early stage AcpiGetTable() call must be paired * with an early stage AcpiPutTable() call. otherwise the table * pointer mapped by the early stage mapping implementation may be * erroneously unmapped by the late stage unmapping implementation * in an AcpiPutTable() invoked during the late stage. * ******************************************************************************/ ACPI_STATUS AcpiGetTable ( char *Signature, UINT32 Instance, ACPI_TABLE_HEADER **OutTable) { UINT32 i; UINT32 j; ACPI_STATUS Status = AE_NOT_FOUND; ACPI_TABLE_DESC *TableDesc; /* Parameter validation */ if (!Signature || !OutTable) { return (AE_BAD_PARAMETER); } /* * Note that the following line is required by some OSPMs, they only * check if the returned table is NULL instead of the returned status * to determined if this function is succeeded. */ *OutTable = NULL; (void) AcpiUtAcquireMutex (ACPI_MTX_TABLES); /* Walk the root table list */ for (i = 0, j = 0; i < AcpiGbl_RootTableList.CurrentTableCount; i++) { TableDesc = &AcpiGbl_RootTableList.Tables[i]; if (!ACPI_COMPARE_NAMESEG (&TableDesc->Signature, Signature)) { continue; } if (++j < Instance) { continue; } Status = AcpiTbGetTable (TableDesc, OutTable); break; } (void) AcpiUtReleaseMutex (ACPI_MTX_TABLES); return (Status); } ACPI_EXPORT_SYMBOL (AcpiGetTable) /******************************************************************************* * * FUNCTION: AcpiPutTable * * PARAMETERS: Table - The pointer to the table * * RETURN: None * * DESCRIPTION: Release a table returned by AcpiGetTable() and its clones. * Note that it is not safe if this function was invoked after an * uninstallation happened to the original table descriptor. * Currently there is no OSPMs' requirement to handle such * situations. * ******************************************************************************/ void AcpiPutTable ( ACPI_TABLE_HEADER *Table) { UINT32 i; ACPI_TABLE_DESC *TableDesc; ACPI_FUNCTION_TRACE (AcpiPutTable); if (!Table) { return_VOID; } (void) AcpiUtAcquireMutex (ACPI_MTX_TABLES); /* Walk the root table list */ for (i = 0; i < AcpiGbl_RootTableList.CurrentTableCount; i++) { TableDesc = &AcpiGbl_RootTableList.Tables[i]; if (TableDesc->Pointer != Table) { continue; } AcpiTbPutTable (TableDesc); break; } (void) AcpiUtReleaseMutex (ACPI_MTX_TABLES); return_VOID; } ACPI_EXPORT_SYMBOL (AcpiPutTable) /******************************************************************************* * * FUNCTION: AcpiGetTableByIndex * * PARAMETERS: TableIndex - Table index * OutTable - Where the pointer to the table is returned * * RETURN: Status and pointer to the requested table * * DESCRIPTION: Obtain a table by an index into the global table list. Used * internally also. * ******************************************************************************/ ACPI_STATUS AcpiGetTableByIndex ( UINT32 TableIndex, ACPI_TABLE_HEADER **OutTable) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (AcpiGetTableByIndex); /* Parameter validation */ if (!OutTable) { return_ACPI_STATUS (AE_BAD_PARAMETER); } /* * Note that the following line is required by some OSPMs, they only * check if the returned table is NULL instead of the returned status * to determined if this function is succeeded. */ *OutTable = NULL; (void) AcpiUtAcquireMutex (ACPI_MTX_TABLES); /* Validate index */ if (TableIndex >= AcpiGbl_RootTableList.CurrentTableCount) { Status = AE_BAD_PARAMETER; goto UnlockAndExit; } Status = AcpiTbGetTable ( &AcpiGbl_RootTableList.Tables[TableIndex], OutTable); UnlockAndExit: (void) AcpiUtReleaseMutex (ACPI_MTX_TABLES); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiGetTableByIndex) /******************************************************************************* * * FUNCTION: AcpiInstallTableHandler * * PARAMETERS: Handler - Table event handler * Context - Value passed to the handler on each event * * RETURN: Status * * DESCRIPTION: Install a global table event handler. * ******************************************************************************/ ACPI_STATUS AcpiInstallTableHandler ( ACPI_TABLE_HANDLER Handler, void *Context) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (AcpiInstallTableHandler); if (!Handler) { return_ACPI_STATUS (AE_BAD_PARAMETER); } Status = AcpiUtAcquireMutex (ACPI_MTX_EVENTS); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* Don't allow more than one handler */ if (AcpiGbl_TableHandler) { Status = AE_ALREADY_EXISTS; goto Cleanup; } /* Install the handler */ AcpiGbl_TableHandler = Handler; AcpiGbl_TableHandlerContext = Context; Cleanup: (void) AcpiUtReleaseMutex (ACPI_MTX_EVENTS); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiInstallTableHandler) /******************************************************************************* * * FUNCTION: AcpiRemoveTableHandler * * PARAMETERS: Handler - Table event handler that was installed * previously. * * RETURN: Status * * DESCRIPTION: Remove a table event handler * ******************************************************************************/ ACPI_STATUS AcpiRemoveTableHandler ( ACPI_TABLE_HANDLER Handler) { ACPI_STATUS Status; ACPI_FUNCTION_TRACE (AcpiRemoveTableHandler); Status = AcpiUtAcquireMutex (ACPI_MTX_EVENTS); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* Make sure that the installed handler is the same */ if (!Handler || Handler != AcpiGbl_TableHandler) { Status = AE_BAD_PARAMETER; goto Cleanup; } /* Remove the handler */ AcpiGbl_TableHandler = NULL; Cleanup: (void) AcpiUtReleaseMutex (ACPI_MTX_EVENTS); return_ACPI_STATUS (Status); } ACPI_EXPORT_SYMBOL (AcpiRemoveTableHandler) Index: projects/clang1000-import/sys/contrib/dev/acpica/components/utilities/utobject.c =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/components/utilities/utobject.c (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/components/utilities/utobject.c (revision 357966) @@ -1,892 +1,892 @@ /****************************************************************************** * * Module Name: utobject - ACPI object create/delete/size/cache routines * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #include #include #include #define _COMPONENT ACPI_UTILITIES ACPI_MODULE_NAME ("utobject") /* Local prototypes */ static ACPI_STATUS AcpiUtGetSimpleObjectSize ( ACPI_OPERAND_OBJECT *Obj, ACPI_SIZE *ObjLength); static ACPI_STATUS AcpiUtGetPackageObjectSize ( ACPI_OPERAND_OBJECT *Obj, ACPI_SIZE *ObjLength); static ACPI_STATUS AcpiUtGetElementLength ( UINT8 ObjectType, ACPI_OPERAND_OBJECT *SourceObject, ACPI_GENERIC_STATE *State, void *Context); /******************************************************************************* * * FUNCTION: AcpiUtCreateInternalObjectDbg * * PARAMETERS: ModuleName - Source file name of caller * LineNumber - Line number of caller * ComponentId - Component type of caller * Type - ACPI Type of the new object * * RETURN: A new internal object, null on failure * * DESCRIPTION: Create and initialize a new internal object. * * NOTE: We always allocate the worst-case object descriptor because * these objects are cached, and we want them to be - * one-size-satisifies-any-request. This in itself may not be + * one-size-satisfies-any-request. This in itself may not be * the most memory efficient, but the efficiency of the object * cache should more than make up for this! * ******************************************************************************/ ACPI_OPERAND_OBJECT * AcpiUtCreateInternalObjectDbg ( const char *ModuleName, UINT32 LineNumber, UINT32 ComponentId, ACPI_OBJECT_TYPE Type) { ACPI_OPERAND_OBJECT *Object; ACPI_OPERAND_OBJECT *SecondObject; ACPI_FUNCTION_TRACE_STR (UtCreateInternalObjectDbg, AcpiUtGetTypeName (Type)); /* Allocate the raw object descriptor */ Object = AcpiUtAllocateObjectDescDbg ( ModuleName, LineNumber, ComponentId); if (!Object) { return_PTR (NULL); } switch (Type) { case ACPI_TYPE_REGION: case ACPI_TYPE_BUFFER_FIELD: case ACPI_TYPE_LOCAL_BANK_FIELD: /* These types require a secondary object */ SecondObject = AcpiUtAllocateObjectDescDbg ( ModuleName, LineNumber, ComponentId); if (!SecondObject) { AcpiUtDeleteObjectDesc (Object); return_PTR (NULL); } SecondObject->Common.Type = ACPI_TYPE_LOCAL_EXTRA; SecondObject->Common.ReferenceCount = 1; /* Link the second object to the first */ Object->Common.NextObject = SecondObject; break; default: /* All others have no secondary object */ break; } /* Save the object type in the object descriptor */ Object->Common.Type = (UINT8) Type; /* Init the reference count */ Object->Common.ReferenceCount = 1; /* Any per-type initialization should go here */ return_PTR (Object); } /******************************************************************************* * * FUNCTION: AcpiUtCreatePackageObject * * PARAMETERS: Count - Number of package elements * * RETURN: Pointer to a new Package object, null on failure * * DESCRIPTION: Create a fully initialized package object * ******************************************************************************/ ACPI_OPERAND_OBJECT * AcpiUtCreatePackageObject ( UINT32 Count) { ACPI_OPERAND_OBJECT *PackageDesc; ACPI_OPERAND_OBJECT **PackageElements; ACPI_FUNCTION_TRACE_U32 (UtCreatePackageObject, Count); /* Create a new Package object */ PackageDesc = AcpiUtCreateInternalObject (ACPI_TYPE_PACKAGE); if (!PackageDesc) { return_PTR (NULL); } /* * Create the element array. Count+1 allows the array to be null * terminated. */ PackageElements = ACPI_ALLOCATE_ZEROED ( ((ACPI_SIZE) Count + 1) * sizeof (void *)); if (!PackageElements) { ACPI_FREE (PackageDesc); return_PTR (NULL); } PackageDesc->Package.Count = Count; PackageDesc->Package.Elements = PackageElements; return_PTR (PackageDesc); } /******************************************************************************* * * FUNCTION: AcpiUtCreateIntegerObject * * PARAMETERS: InitialValue - Initial value for the integer * * RETURN: Pointer to a new Integer object, null on failure * * DESCRIPTION: Create an initialized integer object * ******************************************************************************/ ACPI_OPERAND_OBJECT * AcpiUtCreateIntegerObject ( UINT64 InitialValue) { ACPI_OPERAND_OBJECT *IntegerDesc; ACPI_FUNCTION_TRACE (UtCreateIntegerObject); /* Create and initialize a new integer object */ IntegerDesc = AcpiUtCreateInternalObject (ACPI_TYPE_INTEGER); if (!IntegerDesc) { return_PTR (NULL); } IntegerDesc->Integer.Value = InitialValue; return_PTR (IntegerDesc); } /******************************************************************************* * * FUNCTION: AcpiUtCreateBufferObject * * PARAMETERS: BufferSize - Size of buffer to be created * * RETURN: Pointer to a new Buffer object, null on failure * * DESCRIPTION: Create a fully initialized buffer object * ******************************************************************************/ ACPI_OPERAND_OBJECT * AcpiUtCreateBufferObject ( ACPI_SIZE BufferSize) { ACPI_OPERAND_OBJECT *BufferDesc; UINT8 *Buffer = NULL; ACPI_FUNCTION_TRACE_U32 (UtCreateBufferObject, BufferSize); /* Create a new Buffer object */ BufferDesc = AcpiUtCreateInternalObject (ACPI_TYPE_BUFFER); if (!BufferDesc) { return_PTR (NULL); } /* Create an actual buffer only if size > 0 */ if (BufferSize > 0) { /* Allocate the actual buffer */ Buffer = ACPI_ALLOCATE_ZEROED (BufferSize); if (!Buffer) { ACPI_ERROR ((AE_INFO, "Could not allocate size %u", (UINT32) BufferSize)); AcpiUtRemoveReference (BufferDesc); return_PTR (NULL); } } /* Complete buffer object initialization */ BufferDesc->Buffer.Flags |= AOPOBJ_DATA_VALID; BufferDesc->Buffer.Pointer = Buffer; BufferDesc->Buffer.Length = (UINT32) BufferSize; /* Return the new buffer descriptor */ return_PTR (BufferDesc); } /******************************************************************************* * * FUNCTION: AcpiUtCreateStringObject * * PARAMETERS: StringSize - Size of string to be created. Does not * include NULL terminator, this is added * automatically. * * RETURN: Pointer to a new String object * * DESCRIPTION: Create a fully initialized string object * ******************************************************************************/ ACPI_OPERAND_OBJECT * AcpiUtCreateStringObject ( ACPI_SIZE StringSize) { ACPI_OPERAND_OBJECT *StringDesc; char *String; ACPI_FUNCTION_TRACE_U32 (UtCreateStringObject, StringSize); /* Create a new String object */ StringDesc = AcpiUtCreateInternalObject (ACPI_TYPE_STRING); if (!StringDesc) { return_PTR (NULL); } /* * Allocate the actual string buffer -- (Size + 1) for NULL terminator. * NOTE: Zero-length strings are NULL terminated */ String = ACPI_ALLOCATE_ZEROED (StringSize + 1); if (!String) { ACPI_ERROR ((AE_INFO, "Could not allocate size %u", (UINT32) StringSize)); AcpiUtRemoveReference (StringDesc); return_PTR (NULL); } /* Complete string object initialization */ StringDesc->String.Pointer = String; StringDesc->String.Length = (UINT32) StringSize; /* Return the new string descriptor */ return_PTR (StringDesc); } /******************************************************************************* * * FUNCTION: AcpiUtValidInternalObject * * PARAMETERS: Object - Object to be validated * * RETURN: TRUE if object is valid, FALSE otherwise * * DESCRIPTION: Validate a pointer to be of type ACPI_OPERAND_OBJECT * ******************************************************************************/ BOOLEAN AcpiUtValidInternalObject ( void *Object) { ACPI_FUNCTION_NAME (UtValidInternalObject); /* Check for a null pointer */ if (!Object) { ACPI_DEBUG_PRINT ((ACPI_DB_EXEC, "**** Null Object Ptr\n")); return (FALSE); } /* Check the descriptor type field */ switch (ACPI_GET_DESCRIPTOR_TYPE (Object)) { case ACPI_DESC_TYPE_OPERAND: /* The object appears to be a valid ACPI_OPERAND_OBJECT */ return (TRUE); default: ACPI_DEBUG_PRINT ((ACPI_DB_EXEC, "%p is not an ACPI operand obj [%s]\n", Object, AcpiUtGetDescriptorName (Object))); break; } return (FALSE); } /******************************************************************************* * * FUNCTION: AcpiUtAllocateObjectDescDbg * * PARAMETERS: ModuleName - Caller's module name (for error output) * LineNumber - Caller's line number (for error output) * ComponentId - Caller's component ID (for error output) * * RETURN: Pointer to newly allocated object descriptor. Null on error * * DESCRIPTION: Allocate a new object descriptor. Gracefully handle * error conditions. * ******************************************************************************/ void * AcpiUtAllocateObjectDescDbg ( const char *ModuleName, UINT32 LineNumber, UINT32 ComponentId) { ACPI_OPERAND_OBJECT *Object; ACPI_FUNCTION_TRACE (UtAllocateObjectDescDbg); Object = AcpiOsAcquireObject (AcpiGbl_OperandCache); if (!Object) { ACPI_ERROR ((ModuleName, LineNumber, "Could not allocate an object descriptor")); return_PTR (NULL); } /* Mark the descriptor type */ ACPI_SET_DESCRIPTOR_TYPE (Object, ACPI_DESC_TYPE_OPERAND); ACPI_DEBUG_PRINT ((ACPI_DB_ALLOCATIONS, "%p Size %X\n", Object, (UINT32) sizeof (ACPI_OPERAND_OBJECT))); return_PTR (Object); } /******************************************************************************* * * FUNCTION: AcpiUtDeleteObjectDesc * * PARAMETERS: Object - An Acpi internal object to be deleted * * RETURN: None. * * DESCRIPTION: Free an ACPI object descriptor or add it to the object cache * ******************************************************************************/ void AcpiUtDeleteObjectDesc ( ACPI_OPERAND_OBJECT *Object) { ACPI_FUNCTION_TRACE_PTR (UtDeleteObjectDesc, Object); /* Object must be of type ACPI_OPERAND_OBJECT */ if (ACPI_GET_DESCRIPTOR_TYPE (Object) != ACPI_DESC_TYPE_OPERAND) { ACPI_ERROR ((AE_INFO, "%p is not an ACPI Operand object [%s]", Object, AcpiUtGetDescriptorName (Object))); return_VOID; } (void) AcpiOsReleaseObject (AcpiGbl_OperandCache, Object); return_VOID; } /******************************************************************************* * * FUNCTION: AcpiUtGetSimpleObjectSize * * PARAMETERS: InternalObject - An ACPI operand object * ObjLength - Where the length is returned * * RETURN: Status * * DESCRIPTION: This function is called to determine the space required to * contain a simple object for return to an external user. * * The length includes the object structure plus any additional * needed space. * ******************************************************************************/ static ACPI_STATUS AcpiUtGetSimpleObjectSize ( ACPI_OPERAND_OBJECT *InternalObject, ACPI_SIZE *ObjLength) { ACPI_SIZE Length; ACPI_SIZE Size; ACPI_STATUS Status = AE_OK; ACPI_FUNCTION_TRACE_PTR (UtGetSimpleObjectSize, InternalObject); /* Start with the length of the (external) Acpi object */ Length = sizeof (ACPI_OBJECT); /* A NULL object is allowed, can be a legal uninitialized package element */ if (!InternalObject) { /* * Object is NULL, just return the length of ACPI_OBJECT * (A NULL ACPI_OBJECT is an object of all zeroes.) */ *ObjLength = ACPI_ROUND_UP_TO_NATIVE_WORD (Length); return_ACPI_STATUS (AE_OK); } /* A Namespace Node should never appear here */ if (ACPI_GET_DESCRIPTOR_TYPE (InternalObject) == ACPI_DESC_TYPE_NAMED) { /* A namespace node should never get here */ ACPI_ERROR ((AE_INFO, "Received a namespace node [%4.4s] " "where an operand object is required", ACPI_CAST_PTR (ACPI_NAMESPACE_NODE, InternalObject)->Name.Ascii)); return_ACPI_STATUS (AE_AML_INTERNAL); } /* * The final length depends on the object type * Strings and Buffers are packed right up against the parent object and * must be accessed bytewise or there may be alignment problems on * certain processors */ switch (InternalObject->Common.Type) { case ACPI_TYPE_STRING: Length += (ACPI_SIZE) InternalObject->String.Length + 1; break; case ACPI_TYPE_BUFFER: Length += (ACPI_SIZE) InternalObject->Buffer.Length; break; case ACPI_TYPE_INTEGER: case ACPI_TYPE_PROCESSOR: case ACPI_TYPE_POWER: /* No extra data for these types */ break; case ACPI_TYPE_LOCAL_REFERENCE: switch (InternalObject->Reference.Class) { case ACPI_REFCLASS_NAME: /* * Get the actual length of the full pathname to this object. * The reference will be converted to the pathname to the object */ Size = AcpiNsGetPathnameLength (InternalObject->Reference.Node); if (!Size) { return_ACPI_STATUS (AE_BAD_PARAMETER); } Length += ACPI_ROUND_UP_TO_NATIVE_WORD (Size); break; default: /* * No other reference opcodes are supported. * Notably, Locals and Args are not supported, but this may be * required eventually. */ ACPI_ERROR ((AE_INFO, "Cannot convert to external object - " "unsupported Reference Class [%s] 0x%X in object %p", AcpiUtGetReferenceName (InternalObject), InternalObject->Reference.Class, InternalObject)); Status = AE_TYPE; break; } break; default: ACPI_ERROR ((AE_INFO, "Cannot convert to external object - " "unsupported type [%s] 0x%X in object %p", AcpiUtGetObjectTypeName (InternalObject), InternalObject->Common.Type, InternalObject)); Status = AE_TYPE; break; } /* * Account for the space required by the object rounded up to the next * multiple of the machine word size. This keeps each object aligned * on a machine word boundary. (preventing alignment faults on some * machines.) */ *ObjLength = ACPI_ROUND_UP_TO_NATIVE_WORD (Length); return_ACPI_STATUS (Status); } /******************************************************************************* * * FUNCTION: AcpiUtGetElementLength * * PARAMETERS: ACPI_PKG_CALLBACK * * RETURN: Status * * DESCRIPTION: Get the length of one package element. * ******************************************************************************/ static ACPI_STATUS AcpiUtGetElementLength ( UINT8 ObjectType, ACPI_OPERAND_OBJECT *SourceObject, ACPI_GENERIC_STATE *State, void *Context) { ACPI_STATUS Status = AE_OK; ACPI_PKG_INFO *Info = (ACPI_PKG_INFO *) Context; ACPI_SIZE ObjectSpace; switch (ObjectType) { case ACPI_COPY_TYPE_SIMPLE: /* * Simple object - just get the size (Null object/entry is handled * here also) and sum it into the running package length */ Status = AcpiUtGetSimpleObjectSize (SourceObject, &ObjectSpace); if (ACPI_FAILURE (Status)) { return (Status); } Info->Length += ObjectSpace; break; case ACPI_COPY_TYPE_PACKAGE: /* Package object - nothing much to do here, let the walk handle it */ Info->NumPackages++; State->Pkg.ThisTargetObj = NULL; break; default: /* No other types allowed */ return (AE_BAD_PARAMETER); } return (Status); } /******************************************************************************* * * FUNCTION: AcpiUtGetPackageObjectSize * * PARAMETERS: InternalObject - An ACPI internal object * ObjLength - Where the length is returned * * RETURN: Status * * DESCRIPTION: This function is called to determine the space required to * contain a package object for return to an external user. * * This is moderately complex since a package contains other * objects including packages. * ******************************************************************************/ static ACPI_STATUS AcpiUtGetPackageObjectSize ( ACPI_OPERAND_OBJECT *InternalObject, ACPI_SIZE *ObjLength) { ACPI_STATUS Status; ACPI_PKG_INFO Info; ACPI_FUNCTION_TRACE_PTR (UtGetPackageObjectSize, InternalObject); Info.Length = 0; Info.ObjectSpace = 0; Info.NumPackages = 1; Status = AcpiUtWalkPackageTree ( InternalObject, NULL, AcpiUtGetElementLength, &Info); if (ACPI_FAILURE (Status)) { return_ACPI_STATUS (Status); } /* * We have handled all of the objects in all levels of the package. * just add the length of the package objects themselves. * Round up to the next machine word. */ Info.Length += ACPI_ROUND_UP_TO_NATIVE_WORD ( sizeof (ACPI_OBJECT)) * (ACPI_SIZE) Info.NumPackages; /* Return the total package length */ *ObjLength = Info.Length; return_ACPI_STATUS (Status); } /******************************************************************************* * * FUNCTION: AcpiUtGetObjectSize * * PARAMETERS: InternalObject - An ACPI internal object * ObjLength - Where the length will be returned * * RETURN: Status * * DESCRIPTION: This function is called to determine the space required to * contain an object for return to an API user. * ******************************************************************************/ ACPI_STATUS AcpiUtGetObjectSize ( ACPI_OPERAND_OBJECT *InternalObject, ACPI_SIZE *ObjLength) { ACPI_STATUS Status; ACPI_FUNCTION_ENTRY (); if ((ACPI_GET_DESCRIPTOR_TYPE (InternalObject) == ACPI_DESC_TYPE_OPERAND) && (InternalObject->Common.Type == ACPI_TYPE_PACKAGE)) { Status = AcpiUtGetPackageObjectSize (InternalObject, ObjLength); } else { Status = AcpiUtGetSimpleObjectSize (InternalObject, ObjLength); } return (Status); } Index: projects/clang1000-import/sys/contrib/dev/acpica/include/acconvert.h =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/include/acconvert.h (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/include/acconvert.h (revision 357966) @@ -1,312 +1,311 @@ /****************************************************************************** * * Module Name: acapps - common include for ACPI applications/tools * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #ifndef _ACCONVERT #define _ACCONVERT /* Definitions for comment state */ #define ASL_COMMENT_STANDARD 1 #define ASLCOMMENT_INLINE 2 #define ASL_COMMENT_OPEN_PAREN 3 #define ASL_COMMENT_CLOSE_PAREN 4 #define ASL_COMMENT_CLOSE_BRACE 5 /* Definitions for comment print function*/ #define AML_COMMENT_STANDARD 1 #define AMLCOMMENT_INLINE 2 #define AML_COMMENT_END_NODE 3 #define AML_NAMECOMMENT 4 #define AML_COMMENT_CLOSE_BRACE 5 #define AML_COMMENT_ENDBLK 6 #define AML_COMMENT_INCLUDE 7 #ifdef ACPI_ASL_COMPILER /* * cvcompiler */ void CvProcessComment ( ASL_COMMENT_STATE CurrentState, char *StringBuffer, int c1); void CvProcessCommentType2 ( ASL_COMMENT_STATE CurrentState, char *StringBuffer); UINT32 CvCalculateCommentLengths( ACPI_PARSE_OBJECT *Op); void CvProcessCommentState ( char input); char* CvAppendInlineComment ( char *InlineComment, char *ToAdd); void CvAddToCommentList ( char* ToAdd); void CvPlaceComment ( UINT8 Type, char *CommentString); UINT32 CvParseOpBlockType ( ACPI_PARSE_OBJECT *Op); ACPI_COMMENT_NODE* CvCommentNodeCalloc ( void); void CgWriteAmlDefBlockComment ( ACPI_PARSE_OBJECT *Op); void CgWriteOneAmlComment ( ACPI_PARSE_OBJECT *Op, char* CommentToPrint, UINT8 InputOption); void CgWriteAmlComment ( ACPI_PARSE_OBJECT *Op); /* * cvparser */ void CvInitFileTree ( ACPI_TABLE_HEADER *Table, - UINT8 *AmlStart, - UINT32 AmlLength); + FILE *RootFile); void CvClearOpComments ( ACPI_PARSE_OBJECT *Op); ACPI_FILE_NODE* CvFilenameExists ( char *Filename, ACPI_FILE_NODE *Head); void CvLabelFileNode ( ACPI_PARSE_OBJECT *Op); void CvCaptureListComments ( ACPI_PARSE_STATE *ParserState, ACPI_COMMENT_NODE *ListHead, ACPI_COMMENT_NODE *ListTail); void CvCaptureCommentsOnly ( ACPI_PARSE_STATE *ParserState); void CvCaptureComments ( ACPI_WALK_STATE *WalkState); void CvTransferComments ( ACPI_PARSE_OBJECT *Op); /* * cvdisasm */ void CvSwitchFiles ( UINT32 level, ACPI_PARSE_OBJECT *op); BOOLEAN CvFileHasSwitched ( ACPI_PARSE_OBJECT *Op); void CvCloseParenWriteComment ( ACPI_PARSE_OBJECT *Op, UINT32 Level); void CvCloseBraceWriteComment ( ACPI_PARSE_OBJECT *Op, UINT32 Level); void CvPrintOneCommentList ( ACPI_COMMENT_NODE *CommentList, UINT32 Level); void CvPrintOneCommentType ( ACPI_PARSE_OBJECT *Op, UINT8 CommentType, char* EndStr, UINT32 Level); #endif #endif /* _ACCONVERT */ Index: projects/clang1000-import/sys/contrib/dev/acpica/include/achware.h =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/include/achware.h (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/include/achware.h (revision 357966) @@ -1,335 +1,339 @@ /****************************************************************************** * * Name: achware.h -- hardware specific interfaces * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #ifndef __ACHWARE_H__ #define __ACHWARE_H__ /* Values for the _SST predefined method */ #define ACPI_SST_INDICATOR_OFF 0 #define ACPI_SST_WORKING 1 #define ACPI_SST_WAKING 2 #define ACPI_SST_SLEEPING 3 #define ACPI_SST_SLEEP_CONTEXT 4 /* * hwacpi - high level functions */ ACPI_STATUS AcpiHwSetMode ( UINT32 Mode); UINT32 AcpiHwGetMode ( void); /* * hwregs - ACPI Register I/O */ ACPI_STATUS AcpiHwValidateRegister ( ACPI_GENERIC_ADDRESS *Reg, UINT8 MaxBitWidth, UINT64 *Address); ACPI_STATUS AcpiHwRead ( UINT64 *Value, ACPI_GENERIC_ADDRESS *Reg); ACPI_STATUS AcpiHwWrite ( UINT64 Value, ACPI_GENERIC_ADDRESS *Reg); ACPI_BIT_REGISTER_INFO * AcpiHwGetBitRegisterInfo ( UINT32 RegisterId); ACPI_STATUS AcpiHwWritePm1Control ( UINT32 Pm1aControl, UINT32 Pm1bControl); ACPI_STATUS AcpiHwRegisterRead ( UINT32 RegisterId, UINT32 *ReturnValue); ACPI_STATUS AcpiHwRegisterWrite ( UINT32 RegisterId, UINT32 Value); ACPI_STATUS AcpiHwClearAcpiStatus ( void); /* * hwsleep - sleep/wake support (Legacy sleep registers) */ ACPI_STATUS AcpiHwLegacySleep ( UINT8 SleepState); ACPI_STATUS AcpiHwLegacyWakePrep ( UINT8 SleepState); ACPI_STATUS AcpiHwLegacyWake ( UINT8 SleepState); /* * hwesleep - sleep/wake support (Extended FADT-V5 sleep registers) */ void AcpiHwExecuteSleepMethod ( char *MethodName, UINT32 IntegerArgument); ACPI_STATUS AcpiHwExtendedSleep ( UINT8 SleepState); ACPI_STATUS AcpiHwExtendedWakePrep ( UINT8 SleepState); ACPI_STATUS AcpiHwExtendedWake ( UINT8 SleepState); /* * hwvalid - Port I/O with validation */ ACPI_STATUS AcpiHwReadPort ( ACPI_IO_ADDRESS Address, UINT32 *Value, UINT32 Width); ACPI_STATUS AcpiHwWritePort ( ACPI_IO_ADDRESS Address, UINT32 Value, UINT32 Width); /* * hwgpe - GPE support */ UINT32 AcpiHwGetGpeRegisterBit ( ACPI_GPE_EVENT_INFO *GpeEventInfo); ACPI_STATUS AcpiHwLowSetGpe ( ACPI_GPE_EVENT_INFO *GpeEventInfo, UINT32 Action); ACPI_STATUS AcpiHwDisableGpeBlock ( ACPI_GPE_XRUPT_INFO *GpeXruptInfo, ACPI_GPE_BLOCK_INFO *GpeBlock, void *Context); ACPI_STATUS AcpiHwClearGpe ( ACPI_GPE_EVENT_INFO *GpeEventInfo); ACPI_STATUS AcpiHwClearGpeBlock ( ACPI_GPE_XRUPT_INFO *GpeXruptInfo, ACPI_GPE_BLOCK_INFO *GpeBlock, void *Context); ACPI_STATUS AcpiHwGetGpeStatus ( ACPI_GPE_EVENT_INFO *GpeEventInfo, ACPI_EVENT_STATUS *EventStatus); ACPI_STATUS AcpiHwDisableAllGpes ( void); ACPI_STATUS AcpiHwEnableAllRuntimeGpes ( void); ACPI_STATUS AcpiHwEnableAllWakeupGpes ( void); +UINT8 +AcpiHwCheckAllGpes ( + void); + ACPI_STATUS AcpiHwEnableRuntimeGpeBlock ( ACPI_GPE_XRUPT_INFO *GpeXruptInfo, ACPI_GPE_BLOCK_INFO *GpeBlock, void *Context); /* * hwpci - PCI configuration support */ ACPI_STATUS AcpiHwDerivePciId ( ACPI_PCI_ID *PciId, ACPI_HANDLE RootPciDevice, ACPI_HANDLE PciRegion); #endif /* __ACHWARE_H__ */ Index: projects/clang1000-import/sys/contrib/dev/acpica/include/acmacros.h =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/include/acmacros.h (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/include/acmacros.h (revision 357966) @@ -1,647 +1,647 @@ /****************************************************************************** * * Name: acmacros.h - C macros for the entire subsystem. * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #ifndef __ACMACROS_H__ #define __ACMACROS_H__ /* * Extract data using a pointer. Any more than a byte and we * get into potential alignment issues -- see the STORE macros below. * Use with care. */ #define ACPI_CAST8(ptr) ACPI_CAST_PTR (UINT8, (ptr)) #define ACPI_CAST16(ptr) ACPI_CAST_PTR (UINT16, (ptr)) #define ACPI_CAST32(ptr) ACPI_CAST_PTR (UINT32, (ptr)) #define ACPI_CAST64(ptr) ACPI_CAST_PTR (UINT64, (ptr)) #define ACPI_GET8(ptr) (*ACPI_CAST8 (ptr)) #define ACPI_GET16(ptr) (*ACPI_CAST16 (ptr)) #define ACPI_GET32(ptr) (*ACPI_CAST32 (ptr)) #define ACPI_GET64(ptr) (*ACPI_CAST64 (ptr)) #define ACPI_SET8(ptr, val) (*ACPI_CAST8 (ptr) = (UINT8) (val)) #define ACPI_SET16(ptr, val) (*ACPI_CAST16 (ptr) = (UINT16) (val)) #define ACPI_SET32(ptr, val) (*ACPI_CAST32 (ptr) = (UINT32) (val)) #define ACPI_SET64(ptr, val) (*ACPI_CAST64 (ptr) = (UINT64) (val)) /* * printf() format helper. This macro is a workaround for the difficulties * with emitting 64-bit integers and 64-bit pointers with the same code * for both 32-bit and 64-bit hosts. */ #define ACPI_FORMAT_UINT64(i) ACPI_HIDWORD(i), ACPI_LODWORD(i) /* * Macros for moving data around to/from buffers that are possibly unaligned. * If the hardware supports the transfer of unaligned data, just do the store. * Otherwise, we have to move one byte at a time. */ #ifdef ACPI_BIG_ENDIAN /* * Macros for big-endian machines */ /* These macros reverse the bytes during the move, converting little-endian to big endian */ /* Big Endian <== Little Endian */ /* Hi...Lo Lo...Hi */ /* 16-bit source, 16/32/64 destination */ #define ACPI_MOVE_16_TO_16(d, s) {(( UINT8 *)(void *)(d))[0] = ((UINT8 *)(void *)(s))[1];\ (( UINT8 *)(void *)(d))[1] = ((UINT8 *)(void *)(s))[0];} #define ACPI_MOVE_16_TO_32(d, s) {(*(UINT32 *)(void *)(d))=0;\ ((UINT8 *)(void *)(d))[2] = ((UINT8 *)(void *)(s))[1];\ ((UINT8 *)(void *)(d))[3] = ((UINT8 *)(void *)(s))[0];} #define ACPI_MOVE_16_TO_64(d, s) {(*(UINT64 *)(void *)(d))=0;\ ((UINT8 *)(void *)(d))[6] = ((UINT8 *)(void *)(s))[1];\ ((UINT8 *)(void *)(d))[7] = ((UINT8 *)(void *)(s))[0];} /* 32-bit source, 16/32/64 destination */ #define ACPI_MOVE_32_TO_16(d, s) ACPI_MOVE_16_TO_16(d, s) /* Truncate to 16 */ #define ACPI_MOVE_32_TO_32(d, s) {(( UINT8 *)(void *)(d))[0] = ((UINT8 *)(void *)(s))[3];\ (( UINT8 *)(void *)(d))[1] = ((UINT8 *)(void *)(s))[2];\ (( UINT8 *)(void *)(d))[2] = ((UINT8 *)(void *)(s))[1];\ (( UINT8 *)(void *)(d))[3] = ((UINT8 *)(void *)(s))[0];} #define ACPI_MOVE_32_TO_64(d, s) {(*(UINT64 *)(void *)(d))=0;\ ((UINT8 *)(void *)(d))[4] = ((UINT8 *)(void *)(s))[3];\ ((UINT8 *)(void *)(d))[5] = ((UINT8 *)(void *)(s))[2];\ ((UINT8 *)(void *)(d))[6] = ((UINT8 *)(void *)(s))[1];\ ((UINT8 *)(void *)(d))[7] = ((UINT8 *)(void *)(s))[0];} /* 64-bit source, 16/32/64 destination */ #define ACPI_MOVE_64_TO_16(d, s) ACPI_MOVE_16_TO_16(d, s) /* Truncate to 16 */ #define ACPI_MOVE_64_TO_32(d, s) ACPI_MOVE_32_TO_32(d, s) /* Truncate to 32 */ #define ACPI_MOVE_64_TO_64(d, s) {(( UINT8 *)(void *)(d))[0] = ((UINT8 *)(void *)(s))[7];\ (( UINT8 *)(void *)(d))[1] = ((UINT8 *)(void *)(s))[6];\ (( UINT8 *)(void *)(d))[2] = ((UINT8 *)(void *)(s))[5];\ (( UINT8 *)(void *)(d))[3] = ((UINT8 *)(void *)(s))[4];\ (( UINT8 *)(void *)(d))[4] = ((UINT8 *)(void *)(s))[3];\ (( UINT8 *)(void *)(d))[5] = ((UINT8 *)(void *)(s))[2];\ (( UINT8 *)(void *)(d))[6] = ((UINT8 *)(void *)(s))[1];\ (( UINT8 *)(void *)(d))[7] = ((UINT8 *)(void *)(s))[0];} #else /* * Macros for little-endian machines */ #ifndef ACPI_MISALIGNMENT_NOT_SUPPORTED /* The hardware supports unaligned transfers, just do the little-endian move */ /* 16-bit source, 16/32/64 destination */ #define ACPI_MOVE_16_TO_16(d, s) *(UINT16 *)(void *)(d) = *(UINT16 *)(void *)(s) #define ACPI_MOVE_16_TO_32(d, s) *(UINT32 *)(void *)(d) = *(UINT16 *)(void *)(s) #define ACPI_MOVE_16_TO_64(d, s) *(UINT64 *)(void *)(d) = *(UINT16 *)(void *)(s) /* 32-bit source, 16/32/64 destination */ #define ACPI_MOVE_32_TO_16(d, s) ACPI_MOVE_16_TO_16(d, s) /* Truncate to 16 */ #define ACPI_MOVE_32_TO_32(d, s) *(UINT32 *)(void *)(d) = *(UINT32 *)(void *)(s) #define ACPI_MOVE_32_TO_64(d, s) *(UINT64 *)(void *)(d) = *(UINT32 *)(void *)(s) /* 64-bit source, 16/32/64 destination */ #define ACPI_MOVE_64_TO_16(d, s) ACPI_MOVE_16_TO_16(d, s) /* Truncate to 16 */ #define ACPI_MOVE_64_TO_32(d, s) ACPI_MOVE_32_TO_32(d, s) /* Truncate to 32 */ #define ACPI_MOVE_64_TO_64(d, s) *(UINT64 *)(void *)(d) = *(UINT64 *)(void *)(s) #else /* * The hardware does not support unaligned transfers. We must move the * data one byte at a time. These macros work whether the source or * the destination (or both) is/are unaligned. (Little-endian move) */ /* 16-bit source, 16/32/64 destination */ #define ACPI_MOVE_16_TO_16(d, s) {(( UINT8 *)(void *)(d))[0] = ((UINT8 *)(void *)(s))[0];\ (( UINT8 *)(void *)(d))[1] = ((UINT8 *)(void *)(s))[1];} #define ACPI_MOVE_16_TO_32(d, s) {(*(UINT32 *)(void *)(d)) = 0; ACPI_MOVE_16_TO_16(d, s);} #define ACPI_MOVE_16_TO_64(d, s) {(*(UINT64 *)(void *)(d)) = 0; ACPI_MOVE_16_TO_16(d, s);} /* 32-bit source, 16/32/64 destination */ #define ACPI_MOVE_32_TO_16(d, s) ACPI_MOVE_16_TO_16(d, s) /* Truncate to 16 */ #define ACPI_MOVE_32_TO_32(d, s) {(( UINT8 *)(void *)(d))[0] = ((UINT8 *)(void *)(s))[0];\ (( UINT8 *)(void *)(d))[1] = ((UINT8 *)(void *)(s))[1];\ (( UINT8 *)(void *)(d))[2] = ((UINT8 *)(void *)(s))[2];\ (( UINT8 *)(void *)(d))[3] = ((UINT8 *)(void *)(s))[3];} #define ACPI_MOVE_32_TO_64(d, s) {(*(UINT64 *)(void *)(d)) = 0; ACPI_MOVE_32_TO_32(d, s);} /* 64-bit source, 16/32/64 destination */ #define ACPI_MOVE_64_TO_16(d, s) ACPI_MOVE_16_TO_16(d, s) /* Truncate to 16 */ #define ACPI_MOVE_64_TO_32(d, s) ACPI_MOVE_32_TO_32(d, s) /* Truncate to 32 */ #define ACPI_MOVE_64_TO_64(d, s) {(( UINT8 *)(void *)(d))[0] = ((UINT8 *)(void *)(s))[0];\ (( UINT8 *)(void *)(d))[1] = ((UINT8 *)(void *)(s))[1];\ (( UINT8 *)(void *)(d))[2] = ((UINT8 *)(void *)(s))[2];\ (( UINT8 *)(void *)(d))[3] = ((UINT8 *)(void *)(s))[3];\ (( UINT8 *)(void *)(d))[4] = ((UINT8 *)(void *)(s))[4];\ (( UINT8 *)(void *)(d))[5] = ((UINT8 *)(void *)(s))[5];\ (( UINT8 *)(void *)(d))[6] = ((UINT8 *)(void *)(s))[6];\ (( UINT8 *)(void *)(d))[7] = ((UINT8 *)(void *)(s))[7];} #endif #endif /* * Fast power-of-two math macros for non-optimized compilers */ #define _ACPI_DIV(value, PowerOf2) ((UINT32) ((value) >> (PowerOf2))) #define _ACPI_MUL(value, PowerOf2) ((UINT32) ((value) << (PowerOf2))) #define _ACPI_MOD(value, Divisor) ((UINT32) ((value) & ((Divisor) -1))) #define ACPI_DIV_2(a) _ACPI_DIV(a, 1) #define ACPI_MUL_2(a) _ACPI_MUL(a, 1) #define ACPI_MOD_2(a) _ACPI_MOD(a, 2) #define ACPI_DIV_4(a) _ACPI_DIV(a, 2) #define ACPI_MUL_4(a) _ACPI_MUL(a, 2) #define ACPI_MOD_4(a) _ACPI_MOD(a, 4) #define ACPI_DIV_8(a) _ACPI_DIV(a, 3) #define ACPI_MUL_8(a) _ACPI_MUL(a, 3) #define ACPI_MOD_8(a) _ACPI_MOD(a, 8) #define ACPI_DIV_16(a) _ACPI_DIV(a, 4) #define ACPI_MUL_16(a) _ACPI_MUL(a, 4) #define ACPI_MOD_16(a) _ACPI_MOD(a, 16) #define ACPI_DIV_32(a) _ACPI_DIV(a, 5) #define ACPI_MUL_32(a) _ACPI_MUL(a, 5) #define ACPI_MOD_32(a) _ACPI_MOD(a, 32) /* Test for ASCII character */ #define ACPI_IS_ASCII(c) ((c) < 0x80) /* Signed integers */ #define ACPI_SIGN_POSITIVE 0 #define ACPI_SIGN_NEGATIVE 1 /* * Rounding macros (Power of two boundaries only) */ #define ACPI_ROUND_DOWN(value, boundary) (((ACPI_SIZE)(value)) & \ (~(((ACPI_SIZE) boundary)-1))) #define ACPI_ROUND_UP(value, boundary) ((((ACPI_SIZE)(value)) + \ (((ACPI_SIZE) boundary)-1)) & \ (~(((ACPI_SIZE) boundary)-1))) /* Note: sizeof(ACPI_SIZE) evaluates to either 4 or 8 (32- vs 64-bit mode) */ #define ACPI_ROUND_DOWN_TO_32BIT(a) ACPI_ROUND_DOWN(a, 4) #define ACPI_ROUND_DOWN_TO_64BIT(a) ACPI_ROUND_DOWN(a, 8) #define ACPI_ROUND_DOWN_TO_NATIVE_WORD(a) ACPI_ROUND_DOWN(a, sizeof(ACPI_SIZE)) #define ACPI_ROUND_UP_TO_32BIT(a) ACPI_ROUND_UP(a, 4) #define ACPI_ROUND_UP_TO_64BIT(a) ACPI_ROUND_UP(a, 8) #define ACPI_ROUND_UP_TO_NATIVE_WORD(a) ACPI_ROUND_UP(a, sizeof(ACPI_SIZE)) #define ACPI_ROUND_BITS_UP_TO_BYTES(a) ACPI_DIV_8((a) + 7) #define ACPI_ROUND_BITS_DOWN_TO_BYTES(a) ACPI_DIV_8((a)) #define ACPI_ROUND_UP_TO_1K(a) (((a) + 1023) >> 10) /* Generic (non-power-of-two) rounding */ #define ACPI_ROUND_UP_TO(value, boundary) (((value) + ((boundary)-1)) / (boundary)) #define ACPI_IS_MISALIGNED(value) (((ACPI_SIZE) value) & (sizeof(ACPI_SIZE)-1)) /* Generic bit manipulation */ #ifndef ACPI_USE_NATIVE_BIT_FINDER #define __ACPI_FIND_LAST_BIT_2(a, r) ((((UINT8) (a)) & 0x02) ? (r)+1 : (r)) #define __ACPI_FIND_LAST_BIT_4(a, r) ((((UINT8) (a)) & 0x0C) ? \ __ACPI_FIND_LAST_BIT_2 ((a)>>2, (r)+2) : \ __ACPI_FIND_LAST_BIT_2 ((a), (r))) #define __ACPI_FIND_LAST_BIT_8(a, r) ((((UINT8) (a)) & 0xF0) ? \ __ACPI_FIND_LAST_BIT_4 ((a)>>4, (r)+4) : \ __ACPI_FIND_LAST_BIT_4 ((a), (r))) #define __ACPI_FIND_LAST_BIT_16(a, r) ((((UINT16) (a)) & 0xFF00) ? \ __ACPI_FIND_LAST_BIT_8 ((a)>>8, (r)+8) : \ __ACPI_FIND_LAST_BIT_8 ((a), (r))) #define __ACPI_FIND_LAST_BIT_32(a, r) ((((UINT32) (a)) & 0xFFFF0000) ? \ __ACPI_FIND_LAST_BIT_16 ((a)>>16, (r)+16) : \ __ACPI_FIND_LAST_BIT_16 ((a), (r))) #define __ACPI_FIND_LAST_BIT_64(a, r) ((((UINT64) (a)) & 0xFFFFFFFF00000000) ? \ __ACPI_FIND_LAST_BIT_32 ((a)>>32, (r)+32) : \ __ACPI_FIND_LAST_BIT_32 ((a), (r))) #define ACPI_FIND_LAST_BIT_8(a) ((a) ? __ACPI_FIND_LAST_BIT_8 (a, 1) : 0) #define ACPI_FIND_LAST_BIT_16(a) ((a) ? __ACPI_FIND_LAST_BIT_16 (a, 1) : 0) #define ACPI_FIND_LAST_BIT_32(a) ((a) ? __ACPI_FIND_LAST_BIT_32 (a, 1) : 0) #define ACPI_FIND_LAST_BIT_64(a) ((a) ? __ACPI_FIND_LAST_BIT_64 (a, 1) : 0) #define __ACPI_FIND_FIRST_BIT_2(a, r) ((((UINT8) (a)) & 0x01) ? (r) : (r)+1) #define __ACPI_FIND_FIRST_BIT_4(a, r) ((((UINT8) (a)) & 0x03) ? \ __ACPI_FIND_FIRST_BIT_2 ((a), (r)) : \ __ACPI_FIND_FIRST_BIT_2 ((a)>>2, (r)+2)) #define __ACPI_FIND_FIRST_BIT_8(a, r) ((((UINT8) (a)) & 0x0F) ? \ __ACPI_FIND_FIRST_BIT_4 ((a), (r)) : \ __ACPI_FIND_FIRST_BIT_4 ((a)>>4, (r)+4)) #define __ACPI_FIND_FIRST_BIT_16(a, r) ((((UINT16) (a)) & 0x00FF) ? \ __ACPI_FIND_FIRST_BIT_8 ((a), (r)) : \ __ACPI_FIND_FIRST_BIT_8 ((a)>>8, (r)+8)) #define __ACPI_FIND_FIRST_BIT_32(a, r) ((((UINT32) (a)) & 0x0000FFFF) ? \ __ACPI_FIND_FIRST_BIT_16 ((a), (r)) : \ __ACPI_FIND_FIRST_BIT_16 ((a)>>16, (r)+16)) #define __ACPI_FIND_FIRST_BIT_64(a, r) ((((UINT64) (a)) & 0x00000000FFFFFFFF) ? \ __ACPI_FIND_FIRST_BIT_32 ((a), (r)) : \ __ACPI_FIND_FIRST_BIT_32 ((a)>>32, (r)+32)) #define ACPI_FIND_FIRST_BIT_8(a) ((a) ? __ACPI_FIND_FIRST_BIT_8 (a, 1) : 0) #define ACPI_FIND_FIRST_BIT_16(a) ((a) ? __ACPI_FIND_FIRST_BIT_16 (a, 1) : 0) #define ACPI_FIND_FIRST_BIT_32(a) ((a) ? __ACPI_FIND_FIRST_BIT_32 (a, 1) : 0) #define ACPI_FIND_FIRST_BIT_64(a) ((a) ? __ACPI_FIND_FIRST_BIT_64 (a, 1) : 0) #endif /* ACPI_USE_NATIVE_BIT_FINDER */ /* Generic (power-of-two) rounding */ #define ACPI_ROUND_UP_POWER_OF_TWO_8(a) ((UINT8) \ (((UINT16) 1) << ACPI_FIND_LAST_BIT_8 ((a) - 1))) #define ACPI_ROUND_DOWN_POWER_OF_TWO_8(a) ((UINT8) \ (((UINT16) 1) << (ACPI_FIND_LAST_BIT_8 ((a)) - 1))) #define ACPI_ROUND_UP_POWER_OF_TWO_16(a) ((UINT16) \ (((UINT32) 1) << ACPI_FIND_LAST_BIT_16 ((a) - 1))) #define ACPI_ROUND_DOWN_POWER_OF_TWO_16(a) ((UINT16) \ (((UINT32) 1) << (ACPI_FIND_LAST_BIT_16 ((a)) - 1))) #define ACPI_ROUND_UP_POWER_OF_TWO_32(a) ((UINT32) \ (((UINT64) 1) << ACPI_FIND_LAST_BIT_32 ((a) - 1))) #define ACPI_ROUND_DOWN_POWER_OF_TWO_32(a) ((UINT32) \ (((UINT64) 1) << (ACPI_FIND_LAST_BIT_32 ((a)) - 1))) #define ACPI_IS_ALIGNED(a, s) (((a) & ((s) - 1)) == 0) #define ACPI_IS_POWER_OF_TWO(a) ACPI_IS_ALIGNED(a, a) /* * Bitmask creation * Bit positions start at zero. * MASK_BITS_ABOVE creates a mask starting AT the position and above * MASK_BITS_BELOW creates a mask starting one bit BELOW the position * MASK_BITS_ABOVE/BELOW accepts a bit offset to create a mask * MASK_BITS_ABOVE/BELOW_32/64 accepts a bit width to create a mask * Note: The ACPI_INTEGER_BIT_SIZE check is used to bypass compiler * differences with the shift operator */ #define ACPI_MASK_BITS_ABOVE(position) (~((ACPI_UINT64_MAX) << ((UINT32) (position)))) #define ACPI_MASK_BITS_BELOW(position) ((ACPI_UINT64_MAX) << ((UINT32) (position))) #define ACPI_MASK_BITS_ABOVE_32(width) ((UINT32) ACPI_MASK_BITS_ABOVE(width)) #define ACPI_MASK_BITS_BELOW_32(width) ((UINT32) ACPI_MASK_BITS_BELOW(width)) #define ACPI_MASK_BITS_ABOVE_64(width) ((width) == ACPI_INTEGER_BIT_SIZE ? \ ACPI_UINT64_MAX : \ ACPI_MASK_BITS_ABOVE(width)) #define ACPI_MASK_BITS_BELOW_64(width) ((width) == ACPI_INTEGER_BIT_SIZE ? \ (UINT64) 0 : \ ACPI_MASK_BITS_BELOW(width)) /* Bitfields within ACPI registers */ #define ACPI_REGISTER_PREPARE_BITS(Val, Pos, Mask) \ ((Val << Pos) & Mask) #define ACPI_REGISTER_INSERT_VALUE(Reg, Pos, Mask, Val) \ Reg = (Reg & (~(Mask))) | ACPI_REGISTER_PREPARE_BITS(Val, Pos, Mask) #define ACPI_INSERT_BITS(Target, Mask, Source) \ Target = ((Target & (~(Mask))) | (Source & Mask)) /* Generic bitfield macros and masks */ #define ACPI_GET_BITS(SourcePtr, Position, Mask) \ ((*(SourcePtr) >> (Position)) & (Mask)) #define ACPI_SET_BITS(TargetPtr, Position, Mask, Value) \ (*(TargetPtr) |= (((Value) & (Mask)) << (Position))) #define ACPI_1BIT_MASK 0x00000001 #define ACPI_2BIT_MASK 0x00000003 #define ACPI_3BIT_MASK 0x00000007 #define ACPI_4BIT_MASK 0x0000000F #define ACPI_5BIT_MASK 0x0000001F #define ACPI_6BIT_MASK 0x0000003F #define ACPI_7BIT_MASK 0x0000007F #define ACPI_8BIT_MASK 0x000000FF #define ACPI_16BIT_MASK 0x0000FFFF #define ACPI_24BIT_MASK 0x00FFFFFF /* Macros to extract flag bits from position zero */ #define ACPI_GET_1BIT_FLAG(Value) ((Value) & ACPI_1BIT_MASK) #define ACPI_GET_2BIT_FLAG(Value) ((Value) & ACPI_2BIT_MASK) #define ACPI_GET_3BIT_FLAG(Value) ((Value) & ACPI_3BIT_MASK) #define ACPI_GET_4BIT_FLAG(Value) ((Value) & ACPI_4BIT_MASK) /* Macros to extract flag bits from position one and above */ #define ACPI_EXTRACT_1BIT_FLAG(Field, Position) (ACPI_GET_1BIT_FLAG ((Field) >> Position)) #define ACPI_EXTRACT_2BIT_FLAG(Field, Position) (ACPI_GET_2BIT_FLAG ((Field) >> Position)) #define ACPI_EXTRACT_3BIT_FLAG(Field, Position) (ACPI_GET_3BIT_FLAG ((Field) >> Position)) #define ACPI_EXTRACT_4BIT_FLAG(Field, Position) (ACPI_GET_4BIT_FLAG ((Field) >> Position)) /* ACPI Pathname helpers */ #define ACPI_IS_ROOT_PREFIX(c) ((c) == (UINT8) 0x5C) /* Backslash */ #define ACPI_IS_PARENT_PREFIX(c) ((c) == (UINT8) 0x5E) /* Carat */ #define ACPI_IS_PATH_SEPARATOR(c) ((c) == (UINT8) 0x2E) /* Period (dot) */ /* * An object of type ACPI_NAMESPACE_NODE can appear in some contexts * where a pointer to an object of type ACPI_OPERAND_OBJECT can also * appear. This macro is used to distinguish them. * * The "DescriptorType" field is the second field in both structures. */ #define ACPI_GET_DESCRIPTOR_PTR(d) (((ACPI_DESCRIPTOR *)(void *)(d))->Common.CommonPointer) #define ACPI_SET_DESCRIPTOR_PTR(d, p) (((ACPI_DESCRIPTOR *)(void *)(d))->Common.CommonPointer = (p)) #define ACPI_GET_DESCRIPTOR_TYPE(d) (((ACPI_DESCRIPTOR *)(void *)(d))->Common.DescriptorType) #define ACPI_SET_DESCRIPTOR_TYPE(d, t) (((ACPI_DESCRIPTOR *)(void *)(d))->Common.DescriptorType = (t)) /* * Macros for the master AML opcode table */ #if defined (ACPI_DISASSEMBLER) || defined (ACPI_DEBUG_OUTPUT) #define ACPI_OP(Name, PArgs, IArgs, ObjType, Class, Type, Flags) \ {Name, (UINT32)(PArgs), (UINT32)(IArgs), (UINT32)(Flags), ObjType, Class, Type} #else #define ACPI_OP(Name, PArgs, IArgs, ObjType, Class, Type, Flags) \ {(UINT32)(PArgs), (UINT32)(IArgs), (UINT32)(Flags), ObjType, Class, Type} #endif #define ARG_TYPE_WIDTH 5 #define ARG_1(x) ((UINT32)(x)) #define ARG_2(x) ((UINT32)(x) << (1 * ARG_TYPE_WIDTH)) #define ARG_3(x) ((UINT32)(x) << (2 * ARG_TYPE_WIDTH)) #define ARG_4(x) ((UINT32)(x) << (3 * ARG_TYPE_WIDTH)) #define ARG_5(x) ((UINT32)(x) << (4 * ARG_TYPE_WIDTH)) #define ARG_6(x) ((UINT32)(x) << (5 * ARG_TYPE_WIDTH)) #define ARGI_LIST1(a) (ARG_1(a)) #define ARGI_LIST2(a, b) (ARG_1(b)|ARG_2(a)) #define ARGI_LIST3(a, b, c) (ARG_1(c)|ARG_2(b)|ARG_3(a)) #define ARGI_LIST4(a, b, c, d) (ARG_1(d)|ARG_2(c)|ARG_3(b)|ARG_4(a)) #define ARGI_LIST5(a, b, c, d, e) (ARG_1(e)|ARG_2(d)|ARG_3(c)|ARG_4(b)|ARG_5(a)) #define ARGI_LIST6(a, b, c, d, e, f) (ARG_1(f)|ARG_2(e)|ARG_3(d)|ARG_4(c)|ARG_5(b)|ARG_6(a)) #define ARGP_LIST1(a) (ARG_1(a)) #define ARGP_LIST2(a, b) (ARG_1(a)|ARG_2(b)) #define ARGP_LIST3(a, b, c) (ARG_1(a)|ARG_2(b)|ARG_3(c)) #define ARGP_LIST4(a, b, c, d) (ARG_1(a)|ARG_2(b)|ARG_3(c)|ARG_4(d)) #define ARGP_LIST5(a, b, c, d, e) (ARG_1(a)|ARG_2(b)|ARG_3(c)|ARG_4(d)|ARG_5(e)) #define ARGP_LIST6(a, b, c, d, e, f) (ARG_1(a)|ARG_2(b)|ARG_3(c)|ARG_4(d)|ARG_5(e)|ARG_6(f)) #define GET_CURRENT_ARG_TYPE(List) (List & ((UINT32) 0x1F)) #define INCREMENT_ARG_LIST(List) (List >>= ((UINT32) ARG_TYPE_WIDTH)) /* * Ascii error messages can be configured out */ #ifndef ACPI_NO_ERROR_MESSAGES /* * Error reporting. The callers module and line number are inserted by AE_INFO, * the plist contains a set of parens to allow variable-length lists. * These macros are used for both the debug and non-debug versions of the code. */ #define ACPI_ERROR_NAMESPACE(s, p, e) AcpiUtPrefixedNamespaceError (AE_INFO, s, p, e); #define ACPI_ERROR_METHOD(s, n, p, e) AcpiUtMethodError (AE_INFO, s, n, p, e); #define ACPI_WARN_PREDEFINED(plist) AcpiUtPredefinedWarning plist #define ACPI_INFO_PREDEFINED(plist) AcpiUtPredefinedInfo plist #define ACPI_BIOS_ERROR_PREDEFINED(plist) AcpiUtPredefinedBiosError plist #define ACPI_ERROR_ONLY(s) s #else /* No error messages */ #define ACPI_ERROR_NAMESPACE(s, p, e) #define ACPI_ERROR_METHOD(s, n, p, e) #define ACPI_WARN_PREDEFINED(plist) #define ACPI_INFO_PREDEFINED(plist) #define ACPI_BIOS_ERROR_PREDEFINED(plist) #define ACPI_ERROR_ONLY(s) #endif /* ACPI_NO_ERROR_MESSAGES */ #if (!ACPI_REDUCED_HARDWARE) #define ACPI_HW_OPTIONAL_FUNCTION(addr) addr #else #define ACPI_HW_OPTIONAL_FUNCTION(addr) NULL #endif /* * Macros used for ACPICA utilities only */ /* Generate a UUID */ #define ACPI_INIT_UUID(a, b, c, d0, d1, d2, d3, d4, d5, d6, d7) \ (a) & 0xFF, ((a) >> 8) & 0xFF, ((a) >> 16) & 0xFF, ((a) >> 24) & 0xFF, \ (b) & 0xFF, ((b) >> 8) & 0xFF, \ (c) & 0xFF, ((c) >> 8) & 0xFF, \ (d0), (d1), (d2), (d3), (d4), (d5), (d6), (d7) #define ACPI_IS_OCTAL_DIGIT(d) (((char)(d) >= '0') && ((char)(d) <= '7')) /* * Macros used for the ASL-/ASL+ converter utility */ #ifdef ACPI_ASL_COMPILER #define ASL_CV_LABEL_FILENODE(a) CvLabelFileNode(a); #define ASL_CV_CAPTURE_COMMENTS_ONLY(a) CvCaptureCommentsOnly (a); #define ASL_CV_CAPTURE_COMMENTS(a) CvCaptureComments (a); #define ASL_CV_TRANSFER_COMMENTS(a) CvTransferComments (a); #define ASL_CV_CLOSE_PAREN(a,b) CvCloseParenWriteComment(a,b); #define ASL_CV_CLOSE_BRACE(a,b) CvCloseBraceWriteComment(a,b); #define ASL_CV_SWITCH_FILES(a,b) CvSwitchFiles(a,b); #define ASL_CV_CLEAR_OP_COMMENTS(a) CvClearOpComments(a); #define ASL_CV_PRINT_ONE_COMMENT(a,b,c,d) CvPrintOneCommentType (a,b,c,d); #define ASL_CV_PRINT_ONE_COMMENT_LIST(a,b) CvPrintOneCommentList (a,b); #define ASL_CV_FILE_HAS_SWITCHED(a) CvFileHasSwitched(a) -#define ASL_CV_INIT_FILETREE(a,b,c) CvInitFileTree(a,b,c); +#define ASL_CV_INIT_FILETREE(a,b) CvInitFileTree(a,b); #else #define ASL_CV_LABEL_FILENODE(a) #define ASL_CV_CAPTURE_COMMENTS_ONLY(a) #define ASL_CV_CAPTURE_COMMENTS(a) #define ASL_CV_TRANSFER_COMMENTS(a) #define ASL_CV_CLOSE_PAREN(a,b) AcpiOsPrintf (")"); #define ASL_CV_CLOSE_BRACE(a,b) AcpiOsPrintf ("}"); #define ASL_CV_SWITCH_FILES(a,b) #define ASL_CV_CLEAR_OP_COMMENTS(a) #define ASL_CV_PRINT_ONE_COMMENT(a,b,c,d) #define ASL_CV_PRINT_ONE_COMMENT_LIST(a,b) #define ASL_CV_FILE_HAS_SWITCHED(a) 0 -#define ASL_CV_INIT_FILETREE(a,b,c) +#define ASL_CV_INIT_FILETREE(a,b) #endif #endif /* ACMACROS_H */ Index: projects/clang1000-import/sys/contrib/dev/acpica/include/acpixf.h =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/include/acpixf.h (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/include/acpixf.h (revision 357966) @@ -1,1421 +1,1425 @@ /****************************************************************************** * * Name: acpixf.h - External interfaces to the ACPI subsystem * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #ifndef __ACXFACE_H__ #define __ACXFACE_H__ /* Current ACPICA subsystem version in YYYYMMDD format */ -#define ACPI_CA_VERSION 0x20200110 +#define ACPI_CA_VERSION 0x20200214 #include #include #include #include /***************************************************************************** * * Macros used for ACPICA globals and configuration * ****************************************************************************/ /* * Ensure that global variables are defined and initialized only once. * * The use of these macros allows for a single list of globals (here) * in order to simplify maintenance of the code. */ #ifdef DEFINE_ACPI_GLOBALS #define ACPI_GLOBAL(type,name) \ extern type name; \ type name #define ACPI_INIT_GLOBAL(type,name,value) \ type name=value #else #ifndef ACPI_GLOBAL #define ACPI_GLOBAL(type,name) \ extern type name #endif #ifndef ACPI_INIT_GLOBAL #define ACPI_INIT_GLOBAL(type,name,value) \ extern type name #endif #endif /* * These macros configure the various ACPICA interfaces. They are * useful for generating stub inline functions for features that are * configured out of the current kernel or ACPICA application. */ #ifndef ACPI_EXTERNAL_RETURN_STATUS #define ACPI_EXTERNAL_RETURN_STATUS(Prototype) \ Prototype; #endif #ifndef ACPI_EXTERNAL_RETURN_OK #define ACPI_EXTERNAL_RETURN_OK(Prototype) \ Prototype; #endif #ifndef ACPI_EXTERNAL_RETURN_VOID #define ACPI_EXTERNAL_RETURN_VOID(Prototype) \ Prototype; #endif #ifndef ACPI_EXTERNAL_RETURN_UINT32 #define ACPI_EXTERNAL_RETURN_UINT32(Prototype) \ Prototype; #endif #ifndef ACPI_EXTERNAL_RETURN_PTR #define ACPI_EXTERNAL_RETURN_PTR(Prototype) \ Prototype; #endif /***************************************************************************** * * Public globals and runtime configuration options * ****************************************************************************/ /* * Enable "slack mode" of the AML interpreter? Default is FALSE, and the * interpreter strictly follows the ACPI specification. Setting to TRUE * allows the interpreter to ignore certain errors and/or bad AML constructs. * * Currently, these features are enabled by this flag: * * 1) Allow "implicit return" of last value in a control method * 2) Allow access beyond the end of an operation region * 3) Allow access to uninitialized locals/args (auto-init to integer 0) * 4) Allow ANY object type to be a source operand for the Store() operator * 5) Allow unresolved references (invalid target name) in package objects * 6) Enable warning messages for behavior that is not ACPI spec compliant */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_EnableInterpreterSlack, FALSE); /* * Automatically serialize all methods that create named objects? Default * is TRUE, meaning that all NonSerialized methods are scanned once at * table load time to determine those that create named objects. Methods * that create named objects are marked Serialized in order to prevent * possible run-time problems if they are entered by more than one thread. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_AutoSerializeMethods, TRUE); /* * Create the predefined _OSI method in the namespace? Default is TRUE * because ACPICA is fully compatible with other ACPI implementations. * Changing this will revert ACPICA (and machine ASL) to pre-OSI behavior. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_CreateOsiMethod, TRUE); /* * Optionally use default values for the ACPI register widths. Set this to * TRUE to use the defaults, if an FADT contains incorrect widths/lengths. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_UseDefaultRegisterWidths, TRUE); /* * Whether or not to validate (map) an entire table to verify * checksum/duplication in early stage before install. Set this to TRUE to * allow early table validation before install it to the table manager. * Note that enabling this option causes errors to happen in some OSPMs * during early initialization stages. Default behavior is to allow such * validation. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_EnableTableValidation, TRUE); /* * Optionally enable output from the AML Debug Object. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_EnableAmlDebugObject, FALSE); /* * Optionally copy the entire DSDT to local memory (instead of simply * mapping it.) There are some BIOSs that corrupt or replace the original * DSDT, creating the need for this option. Default is FALSE, do not copy * the DSDT. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_CopyDsdtLocally, FALSE); /* * Optionally ignore an XSDT if present and use the RSDT instead. * Although the ACPI specification requires that an XSDT be used instead * of the RSDT, the XSDT has been found to be corrupt or ill-formed on * some machines. Default behavior is to use the XSDT if present. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_DoNotUseXsdt, FALSE); /* * Optionally use 32-bit FADT addresses if and when there is a conflict * (address mismatch) between the 32-bit and 64-bit versions of the * address. Although ACPICA adheres to the ACPI specification which * requires the use of the corresponding 64-bit address if it is non-zero, * some machines have been found to have a corrupted non-zero 64-bit * address. Default is FALSE, do not favor the 32-bit addresses. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_Use32BitFadtAddresses, FALSE); /* * Optionally use 32-bit FACS table addresses. * It is reported that some platforms fail to resume from system suspending * if 64-bit FACS table address is selected: * https://bugzilla.kernel.org/show_bug.cgi?id=74021 * Default is TRUE, favor the 32-bit addresses. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_Use32BitFacsAddresses, TRUE); /* * Optionally truncate I/O addresses to 16 bits. Provides compatibility * with other ACPI implementations. NOTE: During ACPICA initialization, * this value is set to TRUE if any Windows OSI strings have been * requested by the BIOS. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_TruncateIoAddresses, FALSE); /* * Disable runtime checking and repair of values returned by control methods. * Use only if the repair is causing a problem on a particular machine. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_DisableAutoRepair, FALSE); /* * Optionally do not install any SSDTs from the RSDT/XSDT during initialization. * This can be useful for debugging ACPI problems on some machines. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_DisableSsdtTableInstall, FALSE); /* * Optionally enable runtime namespace override. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_RuntimeNamespaceOverride, TRUE); /* * We keep track of the latest version of Windows that has been requested by * the BIOS. ACPI 5.0. */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_OsiData, 0); /* * ACPI 5.0 introduces the concept of a "reduced hardware platform", meaning * that the ACPI hardware is no longer required. A flag in the FADT indicates * a reduced HW machine, and that flag is duplicated here for convenience. */ ACPI_INIT_GLOBAL (BOOLEAN, AcpiGbl_ReducedHardware, FALSE); /* * Maximum timeout for While() loop iterations before forced method abort. * This mechanism is intended to prevent infinite loops during interpreter * execution within a host kernel. */ ACPI_INIT_GLOBAL (UINT32, AcpiGbl_MaxLoopIterations, ACPI_MAX_LOOP_TIMEOUT); /* * Optionally ignore AE_NOT_FOUND errors from named reference package elements * during DSDT/SSDT table loading. This reduces error "noise" in platforms * whose firmware is carrying around a bunch of unused package objects that * refer to non-existent named objects. However, If the AML actually tries to * use such a package, the unresolved element(s) will be replaced with NULL * elements. */ ACPI_INIT_GLOBAL (BOOLEAN, AcpiGbl_IgnorePackageResolutionErrors, FALSE); /* * This mechanism is used to trace a specified AML method. The method is * traced each time it is executed. */ ACPI_INIT_GLOBAL (UINT32, AcpiGbl_TraceFlags, 0); ACPI_INIT_GLOBAL (const char *, AcpiGbl_TraceMethodName, NULL); ACPI_INIT_GLOBAL (UINT32, AcpiGbl_TraceDbgLevel, ACPI_TRACE_LEVEL_DEFAULT); ACPI_INIT_GLOBAL (UINT32, AcpiGbl_TraceDbgLayer, ACPI_TRACE_LAYER_DEFAULT); /* * Runtime configuration of debug output control masks. We want the debug * switches statically initialized so they are already set when the debugger * is entered. */ #ifdef ACPI_DEBUG_OUTPUT ACPI_INIT_GLOBAL (UINT32, AcpiDbgLevel, ACPI_DEBUG_DEFAULT); #else ACPI_INIT_GLOBAL (UINT32, AcpiDbgLevel, ACPI_NORMAL_DEFAULT); #endif ACPI_INIT_GLOBAL (UINT32, AcpiDbgLayer, ACPI_COMPONENT_DEFAULT); /* Optionally enable timer output with Debug Object output */ ACPI_INIT_GLOBAL (UINT8, AcpiGbl_DisplayDebugTimer, FALSE); /* * Debugger command handshake globals. Host OSes need to access these * variables to implement their own command handshake mechanism. */ #ifdef ACPI_DEBUGGER ACPI_INIT_GLOBAL (BOOLEAN, AcpiGbl_MethodExecuting, FALSE); ACPI_GLOBAL (char, AcpiGbl_DbLineBuf[ACPI_DB_LINE_BUFFER_SIZE]); #endif /* * Other miscellaneous globals */ ACPI_GLOBAL (ACPI_TABLE_FADT, AcpiGbl_FADT); ACPI_GLOBAL (UINT32, AcpiCurrentGpeCount); ACPI_GLOBAL (BOOLEAN, AcpiGbl_SystemAwakeAndRunning); /***************************************************************************** * * ACPICA public interface configuration. * * Interfaces that are configured out of the ACPICA build are replaced * by inlined stubs by default. * ****************************************************************************/ /* * Hardware-reduced prototypes (default: Not hardware reduced). * * All ACPICA hardware-related interfaces that use these macros will be * configured out of the ACPICA build if the ACPI_REDUCED_HARDWARE flag * is set to TRUE. * * Note: This static build option for reduced hardware is intended to * reduce ACPICA code size if desired or necessary. However, even if this * option is not specified, the runtime behavior of ACPICA is dependent * on the actual FADT reduced hardware flag (HW_REDUCED_ACPI). If set, * the flag will enable similar behavior -- ACPICA will not attempt * to access any ACPI-relate hardware (SCI, GPEs, Fixed Events, etc.) */ #if (!ACPI_REDUCED_HARDWARE) #define ACPI_HW_DEPENDENT_RETURN_STATUS(Prototype) \ ACPI_EXTERNAL_RETURN_STATUS(Prototype) #define ACPI_HW_DEPENDENT_RETURN_OK(Prototype) \ ACPI_EXTERNAL_RETURN_OK(Prototype) #define ACPI_HW_DEPENDENT_RETURN_UINT32(prototype) \ ACPI_EXTERNAL_RETURN_UINT32(prototype) #define ACPI_HW_DEPENDENT_RETURN_VOID(Prototype) \ ACPI_EXTERNAL_RETURN_VOID(Prototype) #else #define ACPI_HW_DEPENDENT_RETURN_STATUS(Prototype) \ static ACPI_INLINE Prototype {return(AE_NOT_CONFIGURED);} #define ACPI_HW_DEPENDENT_RETURN_OK(Prototype) \ static ACPI_INLINE Prototype {return(AE_OK);} #define ACPI_HW_DEPENDENT_RETURN_UINT32(prototype) \ static ACPI_INLINE prototype {return(0);} #define ACPI_HW_DEPENDENT_RETURN_VOID(Prototype) \ static ACPI_INLINE Prototype {return;} #endif /* !ACPI_REDUCED_HARDWARE */ /* * Error message prototypes (default: error messages enabled). * * All interfaces related to error and warning messages * will be configured out of the ACPICA build if the * ACPI_NO_ERROR_MESSAGE flag is defined. */ #ifndef ACPI_NO_ERROR_MESSAGES #define ACPI_MSG_DEPENDENT_RETURN_VOID(Prototype) \ Prototype; #else #define ACPI_MSG_DEPENDENT_RETURN_VOID(Prototype) \ static ACPI_INLINE Prototype {return;} #endif /* ACPI_NO_ERROR_MESSAGES */ /* * Debugging output prototypes (default: no debug output). * * All interfaces related to debug output messages * will be configured out of the ACPICA build unless the * ACPI_DEBUG_OUTPUT flag is defined. */ #ifdef ACPI_DEBUG_OUTPUT #define ACPI_DBG_DEPENDENT_RETURN_VOID(Prototype) \ Prototype; #else #define ACPI_DBG_DEPENDENT_RETURN_VOID(Prototype) \ static ACPI_INLINE Prototype {return;} #endif /* ACPI_DEBUG_OUTPUT */ /* * Application prototypes * * All interfaces used by application will be configured * out of the ACPICA build unless the ACPI_APPLICATION * flag is defined. */ #ifdef ACPI_APPLICATION #define ACPI_APP_DEPENDENT_RETURN_VOID(Prototype) \ Prototype; #else #define ACPI_APP_DEPENDENT_RETURN_VOID(Prototype) \ static ACPI_INLINE Prototype {return;} #endif /* ACPI_APPLICATION */ /* * Debugger prototypes * * All interfaces used by debugger will be configured * out of the ACPICA build unless the ACPI_DEBUGGER * flag is defined. */ #ifdef ACPI_DEBUGGER #define ACPI_DBR_DEPENDENT_RETURN_OK(Prototype) \ ACPI_EXTERNAL_RETURN_OK(Prototype) #define ACPI_DBR_DEPENDENT_RETURN_VOID(Prototype) \ ACPI_EXTERNAL_RETURN_VOID(Prototype) #else #define ACPI_DBR_DEPENDENT_RETURN_OK(Prototype) \ static ACPI_INLINE Prototype {return(AE_OK);} #define ACPI_DBR_DEPENDENT_RETURN_VOID(Prototype) \ static ACPI_INLINE Prototype {return;} #endif /* ACPI_DEBUGGER */ /***************************************************************************** * * ACPICA public interface prototypes * ****************************************************************************/ /* * Initialization */ ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS ACPI_INIT_FUNCTION AcpiInitializeTables ( ACPI_TABLE_DESC *InitialStorage, UINT32 InitialTableCount, BOOLEAN AllowResize)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS ACPI_INIT_FUNCTION AcpiInitializeSubsystem ( void)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS ACPI_INIT_FUNCTION AcpiEnableSubsystem ( UINT32 Flags)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS ACPI_INIT_FUNCTION AcpiInitializeObjects ( UINT32 Flags)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS ACPI_INIT_FUNCTION AcpiTerminate ( void)) /* * Miscellaneous global interfaces */ ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiEnable ( void)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiDisable ( void)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiSubsystemStatus ( void)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetSystemInfo ( ACPI_BUFFER *RetBuffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetStatistics ( ACPI_STATISTICS *Stats)) ACPI_EXTERNAL_RETURN_PTR ( const char * AcpiFormatException ( ACPI_STATUS Exception)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiPurgeCachedObjects ( void)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiInstallInterface ( ACPI_STRING InterfaceName)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiRemoveInterface ( ACPI_STRING InterfaceName)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiUpdateInterfaces ( UINT8 Action)) ACPI_EXTERNAL_RETURN_UINT32 ( UINT32 AcpiCheckAddressRange ( ACPI_ADR_SPACE_TYPE SpaceId, ACPI_PHYSICAL_ADDRESS Address, ACPI_SIZE Length, BOOLEAN Warn)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiDecodePldBuffer ( UINT8 *InBuffer, ACPI_SIZE Length, ACPI_PLD_INFO **ReturnBuffer)) /* * ACPI table load/unload interfaces */ ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS ACPI_INIT_FUNCTION AcpiInstallTable ( ACPI_PHYSICAL_ADDRESS Address, BOOLEAN Physical)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiLoadTable ( ACPI_TABLE_HEADER *Table, UINT32 *TableIdx)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiUnloadTable ( UINT32 TableIndex)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiUnloadParentTable ( ACPI_HANDLE Object)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS ACPI_INIT_FUNCTION AcpiLoadTables ( void)) /* * ACPI table manipulation interfaces */ ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS ACPI_INIT_FUNCTION AcpiReallocateRootTable ( void)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS ACPI_INIT_FUNCTION AcpiFindRootPointer ( ACPI_PHYSICAL_ADDRESS *RsdpAddress)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetTableHeader ( ACPI_STRING Signature, UINT32 Instance, ACPI_TABLE_HEADER *OutTableHeader)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetTable ( ACPI_STRING Signature, UINT32 Instance, ACPI_TABLE_HEADER **OutTable)) ACPI_EXTERNAL_RETURN_VOID ( void AcpiPutTable ( ACPI_TABLE_HEADER *Table)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetTableByIndex ( UINT32 TableIndex, ACPI_TABLE_HEADER **OutTable)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiInstallTableHandler ( ACPI_TABLE_HANDLER Handler, void *Context)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiRemoveTableHandler ( ACPI_TABLE_HANDLER Handler)) /* * Namespace and name interfaces */ ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiWalkNamespace ( ACPI_OBJECT_TYPE Type, ACPI_HANDLE StartObject, UINT32 MaxDepth, ACPI_WALK_CALLBACK DescendingCallback, ACPI_WALK_CALLBACK AscendingCallback, void *Context, void **ReturnValue)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetDevices ( char *HID, ACPI_WALK_CALLBACK UserFunction, void *Context, void **ReturnValue)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetName ( ACPI_HANDLE Object, UINT32 NameType, ACPI_BUFFER *RetPathPtr)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetHandle ( ACPI_HANDLE Parent, ACPI_STRING Pathname, ACPI_HANDLE *RetHandle)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiAttachData ( ACPI_HANDLE Object, ACPI_OBJECT_HANDLER Handler, void *Data)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiDetachData ( ACPI_HANDLE Object, ACPI_OBJECT_HANDLER Handler)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetData ( ACPI_HANDLE Object, ACPI_OBJECT_HANDLER Handler, void **Data)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiDebugTrace ( const char *Name, UINT32 DebugLevel, UINT32 DebugLayer, UINT32 Flags)) /* * Object manipulation and enumeration */ ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiEvaluateObject ( ACPI_HANDLE Object, ACPI_STRING Pathname, ACPI_OBJECT_LIST *ParameterObjects, ACPI_BUFFER *ReturnObjectBuffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiEvaluateObjectTyped ( ACPI_HANDLE Object, ACPI_STRING Pathname, ACPI_OBJECT_LIST *ExternalParams, ACPI_BUFFER *ReturnBuffer, ACPI_OBJECT_TYPE ReturnType)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetObjectInfo ( ACPI_HANDLE Object, ACPI_DEVICE_INFO **ReturnBuffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiInstallMethod ( UINT8 *Buffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetNextObject ( ACPI_OBJECT_TYPE Type, ACPI_HANDLE Parent, ACPI_HANDLE Child, ACPI_HANDLE *OutHandle)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetType ( ACPI_HANDLE Object, ACPI_OBJECT_TYPE *OutType)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetParent ( ACPI_HANDLE Object, ACPI_HANDLE *OutHandle)) /* * Handler interfaces */ ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiInstallInitializationHandler ( ACPI_INIT_HANDLER Handler, UINT32 Function)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiInstallSciHandler ( ACPI_SCI_HANDLER Address, void *Context)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiRemoveSciHandler ( ACPI_SCI_HANDLER Address)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiInstallGlobalEventHandler ( ACPI_GBL_EVENT_HANDLER Handler, void *Context)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiInstallFixedEventHandler ( UINT32 AcpiEvent, ACPI_EVENT_HANDLER Handler, void *Context)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiRemoveFixedEventHandler ( UINT32 AcpiEvent, ACPI_EVENT_HANDLER Handler)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiInstallGpeHandler ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, UINT32 Type, ACPI_GPE_HANDLER Address, void *Context)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiInstallGpeRawHandler ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, UINT32 Type, ACPI_GPE_HANDLER Address, void *Context)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiRemoveGpeHandler ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, ACPI_GPE_HANDLER Address)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiInstallNotifyHandler ( ACPI_HANDLE Device, UINT32 HandlerType, ACPI_NOTIFY_HANDLER Handler, void *Context)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiRemoveNotifyHandler ( ACPI_HANDLE Device, UINT32 HandlerType, ACPI_NOTIFY_HANDLER Handler)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiInstallAddressSpaceHandler ( ACPI_HANDLE Device, ACPI_ADR_SPACE_TYPE SpaceId, ACPI_ADR_SPACE_HANDLER Handler, ACPI_ADR_SPACE_SETUP Setup, void *Context)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiRemoveAddressSpaceHandler ( ACPI_HANDLE Device, ACPI_ADR_SPACE_TYPE SpaceId, ACPI_ADR_SPACE_HANDLER Handler)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiInstallExceptionHandler ( ACPI_EXCEPTION_HANDLER Handler)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiInstallInterfaceHandler ( ACPI_INTERFACE_HANDLER Handler)) /* * Global Lock interfaces */ ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiAcquireGlobalLock ( UINT16 Timeout, UINT32 *Handle)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiReleaseGlobalLock ( UINT32 Handle)) /* * Interfaces to AML mutex objects */ ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiAcquireMutex ( ACPI_HANDLE Handle, ACPI_STRING Pathname, UINT16 Timeout)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiReleaseMutex ( ACPI_HANDLE Handle, ACPI_STRING Pathname)) /* * Fixed Event interfaces */ ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiEnableEvent ( UINT32 Event, UINT32 Flags)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiDisableEvent ( UINT32 Event, UINT32 Flags)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiClearEvent ( UINT32 Event)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiGetEventStatus ( UINT32 Event, ACPI_EVENT_STATUS *EventStatus)) /* * General Purpose Event (GPE) Interfaces */ ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiUpdateAllGpes ( void)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiEnableGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiDisableGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiClearGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiSetGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, UINT8 Action)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiFinishGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiMaskGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, BOOLEAN IsMasked)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiMarkGpeForWake ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiSetupGpeForWake ( ACPI_HANDLE ParentDevice, ACPI_HANDLE GpeDevice, UINT32 GpeNumber)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiSetGpeWakeMask ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, UINT8 Action)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiGetGpeStatus ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, ACPI_EVENT_STATUS *EventStatus)) ACPI_HW_DEPENDENT_RETURN_UINT32 ( UINT32 AcpiDispatchGpe ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiDisableAllGpes ( void)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiEnableAllRuntimeGpes ( void)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiEnableAllWakeupGpes ( + void)) + +ACPI_HW_DEPENDENT_RETURN_UINT32 ( + UINT32 AcpiAnyGpeStatusSet ( void)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiGetGpeDevice ( UINT32 GpeIndex, ACPI_HANDLE *GpeDevice)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiInstallGpeBlock ( ACPI_HANDLE GpeDevice, ACPI_GENERIC_ADDRESS *GpeBlockAddress, UINT32 RegisterCount, UINT32 InterruptNumber)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiRemoveGpeBlock ( ACPI_HANDLE GpeDevice)) /* * Resource interfaces */ typedef ACPI_STATUS (*ACPI_WALK_RESOURCE_CALLBACK) ( ACPI_RESOURCE *Resource, void *Context); ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetVendorResource ( ACPI_HANDLE Device, char *Name, ACPI_VENDOR_UUID *Uuid, ACPI_BUFFER *RetBuffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetCurrentResources ( ACPI_HANDLE Device, ACPI_BUFFER *RetBuffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetPossibleResources ( ACPI_HANDLE Device, ACPI_BUFFER *RetBuffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetEventResources ( ACPI_HANDLE DeviceHandle, ACPI_BUFFER *RetBuffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiWalkResourceBuffer ( ACPI_BUFFER *Buffer, ACPI_WALK_RESOURCE_CALLBACK UserFunction, void *Context)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiWalkResources ( ACPI_HANDLE Device, char *Name, ACPI_WALK_RESOURCE_CALLBACK UserFunction, void *Context)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiSetCurrentResources ( ACPI_HANDLE Device, ACPI_BUFFER *InBuffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetIrqRoutingTable ( ACPI_HANDLE Device, ACPI_BUFFER *RetBuffer)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiResourceToAddress64 ( ACPI_RESOURCE *Resource, ACPI_RESOURCE_ADDRESS64 *Out)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiBufferToResource ( UINT8 *AmlBuffer, UINT16 AmlBufferLength, ACPI_RESOURCE **ResourcePtr)) /* * Hardware (ACPI device) interfaces */ ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiReset ( void)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiRead ( UINT64 *Value, ACPI_GENERIC_ADDRESS *Reg)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiWrite ( UINT64 Value, ACPI_GENERIC_ADDRESS *Reg)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiReadBitRegister ( UINT32 RegisterId, UINT32 *ReturnValue)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiWriteBitRegister ( UINT32 RegisterId, UINT32 Value)) /* * Sleep/Wake interfaces */ ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiGetSleepTypeData ( UINT8 SleepState, UINT8 *Slp_TypA, UINT8 *Slp_TypB)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiEnterSleepStatePrep ( UINT8 SleepState)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiEnterSleepState ( UINT8 SleepState)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiEnterSleepStateS4bios ( void)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiLeaveSleepStatePrep ( UINT8 SleepState)) ACPI_EXTERNAL_RETURN_STATUS ( ACPI_STATUS AcpiLeaveSleepState ( UINT8 SleepState)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiSetFirmwareWakingVector ( ACPI_PHYSICAL_ADDRESS PhysicalAddress, ACPI_PHYSICAL_ADDRESS PhysicalAddress64)) /* * ACPI Timer interfaces */ ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiGetTimerResolution ( UINT32 *Resolution)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiGetTimer ( UINT32 *Ticks)) ACPI_HW_DEPENDENT_RETURN_STATUS ( ACPI_STATUS AcpiGetTimerDuration ( UINT32 StartTicks, UINT32 EndTicks, UINT32 *TimeElapsed)) /* * Error/Warning output */ ACPI_MSG_DEPENDENT_RETURN_VOID ( ACPI_PRINTF_LIKE(3) void ACPI_INTERNAL_VAR_XFACE AcpiError ( const char *ModuleName, UINT32 LineNumber, const char *Format, ...)) ACPI_MSG_DEPENDENT_RETURN_VOID ( ACPI_PRINTF_LIKE(4) void ACPI_INTERNAL_VAR_XFACE AcpiException ( const char *ModuleName, UINT32 LineNumber, ACPI_STATUS Status, const char *Format, ...)) ACPI_MSG_DEPENDENT_RETURN_VOID ( ACPI_PRINTF_LIKE(3) void ACPI_INTERNAL_VAR_XFACE AcpiWarning ( const char *ModuleName, UINT32 LineNumber, const char *Format, ...)) ACPI_MSG_DEPENDENT_RETURN_VOID ( ACPI_PRINTF_LIKE(1) void ACPI_INTERNAL_VAR_XFACE AcpiInfo ( const char *Format, ...)) ACPI_MSG_DEPENDENT_RETURN_VOID ( ACPI_PRINTF_LIKE(3) void ACPI_INTERNAL_VAR_XFACE AcpiBiosError ( const char *ModuleName, UINT32 LineNumber, const char *Format, ...)) ACPI_MSG_DEPENDENT_RETURN_VOID ( ACPI_PRINTF_LIKE(4) void ACPI_INTERNAL_VAR_XFACE AcpiBiosException ( const char *ModuleName, UINT32 LineNumber, ACPI_STATUS Status, const char *Format, ...)) ACPI_MSG_DEPENDENT_RETURN_VOID ( ACPI_PRINTF_LIKE(3) void ACPI_INTERNAL_VAR_XFACE AcpiBiosWarning ( const char *ModuleName, UINT32 LineNumber, const char *Format, ...)) /* * Debug output */ ACPI_DBG_DEPENDENT_RETURN_VOID ( ACPI_PRINTF_LIKE(6) void ACPI_INTERNAL_VAR_XFACE AcpiDebugPrint ( UINT32 RequestedDebugLevel, UINT32 LineNumber, const char *FunctionName, const char *ModuleName, UINT32 ComponentId, const char *Format, ...)) ACPI_DBG_DEPENDENT_RETURN_VOID ( ACPI_PRINTF_LIKE(6) void ACPI_INTERNAL_VAR_XFACE AcpiDebugPrintRaw ( UINT32 RequestedDebugLevel, UINT32 LineNumber, const char *FunctionName, const char *ModuleName, UINT32 ComponentId, const char *Format, ...)) ACPI_DBG_DEPENDENT_RETURN_VOID ( void AcpiTracePoint ( ACPI_TRACE_EVENT_TYPE Type, BOOLEAN Begin, UINT8 *Aml, char *Pathname)) ACPI_STATUS AcpiInitializeDebugger ( void); void AcpiTerminateDebugger ( void); void AcpiRunDebugger ( char *BatchBuffer); void AcpiSetDebuggerThreadId ( ACPI_THREAD_ID ThreadId); #endif /* __ACXFACE_H__ */ Index: projects/clang1000-import/sys/contrib/dev/acpica/include/actbl1.h =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/include/actbl1.h (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/include/actbl1.h (revision 357966) @@ -1,2052 +1,2052 @@ /****************************************************************************** * * Name: actbl1.h - Additional ACPI table definitions * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #ifndef __ACTBL1_H__ #define __ACTBL1_H__ /******************************************************************************* * * Additional ACPI Tables * * These tables are not consumed directly by the ACPICA subsystem, but are * included here to support device drivers and the AML disassembler. * ******************************************************************************/ /* * Values for description table header signatures for tables defined in this * file. Useful because they make it more difficult to inadvertently type in * the wrong signature. */ #define ACPI_SIG_ASF "ASF!" /* Alert Standard Format table */ #define ACPI_SIG_BERT "BERT" /* Boot Error Record Table */ #define ACPI_SIG_BGRT "BGRT" /* Boot Graphics Resource Table */ #define ACPI_SIG_BOOT "BOOT" /* Simple Boot Flag Table */ #define ACPI_SIG_CPEP "CPEP" /* Corrected Platform Error Polling table */ #define ACPI_SIG_CSRT "CSRT" /* Core System Resource Table */ #define ACPI_SIG_DBG2 "DBG2" /* Debug Port table type 2 */ #define ACPI_SIG_DBGP "DBGP" /* Debug Port table */ #define ACPI_SIG_DMAR "DMAR" /* DMA Remapping table */ #define ACPI_SIG_DRTM "DRTM" /* Dynamic Root of Trust for Measurement table */ #define ACPI_SIG_ECDT "ECDT" /* Embedded Controller Boot Resources Table */ #define ACPI_SIG_EINJ "EINJ" /* Error Injection table */ #define ACPI_SIG_ERST "ERST" /* Error Record Serialization Table */ #define ACPI_SIG_FPDT "FPDT" /* Firmware Performance Data Table */ #define ACPI_SIG_GTDT "GTDT" /* Generic Timer Description Table */ #define ACPI_SIG_HEST "HEST" /* Hardware Error Source Table */ #define ACPI_SIG_HMAT "HMAT" /* Heterogeneous Memory Attributes Table */ #define ACPI_SIG_HPET "HPET" /* High Precision Event Timer table */ #define ACPI_SIG_IBFT "IBFT" /* iSCSI Boot Firmware Table */ #define ACPI_SIG_S3PT "S3PT" /* S3 Performance (sub)Table */ #define ACPI_SIG_PCCS "PCC" /* PCC Shared Memory Region */ /* Reserved table signatures */ #define ACPI_SIG_MATR "MATR" /* Memory Address Translation Table */ #define ACPI_SIG_MSDM "MSDM" /* Microsoft Data Management Table */ /* * These tables have been seen in the field, but no definition has been found */ #ifdef ACPI_UNDEFINED_TABLES #define ACPI_SIG_ATKG "ATKG" #define ACPI_SIG_GSCI "GSCI" /* GMCH SCI table */ #define ACPI_SIG_IEIT "IEIT" #endif /* * All tables must be byte-packed to match the ACPI specification, since * the tables are provided by the system BIOS. */ #pragma pack(1) /* * Note: C bitfields are not used for this reason: * * "Bitfields are great and easy to read, but unfortunately the C language * does not specify the layout of bitfields in memory, which means they are * essentially useless for dealing with packed data in on-disk formats or * binary wire protocols." (Or ACPI tables and buffers.) "If you ask me, * this decision was a design error in C. Ritchie could have picked an order * and stuck with it." Norman Ramsey. * See http://stackoverflow.com/a/1053662/41661 */ /******************************************************************************* * * Common subtable headers * ******************************************************************************/ /* Generic subtable header (used in MADT, SRAT, etc.) */ typedef struct acpi_subtable_header { UINT8 Type; UINT8 Length; } ACPI_SUBTABLE_HEADER; /* Subtable header for WHEA tables (EINJ, ERST, WDAT) */ typedef struct acpi_whea_header { UINT8 Action; UINT8 Instruction; UINT8 Flags; UINT8 Reserved; ACPI_GENERIC_ADDRESS RegisterRegion; UINT64 Value; /* Value used with Read/Write register */ UINT64 Mask; /* Bitmask required for this register instruction */ } ACPI_WHEA_HEADER; /******************************************************************************* * * ASF - Alert Standard Format table (Signature "ASF!") * Revision 0x10 * * Conforms to the Alert Standard Format Specification V2.0, 23 April 2003 * ******************************************************************************/ typedef struct acpi_table_asf { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ } ACPI_TABLE_ASF; /* ASF subtable header */ typedef struct acpi_asf_header { UINT8 Type; UINT8 Reserved; UINT16 Length; } ACPI_ASF_HEADER; /* Values for Type field above */ enum AcpiAsfType { ACPI_ASF_TYPE_INFO = 0, ACPI_ASF_TYPE_ALERT = 1, ACPI_ASF_TYPE_CONTROL = 2, ACPI_ASF_TYPE_BOOT = 3, ACPI_ASF_TYPE_ADDRESS = 4, ACPI_ASF_TYPE_RESERVED = 5 }; /* * ASF subtables */ /* 0: ASF Information */ typedef struct acpi_asf_info { ACPI_ASF_HEADER Header; UINT8 MinResetValue; UINT8 MinPollInterval; UINT16 SystemId; UINT32 MfgId; UINT8 Flags; UINT8 Reserved2[3]; } ACPI_ASF_INFO; /* Masks for Flags field above */ #define ACPI_ASF_SMBUS_PROTOCOLS (1) /* 1: ASF Alerts */ typedef struct acpi_asf_alert { ACPI_ASF_HEADER Header; UINT8 AssertMask; UINT8 DeassertMask; UINT8 Alerts; UINT8 DataLength; } ACPI_ASF_ALERT; typedef struct acpi_asf_alert_data { UINT8 Address; UINT8 Command; UINT8 Mask; UINT8 Value; UINT8 SensorType; UINT8 Type; UINT8 Offset; UINT8 SourceType; UINT8 Severity; UINT8 SensorNumber; UINT8 Entity; UINT8 Instance; } ACPI_ASF_ALERT_DATA; /* 2: ASF Remote Control */ typedef struct acpi_asf_remote { ACPI_ASF_HEADER Header; UINT8 Controls; UINT8 DataLength; UINT16 Reserved2; } ACPI_ASF_REMOTE; typedef struct acpi_asf_control_data { UINT8 Function; UINT8 Address; UINT8 Command; UINT8 Value; } ACPI_ASF_CONTROL_DATA; /* 3: ASF RMCP Boot Options */ typedef struct acpi_asf_rmcp { ACPI_ASF_HEADER Header; UINT8 Capabilities[7]; UINT8 CompletionCode; UINT32 EnterpriseId; UINT8 Command; UINT16 Parameter; UINT16 BootOptions; UINT16 OemParameters; } ACPI_ASF_RMCP; /* 4: ASF Address */ typedef struct acpi_asf_address { ACPI_ASF_HEADER Header; UINT8 EpromAddress; UINT8 Devices; } ACPI_ASF_ADDRESS; /******************************************************************************* * * BERT - Boot Error Record Table (ACPI 4.0) * Version 1 * ******************************************************************************/ typedef struct acpi_table_bert { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT32 RegionLength; /* Length of the boot error region */ UINT64 Address; /* Physical address of the error region */ } ACPI_TABLE_BERT; /* Boot Error Region (not a subtable, pointed to by Address field above) */ typedef struct acpi_bert_region { UINT32 BlockStatus; /* Type of error information */ UINT32 RawDataOffset; /* Offset to raw error data */ UINT32 RawDataLength; /* Length of raw error data */ UINT32 DataLength; /* Length of generic error data */ UINT32 ErrorSeverity; /* Severity code */ } ACPI_BERT_REGION; /* Values for BlockStatus flags above */ #define ACPI_BERT_UNCORRECTABLE (1) #define ACPI_BERT_CORRECTABLE (1<<1) #define ACPI_BERT_MULTIPLE_UNCORRECTABLE (1<<2) #define ACPI_BERT_MULTIPLE_CORRECTABLE (1<<3) #define ACPI_BERT_ERROR_ENTRY_COUNT (0xFF<<4) /* 8 bits, error count */ /* Values for ErrorSeverity above */ enum AcpiBertErrorSeverity { ACPI_BERT_ERROR_CORRECTABLE = 0, ACPI_BERT_ERROR_FATAL = 1, ACPI_BERT_ERROR_CORRECTED = 2, ACPI_BERT_ERROR_NONE = 3, ACPI_BERT_ERROR_RESERVED = 4 /* 4 and greater are reserved */ }; /* * Note: The generic error data that follows the ErrorSeverity field above * uses the ACPI_HEST_GENERIC_DATA defined under the HEST table below */ /******************************************************************************* * * BGRT - Boot Graphics Resource Table (ACPI 5.0) * Version 1 * ******************************************************************************/ typedef struct acpi_table_bgrt { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT16 Version; UINT8 Status; UINT8 ImageType; UINT64 ImageAddress; UINT32 ImageOffsetX; UINT32 ImageOffsetY; } ACPI_TABLE_BGRT; /* Flags for Status field above */ #define ACPI_BGRT_DISPLAYED (1) #define ACPI_BGRT_ORIENTATION_OFFSET (3 << 1) /******************************************************************************* * * BOOT - Simple Boot Flag Table * Version 1 * * Conforms to the "Simple Boot Flag Specification", Version 2.1 * ******************************************************************************/ typedef struct acpi_table_boot { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT8 CmosIndex; /* Index in CMOS RAM for the boot register */ UINT8 Reserved[3]; } ACPI_TABLE_BOOT; /******************************************************************************* * * CPEP - Corrected Platform Error Polling table (ACPI 4.0) * Version 1 * ******************************************************************************/ typedef struct acpi_table_cpep { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT64 Reserved; } ACPI_TABLE_CPEP; /* Subtable */ typedef struct acpi_cpep_polling { ACPI_SUBTABLE_HEADER Header; UINT8 Id; /* Processor ID */ UINT8 Eid; /* Processor EID */ UINT32 Interval; /* Polling interval (msec) */ } ACPI_CPEP_POLLING; /******************************************************************************* * * CSRT - Core System Resource Table * Version 0 * * Conforms to the "Core System Resource Table (CSRT)", November 14, 2011 * ******************************************************************************/ typedef struct acpi_table_csrt { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ } ACPI_TABLE_CSRT; /* Resource Group subtable */ typedef struct acpi_csrt_group { UINT32 Length; UINT32 VendorId; UINT32 SubvendorId; UINT16 DeviceId; UINT16 SubdeviceId; UINT16 Revision; UINT16 Reserved; UINT32 SharedInfoLength; /* Shared data immediately follows (Length = SharedInfoLength) */ } ACPI_CSRT_GROUP; /* Shared Info subtable */ typedef struct acpi_csrt_shared_info { UINT16 MajorVersion; UINT16 MinorVersion; UINT32 MmioBaseLow; UINT32 MmioBaseHigh; UINT32 GsiInterrupt; UINT8 InterruptPolarity; UINT8 InterruptMode; UINT8 NumChannels; UINT8 DmaAddressWidth; UINT16 BaseRequestLine; UINT16 NumHandshakeSignals; UINT32 MaxBlockSize; /* Resource descriptors immediately follow (Length = Group Length - SharedInfoLength) */ } ACPI_CSRT_SHARED_INFO; /* Resource Descriptor subtable */ typedef struct acpi_csrt_descriptor { UINT32 Length; UINT16 Type; UINT16 Subtype; UINT32 Uid; /* Resource-specific information immediately follows */ } ACPI_CSRT_DESCRIPTOR; /* Resource Types */ #define ACPI_CSRT_TYPE_INTERRUPT 0x0001 #define ACPI_CSRT_TYPE_TIMER 0x0002 #define ACPI_CSRT_TYPE_DMA 0x0003 /* Resource Subtypes */ #define ACPI_CSRT_XRUPT_LINE 0x0000 #define ACPI_CSRT_XRUPT_CONTROLLER 0x0001 #define ACPI_CSRT_TIMER 0x0000 #define ACPI_CSRT_DMA_CHANNEL 0x0000 #define ACPI_CSRT_DMA_CONTROLLER 0x0001 /******************************************************************************* * * DBG2 - Debug Port Table 2 * Version 0 (Both main table and subtables) * * Conforms to "Microsoft Debug Port Table 2 (DBG2)", December 10, 2015 * ******************************************************************************/ typedef struct acpi_table_dbg2 { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT32 InfoOffset; UINT32 InfoCount; } ACPI_TABLE_DBG2; typedef struct acpi_dbg2_header { UINT32 InfoOffset; UINT32 InfoCount; } ACPI_DBG2_HEADER; /* Debug Device Information Subtable */ typedef struct acpi_dbg2_device { UINT8 Revision; UINT16 Length; UINT8 RegisterCount; /* Number of BaseAddress registers */ UINT16 NamepathLength; UINT16 NamepathOffset; UINT16 OemDataLength; UINT16 OemDataOffset; UINT16 PortType; UINT16 PortSubtype; UINT16 Reserved; UINT16 BaseAddressOffset; UINT16 AddressSizeOffset; /* * Data that follows: * BaseAddress (required) - Each in 12-byte Generic Address Structure format. * AddressSize (required) - Array of UINT32 sizes corresponding to each BaseAddress register. * Namepath (required) - Null terminated string. Single dot if not supported. * OemData (optional) - Length is OemDataLength. */ } ACPI_DBG2_DEVICE; /* Types for PortType field above */ #define ACPI_DBG2_SERIAL_PORT 0x8000 #define ACPI_DBG2_1394_PORT 0x8001 #define ACPI_DBG2_USB_PORT 0x8002 #define ACPI_DBG2_NET_PORT 0x8003 /* Subtypes for PortSubtype field above */ #define ACPI_DBG2_16550_COMPATIBLE 0x0000 #define ACPI_DBG2_16550_SUBSET 0x0001 #define ACPI_DBG2_ARM_PL011 0x0003 #define ACPI_DBG2_ARM_SBSA_32BIT 0x000D #define ACPI_DBG2_ARM_SBSA_GENERIC 0x000E #define ACPI_DBG2_ARM_DCC 0x000F #define ACPI_DBG2_BCM2835 0x0010 #define ACPI_DBG2_1394_STANDARD 0x0000 #define ACPI_DBG2_USB_XHCI 0x0000 #define ACPI_DBG2_USB_EHCI 0x0001 /******************************************************************************* * * DBGP - Debug Port table * Version 1 * * Conforms to the "Debug Port Specification", Version 1.00, 2/9/2000 * ******************************************************************************/ typedef struct acpi_table_dbgp { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT8 Type; /* 0=full 16550, 1=subset of 16550 */ UINT8 Reserved[3]; ACPI_GENERIC_ADDRESS DebugPort; } ACPI_TABLE_DBGP; /******************************************************************************* * * DMAR - DMA Remapping table * Version 1 * * Conforms to "Intel Virtualization Technology for Directed I/O", * Version 2.3, October 2014 * ******************************************************************************/ typedef struct acpi_table_dmar { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT8 Width; /* Host Address Width */ UINT8 Flags; UINT8 Reserved[10]; } ACPI_TABLE_DMAR; /* Masks for Flags field above */ #define ACPI_DMAR_INTR_REMAP (1) #define ACPI_DMAR_X2APIC_OPT_OUT (1<<1) #define ACPI_DMAR_X2APIC_MODE (1<<2) /* DMAR subtable header */ typedef struct acpi_dmar_header { UINT16 Type; UINT16 Length; } ACPI_DMAR_HEADER; /* Values for subtable type in ACPI_DMAR_HEADER */ enum AcpiDmarType { ACPI_DMAR_TYPE_HARDWARE_UNIT = 0, ACPI_DMAR_TYPE_RESERVED_MEMORY = 1, ACPI_DMAR_TYPE_ROOT_ATS = 2, ACPI_DMAR_TYPE_HARDWARE_AFFINITY = 3, ACPI_DMAR_TYPE_NAMESPACE = 4, ACPI_DMAR_TYPE_RESERVED = 5 /* 5 and greater are reserved */ }; /* DMAR Device Scope structure */ typedef struct acpi_dmar_device_scope { UINT8 EntryType; UINT8 Length; UINT16 Reserved; UINT8 EnumerationId; UINT8 Bus; } ACPI_DMAR_DEVICE_SCOPE; /* Values for EntryType in ACPI_DMAR_DEVICE_SCOPE - device types */ enum AcpiDmarScopeType { ACPI_DMAR_SCOPE_TYPE_NOT_USED = 0, ACPI_DMAR_SCOPE_TYPE_ENDPOINT = 1, ACPI_DMAR_SCOPE_TYPE_BRIDGE = 2, ACPI_DMAR_SCOPE_TYPE_IOAPIC = 3, ACPI_DMAR_SCOPE_TYPE_HPET = 4, ACPI_DMAR_SCOPE_TYPE_NAMESPACE = 5, ACPI_DMAR_SCOPE_TYPE_RESERVED = 6 /* 6 and greater are reserved */ }; typedef struct acpi_dmar_pci_path { UINT8 Device; UINT8 Function; } ACPI_DMAR_PCI_PATH; /* * DMAR Subtables, correspond to Type in ACPI_DMAR_HEADER */ /* 0: Hardware Unit Definition */ typedef struct acpi_dmar_hardware_unit { ACPI_DMAR_HEADER Header; UINT8 Flags; UINT8 Reserved; UINT16 Segment; UINT64 Address; /* Register Base Address */ } ACPI_DMAR_HARDWARE_UNIT; /* Masks for Flags field above */ #define ACPI_DMAR_INCLUDE_ALL (1) /* 1: Reserved Memory Definition */ typedef struct acpi_dmar_reserved_memory { ACPI_DMAR_HEADER Header; UINT16 Reserved; UINT16 Segment; UINT64 BaseAddress; /* 4K aligned base address */ UINT64 EndAddress; /* 4K aligned limit address */ } ACPI_DMAR_RESERVED_MEMORY; /* Masks for Flags field above */ #define ACPI_DMAR_ALLOW_ALL (1) /* 2: Root Port ATS Capability Reporting Structure */ typedef struct acpi_dmar_atsr { ACPI_DMAR_HEADER Header; UINT8 Flags; UINT8 Reserved; UINT16 Segment; } ACPI_DMAR_ATSR; /* Masks for Flags field above */ #define ACPI_DMAR_ALL_PORTS (1) /* 3: Remapping Hardware Static Affinity Structure */ typedef struct acpi_dmar_rhsa { ACPI_DMAR_HEADER Header; UINT32 Reserved; UINT64 BaseAddress; UINT32 ProximityDomain; } ACPI_DMAR_RHSA; /* 4: ACPI Namespace Device Declaration Structure */ typedef struct acpi_dmar_andd { ACPI_DMAR_HEADER Header; UINT8 Reserved[3]; UINT8 DeviceNumber; char DeviceName[1]; } ACPI_DMAR_ANDD; /******************************************************************************* * * DRTM - Dynamic Root of Trust for Measurement table * Conforms to "TCG D-RTM Architecture" June 17 2013, Version 1.0.0 * Table version 1 * ******************************************************************************/ typedef struct acpi_table_drtm { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT64 EntryBaseAddress; UINT64 EntryLength; UINT32 EntryAddress32; UINT64 EntryAddress64; UINT64 ExitAddress; UINT64 LogAreaAddress; UINT32 LogAreaLength; UINT64 ArchDependentAddress; UINT32 Flags; } ACPI_TABLE_DRTM; /* Flag Definitions for above */ #define ACPI_DRTM_ACCESS_ALLOWED (1) #define ACPI_DRTM_ENABLE_GAP_CODE (1<<1) #define ACPI_DRTM_INCOMPLETE_MEASUREMENTS (1<<2) #define ACPI_DRTM_AUTHORITY_ORDER (1<<3) /* 1) Validated Tables List (64-bit addresses) */ typedef struct acpi_drtm_vtable_list { UINT32 ValidatedTableCount; UINT64 ValidatedTables[1]; } ACPI_DRTM_VTABLE_LIST; /* 2) Resources List (of Resource Descriptors) */ /* Resource Descriptor */ typedef struct acpi_drtm_resource { UINT8 Size[7]; UINT8 Type; UINT64 Address; } ACPI_DRTM_RESOURCE; typedef struct acpi_drtm_resource_list { UINT32 ResourceCount; ACPI_DRTM_RESOURCE Resources[1]; } ACPI_DRTM_RESOURCE_LIST; /* 3) Platform-specific Identifiers List */ typedef struct acpi_drtm_dps_id { UINT32 DpsIdLength; UINT8 DpsId[16]; } ACPI_DRTM_DPS_ID; /******************************************************************************* * * ECDT - Embedded Controller Boot Resources Table * Version 1 * ******************************************************************************/ typedef struct acpi_table_ecdt { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ ACPI_GENERIC_ADDRESS Control; /* Address of EC command/status register */ ACPI_GENERIC_ADDRESS Data; /* Address of EC data register */ UINT32 Uid; /* Unique ID - must be same as the EC _UID method */ UINT8 Gpe; /* The GPE for the EC */ UINT8 Id[1]; /* Full namepath of the EC in the ACPI namespace */ } ACPI_TABLE_ECDT; /******************************************************************************* * * EINJ - Error Injection Table (ACPI 4.0) * Version 1 * ******************************************************************************/ typedef struct acpi_table_einj { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT32 HeaderLength; UINT8 Flags; UINT8 Reserved[3]; UINT32 Entries; } ACPI_TABLE_EINJ; /* EINJ Injection Instruction Entries (actions) */ typedef struct acpi_einj_entry { ACPI_WHEA_HEADER WheaHeader; /* Common header for WHEA tables */ } ACPI_EINJ_ENTRY; /* Masks for Flags field above */ #define ACPI_EINJ_PRESERVE (1) /* Values for Action field above */ enum AcpiEinjActions { ACPI_EINJ_BEGIN_OPERATION = 0, ACPI_EINJ_GET_TRIGGER_TABLE = 1, ACPI_EINJ_SET_ERROR_TYPE = 2, ACPI_EINJ_GET_ERROR_TYPE = 3, ACPI_EINJ_END_OPERATION = 4, ACPI_EINJ_EXECUTE_OPERATION = 5, ACPI_EINJ_CHECK_BUSY_STATUS = 6, ACPI_EINJ_GET_COMMAND_STATUS = 7, ACPI_EINJ_SET_ERROR_TYPE_WITH_ADDRESS = 8, ACPI_EINJ_GET_EXECUTE_TIMINGS = 9, ACPI_EINJ_ACTION_RESERVED = 10, /* 10 and greater are reserved */ ACPI_EINJ_TRIGGER_ERROR = 0xFF /* Except for this value */ }; /* Values for Instruction field above */ enum AcpiEinjInstructions { ACPI_EINJ_READ_REGISTER = 0, ACPI_EINJ_READ_REGISTER_VALUE = 1, ACPI_EINJ_WRITE_REGISTER = 2, ACPI_EINJ_WRITE_REGISTER_VALUE = 3, ACPI_EINJ_NOOP = 4, ACPI_EINJ_FLUSH_CACHELINE = 5, ACPI_EINJ_INSTRUCTION_RESERVED = 6 /* 6 and greater are reserved */ }; typedef struct acpi_einj_error_type_with_addr { UINT32 ErrorType; UINT32 VendorStructOffset; UINT32 Flags; UINT32 ApicId; UINT64 Address; UINT64 Range; UINT32 PcieId; } ACPI_EINJ_ERROR_TYPE_WITH_ADDR; typedef struct acpi_einj_vendor { UINT32 Length; UINT32 PcieId; UINT16 VendorId; UINT16 DeviceId; UINT8 RevisionId; UINT8 Reserved[3]; } ACPI_EINJ_VENDOR; /* EINJ Trigger Error Action Table */ typedef struct acpi_einj_trigger { UINT32 HeaderSize; UINT32 Revision; UINT32 TableSize; UINT32 EntryCount; } ACPI_EINJ_TRIGGER; /* Command status return values */ enum AcpiEinjCommandStatus { ACPI_EINJ_SUCCESS = 0, ACPI_EINJ_FAILURE = 1, ACPI_EINJ_INVALID_ACCESS = 2, ACPI_EINJ_STATUS_RESERVED = 3 /* 3 and greater are reserved */ }; /* Error types returned from ACPI_EINJ_GET_ERROR_TYPE (bitfield) */ #define ACPI_EINJ_PROCESSOR_CORRECTABLE (1) #define ACPI_EINJ_PROCESSOR_UNCORRECTABLE (1<<1) #define ACPI_EINJ_PROCESSOR_FATAL (1<<2) #define ACPI_EINJ_MEMORY_CORRECTABLE (1<<3) #define ACPI_EINJ_MEMORY_UNCORRECTABLE (1<<4) #define ACPI_EINJ_MEMORY_FATAL (1<<5) #define ACPI_EINJ_PCIX_CORRECTABLE (1<<6) #define ACPI_EINJ_PCIX_UNCORRECTABLE (1<<7) #define ACPI_EINJ_PCIX_FATAL (1<<8) #define ACPI_EINJ_PLATFORM_CORRECTABLE (1<<9) #define ACPI_EINJ_PLATFORM_UNCORRECTABLE (1<<10) #define ACPI_EINJ_PLATFORM_FATAL (1<<11) #define ACPI_EINJ_VENDOR_DEFINED (1<<31) /******************************************************************************* * * ERST - Error Record Serialization Table (ACPI 4.0) * Version 1 * ******************************************************************************/ typedef struct acpi_table_erst { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT32 HeaderLength; UINT32 Reserved; UINT32 Entries; } ACPI_TABLE_ERST; /* ERST Serialization Entries (actions) */ typedef struct acpi_erst_entry { ACPI_WHEA_HEADER WheaHeader; /* Common header for WHEA tables */ } ACPI_ERST_ENTRY; /* Masks for Flags field above */ #define ACPI_ERST_PRESERVE (1) /* Values for Action field above */ enum AcpiErstActions { ACPI_ERST_BEGIN_WRITE = 0, ACPI_ERST_BEGIN_READ = 1, ACPI_ERST_BEGIN_CLEAR = 2, ACPI_ERST_END = 3, ACPI_ERST_SET_RECORD_OFFSET = 4, ACPI_ERST_EXECUTE_OPERATION = 5, ACPI_ERST_CHECK_BUSY_STATUS = 6, ACPI_ERST_GET_COMMAND_STATUS = 7, ACPI_ERST_GET_RECORD_ID = 8, ACPI_ERST_SET_RECORD_ID = 9, ACPI_ERST_GET_RECORD_COUNT = 10, ACPI_ERST_BEGIN_DUMMY_WRIITE = 11, ACPI_ERST_NOT_USED = 12, ACPI_ERST_GET_ERROR_RANGE = 13, ACPI_ERST_GET_ERROR_LENGTH = 14, ACPI_ERST_GET_ERROR_ATTRIBUTES = 15, ACPI_ERST_EXECUTE_TIMINGS = 16, ACPI_ERST_ACTION_RESERVED = 17 /* 17 and greater are reserved */ }; /* Values for Instruction field above */ enum AcpiErstInstructions { ACPI_ERST_READ_REGISTER = 0, ACPI_ERST_READ_REGISTER_VALUE = 1, ACPI_ERST_WRITE_REGISTER = 2, ACPI_ERST_WRITE_REGISTER_VALUE = 3, ACPI_ERST_NOOP = 4, ACPI_ERST_LOAD_VAR1 = 5, ACPI_ERST_LOAD_VAR2 = 6, ACPI_ERST_STORE_VAR1 = 7, ACPI_ERST_ADD = 8, ACPI_ERST_SUBTRACT = 9, ACPI_ERST_ADD_VALUE = 10, ACPI_ERST_SUBTRACT_VALUE = 11, ACPI_ERST_STALL = 12, ACPI_ERST_STALL_WHILE_TRUE = 13, ACPI_ERST_SKIP_NEXT_IF_TRUE = 14, ACPI_ERST_GOTO = 15, ACPI_ERST_SET_SRC_ADDRESS_BASE = 16, ACPI_ERST_SET_DST_ADDRESS_BASE = 17, ACPI_ERST_MOVE_DATA = 18, ACPI_ERST_INSTRUCTION_RESERVED = 19 /* 19 and greater are reserved */ }; /* Command status return values */ enum AcpiErstCommandStatus { - ACPI_ERST_SUCESS = 0, + ACPI_ERST_SUCCESS = 0, ACPI_ERST_NO_SPACE = 1, ACPI_ERST_NOT_AVAILABLE = 2, ACPI_ERST_FAILURE = 3, ACPI_ERST_RECORD_EMPTY = 4, ACPI_ERST_NOT_FOUND = 5, ACPI_ERST_STATUS_RESERVED = 6 /* 6 and greater are reserved */ }; /* Error Record Serialization Information */ typedef struct acpi_erst_info { UINT16 Signature; /* Should be "ER" */ UINT8 Data[48]; } ACPI_ERST_INFO; /******************************************************************************* * * FPDT - Firmware Performance Data Table (ACPI 5.0) * Version 1 * ******************************************************************************/ typedef struct acpi_table_fpdt { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ } ACPI_TABLE_FPDT; /* FPDT subtable header (Performance Record Structure) */ typedef struct acpi_fpdt_header { UINT16 Type; UINT8 Length; UINT8 Revision; } ACPI_FPDT_HEADER; /* Values for Type field above */ enum AcpiFpdtType { ACPI_FPDT_TYPE_BOOT = 0, ACPI_FPDT_TYPE_S3PERF = 1 }; /* * FPDT subtables */ /* 0: Firmware Basic Boot Performance Record */ typedef struct acpi_fpdt_boot_pointer { ACPI_FPDT_HEADER Header; UINT8 Reserved[4]; UINT64 Address; } ACPI_FPDT_BOOT_POINTER; /* 1: S3 Performance Table Pointer Record */ typedef struct acpi_fpdt_s3pt_pointer { ACPI_FPDT_HEADER Header; UINT8 Reserved[4]; UINT64 Address; } ACPI_FPDT_S3PT_POINTER; /* * S3PT - S3 Performance Table. This table is pointed to by the * S3 Pointer Record above. */ typedef struct acpi_table_s3pt { UINT8 Signature[4]; /* "S3PT" */ UINT32 Length; } ACPI_TABLE_S3PT; /* * S3PT Subtables (Not part of the actual FPDT) */ /* Values for Type field in S3PT header */ enum AcpiS3ptType { ACPI_S3PT_TYPE_RESUME = 0, ACPI_S3PT_TYPE_SUSPEND = 1, ACPI_FPDT_BOOT_PERFORMANCE = 2 }; typedef struct acpi_s3pt_resume { ACPI_FPDT_HEADER Header; UINT32 ResumeCount; UINT64 FullResume; UINT64 AverageResume; } ACPI_S3PT_RESUME; typedef struct acpi_s3pt_suspend { ACPI_FPDT_HEADER Header; UINT64 SuspendStart; UINT64 SuspendEnd; } ACPI_S3PT_SUSPEND; /* * FPDT Boot Performance Record (Not part of the actual FPDT) */ typedef struct acpi_fpdt_boot { ACPI_FPDT_HEADER Header; UINT8 Reserved[4]; UINT64 ResetEnd; UINT64 LoadStart; UINT64 StartupStart; UINT64 ExitServicesEntry; UINT64 ExitServicesExit; } ACPI_FPDT_BOOT; /******************************************************************************* * * GTDT - Generic Timer Description Table (ACPI 5.1) * Version 2 * ******************************************************************************/ typedef struct acpi_table_gtdt { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT64 CounterBlockAddresss; UINT32 Reserved; UINT32 SecureEl1Interrupt; UINT32 SecureEl1Flags; UINT32 NonSecureEl1Interrupt; UINT32 NonSecureEl1Flags; UINT32 VirtualTimerInterrupt; UINT32 VirtualTimerFlags; UINT32 NonSecureEl2Interrupt; UINT32 NonSecureEl2Flags; UINT64 CounterReadBlockAddress; UINT32 PlatformTimerCount; UINT32 PlatformTimerOffset; } ACPI_TABLE_GTDT; /* Flag Definitions: Timer Block Physical Timers and Virtual timers */ #define ACPI_GTDT_INTERRUPT_MODE (1) #define ACPI_GTDT_INTERRUPT_POLARITY (1<<1) #define ACPI_GTDT_ALWAYS_ON (1<<2) typedef struct acpi_gtdt_el2 { UINT32 VirtualEL2TimerGsiv; UINT32 VirtualEL2TimerFlags; } ACPI_GTDT_EL2; /* Common GTDT subtable header */ typedef struct acpi_gtdt_header { UINT8 Type; UINT16 Length; } ACPI_GTDT_HEADER; /* Values for GTDT subtable type above */ enum AcpiGtdtType { ACPI_GTDT_TYPE_TIMER_BLOCK = 0, ACPI_GTDT_TYPE_WATCHDOG = 1, ACPI_GTDT_TYPE_RESERVED = 2 /* 2 and greater are reserved */ }; /* GTDT Subtables, correspond to Type in acpi_gtdt_header */ /* 0: Generic Timer Block */ typedef struct acpi_gtdt_timer_block { ACPI_GTDT_HEADER Header; UINT8 Reserved; UINT64 BlockAddress; UINT32 TimerCount; UINT32 TimerOffset; } ACPI_GTDT_TIMER_BLOCK; /* Timer Sub-Structure, one per timer */ typedef struct acpi_gtdt_timer_entry { UINT8 FrameNumber; UINT8 Reserved[3]; UINT64 BaseAddress; UINT64 El0BaseAddress; UINT32 TimerInterrupt; UINT32 TimerFlags; UINT32 VirtualTimerInterrupt; UINT32 VirtualTimerFlags; UINT32 CommonFlags; } ACPI_GTDT_TIMER_ENTRY; /* Flag Definitions: TimerFlags and VirtualTimerFlags above */ #define ACPI_GTDT_GT_IRQ_MODE (1) #define ACPI_GTDT_GT_IRQ_POLARITY (1<<1) /* Flag Definitions: CommonFlags above */ #define ACPI_GTDT_GT_IS_SECURE_TIMER (1) #define ACPI_GTDT_GT_ALWAYS_ON (1<<1) /* 1: SBSA Generic Watchdog Structure */ typedef struct acpi_gtdt_watchdog { ACPI_GTDT_HEADER Header; UINT8 Reserved; UINT64 RefreshFrameAddress; UINT64 ControlFrameAddress; UINT32 TimerInterrupt; UINT32 TimerFlags; } ACPI_GTDT_WATCHDOG; /* Flag Definitions: TimerFlags above */ #define ACPI_GTDT_WATCHDOG_IRQ_MODE (1) #define ACPI_GTDT_WATCHDOG_IRQ_POLARITY (1<<1) #define ACPI_GTDT_WATCHDOG_SECURE (1<<2) /******************************************************************************* * * HEST - Hardware Error Source Table (ACPI 4.0) * Version 1 * ******************************************************************************/ typedef struct acpi_table_hest { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT32 ErrorSourceCount; } ACPI_TABLE_HEST; /* HEST subtable header */ typedef struct acpi_hest_header { UINT16 Type; UINT16 SourceId; } ACPI_HEST_HEADER; /* Values for Type field above for subtables */ enum AcpiHestTypes { ACPI_HEST_TYPE_IA32_CHECK = 0, ACPI_HEST_TYPE_IA32_CORRECTED_CHECK = 1, ACPI_HEST_TYPE_IA32_NMI = 2, ACPI_HEST_TYPE_NOT_USED3 = 3, ACPI_HEST_TYPE_NOT_USED4 = 4, ACPI_HEST_TYPE_NOT_USED5 = 5, ACPI_HEST_TYPE_AER_ROOT_PORT = 6, ACPI_HEST_TYPE_AER_ENDPOINT = 7, ACPI_HEST_TYPE_AER_BRIDGE = 8, ACPI_HEST_TYPE_GENERIC_ERROR = 9, ACPI_HEST_TYPE_GENERIC_ERROR_V2 = 10, ACPI_HEST_TYPE_IA32_DEFERRED_CHECK = 11, ACPI_HEST_TYPE_RESERVED = 12 /* 12 and greater are reserved */ }; /* * HEST substructures contained in subtables */ /* * IA32 Error Bank(s) - Follows the ACPI_HEST_IA_MACHINE_CHECK and * ACPI_HEST_IA_CORRECTED structures. */ typedef struct acpi_hest_ia_error_bank { UINT8 BankNumber; UINT8 ClearStatusOnInit; UINT8 StatusFormat; UINT8 Reserved; UINT32 ControlRegister; UINT64 ControlData; UINT32 StatusRegister; UINT32 AddressRegister; UINT32 MiscRegister; } ACPI_HEST_IA_ERROR_BANK; /* Common HEST sub-structure for PCI/AER structures below (6,7,8) */ typedef struct acpi_hest_aer_common { UINT16 Reserved1; UINT8 Flags; UINT8 Enabled; UINT32 RecordsToPreallocate; UINT32 MaxSectionsPerRecord; UINT32 Bus; /* Bus and Segment numbers */ UINT16 Device; UINT16 Function; UINT16 DeviceControl; UINT16 Reserved2; UINT32 UncorrectableMask; UINT32 UncorrectableSeverity; UINT32 CorrectableMask; UINT32 AdvancedCapabilities; } ACPI_HEST_AER_COMMON; /* Masks for HEST Flags fields */ #define ACPI_HEST_FIRMWARE_FIRST (1) #define ACPI_HEST_GLOBAL (1<<1) #define ACPI_HEST_GHES_ASSIST (1<<2) /* * Macros to access the bus/segment numbers in Bus field above: * Bus number is encoded in bits 7:0 * Segment number is encoded in bits 23:8 */ #define ACPI_HEST_BUS(Bus) ((Bus) & 0xFF) #define ACPI_HEST_SEGMENT(Bus) (((Bus) >> 8) & 0xFFFF) /* Hardware Error Notification */ typedef struct acpi_hest_notify { UINT8 Type; UINT8 Length; UINT16 ConfigWriteEnable; UINT32 PollInterval; UINT32 Vector; UINT32 PollingThresholdValue; UINT32 PollingThresholdWindow; UINT32 ErrorThresholdValue; UINT32 ErrorThresholdWindow; } ACPI_HEST_NOTIFY; /* Values for Notify Type field above */ enum AcpiHestNotifyTypes { ACPI_HEST_NOTIFY_POLLED = 0, ACPI_HEST_NOTIFY_EXTERNAL = 1, ACPI_HEST_NOTIFY_LOCAL = 2, ACPI_HEST_NOTIFY_SCI = 3, ACPI_HEST_NOTIFY_NMI = 4, ACPI_HEST_NOTIFY_CMCI = 5, /* ACPI 5.0 */ ACPI_HEST_NOTIFY_MCE = 6, /* ACPI 5.0 */ ACPI_HEST_NOTIFY_GPIO = 7, /* ACPI 6.0 */ ACPI_HEST_NOTIFY_SEA = 8, /* ACPI 6.1 */ ACPI_HEST_NOTIFY_SEI = 9, /* ACPI 6.1 */ ACPI_HEST_NOTIFY_GSIV = 10, /* ACPI 6.1 */ ACPI_HEST_NOTIFY_SOFTWARE_DELEGATED = 11, /* ACPI 6.2 */ ACPI_HEST_NOTIFY_RESERVED = 12 /* 12 and greater are reserved */ }; /* Values for ConfigWriteEnable bitfield above */ #define ACPI_HEST_TYPE (1) #define ACPI_HEST_POLL_INTERVAL (1<<1) #define ACPI_HEST_POLL_THRESHOLD_VALUE (1<<2) #define ACPI_HEST_POLL_THRESHOLD_WINDOW (1<<3) #define ACPI_HEST_ERR_THRESHOLD_VALUE (1<<4) #define ACPI_HEST_ERR_THRESHOLD_WINDOW (1<<5) /* * HEST subtables */ /* 0: IA32 Machine Check Exception */ typedef struct acpi_hest_ia_machine_check { ACPI_HEST_HEADER Header; UINT16 Reserved1; UINT8 Flags; /* See flags ACPI_HEST_GLOBAL, etc. above */ UINT8 Enabled; UINT32 RecordsToPreallocate; UINT32 MaxSectionsPerRecord; UINT64 GlobalCapabilityData; UINT64 GlobalControlData; UINT8 NumHardwareBanks; UINT8 Reserved3[7]; } ACPI_HEST_IA_MACHINE_CHECK; /* 1: IA32 Corrected Machine Check */ typedef struct acpi_hest_ia_corrected { ACPI_HEST_HEADER Header; UINT16 Reserved1; UINT8 Flags; /* See flags ACPI_HEST_GLOBAL, etc. above */ UINT8 Enabled; UINT32 RecordsToPreallocate; UINT32 MaxSectionsPerRecord; ACPI_HEST_NOTIFY Notify; UINT8 NumHardwareBanks; UINT8 Reserved2[3]; } ACPI_HEST_IA_CORRECTED; /* 2: IA32 Non-Maskable Interrupt */ typedef struct acpi_hest_ia_nmi { ACPI_HEST_HEADER Header; UINT32 Reserved; UINT32 RecordsToPreallocate; UINT32 MaxSectionsPerRecord; UINT32 MaxRawDataLength; } ACPI_HEST_IA_NMI; /* 3,4,5: Not used */ /* 6: PCI Express Root Port AER */ typedef struct acpi_hest_aer_root { ACPI_HEST_HEADER Header; ACPI_HEST_AER_COMMON Aer; UINT32 RootErrorCommand; } ACPI_HEST_AER_ROOT; /* 7: PCI Express AER (AER Endpoint) */ typedef struct acpi_hest_aer { ACPI_HEST_HEADER Header; ACPI_HEST_AER_COMMON Aer; } ACPI_HEST_AER; /* 8: PCI Express/PCI-X Bridge AER */ typedef struct acpi_hest_aer_bridge { ACPI_HEST_HEADER Header; ACPI_HEST_AER_COMMON Aer; UINT32 UncorrectableMask2; UINT32 UncorrectableSeverity2; UINT32 AdvancedCapabilities2; } ACPI_HEST_AER_BRIDGE; /* 9: Generic Hardware Error Source */ typedef struct acpi_hest_generic { ACPI_HEST_HEADER Header; UINT16 RelatedSourceId; UINT8 Reserved; UINT8 Enabled; UINT32 RecordsToPreallocate; UINT32 MaxSectionsPerRecord; UINT32 MaxRawDataLength; ACPI_GENERIC_ADDRESS ErrorStatusAddress; ACPI_HEST_NOTIFY Notify; UINT32 ErrorBlockLength; } ACPI_HEST_GENERIC; /* 10: Generic Hardware Error Source, version 2 */ typedef struct acpi_hest_generic_v2 { ACPI_HEST_HEADER Header; UINT16 RelatedSourceId; UINT8 Reserved; UINT8 Enabled; UINT32 RecordsToPreallocate; UINT32 MaxSectionsPerRecord; UINT32 MaxRawDataLength; ACPI_GENERIC_ADDRESS ErrorStatusAddress; ACPI_HEST_NOTIFY Notify; UINT32 ErrorBlockLength; ACPI_GENERIC_ADDRESS ReadAckRegister; UINT64 ReadAckPreserve; UINT64 ReadAckWrite; } ACPI_HEST_GENERIC_V2; /* Generic Error Status block */ typedef struct acpi_hest_generic_status { UINT32 BlockStatus; UINT32 RawDataOffset; UINT32 RawDataLength; UINT32 DataLength; UINT32 ErrorSeverity; } ACPI_HEST_GENERIC_STATUS; /* Values for BlockStatus flags above */ #define ACPI_HEST_UNCORRECTABLE (1) #define ACPI_HEST_CORRECTABLE (1<<1) #define ACPI_HEST_MULTIPLE_UNCORRECTABLE (1<<2) #define ACPI_HEST_MULTIPLE_CORRECTABLE (1<<3) #define ACPI_HEST_ERROR_ENTRY_COUNT (0xFF<<4) /* 8 bits, error count */ /* Generic Error Data entry */ typedef struct acpi_hest_generic_data { UINT8 SectionType[16]; UINT32 ErrorSeverity; UINT16 Revision; UINT8 ValidationBits; UINT8 Flags; UINT32 ErrorDataLength; UINT8 FruId[16]; UINT8 FruText[20]; } ACPI_HEST_GENERIC_DATA; /* Extension for revision 0x0300 */ typedef struct acpi_hest_generic_data_v300 { UINT8 SectionType[16]; UINT32 ErrorSeverity; UINT16 Revision; UINT8 ValidationBits; UINT8 Flags; UINT32 ErrorDataLength; UINT8 FruId[16]; UINT8 FruText[20]; UINT64 TimeStamp; } ACPI_HEST_GENERIC_DATA_V300; /* Values for ErrorSeverity above */ #define ACPI_HEST_GEN_ERROR_RECOVERABLE 0 #define ACPI_HEST_GEN_ERROR_FATAL 1 #define ACPI_HEST_GEN_ERROR_CORRECTED 2 #define ACPI_HEST_GEN_ERROR_NONE 3 /* Flags for ValidationBits above */ #define ACPI_HEST_GEN_VALID_FRU_ID (1) #define ACPI_HEST_GEN_VALID_FRU_STRING (1<<1) #define ACPI_HEST_GEN_VALID_TIMESTAMP (1<<2) /* 11: IA32 Deferred Machine Check Exception (ACPI 6.2) */ typedef struct acpi_hest_ia_deferred_check { ACPI_HEST_HEADER Header; UINT16 Reserved1; UINT8 Flags; /* See flags ACPI_HEST_GLOBAL, etc. above */ UINT8 Enabled; UINT32 RecordsToPreallocate; UINT32 MaxSectionsPerRecord; ACPI_HEST_NOTIFY Notify; UINT8 NumHardwareBanks; UINT8 Reserved2[3]; } ACPI_HEST_IA_DEFERRED_CHECK; /******************************************************************************* * * HMAT - Heterogeneous Memory Attributes Table (ACPI 6.2) * Version 1 * ******************************************************************************/ typedef struct acpi_table_hmat { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT32 Reserved; } ACPI_TABLE_HMAT; /* Values for HMAT structure types */ enum AcpiHmatType { ACPI_HMAT_TYPE_ADDRESS_RANGE = 0, /* Memory subsystem address range */ ACPI_HMAT_TYPE_LOCALITY = 1, /* System locality latency and bandwidth information */ ACPI_HMAT_TYPE_CACHE = 2, /* Memory side cache information */ ACPI_HMAT_TYPE_RESERVED = 3 /* 3 and greater are reserved */ }; typedef struct acpi_hmat_structure { UINT16 Type; UINT16 Reserved; UINT32 Length; } ACPI_HMAT_STRUCTURE; /* * HMAT Structures, correspond to Type in ACPI_HMAT_STRUCTURE */ /* 0: Memory proximity domain attributes */ typedef struct acpi_hmat_proximity_domain { ACPI_HMAT_STRUCTURE Header; UINT16 Flags; UINT16 Reserved1; UINT32 ProcessorPD; /* Processor proximity domain */ UINT32 MemoryPD; /* Memory proximity domain */ UINT32 Reserved2; UINT64 Reserved3; UINT64 Reserved4; } ACPI_HMAT_PROXIMITY_DOMAIN; /* Masks for Flags field above */ #define ACPI_HMAT_PROCESSOR_PD_VALID (1) /* 1: ProcessorPD field is valid */ #define ACPI_HMAT_MEMORY_PD_VALID (1<<1) /* 1: MemoryPD field is valid */ #define ACPI_HMAT_RESERVATION_HINT (1<<2) /* 1: Reservation hint */ /* 1: System locality latency and bandwidth information */ typedef struct acpi_hmat_locality { ACPI_HMAT_STRUCTURE Header; UINT8 Flags; UINT8 DataType; UINT16 Reserved1; UINT32 NumberOfInitiatorPDs; UINT32 NumberOfTargetPDs; UINT32 Reserved2; UINT64 EntryBaseUnit; } ACPI_HMAT_LOCALITY; /* Masks for Flags field above */ #define ACPI_HMAT_MEMORY_HIERARCHY (0x0F) /* Values for Memory Hierarchy flag */ #define ACPI_HMAT_MEMORY 0 #define ACPI_HMAT_LAST_LEVEL_CACHE 1 #define ACPI_HMAT_1ST_LEVEL_CACHE 2 #define ACPI_HMAT_2ND_LEVEL_CACHE 3 #define ACPI_HMAT_3RD_LEVEL_CACHE 4 /* Values for DataType field above */ #define ACPI_HMAT_ACCESS_LATENCY 0 #define ACPI_HMAT_READ_LATENCY 1 #define ACPI_HMAT_WRITE_LATENCY 2 #define ACPI_HMAT_ACCESS_BANDWIDTH 3 #define ACPI_HMAT_READ_BANDWIDTH 4 #define ACPI_HMAT_WRITE_BANDWIDTH 5 /* 2: Memory side cache information */ typedef struct acpi_hmat_cache { ACPI_HMAT_STRUCTURE Header; UINT32 MemoryPD; UINT32 Reserved1; UINT64 CacheSize; UINT32 CacheAttributes; UINT16 Reserved2; UINT16 NumberOfSMBIOSHandles; } ACPI_HMAT_CACHE; /* Masks for CacheAttributes field above */ #define ACPI_HMAT_TOTAL_CACHE_LEVEL (0x0000000F) #define ACPI_HMAT_CACHE_LEVEL (0x000000F0) #define ACPI_HMAT_CACHE_ASSOCIATIVITY (0x00000F00) #define ACPI_HMAT_WRITE_POLICY (0x0000F000) #define ACPI_HMAT_CACHE_LINE_SIZE (0xFFFF0000) /* Values for cache associativity flag */ #define ACPI_HMAT_CA_NONE (0) #define ACPI_HMAT_CA_DIRECT_MAPPED (1) #define ACPI_HMAT_CA_COMPLEX_CACHE_INDEXING (2) /* Values for write policy flag */ #define ACPI_HMAT_CP_NONE (0) #define ACPI_HMAT_CP_WB (1) #define ACPI_HMAT_CP_WT (2) /******************************************************************************* * * HPET - High Precision Event Timer table * Version 1 * * Conforms to "IA-PC HPET (High Precision Event Timers) Specification", * Version 1.0a, October 2004 * ******************************************************************************/ typedef struct acpi_table_hpet { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT32 Id; /* Hardware ID of event timer block */ ACPI_GENERIC_ADDRESS Address; /* Address of event timer block */ UINT8 Sequence; /* HPET sequence number */ UINT16 MinimumTick; /* Main counter min tick, periodic mode */ UINT8 Flags; } ACPI_TABLE_HPET; /* Masks for Flags field above */ #define ACPI_HPET_PAGE_PROTECT_MASK (3) /* Values for Page Protect flags */ enum AcpiHpetPageProtect { ACPI_HPET_NO_PAGE_PROTECT = 0, ACPI_HPET_PAGE_PROTECT4 = 1, ACPI_HPET_PAGE_PROTECT64 = 2 }; /******************************************************************************* * * IBFT - Boot Firmware Table * Version 1 * * Conforms to "iSCSI Boot Firmware Table (iBFT) as Defined in ACPI 3.0b * Specification", Version 1.01, March 1, 2007 * * Note: It appears that this table is not intended to appear in the RSDT/XSDT. * Therefore, it is not currently supported by the disassembler. * ******************************************************************************/ typedef struct acpi_table_ibft { ACPI_TABLE_HEADER Header; /* Common ACPI table header */ UINT8 Reserved[12]; } ACPI_TABLE_IBFT; /* IBFT common subtable header */ typedef struct acpi_ibft_header { UINT8 Type; UINT8 Version; UINT16 Length; UINT8 Index; UINT8 Flags; } ACPI_IBFT_HEADER; /* Values for Type field above */ enum AcpiIbftType { ACPI_IBFT_TYPE_NOT_USED = 0, ACPI_IBFT_TYPE_CONTROL = 1, ACPI_IBFT_TYPE_INITIATOR = 2, ACPI_IBFT_TYPE_NIC = 3, ACPI_IBFT_TYPE_TARGET = 4, ACPI_IBFT_TYPE_EXTENSIONS = 5, ACPI_IBFT_TYPE_RESERVED = 6 /* 6 and greater are reserved */ }; /* IBFT subtables */ typedef struct acpi_ibft_control { ACPI_IBFT_HEADER Header; UINT16 Extensions; UINT16 InitiatorOffset; UINT16 Nic0Offset; UINT16 Target0Offset; UINT16 Nic1Offset; UINT16 Target1Offset; } ACPI_IBFT_CONTROL; typedef struct acpi_ibft_initiator { ACPI_IBFT_HEADER Header; UINT8 SnsServer[16]; UINT8 SlpServer[16]; UINT8 PrimaryServer[16]; UINT8 SecondaryServer[16]; UINT16 NameLength; UINT16 NameOffset; } ACPI_IBFT_INITIATOR; typedef struct acpi_ibft_nic { ACPI_IBFT_HEADER Header; UINT8 IpAddress[16]; UINT8 SubnetMaskPrefix; UINT8 Origin; UINT8 Gateway[16]; UINT8 PrimaryDns[16]; UINT8 SecondaryDns[16]; UINT8 Dhcp[16]; UINT16 Vlan; UINT8 MacAddress[6]; UINT16 PciAddress; UINT16 NameLength; UINT16 NameOffset; } ACPI_IBFT_NIC; typedef struct acpi_ibft_target { ACPI_IBFT_HEADER Header; UINT8 TargetIpAddress[16]; UINT16 TargetIpSocket; UINT8 TargetBootLun[8]; UINT8 ChapType; UINT8 NicAssociation; UINT16 TargetNameLength; UINT16 TargetNameOffset; UINT16 ChapNameLength; UINT16 ChapNameOffset; UINT16 ChapSecretLength; UINT16 ChapSecretOffset; UINT16 ReverseChapNameLength; UINT16 ReverseChapNameOffset; UINT16 ReverseChapSecretLength; UINT16 ReverseChapSecretOffset; } ACPI_IBFT_TARGET; /* Reset to default packing */ #pragma pack() #endif /* __ACTBL1_H__ */ Index: projects/clang1000-import/sys/contrib/dev/acpica/include/actypes.h =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica/include/actypes.h (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica/include/actypes.h (revision 357966) @@ -1,1531 +1,1532 @@ /****************************************************************************** * * Name: actypes.h - Common data types for the entire ACPI subsystem * *****************************************************************************/ /****************************************************************************** * * 1. Copyright Notice * * Some or all of this work - Copyright (c) 1999 - 2020, Intel Corp. * All rights reserved. * * 2. License * * 2.1. This is your license from Intel Corp. under its intellectual property * rights. You may have additional license terms from the party that provided * you this software, covering your right to use that party's intellectual * property rights. * * 2.2. Intel grants, free of charge, to any person ("Licensee") obtaining a * copy of the source code appearing in this file ("Covered Code") an * irrevocable, perpetual, worldwide license under Intel's copyrights in the * base code distributed originally by Intel ("Original Intel Code") to copy, * make derivatives, distribute, use and display any portion of the Covered * Code in any form, with the right to sublicense such rights; and * * 2.3. Intel grants Licensee a non-exclusive and non-transferable patent * license (with the right to sublicense), under only those claims of Intel * patents that are infringed by the Original Intel Code, to make, use, sell, * offer to sell, and import the Covered Code and derivative works thereof * solely to the minimum extent necessary to exercise the above copyright * license, and in no event shall the patent license extend to any additions * to or modifications of the Original Intel Code. No other license or right * is granted directly or by implication, estoppel or otherwise; * * The above copyright and patent license is granted only if the following * conditions are met: * * 3. Conditions * * 3.1. Redistribution of Source with Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification with rights to further distribute source must include * the above Copyright Notice, the above License, this list of Conditions, * and the following Disclaimer and Export Compliance provision. In addition, * Licensee must cause all Covered Code to which Licensee contributes to * contain a file documenting the changes Licensee made to create that Covered * Code and the date of any change. Licensee must include in that file the * documentation of any changes made by any predecessor Licensee. Licensee * must include a prominent statement that the modification is derived, * directly or indirectly, from Original Intel Code. * * 3.2. Redistribution of Source with no Rights to Further Distribute Source. * Redistribution of source code of any substantial portion of the Covered * Code or modification without rights to further distribute source must * include the following Disclaimer and Export Compliance provision in the * documentation and/or other materials provided with distribution. In * addition, Licensee may not authorize further sublicense of source of any * portion of the Covered Code, and must include terms to the effect that the * license from Licensee to its licensee is limited to the intellectual * property embodied in the software Licensee provides to its licensee, and * not to intellectual property embodied in modifications its licensee may * make. * * 3.3. Redistribution of Executable. Redistribution in executable form of any * substantial portion of the Covered Code or modification must reproduce the * above Copyright Notice, and the following Disclaimer and Export Compliance * provision in the documentation and/or other materials provided with the * distribution. * * 3.4. Intel retains all right, title, and interest in and to the Original * Intel Code. * * 3.5. Neither the name Intel nor any other trademark owned or controlled by * Intel shall be used in advertising or otherwise to promote the sale, use or * other dealings in products derived from or relating to the Covered Code * without prior written authorization from Intel. * * 4. Disclaimer and Export Compliance * * 4.1. INTEL MAKES NO WARRANTY OF ANY KIND REGARDING ANY SOFTWARE PROVIDED * HERE. ANY SOFTWARE ORIGINATING FROM INTEL OR DERIVED FROM INTEL SOFTWARE * IS PROVIDED "AS IS," AND INTEL WILL NOT PROVIDE ANY SUPPORT, ASSISTANCE, * INSTALLATION, TRAINING OR OTHER SERVICES. INTEL WILL NOT PROVIDE ANY * UPDATES, ENHANCEMENTS OR EXTENSIONS. INTEL SPECIFICALLY DISCLAIMS ANY * IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS FOR A * PARTICULAR PURPOSE. * * 4.2. IN NO EVENT SHALL INTEL HAVE ANY LIABILITY TO LICENSEE, ITS LICENSEES * OR ANY OTHER THIRD PARTY, FOR ANY LOST PROFITS, LOST DATA, LOSS OF USE OR * COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR FOR ANY INDIRECT, * SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THIS AGREEMENT, UNDER ANY * CAUSE OF ACTION OR THEORY OF LIABILITY, AND IRRESPECTIVE OF WHETHER INTEL * HAS ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES. THESE LIMITATIONS * SHALL APPLY NOTWITHSTANDING THE FAILURE OF THE ESSENTIAL PURPOSE OF ANY * LIMITED REMEDY. * * 4.3. Licensee shall not export, either directly or indirectly, any of this * software or system incorporating such software without first obtaining any * required license or other approval from the U. S. Department of Commerce or * any other agency or department of the United States Government. In the * event Licensee exports any such software from the United States or * re-exports any such software from a foreign destination, Licensee shall * ensure that the distribution and export/re-export of the software is in * compliance with all laws, regulations, orders, or other restrictions of the * U.S. Export Administration Regulations. Licensee agrees that neither it nor * any of its subsidiaries will export/re-export any technical data, process, * software, or service, directly or indirectly, to any country for which the * United States government or any agency thereof requires an export license, * other governmental approval, or letter of assurance, without first obtaining * such license, approval or letter. * ***************************************************************************** * * Alternatively, you may choose to be licensed under the terms of the * following license: * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * substantially similar to the "NO WARRANTY" disclaimer below * ("Disclaimer") and any redistribution must be conditioned upon * including a substantially similar Disclaimer requirement for further * binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * Alternatively, you may choose to be licensed under the terms of the * GNU General Public License ("GPL") version 2 as published by the Free * Software Foundation. * *****************************************************************************/ #ifndef __ACTYPES_H__ #define __ACTYPES_H__ /* acpisrc:StructDefs -- for acpisrc conversion */ /* * ACPI_MACHINE_WIDTH must be specified in an OS- or compiler-dependent * header and must be either 32 or 64. 16-bit ACPICA is no longer * supported, as of 12/2006. */ #ifndef ACPI_MACHINE_WIDTH #error ACPI_MACHINE_WIDTH not defined #endif /* * Data type ranges * Note: These macros are designed to be compiler independent as well as * working around problems that some 32-bit compilers have with 64-bit * constants. */ #define ACPI_UINT8_MAX (UINT8) (~((UINT8) 0)) /* 0xFF */ #define ACPI_UINT16_MAX (UINT16)(~((UINT16) 0)) /* 0xFFFF */ #define ACPI_UINT32_MAX (UINT32)(~((UINT32) 0)) /* 0xFFFFFFFF */ #define ACPI_UINT64_MAX (UINT64)(~((UINT64) 0)) /* 0xFFFFFFFFFFFFFFFF */ #define ACPI_ASCII_MAX 0x7F /* * Architecture-specific ACPICA Subsystem Data Types * * The goal of these types is to provide source code portability across * 16-bit, 32-bit, and 64-bit targets. * * 1) The following types are of fixed size for all targets (16/32/64): * * BOOLEAN Logical boolean * * UINT8 8-bit (1 byte) unsigned value * UINT16 16-bit (2 byte) unsigned value * UINT32 32-bit (4 byte) unsigned value * UINT64 64-bit (8 byte) unsigned value * * INT16 16-bit (2 byte) signed value * INT32 32-bit (4 byte) signed value * INT64 64-bit (8 byte) signed value * * COMPILER_DEPENDENT_UINT64/INT64 - These types are defined in the * compiler-dependent header(s) and were introduced because there is no * common 64-bit integer type across the various compilation models, as * shown in the table below. * * Datatype LP64 ILP64 LLP64 ILP32 LP32 16bit * char 8 8 8 8 8 8 * short 16 16 16 16 16 16 * _int32 32 * int 32 64 32 32 16 16 * long 64 64 32 32 32 32 * long long 64 64 * pointer 64 64 64 32 32 32 * * Note: ILP64 and LP32 are currently not supported. * * * 2) These types represent the native word size of the target mode of the * processor, and may be 16-bit, 32-bit, or 64-bit as required. They are * usually used for memory allocation, efficient loop counters, and array * indexes. The types are similar to the size_t type in the C library and * are required because there is no C type that consistently represents the * native data width. ACPI_SIZE is needed because there is no guarantee * that a kernel-level C library is present. * * ACPI_SIZE 16/32/64-bit unsigned value * ACPI_NATIVE_INT 16/32/64-bit signed value */ /******************************************************************************* * * Common types for all compilers, all targets * ******************************************************************************/ #ifndef ACPI_USE_SYSTEM_INTTYPES typedef unsigned char BOOLEAN; typedef unsigned char UINT8; typedef unsigned short UINT16; typedef short INT16; typedef COMPILER_DEPENDENT_UINT64 UINT64; typedef COMPILER_DEPENDENT_INT64 INT64; #endif /* ACPI_USE_SYSTEM_INTTYPES */ /* * Value returned by AcpiOsGetThreadId. There is no standard "thread_id" * across operating systems or even the various UNIX systems. Since ACPICA * only needs the thread ID as a unique thread identifier, we use a UINT64 * as the only common data type - it will accommodate any type of pointer or * any type of integer. It is up to the host-dependent OSL to cast the * native thread ID type to a UINT64 (in AcpiOsGetThreadId). */ #define ACPI_THREAD_ID UINT64 /******************************************************************************* * * Types specific to 64-bit targets * ******************************************************************************/ #if ACPI_MACHINE_WIDTH == 64 #ifndef ACPI_USE_SYSTEM_INTTYPES typedef unsigned int UINT32; typedef int INT32; #endif /* ACPI_USE_SYSTEM_INTTYPES */ typedef INT64 ACPI_NATIVE_INT; typedef UINT64 ACPI_SIZE; typedef UINT64 ACPI_IO_ADDRESS; typedef UINT64 ACPI_PHYSICAL_ADDRESS; #define ACPI_MAX_PTR ACPI_UINT64_MAX #define ACPI_SIZE_MAX ACPI_UINT64_MAX #define ACPI_USE_NATIVE_DIVIDE /* Has native 64-bit integer support */ #define ACPI_USE_NATIVE_MATH64 /* Has native 64-bit integer support */ /* * In the case of the Itanium Processor Family (IPF), the hardware does not * support misaligned memory transfers. Set the MISALIGNMENT_NOT_SUPPORTED * flag to indicate that special precautions must be taken to avoid alignment * faults. (IA64 or ia64 is currently used by existing compilers to indicate * IPF.) * * Note: EM64T and other X86-64 processors support misaligned transfers, * so there is no need to define this flag. */ #if defined (__IA64__) || defined (__ia64__) #define ACPI_MISALIGNMENT_NOT_SUPPORTED #endif /******************************************************************************* * * Types specific to 32-bit targets * ******************************************************************************/ #elif ACPI_MACHINE_WIDTH == 32 #ifndef ACPI_USE_SYSTEM_INTTYPES typedef unsigned int UINT32; typedef int INT32; #endif /* ACPI_USE_SYSTEM_INTTYPES */ typedef INT32 ACPI_NATIVE_INT; typedef UINT32 ACPI_SIZE; #ifdef ACPI_32BIT_PHYSICAL_ADDRESS /* * OSPMs can define this to shrink the size of the structures for 32-bit * none PAE environment. ASL compiler may always define this to generate * 32-bit OSPM compliant tables. */ typedef UINT32 ACPI_IO_ADDRESS; typedef UINT32 ACPI_PHYSICAL_ADDRESS; #else /* ACPI_32BIT_PHYSICAL_ADDRESS */ /* * It is reported that, after some calculations, the physical addresses can * wrap over the 32-bit boundary on 32-bit PAE environment. * https://bugzilla.kernel.org/show_bug.cgi?id=87971 */ typedef UINT64 ACPI_IO_ADDRESS; typedef UINT64 ACPI_PHYSICAL_ADDRESS; #endif /* ACPI_32BIT_PHYSICAL_ADDRESS */ #define ACPI_MAX_PTR ACPI_UINT32_MAX #define ACPI_SIZE_MAX ACPI_UINT32_MAX #else /* ACPI_MACHINE_WIDTH must be either 64 or 32 */ #error unknown ACPI_MACHINE_WIDTH #endif /******************************************************************************* * * OS-dependent types * * If the defaults below are not appropriate for the host system, they can * be defined in the OS-specific header, and this will take precedence. * ******************************************************************************/ /* Flags for AcpiOsAcquireLock/AcpiOsReleaseLock */ #ifndef ACPI_CPU_FLAGS #define ACPI_CPU_FLAGS ACPI_SIZE #endif /* Object returned from AcpiOsCreateCache */ #ifndef ACPI_CACHE_T #ifdef ACPI_USE_LOCAL_CACHE #define ACPI_CACHE_T ACPI_MEMORY_LIST #else #define ACPI_CACHE_T void * #endif #endif /* * Synchronization objects - Mutexes, Semaphores, and SpinLocks */ #if (ACPI_MUTEX_TYPE == ACPI_BINARY_SEMAPHORE) /* * These macros are used if the host OS does not support a mutex object. * Map the OSL Mutex interfaces to binary semaphores. */ #define ACPI_MUTEX ACPI_SEMAPHORE #define AcpiOsCreateMutex(OutHandle) AcpiOsCreateSemaphore (1, 1, OutHandle) #define AcpiOsDeleteMutex(Handle) (void) AcpiOsDeleteSemaphore (Handle) #define AcpiOsAcquireMutex(Handle,Time) AcpiOsWaitSemaphore (Handle, 1, Time) #define AcpiOsReleaseMutex(Handle) (void) AcpiOsSignalSemaphore (Handle, 1) #endif /* Configurable types for synchronization objects */ #ifndef ACPI_SPINLOCK #define ACPI_SPINLOCK void * #endif #ifndef ACPI_SEMAPHORE #define ACPI_SEMAPHORE void * #endif #ifndef ACPI_MUTEX #define ACPI_MUTEX void * #endif /******************************************************************************* * * Compiler-dependent types * * If the defaults below are not appropriate for the host compiler, they can * be defined in the compiler-specific header, and this will take precedence. * ******************************************************************************/ /* Use C99 uintptr_t for pointer casting if available, "void *" otherwise */ #ifndef ACPI_UINTPTR_T #define ACPI_UINTPTR_T void * #endif /* * ACPI_PRINTF_LIKE is used to tag functions as "printf-like" because * some compilers can catch printf format string problems */ #ifndef ACPI_PRINTF_LIKE #define ACPI_PRINTF_LIKE(c) #endif /* * Some compilers complain about unused variables. Sometimes we don't want * to use all the variables (for example, _AcpiModuleName). This allows us * to tell the compiler in a per-variable manner that a variable * is unused */ #ifndef ACPI_UNUSED_VAR #define ACPI_UNUSED_VAR #endif /* * All ACPICA external functions that are available to the rest of the * kernel are tagged with these macros which can be defined as appropriate * for the host. * * Notes: * ACPI_EXPORT_SYMBOL_INIT is used for initialization and termination * interfaces that may need special processing. * ACPI_EXPORT_SYMBOL is used for all other public external functions. */ #ifndef ACPI_EXPORT_SYMBOL_INIT #define ACPI_EXPORT_SYMBOL_INIT(Symbol) #endif #ifndef ACPI_EXPORT_SYMBOL #define ACPI_EXPORT_SYMBOL(Symbol) #endif /* * Compiler/Clibrary-dependent debug initialization. Used for ACPICA * utilities only. */ #ifndef ACPI_DEBUG_INITIALIZE #define ACPI_DEBUG_INITIALIZE() #endif /******************************************************************************* * * Configuration * ******************************************************************************/ #ifdef ACPI_NO_MEM_ALLOCATIONS #define ACPI_ALLOCATE(a) NULL #define ACPI_ALLOCATE_ZEROED(a) NULL #define ACPI_FREE(a) #define ACPI_MEM_TRACKING(a) #else /* ACPI_NO_MEM_ALLOCATIONS */ #ifdef ACPI_DBG_TRACK_ALLOCATIONS /* * Memory allocation tracking (used by AcpiExec to detect memory leaks) */ #define ACPI_MEM_PARAMETERS _COMPONENT, _AcpiModuleName, __LINE__ #define ACPI_ALLOCATE(a) AcpiUtAllocateAndTrack ((ACPI_SIZE) (a), ACPI_MEM_PARAMETERS) #define ACPI_ALLOCATE_ZEROED(a) AcpiUtAllocateZeroedAndTrack ((ACPI_SIZE) (a), ACPI_MEM_PARAMETERS) #define ACPI_FREE(a) AcpiUtFreeAndTrack (a, ACPI_MEM_PARAMETERS) #define ACPI_MEM_TRACKING(a) a #else /* * Normal memory allocation directly via the OS services layer */ #define ACPI_ALLOCATE(a) AcpiOsAllocate ((ACPI_SIZE) (a)) #define ACPI_ALLOCATE_ZEROED(a) AcpiOsAllocateZeroed ((ACPI_SIZE) (a)) #define ACPI_FREE(a) AcpiOsFree (a) #define ACPI_MEM_TRACKING(a) #endif /* ACPI_DBG_TRACK_ALLOCATIONS */ #endif /* ACPI_NO_MEM_ALLOCATIONS */ /****************************************************************************** * * ACPI Specification constants (Do not change unless the specification * changes) * *****************************************************************************/ /* Number of distinct FADT-based GPE register blocks (GPE0 and GPE1) */ #define ACPI_MAX_GPE_BLOCKS 2 /* Default ACPI register widths */ #define ACPI_GPE_REGISTER_WIDTH 8 #define ACPI_PM1_REGISTER_WIDTH 16 #define ACPI_PM2_REGISTER_WIDTH 8 #define ACPI_PM_TIMER_WIDTH 32 #define ACPI_RESET_REGISTER_WIDTH 8 /* Names within the namespace are 4 bytes long */ #define ACPI_NAMESEG_SIZE 4 /* Fixed by ACPI spec */ #define ACPI_PATH_SEGMENT_LENGTH 5 /* 4 chars for name + 1 char for separator */ #define ACPI_PATH_SEPARATOR '.' /* Sizes for ACPI table headers */ #define ACPI_OEM_ID_SIZE 6 #define ACPI_OEM_TABLE_ID_SIZE 8 /* ACPI/PNP hardware IDs */ #define PCI_ROOT_HID_STRING "PNP0A03" #define PCI_EXPRESS_ROOT_HID_STRING "PNP0A08" /* PM Timer ticks per second (HZ) */ #define ACPI_PM_TIMER_FREQUENCY 3579545 /******************************************************************************* * * Independent types * ******************************************************************************/ /* Logical defines and NULL */ #ifdef FALSE #undef FALSE #endif #define FALSE (1 == 0) #ifdef TRUE #undef TRUE #endif #define TRUE (1 == 1) #ifndef NULL #define NULL (void *) 0 #endif /* * Miscellaneous types */ typedef UINT32 ACPI_STATUS; /* All ACPI Exceptions */ typedef UINT32 ACPI_NAME; /* 4-byte ACPI name */ typedef char * ACPI_STRING; /* Null terminated ASCII string */ typedef void * ACPI_HANDLE; /* Actually a ptr to a NS Node */ /* Time constants for timer calculations */ #define ACPI_MSEC_PER_SEC 1000L #define ACPI_USEC_PER_MSEC 1000L #define ACPI_USEC_PER_SEC 1000000L #define ACPI_100NSEC_PER_USEC 10L #define ACPI_100NSEC_PER_MSEC 10000L #define ACPI_100NSEC_PER_SEC 10000000L #define ACPI_NSEC_PER_USEC 1000L #define ACPI_NSEC_PER_MSEC 1000000L #define ACPI_NSEC_PER_SEC 1000000000L #define ACPI_TIME_AFTER(a, b) ((INT64)((b) - (a)) < 0) /* Owner IDs are used to track namespace nodes for selective deletion */ typedef UINT16 ACPI_OWNER_ID; #define ACPI_OWNER_ID_MAX 0xFFF /* 4095 possible owner IDs */ #define ACPI_INTEGER_BIT_SIZE 64 #define ACPI_MAX_DECIMAL_DIGITS 20 /* 2^64 = 18,446,744,073,709,551,616 */ #define ACPI_MAX64_DECIMAL_DIGITS 20 #define ACPI_MAX32_DECIMAL_DIGITS 10 #define ACPI_MAX16_DECIMAL_DIGITS 5 #define ACPI_MAX8_DECIMAL_DIGITS 3 /* * Constants with special meanings */ #define ACPI_ROOT_OBJECT ((ACPI_HANDLE) ACPI_TO_POINTER (ACPI_MAX_PTR)) #define ACPI_WAIT_FOREVER 0xFFFF /* UINT16, as per ACPI spec */ #define ACPI_DO_NOT_WAIT 0 /* * Obsolete: Acpi integer width. In ACPI version 1 (1996), integers are * 32 bits. In ACPI version 2 (2000) and later, integers are max 64 bits. * Note that this pertains to the ACPI integer type only, not to other * integers used in the implementation of the ACPICA subsystem. * * 01/2010: This type is obsolete and has been removed from the entire ACPICA * code base. It remains here for compatibility with device drivers that use * the type. However, it will be removed in the future. */ typedef UINT64 ACPI_INTEGER; #define ACPI_INTEGER_MAX ACPI_UINT64_MAX /******************************************************************************* * * Commonly used macros * ******************************************************************************/ /* Data manipulation */ #define ACPI_LOBYTE(Integer) ((UINT8) (UINT16)(Integer)) #define ACPI_HIBYTE(Integer) ((UINT8) (((UINT16)(Integer)) >> 8)) #define ACPI_LOWORD(Integer) ((UINT16) (UINT32)(Integer)) #define ACPI_HIWORD(Integer) ((UINT16)(((UINT32)(Integer)) >> 16)) #define ACPI_LODWORD(Integer64) ((UINT32) (UINT64)(Integer64)) #define ACPI_HIDWORD(Integer64) ((UINT32)(((UINT64)(Integer64)) >> 32)) #define ACPI_SET_BIT(target,bit) ((target) |= (bit)) #define ACPI_CLEAR_BIT(target,bit) ((target) &= ~(bit)) #define ACPI_MIN(a,b) (((a)<(b))?(a):(b)) #define ACPI_MAX(a,b) (((a)>(b))?(a):(b)) /* Size calculation */ #define ACPI_ARRAY_LENGTH(x) (sizeof(x) / sizeof((x)[0])) /* Pointer manipulation */ #define ACPI_CAST_PTR(t, p) ((t *) (ACPI_UINTPTR_T) (p)) #define ACPI_CAST_INDIRECT_PTR(t, p) ((t **) (ACPI_UINTPTR_T) (p)) #define ACPI_ADD_PTR(t, a, b) ACPI_CAST_PTR (t, (ACPI_CAST_PTR (UINT8, (a)) + (ACPI_SIZE)(b))) #define ACPI_SUB_PTR(t, a, b) ACPI_CAST_PTR (t, (ACPI_CAST_PTR (UINT8, (a)) - (ACPI_SIZE)(b))) #define ACPI_PTR_DIFF(a, b) ((ACPI_SIZE) (ACPI_CAST_PTR (UINT8, (a)) - ACPI_CAST_PTR (UINT8, (b)))) /* Pointer/Integer type conversions */ #define ACPI_TO_POINTER(i) ACPI_CAST_PTR (void, (ACPI_SIZE) (i)) #define ACPI_TO_INTEGER(p) ACPI_PTR_DIFF (p, (void *) 0) #define ACPI_OFFSET(d, f) ACPI_PTR_DIFF (&(((d *) 0)->f), (void *) 0) #define ACPI_PHYSADDR_TO_PTR(i) ACPI_TO_POINTER(i) #define ACPI_PTR_TO_PHYSADDR(i) ACPI_TO_INTEGER(i) /* Optimizations for 4-character (32-bit) ACPI_NAME manipulation */ #ifndef ACPI_MISALIGNMENT_NOT_SUPPORTED #define ACPI_COMPARE_NAMESEG(a,b) (*ACPI_CAST_PTR (UINT32, (a)) == *ACPI_CAST_PTR (UINT32, (b))) #define ACPI_COPY_NAMESEG(dest,src) (*ACPI_CAST_PTR (UINT32, (dest)) = *ACPI_CAST_PTR (UINT32, (src))) #else #define ACPI_COMPARE_NAMESEG(a,b) (!strncmp (ACPI_CAST_PTR (char, (a)), ACPI_CAST_PTR (char, (b)), ACPI_NAMESEG_SIZE)) #define ACPI_COPY_NAMESEG(dest,src) (strncpy (ACPI_CAST_PTR (char, (dest)), ACPI_CAST_PTR (char, (src)), ACPI_NAMESEG_SIZE)) #endif /* Support for the special RSDP signature (8 characters) */ #define ACPI_VALIDATE_RSDP_SIG(a) (!strncmp (ACPI_CAST_PTR (char, (a)), ACPI_SIG_RSDP, 8)) #define ACPI_MAKE_RSDP_SIG(dest) (memcpy (ACPI_CAST_PTR (char, (dest)), ACPI_SIG_RSDP, 8)) /* Support for OEMx signature (x can be any character) */ #define ACPI_IS_OEM_SIG(a) (!strncmp (ACPI_CAST_PTR (char, (a)), ACPI_OEM_NAME, 3) &&\ strnlen (a, ACPI_NAMESEG_SIZE) == ACPI_NAMESEG_SIZE) /* - * Algorithm to obtain access bit width. - * Can be used with AccessWidth of ACPI_GENERIC_ADDRESS and AccessSize of + * Algorithm to obtain access bit or byte width. + * Can be used with AccessSize field of ACPI_GENERIC_ADDRESS and * ACPI_RESOURCE_GENERIC_REGISTER. */ -#define ACPI_ACCESS_BIT_WIDTH(size) (1 << ((size) + 2)) +#define ACPI_ACCESS_BIT_WIDTH(AccessSize) (1 << ((AccessSize) + 2)) +#define ACPI_ACCESS_BYTE_WIDTH(AccessSize) (1 << ((AccessSize) - 1)) /******************************************************************************* * * Miscellaneous constants * ******************************************************************************/ /* * Initialization sequence options */ #define ACPI_FULL_INITIALIZATION 0x0000 #define ACPI_NO_FACS_INIT 0x0001 #define ACPI_NO_ACPI_ENABLE 0x0002 #define ACPI_NO_HARDWARE_INIT 0x0004 #define ACPI_NO_EVENT_INIT 0x0008 #define ACPI_NO_HANDLER_INIT 0x0010 #define ACPI_NO_OBJECT_INIT 0x0020 #define ACPI_NO_DEVICE_INIT 0x0040 #define ACPI_NO_ADDRESS_SPACE_INIT 0x0080 /* * Initialization state */ #define ACPI_SUBSYSTEM_INITIALIZE 0x01 #define ACPI_INITIALIZED_OK 0x02 /* * Power state values */ #define ACPI_STATE_UNKNOWN (UINT8) 0xFF #define ACPI_STATE_S0 (UINT8) 0 #define ACPI_STATE_S1 (UINT8) 1 #define ACPI_STATE_S2 (UINT8) 2 #define ACPI_STATE_S3 (UINT8) 3 #define ACPI_STATE_S4 (UINT8) 4 #define ACPI_STATE_S5 (UINT8) 5 #define ACPI_S_STATES_MAX ACPI_STATE_S5 #define ACPI_S_STATE_COUNT 6 #define ACPI_STATE_D0 (UINT8) 0 #define ACPI_STATE_D1 (UINT8) 1 #define ACPI_STATE_D2 (UINT8) 2 #define ACPI_STATE_D3 (UINT8) 3 #define ACPI_D_STATES_MAX ACPI_STATE_D3 #define ACPI_D_STATE_COUNT 4 #define ACPI_STATE_C0 (UINT8) 0 #define ACPI_STATE_C1 (UINT8) 1 #define ACPI_STATE_C2 (UINT8) 2 #define ACPI_STATE_C3 (UINT8) 3 #define ACPI_C_STATES_MAX ACPI_STATE_C3 #define ACPI_C_STATE_COUNT 4 /* * Sleep type invalid value */ #define ACPI_SLEEP_TYPE_MAX 0x7 #define ACPI_SLEEP_TYPE_INVALID 0xFF /* * Standard notify values */ #define ACPI_NOTIFY_BUS_CHECK (UINT8) 0x00 #define ACPI_NOTIFY_DEVICE_CHECK (UINT8) 0x01 #define ACPI_NOTIFY_DEVICE_WAKE (UINT8) 0x02 #define ACPI_NOTIFY_EJECT_REQUEST (UINT8) 0x03 #define ACPI_NOTIFY_DEVICE_CHECK_LIGHT (UINT8) 0x04 #define ACPI_NOTIFY_FREQUENCY_MISMATCH (UINT8) 0x05 #define ACPI_NOTIFY_BUS_MODE_MISMATCH (UINT8) 0x06 #define ACPI_NOTIFY_POWER_FAULT (UINT8) 0x07 #define ACPI_NOTIFY_CAPABILITIES_CHECK (UINT8) 0x08 #define ACPI_NOTIFY_DEVICE_PLD_CHECK (UINT8) 0x09 #define ACPI_NOTIFY_RESERVED (UINT8) 0x0A #define ACPI_NOTIFY_LOCALITY_UPDATE (UINT8) 0x0B #define ACPI_NOTIFY_SHUTDOWN_REQUEST (UINT8) 0x0C #define ACPI_NOTIFY_AFFINITY_UPDATE (UINT8) 0x0D #define ACPI_NOTIFY_MEMORY_UPDATE (UINT8) 0x0E #define ACPI_NOTIFY_DISCONNECT_RECOVER (UINT8) 0x0F #define ACPI_GENERIC_NOTIFY_MAX 0x0F #define ACPI_SPECIFIC_NOTIFY_MAX 0x84 /* * Types associated with ACPI names and objects. The first group of * values (up to ACPI_TYPE_EXTERNAL_MAX) correspond to the definition * of the ACPI ObjectType() operator (See the ACPI Spec). Therefore, * only add to the first group if the spec changes. * * NOTE: Types must be kept in sync with the global AcpiNsProperties * and AcpiNsTypeNames arrays. */ typedef UINT32 ACPI_OBJECT_TYPE; #define ACPI_TYPE_ANY 0x00 #define ACPI_TYPE_INTEGER 0x01 /* Byte/Word/Dword/Zero/One/Ones */ #define ACPI_TYPE_STRING 0x02 #define ACPI_TYPE_BUFFER 0x03 #define ACPI_TYPE_PACKAGE 0x04 /* ByteConst, multiple DataTerm/Constant/SuperName */ #define ACPI_TYPE_FIELD_UNIT 0x05 #define ACPI_TYPE_DEVICE 0x06 /* Name, multiple Node */ #define ACPI_TYPE_EVENT 0x07 #define ACPI_TYPE_METHOD 0x08 /* Name, ByteConst, multiple Code */ #define ACPI_TYPE_MUTEX 0x09 #define ACPI_TYPE_REGION 0x0A #define ACPI_TYPE_POWER 0x0B /* Name,ByteConst,WordConst,multi Node */ #define ACPI_TYPE_PROCESSOR 0x0C /* Name,ByteConst,DWordConst,ByteConst,multi NmO */ #define ACPI_TYPE_THERMAL 0x0D /* Name, multiple Node */ #define ACPI_TYPE_BUFFER_FIELD 0x0E #define ACPI_TYPE_DDB_HANDLE 0x0F #define ACPI_TYPE_DEBUG_OBJECT 0x10 #define ACPI_TYPE_EXTERNAL_MAX 0x10 #define ACPI_NUM_TYPES (ACPI_TYPE_EXTERNAL_MAX + 1) /* * These are object types that do not map directly to the ACPI * ObjectType() operator. They are used for various internal purposes * only. If new predefined ACPI_TYPEs are added (via the ACPI * specification), these internal types must move upwards. (There * is code that depends on these values being contiguous with the * external types above.) */ #define ACPI_TYPE_LOCAL_REGION_FIELD 0x11 #define ACPI_TYPE_LOCAL_BANK_FIELD 0x12 #define ACPI_TYPE_LOCAL_INDEX_FIELD 0x13 #define ACPI_TYPE_LOCAL_REFERENCE 0x14 /* Arg#, Local#, Name, Debug, RefOf, Index */ #define ACPI_TYPE_LOCAL_ALIAS 0x15 #define ACPI_TYPE_LOCAL_METHOD_ALIAS 0x16 #define ACPI_TYPE_LOCAL_NOTIFY 0x17 #define ACPI_TYPE_LOCAL_ADDRESS_HANDLER 0x18 #define ACPI_TYPE_LOCAL_RESOURCE 0x19 #define ACPI_TYPE_LOCAL_RESOURCE_FIELD 0x1A #define ACPI_TYPE_LOCAL_SCOPE 0x1B /* 1 Name, multiple ObjectList Nodes */ #define ACPI_TYPE_NS_NODE_MAX 0x1B /* Last typecode used within a NS Node */ #define ACPI_TOTAL_TYPES (ACPI_TYPE_NS_NODE_MAX + 1) /* * These are special object types that never appear in * a Namespace node, only in an object of ACPI_OPERAND_OBJECT */ #define ACPI_TYPE_LOCAL_EXTRA 0x1C #define ACPI_TYPE_LOCAL_DATA 0x1D #define ACPI_TYPE_LOCAL_MAX 0x1D /* All types above here are invalid */ #define ACPI_TYPE_INVALID 0x1E #define ACPI_TYPE_NOT_FOUND 0xFF #define ACPI_NUM_NS_TYPES (ACPI_TYPE_INVALID + 1) /* * All I/O */ #define ACPI_READ 0 #define ACPI_WRITE 1 #define ACPI_IO_MASK 1 /* * Event Types: Fixed & General Purpose */ typedef UINT32 ACPI_EVENT_TYPE; /* * Fixed events */ #define ACPI_EVENT_PMTIMER 0 #define ACPI_EVENT_GLOBAL 1 #define ACPI_EVENT_POWER_BUTTON 2 #define ACPI_EVENT_SLEEP_BUTTON 3 #define ACPI_EVENT_RTC 4 #define ACPI_EVENT_MAX 4 #define ACPI_NUM_FIXED_EVENTS ACPI_EVENT_MAX + 1 /* * Event Status - Per event * ------------- * The encoding of ACPI_EVENT_STATUS is illustrated below. * Note that a set bit (1) indicates the property is TRUE * (e.g. if bit 0 is set then the event is enabled). * +-------------+-+-+-+-+-+-+ * | Bits 31:6 |5|4|3|2|1|0| * +-------------+-+-+-+-+-+-+ * | | | | | | | * | | | | | | +- Enabled? * | | | | | +--- Enabled for wake? * | | | | +----- Status bit set? * | | | +------- Enable bit set? * | | +--------- Has a handler? * | +----------- Masked? * +----------------- */ typedef UINT32 ACPI_EVENT_STATUS; #define ACPI_EVENT_FLAG_DISABLED (ACPI_EVENT_STATUS) 0x00 #define ACPI_EVENT_FLAG_ENABLED (ACPI_EVENT_STATUS) 0x01 #define ACPI_EVENT_FLAG_WAKE_ENABLED (ACPI_EVENT_STATUS) 0x02 #define ACPI_EVENT_FLAG_STATUS_SET (ACPI_EVENT_STATUS) 0x04 #define ACPI_EVENT_FLAG_ENABLE_SET (ACPI_EVENT_STATUS) 0x08 #define ACPI_EVENT_FLAG_HAS_HANDLER (ACPI_EVENT_STATUS) 0x10 #define ACPI_EVENT_FLAG_MASKED (ACPI_EVENT_STATUS) 0x20 #define ACPI_EVENT_FLAG_SET ACPI_EVENT_FLAG_STATUS_SET /* Actions for AcpiSetGpe, AcpiGpeWakeup, AcpiHwLowSetGpe */ #define ACPI_GPE_ENABLE 0 #define ACPI_GPE_DISABLE 1 #define ACPI_GPE_CONDITIONAL_ENABLE 2 /* * GPE info flags - Per GPE * +---+-+-+-+---+ * |7:6|5|4|3|2:0| * +---+-+-+-+---+ * | | | | | * | | | | +-- Type of dispatch:to method, handler, notify, or none * | | | +----- Interrupt type: edge or level triggered * | | +------- Is a Wake GPE * | +--------- Has been enabled automatically at init time * +------------ */ #define ACPI_GPE_DISPATCH_NONE (UINT8) 0x00 #define ACPI_GPE_DISPATCH_METHOD (UINT8) 0x01 #define ACPI_GPE_DISPATCH_HANDLER (UINT8) 0x02 #define ACPI_GPE_DISPATCH_NOTIFY (UINT8) 0x03 #define ACPI_GPE_DISPATCH_RAW_HANDLER (UINT8) 0x04 #define ACPI_GPE_DISPATCH_MASK (UINT8) 0x07 #define ACPI_GPE_DISPATCH_TYPE(flags) ((UINT8) ((flags) & ACPI_GPE_DISPATCH_MASK)) #define ACPI_GPE_LEVEL_TRIGGERED (UINT8) 0x08 #define ACPI_GPE_EDGE_TRIGGERED (UINT8) 0x00 #define ACPI_GPE_XRUPT_TYPE_MASK (UINT8) 0x08 #define ACPI_GPE_CAN_WAKE (UINT8) 0x10 #define ACPI_GPE_AUTO_ENABLED (UINT8) 0x20 #define ACPI_GPE_INITIALIZED (UINT8) 0x40 /* * Flags for GPE and Lock interfaces */ #define ACPI_NOT_ISR 0x1 #define ACPI_ISR 0x0 /* Notify types */ #define ACPI_SYSTEM_NOTIFY 0x1 #define ACPI_DEVICE_NOTIFY 0x2 #define ACPI_ALL_NOTIFY (ACPI_SYSTEM_NOTIFY | ACPI_DEVICE_NOTIFY) #define ACPI_MAX_NOTIFY_HANDLER_TYPE 0x3 #define ACPI_NUM_NOTIFY_TYPES 2 #define ACPI_MAX_SYS_NOTIFY 0x7F #define ACPI_MAX_DEVICE_SPECIFIC_NOTIFY 0xBF #define ACPI_SYSTEM_HANDLER_LIST 0 /* Used as index, must be SYSTEM_NOTIFY -1 */ #define ACPI_DEVICE_HANDLER_LIST 1 /* Used as index, must be DEVICE_NOTIFY -1 */ /* Address Space (Operation Region) Types */ typedef UINT8 ACPI_ADR_SPACE_TYPE; #define ACPI_ADR_SPACE_SYSTEM_MEMORY (ACPI_ADR_SPACE_TYPE) 0 #define ACPI_ADR_SPACE_SYSTEM_IO (ACPI_ADR_SPACE_TYPE) 1 #define ACPI_ADR_SPACE_PCI_CONFIG (ACPI_ADR_SPACE_TYPE) 2 #define ACPI_ADR_SPACE_EC (ACPI_ADR_SPACE_TYPE) 3 #define ACPI_ADR_SPACE_SMBUS (ACPI_ADR_SPACE_TYPE) 4 #define ACPI_ADR_SPACE_CMOS (ACPI_ADR_SPACE_TYPE) 5 #define ACPI_ADR_SPACE_PCI_BAR_TARGET (ACPI_ADR_SPACE_TYPE) 6 #define ACPI_ADR_SPACE_IPMI (ACPI_ADR_SPACE_TYPE) 7 #define ACPI_ADR_SPACE_GPIO (ACPI_ADR_SPACE_TYPE) 8 #define ACPI_ADR_SPACE_GSBUS (ACPI_ADR_SPACE_TYPE) 9 #define ACPI_ADR_SPACE_PLATFORM_COMM (ACPI_ADR_SPACE_TYPE) 10 #define ACPI_NUM_PREDEFINED_REGIONS 11 /* * Special Address Spaces * * Note: A Data Table region is a special type of operation region * that has its own AML opcode. However, internally, the AML * interpreter simply creates an operation region with an an address * space type of ACPI_ADR_SPACE_DATA_TABLE. */ #define ACPI_ADR_SPACE_DATA_TABLE (ACPI_ADR_SPACE_TYPE) 0x7E /* Internal to ACPICA only */ #define ACPI_ADR_SPACE_FIXED_HARDWARE (ACPI_ADR_SPACE_TYPE) 0x7F /* Values for _REG connection code */ #define ACPI_REG_DISCONNECT 0 #define ACPI_REG_CONNECT 1 /* * BitRegister IDs * * These values are intended to be used by the hardware interfaces * and are mapped to individual bitfields defined within the ACPI * registers. See the AcpiGbl_BitRegisterInfo global table in utglobal.c * for this mapping. */ /* PM1 Status register */ #define ACPI_BITREG_TIMER_STATUS 0x00 #define ACPI_BITREG_BUS_MASTER_STATUS 0x01 #define ACPI_BITREG_GLOBAL_LOCK_STATUS 0x02 #define ACPI_BITREG_POWER_BUTTON_STATUS 0x03 #define ACPI_BITREG_SLEEP_BUTTON_STATUS 0x04 #define ACPI_BITREG_RT_CLOCK_STATUS 0x05 #define ACPI_BITREG_WAKE_STATUS 0x06 #define ACPI_BITREG_PCIEXP_WAKE_STATUS 0x07 /* PM1 Enable register */ #define ACPI_BITREG_TIMER_ENABLE 0x08 #define ACPI_BITREG_GLOBAL_LOCK_ENABLE 0x09 #define ACPI_BITREG_POWER_BUTTON_ENABLE 0x0A #define ACPI_BITREG_SLEEP_BUTTON_ENABLE 0x0B #define ACPI_BITREG_RT_CLOCK_ENABLE 0x0C #define ACPI_BITREG_PCIEXP_WAKE_DISABLE 0x0D /* PM1 Control register */ #define ACPI_BITREG_SCI_ENABLE 0x0E #define ACPI_BITREG_BUS_MASTER_RLD 0x0F #define ACPI_BITREG_GLOBAL_LOCK_RELEASE 0x10 #define ACPI_BITREG_SLEEP_TYPE 0x11 #define ACPI_BITREG_SLEEP_ENABLE 0x12 /* PM2 Control register */ #define ACPI_BITREG_ARB_DISABLE 0x13 #define ACPI_BITREG_MAX 0x13 #define ACPI_NUM_BITREG ACPI_BITREG_MAX + 1 /* Status register values. A 1 clears a status bit. 0 = no effect */ #define ACPI_CLEAR_STATUS 1 /* Enable and Control register values */ #define ACPI_ENABLE_EVENT 1 #define ACPI_DISABLE_EVENT 0 /* Sleep function dispatch */ typedef ACPI_STATUS (*ACPI_SLEEP_FUNCTION) ( UINT8 SleepState); typedef struct acpi_sleep_functions { ACPI_SLEEP_FUNCTION LegacyFunction; ACPI_SLEEP_FUNCTION ExtendedFunction; } ACPI_SLEEP_FUNCTIONS; /* * External ACPI object definition */ /* * Note: Type == ACPI_TYPE_ANY (0) is used to indicate a NULL package * element or an unresolved named reference. */ typedef union acpi_object { ACPI_OBJECT_TYPE Type; /* See definition of AcpiNsType for values */ struct { ACPI_OBJECT_TYPE Type; /* ACPI_TYPE_INTEGER */ UINT64 Value; /* The actual number */ } Integer; struct { ACPI_OBJECT_TYPE Type; /* ACPI_TYPE_STRING */ UINT32 Length; /* # of bytes in string, excluding trailing null */ char *Pointer; /* points to the string value */ } String; struct { ACPI_OBJECT_TYPE Type; /* ACPI_TYPE_BUFFER */ UINT32 Length; /* # of bytes in buffer */ UINT8 *Pointer; /* points to the buffer */ } Buffer; struct { ACPI_OBJECT_TYPE Type; /* ACPI_TYPE_PACKAGE */ UINT32 Count; /* # of elements in package */ union acpi_object *Elements; /* Pointer to an array of ACPI_OBJECTs */ } Package; struct { ACPI_OBJECT_TYPE Type; /* ACPI_TYPE_LOCAL_REFERENCE */ ACPI_OBJECT_TYPE ActualType; /* Type associated with the Handle */ ACPI_HANDLE Handle; /* object reference */ } Reference; struct { ACPI_OBJECT_TYPE Type; /* ACPI_TYPE_PROCESSOR */ UINT32 ProcId; ACPI_IO_ADDRESS PblkAddress; UINT32 PblkLength; } Processor; struct { ACPI_OBJECT_TYPE Type; /* ACPI_TYPE_POWER */ UINT32 SystemLevel; UINT32 ResourceOrder; } PowerResource; } ACPI_OBJECT; /* * List of objects, used as a parameter list for control method evaluation */ typedef struct acpi_object_list { UINT32 Count; ACPI_OBJECT *Pointer; } ACPI_OBJECT_LIST; /* * Miscellaneous common Data Structures used by the interfaces */ #define ACPI_NO_BUFFER 0 #ifdef ACPI_NO_MEM_ALLOCATIONS #define ACPI_ALLOCATE_BUFFER (ACPI_SIZE) (0) #define ACPI_ALLOCATE_LOCAL_BUFFER (ACPI_SIZE) (0) #else /* ACPI_NO_MEM_ALLOCATIONS */ #define ACPI_ALLOCATE_BUFFER (ACPI_SIZE) (-1) /* Let ACPICA allocate buffer */ #define ACPI_ALLOCATE_LOCAL_BUFFER (ACPI_SIZE) (-2) /* For internal use only (enables tracking) */ #endif /* ACPI_NO_MEM_ALLOCATIONS */ typedef struct acpi_buffer { ACPI_SIZE Length; /* Length in bytes of the buffer */ void *Pointer; /* pointer to buffer */ } ACPI_BUFFER; /* * NameType for AcpiGetName */ #define ACPI_FULL_PATHNAME 0 #define ACPI_SINGLE_NAME 1 #define ACPI_FULL_PATHNAME_NO_TRAILING 2 #define ACPI_NAME_TYPE_MAX 2 /* * Predefined Namespace items */ typedef struct acpi_predefined_names { const char *Name; UINT8 Type; char *Val; } ACPI_PREDEFINED_NAMES; /* * Structure and flags for AcpiGetSystemInfo */ #define ACPI_SYS_MODE_UNKNOWN 0x0000 #define ACPI_SYS_MODE_ACPI 0x0001 #define ACPI_SYS_MODE_LEGACY 0x0002 #define ACPI_SYS_MODES_MASK 0x0003 /* * System info returned by AcpiGetSystemInfo() */ typedef struct acpi_system_info { UINT32 AcpiCaVersion; UINT32 Flags; UINT32 TimerResolution; UINT32 Reserved1; UINT32 Reserved2; UINT32 DebugLevel; UINT32 DebugLayer; } ACPI_SYSTEM_INFO; /* * System statistics returned by AcpiGetStatistics() */ typedef struct acpi_statistics { UINT32 SciCount; UINT32 GpeCount; UINT32 FixedEventCount[ACPI_NUM_FIXED_EVENTS]; UINT32 MethodCount; } ACPI_STATISTICS; /* * Types specific to the OS service interfaces */ typedef UINT32 (ACPI_SYSTEM_XFACE *ACPI_OSD_HANDLER) ( void *Context); typedef void (ACPI_SYSTEM_XFACE *ACPI_OSD_EXEC_CALLBACK) ( void *Context); /* * Various handlers and callback procedures */ typedef UINT32 (*ACPI_SCI_HANDLER) ( void *Context); typedef void (*ACPI_GBL_EVENT_HANDLER) ( UINT32 EventType, ACPI_HANDLE Device, UINT32 EventNumber, void *Context); #define ACPI_EVENT_TYPE_GPE 0 #define ACPI_EVENT_TYPE_FIXED 1 typedef UINT32 (*ACPI_EVENT_HANDLER) ( void *Context); typedef UINT32 (*ACPI_GPE_HANDLER) ( ACPI_HANDLE GpeDevice, UINT32 GpeNumber, void *Context); typedef void (*ACPI_NOTIFY_HANDLER) ( ACPI_HANDLE Device, UINT32 Value, void *Context); typedef void (*ACPI_OBJECT_HANDLER) ( ACPI_HANDLE Object, void *Data); typedef ACPI_STATUS (*ACPI_INIT_HANDLER) ( ACPI_HANDLE Object, UINT32 Function); #define ACPI_INIT_DEVICE_INI 1 typedef ACPI_STATUS (*ACPI_EXCEPTION_HANDLER) ( ACPI_STATUS AmlStatus, ACPI_NAME Name, UINT16 Opcode, UINT32 AmlOffset, void *Context); /* Table Event handler (Load, LoadTable, etc.) and types */ typedef ACPI_STATUS (*ACPI_TABLE_HANDLER) ( UINT32 Event, void *Table, void *Context); /* Table Event Types */ #define ACPI_TABLE_EVENT_LOAD 0x0 #define ACPI_TABLE_EVENT_UNLOAD 0x1 #define ACPI_TABLE_EVENT_INSTALL 0x2 #define ACPI_TABLE_EVENT_UNINSTALL 0x3 #define ACPI_NUM_TABLE_EVENTS 4 /* Address Spaces (For Operation Regions) */ typedef ACPI_STATUS (*ACPI_ADR_SPACE_HANDLER) ( UINT32 Function, ACPI_PHYSICAL_ADDRESS Address, UINT32 BitWidth, UINT64 *Value, void *HandlerContext, void *RegionContext); #define ACPI_DEFAULT_HANDLER NULL /* Special Context data for GenericSerialBus/GeneralPurposeIo (ACPI 5.0) */ typedef struct acpi_connection_info { UINT8 *Connection; UINT16 Length; UINT8 AccessLength; } ACPI_CONNECTION_INFO; typedef ACPI_STATUS (*ACPI_ADR_SPACE_SETUP) ( ACPI_HANDLE RegionHandle, UINT32 Function, void *HandlerContext, void **RegionContext); #define ACPI_REGION_ACTIVATE 0 #define ACPI_REGION_DEACTIVATE 1 typedef ACPI_STATUS (*ACPI_WALK_CALLBACK) ( ACPI_HANDLE Object, UINT32 NestingLevel, void *Context, void **ReturnValue); typedef UINT32 (*ACPI_INTERFACE_HANDLER) ( ACPI_STRING InterfaceName, UINT32 Supported); /* Interrupt handler return values */ #define ACPI_INTERRUPT_NOT_HANDLED 0x00 #define ACPI_INTERRUPT_HANDLED 0x01 /* GPE handler return values */ #define ACPI_REENABLE_GPE 0x80 /* Length of 32-bit EISAID values when converted back to a string */ #define ACPI_EISAID_STRING_SIZE 8 /* Includes null terminator */ /* Length of UUID (string) values */ #define ACPI_UUID_LENGTH 16 /* Length of 3-byte PCI class code values when converted back to a string */ #define ACPI_PCICLS_STRING_SIZE 7 /* Includes null terminator */ /* Structures used for device/processor HID, UID, CID */ typedef struct acpi_pnp_device_id { UINT32 Length; /* Length of string + null */ char *String; } ACPI_PNP_DEVICE_ID; typedef struct acpi_pnp_device_id_list { UINT32 Count; /* Number of IDs in Ids array */ UINT32 ListSize; /* Size of list, including ID strings */ ACPI_PNP_DEVICE_ID Ids[1]; /* ID array */ } ACPI_PNP_DEVICE_ID_LIST; /* * Structure returned from AcpiGetObjectInfo. * Optimized for both 32-bit and 64-bit builds. */ typedef struct acpi_device_info { UINT32 InfoSize; /* Size of info, including ID strings */ UINT32 Name; /* ACPI object Name */ ACPI_OBJECT_TYPE Type; /* ACPI object Type */ UINT8 ParamCount; /* If a method, required parameter count */ UINT16 Valid; /* Indicates which optional fields are valid */ UINT8 Flags; /* Miscellaneous info */ UINT8 HighestDstates[4]; /* _SxD values: 0xFF indicates not valid */ UINT8 LowestDstates[5]; /* _SxW values: 0xFF indicates not valid */ UINT64 Address; /* _ADR value */ ACPI_PNP_DEVICE_ID HardwareId; /* _HID value */ ACPI_PNP_DEVICE_ID UniqueId; /* _UID value */ ACPI_PNP_DEVICE_ID ClassCode; /* _CLS value */ ACPI_PNP_DEVICE_ID_LIST CompatibleIdList; /* _CID list */ } ACPI_DEVICE_INFO; /* Values for Flags field above (AcpiGetObjectInfo) */ #define ACPI_PCI_ROOT_BRIDGE 0x01 /* Flags for Valid field above (AcpiGetObjectInfo) */ #define ACPI_VALID_ADR 0x0002 #define ACPI_VALID_HID 0x0004 #define ACPI_VALID_UID 0x0008 #define ACPI_VALID_CID 0x0020 #define ACPI_VALID_CLS 0x0040 #define ACPI_VALID_SXDS 0x0100 #define ACPI_VALID_SXWS 0x0200 /* Flags for _STA method */ #define ACPI_STA_DEVICE_PRESENT 0x01 #define ACPI_STA_DEVICE_ENABLED 0x02 #define ACPI_STA_DEVICE_UI 0x04 #define ACPI_STA_DEVICE_FUNCTIONING 0x08 #define ACPI_STA_DEVICE_OK 0x08 /* Synonym */ #define ACPI_STA_BATTERY_PRESENT 0x10 /* Context structs for address space handlers */ typedef struct acpi_pci_id { UINT16 Segment; UINT16 Bus; UINT16 Device; UINT16 Function; } ACPI_PCI_ID; typedef struct acpi_mem_space_context { UINT32 Length; ACPI_PHYSICAL_ADDRESS Address; ACPI_PHYSICAL_ADDRESS MappedPhysicalAddress; UINT8 *MappedLogicalAddress; ACPI_SIZE MappedLength; } ACPI_MEM_SPACE_CONTEXT; /* * ACPI_MEMORY_LIST is used only if the ACPICA local cache is enabled */ typedef struct acpi_memory_list { const char *ListName; void *ListHead; UINT16 ObjectSize; UINT16 MaxDepth; UINT16 CurrentDepth; #ifdef ACPI_DBG_TRACK_ALLOCATIONS /* Statistics for debug memory tracking only */ UINT32 TotalAllocated; UINT32 TotalFreed; UINT32 MaxOccupied; UINT32 TotalSize; UINT32 CurrentTotalSize; UINT32 Requests; UINT32 Hits; #endif } ACPI_MEMORY_LIST; /* Definitions of trace event types */ typedef enum { ACPI_TRACE_AML_METHOD, ACPI_TRACE_AML_OPCODE, ACPI_TRACE_AML_REGION } ACPI_TRACE_EVENT_TYPE; /* Definitions of _OSI support */ #define ACPI_VENDOR_STRINGS 0x01 #define ACPI_FEATURE_STRINGS 0x02 #define ACPI_ENABLE_INTERFACES 0x00 #define ACPI_DISABLE_INTERFACES 0x04 #define ACPI_DISABLE_ALL_VENDOR_STRINGS (ACPI_DISABLE_INTERFACES | ACPI_VENDOR_STRINGS) #define ACPI_DISABLE_ALL_FEATURE_STRINGS (ACPI_DISABLE_INTERFACES | ACPI_FEATURE_STRINGS) #define ACPI_DISABLE_ALL_STRINGS (ACPI_DISABLE_INTERFACES | ACPI_VENDOR_STRINGS | ACPI_FEATURE_STRINGS) #define ACPI_ENABLE_ALL_VENDOR_STRINGS (ACPI_ENABLE_INTERFACES | ACPI_VENDOR_STRINGS) #define ACPI_ENABLE_ALL_FEATURE_STRINGS (ACPI_ENABLE_INTERFACES | ACPI_FEATURE_STRINGS) #define ACPI_ENABLE_ALL_STRINGS (ACPI_ENABLE_INTERFACES | ACPI_VENDOR_STRINGS | ACPI_FEATURE_STRINGS) #define ACPI_OSI_WIN_2000 0x01 #define ACPI_OSI_WIN_XP 0x02 #define ACPI_OSI_WIN_XP_SP1 0x03 #define ACPI_OSI_WINSRV_2003 0x04 #define ACPI_OSI_WIN_XP_SP2 0x05 #define ACPI_OSI_WINSRV_2003_SP1 0x06 #define ACPI_OSI_WIN_VISTA 0x07 #define ACPI_OSI_WINSRV_2008 0x08 #define ACPI_OSI_WIN_VISTA_SP1 0x09 #define ACPI_OSI_WIN_VISTA_SP2 0x0A #define ACPI_OSI_WIN_7 0x0B #define ACPI_OSI_WIN_8 0x0C #define ACPI_OSI_WIN_8_1 0x0D #define ACPI_OSI_WIN_10 0x0E #define ACPI_OSI_WIN_10_RS1 0x0F #define ACPI_OSI_WIN_10_RS2 0x10 #define ACPI_OSI_WIN_10_RS3 0x11 #define ACPI_OSI_WIN_10_RS4 0x12 #define ACPI_OSI_WIN_10_RS5 0x13 #define ACPI_OSI_WIN_10_19H1 0x14 /* Definitions of getopt */ #define ACPI_OPT_END -1 #endif /* __ACTYPES_H__ */ Index: projects/clang1000-import/sys/contrib/dev/acpica =================================================================== --- projects/clang1000-import/sys/contrib/dev/acpica (revision 357965) +++ projects/clang1000-import/sys/contrib/dev/acpica (revision 357966) Property changes on: projects/clang1000-import/sys/contrib/dev/acpica ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,2 ## Merged /head/sys/contrib/dev/acpica:r356848-357965 Merged /vendor-sys/acpica/dist/source:r357927 Index: projects/clang1000-import/sys/fs/tmpfs/tmpfs_vfsops.c =================================================================== --- projects/clang1000-import/sys/fs/tmpfs/tmpfs_vfsops.c (revision 357965) +++ projects/clang1000-import/sys/fs/tmpfs/tmpfs_vfsops.c (revision 357966) @@ -1,686 +1,676 @@ /* $NetBSD: tmpfs_vfsops.c,v 1.10 2005/12/11 12:24:29 christos Exp $ */ /*- * SPDX-License-Identifier: BSD-2-Clause-NetBSD * * Copyright (c) 2005 The NetBSD Foundation, Inc. * All rights reserved. * * This code is derived from software contributed to The NetBSD Foundation * by Julio M. Merino Vidal, developed as part of Google's Summer of Code * 2005 program. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ /* * Efficient memory file system. * * tmpfs is a file system that uses FreeBSD's virtual memory * sub-system to store file data and metadata in an efficient way. * This means that it does not follow the structure of an on-disk file * system because it simply does not need to. Instead, it uses * memory-specific data structures and algorithms to automatically * allocate and release resources. */ #include "opt_tmpfs.h" #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* * Default permission for root node */ #define TMPFS_DEFAULT_ROOT_MODE (S_IRWXU|S_IRGRP|S_IXGRP|S_IROTH|S_IXOTH) MALLOC_DEFINE(M_TMPFSMNT, "tmpfs mount", "tmpfs mount structures"); MALLOC_DEFINE(M_TMPFSNAME, "tmpfs name", "tmpfs file names"); static int tmpfs_mount(struct mount *); static int tmpfs_unmount(struct mount *, int); static int tmpfs_root(struct mount *, int flags, struct vnode **); static int tmpfs_fhtovp(struct mount *, struct fid *, int, struct vnode **); static int tmpfs_statfs(struct mount *, struct statfs *); -static void tmpfs_susp_clean(struct mount *); static const char *tmpfs_opts[] = { "from", "size", "maxfilesize", "inodes", "uid", "gid", "mode", "export", "union", "nonc", "nomtime", NULL }; static const char *tmpfs_updateopts[] = { "from", "export", "nomtime", "size", NULL }; /* * Handle updates of time from writes to mmaped regions, if allowed. * Use MNT_VNODE_FOREACH_ALL instead of MNT_VNODE_FOREACH_LAZY, since * unmap of the tmpfs-backed vnode does not call vinactive(), due to * vm object type is OBJT_SWAP. If lazy, only handle delayed update * of mtime due to the writes to mapped files. */ static void tmpfs_update_mtime(struct mount *mp, bool lazy) { struct vnode *vp, *mvp; struct vm_object *obj; if (VFS_TO_TMPFS(mp)->tm_nomtime) return; MNT_VNODE_FOREACH_ALL(vp, mp, mvp) { if (vp->v_type != VREG) { VI_UNLOCK(vp); continue; } obj = vp->v_object; KASSERT((obj->flags & (OBJ_TMPFS_NODE | OBJ_TMPFS)) == (OBJ_TMPFS_NODE | OBJ_TMPFS), ("non-tmpfs obj")); /* * In lazy case, do unlocked read, avoid taking vnode * lock if not needed. Lost update will be handled on * the next call. * For non-lazy case, we must flush all pending * metadata changes now. */ if (!lazy || obj->generation != obj->cleangeneration) { if (vget(vp, LK_EXCLUSIVE | LK_INTERLOCK, curthread) != 0) continue; tmpfs_check_mtime(vp); if (!lazy) tmpfs_update(vp); vput(vp); } else { VI_UNLOCK(vp); continue; } } } struct tmpfs_check_rw_maps_arg { bool found; }; static bool tmpfs_check_rw_maps_cb(struct mount *mp __unused, vm_map_t map __unused, vm_map_entry_t entry __unused, void *arg) { struct tmpfs_check_rw_maps_arg *a; a = arg; a->found = true; return (true); } /* * Revoke write permissions from all mappings of regular files * belonging to the specified tmpfs mount. */ static bool tmpfs_revoke_rw_maps_cb(struct mount *mp __unused, vm_map_t map, vm_map_entry_t entry, void *arg __unused) { /* * XXXKIB: might be invalidate the mapping * instead ? The process is not going to be * happy in any case. */ entry->max_protection &= ~VM_PROT_WRITE; if ((entry->protection & VM_PROT_WRITE) != 0) { entry->protection &= ~VM_PROT_WRITE; pmap_protect(map->pmap, entry->start, entry->end, entry->protection); } return (false); } static void tmpfs_all_rw_maps(struct mount *mp, bool (*cb)(struct mount *mp, vm_map_t, vm_map_entry_t, void *), void *cb_arg) { struct proc *p; struct vmspace *vm; vm_map_t map; vm_map_entry_t entry; vm_object_t object; struct vnode *vp; int gen; bool terminate; terminate = false; sx_slock(&allproc_lock); again: gen = allproc_gen; FOREACH_PROC_IN_SYSTEM(p) { PROC_LOCK(p); if (p->p_state != PRS_NORMAL || (p->p_flag & (P_INEXEC | P_SYSTEM | P_WEXIT)) != 0) { PROC_UNLOCK(p); continue; } vm = vmspace_acquire_ref(p); _PHOLD_LITE(p); PROC_UNLOCK(p); if (vm == NULL) { PRELE(p); continue; } sx_sunlock(&allproc_lock); map = &vm->vm_map; vm_map_lock(map); if (map->busy) vm_map_wait_busy(map); VM_MAP_ENTRY_FOREACH(entry, map) { if ((entry->eflags & (MAP_ENTRY_GUARD | MAP_ENTRY_IS_SUB_MAP | MAP_ENTRY_COW)) != 0 || (entry->max_protection & VM_PROT_WRITE) == 0) continue; object = entry->object.vm_object; if (object == NULL || object->type != OBJT_SWAP || (object->flags & OBJ_TMPFS_NODE) == 0) continue; /* * No need to dig into shadow chain, mapping * of the object not at top is readonly. */ VM_OBJECT_RLOCK(object); if (object->type == OBJT_DEAD) { VM_OBJECT_RUNLOCK(object); continue; } MPASS(object->ref_count > 1); if ((object->flags & (OBJ_TMPFS_NODE | OBJ_TMPFS)) != (OBJ_TMPFS_NODE | OBJ_TMPFS)) { VM_OBJECT_RUNLOCK(object); continue; } vp = object->un_pager.swp.swp_tmpfs; if (vp->v_mount != mp) { VM_OBJECT_RUNLOCK(object); continue; } terminate = cb(mp, map, entry, cb_arg); VM_OBJECT_RUNLOCK(object); if (terminate) break; } vm_map_unlock(map); vmspace_free(vm); sx_slock(&allproc_lock); PRELE(p); if (terminate) break; } if (!terminate && gen != allproc_gen) goto again; sx_sunlock(&allproc_lock); } static bool tmpfs_check_rw_maps(struct mount *mp) { struct tmpfs_check_rw_maps_arg ca; ca.found = false; tmpfs_all_rw_maps(mp, tmpfs_check_rw_maps_cb, &ca); return (ca.found); } static int tmpfs_rw_to_ro(struct mount *mp) { int error, flags; bool forced; forced = (mp->mnt_flag & MNT_FORCE) != 0; flags = WRITECLOSE | (forced ? FORCECLOSE : 0); if ((error = vn_start_write(NULL, &mp, V_WAIT)) != 0) return (error); error = vfs_write_suspend_umnt(mp); if (error != 0) return (error); if (!forced && tmpfs_check_rw_maps(mp)) { error = EBUSY; goto out; } VFS_TO_TMPFS(mp)->tm_ronly = 1; MNT_ILOCK(mp); mp->mnt_flag |= MNT_RDONLY; MNT_IUNLOCK(mp); for (;;) { tmpfs_all_rw_maps(mp, tmpfs_revoke_rw_maps_cb, NULL); tmpfs_update_mtime(mp, false); error = vflush(mp, 0, flags, curthread); if (error != 0) { VFS_TO_TMPFS(mp)->tm_ronly = 0; MNT_ILOCK(mp); mp->mnt_flag &= ~MNT_RDONLY; MNT_IUNLOCK(mp); goto out; } if (!tmpfs_check_rw_maps(mp)) break; } out: vfs_write_resume(mp, 0); return (error); } static int tmpfs_mount(struct mount *mp) { const size_t nodes_per_page = howmany(PAGE_SIZE, sizeof(struct tmpfs_dirent) + sizeof(struct tmpfs_node)); struct tmpfs_mount *tmp; struct tmpfs_node *root; int error; bool nomtime, nonc; /* Size counters. */ u_quad_t pages; off_t nodes_max, size_max, maxfilesize; /* Root node attributes. */ uid_t root_uid; gid_t root_gid; mode_t root_mode; struct vattr va; if (vfs_filteropt(mp->mnt_optnew, tmpfs_opts)) return (EINVAL); if (mp->mnt_flag & MNT_UPDATE) { /* Only support update mounts for certain options. */ if (vfs_filteropt(mp->mnt_optnew, tmpfs_updateopts) != 0) return (EOPNOTSUPP); tmp = VFS_TO_TMPFS(mp); if (vfs_getopt_size(mp->mnt_optnew, "size", &size_max) == 0) { /* * On-the-fly resizing is not supported (yet). We still * need to have "size" listed as "supported", otherwise * trying to update fs that is listed in fstab with size * parameter, say trying to change rw to ro or vice * versa, would cause vfs_filteropt() to bail. */ if (size_max != tmp->tm_size_max) return (EOPNOTSUPP); } if (vfs_flagopt(mp->mnt_optnew, "ro", NULL, 0) && !tmp->tm_ronly) { /* RW -> RO */ return (tmpfs_rw_to_ro(mp)); } else if (!vfs_flagopt(mp->mnt_optnew, "ro", NULL, 0) && tmp->tm_ronly) { /* RO -> RW */ tmp->tm_ronly = 0; MNT_ILOCK(mp); mp->mnt_flag &= ~MNT_RDONLY; MNT_IUNLOCK(mp); } tmp->tm_nomtime = vfs_getopt(mp->mnt_optnew, "nomtime", NULL, 0) == 0; return (0); } vn_lock(mp->mnt_vnodecovered, LK_SHARED | LK_RETRY); error = VOP_GETATTR(mp->mnt_vnodecovered, &va, mp->mnt_cred); VOP_UNLOCK(mp->mnt_vnodecovered); if (error) return (error); if (mp->mnt_cred->cr_ruid != 0 || vfs_scanopt(mp->mnt_optnew, "gid", "%d", &root_gid) != 1) root_gid = va.va_gid; if (mp->mnt_cred->cr_ruid != 0 || vfs_scanopt(mp->mnt_optnew, "uid", "%d", &root_uid) != 1) root_uid = va.va_uid; if (mp->mnt_cred->cr_ruid != 0 || vfs_scanopt(mp->mnt_optnew, "mode", "%ho", &root_mode) != 1) root_mode = va.va_mode; if (vfs_getopt_size(mp->mnt_optnew, "inodes", &nodes_max) != 0) nodes_max = 0; if (vfs_getopt_size(mp->mnt_optnew, "size", &size_max) != 0) size_max = 0; if (vfs_getopt_size(mp->mnt_optnew, "maxfilesize", &maxfilesize) != 0) maxfilesize = 0; nonc = vfs_getopt(mp->mnt_optnew, "nonc", NULL, NULL) == 0; nomtime = vfs_getopt(mp->mnt_optnew, "nomtime", NULL, NULL) == 0; /* Do not allow mounts if we do not have enough memory to preserve * the minimum reserved pages. */ if (tmpfs_mem_avail() < TMPFS_PAGES_MINRESERVED) return (ENOSPC); /* Get the maximum number of memory pages this file system is * allowed to use, based on the maximum size the user passed in * the mount structure. A value of zero is treated as if the * maximum available space was requested. */ if (size_max == 0 || size_max > OFF_MAX - PAGE_SIZE || (SIZE_MAX < OFF_MAX && size_max / PAGE_SIZE >= SIZE_MAX)) pages = SIZE_MAX; else { size_max = roundup(size_max, PAGE_SIZE); pages = howmany(size_max, PAGE_SIZE); } MPASS(pages > 0); if (nodes_max <= 3) { if (pages < INT_MAX / nodes_per_page) nodes_max = pages * nodes_per_page; else nodes_max = INT_MAX; } if (nodes_max > INT_MAX) nodes_max = INT_MAX; MPASS(nodes_max >= 3); /* Allocate the tmpfs mount structure and fill it. */ tmp = (struct tmpfs_mount *)malloc(sizeof(struct tmpfs_mount), M_TMPFSMNT, M_WAITOK | M_ZERO); mtx_init(&tmp->tm_allnode_lock, "tmpfs allnode lock", NULL, MTX_DEF); tmp->tm_nodes_max = nodes_max; tmp->tm_nodes_inuse = 0; tmp->tm_refcount = 1; tmp->tm_maxfilesize = maxfilesize > 0 ? maxfilesize : OFF_MAX; LIST_INIT(&tmp->tm_nodes_used); tmp->tm_size_max = size_max; tmp->tm_pages_max = pages; tmp->tm_pages_used = 0; new_unrhdr64(&tmp->tm_ino_unr, 2); tmp->tm_ronly = (mp->mnt_flag & MNT_RDONLY) != 0; tmp->tm_nonc = nonc; tmp->tm_nomtime = nomtime; /* Allocate the root node. */ error = tmpfs_alloc_node(mp, tmp, VDIR, root_uid, root_gid, root_mode & ALLPERMS, NULL, NULL, VNOVAL, &root); if (error != 0 || root == NULL) { free(tmp, M_TMPFSMNT); return (error); } KASSERT(root->tn_id == 2, ("tmpfs root with invalid ino: %ju", (uintmax_t)root->tn_id)); tmp->tm_root = root; MNT_ILOCK(mp); mp->mnt_flag |= MNT_LOCAL; mp->mnt_kern_flag |= MNTK_LOOKUP_SHARED | MNTK_EXTENDED_SHARED | MNTK_TEXT_REFS | MNTK_NOMSYNC; MNT_IUNLOCK(mp); mp->mnt_data = tmp; mp->mnt_stat.f_namemax = MAXNAMLEN; vfs_getnewfsid(mp); vfs_mountedfrom(mp, "tmpfs"); return 0; } /* ARGSUSED2 */ static int tmpfs_unmount(struct mount *mp, int mntflags) { struct tmpfs_mount *tmp; struct tmpfs_node *node; int error, flags; flags = (mntflags & MNT_FORCE) != 0 ? FORCECLOSE : 0; tmp = VFS_TO_TMPFS(mp); /* Stop writers */ error = vfs_write_suspend_umnt(mp); if (error != 0) return (error); /* * At this point, nodes cannot be destroyed by any other * thread because write suspension is started. */ for (;;) { error = vflush(mp, 0, flags, curthread); if (error != 0) { vfs_write_resume(mp, VR_START_WRITE); return (error); } MNT_ILOCK(mp); if (mp->mnt_nvnodelistsize == 0) { MNT_IUNLOCK(mp); break; } MNT_IUNLOCK(mp); if ((mntflags & MNT_FORCE) == 0) { vfs_write_resume(mp, VR_START_WRITE); return (EBUSY); } } TMPFS_LOCK(tmp); while ((node = LIST_FIRST(&tmp->tm_nodes_used)) != NULL) { TMPFS_NODE_LOCK(node); if (node->tn_type == VDIR) tmpfs_dir_destroy(tmp, node); if (tmpfs_free_node_locked(tmp, node, true)) TMPFS_LOCK(tmp); else TMPFS_NODE_UNLOCK(node); } mp->mnt_data = NULL; tmpfs_free_tmp(tmp); vfs_write_resume(mp, VR_START_WRITE); MNT_ILOCK(mp); mp->mnt_flag &= ~MNT_LOCAL; MNT_IUNLOCK(mp); return (0); } void tmpfs_free_tmp(struct tmpfs_mount *tmp) { MPASS(tmp->tm_refcount > 0); tmp->tm_refcount--; if (tmp->tm_refcount > 0) { TMPFS_UNLOCK(tmp); return; } TMPFS_UNLOCK(tmp); mtx_destroy(&tmp->tm_allnode_lock); MPASS(tmp->tm_pages_used == 0); MPASS(tmp->tm_nodes_inuse == 0); free(tmp, M_TMPFSMNT); } static int tmpfs_root(struct mount *mp, int flags, struct vnode **vpp) { int error; error = tmpfs_alloc_vp(mp, VFS_TO_TMPFS(mp)->tm_root, flags, vpp); if (error == 0) (*vpp)->v_vflag |= VV_ROOT; return (error); } static int tmpfs_fhtovp(struct mount *mp, struct fid *fhp, int flags, struct vnode **vpp) { struct tmpfs_fid *tfhp; struct tmpfs_mount *tmp; struct tmpfs_node *node; int error; tmp = VFS_TO_TMPFS(mp); tfhp = (struct tmpfs_fid *)fhp; if (tfhp->tf_len != sizeof(struct tmpfs_fid)) return (EINVAL); if (tfhp->tf_id >= tmp->tm_nodes_max) return (EINVAL); TMPFS_LOCK(tmp); LIST_FOREACH(node, &tmp->tm_nodes_used, tn_entries) { if (node->tn_id == tfhp->tf_id && node->tn_gen == tfhp->tf_gen) { tmpfs_ref_node(node); break; } } TMPFS_UNLOCK(tmp); if (node != NULL) { error = tmpfs_alloc_vp(mp, node, LK_EXCLUSIVE, vpp); tmpfs_free_node(tmp, node); } else error = EINVAL; return (error); } /* ARGSUSED2 */ static int tmpfs_statfs(struct mount *mp, struct statfs *sbp) { struct tmpfs_mount *tmp; size_t used; tmp = VFS_TO_TMPFS(mp); sbp->f_iosize = PAGE_SIZE; sbp->f_bsize = PAGE_SIZE; used = tmpfs_pages_used(tmp); if (tmp->tm_pages_max != ULONG_MAX) sbp->f_blocks = tmp->tm_pages_max; else sbp->f_blocks = used + tmpfs_mem_avail(); if (sbp->f_blocks <= used) sbp->f_bavail = 0; else sbp->f_bavail = sbp->f_blocks - used; sbp->f_bfree = sbp->f_bavail; used = tmp->tm_nodes_inuse; sbp->f_files = tmp->tm_nodes_max; if (sbp->f_files <= used) sbp->f_ffree = 0; else sbp->f_ffree = sbp->f_files - used; /* sbp->f_owner = tmp->tn_uid; */ return 0; } static int tmpfs_sync(struct mount *mp, int waitfor) { if (waitfor == MNT_SUSPEND) { MNT_ILOCK(mp); mp->mnt_kern_flag |= MNTK_SUSPEND2 | MNTK_SUSPENDED; MNT_IUNLOCK(mp); } else if (waitfor == MNT_LAZY) { tmpfs_update_mtime(mp, true); } return (0); } -/* - * The presence of a susp_clean method tells the VFS to track writes. - */ -static void -tmpfs_susp_clean(struct mount *mp __unused) -{ -} - static int tmpfs_init(struct vfsconf *conf) { tmpfs_subr_init(); return (0); } static int tmpfs_uninit(struct vfsconf *conf) { tmpfs_subr_uninit(); return (0); } /* * tmpfs vfs operations. */ struct vfsops tmpfs_vfsops = { .vfs_mount = tmpfs_mount, .vfs_unmount = tmpfs_unmount, .vfs_root = vfs_cache_root, .vfs_cachedroot = tmpfs_root, .vfs_statfs = tmpfs_statfs, .vfs_fhtovp = tmpfs_fhtovp, .vfs_sync = tmpfs_sync, - .vfs_susp_clean = tmpfs_susp_clean, .vfs_init = tmpfs_init, .vfs_uninit = tmpfs_uninit, }; VFS_SET(tmpfs_vfsops, tmpfs, VFCF_JAIL); Index: projects/clang1000-import/sys/i386/i386/trap.c =================================================================== --- projects/clang1000-import/sys/i386/i386/trap.c (revision 357965) +++ projects/clang1000-import/sys/i386/i386/trap.c (revision 357966) @@ -1,1155 +1,1155 @@ /*- * SPDX-License-Identifier: BSD-4-Clause * * Copyright (C) 1994, David Greenman * Copyright (c) 1990, 1993 * The Regents of the University of California. All rights reserved. * * This code is derived from software contributed to Berkeley by * the University of Utah, and William Jolitz. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by the University of * California, Berkeley and its contributors. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * from: @(#)trap.c 7.4 (Berkeley) 5/13/91 */ #include __FBSDID("$FreeBSD$"); /* * 386 Trap and System call handling */ #include "opt_clock.h" #include "opt_compat.h" #include "opt_cpu.h" #include "opt_hwpmc_hooks.h" #include "opt_isa.h" #include "opt_kdb.h" #include "opt_trap.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef HWPMC_HOOKS #include PMC_SOFT_DEFINE( , , page_fault, all); PMC_SOFT_DEFINE( , , page_fault, read); PMC_SOFT_DEFINE( , , page_fault, write); #endif #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef SMP #include #endif #include #include #include #include #ifdef POWERFAIL_NMI #include #include #endif #ifdef KDTRACE_HOOKS #include #endif void trap(struct trapframe *frame); void syscall(struct trapframe *frame); static int trap_pfault(struct trapframe *, bool, vm_offset_t, int *, int *); static void trap_fatal(struct trapframe *, vm_offset_t); #ifdef KDTRACE_HOOKS static bool trap_user_dtrace(struct trapframe *, int (**hook)(struct trapframe *)); #endif void dblfault_handler(void); extern inthand_t IDTVEC(bpt), IDTVEC(dbg), IDTVEC(int0x80_syscall); extern uint64_t pg_nx; struct trap_data { bool ei; const char *msg; }; static const struct trap_data trap_data[] = { [T_PRIVINFLT] = { .ei = true, .msg = "privileged instruction fault" }, [T_BPTFLT] = { .ei = false, .msg = "breakpoint instruction fault" }, [T_ARITHTRAP] = { .ei = true, .msg = "arithmetic trap" }, [T_PROTFLT] = { .ei = true, .msg = "general protection fault" }, [T_TRCTRAP] = { .ei = false, .msg = "debug exception" }, [T_PAGEFLT] = { .ei = true, .msg = "page fault" }, [T_ALIGNFLT] = { .ei = true, .msg = "alignment fault" }, [T_DIVIDE] = { .ei = true, .msg = "integer divide fault" }, [T_NMI] = { .ei = false, .msg = "non-maskable interrupt trap" }, [T_OFLOW] = { .ei = true, .msg = "overflow trap" }, [T_BOUND] = { .ei = true, .msg = "FPU bounds check fault" }, [T_DNA] = { .ei = true, .msg = "FPU device not available" }, [T_DOUBLEFLT] = { .ei = false, .msg = "double fault" }, [T_FPOPFLT] = { .ei = true, .msg = "FPU operand fetch fault" }, [T_TSSFLT] = { .ei = true, .msg = "invalid TSS fault" }, [T_SEGNPFLT] = { .ei = true, .msg = "segment not present fault" }, [T_STKFLT] = { .ei = true, .msg = "stack fault" }, [T_MCHK] = { .ei = true, .msg = "machine check trap" }, [T_XMMFLT] = { .ei = true, .msg = "SIMD floating-point exception" }, [T_DTRACE_RET] ={ .ei = true, .msg = "DTrace pid return trap" }, }; static bool trap_enable_intr(int trapno) { MPASS(trapno > 0); if (trapno < nitems(trap_data) && trap_data[trapno].msg != NULL) return (trap_data[trapno].ei); return (false); } static const char * trap_msg(int trapno) { const char *res; static const char unkn[] = "UNKNOWN"; res = NULL; if (trapno < nitems(trap_data)) res = trap_data[trapno].msg; if (res == NULL) res = unkn; return (res); } #if defined(I586_CPU) && !defined(NO_F00F_HACK) int has_f00f_bug = 0; /* Initialized so that it can be patched. */ #endif static int uprintf_signal; SYSCTL_INT(_machdep, OID_AUTO, uprintf_signal, CTLFLAG_RW, &uprintf_signal, 0, "Print debugging information on trap signal to ctty"); /* * Exception, fault, and trap interface to the FreeBSD kernel. * This common code is called from assembly language IDT gate entry * routines that prepare a suitable stack frame, and restore this * frame after the exception has been processed. */ void trap(struct trapframe *frame) { ksiginfo_t ksi; struct thread *td; struct proc *p; int pf, signo, ucode; u_int type; register_t addr, dr6; vm_offset_t eva; #ifdef POWERFAIL_NMI static int lastalert = 0; #endif td = curthread; p = td->td_proc; dr6 = 0; VM_CNT_INC(v_trap); type = frame->tf_trapno; KASSERT((read_eflags() & PSL_I) == 0, ("trap: interrupts enabled, type %d frame %p", type, frame)); #ifdef SMP /* Handler for NMI IPIs used for stopping CPUs. */ if (type == T_NMI && ipi_nmi_handler() == 0) return; #endif /* SMP */ #ifdef KDB if (kdb_active) { kdb_reenter(); return; } #endif if (type == T_RESERVED) { trap_fatal(frame, 0); return; } if (type == T_NMI) { #ifdef HWPMC_HOOKS /* * CPU PMCs interrupt using an NMI so we check for that first. * If the HWPMC module is active, 'pmc_hook' will point to * the function to be called. A non-zero return value from the * hook means that the NMI was consumed by it and that we can * return immediately. */ if (pmc_intr != NULL && (*pmc_intr)(frame) != 0) return; #endif } if (type == T_MCHK) { mca_intr(); return; } #ifdef KDTRACE_HOOKS /* * A trap can occur while DTrace executes a probe. Before * executing the probe, DTrace blocks re-scheduling and sets * a flag in its per-cpu flags to indicate that it doesn't * want to fault. On returning from the probe, the no-fault * flag is cleared and finally re-scheduling is enabled. */ if ((type == T_PROTFLT || type == T_PAGEFLT) && dtrace_trap_func != NULL && (*dtrace_trap_func)(frame, type)) return; #endif /* * We must not allow context switches until %cr2 is read. * Also, for some Cyrix CPUs, %cr2 is clobbered by interrupts. * All faults use interrupt gates, so %cr2 can be safely read * now, before optional enable of the interrupts below. */ if (type == T_PAGEFLT) eva = rcr2(); /* * Buggy application or kernel code has disabled interrupts * and then trapped. Enabling interrupts now is wrong, but it * is better than running with interrupts disabled until they * are accidentally enabled later. */ if ((frame->tf_eflags & PSL_I) == 0 && TRAPF_USERMODE(frame) && (curpcb->pcb_flags & PCB_VM86CALL) == 0) uprintf("pid %ld (%s): trap %d with interrupts disabled\n", (long)curproc->p_pid, curthread->td_name, type); /* * Conditionally reenable interrupts. If we hold a spin lock, * then we must not reenable interrupts. This might be a * spurious page fault. */ if (trap_enable_intr(type) && td->td_md.md_spinlock_count == 0 && frame->tf_eip != (int)cpu_switch_load_gs) enable_intr(); if (TRAPF_USERMODE(frame) && (curpcb->pcb_flags & PCB_VM86CALL) == 0) { /* user trap */ td->td_pticks = 0; td->td_frame = frame; addr = frame->tf_eip; if (td->td_cowgen != p->p_cowgen) thread_cow_update(td); switch (type) { case T_PRIVINFLT: /* privileged instruction fault */ signo = SIGILL; ucode = ILL_PRVOPC; break; case T_BPTFLT: /* bpt instruction fault */ #ifdef KDTRACE_HOOKS if (trap_user_dtrace(frame, &dtrace_pid_probe_ptr)) return; #else enable_intr(); #endif signo = SIGTRAP; ucode = TRAP_BRKPT; break; case T_TRCTRAP: /* debug exception */ enable_intr(); user_trctrap_out: signo = SIGTRAP; ucode = TRAP_TRACE; dr6 = rdr6(); if ((dr6 & DBREG_DR6_BS) != 0) { PROC_LOCK(td->td_proc); if ((td->td_dbgflags & TDB_STEP) != 0) { td->td_frame->tf_eflags &= ~PSL_T; td->td_dbgflags &= ~TDB_STEP; } PROC_UNLOCK(td->td_proc); } break; case T_ARITHTRAP: /* arithmetic trap */ ucode = npxtrap_x87(); if (ucode == -1) return; signo = SIGFPE; break; /* * The following two traps can happen in vm86 mode, * and, if so, we want to handle them specially. */ case T_PROTFLT: /* general protection fault */ case T_STKFLT: /* stack fault */ if (frame->tf_eflags & PSL_VM) { signo = vm86_emulate((struct vm86frame *)frame); ucode = 0; /* XXXKIB: better code ? */ if (signo == SIGTRAP) { load_dr6(rdr6() | 0x4000); goto user_trctrap_out; } if (signo == 0) goto user; break; } signo = SIGBUS; ucode = (type == T_PROTFLT) ? BUS_OBJERR : BUS_ADRERR; break; case T_SEGNPFLT: /* segment not present fault */ signo = SIGBUS; ucode = BUS_ADRERR; break; case T_TSSFLT: /* invalid TSS fault */ signo = SIGBUS; ucode = BUS_OBJERR; break; case T_ALIGNFLT: signo = SIGBUS; ucode = BUS_ADRALN; break; case T_DOUBLEFLT: /* double fault */ default: signo = SIGBUS; ucode = BUS_OBJERR; break; case T_PAGEFLT: /* page fault */ addr = eva; pf = trap_pfault(frame, true, eva, &signo, &ucode); #if defined(I586_CPU) && !defined(NO_F00F_HACK) if (pf == -2) { /* * The f00f hack workaround has triggered, so * treat the fault as an illegal instruction * (T_PRIVINFLT) instead of a page fault. */ type = frame->tf_trapno = T_PRIVINFLT; break; } #endif if (pf == -1) return; if (pf == 0) goto user; break; case T_DIVIDE: /* integer divide fault */ ucode = FPE_INTDIV; signo = SIGFPE; break; #ifdef DEV_ISA case T_NMI: #ifdef POWERFAIL_NMI #ifndef TIMER_FREQ # define TIMER_FREQ 1193182 #endif if (time_second - lastalert > 10) { log(LOG_WARNING, "NMI: power fail\n"); sysbeep(880, hz); lastalert = time_second; } return; #else /* !POWERFAIL_NMI */ nmi_handle_intr(type, frame); return; #endif /* POWERFAIL_NMI */ #endif /* DEV_ISA */ case T_OFLOW: /* integer overflow fault */ ucode = FPE_INTOVF; signo = SIGFPE; break; case T_BOUND: /* bounds check fault */ ucode = FPE_FLTSUB; signo = SIGFPE; break; case T_DNA: KASSERT(PCB_USER_FPU(td->td_pcb), ("kernel FPU ctx has leaked")); /* transparent fault (due to context switch "late") */ if (npxdna()) return; uprintf("pid %d killed due to lack of floating point\n", p->p_pid); signo = SIGKILL; ucode = 0; break; case T_FPOPFLT: /* FPU operand fetch fault */ ucode = ILL_COPROC; signo = SIGILL; break; case T_XMMFLT: /* SIMD floating-point exception */ ucode = npxtrap_sse(); if (ucode == -1) return; signo = SIGFPE; break; #ifdef KDTRACE_HOOKS case T_DTRACE_RET: (void)trap_user_dtrace(frame, &dtrace_return_probe_ptr); return; #endif } } else { /* kernel trap */ KASSERT(cold || td->td_ucred != NULL, ("kernel trap doesn't have ucred")); switch (type) { case T_PAGEFLT: /* page fault */ (void)trap_pfault(frame, false, eva, NULL, NULL); return; case T_DNA: if (PCB_USER_FPU(td->td_pcb)) panic("Unregistered use of FPU in kernel"); if (npxdna()) return; break; case T_ARITHTRAP: /* arithmetic trap */ case T_XMMFLT: /* SIMD floating-point exception */ case T_FPOPFLT: /* FPU operand fetch fault */ /* * XXXKIB for now disable any FPU traps in kernel * handler registration seems to be overkill */ trap_fatal(frame, 0); return; /* * The following two traps can happen in * vm86 mode, and, if so, we want to handle * them specially. */ case T_PROTFLT: /* general protection fault */ case T_STKFLT: /* stack fault */ if (frame->tf_eflags & PSL_VM) { signo = vm86_emulate((struct vm86frame *)frame); if (signo == SIGTRAP) { type = T_TRCTRAP; load_dr6(rdr6() | 0x4000); goto kernel_trctrap; } if (signo != 0) /* * returns to original process */ vm86_trap((struct vm86frame *)frame); return; } /* FALL THROUGH */ case T_SEGNPFLT: /* segment not present fault */ if (curpcb->pcb_flags & PCB_VM86CALL) break; /* * Invalid %fs's and %gs's can be created using * procfs or PT_SETREGS or by invalidating the * underlying LDT entry. This causes a fault * in kernel mode when the kernel attempts to * switch contexts. Lose the bad context * (XXX) so that we can continue, and generate * a signal. */ if (frame->tf_eip == (int)cpu_switch_load_gs) { curpcb->pcb_gs = 0; #if 0 PROC_LOCK(p); kern_psignal(p, SIGBUS); PROC_UNLOCK(p); #endif return; } if (td->td_intr_nesting_level != 0) break; /* * Invalid segment selectors and out of bounds * %eip's and %esp's can be set up in user mode. * This causes a fault in kernel mode when the * kernel tries to return to user mode. We want * to get this fault so that we can fix the * problem here and not have to check all the * selectors and pointers when the user changes * them. * * N.B. Comparing to long mode, 32-bit mode * does not push %esp on the trap frame, * because iretl faulted while in ring 0. As * the consequence, there is no need to fixup * the stack pointer for doreti_iret_fault, * the fixup and the complimentary trap() call * are executed on the main thread stack, not * on the trampoline stack. */ if (frame->tf_eip == (int)doreti_iret + setidt_disp) { frame->tf_eip = (int)doreti_iret_fault + setidt_disp; return; } if (type == T_STKFLT) break; if (frame->tf_eip == (int)doreti_popl_ds + setidt_disp) { frame->tf_eip = (int)doreti_popl_ds_fault + setidt_disp; return; } if (frame->tf_eip == (int)doreti_popl_es + setidt_disp) { frame->tf_eip = (int)doreti_popl_es_fault + setidt_disp; return; } if (frame->tf_eip == (int)doreti_popl_fs + setidt_disp) { frame->tf_eip = (int)doreti_popl_fs_fault + setidt_disp; return; } if (curpcb->pcb_onfault != NULL) { frame->tf_eip = (int)curpcb->pcb_onfault; return; } break; case T_TSSFLT: /* * PSL_NT can be set in user mode and isn't cleared * automatically when the kernel is entered. This * causes a TSS fault when the kernel attempts to * `iret' because the TSS link is uninitialized. We * want to get this fault so that we can fix the * problem here and not every time the kernel is * entered. */ if (frame->tf_eflags & PSL_NT) { frame->tf_eflags &= ~PSL_NT; return; } break; case T_TRCTRAP: /* debug exception */ kernel_trctrap: /* Clear any pending debug events. */ dr6 = rdr6(); load_dr6(0); /* * Ignore debug register exceptions due to * accesses in the user's address space, which * can happen under several conditions such as * if a user sets a watchpoint on a buffer and * then passes that buffer to a system call. * We still want to get TRCTRAPS for addresses * in kernel space because that is useful when * debugging the kernel. */ if (user_dbreg_trap(dr6) && !(curpcb->pcb_flags & PCB_VM86CALL)) return; /* * Malicious user code can configure a debug * register watchpoint to trap on data access * to the top of stack and then execute 'pop * %ss; int 3'. Due to exception deferral for * 'pop %ss', the CPU will not interrupt 'int * 3' to raise the DB# exception for the debug * register but will postpone the DB# until * execution of the first instruction of the * BP# handler (in kernel mode). Normally the * previous check would ignore DB# exceptions * for watchpoints on user addresses raised in * kernel mode. However, some CPU errata * include cases where DB# exceptions do not * properly set bits in %dr6, e.g. Haswell * HSD23 and Skylake-X SKZ24. * * A deferred DB# can also be raised on the * first instructions of system call entry * points or single-step traps via similar use * of 'pop %ss' or 'mov xxx, %ss'. */ if (frame->tf_eip == (uintptr_t)IDTVEC(int0x80_syscall) + setidt_disp || frame->tf_eip == (uintptr_t)IDTVEC(bpt) + setidt_disp || frame->tf_eip == (uintptr_t)IDTVEC(dbg) + setidt_disp) return; /* * FALLTHROUGH (TRCTRAP kernel mode, kernel address) */ case T_BPTFLT: /* * If KDB is enabled, let it handle the debugger trap. * Otherwise, debugger traps "can't happen". */ #ifdef KDB if (kdb_trap(type, dr6, frame)) return; #endif break; #ifdef DEV_ISA case T_NMI: #ifdef POWERFAIL_NMI if (time_second - lastalert > 10) { log(LOG_WARNING, "NMI: power fail\n"); sysbeep(880, hz); lastalert = time_second; } return; #else /* !POWERFAIL_NMI */ nmi_handle_intr(type, frame); return; #endif /* POWERFAIL_NMI */ #endif /* DEV_ISA */ } trap_fatal(frame, eva); return; } /* Translate fault for emulators (e.g. Linux) */ if (*p->p_sysent->sv_transtrap != NULL) signo = (*p->p_sysent->sv_transtrap)(signo, type); ksiginfo_init_trap(&ksi); ksi.ksi_signo = signo; ksi.ksi_code = ucode; ksi.ksi_addr = (void *)addr; ksi.ksi_trapno = type; if (uprintf_signal) { uprintf("pid %d comm %s: signal %d err %x code %d type %d " "addr 0x%x ss 0x%04x esp 0x%08x cs 0x%04x eip 0x%08x " "<%02x %02x %02x %02x %02x %02x %02x %02x>\n", p->p_pid, p->p_comm, signo, frame->tf_err, ucode, type, addr, frame->tf_ss, frame->tf_esp, frame->tf_cs, frame->tf_eip, fubyte((void *)(frame->tf_eip + 0)), fubyte((void *)(frame->tf_eip + 1)), fubyte((void *)(frame->tf_eip + 2)), fubyte((void *)(frame->tf_eip + 3)), fubyte((void *)(frame->tf_eip + 4)), fubyte((void *)(frame->tf_eip + 5)), fubyte((void *)(frame->tf_eip + 6)), fubyte((void *)(frame->tf_eip + 7))); } KASSERT((read_eflags() & PSL_I) != 0, ("interrupts disabled")); trapsignal(td, &ksi); user: userret(td, frame); KASSERT(PCB_USER_FPU(td->td_pcb), ("Return from trap with kernel FPU ctx leaked")); } /* * Handle all details of a page fault. * Returns: * -2 if the fault was caused by triggered workaround for Intel Pentium * 0xf00f bug. * -1 if this fault was fatal, typically from kernel mode * (cannot happen, but we need to return something). * 0 if this fault was handled by updating either the user or kernel * page table, execution can continue. * 1 if this fault was from usermode and it was not handled, a synchronous * signal should be delivered to the thread. *signo returns the signal * number, *ucode gives si_code. */ static int trap_pfault(struct trapframe *frame, bool usermode, vm_offset_t eva, int *signo, int *ucode) { struct thread *td; struct proc *p; vm_map_t map; int rv; vm_prot_t ftype; MPASS(!usermode || (signo != NULL && ucode != NULL)); td = curthread; p = td->td_proc; if (__predict_false((td->td_pflags & TDP_NOFAULTING) != 0)) { /* * Due to both processor errata and lazy TLB invalidation when * access restrictions are removed from virtual pages, memory * accesses that are allowed by the physical mapping layer may * nonetheless cause one spurious page fault per virtual page. * When the thread is executing a "no faulting" section that * is bracketed by vm_fault_{disable,enable}_pagefaults(), * every page fault is treated as a spurious page fault, * unless it accesses the same virtual address as the most * recent page fault within the same "no faulting" section. */ if (td->td_md.md_spurflt_addr != eva || (td->td_pflags & TDP_RESETSPUR) != 0) { /* * Do nothing to the TLB. A stale TLB entry is * flushed automatically by a page fault. */ td->td_md.md_spurflt_addr = eva; td->td_pflags &= ~TDP_RESETSPUR; return (0); } } else { /* * If we get a page fault while in a critical section, then * it is most likely a fatal kernel page fault. The kernel * is already going to panic trying to get a sleep lock to * do the VM lookup, so just consider it a fatal trap so the * kernel can print out a useful trap message and even get * to the debugger. * * If we get a page fault while holding a non-sleepable * lock, then it is most likely a fatal kernel page fault. * If WITNESS is enabled, then it's going to whine about * bogus LORs with various VM locks, so just skip to the * fatal trap handling directly. */ if (td->td_critnest != 0 || WITNESS_CHECK(WARN_SLEEPOK | WARN_GIANTOK, NULL, "Kernel page fault") != 0) { trap_fatal(frame, eva); return (-1); } } if (eva >= PMAP_TRM_MIN_ADDRESS) { /* * Don't allow user-mode faults in kernel address space. * An exception: if the faulting address is the invalid * instruction entry in the IDT, then the Intel Pentium * F00F bug workaround was triggered, and we need to * treat it is as an illegal instruction, and not a page * fault. */ #if defined(I586_CPU) && !defined(NO_F00F_HACK) if ((eva == (unsigned int)&idt[6]) && has_f00f_bug) { *ucode = ILL_PRVOPC; *signo = SIGILL; return (-2); } #endif if (usermode) { *signo = SIGSEGV; *ucode = SEGV_MAPERR; return (1); } trap_fatal(frame, eva); return (-1); } else { map = usermode ? &p->p_vmspace->vm_map : kernel_map; /* * Kernel cannot access a user-space address directly * because user pages are not mapped. Also, page * faults must not be caused during the interrupts. */ if (!usermode && td->td_intr_nesting_level != 0) { trap_fatal(frame, eva); return (-1); } } /* * If the trap was caused by errant bits in the PTE then panic. */ if (frame->tf_err & PGEX_RSV) { trap_fatal(frame, eva); return (-1); } /* * PGEX_I is defined only if the execute disable bit capability is * supported and enabled. */ if (frame->tf_err & PGEX_W) ftype = VM_PROT_WRITE; else if ((frame->tf_err & PGEX_I) && pg_nx != 0) ftype = VM_PROT_EXECUTE; else ftype = VM_PROT_READ; /* Fault in the page. */ rv = vm_fault_trap(map, eva, ftype, VM_FAULT_NORMAL, signo, ucode); if (rv == KERN_SUCCESS) { #ifdef HWPMC_HOOKS if (ftype == VM_PROT_READ || ftype == VM_PROT_WRITE) { PMC_SOFT_CALL_TF( , , page_fault, all, frame); if (ftype == VM_PROT_READ) PMC_SOFT_CALL_TF( , , page_fault, read, frame); else PMC_SOFT_CALL_TF( , , page_fault, write, frame); } #endif return (0); } if (usermode) return (1); if (td->td_intr_nesting_level == 0 && curpcb->pcb_onfault != NULL) { frame->tf_eip = (int)curpcb->pcb_onfault; return (0); } trap_fatal(frame, eva); return (-1); } static void trap_fatal(frame, eva) struct trapframe *frame; vm_offset_t eva; { int code, ss, esp; u_int type; struct soft_segment_descriptor softseg; #ifdef KDB bool handled; #endif code = frame->tf_err; type = frame->tf_trapno; sdtossd(&gdt[IDXSEL(frame->tf_cs & 0xffff)].sd, &softseg); printf("\n\nFatal trap %d: %s while in %s mode\n", type, trap_msg(type), frame->tf_eflags & PSL_VM ? "vm86" : ISPL(frame->tf_cs) == SEL_UPL ? "user" : "kernel"); #ifdef SMP /* two separate prints in case of a trap on an unmapped page */ printf("cpuid = %d; ", PCPU_GET(cpuid)); printf("apic id = %02x\n", PCPU_GET(apic_id)); #endif if (type == T_PAGEFLT) { printf("fault virtual address = 0x%x\n", eva); printf("fault code = %s %s%s, %s\n", code & PGEX_U ? "user" : "supervisor", code & PGEX_W ? "write" : "read", pg_nx != 0 ? (code & PGEX_I ? " instruction" : " data") : "", code & PGEX_RSV ? "reserved bits in PTE" : code & PGEX_P ? "protection violation" : "page not present"); } else { printf("error code = %#x\n", code); } printf("instruction pointer = 0x%x:0x%x\n", frame->tf_cs & 0xffff, frame->tf_eip); if (TF_HAS_STACKREGS(frame)) { ss = frame->tf_ss & 0xffff; esp = frame->tf_esp; } else { ss = GSEL(GDATA_SEL, SEL_KPL); esp = (int)&frame->tf_esp; } printf("stack pointer = 0x%x:0x%x\n", ss, esp); printf("frame pointer = 0x%x:0x%x\n", ss, frame->tf_ebp); printf("code segment = base 0x%x, limit 0x%x, type 0x%x\n", softseg.ssd_base, softseg.ssd_limit, softseg.ssd_type); printf(" = DPL %d, pres %d, def32 %d, gran %d\n", softseg.ssd_dpl, softseg.ssd_p, softseg.ssd_def32, softseg.ssd_gran); printf("processor eflags = "); if (frame->tf_eflags & PSL_T) printf("trace trap, "); if (frame->tf_eflags & PSL_I) printf("interrupt enabled, "); if (frame->tf_eflags & PSL_NT) printf("nested task, "); if (frame->tf_eflags & PSL_RF) printf("resume, "); if (frame->tf_eflags & PSL_VM) printf("vm86, "); printf("IOPL = %d\n", (frame->tf_eflags & PSL_IOPL) >> 12); printf("current process = %d (%s)\n", curproc->p_pid, curthread->td_name); #ifdef KDB if (debugger_on_trap) { kdb_why = KDB_WHY_TRAP; frame->tf_err = eva; /* smuggle fault address to ddb */ handled = kdb_trap(type, 0, frame); frame->tf_err = code; /* restore error code */ kdb_why = KDB_WHY_UNSET; if (handled) return; } #endif printf("trap number = %d\n", type); if (trap_msg(type) != NULL) panic("%s", trap_msg(type)); else panic("unknown/reserved trap"); } #ifdef KDTRACE_HOOKS /* * Invoke a userspace DTrace hook. The hook pointer is cleared when no * userspace probes are enabled, so we must synchronize with DTrace to ensure * that a trapping thread is able to call the hook before it is cleared. */ static bool trap_user_dtrace(struct trapframe *frame, int (**hookp)(struct trapframe *)) { int (*hook)(struct trapframe *); - hook = (int (*)(struct trapframe *))atomic_load_ptr(hookp); + hook = atomic_load_ptr(hookp); enable_intr(); if (hook != NULL) return ((hook)(frame) == 0); return (false); } #endif /* * Double fault handler. Called when a fault occurs while writing * a frame for a trap/exception onto the stack. This usually occurs * when the stack overflows (such is the case with infinite recursion, * for example). * * XXX Note that the current PTD gets replaced by IdlePTD when the * task switch occurs. This means that the stack that was active at * the time of the double fault is not available at unless * the machine was idle when the double fault occurred. The downside * of this is that "trace " in ddb won't work. */ void dblfault_handler(void) { #ifdef KDTRACE_HOOKS if (dtrace_doubletrap_func != NULL) (*dtrace_doubletrap_func)(); #endif printf("\nFatal double fault:\n"); printf("eip = 0x%x\n", PCPU_GET(common_tssp)->tss_eip); printf("esp = 0x%x\n", PCPU_GET(common_tssp)->tss_esp); printf("ebp = 0x%x\n", PCPU_GET(common_tssp)->tss_ebp); #ifdef SMP /* two separate prints in case of a trap on an unmapped page */ printf("cpuid = %d; ", PCPU_GET(cpuid)); printf("apic id = %02x\n", PCPU_GET(apic_id)); #endif panic("double fault"); } int cpu_fetch_syscall_args(struct thread *td) { struct proc *p; struct trapframe *frame; struct syscall_args *sa; caddr_t params; long tmp; int error; #ifdef COMPAT_43 u_int32_t eip; int cs; #endif p = td->td_proc; frame = td->td_frame; sa = &td->td_sa; #ifdef COMPAT_43 if (__predict_false(frame->tf_cs == 7 && frame->tf_eip == 2)) { /* * In lcall $7,$0 after int $0x80. Convert the user * frame to what it would be for a direct int 0x80 instead * of lcall $7,$0, by popping the lcall return address. */ error = fueword32((void *)frame->tf_esp, &eip); if (error == -1) return (EFAULT); cs = fuword16((void *)(frame->tf_esp + sizeof(u_int32_t))); if (cs == -1) return (EFAULT); /* * Unwind in-kernel frame after all stack frame pieces * were successfully read. */ frame->tf_eip = eip; frame->tf_cs = cs; frame->tf_esp += 2 * sizeof(u_int32_t); frame->tf_err = 7; /* size of lcall $7,$0 */ } #endif sa->code = frame->tf_eax; params = (caddr_t)frame->tf_esp + sizeof(uint32_t); /* * Need to check if this is a 32 bit or 64 bit syscall. */ if (sa->code == SYS_syscall) { /* * Code is first argument, followed by actual args. */ error = fueword(params, &tmp); if (error == -1) return (EFAULT); sa->code = tmp; params += sizeof(uint32_t); } else if (sa->code == SYS___syscall) { /* * Like syscall, but code is a quad, so as to maintain * quad alignment for the rest of the arguments. */ error = fueword(params, &tmp); if (error == -1) return (EFAULT); sa->code = tmp; params += sizeof(quad_t); } if (sa->code >= p->p_sysent->sv_size) sa->callp = &p->p_sysent->sv_table[0]; else sa->callp = &p->p_sysent->sv_table[sa->code]; sa->narg = sa->callp->sy_narg; if (params != NULL && sa->narg != 0) error = copyin(params, (caddr_t)sa->args, (u_int)(sa->narg * sizeof(uint32_t))); else error = 0; if (error == 0) { td->td_retval[0] = 0; td->td_retval[1] = frame->tf_edx; } return (error); } #include "../../kern/subr_syscall.c" /* * syscall - system call request C handler. A system call is * essentially treated as a trap by reusing the frame layout. */ void syscall(struct trapframe *frame) { struct thread *td; register_t orig_tf_eflags; ksiginfo_t ksi; #ifdef DIAGNOSTIC if (!(TRAPF_USERMODE(frame) && (curpcb->pcb_flags & PCB_VM86CALL) == 0)) { panic("syscall"); /* NOT REACHED */ } #endif orig_tf_eflags = frame->tf_eflags; td = curthread; td->td_frame = frame; syscallenter(td); /* * Traced syscall. */ if ((orig_tf_eflags & PSL_T) && !(orig_tf_eflags & PSL_VM)) { frame->tf_eflags &= ~PSL_T; ksiginfo_init_trap(&ksi); ksi.ksi_signo = SIGTRAP; ksi.ksi_code = TRAP_TRACE; ksi.ksi_addr = (void *)frame->tf_eip; trapsignal(td, &ksi); } KASSERT(PCB_USER_FPU(td->td_pcb), ("System call %s returning with kernel FPU ctx leaked", syscallname(td->td_proc, td->td_sa.code))); KASSERT(td->td_pcb->pcb_save == get_pcb_user_save_td(td), ("System call %s returning with mangled pcb_save", syscallname(td->td_proc, td->td_sa.code))); syscallret(td); } Index: projects/clang1000-import/sys/kern/kern_descrip.c =================================================================== --- projects/clang1000-import/sys/kern/kern_descrip.c (revision 357965) +++ projects/clang1000-import/sys/kern/kern_descrip.c (revision 357966) @@ -1,4402 +1,4402 @@ /*- * SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 1982, 1986, 1989, 1991, 1993 * The Regents of the University of California. All rights reserved. * (c) UNIX System Laboratories, Inc. * All or some portions of this file are derived from material licensed * to the University of California by American Telephone and Telegraph * Co. or Unix System Laboratories, Inc. and are reproduced herein with * the permission of UNIX System Laboratories, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)kern_descrip.c 8.6 (Berkeley) 4/19/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_ddb.h" #include "opt_ktrace.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef KTRACE #include #endif #include #include #include #include #include static MALLOC_DEFINE(M_FILEDESC, "filedesc", "Open file descriptor table"); static MALLOC_DEFINE(M_FILEDESC_TO_LEADER, "filedesc_to_leader", "file desc to leader structures"); static MALLOC_DEFINE(M_SIGIO, "sigio", "sigio structures"); MALLOC_DEFINE(M_FILECAPS, "filecaps", "descriptor capabilities"); MALLOC_DECLARE(M_FADVISE); static __read_mostly uma_zone_t file_zone; static __read_mostly uma_zone_t filedesc0_zone; static int closefp(struct filedesc *fdp, int fd, struct file *fp, struct thread *td, int holdleaders); static int fd_first_free(struct filedesc *fdp, int low, int size); static int fd_last_used(struct filedesc *fdp, int size); static void fdgrowtable(struct filedesc *fdp, int nfd); static void fdgrowtable_exp(struct filedesc *fdp, int nfd); static void fdunused(struct filedesc *fdp, int fd); static void fdused(struct filedesc *fdp, int fd); static int getmaxfd(struct thread *td); static u_long *filecaps_copy_prep(const struct filecaps *src); static void filecaps_copy_finish(const struct filecaps *src, struct filecaps *dst, u_long *ioctls); static u_long *filecaps_free_prep(struct filecaps *fcaps); static void filecaps_free_finish(u_long *ioctls); /* * Each process has: * * - An array of open file descriptors (fd_ofiles) * - An array of file flags (fd_ofileflags) * - A bitmap recording which descriptors are in use (fd_map) * * A process starts out with NDFILE descriptors. The value of NDFILE has * been selected based the historical limit of 20 open files, and an * assumption that the majority of processes, especially short-lived * processes like shells, will never need more. * * If this initial allocation is exhausted, a larger descriptor table and * map are allocated dynamically, and the pointers in the process's struct * filedesc are updated to point to those. This is repeated every time * the process runs out of file descriptors (provided it hasn't hit its * resource limit). * * Since threads may hold references to individual descriptor table * entries, the tables are never freed. Instead, they are placed on a * linked list and freed only when the struct filedesc is released. */ #define NDFILE 20 #define NDSLOTSIZE sizeof(NDSLOTTYPE) #define NDENTRIES (NDSLOTSIZE * __CHAR_BIT) #define NDSLOT(x) ((x) / NDENTRIES) #define NDBIT(x) ((NDSLOTTYPE)1 << ((x) % NDENTRIES)) #define NDSLOTS(x) (((x) + NDENTRIES - 1) / NDENTRIES) /* * SLIST entry used to keep track of ofiles which must be reclaimed when * the process exits. */ struct freetable { struct fdescenttbl *ft_table; SLIST_ENTRY(freetable) ft_next; }; /* * Initial allocation: a filedesc structure + the head of SLIST used to * keep track of old ofiles + enough space for NDFILE descriptors. */ struct fdescenttbl0 { int fdt_nfiles; struct filedescent fdt_ofiles[NDFILE]; }; struct filedesc0 { struct filedesc fd_fd; SLIST_HEAD(, freetable) fd_free; struct fdescenttbl0 fd_dfiles; NDSLOTTYPE fd_dmap[NDSLOTS(NDFILE)]; }; /* * Descriptor management. */ static int __exclusive_cache_line openfiles; /* actual number of open files */ struct mtx sigio_lock; /* mtx to protect pointers to sigio */ void __read_mostly (*mq_fdclose)(struct thread *td, int fd, struct file *fp); /* * If low >= size, just return low. Otherwise find the first zero bit in the * given bitmap, starting at low and not exceeding size - 1. Return size if * not found. */ static int fd_first_free(struct filedesc *fdp, int low, int size) { NDSLOTTYPE *map = fdp->fd_map; NDSLOTTYPE mask; int off, maxoff; if (low >= size) return (low); off = NDSLOT(low); if (low % NDENTRIES) { mask = ~(~(NDSLOTTYPE)0 >> (NDENTRIES - (low % NDENTRIES))); if ((mask &= ~map[off]) != 0UL) return (off * NDENTRIES + ffsl(mask) - 1); ++off; } for (maxoff = NDSLOTS(size); off < maxoff; ++off) if (map[off] != ~0UL) return (off * NDENTRIES + ffsl(~map[off]) - 1); return (size); } /* * Find the highest non-zero bit in the given bitmap, starting at 0 and * not exceeding size - 1. Return -1 if not found. */ static int fd_last_used(struct filedesc *fdp, int size) { NDSLOTTYPE *map = fdp->fd_map; NDSLOTTYPE mask; int off, minoff; off = NDSLOT(size); if (size % NDENTRIES) { mask = ~(~(NDSLOTTYPE)0 << (size % NDENTRIES)); if ((mask &= map[off]) != 0) return (off * NDENTRIES + flsl(mask) - 1); --off; } for (minoff = NDSLOT(0); off >= minoff; --off) if (map[off] != 0) return (off * NDENTRIES + flsl(map[off]) - 1); return (-1); } static int fdisused(struct filedesc *fdp, int fd) { KASSERT(fd >= 0 && fd < fdp->fd_nfiles, ("file descriptor %d out of range (0, %d)", fd, fdp->fd_nfiles)); return ((fdp->fd_map[NDSLOT(fd)] & NDBIT(fd)) != 0); } /* * Mark a file descriptor as used. */ static void fdused_init(struct filedesc *fdp, int fd) { KASSERT(!fdisused(fdp, fd), ("fd=%d is already used", fd)); fdp->fd_map[NDSLOT(fd)] |= NDBIT(fd); } static void fdused(struct filedesc *fdp, int fd) { FILEDESC_XLOCK_ASSERT(fdp); fdused_init(fdp, fd); if (fd > fdp->fd_lastfile) fdp->fd_lastfile = fd; if (fd == fdp->fd_freefile) fdp->fd_freefile++; } /* * Mark a file descriptor as unused. */ static void fdunused(struct filedesc *fdp, int fd) { FILEDESC_XLOCK_ASSERT(fdp); KASSERT(fdisused(fdp, fd), ("fd=%d is already unused", fd)); KASSERT(fdp->fd_ofiles[fd].fde_file == NULL, ("fd=%d is still in use", fd)); fdp->fd_map[NDSLOT(fd)] &= ~NDBIT(fd); if (fd < fdp->fd_freefile) fdp->fd_freefile = fd; if (fd == fdp->fd_lastfile) fdp->fd_lastfile = fd_last_used(fdp, fd); } /* * Free a file descriptor. * * Avoid some work if fdp is about to be destroyed. */ static inline void fdefree_last(struct filedescent *fde) { filecaps_free(&fde->fde_caps); } static inline void fdfree(struct filedesc *fdp, int fd) { struct filedescent *fde; fde = &fdp->fd_ofiles[fd]; #ifdef CAPABILITIES seqc_write_begin(&fde->fde_seqc); #endif fde->fde_file = NULL; #ifdef CAPABILITIES seqc_write_end(&fde->fde_seqc); #endif fdefree_last(fde); fdunused(fdp, fd); } void pwd_ensure_dirs(void) { struct filedesc *fdp; fdp = curproc->p_fd; FILEDESC_XLOCK(fdp); if (fdp->fd_cdir == NULL) { fdp->fd_cdir = rootvnode; vrefact(rootvnode); } if (fdp->fd_rdir == NULL) { fdp->fd_rdir = rootvnode; vrefact(rootvnode); } FILEDESC_XUNLOCK(fdp); } /* * System calls on descriptors. */ #ifndef _SYS_SYSPROTO_H_ struct getdtablesize_args { int dummy; }; #endif /* ARGSUSED */ int sys_getdtablesize(struct thread *td, struct getdtablesize_args *uap) { #ifdef RACCT uint64_t lim; #endif td->td_retval[0] = getmaxfd(td); #ifdef RACCT PROC_LOCK(td->td_proc); lim = racct_get_limit(td->td_proc, RACCT_NOFILE); PROC_UNLOCK(td->td_proc); if (lim < td->td_retval[0]) td->td_retval[0] = lim; #endif return (0); } /* * Duplicate a file descriptor to a particular value. * * Note: keep in mind that a potential race condition exists when closing * descriptors from a shared descriptor table (via rfork). */ #ifndef _SYS_SYSPROTO_H_ struct dup2_args { u_int from; u_int to; }; #endif /* ARGSUSED */ int sys_dup2(struct thread *td, struct dup2_args *uap) { return (kern_dup(td, FDDUP_FIXED, 0, (int)uap->from, (int)uap->to)); } /* * Duplicate a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct dup_args { u_int fd; }; #endif /* ARGSUSED */ int sys_dup(struct thread *td, struct dup_args *uap) { return (kern_dup(td, FDDUP_NORMAL, 0, (int)uap->fd, 0)); } /* * The file control system call. */ #ifndef _SYS_SYSPROTO_H_ struct fcntl_args { int fd; int cmd; long arg; }; #endif /* ARGSUSED */ int sys_fcntl(struct thread *td, struct fcntl_args *uap) { return (kern_fcntl_freebsd(td, uap->fd, uap->cmd, uap->arg)); } int kern_fcntl_freebsd(struct thread *td, int fd, int cmd, long arg) { struct flock fl; struct __oflock ofl; intptr_t arg1; int error, newcmd; error = 0; newcmd = cmd; switch (cmd) { case F_OGETLK: case F_OSETLK: case F_OSETLKW: /* * Convert old flock structure to new. */ error = copyin((void *)(intptr_t)arg, &ofl, sizeof(ofl)); fl.l_start = ofl.l_start; fl.l_len = ofl.l_len; fl.l_pid = ofl.l_pid; fl.l_type = ofl.l_type; fl.l_whence = ofl.l_whence; fl.l_sysid = 0; switch (cmd) { case F_OGETLK: newcmd = F_GETLK; break; case F_OSETLK: newcmd = F_SETLK; break; case F_OSETLKW: newcmd = F_SETLKW; break; } arg1 = (intptr_t)&fl; break; case F_GETLK: case F_SETLK: case F_SETLKW: case F_SETLK_REMOTE: error = copyin((void *)(intptr_t)arg, &fl, sizeof(fl)); arg1 = (intptr_t)&fl; break; default: arg1 = arg; break; } if (error) return (error); error = kern_fcntl(td, fd, newcmd, arg1); if (error) return (error); if (cmd == F_OGETLK) { ofl.l_start = fl.l_start; ofl.l_len = fl.l_len; ofl.l_pid = fl.l_pid; ofl.l_type = fl.l_type; ofl.l_whence = fl.l_whence; error = copyout(&ofl, (void *)(intptr_t)arg, sizeof(ofl)); } else if (cmd == F_GETLK) { error = copyout(&fl, (void *)(intptr_t)arg, sizeof(fl)); } return (error); } int kern_fcntl(struct thread *td, int fd, int cmd, intptr_t arg) { struct filedesc *fdp; struct flock *flp; struct file *fp, *fp2; struct filedescent *fde; struct proc *p; struct vnode *vp; struct mount *mp; int error, flg, seals, tmp; uint64_t bsize; off_t foffset; error = 0; flg = F_POSIX; p = td->td_proc; fdp = p->p_fd; AUDIT_ARG_FD(cmd); AUDIT_ARG_CMD(cmd); switch (cmd) { case F_DUPFD: tmp = arg; error = kern_dup(td, FDDUP_FCNTL, 0, fd, tmp); break; case F_DUPFD_CLOEXEC: tmp = arg; error = kern_dup(td, FDDUP_FCNTL, FDDUP_FLAG_CLOEXEC, fd, tmp); break; case F_DUP2FD: tmp = arg; error = kern_dup(td, FDDUP_FIXED, 0, fd, tmp); break; case F_DUP2FD_CLOEXEC: tmp = arg; error = kern_dup(td, FDDUP_FIXED, FDDUP_FLAG_CLOEXEC, fd, tmp); break; case F_GETFD: error = EBADF; FILEDESC_SLOCK(fdp); fde = fdeget_locked(fdp, fd); if (fde != NULL) { td->td_retval[0] = (fde->fde_flags & UF_EXCLOSE) ? FD_CLOEXEC : 0; error = 0; } FILEDESC_SUNLOCK(fdp); break; case F_SETFD: error = EBADF; FILEDESC_XLOCK(fdp); fde = fdeget_locked(fdp, fd); if (fde != NULL) { fde->fde_flags = (fde->fde_flags & ~UF_EXCLOSE) | (arg & FD_CLOEXEC ? UF_EXCLOSE : 0); error = 0; } FILEDESC_XUNLOCK(fdp); break; case F_GETFL: error = fget_fcntl(td, fd, &cap_fcntl_rights, F_GETFL, &fp); if (error != 0) break; td->td_retval[0] = OFLAGS(fp->f_flag); fdrop(fp, td); break; case F_SETFL: error = fget_fcntl(td, fd, &cap_fcntl_rights, F_SETFL, &fp); if (error != 0) break; do { tmp = flg = fp->f_flag; tmp &= ~FCNTLFLAGS; tmp |= FFLAGS(arg & ~O_ACCMODE) & FCNTLFLAGS; } while(atomic_cmpset_int(&fp->f_flag, flg, tmp) == 0); tmp = fp->f_flag & FNONBLOCK; error = fo_ioctl(fp, FIONBIO, &tmp, td->td_ucred, td); if (error != 0) { fdrop(fp, td); break; } tmp = fp->f_flag & FASYNC; error = fo_ioctl(fp, FIOASYNC, &tmp, td->td_ucred, td); if (error == 0) { fdrop(fp, td); break; } atomic_clear_int(&fp->f_flag, FNONBLOCK); tmp = 0; (void)fo_ioctl(fp, FIONBIO, &tmp, td->td_ucred, td); fdrop(fp, td); break; case F_GETOWN: error = fget_fcntl(td, fd, &cap_fcntl_rights, F_GETOWN, &fp); if (error != 0) break; error = fo_ioctl(fp, FIOGETOWN, &tmp, td->td_ucred, td); if (error == 0) td->td_retval[0] = tmp; fdrop(fp, td); break; case F_SETOWN: error = fget_fcntl(td, fd, &cap_fcntl_rights, F_SETOWN, &fp); if (error != 0) break; tmp = arg; error = fo_ioctl(fp, FIOSETOWN, &tmp, td->td_ucred, td); fdrop(fp, td); break; case F_SETLK_REMOTE: error = priv_check(td, PRIV_NFS_LOCKD); if (error != 0) return (error); flg = F_REMOTE; goto do_setlk; case F_SETLKW: flg |= F_WAIT; /* FALLTHROUGH F_SETLK */ case F_SETLK: do_setlk: flp = (struct flock *)arg; if ((flg & F_REMOTE) != 0 && flp->l_sysid == 0) { error = EINVAL; break; } error = fget_unlocked(fdp, fd, &cap_flock_rights, &fp); if (error != 0) break; if (fp->f_type != DTYPE_VNODE) { error = EBADF; fdrop(fp, td); break; } if (flp->l_whence == SEEK_CUR) { foffset = foffset_get(fp); if (foffset < 0 || (flp->l_start > 0 && foffset > OFF_MAX - flp->l_start)) { error = EOVERFLOW; fdrop(fp, td); break; } flp->l_start += foffset; } vp = fp->f_vnode; switch (flp->l_type) { case F_RDLCK: if ((fp->f_flag & FREAD) == 0) { error = EBADF; break; } if ((p->p_leader->p_flag & P_ADVLOCK) == 0) { PROC_LOCK(p->p_leader); p->p_leader->p_flag |= P_ADVLOCK; PROC_UNLOCK(p->p_leader); } error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_SETLK, flp, flg); break; case F_WRLCK: if ((fp->f_flag & FWRITE) == 0) { error = EBADF; break; } if ((p->p_leader->p_flag & P_ADVLOCK) == 0) { PROC_LOCK(p->p_leader); p->p_leader->p_flag |= P_ADVLOCK; PROC_UNLOCK(p->p_leader); } error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_SETLK, flp, flg); break; case F_UNLCK: error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_UNLCK, flp, flg); break; case F_UNLCKSYS: if (flg != F_REMOTE) { error = EINVAL; break; } error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_UNLCKSYS, flp, flg); break; default: error = EINVAL; break; } if (error != 0 || flp->l_type == F_UNLCK || flp->l_type == F_UNLCKSYS) { fdrop(fp, td); break; } /* * Check for a race with close. * * The vnode is now advisory locked (or unlocked, but this case * is not really important) as the caller requested. * We had to drop the filedesc lock, so we need to recheck if * the descriptor is still valid, because if it was closed * in the meantime we need to remove advisory lock from the * vnode - close on any descriptor leading to an advisory * locked vnode, removes that lock. * We will return 0 on purpose in that case, as the result of * successful advisory lock might have been externally visible * already. This is fine - effectively we pretend to the caller * that the closing thread was a bit slower and that the * advisory lock succeeded before the close. */ error = fget_unlocked(fdp, fd, &cap_no_rights, &fp2); if (error != 0) { fdrop(fp, td); break; } if (fp != fp2) { flp->l_whence = SEEK_SET; flp->l_start = 0; flp->l_len = 0; flp->l_type = F_UNLCK; (void) VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_UNLCK, flp, F_POSIX); } fdrop(fp, td); fdrop(fp2, td); break; case F_GETLK: error = fget_unlocked(fdp, fd, &cap_flock_rights, &fp); if (error != 0) break; if (fp->f_type != DTYPE_VNODE) { error = EBADF; fdrop(fp, td); break; } flp = (struct flock *)arg; if (flp->l_type != F_RDLCK && flp->l_type != F_WRLCK && flp->l_type != F_UNLCK) { error = EINVAL; fdrop(fp, td); break; } if (flp->l_whence == SEEK_CUR) { foffset = foffset_get(fp); if ((flp->l_start > 0 && foffset > OFF_MAX - flp->l_start) || (flp->l_start < 0 && foffset < OFF_MIN - flp->l_start)) { error = EOVERFLOW; fdrop(fp, td); break; } flp->l_start += foffset; } vp = fp->f_vnode; error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_GETLK, flp, F_POSIX); fdrop(fp, td); break; case F_ADD_SEALS: error = fget_unlocked(fdp, fd, &cap_no_rights, &fp); if (error != 0) break; error = fo_add_seals(fp, arg); fdrop(fp, td); break; case F_GET_SEALS: error = fget_unlocked(fdp, fd, &cap_no_rights, &fp); if (error != 0) break; if (fo_get_seals(fp, &seals) == 0) td->td_retval[0] = seals; else error = EINVAL; fdrop(fp, td); break; case F_RDAHEAD: arg = arg ? 128 * 1024: 0; /* FALLTHROUGH */ case F_READAHEAD: error = fget_unlocked(fdp, fd, &cap_no_rights, &fp); if (error != 0) break; if (fp->f_type != DTYPE_VNODE) { fdrop(fp, td); error = EBADF; break; } vp = fp->f_vnode; if (vp->v_type != VREG) { fdrop(fp, td); error = ENOTTY; break; } /* * Exclusive lock synchronizes against f_seqcount reads and * writes in sequential_heuristic(). */ error = vn_lock(vp, LK_EXCLUSIVE); if (error != 0) { fdrop(fp, td); break; } if (arg >= 0) { bsize = fp->f_vnode->v_mount->mnt_stat.f_iosize; arg = MIN(arg, INT_MAX - bsize + 1); fp->f_seqcount = MIN(IO_SEQMAX, (arg + bsize - 1) / bsize); atomic_set_int(&fp->f_flag, FRDAHEAD); } else { atomic_clear_int(&fp->f_flag, FRDAHEAD); } VOP_UNLOCK(vp); fdrop(fp, td); break; case F_ISUNIONSTACK: /* * Check if the vnode is part of a union stack (either the * "union" flag from mount(2) or unionfs). * * Prior to introduction of this op libc's readdir would call * fstatfs(2), in effect unnecessarily copying kilobytes of * data just to check fs name and a mount flag. * * Fixing the code to handle everything in the kernel instead * is a non-trivial endeavor and has low priority, thus this * horrible kludge facilitates the current behavior in a much * cheaper manner until someone(tm) sorts this out. */ error = fget_unlocked(fdp, fd, &cap_no_rights, &fp); if (error != 0) break; if (fp->f_type != DTYPE_VNODE) { fdrop(fp, td); error = EBADF; break; } vp = fp->f_vnode; /* * Since we don't prevent dooming the vnode even non-null mp * found can become immediately stale. This is tolerable since * mount points are type-stable (providing safe memory access) * and any vfs op on this vnode going forward will return an * error (meaning return value in this case is meaningless). */ - mp = (struct mount *)atomic_load_ptr(&vp->v_mount); + mp = atomic_load_ptr(&vp->v_mount); if (__predict_false(mp == NULL)) { fdrop(fp, td); error = EBADF; break; } td->td_retval[0] = 0; if (mp->mnt_kern_flag & MNTK_UNIONFS || mp->mnt_flag & MNT_UNION) td->td_retval[0] = 1; fdrop(fp, td); break; default: error = EINVAL; break; } return (error); } static int getmaxfd(struct thread *td) { return (min((int)lim_cur(td, RLIMIT_NOFILE), maxfilesperproc)); } /* * Common code for dup, dup2, fcntl(F_DUPFD) and fcntl(F_DUP2FD). */ int kern_dup(struct thread *td, u_int mode, int flags, int old, int new) { struct filedesc *fdp; struct filedescent *oldfde, *newfde; struct proc *p; struct file *delfp; u_long *oioctls, *nioctls; int error, maxfd; p = td->td_proc; fdp = p->p_fd; oioctls = NULL; MPASS((flags & ~(FDDUP_FLAG_CLOEXEC)) == 0); MPASS(mode < FDDUP_LASTMODE); AUDIT_ARG_FD(old); /* XXXRW: if (flags & FDDUP_FIXED) AUDIT_ARG_FD2(new); */ /* * Verify we have a valid descriptor to dup from and possibly to * dup to. Unlike dup() and dup2(), fcntl()'s F_DUPFD should * return EINVAL when the new descriptor is out of bounds. */ if (old < 0) return (EBADF); if (new < 0) return (mode == FDDUP_FCNTL ? EINVAL : EBADF); maxfd = getmaxfd(td); if (new >= maxfd) return (mode == FDDUP_FCNTL ? EINVAL : EBADF); error = EBADF; FILEDESC_XLOCK(fdp); if (fget_locked(fdp, old) == NULL) goto unlock; if ((mode == FDDUP_FIXED || mode == FDDUP_MUSTREPLACE) && old == new) { td->td_retval[0] = new; if (flags & FDDUP_FLAG_CLOEXEC) fdp->fd_ofiles[new].fde_flags |= UF_EXCLOSE; error = 0; goto unlock; } oldfde = &fdp->fd_ofiles[old]; if (!fhold(oldfde->fde_file)) goto unlock; /* * If the caller specified a file descriptor, make sure the file * table is large enough to hold it, and grab it. Otherwise, just * allocate a new descriptor the usual way. */ switch (mode) { case FDDUP_NORMAL: case FDDUP_FCNTL: if ((error = fdalloc(td, new, &new)) != 0) { fdrop(oldfde->fde_file, td); goto unlock; } break; case FDDUP_MUSTREPLACE: /* Target file descriptor must exist. */ if (fget_locked(fdp, new) == NULL) { fdrop(oldfde->fde_file, td); goto unlock; } break; case FDDUP_FIXED: if (new >= fdp->fd_nfiles) { /* * The resource limits are here instead of e.g. * fdalloc(), because the file descriptor table may be * shared between processes, so we can't really use * racct_add()/racct_sub(). Instead of counting the * number of actually allocated descriptors, just put * the limit on the size of the file descriptor table. */ #ifdef RACCT if (RACCT_ENABLED()) { error = racct_set_unlocked(p, RACCT_NOFILE, new + 1); if (error != 0) { error = EMFILE; fdrop(oldfde->fde_file, td); goto unlock; } } #endif fdgrowtable_exp(fdp, new + 1); } if (!fdisused(fdp, new)) fdused(fdp, new); break; default: KASSERT(0, ("%s unsupported mode %d", __func__, mode)); } KASSERT(old != new, ("new fd is same as old")); newfde = &fdp->fd_ofiles[new]; delfp = newfde->fde_file; oioctls = filecaps_free_prep(&newfde->fde_caps); nioctls = filecaps_copy_prep(&oldfde->fde_caps); /* * Duplicate the source descriptor. */ #ifdef CAPABILITIES seqc_write_begin(&newfde->fde_seqc); #endif memcpy(newfde, oldfde, fde_change_size); filecaps_copy_finish(&oldfde->fde_caps, &newfde->fde_caps, nioctls); if ((flags & FDDUP_FLAG_CLOEXEC) != 0) newfde->fde_flags = oldfde->fde_flags | UF_EXCLOSE; else newfde->fde_flags = oldfde->fde_flags & ~UF_EXCLOSE; #ifdef CAPABILITIES seqc_write_end(&newfde->fde_seqc); #endif td->td_retval[0] = new; error = 0; if (delfp != NULL) { (void) closefp(fdp, new, delfp, td, 1); FILEDESC_UNLOCK_ASSERT(fdp); } else { unlock: FILEDESC_XUNLOCK(fdp); } filecaps_free_finish(oioctls); return (error); } /* * If sigio is on the list associated with a process or process group, * disable signalling from the device, remove sigio from the list and * free sigio. */ void funsetown(struct sigio **sigiop) { struct sigio *sigio; if (*sigiop == NULL) return; SIGIO_LOCK(); sigio = *sigiop; if (sigio == NULL) { SIGIO_UNLOCK(); return; } *(sigio->sio_myref) = NULL; if ((sigio)->sio_pgid < 0) { struct pgrp *pg = (sigio)->sio_pgrp; PGRP_LOCK(pg); SLIST_REMOVE(&sigio->sio_pgrp->pg_sigiolst, sigio, sigio, sio_pgsigio); PGRP_UNLOCK(pg); } else { struct proc *p = (sigio)->sio_proc; PROC_LOCK(p); SLIST_REMOVE(&sigio->sio_proc->p_sigiolst, sigio, sigio, sio_pgsigio); PROC_UNLOCK(p); } SIGIO_UNLOCK(); crfree(sigio->sio_ucred); free(sigio, M_SIGIO); } /* * Free a list of sigio structures. * We only need to lock the SIGIO_LOCK because we have made ourselves * inaccessible to callers of fsetown and therefore do not need to lock * the proc or pgrp struct for the list manipulation. */ void funsetownlst(struct sigiolst *sigiolst) { struct proc *p; struct pgrp *pg; struct sigio *sigio; sigio = SLIST_FIRST(sigiolst); if (sigio == NULL) return; p = NULL; pg = NULL; /* * Every entry of the list should belong * to a single proc or pgrp. */ if (sigio->sio_pgid < 0) { pg = sigio->sio_pgrp; PGRP_LOCK_ASSERT(pg, MA_NOTOWNED); } else /* if (sigio->sio_pgid > 0) */ { p = sigio->sio_proc; PROC_LOCK_ASSERT(p, MA_NOTOWNED); } SIGIO_LOCK(); while ((sigio = SLIST_FIRST(sigiolst)) != NULL) { *(sigio->sio_myref) = NULL; if (pg != NULL) { KASSERT(sigio->sio_pgid < 0, ("Proc sigio in pgrp sigio list")); KASSERT(sigio->sio_pgrp == pg, ("Bogus pgrp in sigio list")); PGRP_LOCK(pg); SLIST_REMOVE(&pg->pg_sigiolst, sigio, sigio, sio_pgsigio); PGRP_UNLOCK(pg); } else /* if (p != NULL) */ { KASSERT(sigio->sio_pgid > 0, ("Pgrp sigio in proc sigio list")); KASSERT(sigio->sio_proc == p, ("Bogus proc in sigio list")); PROC_LOCK(p); SLIST_REMOVE(&p->p_sigiolst, sigio, sigio, sio_pgsigio); PROC_UNLOCK(p); } SIGIO_UNLOCK(); crfree(sigio->sio_ucred); free(sigio, M_SIGIO); SIGIO_LOCK(); } SIGIO_UNLOCK(); } /* * This is common code for FIOSETOWN ioctl called by fcntl(fd, F_SETOWN, arg). * * After permission checking, add a sigio structure to the sigio list for * the process or process group. */ int fsetown(pid_t pgid, struct sigio **sigiop) { struct proc *proc; struct pgrp *pgrp; struct sigio *sigio; int ret; if (pgid == 0) { funsetown(sigiop); return (0); } ret = 0; /* Allocate and fill in the new sigio out of locks. */ sigio = malloc(sizeof(struct sigio), M_SIGIO, M_WAITOK); sigio->sio_pgid = pgid; sigio->sio_ucred = crhold(curthread->td_ucred); sigio->sio_myref = sigiop; sx_slock(&proctree_lock); if (pgid > 0) { proc = pfind(pgid); if (proc == NULL) { ret = ESRCH; goto fail; } /* * Policy - Don't allow a process to FSETOWN a process * in another session. * * Remove this test to allow maximum flexibility or * restrict FSETOWN to the current process or process * group for maximum safety. */ PROC_UNLOCK(proc); if (proc->p_session != curthread->td_proc->p_session) { ret = EPERM; goto fail; } pgrp = NULL; } else /* if (pgid < 0) */ { pgrp = pgfind(-pgid); if (pgrp == NULL) { ret = ESRCH; goto fail; } PGRP_UNLOCK(pgrp); /* * Policy - Don't allow a process to FSETOWN a process * in another session. * * Remove this test to allow maximum flexibility or * restrict FSETOWN to the current process or process * group for maximum safety. */ if (pgrp->pg_session != curthread->td_proc->p_session) { ret = EPERM; goto fail; } proc = NULL; } funsetown(sigiop); if (pgid > 0) { PROC_LOCK(proc); /* * Since funsetownlst() is called without the proctree * locked, we need to check for P_WEXIT. * XXX: is ESRCH correct? */ if ((proc->p_flag & P_WEXIT) != 0) { PROC_UNLOCK(proc); ret = ESRCH; goto fail; } SLIST_INSERT_HEAD(&proc->p_sigiolst, sigio, sio_pgsigio); sigio->sio_proc = proc; PROC_UNLOCK(proc); } else { PGRP_LOCK(pgrp); SLIST_INSERT_HEAD(&pgrp->pg_sigiolst, sigio, sio_pgsigio); sigio->sio_pgrp = pgrp; PGRP_UNLOCK(pgrp); } sx_sunlock(&proctree_lock); SIGIO_LOCK(); *sigiop = sigio; SIGIO_UNLOCK(); return (0); fail: sx_sunlock(&proctree_lock); crfree(sigio->sio_ucred); free(sigio, M_SIGIO); return (ret); } /* * This is common code for FIOGETOWN ioctl called by fcntl(fd, F_GETOWN, arg). */ pid_t fgetown(struct sigio **sigiop) { pid_t pgid; SIGIO_LOCK(); pgid = (*sigiop != NULL) ? (*sigiop)->sio_pgid : 0; SIGIO_UNLOCK(); return (pgid); } /* * Function drops the filedesc lock on return. */ static int closefp(struct filedesc *fdp, int fd, struct file *fp, struct thread *td, int holdleaders) { int error; FILEDESC_XLOCK_ASSERT(fdp); if (holdleaders) { if (td->td_proc->p_fdtol != NULL) { /* * Ask fdfree() to sleep to ensure that all relevant * process leaders can be traversed in closef(). */ fdp->fd_holdleaderscount++; } else { holdleaders = 0; } } /* * We now hold the fp reference that used to be owned by the * descriptor array. We have to unlock the FILEDESC *AFTER* * knote_fdclose to prevent a race of the fd getting opened, a knote * added, and deleteing a knote for the new fd. */ if (__predict_false(!TAILQ_EMPTY(&fdp->fd_kqlist))) knote_fdclose(td, fd); /* * We need to notify mqueue if the object is of type mqueue. */ if (__predict_false(fp->f_type == DTYPE_MQUEUE)) mq_fdclose(td, fd, fp); FILEDESC_XUNLOCK(fdp); error = closef(fp, td); if (holdleaders) { FILEDESC_XLOCK(fdp); fdp->fd_holdleaderscount--; if (fdp->fd_holdleaderscount == 0 && fdp->fd_holdleaderswakeup != 0) { fdp->fd_holdleaderswakeup = 0; wakeup(&fdp->fd_holdleaderscount); } FILEDESC_XUNLOCK(fdp); } return (error); } /* * Close a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct close_args { int fd; }; #endif /* ARGSUSED */ int sys_close(struct thread *td, struct close_args *uap) { return (kern_close(td, uap->fd)); } int kern_close(struct thread *td, int fd) { struct filedesc *fdp; struct file *fp; fdp = td->td_proc->p_fd; AUDIT_SYSCLOSE(td, fd); FILEDESC_XLOCK(fdp); if ((fp = fget_locked(fdp, fd)) == NULL) { FILEDESC_XUNLOCK(fdp); return (EBADF); } fdfree(fdp, fd); /* closefp() drops the FILEDESC lock for us. */ return (closefp(fdp, fd, fp, td, 1)); } /* * Close open file descriptors. */ #ifndef _SYS_SYSPROTO_H_ struct closefrom_args { int lowfd; }; #endif /* ARGSUSED */ int sys_closefrom(struct thread *td, struct closefrom_args *uap) { struct filedesc *fdp; int fd; fdp = td->td_proc->p_fd; AUDIT_ARG_FD(uap->lowfd); /* * Treat negative starting file descriptor values identical to * closefrom(0) which closes all files. */ if (uap->lowfd < 0) uap->lowfd = 0; FILEDESC_SLOCK(fdp); for (fd = uap->lowfd; fd <= fdp->fd_lastfile; fd++) { if (fdp->fd_ofiles[fd].fde_file != NULL) { FILEDESC_SUNLOCK(fdp); (void)kern_close(td, fd); FILEDESC_SLOCK(fdp); } } FILEDESC_SUNLOCK(fdp); return (0); } #if defined(COMPAT_43) /* * Return status information about a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct ofstat_args { int fd; struct ostat *sb; }; #endif /* ARGSUSED */ int ofstat(struct thread *td, struct ofstat_args *uap) { struct ostat oub; struct stat ub; int error; error = kern_fstat(td, uap->fd, &ub); if (error == 0) { cvtstat(&ub, &oub); error = copyout(&oub, uap->sb, sizeof(oub)); } return (error); } #endif /* COMPAT_43 */ #if defined(COMPAT_FREEBSD11) int freebsd11_fstat(struct thread *td, struct freebsd11_fstat_args *uap) { struct stat sb; struct freebsd11_stat osb; int error; error = kern_fstat(td, uap->fd, &sb); if (error != 0) return (error); error = freebsd11_cvtstat(&sb, &osb); if (error == 0) error = copyout(&osb, uap->sb, sizeof(osb)); return (error); } #endif /* COMPAT_FREEBSD11 */ /* * Return status information about a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct fstat_args { int fd; struct stat *sb; }; #endif /* ARGSUSED */ int sys_fstat(struct thread *td, struct fstat_args *uap) { struct stat ub; int error; error = kern_fstat(td, uap->fd, &ub); if (error == 0) error = copyout(&ub, uap->sb, sizeof(ub)); return (error); } int kern_fstat(struct thread *td, int fd, struct stat *sbp) { struct file *fp; int error; AUDIT_ARG_FD(fd); error = fget(td, fd, &cap_fstat_rights, &fp); if (__predict_false(error != 0)) return (error); AUDIT_ARG_FILE(td->td_proc, fp); error = fo_stat(fp, sbp, td->td_ucred, td); fdrop(fp, td); #ifdef __STAT_TIME_T_EXT sbp->st_atim_ext = 0; sbp->st_mtim_ext = 0; sbp->st_ctim_ext = 0; sbp->st_btim_ext = 0; #endif #ifdef KTRACE if (KTRPOINT(td, KTR_STRUCT)) ktrstat_error(sbp, error); #endif return (error); } #if defined(COMPAT_FREEBSD11) /* * Return status information about a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct freebsd11_nfstat_args { int fd; struct nstat *sb; }; #endif /* ARGSUSED */ int freebsd11_nfstat(struct thread *td, struct freebsd11_nfstat_args *uap) { struct nstat nub; struct stat ub; int error; error = kern_fstat(td, uap->fd, &ub); if (error == 0) { freebsd11_cvtnstat(&ub, &nub); error = copyout(&nub, uap->sb, sizeof(nub)); } return (error); } #endif /* COMPAT_FREEBSD11 */ /* * Return pathconf information about a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct fpathconf_args { int fd; int name; }; #endif /* ARGSUSED */ int sys_fpathconf(struct thread *td, struct fpathconf_args *uap) { long value; int error; error = kern_fpathconf(td, uap->fd, uap->name, &value); if (error == 0) td->td_retval[0] = value; return (error); } int kern_fpathconf(struct thread *td, int fd, int name, long *valuep) { struct file *fp; struct vnode *vp; int error; error = fget(td, fd, &cap_fpathconf_rights, &fp); if (error != 0) return (error); if (name == _PC_ASYNC_IO) { *valuep = _POSIX_ASYNCHRONOUS_IO; goto out; } vp = fp->f_vnode; if (vp != NULL) { vn_lock(vp, LK_SHARED | LK_RETRY); error = VOP_PATHCONF(vp, name, valuep); VOP_UNLOCK(vp); } else if (fp->f_type == DTYPE_PIPE || fp->f_type == DTYPE_SOCKET) { if (name != _PC_PIPE_BUF) { error = EINVAL; } else { *valuep = PIPE_BUF; error = 0; } } else { error = EOPNOTSUPP; } out: fdrop(fp, td); return (error); } /* * Copy filecaps structure allocating memory for ioctls array if needed. * * The last parameter indicates whether the fdtable is locked. If it is not and * ioctls are encountered, copying fails and the caller must lock the table. * * Note that if the table was not locked, the caller has to check the relevant * sequence counter to determine whether the operation was successful. */ bool filecaps_copy(const struct filecaps *src, struct filecaps *dst, bool locked) { size_t size; if (src->fc_ioctls != NULL && !locked) return (false); memcpy(dst, src, sizeof(*src)); if (src->fc_ioctls == NULL) return (true); KASSERT(src->fc_nioctls > 0, ("fc_ioctls != NULL, but fc_nioctls=%hd", src->fc_nioctls)); size = sizeof(src->fc_ioctls[0]) * src->fc_nioctls; dst->fc_ioctls = malloc(size, M_FILECAPS, M_WAITOK); memcpy(dst->fc_ioctls, src->fc_ioctls, size); return (true); } static u_long * filecaps_copy_prep(const struct filecaps *src) { u_long *ioctls; size_t size; if (__predict_true(src->fc_ioctls == NULL)) return (NULL); KASSERT(src->fc_nioctls > 0, ("fc_ioctls != NULL, but fc_nioctls=%hd", src->fc_nioctls)); size = sizeof(src->fc_ioctls[0]) * src->fc_nioctls; ioctls = malloc(size, M_FILECAPS, M_WAITOK); return (ioctls); } static void filecaps_copy_finish(const struct filecaps *src, struct filecaps *dst, u_long *ioctls) { size_t size; *dst = *src; if (__predict_true(src->fc_ioctls == NULL)) { MPASS(ioctls == NULL); return; } size = sizeof(src->fc_ioctls[0]) * src->fc_nioctls; dst->fc_ioctls = ioctls; bcopy(src->fc_ioctls, dst->fc_ioctls, size); } /* * Move filecaps structure to the new place and clear the old place. */ void filecaps_move(struct filecaps *src, struct filecaps *dst) { *dst = *src; bzero(src, sizeof(*src)); } /* * Fill the given filecaps structure with full rights. */ static void filecaps_fill(struct filecaps *fcaps) { CAP_ALL(&fcaps->fc_rights); fcaps->fc_ioctls = NULL; fcaps->fc_nioctls = -1; fcaps->fc_fcntls = CAP_FCNTL_ALL; } /* * Free memory allocated within filecaps structure. */ void filecaps_free(struct filecaps *fcaps) { free(fcaps->fc_ioctls, M_FILECAPS); bzero(fcaps, sizeof(*fcaps)); } static u_long * filecaps_free_prep(struct filecaps *fcaps) { u_long *ioctls; ioctls = fcaps->fc_ioctls; bzero(fcaps, sizeof(*fcaps)); return (ioctls); } static void filecaps_free_finish(u_long *ioctls) { free(ioctls, M_FILECAPS); } /* * Validate the given filecaps structure. */ static void filecaps_validate(const struct filecaps *fcaps, const char *func) { KASSERT(cap_rights_is_valid(&fcaps->fc_rights), ("%s: invalid rights", func)); KASSERT((fcaps->fc_fcntls & ~CAP_FCNTL_ALL) == 0, ("%s: invalid fcntls", func)); KASSERT(fcaps->fc_fcntls == 0 || cap_rights_is_set(&fcaps->fc_rights, CAP_FCNTL), ("%s: fcntls without CAP_FCNTL", func)); KASSERT(fcaps->fc_ioctls != NULL ? fcaps->fc_nioctls > 0 : (fcaps->fc_nioctls == -1 || fcaps->fc_nioctls == 0), ("%s: invalid ioctls", func)); KASSERT(fcaps->fc_nioctls == 0 || cap_rights_is_set(&fcaps->fc_rights, CAP_IOCTL), ("%s: ioctls without CAP_IOCTL", func)); } static void fdgrowtable_exp(struct filedesc *fdp, int nfd) { int nfd1; FILEDESC_XLOCK_ASSERT(fdp); nfd1 = fdp->fd_nfiles * 2; if (nfd1 < nfd) nfd1 = nfd; fdgrowtable(fdp, nfd1); } /* * Grow the file table to accommodate (at least) nfd descriptors. */ static void fdgrowtable(struct filedesc *fdp, int nfd) { struct filedesc0 *fdp0; struct freetable *ft; struct fdescenttbl *ntable; struct fdescenttbl *otable; int nnfiles, onfiles; NDSLOTTYPE *nmap, *omap; /* * If lastfile is -1 this struct filedesc was just allocated and we are * growing it to accommodate for the one we are going to copy from. There * is no need to have a lock on this one as it's not visible to anyone. */ if (fdp->fd_lastfile != -1) FILEDESC_XLOCK_ASSERT(fdp); KASSERT(fdp->fd_nfiles > 0, ("zero-length file table")); /* save old values */ onfiles = fdp->fd_nfiles; otable = fdp->fd_files; omap = fdp->fd_map; /* compute the size of the new table */ nnfiles = NDSLOTS(nfd) * NDENTRIES; /* round up */ if (nnfiles <= onfiles) /* the table is already large enough */ return; /* * Allocate a new table. We need enough space for the number of * entries, file entries themselves and the struct freetable we will use * when we decommission the table and place it on the freelist. * We place the struct freetable in the middle so we don't have * to worry about padding. */ ntable = malloc(offsetof(struct fdescenttbl, fdt_ofiles) + nnfiles * sizeof(ntable->fdt_ofiles[0]) + sizeof(struct freetable), M_FILEDESC, M_ZERO | M_WAITOK); /* copy the old data */ ntable->fdt_nfiles = nnfiles; memcpy(ntable->fdt_ofiles, otable->fdt_ofiles, onfiles * sizeof(ntable->fdt_ofiles[0])); /* * Allocate a new map only if the old is not large enough. It will * grow at a slower rate than the table as it can map more * entries than the table can hold. */ if (NDSLOTS(nnfiles) > NDSLOTS(onfiles)) { nmap = malloc(NDSLOTS(nnfiles) * NDSLOTSIZE, M_FILEDESC, M_ZERO | M_WAITOK); /* copy over the old data and update the pointer */ memcpy(nmap, omap, NDSLOTS(onfiles) * sizeof(*omap)); fdp->fd_map = nmap; } /* * Make sure that ntable is correctly initialized before we replace * fd_files poiner. Otherwise fget_unlocked() may see inconsistent * data. */ atomic_store_rel_ptr((volatile void *)&fdp->fd_files, (uintptr_t)ntable); /* * Do not free the old file table, as some threads may still * reference entries within it. Instead, place it on a freelist * which will be processed when the struct filedesc is released. * * Note that if onfiles == NDFILE, we're dealing with the original * static allocation contained within (struct filedesc0 *)fdp, * which must not be freed. */ if (onfiles > NDFILE) { ft = (struct freetable *)&otable->fdt_ofiles[onfiles]; fdp0 = (struct filedesc0 *)fdp; ft->ft_table = otable; SLIST_INSERT_HEAD(&fdp0->fd_free, ft, ft_next); } /* * The map does not have the same possibility of threads still * holding references to it. So always free it as long as it * does not reference the original static allocation. */ if (NDSLOTS(onfiles) > NDSLOTS(NDFILE)) free(omap, M_FILEDESC); } /* * Allocate a file descriptor for the process. */ int fdalloc(struct thread *td, int minfd, int *result) { struct proc *p = td->td_proc; struct filedesc *fdp = p->p_fd; int fd, maxfd, allocfd; #ifdef RACCT int error; #endif FILEDESC_XLOCK_ASSERT(fdp); if (fdp->fd_freefile > minfd) minfd = fdp->fd_freefile; maxfd = getmaxfd(td); /* * Search the bitmap for a free descriptor starting at minfd. * If none is found, grow the file table. */ fd = fd_first_free(fdp, minfd, fdp->fd_nfiles); if (fd >= maxfd) return (EMFILE); if (fd >= fdp->fd_nfiles) { allocfd = min(fd * 2, maxfd); #ifdef RACCT if (RACCT_ENABLED()) { error = racct_set_unlocked(p, RACCT_NOFILE, allocfd); if (error != 0) return (EMFILE); } #endif /* * fd is already equal to first free descriptor >= minfd, so * we only need to grow the table and we are done. */ fdgrowtable_exp(fdp, allocfd); } /* * Perform some sanity checks, then mark the file descriptor as * used and return it to the caller. */ KASSERT(fd >= 0 && fd < min(maxfd, fdp->fd_nfiles), ("invalid descriptor %d", fd)); KASSERT(!fdisused(fdp, fd), ("fd_first_free() returned non-free descriptor")); KASSERT(fdp->fd_ofiles[fd].fde_file == NULL, ("file descriptor isn't free")); fdused(fdp, fd); *result = fd; return (0); } /* * Allocate n file descriptors for the process. */ int fdallocn(struct thread *td, int minfd, int *fds, int n) { struct proc *p = td->td_proc; struct filedesc *fdp = p->p_fd; int i; FILEDESC_XLOCK_ASSERT(fdp); for (i = 0; i < n; i++) if (fdalloc(td, 0, &fds[i]) != 0) break; if (i < n) { for (i--; i >= 0; i--) fdunused(fdp, fds[i]); return (EMFILE); } return (0); } /* * Create a new open file structure and allocate a file descriptor for the * process that refers to it. We add one reference to the file for the * descriptor table and one reference for resultfp. This is to prevent us * being preempted and the entry in the descriptor table closed after we * release the FILEDESC lock. */ int falloc_caps(struct thread *td, struct file **resultfp, int *resultfd, int flags, struct filecaps *fcaps) { struct file *fp; int error, fd; error = falloc_noinstall(td, &fp); if (error) return (error); /* no reference held on error */ error = finstall(td, fp, &fd, flags, fcaps); if (error) { fdrop(fp, td); /* one reference (fp only) */ return (error); } if (resultfp != NULL) *resultfp = fp; /* copy out result */ else fdrop(fp, td); /* release local reference */ if (resultfd != NULL) *resultfd = fd; return (0); } /* * Create a new open file structure without allocating a file descriptor. */ int falloc_noinstall(struct thread *td, struct file **resultfp) { struct file *fp; int maxuserfiles = maxfiles - (maxfiles / 20); int openfiles_new; static struct timeval lastfail; static int curfail; KASSERT(resultfp != NULL, ("%s: resultfp == NULL", __func__)); openfiles_new = atomic_fetchadd_int(&openfiles, 1) + 1; if ((openfiles_new >= maxuserfiles && priv_check(td, PRIV_MAXFILES) != 0) || openfiles_new >= maxfiles) { atomic_subtract_int(&openfiles, 1); if (ppsratecheck(&lastfail, &curfail, 1)) { printf("kern.maxfiles limit exceeded by uid %i, (%s) " "please see tuning(7).\n", td->td_ucred->cr_ruid, td->td_proc->p_comm); } return (ENFILE); } fp = uma_zalloc(file_zone, M_WAITOK); bzero(fp, sizeof(*fp)); refcount_init(&fp->f_count, 1); fp->f_cred = crhold(td->td_ucred); fp->f_ops = &badfileops; *resultfp = fp; return (0); } /* * Install a file in a file descriptor table. */ void _finstall(struct filedesc *fdp, struct file *fp, int fd, int flags, struct filecaps *fcaps) { struct filedescent *fde; MPASS(fp != NULL); if (fcaps != NULL) filecaps_validate(fcaps, __func__); FILEDESC_XLOCK_ASSERT(fdp); fde = &fdp->fd_ofiles[fd]; #ifdef CAPABILITIES seqc_write_begin(&fde->fde_seqc); #endif fde->fde_file = fp; fde->fde_flags = (flags & O_CLOEXEC) != 0 ? UF_EXCLOSE : 0; if (fcaps != NULL) filecaps_move(fcaps, &fde->fde_caps); else filecaps_fill(&fde->fde_caps); #ifdef CAPABILITIES seqc_write_end(&fde->fde_seqc); #endif } int finstall(struct thread *td, struct file *fp, int *fd, int flags, struct filecaps *fcaps) { struct filedesc *fdp = td->td_proc->p_fd; int error; MPASS(fd != NULL); if (!fhold(fp)) return (EBADF); FILEDESC_XLOCK(fdp); error = fdalloc(td, 0, fd); if (__predict_false(error != 0)) { FILEDESC_XUNLOCK(fdp); fdrop(fp, td); return (error); } _finstall(fdp, fp, *fd, flags, fcaps); FILEDESC_XUNLOCK(fdp); return (0); } /* * Build a new filedesc structure from another. * Copy the current, root, and jail root vnode references. * * If fdp is not NULL, return with it shared locked. */ struct filedesc * fdinit(struct filedesc *fdp, bool prepfiles) { struct filedesc0 *newfdp0; struct filedesc *newfdp; newfdp0 = uma_zalloc(filedesc0_zone, M_WAITOK | M_ZERO); newfdp = &newfdp0->fd_fd; /* Create the file descriptor table. */ FILEDESC_LOCK_INIT(newfdp); refcount_init(&newfdp->fd_refcnt, 1); refcount_init(&newfdp->fd_holdcnt, 1); newfdp->fd_cmask = CMASK; newfdp->fd_map = newfdp0->fd_dmap; newfdp->fd_lastfile = -1; newfdp->fd_files = (struct fdescenttbl *)&newfdp0->fd_dfiles; newfdp->fd_files->fdt_nfiles = NDFILE; if (fdp == NULL) return (newfdp); if (prepfiles && fdp->fd_lastfile >= newfdp->fd_nfiles) fdgrowtable(newfdp, fdp->fd_lastfile + 1); FILEDESC_SLOCK(fdp); newfdp->fd_cdir = fdp->fd_cdir; if (newfdp->fd_cdir) vrefact(newfdp->fd_cdir); newfdp->fd_rdir = fdp->fd_rdir; if (newfdp->fd_rdir) vrefact(newfdp->fd_rdir); newfdp->fd_jdir = fdp->fd_jdir; if (newfdp->fd_jdir) vrefact(newfdp->fd_jdir); if (!prepfiles) { FILEDESC_SUNLOCK(fdp); } else { while (fdp->fd_lastfile >= newfdp->fd_nfiles) { FILEDESC_SUNLOCK(fdp); fdgrowtable(newfdp, fdp->fd_lastfile + 1); FILEDESC_SLOCK(fdp); } } return (newfdp); } static struct filedesc * fdhold(struct proc *p) { struct filedesc *fdp; PROC_LOCK_ASSERT(p, MA_OWNED); fdp = p->p_fd; if (fdp != NULL) refcount_acquire(&fdp->fd_holdcnt); return (fdp); } static void fddrop(struct filedesc *fdp) { if (fdp->fd_holdcnt > 1) { if (refcount_release(&fdp->fd_holdcnt) == 0) return; } FILEDESC_LOCK_DESTROY(fdp); uma_zfree(filedesc0_zone, fdp); } /* * Share a filedesc structure. */ struct filedesc * fdshare(struct filedesc *fdp) { refcount_acquire(&fdp->fd_refcnt); return (fdp); } /* * Unshare a filedesc structure, if necessary by making a copy */ void fdunshare(struct thread *td) { struct filedesc *tmp; struct proc *p = td->td_proc; if (p->p_fd->fd_refcnt == 1) return; tmp = fdcopy(p->p_fd); fdescfree(td); p->p_fd = tmp; } void fdinstall_remapped(struct thread *td, struct filedesc *fdp) { fdescfree(td); td->td_proc->p_fd = fdp; } /* * Copy a filedesc structure. A NULL pointer in returns a NULL reference, * this is to ease callers, not catch errors. */ struct filedesc * fdcopy(struct filedesc *fdp) { struct filedesc *newfdp; struct filedescent *nfde, *ofde; int i; MPASS(fdp != NULL); newfdp = fdinit(fdp, true); /* copy all passable descriptors (i.e. not kqueue) */ newfdp->fd_freefile = -1; for (i = 0; i <= fdp->fd_lastfile; ++i) { ofde = &fdp->fd_ofiles[i]; if (ofde->fde_file == NULL || (ofde->fde_file->f_ops->fo_flags & DFLAG_PASSABLE) == 0 || !fhold(ofde->fde_file)) { if (newfdp->fd_freefile == -1) newfdp->fd_freefile = i; continue; } nfde = &newfdp->fd_ofiles[i]; *nfde = *ofde; filecaps_copy(&ofde->fde_caps, &nfde->fde_caps, true); fdused_init(newfdp, i); newfdp->fd_lastfile = i; } if (newfdp->fd_freefile == -1) newfdp->fd_freefile = i; newfdp->fd_cmask = fdp->fd_cmask; FILEDESC_SUNLOCK(fdp); return (newfdp); } /* * Copies a filedesc structure, while remapping all file descriptors * stored inside using a translation table. * * File descriptors are copied over to the new file descriptor table, * regardless of whether the close-on-exec flag is set. */ int fdcopy_remapped(struct filedesc *fdp, const int *fds, size_t nfds, struct filedesc **ret) { struct filedesc *newfdp; struct filedescent *nfde, *ofde; int error, i; MPASS(fdp != NULL); newfdp = fdinit(fdp, true); if (nfds > fdp->fd_lastfile + 1) { /* New table cannot be larger than the old one. */ error = E2BIG; goto bad; } /* Copy all passable descriptors (i.e. not kqueue). */ newfdp->fd_freefile = nfds; for (i = 0; i < nfds; ++i) { if (fds[i] < 0 || fds[i] > fdp->fd_lastfile) { /* File descriptor out of bounds. */ error = EBADF; goto bad; } ofde = &fdp->fd_ofiles[fds[i]]; if (ofde->fde_file == NULL) { /* Unused file descriptor. */ error = EBADF; goto bad; } if ((ofde->fde_file->f_ops->fo_flags & DFLAG_PASSABLE) == 0) { /* File descriptor cannot be passed. */ error = EINVAL; goto bad; } if (!fhold(nfde->fde_file)) { error = EBADF; goto bad; } nfde = &newfdp->fd_ofiles[i]; *nfde = *ofde; filecaps_copy(&ofde->fde_caps, &nfde->fde_caps, true); fdused_init(newfdp, i); newfdp->fd_lastfile = i; } newfdp->fd_cmask = fdp->fd_cmask; FILEDESC_SUNLOCK(fdp); *ret = newfdp; return (0); bad: FILEDESC_SUNLOCK(fdp); fdescfree_remapped(newfdp); return (error); } /* * Clear POSIX style locks. This is only used when fdp looses a reference (i.e. * one of processes using it exits) and the table used to be shared. */ static void fdclearlocks(struct thread *td) { struct filedesc *fdp; struct filedesc_to_leader *fdtol; struct flock lf; struct file *fp; struct proc *p; struct vnode *vp; int i; p = td->td_proc; fdp = p->p_fd; fdtol = p->p_fdtol; MPASS(fdtol != NULL); FILEDESC_XLOCK(fdp); KASSERT(fdtol->fdl_refcount > 0, ("filedesc_to_refcount botch: fdl_refcount=%d", fdtol->fdl_refcount)); if (fdtol->fdl_refcount == 1 && (p->p_leader->p_flag & P_ADVLOCK) != 0) { for (i = 0; i <= fdp->fd_lastfile; i++) { fp = fdp->fd_ofiles[i].fde_file; if (fp == NULL || fp->f_type != DTYPE_VNODE || !fhold(fp)) continue; FILEDESC_XUNLOCK(fdp); lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; lf.l_type = F_UNLCK; vp = fp->f_vnode; (void) VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_UNLCK, &lf, F_POSIX); FILEDESC_XLOCK(fdp); fdrop(fp, td); } } retry: if (fdtol->fdl_refcount == 1) { if (fdp->fd_holdleaderscount > 0 && (p->p_leader->p_flag & P_ADVLOCK) != 0) { /* * close() or kern_dup() has cleared a reference * in a shared file descriptor table. */ fdp->fd_holdleaderswakeup = 1; sx_sleep(&fdp->fd_holdleaderscount, FILEDESC_LOCK(fdp), PLOCK, "fdlhold", 0); goto retry; } if (fdtol->fdl_holdcount > 0) { /* * Ensure that fdtol->fdl_leader remains * valid in closef(). */ fdtol->fdl_wakeup = 1; sx_sleep(fdtol, FILEDESC_LOCK(fdp), PLOCK, "fdlhold", 0); goto retry; } } fdtol->fdl_refcount--; if (fdtol->fdl_refcount == 0 && fdtol->fdl_holdcount == 0) { fdtol->fdl_next->fdl_prev = fdtol->fdl_prev; fdtol->fdl_prev->fdl_next = fdtol->fdl_next; } else fdtol = NULL; p->p_fdtol = NULL; FILEDESC_XUNLOCK(fdp); if (fdtol != NULL) free(fdtol, M_FILEDESC_TO_LEADER); } /* * Release a filedesc structure. */ static void fdescfree_fds(struct thread *td, struct filedesc *fdp, bool needclose) { struct filedesc0 *fdp0; struct freetable *ft, *tft; struct filedescent *fde; struct file *fp; int i; for (i = 0; i <= fdp->fd_lastfile; i++) { fde = &fdp->fd_ofiles[i]; fp = fde->fde_file; if (fp != NULL) { fdefree_last(fde); if (needclose) (void) closef(fp, td); else fdrop(fp, td); } } if (NDSLOTS(fdp->fd_nfiles) > NDSLOTS(NDFILE)) free(fdp->fd_map, M_FILEDESC); if (fdp->fd_nfiles > NDFILE) free(fdp->fd_files, M_FILEDESC); fdp0 = (struct filedesc0 *)fdp; SLIST_FOREACH_SAFE(ft, &fdp0->fd_free, ft_next, tft) free(ft->ft_table, M_FILEDESC); fddrop(fdp); } void fdescfree(struct thread *td) { struct proc *p; struct filedesc *fdp; struct vnode *cdir, *jdir, *rdir; p = td->td_proc; fdp = p->p_fd; MPASS(fdp != NULL); #ifdef RACCT if (RACCT_ENABLED()) racct_set_unlocked(p, RACCT_NOFILE, 0); #endif if (p->p_fdtol != NULL) fdclearlocks(td); PROC_LOCK(p); p->p_fd = NULL; PROC_UNLOCK(p); if (refcount_release(&fdp->fd_refcnt) == 0) return; FILEDESC_XLOCK(fdp); cdir = fdp->fd_cdir; fdp->fd_cdir = NULL; rdir = fdp->fd_rdir; fdp->fd_rdir = NULL; jdir = fdp->fd_jdir; fdp->fd_jdir = NULL; FILEDESC_XUNLOCK(fdp); if (cdir != NULL) vrele(cdir); if (rdir != NULL) vrele(rdir); if (jdir != NULL) vrele(jdir); fdescfree_fds(td, fdp, 1); } void fdescfree_remapped(struct filedesc *fdp) { if (fdp->fd_cdir != NULL) vrele(fdp->fd_cdir); if (fdp->fd_rdir != NULL) vrele(fdp->fd_rdir); if (fdp->fd_jdir != NULL) vrele(fdp->fd_jdir); fdescfree_fds(curthread, fdp, 0); } /* * For setugid programs, we don't want to people to use that setugidness * to generate error messages which write to a file which otherwise would * otherwise be off-limits to the process. We check for filesystems where * the vnode can change out from under us after execve (like [lin]procfs). * * Since fdsetugidsafety calls this only for fd 0, 1 and 2, this check is * sufficient. We also don't check for setugidness since we know we are. */ static bool is_unsafe(struct file *fp) { struct vnode *vp; if (fp->f_type != DTYPE_VNODE) return (false); vp = fp->f_vnode; return ((vp->v_vflag & VV_PROCDEP) != 0); } /* * Make this setguid thing safe, if at all possible. */ void fdsetugidsafety(struct thread *td) { struct filedesc *fdp; struct file *fp; int i; fdp = td->td_proc->p_fd; KASSERT(fdp->fd_refcnt == 1, ("the fdtable should not be shared")); MPASS(fdp->fd_nfiles >= 3); for (i = 0; i <= 2; i++) { fp = fdp->fd_ofiles[i].fde_file; if (fp != NULL && is_unsafe(fp)) { FILEDESC_XLOCK(fdp); knote_fdclose(td, i); /* * NULL-out descriptor prior to close to avoid * a race while close blocks. */ fdfree(fdp, i); FILEDESC_XUNLOCK(fdp); (void) closef(fp, td); } } } /* * If a specific file object occupies a specific file descriptor, close the * file descriptor entry and drop a reference on the file object. This is a * convenience function to handle a subsequent error in a function that calls * falloc() that handles the race that another thread might have closed the * file descriptor out from under the thread creating the file object. */ void fdclose(struct thread *td, struct file *fp, int idx) { struct filedesc *fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); if (fdp->fd_ofiles[idx].fde_file == fp) { fdfree(fdp, idx); FILEDESC_XUNLOCK(fdp); fdrop(fp, td); } else FILEDESC_XUNLOCK(fdp); } /* * Close any files on exec? */ void fdcloseexec(struct thread *td) { struct filedesc *fdp; struct filedescent *fde; struct file *fp; int i; fdp = td->td_proc->p_fd; KASSERT(fdp->fd_refcnt == 1, ("the fdtable should not be shared")); for (i = 0; i <= fdp->fd_lastfile; i++) { fde = &fdp->fd_ofiles[i]; fp = fde->fde_file; if (fp != NULL && (fp->f_type == DTYPE_MQUEUE || (fde->fde_flags & UF_EXCLOSE))) { FILEDESC_XLOCK(fdp); fdfree(fdp, i); (void) closefp(fdp, i, fp, td, 0); FILEDESC_UNLOCK_ASSERT(fdp); } } } /* * It is unsafe for set[ug]id processes to be started with file * descriptors 0..2 closed, as these descriptors are given implicit * significance in the Standard C library. fdcheckstd() will create a * descriptor referencing /dev/null for each of stdin, stdout, and * stderr that is not already open. */ int fdcheckstd(struct thread *td) { struct filedesc *fdp; register_t save; int i, error, devnull; fdp = td->td_proc->p_fd; KASSERT(fdp->fd_refcnt == 1, ("the fdtable should not be shared")); MPASS(fdp->fd_nfiles >= 3); devnull = -1; for (i = 0; i <= 2; i++) { if (fdp->fd_ofiles[i].fde_file != NULL) continue; save = td->td_retval[0]; if (devnull != -1) { error = kern_dup(td, FDDUP_FIXED, 0, devnull, i); } else { error = kern_openat(td, AT_FDCWD, "/dev/null", UIO_SYSSPACE, O_RDWR, 0); if (error == 0) { devnull = td->td_retval[0]; KASSERT(devnull == i, ("we didn't get our fd")); } } td->td_retval[0] = save; if (error != 0) return (error); } return (0); } /* * Internal form of close. Decrement reference count on file structure. * Note: td may be NULL when closing a file that was being passed in a * message. */ int closef(struct file *fp, struct thread *td) { struct vnode *vp; struct flock lf; struct filedesc_to_leader *fdtol; struct filedesc *fdp; /* * POSIX record locking dictates that any close releases ALL * locks owned by this process. This is handled by setting * a flag in the unlock to free ONLY locks obeying POSIX * semantics, and not to free BSD-style file locks. * If the descriptor was in a message, POSIX-style locks * aren't passed with the descriptor, and the thread pointer * will be NULL. Callers should be careful only to pass a * NULL thread pointer when there really is no owning * context that might have locks, or the locks will be * leaked. */ if (fp->f_type == DTYPE_VNODE && td != NULL) { vp = fp->f_vnode; if ((td->td_proc->p_leader->p_flag & P_ADVLOCK) != 0) { lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; lf.l_type = F_UNLCK; (void) VOP_ADVLOCK(vp, (caddr_t)td->td_proc->p_leader, F_UNLCK, &lf, F_POSIX); } fdtol = td->td_proc->p_fdtol; if (fdtol != NULL) { /* * Handle special case where file descriptor table is * shared between multiple process leaders. */ fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); for (fdtol = fdtol->fdl_next; fdtol != td->td_proc->p_fdtol; fdtol = fdtol->fdl_next) { if ((fdtol->fdl_leader->p_flag & P_ADVLOCK) == 0) continue; fdtol->fdl_holdcount++; FILEDESC_XUNLOCK(fdp); lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; lf.l_type = F_UNLCK; vp = fp->f_vnode; (void) VOP_ADVLOCK(vp, (caddr_t)fdtol->fdl_leader, F_UNLCK, &lf, F_POSIX); FILEDESC_XLOCK(fdp); fdtol->fdl_holdcount--; if (fdtol->fdl_holdcount == 0 && fdtol->fdl_wakeup != 0) { fdtol->fdl_wakeup = 0; wakeup(fdtol); } } FILEDESC_XUNLOCK(fdp); } } return (fdrop(fp, td)); } /* * Initialize the file pointer with the specified properties. * * The ops are set with release semantics to be certain that the flags, type, * and data are visible when ops is. This is to prevent ops methods from being * called with bad data. */ void finit(struct file *fp, u_int flag, short type, void *data, struct fileops *ops) { fp->f_data = data; fp->f_flag = flag; fp->f_type = type; atomic_store_rel_ptr((volatile uintptr_t *)&fp->f_ops, (uintptr_t)ops); } int fget_cap_locked(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, struct file **fpp, struct filecaps *havecapsp) { struct filedescent *fde; int error; FILEDESC_LOCK_ASSERT(fdp); fde = fdeget_locked(fdp, fd); if (fde == NULL) { error = EBADF; goto out; } #ifdef CAPABILITIES error = cap_check(cap_rights_fde_inline(fde), needrightsp); if (error != 0) goto out; #endif if (havecapsp != NULL) filecaps_copy(&fde->fde_caps, havecapsp, true); *fpp = fde->fde_file; error = 0; out: return (error); } int fget_cap(struct thread *td, int fd, cap_rights_t *needrightsp, struct file **fpp, struct filecaps *havecapsp) { struct filedesc *fdp = td->td_proc->p_fd; int error; #ifndef CAPABILITIES error = fget_unlocked(fdp, fd, needrightsp, fpp); if (havecapsp != NULL && error == 0) filecaps_fill(havecapsp); #else struct file *fp; seqc_t seq; *fpp = NULL; for (;;) { error = fget_unlocked_seq(fdp, fd, needrightsp, &fp, &seq); if (error != 0) return (error); if (havecapsp != NULL) { if (!filecaps_copy(&fdp->fd_ofiles[fd].fde_caps, havecapsp, false)) { fdrop(fp, td); goto get_locked; } } if (!fd_modified(fdp, fd, seq)) break; fdrop(fp, td); } *fpp = fp; return (0); get_locked: FILEDESC_SLOCK(fdp); error = fget_cap_locked(fdp, fd, needrightsp, fpp, havecapsp); if (error == 0 && !fhold(*fpp)) error = EBADF; FILEDESC_SUNLOCK(fdp); #endif return (error); } int fget_unlocked_seq(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, struct file **fpp, seqc_t *seqp) { #ifdef CAPABILITIES const struct filedescent *fde; #endif const struct fdescenttbl *fdt; struct file *fp; #ifdef CAPABILITIES seqc_t seq; cap_rights_t haverights; int error; #endif fdt = fdp->fd_files; if (__predict_false((u_int)fd >= fdt->fdt_nfiles)) return (EBADF); /* * Fetch the descriptor locklessly. We avoid fdrop() races by * never raising a refcount above 0. To accomplish this we have * to use a cmpset loop rather than an atomic_add. The descriptor * must be re-verified once we acquire a reference to be certain * that the identity is still correct and we did not lose a race * due to preemption. */ for (;;) { #ifdef CAPABILITIES seq = seqc_read(fd_seqc(fdt, fd)); fde = &fdt->fdt_ofiles[fd]; haverights = *cap_rights_fde_inline(fde); fp = fde->fde_file; if (!seqc_consistent(fd_seqc(fdt, fd), seq)) continue; #else fp = fdt->fdt_ofiles[fd].fde_file; #endif if (fp == NULL) return (EBADF); #ifdef CAPABILITIES error = cap_check_inline(&haverights, needrightsp); if (error != 0) return (error); #endif if (__predict_false(!refcount_acquire_if_not_zero(&fp->f_count))) { /* * The count was found either saturated or zero. * This re-read is not any more racy than using the * return value from fcmpset. */ if (fp->f_count != 0) return (EBADF); /* * Force a reload. Other thread could reallocate the * table before this fd was closed, so it is possible * that there is a stale fp pointer in cached version. */ - fdt = (struct fdescenttbl *)atomic_load_ptr(&fdp->fd_files); + fdt = atomic_load_ptr(&fdp->fd_files); continue; } /* * Use an acquire barrier to force re-reading of fdt so it is * refreshed for verification. */ atomic_thread_fence_acq(); fdt = fdp->fd_files; #ifdef CAPABILITIES if (seqc_consistent_nomb(fd_seqc(fdt, fd), seq)) #else if (fp == fdt->fdt_ofiles[fd].fde_file) #endif break; fdrop(fp, curthread); } *fpp = fp; if (seqp != NULL) { #ifdef CAPABILITIES *seqp = seq; #endif } return (0); } /* * See the comments in fget_unlocked_seq for an explanation of how this works. * * This is a simplified variant which bails out to the aforementioned routine * if anything goes wrong. In practice this only happens when userspace is * racing with itself. */ int fget_unlocked(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, struct file **fpp) { #ifdef CAPABILITIES const struct filedescent *fde; #endif const struct fdescenttbl *fdt; struct file *fp; #ifdef CAPABILITIES seqc_t seq; const cap_rights_t *haverights; #endif fdt = fdp->fd_files; if (__predict_false((u_int)fd >= fdt->fdt_nfiles)) return (EBADF); #ifdef CAPABILITIES seq = seqc_read_any(fd_seqc(fdt, fd)); if (__predict_false(seqc_in_modify(seq))) goto out_fallback; fde = &fdt->fdt_ofiles[fd]; haverights = cap_rights_fde_inline(fde); fp = fde->fde_file; #else fp = fdt->fdt_ofiles[fd].fde_file; #endif if (__predict_false(fp == NULL)) goto out_fallback; #ifdef CAPABILITIES if (__predict_false(cap_check_inline_transient(haverights, needrightsp))) goto out_fallback; #endif if (__predict_false(!refcount_acquire_if_not_zero(&fp->f_count))) goto out_fallback; /* * Use an acquire barrier to force re-reading of fdt so it is * refreshed for verification. */ atomic_thread_fence_acq(); fdt = fdp->fd_files; #ifdef CAPABILITIES if (__predict_false(!seqc_consistent_nomb(fd_seqc(fdt, fd), seq))) #else if (__predict_false(fp != fdt->fdt_ofiles[fd].fde_file)) #endif goto out_fdrop; *fpp = fp; return (0); out_fdrop: fdrop(fp, curthread); out_fallback: return (fget_unlocked_seq(fdp, fd, needrightsp, fpp, NULL)); } /* * Extract the file pointer associated with the specified descriptor for the * current user process. * * If the descriptor doesn't exist or doesn't match 'flags', EBADF is * returned. * * File's rights will be checked against the capability rights mask. * * If an error occurred the non-zero error is returned and *fpp is set to * NULL. Otherwise *fpp is held and set and zero is returned. Caller is * responsible for fdrop(). */ static __inline int _fget(struct thread *td, int fd, struct file **fpp, int flags, cap_rights_t *needrightsp) { struct filedesc *fdp; struct file *fp; int error; *fpp = NULL; fdp = td->td_proc->p_fd; error = fget_unlocked(fdp, fd, needrightsp, &fp); if (__predict_false(error != 0)) return (error); if (__predict_false(fp->f_ops == &badfileops)) { fdrop(fp, td); return (EBADF); } /* * FREAD and FWRITE failure return EBADF as per POSIX. */ error = 0; switch (flags) { case FREAD: case FWRITE: if ((fp->f_flag & flags) == 0) error = EBADF; break; case FEXEC: if ((fp->f_flag & (FREAD | FEXEC)) == 0 || ((fp->f_flag & FWRITE) != 0)) error = EBADF; break; case 0: break; default: KASSERT(0, ("wrong flags")); } if (error != 0) { fdrop(fp, td); return (error); } *fpp = fp; return (0); } int fget(struct thread *td, int fd, cap_rights_t *rightsp, struct file **fpp) { return (_fget(td, fd, fpp, 0, rightsp)); } int fget_mmap(struct thread *td, int fd, cap_rights_t *rightsp, vm_prot_t *maxprotp, struct file **fpp) { int error; #ifndef CAPABILITIES error = _fget(td, fd, fpp, 0, rightsp); if (maxprotp != NULL) *maxprotp = VM_PROT_ALL; return (error); #else cap_rights_t fdrights; struct filedesc *fdp; struct file *fp; seqc_t seq; *fpp = NULL; fdp = td->td_proc->p_fd; MPASS(cap_rights_is_set(rightsp, CAP_MMAP)); for (;;) { error = fget_unlocked_seq(fdp, fd, rightsp, &fp, &seq); if (__predict_false(error != 0)) return (error); if (__predict_false(fp->f_ops == &badfileops)) { fdrop(fp, td); return (EBADF); } if (maxprotp != NULL) fdrights = *cap_rights(fdp, fd); if (!fd_modified(fdp, fd, seq)) break; fdrop(fp, td); } /* * If requested, convert capability rights to access flags. */ if (maxprotp != NULL) *maxprotp = cap_rights_to_vmprot(&fdrights); *fpp = fp; return (0); #endif } int fget_read(struct thread *td, int fd, cap_rights_t *rightsp, struct file **fpp) { return (_fget(td, fd, fpp, FREAD, rightsp)); } int fget_write(struct thread *td, int fd, cap_rights_t *rightsp, struct file **fpp) { return (_fget(td, fd, fpp, FWRITE, rightsp)); } int fget_fcntl(struct thread *td, int fd, cap_rights_t *rightsp, int needfcntl, struct file **fpp) { struct filedesc *fdp = td->td_proc->p_fd; #ifndef CAPABILITIES return (fget_unlocked(fdp, fd, rightsp, fpp)); #else struct file *fp; int error; seqc_t seq; *fpp = NULL; MPASS(cap_rights_is_set(rightsp, CAP_FCNTL)); for (;;) { error = fget_unlocked_seq(fdp, fd, rightsp, &fp, &seq); if (error != 0) return (error); error = cap_fcntl_check(fdp, fd, needfcntl); if (!fd_modified(fdp, fd, seq)) break; fdrop(fp, td); } if (error != 0) { fdrop(fp, td); return (error); } *fpp = fp; return (0); #endif } /* * Like fget() but loads the underlying vnode, or returns an error if the * descriptor does not represent a vnode. Note that pipes use vnodes but * never have VM objects. The returned vnode will be vref()'d. * * XXX: what about the unused flags ? */ static __inline int _fgetvp(struct thread *td, int fd, int flags, cap_rights_t *needrightsp, struct vnode **vpp) { struct file *fp; int error; *vpp = NULL; error = _fget(td, fd, &fp, flags, needrightsp); if (error != 0) return (error); if (fp->f_vnode == NULL) { error = EINVAL; } else { *vpp = fp->f_vnode; vrefact(*vpp); } fdrop(fp, td); return (error); } int fgetvp(struct thread *td, int fd, cap_rights_t *rightsp, struct vnode **vpp) { return (_fgetvp(td, fd, 0, rightsp, vpp)); } int fgetvp_rights(struct thread *td, int fd, cap_rights_t *needrightsp, struct filecaps *havecaps, struct vnode **vpp) { struct filedesc *fdp; struct filecaps caps; struct file *fp; int error; fdp = td->td_proc->p_fd; error = fget_cap_locked(fdp, fd, needrightsp, &fp, &caps); if (error != 0) return (error); if (fp->f_ops == &badfileops) { error = EBADF; goto out; } if (fp->f_vnode == NULL) { error = EINVAL; goto out; } *havecaps = caps; *vpp = fp->f_vnode; vrefact(*vpp); return (0); out: filecaps_free(&caps); return (error); } int fgetvp_read(struct thread *td, int fd, cap_rights_t *rightsp, struct vnode **vpp) { return (_fgetvp(td, fd, FREAD, rightsp, vpp)); } int fgetvp_exec(struct thread *td, int fd, cap_rights_t *rightsp, struct vnode **vpp) { return (_fgetvp(td, fd, FEXEC, rightsp, vpp)); } #ifdef notyet int fgetvp_write(struct thread *td, int fd, cap_rights_t *rightsp, struct vnode **vpp) { return (_fgetvp(td, fd, FWRITE, rightsp, vpp)); } #endif /* * Handle the last reference to a file being closed. * * Without the noinline attribute clang keeps inlining the func thorough this * file when fdrop is used. */ int __noinline _fdrop(struct file *fp, struct thread *td) { int error; if (fp->f_count != 0) panic("fdrop: count %d", fp->f_count); error = fo_close(fp, td); atomic_subtract_int(&openfiles, 1); crfree(fp->f_cred); free(fp->f_advice, M_FADVISE); uma_zfree(file_zone, fp); return (error); } /* * Apply an advisory lock on a file descriptor. * * Just attempt to get a record lock of the requested type on the entire file * (l_whence = SEEK_SET, l_start = 0, l_len = 0). */ #ifndef _SYS_SYSPROTO_H_ struct flock_args { int fd; int how; }; #endif /* ARGSUSED */ int sys_flock(struct thread *td, struct flock_args *uap) { struct file *fp; struct vnode *vp; struct flock lf; int error; error = fget(td, uap->fd, &cap_flock_rights, &fp); if (error != 0) return (error); if (fp->f_type != DTYPE_VNODE) { fdrop(fp, td); return (EOPNOTSUPP); } vp = fp->f_vnode; lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; if (uap->how & LOCK_UN) { lf.l_type = F_UNLCK; atomic_clear_int(&fp->f_flag, FHASLOCK); error = VOP_ADVLOCK(vp, (caddr_t)fp, F_UNLCK, &lf, F_FLOCK); goto done2; } if (uap->how & LOCK_EX) lf.l_type = F_WRLCK; else if (uap->how & LOCK_SH) lf.l_type = F_RDLCK; else { error = EBADF; goto done2; } atomic_set_int(&fp->f_flag, FHASLOCK); error = VOP_ADVLOCK(vp, (caddr_t)fp, F_SETLK, &lf, (uap->how & LOCK_NB) ? F_FLOCK : F_FLOCK | F_WAIT); done2: fdrop(fp, td); return (error); } /* * Duplicate the specified descriptor to a free descriptor. */ int dupfdopen(struct thread *td, struct filedesc *fdp, int dfd, int mode, int openerror, int *indxp) { struct filedescent *newfde, *oldfde; struct file *fp; u_long *ioctls; int error, indx; KASSERT(openerror == ENODEV || openerror == ENXIO, ("unexpected error %d in %s", openerror, __func__)); /* * If the to-be-dup'd fd number is greater than the allowed number * of file descriptors, or the fd to be dup'd has already been * closed, then reject. */ FILEDESC_XLOCK(fdp); if ((fp = fget_locked(fdp, dfd)) == NULL) { FILEDESC_XUNLOCK(fdp); return (EBADF); } error = fdalloc(td, 0, &indx); if (error != 0) { FILEDESC_XUNLOCK(fdp); return (error); } /* * There are two cases of interest here. * * For ENODEV simply dup (dfd) to file descriptor (indx) and return. * * For ENXIO steal away the file structure from (dfd) and store it in * (indx). (dfd) is effectively closed by this operation. */ switch (openerror) { case ENODEV: /* * Check that the mode the file is being opened for is a * subset of the mode of the existing descriptor. */ if (((mode & (FREAD|FWRITE)) | fp->f_flag) != fp->f_flag) { fdunused(fdp, indx); FILEDESC_XUNLOCK(fdp); return (EACCES); } if (!fhold(fp)) { fdunused(fdp, indx); FILEDESC_XUNLOCK(fdp); return (EBADF); } newfde = &fdp->fd_ofiles[indx]; oldfde = &fdp->fd_ofiles[dfd]; ioctls = filecaps_copy_prep(&oldfde->fde_caps); #ifdef CAPABILITIES seqc_write_begin(&newfde->fde_seqc); #endif memcpy(newfde, oldfde, fde_change_size); filecaps_copy_finish(&oldfde->fde_caps, &newfde->fde_caps, ioctls); #ifdef CAPABILITIES seqc_write_end(&newfde->fde_seqc); #endif break; case ENXIO: /* * Steal away the file pointer from dfd and stuff it into indx. */ newfde = &fdp->fd_ofiles[indx]; oldfde = &fdp->fd_ofiles[dfd]; #ifdef CAPABILITIES seqc_write_begin(&newfde->fde_seqc); #endif memcpy(newfde, oldfde, fde_change_size); oldfde->fde_file = NULL; fdunused(fdp, dfd); #ifdef CAPABILITIES seqc_write_end(&newfde->fde_seqc); #endif break; } FILEDESC_XUNLOCK(fdp); *indxp = indx; return (0); } /* * This sysctl determines if we will allow a process to chroot(2) if it * has a directory open: * 0: disallowed for all processes. * 1: allowed for processes that were not already chroot(2)'ed. * 2: allowed for all processes. */ static int chroot_allow_open_directories = 1; SYSCTL_INT(_kern, OID_AUTO, chroot_allow_open_directories, CTLFLAG_RW, &chroot_allow_open_directories, 0, "Allow a process to chroot(2) if it has a directory open"); /* * Helper function for raised chroot(2) security function: Refuse if * any filedescriptors are open directories. */ static int chroot_refuse_vdir_fds(struct filedesc *fdp) { struct vnode *vp; struct file *fp; int fd; FILEDESC_LOCK_ASSERT(fdp); for (fd = 0; fd <= fdp->fd_lastfile; fd++) { fp = fget_locked(fdp, fd); if (fp == NULL) continue; if (fp->f_type == DTYPE_VNODE) { vp = fp->f_vnode; if (vp->v_type == VDIR) return (EPERM); } } return (0); } /* * Common routine for kern_chroot() and jail_attach(). The caller is * responsible for invoking priv_check() and mac_vnode_check_chroot() to * authorize this operation. */ int pwd_chroot(struct thread *td, struct vnode *vp) { struct filedesc *fdp; struct vnode *oldvp; int error; fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); if (chroot_allow_open_directories == 0 || (chroot_allow_open_directories == 1 && fdp->fd_rdir != rootvnode)) { error = chroot_refuse_vdir_fds(fdp); if (error != 0) { FILEDESC_XUNLOCK(fdp); return (error); } } oldvp = fdp->fd_rdir; vrefact(vp); fdp->fd_rdir = vp; if (fdp->fd_jdir == NULL) { vrefact(vp); fdp->fd_jdir = vp; } FILEDESC_XUNLOCK(fdp); vrele(oldvp); return (0); } void pwd_chdir(struct thread *td, struct vnode *vp) { struct filedesc *fdp; struct vnode *oldvp; fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); VNASSERT(vp->v_usecount > 0, vp, ("chdir to a vnode with zero usecount")); oldvp = fdp->fd_cdir; fdp->fd_cdir = vp; FILEDESC_XUNLOCK(fdp); vrele(oldvp); } /* * Scan all active processes and prisons to see if any of them have a current * or root directory of `olddp'. If so, replace them with the new mount point. */ void mountcheckdirs(struct vnode *olddp, struct vnode *newdp) { struct filedesc *fdp; struct prison *pr; struct proc *p; int nrele; if (vrefcnt(olddp) == 1) return; nrele = 0; sx_slock(&allproc_lock); FOREACH_PROC_IN_SYSTEM(p) { PROC_LOCK(p); fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) continue; FILEDESC_XLOCK(fdp); if (fdp->fd_cdir == olddp) { vrefact(newdp); fdp->fd_cdir = newdp; nrele++; } if (fdp->fd_rdir == olddp) { vrefact(newdp); fdp->fd_rdir = newdp; nrele++; } if (fdp->fd_jdir == olddp) { vrefact(newdp); fdp->fd_jdir = newdp; nrele++; } FILEDESC_XUNLOCK(fdp); fddrop(fdp); } sx_sunlock(&allproc_lock); if (rootvnode == olddp) { vrefact(newdp); rootvnode = newdp; nrele++; } mtx_lock(&prison0.pr_mtx); if (prison0.pr_root == olddp) { vrefact(newdp); prison0.pr_root = newdp; nrele++; } mtx_unlock(&prison0.pr_mtx); sx_slock(&allprison_lock); TAILQ_FOREACH(pr, &allprison, pr_list) { mtx_lock(&pr->pr_mtx); if (pr->pr_root == olddp) { vrefact(newdp); pr->pr_root = newdp; nrele++; } mtx_unlock(&pr->pr_mtx); } sx_sunlock(&allprison_lock); while (nrele--) vrele(olddp); } struct filedesc_to_leader * filedesc_to_leader_alloc(struct filedesc_to_leader *old, struct filedesc *fdp, struct proc *leader) { struct filedesc_to_leader *fdtol; fdtol = malloc(sizeof(struct filedesc_to_leader), M_FILEDESC_TO_LEADER, M_WAITOK); fdtol->fdl_refcount = 1; fdtol->fdl_holdcount = 0; fdtol->fdl_wakeup = 0; fdtol->fdl_leader = leader; if (old != NULL) { FILEDESC_XLOCK(fdp); fdtol->fdl_next = old->fdl_next; fdtol->fdl_prev = old; old->fdl_next = fdtol; fdtol->fdl_next->fdl_prev = fdtol; FILEDESC_XUNLOCK(fdp); } else { fdtol->fdl_next = fdtol; fdtol->fdl_prev = fdtol; } return (fdtol); } static int sysctl_kern_proc_nfds(SYSCTL_HANDLER_ARGS) { struct filedesc *fdp; int i, count, slots; if (*(int *)arg1 != 0) return (EINVAL); fdp = curproc->p_fd; count = 0; FILEDESC_SLOCK(fdp); slots = NDSLOTS(fdp->fd_lastfile + 1); for (i = 0; i < slots; i++) count += bitcountl(fdp->fd_map[i]); FILEDESC_SUNLOCK(fdp); return (SYSCTL_OUT(req, &count, sizeof(count))); } static SYSCTL_NODE(_kern_proc, KERN_PROC_NFDS, nfds, CTLFLAG_RD|CTLFLAG_CAPRD|CTLFLAG_MPSAFE, sysctl_kern_proc_nfds, "Number of open file descriptors"); /* * Get file structures globally. */ static int sysctl_kern_file(SYSCTL_HANDLER_ARGS) { struct xfile xf; struct filedesc *fdp; struct file *fp; struct proc *p; int error, n; error = sysctl_wire_old_buffer(req, 0); if (error != 0) return (error); if (req->oldptr == NULL) { n = 0; sx_slock(&allproc_lock); FOREACH_PROC_IN_SYSTEM(p) { PROC_LOCK(p); if (p->p_state == PRS_NEW) { PROC_UNLOCK(p); continue; } fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) continue; /* overestimates sparse tables. */ if (fdp->fd_lastfile > 0) n += fdp->fd_lastfile; fddrop(fdp); } sx_sunlock(&allproc_lock); return (SYSCTL_OUT(req, 0, n * sizeof(xf))); } error = 0; bzero(&xf, sizeof(xf)); xf.xf_size = sizeof(xf); sx_slock(&allproc_lock); FOREACH_PROC_IN_SYSTEM(p) { PROC_LOCK(p); if (p->p_state == PRS_NEW) { PROC_UNLOCK(p); continue; } if (p_cansee(req->td, p) != 0) { PROC_UNLOCK(p); continue; } xf.xf_pid = p->p_pid; xf.xf_uid = p->p_ucred->cr_uid; fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) continue; FILEDESC_SLOCK(fdp); for (n = 0; fdp->fd_refcnt > 0 && n <= fdp->fd_lastfile; ++n) { if ((fp = fdp->fd_ofiles[n].fde_file) == NULL) continue; xf.xf_fd = n; xf.xf_file = (uintptr_t)fp; xf.xf_data = (uintptr_t)fp->f_data; xf.xf_vnode = (uintptr_t)fp->f_vnode; xf.xf_type = (uintptr_t)fp->f_type; xf.xf_count = fp->f_count; xf.xf_msgcount = 0; xf.xf_offset = foffset_get(fp); xf.xf_flag = fp->f_flag; error = SYSCTL_OUT(req, &xf, sizeof(xf)); if (error) break; } FILEDESC_SUNLOCK(fdp); fddrop(fdp); if (error) break; } sx_sunlock(&allproc_lock); return (error); } SYSCTL_PROC(_kern, KERN_FILE, file, CTLTYPE_OPAQUE|CTLFLAG_RD|CTLFLAG_MPSAFE, 0, 0, sysctl_kern_file, "S,xfile", "Entire file table"); #ifdef KINFO_FILE_SIZE CTASSERT(sizeof(struct kinfo_file) == KINFO_FILE_SIZE); #endif static int xlate_fflags(int fflags) { static const struct { int fflag; int kf_fflag; } fflags_table[] = { { FAPPEND, KF_FLAG_APPEND }, { FASYNC, KF_FLAG_ASYNC }, { FFSYNC, KF_FLAG_FSYNC }, { FHASLOCK, KF_FLAG_HASLOCK }, { FNONBLOCK, KF_FLAG_NONBLOCK }, { FREAD, KF_FLAG_READ }, { FWRITE, KF_FLAG_WRITE }, { O_CREAT, KF_FLAG_CREAT }, { O_DIRECT, KF_FLAG_DIRECT }, { O_EXCL, KF_FLAG_EXCL }, { O_EXEC, KF_FLAG_EXEC }, { O_EXLOCK, KF_FLAG_EXLOCK }, { O_NOFOLLOW, KF_FLAG_NOFOLLOW }, { O_SHLOCK, KF_FLAG_SHLOCK }, { O_TRUNC, KF_FLAG_TRUNC } }; unsigned int i; int kflags; kflags = 0; for (i = 0; i < nitems(fflags_table); i++) if (fflags & fflags_table[i].fflag) kflags |= fflags_table[i].kf_fflag; return (kflags); } /* Trim unused data from kf_path by truncating the structure size. */ void pack_kinfo(struct kinfo_file *kif) { kif->kf_structsize = offsetof(struct kinfo_file, kf_path) + strlen(kif->kf_path) + 1; kif->kf_structsize = roundup(kif->kf_structsize, sizeof(uint64_t)); } static void export_file_to_kinfo(struct file *fp, int fd, cap_rights_t *rightsp, struct kinfo_file *kif, struct filedesc *fdp, int flags) { int error; bzero(kif, sizeof(*kif)); /* Set a default type to allow for empty fill_kinfo() methods. */ kif->kf_type = KF_TYPE_UNKNOWN; kif->kf_flags = xlate_fflags(fp->f_flag); if (rightsp != NULL) kif->kf_cap_rights = *rightsp; else - cap_rights_init(&kif->kf_cap_rights); + cap_rights_init_zero(&kif->kf_cap_rights); kif->kf_fd = fd; kif->kf_ref_count = fp->f_count; kif->kf_offset = foffset_get(fp); /* * This may drop the filedesc lock, so the 'fp' cannot be * accessed after this call. */ error = fo_fill_kinfo(fp, kif, fdp); if (error == 0) kif->kf_status |= KF_ATTR_VALID; if ((flags & KERN_FILEDESC_PACK_KINFO) != 0) pack_kinfo(kif); else kif->kf_structsize = roundup2(sizeof(*kif), sizeof(uint64_t)); } static void export_vnode_to_kinfo(struct vnode *vp, int fd, int fflags, struct kinfo_file *kif, int flags) { int error; bzero(kif, sizeof(*kif)); kif->kf_type = KF_TYPE_VNODE; error = vn_fill_kinfo_vnode(vp, kif); if (error == 0) kif->kf_status |= KF_ATTR_VALID; kif->kf_flags = xlate_fflags(fflags); - cap_rights_init(&kif->kf_cap_rights); + cap_rights_init_zero(&kif->kf_cap_rights); kif->kf_fd = fd; kif->kf_ref_count = -1; kif->kf_offset = -1; if ((flags & KERN_FILEDESC_PACK_KINFO) != 0) pack_kinfo(kif); else kif->kf_structsize = roundup2(sizeof(*kif), sizeof(uint64_t)); vrele(vp); } struct export_fd_buf { struct filedesc *fdp; struct sbuf *sb; ssize_t remainder; struct kinfo_file kif; int flags; }; static int export_kinfo_to_sb(struct export_fd_buf *efbuf) { struct kinfo_file *kif; kif = &efbuf->kif; if (efbuf->remainder != -1) { if (efbuf->remainder < kif->kf_structsize) { /* Terminate export. */ efbuf->remainder = 0; return (0); } efbuf->remainder -= kif->kf_structsize; } return (sbuf_bcat(efbuf->sb, kif, kif->kf_structsize) == 0 ? 0 : ENOMEM); } static int export_file_to_sb(struct file *fp, int fd, cap_rights_t *rightsp, struct export_fd_buf *efbuf) { int error; if (efbuf->remainder == 0) return (0); export_file_to_kinfo(fp, fd, rightsp, &efbuf->kif, efbuf->fdp, efbuf->flags); FILEDESC_SUNLOCK(efbuf->fdp); error = export_kinfo_to_sb(efbuf); FILEDESC_SLOCK(efbuf->fdp); return (error); } static int export_vnode_to_sb(struct vnode *vp, int fd, int fflags, struct export_fd_buf *efbuf) { int error; if (efbuf->remainder == 0) return (0); if (efbuf->fdp != NULL) FILEDESC_SUNLOCK(efbuf->fdp); export_vnode_to_kinfo(vp, fd, fflags, &efbuf->kif, efbuf->flags); error = export_kinfo_to_sb(efbuf); if (efbuf->fdp != NULL) FILEDESC_SLOCK(efbuf->fdp); return (error); } /* * Store a process file descriptor information to sbuf. * * Takes a locked proc as argument, and returns with the proc unlocked. */ int kern_proc_filedesc_out(struct proc *p, struct sbuf *sb, ssize_t maxlen, int flags) { struct file *fp; struct filedesc *fdp; struct export_fd_buf *efbuf; struct vnode *cttyvp, *textvp, *tracevp; int error, i; cap_rights_t rights; PROC_LOCK_ASSERT(p, MA_OWNED); /* ktrace vnode */ tracevp = p->p_tracevp; if (tracevp != NULL) vrefact(tracevp); /* text vnode */ textvp = p->p_textvp; if (textvp != NULL) vrefact(textvp); /* Controlling tty. */ cttyvp = NULL; if (p->p_pgrp != NULL && p->p_pgrp->pg_session != NULL) { cttyvp = p->p_pgrp->pg_session->s_ttyvp; if (cttyvp != NULL) vrefact(cttyvp); } fdp = fdhold(p); PROC_UNLOCK(p); efbuf = malloc(sizeof(*efbuf), M_TEMP, M_WAITOK); efbuf->fdp = NULL; efbuf->sb = sb; efbuf->remainder = maxlen; efbuf->flags = flags; if (tracevp != NULL) export_vnode_to_sb(tracevp, KF_FD_TYPE_TRACE, FREAD | FWRITE, efbuf); if (textvp != NULL) export_vnode_to_sb(textvp, KF_FD_TYPE_TEXT, FREAD, efbuf); if (cttyvp != NULL) export_vnode_to_sb(cttyvp, KF_FD_TYPE_CTTY, FREAD | FWRITE, efbuf); error = 0; if (fdp == NULL) goto fail; efbuf->fdp = fdp; FILEDESC_SLOCK(fdp); /* working directory */ if (fdp->fd_cdir != NULL) { vrefact(fdp->fd_cdir); export_vnode_to_sb(fdp->fd_cdir, KF_FD_TYPE_CWD, FREAD, efbuf); } /* root directory */ if (fdp->fd_rdir != NULL) { vrefact(fdp->fd_rdir); export_vnode_to_sb(fdp->fd_rdir, KF_FD_TYPE_ROOT, FREAD, efbuf); } /* jail directory */ if (fdp->fd_jdir != NULL) { vrefact(fdp->fd_jdir); export_vnode_to_sb(fdp->fd_jdir, KF_FD_TYPE_JAIL, FREAD, efbuf); } for (i = 0; fdp->fd_refcnt > 0 && i <= fdp->fd_lastfile; i++) { if ((fp = fdp->fd_ofiles[i].fde_file) == NULL) continue; #ifdef CAPABILITIES rights = *cap_rights(fdp, i); #else /* !CAPABILITIES */ rights = cap_no_rights; #endif /* * Create sysctl entry. It is OK to drop the filedesc * lock inside of export_file_to_sb() as we will * re-validate and re-evaluate its properties when the * loop continues. */ error = export_file_to_sb(fp, i, &rights, efbuf); if (error != 0 || efbuf->remainder == 0) break; } FILEDESC_SUNLOCK(fdp); fddrop(fdp); fail: free(efbuf, M_TEMP); return (error); } #define FILEDESC_SBUF_SIZE (sizeof(struct kinfo_file) * 5) /* * Get per-process file descriptors for use by procstat(1), et al. */ static int sysctl_kern_proc_filedesc(SYSCTL_HANDLER_ARGS) { struct sbuf sb; struct proc *p; ssize_t maxlen; int error, error2, *name; name = (int *)arg1; sbuf_new_for_sysctl(&sb, NULL, FILEDESC_SBUF_SIZE, req); sbuf_clear_flags(&sb, SBUF_INCLUDENUL); error = pget((pid_t)name[0], PGET_CANDEBUG | PGET_NOTWEXIT, &p); if (error != 0) { sbuf_delete(&sb); return (error); } maxlen = req->oldptr != NULL ? req->oldlen : -1; error = kern_proc_filedesc_out(p, &sb, maxlen, KERN_FILEDESC_PACK_KINFO); error2 = sbuf_finish(&sb); sbuf_delete(&sb); return (error != 0 ? error : error2); } #ifdef COMPAT_FREEBSD7 #ifdef KINFO_OFILE_SIZE CTASSERT(sizeof(struct kinfo_ofile) == KINFO_OFILE_SIZE); #endif static void kinfo_to_okinfo(struct kinfo_file *kif, struct kinfo_ofile *okif) { okif->kf_structsize = sizeof(*okif); okif->kf_type = kif->kf_type; okif->kf_fd = kif->kf_fd; okif->kf_ref_count = kif->kf_ref_count; okif->kf_flags = kif->kf_flags & (KF_FLAG_READ | KF_FLAG_WRITE | KF_FLAG_APPEND | KF_FLAG_ASYNC | KF_FLAG_FSYNC | KF_FLAG_NONBLOCK | KF_FLAG_DIRECT | KF_FLAG_HASLOCK); okif->kf_offset = kif->kf_offset; if (kif->kf_type == KF_TYPE_VNODE) okif->kf_vnode_type = kif->kf_un.kf_file.kf_file_type; else okif->kf_vnode_type = KF_VTYPE_VNON; strlcpy(okif->kf_path, kif->kf_path, sizeof(okif->kf_path)); if (kif->kf_type == KF_TYPE_SOCKET) { okif->kf_sock_domain = kif->kf_un.kf_sock.kf_sock_domain0; okif->kf_sock_type = kif->kf_un.kf_sock.kf_sock_type0; okif->kf_sock_protocol = kif->kf_un.kf_sock.kf_sock_protocol0; okif->kf_sa_local = kif->kf_un.kf_sock.kf_sa_local; okif->kf_sa_peer = kif->kf_un.kf_sock.kf_sa_peer; } else { okif->kf_sa_local.ss_family = AF_UNSPEC; okif->kf_sa_peer.ss_family = AF_UNSPEC; } } static int export_vnode_for_osysctl(struct vnode *vp, int type, struct kinfo_file *kif, struct kinfo_ofile *okif, struct filedesc *fdp, struct sysctl_req *req) { int error; vrefact(vp); FILEDESC_SUNLOCK(fdp); export_vnode_to_kinfo(vp, type, 0, kif, KERN_FILEDESC_PACK_KINFO); kinfo_to_okinfo(kif, okif); error = SYSCTL_OUT(req, okif, sizeof(*okif)); FILEDESC_SLOCK(fdp); return (error); } /* * Get per-process file descriptors for use by procstat(1), et al. */ static int sysctl_kern_proc_ofiledesc(SYSCTL_HANDLER_ARGS) { struct kinfo_ofile *okif; struct kinfo_file *kif; struct filedesc *fdp; int error, i, *name; struct file *fp; struct proc *p; name = (int *)arg1; error = pget((pid_t)name[0], PGET_CANDEBUG | PGET_NOTWEXIT, &p); if (error != 0) return (error); fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) return (ENOENT); kif = malloc(sizeof(*kif), M_TEMP, M_WAITOK); okif = malloc(sizeof(*okif), M_TEMP, M_WAITOK); FILEDESC_SLOCK(fdp); if (fdp->fd_cdir != NULL) export_vnode_for_osysctl(fdp->fd_cdir, KF_FD_TYPE_CWD, kif, okif, fdp, req); if (fdp->fd_rdir != NULL) export_vnode_for_osysctl(fdp->fd_rdir, KF_FD_TYPE_ROOT, kif, okif, fdp, req); if (fdp->fd_jdir != NULL) export_vnode_for_osysctl(fdp->fd_jdir, KF_FD_TYPE_JAIL, kif, okif, fdp, req); for (i = 0; fdp->fd_refcnt > 0 && i <= fdp->fd_lastfile; i++) { if ((fp = fdp->fd_ofiles[i].fde_file) == NULL) continue; export_file_to_kinfo(fp, i, NULL, kif, fdp, KERN_FILEDESC_PACK_KINFO); FILEDESC_SUNLOCK(fdp); kinfo_to_okinfo(kif, okif); error = SYSCTL_OUT(req, okif, sizeof(*okif)); FILEDESC_SLOCK(fdp); if (error) break; } FILEDESC_SUNLOCK(fdp); fddrop(fdp); free(kif, M_TEMP); free(okif, M_TEMP); return (0); } static SYSCTL_NODE(_kern_proc, KERN_PROC_OFILEDESC, ofiledesc, CTLFLAG_RD|CTLFLAG_MPSAFE, sysctl_kern_proc_ofiledesc, "Process ofiledesc entries"); #endif /* COMPAT_FREEBSD7 */ int vntype_to_kinfo(int vtype) { struct { int vtype; int kf_vtype; } vtypes_table[] = { { VBAD, KF_VTYPE_VBAD }, { VBLK, KF_VTYPE_VBLK }, { VCHR, KF_VTYPE_VCHR }, { VDIR, KF_VTYPE_VDIR }, { VFIFO, KF_VTYPE_VFIFO }, { VLNK, KF_VTYPE_VLNK }, { VNON, KF_VTYPE_VNON }, { VREG, KF_VTYPE_VREG }, { VSOCK, KF_VTYPE_VSOCK } }; unsigned int i; /* * Perform vtype translation. */ for (i = 0; i < nitems(vtypes_table); i++) if (vtypes_table[i].vtype == vtype) return (vtypes_table[i].kf_vtype); return (KF_VTYPE_UNKNOWN); } static SYSCTL_NODE(_kern_proc, KERN_PROC_FILEDESC, filedesc, CTLFLAG_RD|CTLFLAG_MPSAFE, sysctl_kern_proc_filedesc, "Process filedesc entries"); /* * Store a process current working directory information to sbuf. * * Takes a locked proc as argument, and returns with the proc unlocked. */ int kern_proc_cwd_out(struct proc *p, struct sbuf *sb, ssize_t maxlen) { struct filedesc *fdp; struct export_fd_buf *efbuf; int error; PROC_LOCK_ASSERT(p, MA_OWNED); fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) return (EINVAL); efbuf = malloc(sizeof(*efbuf), M_TEMP, M_WAITOK); efbuf->fdp = fdp; efbuf->sb = sb; efbuf->remainder = maxlen; FILEDESC_SLOCK(fdp); if (fdp->fd_cdir == NULL) error = EINVAL; else { vrefact(fdp->fd_cdir); error = export_vnode_to_sb(fdp->fd_cdir, KF_FD_TYPE_CWD, FREAD, efbuf); } FILEDESC_SUNLOCK(fdp); fddrop(fdp); free(efbuf, M_TEMP); return (error); } /* * Get per-process current working directory. */ static int sysctl_kern_proc_cwd(SYSCTL_HANDLER_ARGS) { struct sbuf sb; struct proc *p; ssize_t maxlen; int error, error2, *name; name = (int *)arg1; sbuf_new_for_sysctl(&sb, NULL, sizeof(struct kinfo_file), req); sbuf_clear_flags(&sb, SBUF_INCLUDENUL); error = pget((pid_t)name[0], PGET_CANDEBUG | PGET_NOTWEXIT, &p); if (error != 0) { sbuf_delete(&sb); return (error); } maxlen = req->oldptr != NULL ? req->oldlen : -1; error = kern_proc_cwd_out(p, &sb, maxlen); error2 = sbuf_finish(&sb); sbuf_delete(&sb); return (error != 0 ? error : error2); } static SYSCTL_NODE(_kern_proc, KERN_PROC_CWD, cwd, CTLFLAG_RD|CTLFLAG_MPSAFE, sysctl_kern_proc_cwd, "Process current working directory"); #ifdef DDB /* * For the purposes of debugging, generate a human-readable string for the * file type. */ static const char * file_type_to_name(short type) { switch (type) { case 0: return ("zero"); case DTYPE_VNODE: return ("vnode"); case DTYPE_SOCKET: return ("socket"); case DTYPE_PIPE: return ("pipe"); case DTYPE_FIFO: return ("fifo"); case DTYPE_KQUEUE: return ("kqueue"); case DTYPE_CRYPTO: return ("crypto"); case DTYPE_MQUEUE: return ("mqueue"); case DTYPE_SHM: return ("shm"); case DTYPE_SEM: return ("ksem"); case DTYPE_PTS: return ("pts"); case DTYPE_DEV: return ("dev"); case DTYPE_PROCDESC: return ("proc"); case DTYPE_LINUXEFD: return ("levent"); case DTYPE_LINUXTFD: return ("ltimer"); default: return ("unkn"); } } /* * For the purposes of debugging, identify a process (if any, perhaps one of * many) that references the passed file in its file descriptor array. Return * NULL if none. */ static struct proc * file_to_first_proc(struct file *fp) { struct filedesc *fdp; struct proc *p; int n; FOREACH_PROC_IN_SYSTEM(p) { if (p->p_state == PRS_NEW) continue; fdp = p->p_fd; if (fdp == NULL) continue; for (n = 0; n <= fdp->fd_lastfile; n++) { if (fp == fdp->fd_ofiles[n].fde_file) return (p); } } return (NULL); } static void db_print_file(struct file *fp, int header) { #define XPTRWIDTH ((int)howmany(sizeof(void *) * NBBY, 4)) struct proc *p; if (header) db_printf("%*s %6s %*s %8s %4s %5s %6s %*s %5s %s\n", XPTRWIDTH, "File", "Type", XPTRWIDTH, "Data", "Flag", "GCFl", "Count", "MCount", XPTRWIDTH, "Vnode", "FPID", "FCmd"); p = file_to_first_proc(fp); db_printf("%*p %6s %*p %08x %04x %5d %6d %*p %5d %s\n", XPTRWIDTH, fp, file_type_to_name(fp->f_type), XPTRWIDTH, fp->f_data, fp->f_flag, 0, fp->f_count, 0, XPTRWIDTH, fp->f_vnode, p != NULL ? p->p_pid : -1, p != NULL ? p->p_comm : "-"); #undef XPTRWIDTH } DB_SHOW_COMMAND(file, db_show_file) { struct file *fp; if (!have_addr) { db_printf("usage: show file \n"); return; } fp = (struct file *)addr; db_print_file(fp, 1); } DB_SHOW_COMMAND(files, db_show_files) { struct filedesc *fdp; struct file *fp; struct proc *p; int header; int n; header = 1; FOREACH_PROC_IN_SYSTEM(p) { if (p->p_state == PRS_NEW) continue; if ((fdp = p->p_fd) == NULL) continue; for (n = 0; n <= fdp->fd_lastfile; ++n) { if ((fp = fdp->fd_ofiles[n].fde_file) == NULL) continue; db_print_file(fp, header); header = 0; } } } #endif SYSCTL_INT(_kern, KERN_MAXFILESPERPROC, maxfilesperproc, CTLFLAG_RW, &maxfilesperproc, 0, "Maximum files allowed open per process"); SYSCTL_INT(_kern, KERN_MAXFILES, maxfiles, CTLFLAG_RW, &maxfiles, 0, "Maximum number of files"); SYSCTL_INT(_kern, OID_AUTO, openfiles, CTLFLAG_RD, &openfiles, 0, "System-wide number of open files"); /* ARGSUSED*/ static void filelistinit(void *dummy) { file_zone = uma_zcreate("Files", sizeof(struct file), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE); filedesc0_zone = uma_zcreate("filedesc0", sizeof(struct filedesc0), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); mtx_init(&sigio_lock, "sigio lock", NULL, MTX_DEF); } SYSINIT(select, SI_SUB_LOCK, SI_ORDER_FIRST, filelistinit, NULL); /*-------------------------------------------------------------------*/ static int badfo_readwrite(struct file *fp, struct uio *uio, struct ucred *active_cred, int flags, struct thread *td) { return (EBADF); } static int badfo_truncate(struct file *fp, off_t length, struct ucred *active_cred, struct thread *td) { return (EINVAL); } static int badfo_ioctl(struct file *fp, u_long com, void *data, struct ucred *active_cred, struct thread *td) { return (EBADF); } static int badfo_poll(struct file *fp, int events, struct ucred *active_cred, struct thread *td) { return (0); } static int badfo_kqfilter(struct file *fp, struct knote *kn) { return (EBADF); } static int badfo_stat(struct file *fp, struct stat *sb, struct ucred *active_cred, struct thread *td) { return (EBADF); } static int badfo_close(struct file *fp, struct thread *td) { return (0); } static int badfo_chmod(struct file *fp, mode_t mode, struct ucred *active_cred, struct thread *td) { return (EBADF); } static int badfo_chown(struct file *fp, uid_t uid, gid_t gid, struct ucred *active_cred, struct thread *td) { return (EBADF); } static int badfo_sendfile(struct file *fp, int sockfd, struct uio *hdr_uio, struct uio *trl_uio, off_t offset, size_t nbytes, off_t *sent, int flags, struct thread *td) { return (EBADF); } static int badfo_fill_kinfo(struct file *fp, struct kinfo_file *kif, struct filedesc *fdp) { return (0); } struct fileops badfileops = { .fo_read = badfo_readwrite, .fo_write = badfo_readwrite, .fo_truncate = badfo_truncate, .fo_ioctl = badfo_ioctl, .fo_poll = badfo_poll, .fo_kqfilter = badfo_kqfilter, .fo_stat = badfo_stat, .fo_close = badfo_close, .fo_chmod = badfo_chmod, .fo_chown = badfo_chown, .fo_sendfile = badfo_sendfile, .fo_fill_kinfo = badfo_fill_kinfo, }; int invfo_rdwr(struct file *fp, struct uio *uio, struct ucred *active_cred, int flags, struct thread *td) { return (EOPNOTSUPP); } int invfo_truncate(struct file *fp, off_t length, struct ucred *active_cred, struct thread *td) { return (EINVAL); } int invfo_ioctl(struct file *fp, u_long com, void *data, struct ucred *active_cred, struct thread *td) { return (ENOTTY); } int invfo_poll(struct file *fp, int events, struct ucred *active_cred, struct thread *td) { return (poll_no_poll(events)); } int invfo_kqfilter(struct file *fp, struct knote *kn) { return (EINVAL); } int invfo_chmod(struct file *fp, mode_t mode, struct ucred *active_cred, struct thread *td) { return (EINVAL); } int invfo_chown(struct file *fp, uid_t uid, gid_t gid, struct ucred *active_cred, struct thread *td) { return (EINVAL); } int invfo_sendfile(struct file *fp, int sockfd, struct uio *hdr_uio, struct uio *trl_uio, off_t offset, size_t nbytes, off_t *sent, int flags, struct thread *td) { return (EINVAL); } /*-------------------------------------------------------------------*/ /* * File Descriptor pseudo-device driver (/dev/fd/). * * Opening minor device N dup()s the file (if any) connected to file * descriptor N belonging to the calling process. Note that this driver * consists of only the ``open()'' routine, because all subsequent * references to this file will be direct to the other driver. * * XXX: we could give this one a cloning event handler if necessary. */ /* ARGSUSED */ static int fdopen(struct cdev *dev, int mode, int type, struct thread *td) { /* * XXX Kludge: set curthread->td_dupfd to contain the value of the * the file descriptor being sought for duplication. The error * return ensures that the vnode for this device will be released * by vn_open. Open will detect this special error and take the * actions in dupfdopen below. Other callers of vn_open or VOP_OPEN * will simply report the error. */ td->td_dupfd = dev2unit(dev); return (ENODEV); } static struct cdevsw fildesc_cdevsw = { .d_version = D_VERSION, .d_open = fdopen, .d_name = "FD", }; static void fildesc_drvinit(void *unused) { struct cdev *dev; dev = make_dev_credf(MAKEDEV_ETERNAL, &fildesc_cdevsw, 0, NULL, UID_ROOT, GID_WHEEL, 0666, "fd/0"); make_dev_alias(dev, "stdin"); dev = make_dev_credf(MAKEDEV_ETERNAL, &fildesc_cdevsw, 1, NULL, UID_ROOT, GID_WHEEL, 0666, "fd/1"); make_dev_alias(dev, "stdout"); dev = make_dev_credf(MAKEDEV_ETERNAL, &fildesc_cdevsw, 2, NULL, UID_ROOT, GID_WHEEL, 0666, "fd/2"); make_dev_alias(dev, "stderr"); } SYSINIT(fildescdev, SI_SUB_DRIVERS, SI_ORDER_MIDDLE, fildesc_drvinit, NULL); Index: projects/clang1000-import/sys/kern/kern_event.c =================================================================== --- projects/clang1000-import/sys/kern/kern_event.c (revision 357965) +++ projects/clang1000-import/sys/kern/kern_event.c (revision 357966) @@ -1,2741 +1,2741 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 1999,2000,2001 Jonathan Lemon * Copyright 2004 John-Mark Gurney * Copyright (c) 2009 Apple, Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_ktrace.h" #include "opt_kqueue.h" #ifdef COMPAT_FREEBSD11 #define _WANT_FREEBSD11_KEVENT #endif #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef KTRACE #include #endif #include #include static MALLOC_DEFINE(M_KQUEUE, "kqueue", "memory for kqueue system"); /* * This lock is used if multiple kq locks are required. This possibly * should be made into a per proc lock. */ static struct mtx kq_global; MTX_SYSINIT(kq_global, &kq_global, "kqueue order", MTX_DEF); #define KQ_GLOBAL_LOCK(lck, haslck) do { \ if (!haslck) \ mtx_lock(lck); \ haslck = 1; \ } while (0) #define KQ_GLOBAL_UNLOCK(lck, haslck) do { \ if (haslck) \ mtx_unlock(lck); \ haslck = 0; \ } while (0) TASKQUEUE_DEFINE_THREAD(kqueue_ctx); static int kevent_copyout(void *arg, struct kevent *kevp, int count); static int kevent_copyin(void *arg, struct kevent *kevp, int count); static int kqueue_register(struct kqueue *kq, struct kevent *kev, struct thread *td, int mflag); static int kqueue_acquire(struct file *fp, struct kqueue **kqp); static void kqueue_release(struct kqueue *kq, int locked); static void kqueue_destroy(struct kqueue *kq); static void kqueue_drain(struct kqueue *kq, struct thread *td); static int kqueue_expand(struct kqueue *kq, struct filterops *fops, uintptr_t ident, int mflag); static void kqueue_task(void *arg, int pending); static int kqueue_scan(struct kqueue *kq, int maxevents, struct kevent_copyops *k_ops, const struct timespec *timeout, struct kevent *keva, struct thread *td); static void kqueue_wakeup(struct kqueue *kq); static struct filterops *kqueue_fo_find(int filt); static void kqueue_fo_release(int filt); struct g_kevent_args; static int kern_kevent_generic(struct thread *td, struct g_kevent_args *uap, struct kevent_copyops *k_ops, const char *struct_name); static fo_ioctl_t kqueue_ioctl; static fo_poll_t kqueue_poll; static fo_kqfilter_t kqueue_kqfilter; static fo_stat_t kqueue_stat; static fo_close_t kqueue_close; static fo_fill_kinfo_t kqueue_fill_kinfo; static struct fileops kqueueops = { .fo_read = invfo_rdwr, .fo_write = invfo_rdwr, .fo_truncate = invfo_truncate, .fo_ioctl = kqueue_ioctl, .fo_poll = kqueue_poll, .fo_kqfilter = kqueue_kqfilter, .fo_stat = kqueue_stat, .fo_close = kqueue_close, .fo_chmod = invfo_chmod, .fo_chown = invfo_chown, .fo_sendfile = invfo_sendfile, .fo_fill_kinfo = kqueue_fill_kinfo, }; static int knote_attach(struct knote *kn, struct kqueue *kq); static void knote_drop(struct knote *kn, struct thread *td); static void knote_drop_detached(struct knote *kn, struct thread *td); static void knote_enqueue(struct knote *kn); static void knote_dequeue(struct knote *kn); static void knote_init(void); static struct knote *knote_alloc(int mflag); static void knote_free(struct knote *kn); static void filt_kqdetach(struct knote *kn); static int filt_kqueue(struct knote *kn, long hint); static int filt_procattach(struct knote *kn); static void filt_procdetach(struct knote *kn); static int filt_proc(struct knote *kn, long hint); static int filt_fileattach(struct knote *kn); static void filt_timerexpire(void *knx); static int filt_timerattach(struct knote *kn); static void filt_timerdetach(struct knote *kn); static void filt_timerstart(struct knote *kn, sbintime_t to); static void filt_timertouch(struct knote *kn, struct kevent *kev, u_long type); static int filt_timervalidate(struct knote *kn, sbintime_t *to); static int filt_timer(struct knote *kn, long hint); static int filt_userattach(struct knote *kn); static void filt_userdetach(struct knote *kn); static int filt_user(struct knote *kn, long hint); static void filt_usertouch(struct knote *kn, struct kevent *kev, u_long type); static struct filterops file_filtops = { .f_isfd = 1, .f_attach = filt_fileattach, }; static struct filterops kqread_filtops = { .f_isfd = 1, .f_detach = filt_kqdetach, .f_event = filt_kqueue, }; /* XXX - move to kern_proc.c? */ static struct filterops proc_filtops = { .f_isfd = 0, .f_attach = filt_procattach, .f_detach = filt_procdetach, .f_event = filt_proc, }; static struct filterops timer_filtops = { .f_isfd = 0, .f_attach = filt_timerattach, .f_detach = filt_timerdetach, .f_event = filt_timer, .f_touch = filt_timertouch, }; static struct filterops user_filtops = { .f_attach = filt_userattach, .f_detach = filt_userdetach, .f_event = filt_user, .f_touch = filt_usertouch, }; static uma_zone_t knote_zone; static unsigned int kq_ncallouts = 0; static unsigned int kq_calloutmax = 4 * 1024; SYSCTL_UINT(_kern, OID_AUTO, kq_calloutmax, CTLFLAG_RW, &kq_calloutmax, 0, "Maximum number of callouts allocated for kqueue"); /* XXX - ensure not influx ? */ #define KNOTE_ACTIVATE(kn, islock) do { \ if ((islock)) \ mtx_assert(&(kn)->kn_kq->kq_lock, MA_OWNED); \ else \ KQ_LOCK((kn)->kn_kq); \ (kn)->kn_status |= KN_ACTIVE; \ if (((kn)->kn_status & (KN_QUEUED | KN_DISABLED)) == 0) \ knote_enqueue((kn)); \ if (!(islock)) \ KQ_UNLOCK((kn)->kn_kq); \ } while(0) #define KQ_LOCK(kq) do { \ mtx_lock(&(kq)->kq_lock); \ } while (0) #define KQ_FLUX_WAKEUP(kq) do { \ if (((kq)->kq_state & KQ_FLUXWAIT) == KQ_FLUXWAIT) { \ (kq)->kq_state &= ~KQ_FLUXWAIT; \ wakeup((kq)); \ } \ } while (0) #define KQ_UNLOCK_FLUX(kq) do { \ KQ_FLUX_WAKEUP(kq); \ mtx_unlock(&(kq)->kq_lock); \ } while (0) #define KQ_UNLOCK(kq) do { \ mtx_unlock(&(kq)->kq_lock); \ } while (0) #define KQ_OWNED(kq) do { \ mtx_assert(&(kq)->kq_lock, MA_OWNED); \ } while (0) #define KQ_NOTOWNED(kq) do { \ mtx_assert(&(kq)->kq_lock, MA_NOTOWNED); \ } while (0) static struct knlist * kn_list_lock(struct knote *kn) { struct knlist *knl; knl = kn->kn_knlist; if (knl != NULL) knl->kl_lock(knl->kl_lockarg); return (knl); } static void kn_list_unlock(struct knlist *knl) { bool do_free; if (knl == NULL) return; do_free = knl->kl_autodestroy && knlist_empty(knl); knl->kl_unlock(knl->kl_lockarg); if (do_free) { knlist_destroy(knl); free(knl, M_KQUEUE); } } static bool kn_in_flux(struct knote *kn) { return (kn->kn_influx > 0); } static void kn_enter_flux(struct knote *kn) { KQ_OWNED(kn->kn_kq); MPASS(kn->kn_influx < INT_MAX); kn->kn_influx++; } static bool kn_leave_flux(struct knote *kn) { KQ_OWNED(kn->kn_kq); MPASS(kn->kn_influx > 0); kn->kn_influx--; return (kn->kn_influx == 0); } #define KNL_ASSERT_LOCK(knl, islocked) do { \ if (islocked) \ KNL_ASSERT_LOCKED(knl); \ else \ KNL_ASSERT_UNLOCKED(knl); \ } while (0) #ifdef INVARIANTS #define KNL_ASSERT_LOCKED(knl) do { \ knl->kl_assert_locked((knl)->kl_lockarg); \ } while (0) #define KNL_ASSERT_UNLOCKED(knl) do { \ knl->kl_assert_unlocked((knl)->kl_lockarg); \ } while (0) #else /* !INVARIANTS */ #define KNL_ASSERT_LOCKED(knl) do {} while(0) #define KNL_ASSERT_UNLOCKED(knl) do {} while (0) #endif /* INVARIANTS */ #ifndef KN_HASHSIZE #define KN_HASHSIZE 64 /* XXX should be tunable */ #endif #define KN_HASH(val, mask) (((val) ^ (val >> 8)) & (mask)) static int filt_nullattach(struct knote *kn) { return (ENXIO); }; struct filterops null_filtops = { .f_isfd = 0, .f_attach = filt_nullattach, }; /* XXX - make SYSINIT to add these, and move into respective modules. */ extern struct filterops sig_filtops; extern struct filterops fs_filtops; /* * Table for for all system-defined filters. */ static struct mtx filterops_lock; MTX_SYSINIT(kqueue_filterops, &filterops_lock, "protect sysfilt_ops", MTX_DEF); static struct { struct filterops *for_fop; int for_nolock; int for_refcnt; } sysfilt_ops[EVFILT_SYSCOUNT] = { { &file_filtops, 1 }, /* EVFILT_READ */ { &file_filtops, 1 }, /* EVFILT_WRITE */ { &null_filtops }, /* EVFILT_AIO */ { &file_filtops, 1 }, /* EVFILT_VNODE */ { &proc_filtops, 1 }, /* EVFILT_PROC */ { &sig_filtops, 1 }, /* EVFILT_SIGNAL */ { &timer_filtops, 1 }, /* EVFILT_TIMER */ { &file_filtops, 1 }, /* EVFILT_PROCDESC */ { &fs_filtops, 1 }, /* EVFILT_FS */ { &null_filtops }, /* EVFILT_LIO */ { &user_filtops, 1 }, /* EVFILT_USER */ { &null_filtops }, /* EVFILT_SENDFILE */ { &file_filtops, 1 }, /* EVFILT_EMPTY */ }; /* * Simple redirection for all cdevsw style objects to call their fo_kqfilter * method. */ static int filt_fileattach(struct knote *kn) { return (fo_kqfilter(kn->kn_fp, kn)); } /*ARGSUSED*/ static int kqueue_kqfilter(struct file *fp, struct knote *kn) { struct kqueue *kq = kn->kn_fp->f_data; if (kn->kn_filter != EVFILT_READ) return (EINVAL); kn->kn_status |= KN_KQUEUE; kn->kn_fop = &kqread_filtops; knlist_add(&kq->kq_sel.si_note, kn, 0); return (0); } static void filt_kqdetach(struct knote *kn) { struct kqueue *kq = kn->kn_fp->f_data; knlist_remove(&kq->kq_sel.si_note, kn, 0); } /*ARGSUSED*/ static int filt_kqueue(struct knote *kn, long hint) { struct kqueue *kq = kn->kn_fp->f_data; kn->kn_data = kq->kq_count; return (kn->kn_data > 0); } /* XXX - move to kern_proc.c? */ static int filt_procattach(struct knote *kn) { struct proc *p; int error; bool exiting, immediate; exiting = immediate = false; if (kn->kn_sfflags & NOTE_EXIT) p = pfind_any(kn->kn_id); else p = pfind(kn->kn_id); if (p == NULL) return (ESRCH); if (p->p_flag & P_WEXIT) exiting = true; if ((error = p_cansee(curthread, p))) { PROC_UNLOCK(p); return (error); } kn->kn_ptr.p_proc = p; kn->kn_flags |= EV_CLEAR; /* automatically set */ /* * Internal flag indicating registration done by kernel for the * purposes of getting a NOTE_CHILD notification. */ if (kn->kn_flags & EV_FLAG2) { kn->kn_flags &= ~EV_FLAG2; kn->kn_data = kn->kn_sdata; /* ppid */ kn->kn_fflags = NOTE_CHILD; kn->kn_sfflags &= ~(NOTE_EXIT | NOTE_EXEC | NOTE_FORK); immediate = true; /* Force immediate activation of child note. */ } /* * Internal flag indicating registration done by kernel (for other than * NOTE_CHILD). */ if (kn->kn_flags & EV_FLAG1) { kn->kn_flags &= ~EV_FLAG1; } knlist_add(p->p_klist, kn, 1); /* * Immediately activate any child notes or, in the case of a zombie * target process, exit notes. The latter is necessary to handle the * case where the target process, e.g. a child, dies before the kevent * is registered. */ if (immediate || (exiting && filt_proc(kn, NOTE_EXIT))) KNOTE_ACTIVATE(kn, 0); PROC_UNLOCK(p); return (0); } /* * The knote may be attached to a different process, which may exit, * leaving nothing for the knote to be attached to. So when the process * exits, the knote is marked as DETACHED and also flagged as ONESHOT so * it will be deleted when read out. However, as part of the knote deletion, * this routine is called, so a check is needed to avoid actually performing * a detach, because the original process does not exist any more. */ /* XXX - move to kern_proc.c? */ static void filt_procdetach(struct knote *kn) { knlist_remove(kn->kn_knlist, kn, 0); kn->kn_ptr.p_proc = NULL; } /* XXX - move to kern_proc.c? */ static int filt_proc(struct knote *kn, long hint) { struct proc *p; u_int event; p = kn->kn_ptr.p_proc; if (p == NULL) /* already activated, from attach filter */ return (0); /* Mask off extra data. */ event = (u_int)hint & NOTE_PCTRLMASK; /* If the user is interested in this event, record it. */ if (kn->kn_sfflags & event) kn->kn_fflags |= event; /* Process is gone, so flag the event as finished. */ if (event == NOTE_EXIT) { kn->kn_flags |= EV_EOF | EV_ONESHOT; kn->kn_ptr.p_proc = NULL; if (kn->kn_fflags & NOTE_EXIT) kn->kn_data = KW_EXITCODE(p->p_xexit, p->p_xsig); if (kn->kn_fflags == 0) kn->kn_flags |= EV_DROP; return (1); } return (kn->kn_fflags != 0); } /* * Called when the process forked. It mostly does the same as the * knote(), activating all knotes registered to be activated when the * process forked. Additionally, for each knote attached to the * parent, check whether user wants to track the new process. If so * attach a new knote to it, and immediately report an event with the * child's pid. */ void knote_fork(struct knlist *list, int pid) { struct kqueue *kq; struct knote *kn; struct kevent kev; int error; MPASS(list != NULL); KNL_ASSERT_LOCKED(list); if (SLIST_EMPTY(&list->kl_list)) return; memset(&kev, 0, sizeof(kev)); SLIST_FOREACH(kn, &list->kl_list, kn_selnext) { kq = kn->kn_kq; KQ_LOCK(kq); if (kn_in_flux(kn) && (kn->kn_status & KN_SCAN) == 0) { KQ_UNLOCK(kq); continue; } /* * The same as knote(), activate the event. */ if ((kn->kn_sfflags & NOTE_TRACK) == 0) { if (kn->kn_fop->f_event(kn, NOTE_FORK)) KNOTE_ACTIVATE(kn, 1); KQ_UNLOCK(kq); continue; } /* * The NOTE_TRACK case. In addition to the activation * of the event, we need to register new events to * track the child. Drop the locks in preparation for * the call to kqueue_register(). */ kn_enter_flux(kn); KQ_UNLOCK(kq); list->kl_unlock(list->kl_lockarg); /* * Activate existing knote and register tracking knotes with * new process. * * First register a knote to get just the child notice. This * must be a separate note from a potential NOTE_EXIT * notification since both NOTE_CHILD and NOTE_EXIT are defined * to use the data field (in conflicting ways). */ kev.ident = pid; kev.filter = kn->kn_filter; kev.flags = kn->kn_flags | EV_ADD | EV_ENABLE | EV_ONESHOT | EV_FLAG2; kev.fflags = kn->kn_sfflags; kev.data = kn->kn_id; /* parent */ kev.udata = kn->kn_kevent.udata;/* preserve udata */ error = kqueue_register(kq, &kev, NULL, M_NOWAIT); if (error) kn->kn_fflags |= NOTE_TRACKERR; /* * Then register another knote to track other potential events * from the new process. */ kev.ident = pid; kev.filter = kn->kn_filter; kev.flags = kn->kn_flags | EV_ADD | EV_ENABLE | EV_FLAG1; kev.fflags = kn->kn_sfflags; kev.data = kn->kn_id; /* parent */ kev.udata = kn->kn_kevent.udata;/* preserve udata */ error = kqueue_register(kq, &kev, NULL, M_NOWAIT); if (error) kn->kn_fflags |= NOTE_TRACKERR; if (kn->kn_fop->f_event(kn, NOTE_FORK)) KNOTE_ACTIVATE(kn, 0); list->kl_lock(list->kl_lockarg); KQ_LOCK(kq); kn_leave_flux(kn); KQ_UNLOCK_FLUX(kq); } } /* * XXX: EVFILT_TIMER should perhaps live in kern_time.c beside the * interval timer support code. */ #define NOTE_TIMER_PRECMASK \ (NOTE_SECONDS | NOTE_MSECONDS | NOTE_USECONDS | NOTE_NSECONDS) static sbintime_t timer2sbintime(intptr_t data, int flags) { int64_t secs; /* * Macros for converting to the fractional second portion of an * sbintime_t using 64bit multiplication to improve precision. */ #define NS_TO_SBT(ns) (((ns) * (((uint64_t)1 << 63) / 500000000)) >> 32) #define US_TO_SBT(us) (((us) * (((uint64_t)1 << 63) / 500000)) >> 32) #define MS_TO_SBT(ms) (((ms) * (((uint64_t)1 << 63) / 500)) >> 32) switch (flags & NOTE_TIMER_PRECMASK) { case NOTE_SECONDS: #ifdef __LP64__ if (data > (SBT_MAX / SBT_1S)) return (SBT_MAX); #endif return ((sbintime_t)data << 32); case NOTE_MSECONDS: /* FALLTHROUGH */ case 0: if (data >= 1000) { secs = data / 1000; #ifdef __LP64__ if (secs > (SBT_MAX / SBT_1S)) return (SBT_MAX); #endif return (secs << 32 | MS_TO_SBT(data % 1000)); } return (MS_TO_SBT(data)); case NOTE_USECONDS: if (data >= 1000000) { secs = data / 1000000; #ifdef __LP64__ if (secs > (SBT_MAX / SBT_1S)) return (SBT_MAX); #endif return (secs << 32 | US_TO_SBT(data % 1000000)); } return (US_TO_SBT(data)); case NOTE_NSECONDS: if (data >= 1000000000) { secs = data / 1000000000; #ifdef __LP64__ if (secs > (SBT_MAX / SBT_1S)) return (SBT_MAX); #endif return (secs << 32 | US_TO_SBT(data % 1000000000)); } return (NS_TO_SBT(data)); default: break; } return (-1); } struct kq_timer_cb_data { struct callout c; sbintime_t next; /* next timer event fires at */ sbintime_t to; /* precalculated timer period, 0 for abs */ }; static void filt_timerexpire(void *knx) { struct knote *kn; struct kq_timer_cb_data *kc; kn = knx; kn->kn_data++; KNOTE_ACTIVATE(kn, 0); /* XXX - handle locking */ if ((kn->kn_flags & EV_ONESHOT) != 0) return; kc = kn->kn_ptr.p_v; if (kc->to == 0) return; kc->next += kc->to; callout_reset_sbt_on(&kc->c, kc->next, 0, filt_timerexpire, kn, PCPU_GET(cpuid), C_ABSOLUTE); } /* * data contains amount of time to sleep */ static int filt_timervalidate(struct knote *kn, sbintime_t *to) { struct bintime bt; sbintime_t sbt; if (kn->kn_sdata < 0) return (EINVAL); if (kn->kn_sdata == 0 && (kn->kn_flags & EV_ONESHOT) == 0) kn->kn_sdata = 1; /* * The only fflags values supported are the timer unit * (precision) and the absolute time indicator. */ if ((kn->kn_sfflags & ~(NOTE_TIMER_PRECMASK | NOTE_ABSTIME)) != 0) return (EINVAL); *to = timer2sbintime(kn->kn_sdata, kn->kn_sfflags); if ((kn->kn_sfflags & NOTE_ABSTIME) != 0) { getboottimebin(&bt); sbt = bttosbt(bt); *to -= sbt; } if (*to < 0) return (EINVAL); return (0); } static int filt_timerattach(struct knote *kn) { struct kq_timer_cb_data *kc; sbintime_t to; unsigned int ncallouts; int error; error = filt_timervalidate(kn, &to); if (error != 0) return (error); do { ncallouts = kq_ncallouts; if (ncallouts >= kq_calloutmax) return (ENOMEM); } while (!atomic_cmpset_int(&kq_ncallouts, ncallouts, ncallouts + 1)); if ((kn->kn_sfflags & NOTE_ABSTIME) == 0) kn->kn_flags |= EV_CLEAR; /* automatically set */ kn->kn_status &= ~KN_DETACHED; /* knlist_add clears it */ kn->kn_ptr.p_v = kc = malloc(sizeof(*kc), M_KQUEUE, M_WAITOK); callout_init(&kc->c, 1); filt_timerstart(kn, to); return (0); } static void filt_timerstart(struct knote *kn, sbintime_t to) { struct kq_timer_cb_data *kc; kc = kn->kn_ptr.p_v; if ((kn->kn_sfflags & NOTE_ABSTIME) != 0) { kc->next = to; kc->to = 0; } else { kc->next = to + sbinuptime(); kc->to = to; } callout_reset_sbt_on(&kc->c, kc->next, 0, filt_timerexpire, kn, PCPU_GET(cpuid), C_ABSOLUTE); } static void filt_timerdetach(struct knote *kn) { struct kq_timer_cb_data *kc; unsigned int old __unused; kc = kn->kn_ptr.p_v; callout_drain(&kc->c); free(kc, M_KQUEUE); old = atomic_fetchadd_int(&kq_ncallouts, -1); KASSERT(old > 0, ("Number of callouts cannot become negative")); kn->kn_status |= KN_DETACHED; /* knlist_remove sets it */ } static void filt_timertouch(struct knote *kn, struct kevent *kev, u_long type) { struct kq_timer_cb_data *kc; struct kqueue *kq; sbintime_t to; int error; switch (type) { case EVENT_REGISTER: /* Handle re-added timers that update data/fflags */ if (kev->flags & EV_ADD) { kc = kn->kn_ptr.p_v; /* Drain any existing callout. */ callout_drain(&kc->c); /* Throw away any existing undelivered record * of the timer expiration. This is done under * the presumption that if a process is * re-adding this timer with new parameters, * it is no longer interested in what may have * happened under the old parameters. If it is * interested, it can wait for the expiration, * delete the old timer definition, and then * add the new one. * * This has to be done while the kq is locked: * - if enqueued, dequeue * - make it no longer active * - clear the count of expiration events */ kq = kn->kn_kq; KQ_LOCK(kq); if (kn->kn_status & KN_QUEUED) knote_dequeue(kn); kn->kn_status &= ~KN_ACTIVE; kn->kn_data = 0; KQ_UNLOCK(kq); /* Reschedule timer based on new data/fflags */ kn->kn_sfflags = kev->fflags; kn->kn_sdata = kev->data; error = filt_timervalidate(kn, &to); if (error != 0) { kn->kn_flags |= EV_ERROR; kn->kn_data = error; } else filt_timerstart(kn, to); } break; case EVENT_PROCESS: *kev = kn->kn_kevent; if (kn->kn_flags & EV_CLEAR) { kn->kn_data = 0; kn->kn_fflags = 0; } break; default: panic("filt_timertouch() - invalid type (%ld)", type); break; } } static int filt_timer(struct knote *kn, long hint) { return (kn->kn_data != 0); } static int filt_userattach(struct knote *kn) { /* * EVFILT_USER knotes are not attached to anything in the kernel. */ kn->kn_hook = NULL; if (kn->kn_fflags & NOTE_TRIGGER) kn->kn_hookid = 1; else kn->kn_hookid = 0; return (0); } static void filt_userdetach(__unused struct knote *kn) { /* * EVFILT_USER knotes are not attached to anything in the kernel. */ } static int filt_user(struct knote *kn, __unused long hint) { return (kn->kn_hookid); } static void filt_usertouch(struct knote *kn, struct kevent *kev, u_long type) { u_int ffctrl; switch (type) { case EVENT_REGISTER: if (kev->fflags & NOTE_TRIGGER) kn->kn_hookid = 1; ffctrl = kev->fflags & NOTE_FFCTRLMASK; kev->fflags &= NOTE_FFLAGSMASK; switch (ffctrl) { case NOTE_FFNOP: break; case NOTE_FFAND: kn->kn_sfflags &= kev->fflags; break; case NOTE_FFOR: kn->kn_sfflags |= kev->fflags; break; case NOTE_FFCOPY: kn->kn_sfflags = kev->fflags; break; default: /* XXX Return error? */ break; } kn->kn_sdata = kev->data; if (kev->flags & EV_CLEAR) { kn->kn_hookid = 0; kn->kn_data = 0; kn->kn_fflags = 0; } break; case EVENT_PROCESS: *kev = kn->kn_kevent; kev->fflags = kn->kn_sfflags; kev->data = kn->kn_sdata; if (kn->kn_flags & EV_CLEAR) { kn->kn_hookid = 0; kn->kn_data = 0; kn->kn_fflags = 0; } break; default: panic("filt_usertouch() - invalid type (%ld)", type); break; } } int sys_kqueue(struct thread *td, struct kqueue_args *uap) { return (kern_kqueue(td, 0, NULL)); } static void kqueue_init(struct kqueue *kq) { mtx_init(&kq->kq_lock, "kqueue", NULL, MTX_DEF | MTX_DUPOK); TAILQ_INIT(&kq->kq_head); knlist_init_mtx(&kq->kq_sel.si_note, &kq->kq_lock); TASK_INIT(&kq->kq_task, 0, kqueue_task, kq); } int kern_kqueue(struct thread *td, int flags, struct filecaps *fcaps) { struct filedesc *fdp; struct kqueue *kq; struct file *fp; struct ucred *cred; int fd, error; fdp = td->td_proc->p_fd; cred = td->td_ucred; if (!chgkqcnt(cred->cr_ruidinfo, 1, lim_cur(td, RLIMIT_KQUEUES))) return (ENOMEM); error = falloc_caps(td, &fp, &fd, flags, fcaps); if (error != 0) { chgkqcnt(cred->cr_ruidinfo, -1, 0); return (error); } /* An extra reference on `fp' has been held for us by falloc(). */ kq = malloc(sizeof *kq, M_KQUEUE, M_WAITOK | M_ZERO); kqueue_init(kq); kq->kq_fdp = fdp; kq->kq_cred = crhold(cred); FILEDESC_XLOCK(fdp); TAILQ_INSERT_HEAD(&fdp->fd_kqlist, kq, kq_list); FILEDESC_XUNLOCK(fdp); finit(fp, FREAD | FWRITE, DTYPE_KQUEUE, kq, &kqueueops); fdrop(fp, td); td->td_retval[0] = fd; return (0); } struct g_kevent_args { int fd; void *changelist; int nchanges; void *eventlist; int nevents; const struct timespec *timeout; }; int sys_kevent(struct thread *td, struct kevent_args *uap) { struct kevent_copyops k_ops = { .arg = uap, .k_copyout = kevent_copyout, .k_copyin = kevent_copyin, .kevent_size = sizeof(struct kevent), }; struct g_kevent_args gk_args = { .fd = uap->fd, .changelist = uap->changelist, .nchanges = uap->nchanges, .eventlist = uap->eventlist, .nevents = uap->nevents, .timeout = uap->timeout, }; return (kern_kevent_generic(td, &gk_args, &k_ops, "kevent")); } static int kern_kevent_generic(struct thread *td, struct g_kevent_args *uap, struct kevent_copyops *k_ops, const char *struct_name) { struct timespec ts, *tsp; #ifdef KTRACE struct kevent *eventlist = uap->eventlist; #endif int error; if (uap->timeout != NULL) { error = copyin(uap->timeout, &ts, sizeof(ts)); if (error) return (error); tsp = &ts; } else tsp = NULL; #ifdef KTRACE if (KTRPOINT(td, KTR_STRUCT_ARRAY)) ktrstructarray(struct_name, UIO_USERSPACE, uap->changelist, uap->nchanges, k_ops->kevent_size); #endif error = kern_kevent(td, uap->fd, uap->nchanges, uap->nevents, k_ops, tsp); #ifdef KTRACE if (error == 0 && KTRPOINT(td, KTR_STRUCT_ARRAY)) ktrstructarray(struct_name, UIO_USERSPACE, eventlist, td->td_retval[0], k_ops->kevent_size); #endif return (error); } /* * Copy 'count' items into the destination list pointed to by uap->eventlist. */ static int kevent_copyout(void *arg, struct kevent *kevp, int count) { struct kevent_args *uap; int error; KASSERT(count <= KQ_NEVENTS, ("count (%d) > KQ_NEVENTS", count)); uap = (struct kevent_args *)arg; error = copyout(kevp, uap->eventlist, count * sizeof *kevp); if (error == 0) uap->eventlist += count; return (error); } /* * Copy 'count' items from the list pointed to by uap->changelist. */ static int kevent_copyin(void *arg, struct kevent *kevp, int count) { struct kevent_args *uap; int error; KASSERT(count <= KQ_NEVENTS, ("count (%d) > KQ_NEVENTS", count)); uap = (struct kevent_args *)arg; error = copyin(uap->changelist, kevp, count * sizeof *kevp); if (error == 0) uap->changelist += count; return (error); } #ifdef COMPAT_FREEBSD11 static int kevent11_copyout(void *arg, struct kevent *kevp, int count) { struct freebsd11_kevent_args *uap; struct kevent_freebsd11 kev11; int error, i; KASSERT(count <= KQ_NEVENTS, ("count (%d) > KQ_NEVENTS", count)); uap = (struct freebsd11_kevent_args *)arg; for (i = 0; i < count; i++) { kev11.ident = kevp->ident; kev11.filter = kevp->filter; kev11.flags = kevp->flags; kev11.fflags = kevp->fflags; kev11.data = kevp->data; kev11.udata = kevp->udata; error = copyout(&kev11, uap->eventlist, sizeof(kev11)); if (error != 0) break; uap->eventlist++; kevp++; } return (error); } /* * Copy 'count' items from the list pointed to by uap->changelist. */ static int kevent11_copyin(void *arg, struct kevent *kevp, int count) { struct freebsd11_kevent_args *uap; struct kevent_freebsd11 kev11; int error, i; KASSERT(count <= KQ_NEVENTS, ("count (%d) > KQ_NEVENTS", count)); uap = (struct freebsd11_kevent_args *)arg; for (i = 0; i < count; i++) { error = copyin(uap->changelist, &kev11, sizeof(kev11)); if (error != 0) break; kevp->ident = kev11.ident; kevp->filter = kev11.filter; kevp->flags = kev11.flags; kevp->fflags = kev11.fflags; kevp->data = (uintptr_t)kev11.data; kevp->udata = kev11.udata; bzero(&kevp->ext, sizeof(kevp->ext)); uap->changelist++; kevp++; } return (error); } int freebsd11_kevent(struct thread *td, struct freebsd11_kevent_args *uap) { struct kevent_copyops k_ops = { .arg = uap, .k_copyout = kevent11_copyout, .k_copyin = kevent11_copyin, .kevent_size = sizeof(struct kevent_freebsd11), }; struct g_kevent_args gk_args = { .fd = uap->fd, .changelist = uap->changelist, .nchanges = uap->nchanges, .eventlist = uap->eventlist, .nevents = uap->nevents, .timeout = uap->timeout, }; return (kern_kevent_generic(td, &gk_args, &k_ops, "kevent_freebsd11")); } #endif int kern_kevent(struct thread *td, int fd, int nchanges, int nevents, struct kevent_copyops *k_ops, const struct timespec *timeout) { cap_rights_t rights; struct file *fp; int error; - cap_rights_init(&rights); + cap_rights_init_zero(&rights); if (nchanges > 0) - cap_rights_set(&rights, CAP_KQUEUE_CHANGE); + cap_rights_set_one(&rights, CAP_KQUEUE_CHANGE); if (nevents > 0) - cap_rights_set(&rights, CAP_KQUEUE_EVENT); + cap_rights_set_one(&rights, CAP_KQUEUE_EVENT); error = fget(td, fd, &rights, &fp); if (error != 0) return (error); error = kern_kevent_fp(td, fp, nchanges, nevents, k_ops, timeout); fdrop(fp, td); return (error); } static int kqueue_kevent(struct kqueue *kq, struct thread *td, int nchanges, int nevents, struct kevent_copyops *k_ops, const struct timespec *timeout) { struct kevent keva[KQ_NEVENTS]; struct kevent *kevp, *changes; int i, n, nerrors, error; nerrors = 0; while (nchanges > 0) { n = nchanges > KQ_NEVENTS ? KQ_NEVENTS : nchanges; error = k_ops->k_copyin(k_ops->arg, keva, n); if (error) return (error); changes = keva; for (i = 0; i < n; i++) { kevp = &changes[i]; if (!kevp->filter) continue; kevp->flags &= ~EV_SYSFLAGS; error = kqueue_register(kq, kevp, td, M_WAITOK); if (error || (kevp->flags & EV_RECEIPT)) { if (nevents == 0) return (error); kevp->flags = EV_ERROR; kevp->data = error; (void)k_ops->k_copyout(k_ops->arg, kevp, 1); nevents--; nerrors++; } } nchanges -= n; } if (nerrors) { td->td_retval[0] = nerrors; return (0); } return (kqueue_scan(kq, nevents, k_ops, timeout, keva, td)); } int kern_kevent_fp(struct thread *td, struct file *fp, int nchanges, int nevents, struct kevent_copyops *k_ops, const struct timespec *timeout) { struct kqueue *kq; int error; error = kqueue_acquire(fp, &kq); if (error != 0) return (error); error = kqueue_kevent(kq, td, nchanges, nevents, k_ops, timeout); kqueue_release(kq, 0); return (error); } /* * Performs a kevent() call on a temporarily created kqueue. This can be * used to perform one-shot polling, similar to poll() and select(). */ int kern_kevent_anonymous(struct thread *td, int nevents, struct kevent_copyops *k_ops) { struct kqueue kq = {}; int error; kqueue_init(&kq); kq.kq_refcnt = 1; error = kqueue_kevent(&kq, td, nevents, nevents, k_ops, NULL); kqueue_drain(&kq, td); kqueue_destroy(&kq); return (error); } int kqueue_add_filteropts(int filt, struct filterops *filtops) { int error; error = 0; if (filt > 0 || filt + EVFILT_SYSCOUNT < 0) { printf( "trying to add a filterop that is out of range: %d is beyond %d\n", ~filt, EVFILT_SYSCOUNT); return EINVAL; } mtx_lock(&filterops_lock); if (sysfilt_ops[~filt].for_fop != &null_filtops && sysfilt_ops[~filt].for_fop != NULL) error = EEXIST; else { sysfilt_ops[~filt].for_fop = filtops; sysfilt_ops[~filt].for_refcnt = 0; } mtx_unlock(&filterops_lock); return (error); } int kqueue_del_filteropts(int filt) { int error; error = 0; if (filt > 0 || filt + EVFILT_SYSCOUNT < 0) return EINVAL; mtx_lock(&filterops_lock); if (sysfilt_ops[~filt].for_fop == &null_filtops || sysfilt_ops[~filt].for_fop == NULL) error = EINVAL; else if (sysfilt_ops[~filt].for_refcnt != 0) error = EBUSY; else { sysfilt_ops[~filt].for_fop = &null_filtops; sysfilt_ops[~filt].for_refcnt = 0; } mtx_unlock(&filterops_lock); return error; } static struct filterops * kqueue_fo_find(int filt) { if (filt > 0 || filt + EVFILT_SYSCOUNT < 0) return NULL; if (sysfilt_ops[~filt].for_nolock) return sysfilt_ops[~filt].for_fop; mtx_lock(&filterops_lock); sysfilt_ops[~filt].for_refcnt++; if (sysfilt_ops[~filt].for_fop == NULL) sysfilt_ops[~filt].for_fop = &null_filtops; mtx_unlock(&filterops_lock); return sysfilt_ops[~filt].for_fop; } static void kqueue_fo_release(int filt) { if (filt > 0 || filt + EVFILT_SYSCOUNT < 0) return; if (sysfilt_ops[~filt].for_nolock) return; mtx_lock(&filterops_lock); KASSERT(sysfilt_ops[~filt].for_refcnt > 0, ("filter object refcount not valid on release")); sysfilt_ops[~filt].for_refcnt--; mtx_unlock(&filterops_lock); } /* * A ref to kq (obtained via kqueue_acquire) must be held. */ static int kqueue_register(struct kqueue *kq, struct kevent *kev, struct thread *td, int mflag) { struct filterops *fops; struct file *fp; struct knote *kn, *tkn; struct knlist *knl; int error, filt, event; int haskqglobal, filedesc_unlock; if ((kev->flags & (EV_ENABLE | EV_DISABLE)) == (EV_ENABLE | EV_DISABLE)) return (EINVAL); fp = NULL; kn = NULL; knl = NULL; error = 0; haskqglobal = 0; filedesc_unlock = 0; filt = kev->filter; fops = kqueue_fo_find(filt); if (fops == NULL) return EINVAL; if (kev->flags & EV_ADD) { /* * Prevent waiting with locks. Non-sleepable * allocation failures are handled in the loop, only * if the spare knote appears to be actually required. */ tkn = knote_alloc(mflag); } else { tkn = NULL; } findkn: if (fops->f_isfd) { KASSERT(td != NULL, ("td is NULL")); if (kev->ident > INT_MAX) error = EBADF; else error = fget(td, kev->ident, &cap_event_rights, &fp); if (error) goto done; if ((kev->flags & EV_ADD) == EV_ADD && kqueue_expand(kq, fops, kev->ident, M_NOWAIT) != 0) { /* try again */ fdrop(fp, td); fp = NULL; error = kqueue_expand(kq, fops, kev->ident, mflag); if (error) goto done; goto findkn; } if (fp->f_type == DTYPE_KQUEUE) { /* * If we add some intelligence about what we are doing, * we should be able to support events on ourselves. * We need to know when we are doing this to prevent * getting both the knlist lock and the kq lock since * they are the same thing. */ if (fp->f_data == kq) { error = EINVAL; goto done; } /* * Pre-lock the filedesc before the global * lock mutex, see the comment in * kqueue_close(). */ FILEDESC_XLOCK(td->td_proc->p_fd); filedesc_unlock = 1; KQ_GLOBAL_LOCK(&kq_global, haskqglobal); } KQ_LOCK(kq); if (kev->ident < kq->kq_knlistsize) { SLIST_FOREACH(kn, &kq->kq_knlist[kev->ident], kn_link) if (kev->filter == kn->kn_filter) break; } } else { if ((kev->flags & EV_ADD) == EV_ADD) { error = kqueue_expand(kq, fops, kev->ident, mflag); if (error != 0) goto done; } KQ_LOCK(kq); /* * If possible, find an existing knote to use for this kevent. */ if (kev->filter == EVFILT_PROC && (kev->flags & (EV_FLAG1 | EV_FLAG2)) != 0) { /* This is an internal creation of a process tracking * note. Don't attempt to coalesce this with an * existing note. */ ; } else if (kq->kq_knhashmask != 0) { struct klist *list; list = &kq->kq_knhash[ KN_HASH((u_long)kev->ident, kq->kq_knhashmask)]; SLIST_FOREACH(kn, list, kn_link) if (kev->ident == kn->kn_id && kev->filter == kn->kn_filter) break; } } /* knote is in the process of changing, wait for it to stabilize. */ if (kn != NULL && kn_in_flux(kn)) { KQ_GLOBAL_UNLOCK(&kq_global, haskqglobal); if (filedesc_unlock) { FILEDESC_XUNLOCK(td->td_proc->p_fd); filedesc_unlock = 0; } kq->kq_state |= KQ_FLUXWAIT; msleep(kq, &kq->kq_lock, PSOCK | PDROP, "kqflxwt", 0); if (fp != NULL) { fdrop(fp, td); fp = NULL; } goto findkn; } /* * kn now contains the matching knote, or NULL if no match */ if (kn == NULL) { if (kev->flags & EV_ADD) { kn = tkn; tkn = NULL; if (kn == NULL) { KQ_UNLOCK(kq); error = ENOMEM; goto done; } kn->kn_fp = fp; kn->kn_kq = kq; kn->kn_fop = fops; /* * apply reference counts to knote structure, and * do not release it at the end of this routine. */ fops = NULL; fp = NULL; kn->kn_sfflags = kev->fflags; kn->kn_sdata = kev->data; kev->fflags = 0; kev->data = 0; kn->kn_kevent = *kev; kn->kn_kevent.flags &= ~(EV_ADD | EV_DELETE | EV_ENABLE | EV_DISABLE | EV_FORCEONESHOT); kn->kn_status = KN_DETACHED; if ((kev->flags & EV_DISABLE) != 0) kn->kn_status |= KN_DISABLED; kn_enter_flux(kn); error = knote_attach(kn, kq); KQ_UNLOCK(kq); if (error != 0) { tkn = kn; goto done; } if ((error = kn->kn_fop->f_attach(kn)) != 0) { knote_drop_detached(kn, td); goto done; } knl = kn_list_lock(kn); goto done_ev_add; } else { /* No matching knote and the EV_ADD flag is not set. */ KQ_UNLOCK(kq); error = ENOENT; goto done; } } if (kev->flags & EV_DELETE) { kn_enter_flux(kn); KQ_UNLOCK(kq); knote_drop(kn, td); goto done; } if (kev->flags & EV_FORCEONESHOT) { kn->kn_flags |= EV_ONESHOT; KNOTE_ACTIVATE(kn, 1); } if ((kev->flags & EV_ENABLE) != 0) kn->kn_status &= ~KN_DISABLED; else if ((kev->flags & EV_DISABLE) != 0) kn->kn_status |= KN_DISABLED; /* * The user may change some filter values after the initial EV_ADD, * but doing so will not reset any filter which has already been * triggered. */ kn->kn_status |= KN_SCAN; kn_enter_flux(kn); KQ_UNLOCK(kq); knl = kn_list_lock(kn); kn->kn_kevent.udata = kev->udata; if (!fops->f_isfd && fops->f_touch != NULL) { fops->f_touch(kn, kev, EVENT_REGISTER); } else { kn->kn_sfflags = kev->fflags; kn->kn_sdata = kev->data; } done_ev_add: /* * We can get here with kn->kn_knlist == NULL. This can happen when * the initial attach event decides that the event is "completed" * already, e.g., filt_procattach() is called on a zombie process. It * will call filt_proc() which will remove it from the list, and NULL * kn_knlist. * * KN_DISABLED will be stable while the knote is in flux, so the * unlocked read will not race with an update. */ if ((kn->kn_status & KN_DISABLED) == 0) event = kn->kn_fop->f_event(kn, 0); else event = 0; KQ_LOCK(kq); if (event) kn->kn_status |= KN_ACTIVE; if ((kn->kn_status & (KN_ACTIVE | KN_DISABLED | KN_QUEUED)) == KN_ACTIVE) knote_enqueue(kn); kn->kn_status &= ~KN_SCAN; kn_leave_flux(kn); kn_list_unlock(knl); KQ_UNLOCK_FLUX(kq); done: KQ_GLOBAL_UNLOCK(&kq_global, haskqglobal); if (filedesc_unlock) FILEDESC_XUNLOCK(td->td_proc->p_fd); if (fp != NULL) fdrop(fp, td); knote_free(tkn); if (fops != NULL) kqueue_fo_release(filt); return (error); } static int kqueue_acquire(struct file *fp, struct kqueue **kqp) { int error; struct kqueue *kq; error = 0; kq = fp->f_data; if (fp->f_type != DTYPE_KQUEUE || kq == NULL) return (EBADF); *kqp = kq; KQ_LOCK(kq); if ((kq->kq_state & KQ_CLOSING) == KQ_CLOSING) { KQ_UNLOCK(kq); return (EBADF); } kq->kq_refcnt++; KQ_UNLOCK(kq); return error; } static void kqueue_release(struct kqueue *kq, int locked) { if (locked) KQ_OWNED(kq); else KQ_LOCK(kq); kq->kq_refcnt--; if (kq->kq_refcnt == 1) wakeup(&kq->kq_refcnt); if (!locked) KQ_UNLOCK(kq); } static void kqueue_schedtask(struct kqueue *kq) { KQ_OWNED(kq); KASSERT(((kq->kq_state & KQ_TASKDRAIN) != KQ_TASKDRAIN), ("scheduling kqueue task while draining")); if ((kq->kq_state & KQ_TASKSCHED) != KQ_TASKSCHED) { taskqueue_enqueue(taskqueue_kqueue_ctx, &kq->kq_task); kq->kq_state |= KQ_TASKSCHED; } } /* * Expand the kq to make sure we have storage for fops/ident pair. * * Return 0 on success (or no work necessary), return errno on failure. */ static int kqueue_expand(struct kqueue *kq, struct filterops *fops, uintptr_t ident, int mflag) { struct klist *list, *tmp_knhash, *to_free; u_long tmp_knhashmask; int error, fd, size; KQ_NOTOWNED(kq); error = 0; to_free = NULL; if (fops->f_isfd) { fd = ident; if (kq->kq_knlistsize <= fd) { size = kq->kq_knlistsize; while (size <= fd) size += KQEXTENT; list = malloc(size * sizeof(*list), M_KQUEUE, mflag); if (list == NULL) return ENOMEM; KQ_LOCK(kq); if ((kq->kq_state & KQ_CLOSING) != 0) { to_free = list; error = EBADF; } else if (kq->kq_knlistsize > fd) { to_free = list; } else { if (kq->kq_knlist != NULL) { bcopy(kq->kq_knlist, list, kq->kq_knlistsize * sizeof(*list)); to_free = kq->kq_knlist; kq->kq_knlist = NULL; } bzero((caddr_t)list + kq->kq_knlistsize * sizeof(*list), (size - kq->kq_knlistsize) * sizeof(*list)); kq->kq_knlistsize = size; kq->kq_knlist = list; } KQ_UNLOCK(kq); } } else { if (kq->kq_knhashmask == 0) { tmp_knhash = hashinit_flags(KN_HASHSIZE, M_KQUEUE, &tmp_knhashmask, (mflag & M_WAITOK) != 0 ? HASH_WAITOK : HASH_NOWAIT); if (tmp_knhash == NULL) return (ENOMEM); KQ_LOCK(kq); if ((kq->kq_state & KQ_CLOSING) != 0) { to_free = tmp_knhash; error = EBADF; } else if (kq->kq_knhashmask == 0) { kq->kq_knhash = tmp_knhash; kq->kq_knhashmask = tmp_knhashmask; } else { to_free = tmp_knhash; } KQ_UNLOCK(kq); } } free(to_free, M_KQUEUE); KQ_NOTOWNED(kq); return (error); } static void kqueue_task(void *arg, int pending) { struct kqueue *kq; int haskqglobal; haskqglobal = 0; kq = arg; KQ_GLOBAL_LOCK(&kq_global, haskqglobal); KQ_LOCK(kq); KNOTE_LOCKED(&kq->kq_sel.si_note, 0); kq->kq_state &= ~KQ_TASKSCHED; if ((kq->kq_state & KQ_TASKDRAIN) == KQ_TASKDRAIN) { wakeup(&kq->kq_state); } KQ_UNLOCK(kq); KQ_GLOBAL_UNLOCK(&kq_global, haskqglobal); } /* * Scan, update kn_data (if not ONESHOT), and copyout triggered events. * We treat KN_MARKER knotes as if they are in flux. */ static int kqueue_scan(struct kqueue *kq, int maxevents, struct kevent_copyops *k_ops, const struct timespec *tsp, struct kevent *keva, struct thread *td) { struct kevent *kevp; struct knote *kn, *marker; struct knlist *knl; sbintime_t asbt, rsbt; int count, error, haskqglobal, influx, nkev, touch; count = maxevents; nkev = 0; error = 0; haskqglobal = 0; if (maxevents == 0) goto done_nl; rsbt = 0; if (tsp != NULL) { if (tsp->tv_sec < 0 || tsp->tv_nsec < 0 || tsp->tv_nsec >= 1000000000) { error = EINVAL; goto done_nl; } if (timespecisset(tsp)) { if (tsp->tv_sec <= INT32_MAX) { rsbt = tstosbt(*tsp); if (TIMESEL(&asbt, rsbt)) asbt += tc_tick_sbt; if (asbt <= SBT_MAX - rsbt) asbt += rsbt; else asbt = 0; rsbt >>= tc_precexp; } else asbt = 0; } else asbt = -1; } else asbt = 0; marker = knote_alloc(M_WAITOK); marker->kn_status = KN_MARKER; KQ_LOCK(kq); retry: kevp = keva; if (kq->kq_count == 0) { if (asbt == -1) { error = EWOULDBLOCK; } else { kq->kq_state |= KQ_SLEEP; error = msleep_sbt(kq, &kq->kq_lock, PSOCK | PCATCH, "kqread", asbt, rsbt, C_ABSOLUTE); } if (error == 0) goto retry; /* don't restart after signals... */ if (error == ERESTART) error = EINTR; else if (error == EWOULDBLOCK) error = 0; goto done; } TAILQ_INSERT_TAIL(&kq->kq_head, marker, kn_tqe); influx = 0; while (count) { KQ_OWNED(kq); kn = TAILQ_FIRST(&kq->kq_head); if ((kn->kn_status == KN_MARKER && kn != marker) || kn_in_flux(kn)) { if (influx) { influx = 0; KQ_FLUX_WAKEUP(kq); } kq->kq_state |= KQ_FLUXWAIT; error = msleep(kq, &kq->kq_lock, PSOCK, "kqflxwt", 0); continue; } TAILQ_REMOVE(&kq->kq_head, kn, kn_tqe); if ((kn->kn_status & KN_DISABLED) == KN_DISABLED) { kn->kn_status &= ~KN_QUEUED; kq->kq_count--; continue; } if (kn == marker) { KQ_FLUX_WAKEUP(kq); if (count == maxevents) goto retry; goto done; } KASSERT(!kn_in_flux(kn), ("knote %p is unexpectedly in flux", kn)); if ((kn->kn_flags & EV_DROP) == EV_DROP) { kn->kn_status &= ~KN_QUEUED; kn_enter_flux(kn); kq->kq_count--; KQ_UNLOCK(kq); /* * We don't need to lock the list since we've * marked it as in flux. */ knote_drop(kn, td); KQ_LOCK(kq); continue; } else if ((kn->kn_flags & EV_ONESHOT) == EV_ONESHOT) { kn->kn_status &= ~KN_QUEUED; kn_enter_flux(kn); kq->kq_count--; KQ_UNLOCK(kq); /* * We don't need to lock the list since we've * marked the knote as being in flux. */ *kevp = kn->kn_kevent; knote_drop(kn, td); KQ_LOCK(kq); kn = NULL; } else { kn->kn_status |= KN_SCAN; kn_enter_flux(kn); KQ_UNLOCK(kq); if ((kn->kn_status & KN_KQUEUE) == KN_KQUEUE) KQ_GLOBAL_LOCK(&kq_global, haskqglobal); knl = kn_list_lock(kn); if (kn->kn_fop->f_event(kn, 0) == 0) { KQ_LOCK(kq); KQ_GLOBAL_UNLOCK(&kq_global, haskqglobal); kn->kn_status &= ~(KN_QUEUED | KN_ACTIVE | KN_SCAN); kn_leave_flux(kn); kq->kq_count--; kn_list_unlock(knl); influx = 1; continue; } touch = (!kn->kn_fop->f_isfd && kn->kn_fop->f_touch != NULL); if (touch) kn->kn_fop->f_touch(kn, kevp, EVENT_PROCESS); else *kevp = kn->kn_kevent; KQ_LOCK(kq); KQ_GLOBAL_UNLOCK(&kq_global, haskqglobal); if (kn->kn_flags & (EV_CLEAR | EV_DISPATCH)) { /* * Manually clear knotes who weren't * 'touch'ed. */ if (touch == 0 && kn->kn_flags & EV_CLEAR) { kn->kn_data = 0; kn->kn_fflags = 0; } if (kn->kn_flags & EV_DISPATCH) kn->kn_status |= KN_DISABLED; kn->kn_status &= ~(KN_QUEUED | KN_ACTIVE); kq->kq_count--; } else TAILQ_INSERT_TAIL(&kq->kq_head, kn, kn_tqe); kn->kn_status &= ~KN_SCAN; kn_leave_flux(kn); kn_list_unlock(knl); influx = 1; } /* we are returning a copy to the user */ kevp++; nkev++; count--; if (nkev == KQ_NEVENTS) { influx = 0; KQ_UNLOCK_FLUX(kq); error = k_ops->k_copyout(k_ops->arg, keva, nkev); nkev = 0; kevp = keva; KQ_LOCK(kq); if (error) break; } } TAILQ_REMOVE(&kq->kq_head, marker, kn_tqe); done: KQ_OWNED(kq); KQ_UNLOCK_FLUX(kq); knote_free(marker); done_nl: KQ_NOTOWNED(kq); if (nkev != 0) error = k_ops->k_copyout(k_ops->arg, keva, nkev); td->td_retval[0] = maxevents - count; return (error); } /*ARGSUSED*/ static int kqueue_ioctl(struct file *fp, u_long cmd, void *data, struct ucred *active_cred, struct thread *td) { /* * Enabling sigio causes two major problems: * 1) infinite recursion: * Synopsys: kevent is being used to track signals and have FIOASYNC * set. On receipt of a signal this will cause a kqueue to recurse * into itself over and over. Sending the sigio causes the kqueue * to become ready, which in turn posts sigio again, forever. * Solution: this can be solved by setting a flag in the kqueue that * we have a SIGIO in progress. * 2) locking problems: * Synopsys: Kqueue is a leaf subsystem, but adding signalling puts * us above the proc and pgrp locks. * Solution: Post a signal using an async mechanism, being sure to * record a generation count in the delivery so that we do not deliver * a signal to the wrong process. * * Note, these two mechanisms are somewhat mutually exclusive! */ #if 0 struct kqueue *kq; kq = fp->f_data; switch (cmd) { case FIOASYNC: if (*(int *)data) { kq->kq_state |= KQ_ASYNC; } else { kq->kq_state &= ~KQ_ASYNC; } return (0); case FIOSETOWN: return (fsetown(*(int *)data, &kq->kq_sigio)); case FIOGETOWN: *(int *)data = fgetown(&kq->kq_sigio); return (0); } #endif return (ENOTTY); } /*ARGSUSED*/ static int kqueue_poll(struct file *fp, int events, struct ucred *active_cred, struct thread *td) { struct kqueue *kq; int revents = 0; int error; if ((error = kqueue_acquire(fp, &kq))) return POLLERR; KQ_LOCK(kq); if (events & (POLLIN | POLLRDNORM)) { if (kq->kq_count) { revents |= events & (POLLIN | POLLRDNORM); } else { selrecord(td, &kq->kq_sel); if (SEL_WAITING(&kq->kq_sel)) kq->kq_state |= KQ_SEL; } } kqueue_release(kq, 1); KQ_UNLOCK(kq); return (revents); } /*ARGSUSED*/ static int kqueue_stat(struct file *fp, struct stat *st, struct ucred *active_cred, struct thread *td) { bzero((void *)st, sizeof *st); /* * We no longer return kq_count because the unlocked value is useless. * If you spent all this time getting the count, why not spend your * syscall better by calling kevent? * * XXX - This is needed for libc_r. */ st->st_mode = S_IFIFO; return (0); } static void kqueue_drain(struct kqueue *kq, struct thread *td) { struct knote *kn; int i; KQ_LOCK(kq); KASSERT((kq->kq_state & KQ_CLOSING) != KQ_CLOSING, ("kqueue already closing")); kq->kq_state |= KQ_CLOSING; if (kq->kq_refcnt > 1) msleep(&kq->kq_refcnt, &kq->kq_lock, PSOCK, "kqclose", 0); KASSERT(kq->kq_refcnt == 1, ("other refs are out there!")); KASSERT(knlist_empty(&kq->kq_sel.si_note), ("kqueue's knlist not empty")); for (i = 0; i < kq->kq_knlistsize; i++) { while ((kn = SLIST_FIRST(&kq->kq_knlist[i])) != NULL) { if (kn_in_flux(kn)) { kq->kq_state |= KQ_FLUXWAIT; msleep(kq, &kq->kq_lock, PSOCK, "kqclo1", 0); continue; } kn_enter_flux(kn); KQ_UNLOCK(kq); knote_drop(kn, td); KQ_LOCK(kq); } } if (kq->kq_knhashmask != 0) { for (i = 0; i <= kq->kq_knhashmask; i++) { while ((kn = SLIST_FIRST(&kq->kq_knhash[i])) != NULL) { if (kn_in_flux(kn)) { kq->kq_state |= KQ_FLUXWAIT; msleep(kq, &kq->kq_lock, PSOCK, "kqclo2", 0); continue; } kn_enter_flux(kn); KQ_UNLOCK(kq); knote_drop(kn, td); KQ_LOCK(kq); } } } if ((kq->kq_state & KQ_TASKSCHED) == KQ_TASKSCHED) { kq->kq_state |= KQ_TASKDRAIN; msleep(&kq->kq_state, &kq->kq_lock, PSOCK, "kqtqdr", 0); } if ((kq->kq_state & KQ_SEL) == KQ_SEL) { selwakeuppri(&kq->kq_sel, PSOCK); if (!SEL_WAITING(&kq->kq_sel)) kq->kq_state &= ~KQ_SEL; } KQ_UNLOCK(kq); } static void kqueue_destroy(struct kqueue *kq) { KASSERT(kq->kq_fdp == NULL, ("kqueue still attached to a file descriptor")); seldrain(&kq->kq_sel); knlist_destroy(&kq->kq_sel.si_note); mtx_destroy(&kq->kq_lock); if (kq->kq_knhash != NULL) free(kq->kq_knhash, M_KQUEUE); if (kq->kq_knlist != NULL) free(kq->kq_knlist, M_KQUEUE); funsetown(&kq->kq_sigio); } /*ARGSUSED*/ static int kqueue_close(struct file *fp, struct thread *td) { struct kqueue *kq = fp->f_data; struct filedesc *fdp; int error; int filedesc_unlock; if ((error = kqueue_acquire(fp, &kq))) return error; kqueue_drain(kq, td); /* * We could be called due to the knote_drop() doing fdrop(), * called from kqueue_register(). In this case the global * lock is owned, and filedesc sx is locked before, to not * take the sleepable lock after non-sleepable. */ fdp = kq->kq_fdp; kq->kq_fdp = NULL; if (!sx_xlocked(FILEDESC_LOCK(fdp))) { FILEDESC_XLOCK(fdp); filedesc_unlock = 1; } else filedesc_unlock = 0; TAILQ_REMOVE(&fdp->fd_kqlist, kq, kq_list); if (filedesc_unlock) FILEDESC_XUNLOCK(fdp); kqueue_destroy(kq); chgkqcnt(kq->kq_cred->cr_ruidinfo, -1, 0); crfree(kq->kq_cred); free(kq, M_KQUEUE); fp->f_data = NULL; return (0); } static int kqueue_fill_kinfo(struct file *fp, struct kinfo_file *kif, struct filedesc *fdp) { kif->kf_type = KF_TYPE_KQUEUE; return (0); } static void kqueue_wakeup(struct kqueue *kq) { KQ_OWNED(kq); if ((kq->kq_state & KQ_SLEEP) == KQ_SLEEP) { kq->kq_state &= ~KQ_SLEEP; wakeup(kq); } if ((kq->kq_state & KQ_SEL) == KQ_SEL) { selwakeuppri(&kq->kq_sel, PSOCK); if (!SEL_WAITING(&kq->kq_sel)) kq->kq_state &= ~KQ_SEL; } if (!knlist_empty(&kq->kq_sel.si_note)) kqueue_schedtask(kq); if ((kq->kq_state & KQ_ASYNC) == KQ_ASYNC) { pgsigio(&kq->kq_sigio, SIGIO, 0); } } /* * Walk down a list of knotes, activating them if their event has triggered. * * There is a possibility to optimize in the case of one kq watching another. * Instead of scheduling a task to wake it up, you could pass enough state * down the chain to make up the parent kqueue. Make this code functional * first. */ void knote(struct knlist *list, long hint, int lockflags) { struct kqueue *kq; struct knote *kn, *tkn; int error; if (list == NULL) return; KNL_ASSERT_LOCK(list, lockflags & KNF_LISTLOCKED); if ((lockflags & KNF_LISTLOCKED) == 0) list->kl_lock(list->kl_lockarg); /* * If we unlock the list lock (and enter influx), we can * eliminate the kqueue scheduling, but this will introduce * four lock/unlock's for each knote to test. Also, marker * would be needed to keep iteration position, since filters * or other threads could remove events. */ SLIST_FOREACH_SAFE(kn, &list->kl_list, kn_selnext, tkn) { kq = kn->kn_kq; KQ_LOCK(kq); if (kn_in_flux(kn) && (kn->kn_status & KN_SCAN) == 0) { /* * Do not process the influx notes, except for * the influx coming from the kq unlock in the * kqueue_scan(). In the later case, we do * not interfere with the scan, since the code * fragment in kqueue_scan() locks the knlist, * and cannot proceed until we finished. */ KQ_UNLOCK(kq); } else if ((lockflags & KNF_NOKQLOCK) != 0) { kn_enter_flux(kn); KQ_UNLOCK(kq); error = kn->kn_fop->f_event(kn, hint); KQ_LOCK(kq); kn_leave_flux(kn); if (error) KNOTE_ACTIVATE(kn, 1); KQ_UNLOCK_FLUX(kq); } else { if (kn->kn_fop->f_event(kn, hint)) KNOTE_ACTIVATE(kn, 1); KQ_UNLOCK(kq); } } if ((lockflags & KNF_LISTLOCKED) == 0) list->kl_unlock(list->kl_lockarg); } /* * add a knote to a knlist */ void knlist_add(struct knlist *knl, struct knote *kn, int islocked) { KNL_ASSERT_LOCK(knl, islocked); KQ_NOTOWNED(kn->kn_kq); KASSERT(kn_in_flux(kn), ("knote %p not in flux", kn)); KASSERT((kn->kn_status & KN_DETACHED) != 0, ("knote %p was not detached", kn)); if (!islocked) knl->kl_lock(knl->kl_lockarg); SLIST_INSERT_HEAD(&knl->kl_list, kn, kn_selnext); if (!islocked) knl->kl_unlock(knl->kl_lockarg); KQ_LOCK(kn->kn_kq); kn->kn_knlist = knl; kn->kn_status &= ~KN_DETACHED; KQ_UNLOCK(kn->kn_kq); } static void knlist_remove_kq(struct knlist *knl, struct knote *kn, int knlislocked, int kqislocked) { KASSERT(!kqislocked || knlislocked, ("kq locked w/o knl locked")); KNL_ASSERT_LOCK(knl, knlislocked); mtx_assert(&kn->kn_kq->kq_lock, kqislocked ? MA_OWNED : MA_NOTOWNED); KASSERT(kqislocked || kn_in_flux(kn), ("knote %p not in flux", kn)); KASSERT((kn->kn_status & KN_DETACHED) == 0, ("knote %p was already detached", kn)); if (!knlislocked) knl->kl_lock(knl->kl_lockarg); SLIST_REMOVE(&knl->kl_list, kn, knote, kn_selnext); kn->kn_knlist = NULL; if (!knlislocked) kn_list_unlock(knl); if (!kqislocked) KQ_LOCK(kn->kn_kq); kn->kn_status |= KN_DETACHED; if (!kqislocked) KQ_UNLOCK(kn->kn_kq); } /* * remove knote from the specified knlist */ void knlist_remove(struct knlist *knl, struct knote *kn, int islocked) { knlist_remove_kq(knl, kn, islocked, 0); } int knlist_empty(struct knlist *knl) { KNL_ASSERT_LOCKED(knl); return (SLIST_EMPTY(&knl->kl_list)); } static struct mtx knlist_lock; MTX_SYSINIT(knlist_lock, &knlist_lock, "knlist lock for lockless objects", MTX_DEF); static void knlist_mtx_lock(void *arg); static void knlist_mtx_unlock(void *arg); static void knlist_mtx_lock(void *arg) { mtx_lock((struct mtx *)arg); } static void knlist_mtx_unlock(void *arg) { mtx_unlock((struct mtx *)arg); } static void knlist_mtx_assert_locked(void *arg) { mtx_assert((struct mtx *)arg, MA_OWNED); } static void knlist_mtx_assert_unlocked(void *arg) { mtx_assert((struct mtx *)arg, MA_NOTOWNED); } static void knlist_rw_rlock(void *arg) { rw_rlock((struct rwlock *)arg); } static void knlist_rw_runlock(void *arg) { rw_runlock((struct rwlock *)arg); } static void knlist_rw_assert_locked(void *arg) { rw_assert((struct rwlock *)arg, RA_LOCKED); } static void knlist_rw_assert_unlocked(void *arg) { rw_assert((struct rwlock *)arg, RA_UNLOCKED); } void knlist_init(struct knlist *knl, void *lock, void (*kl_lock)(void *), void (*kl_unlock)(void *), void (*kl_assert_locked)(void *), void (*kl_assert_unlocked)(void *)) { if (lock == NULL) knl->kl_lockarg = &knlist_lock; else knl->kl_lockarg = lock; if (kl_lock == NULL) knl->kl_lock = knlist_mtx_lock; else knl->kl_lock = kl_lock; if (kl_unlock == NULL) knl->kl_unlock = knlist_mtx_unlock; else knl->kl_unlock = kl_unlock; if (kl_assert_locked == NULL) knl->kl_assert_locked = knlist_mtx_assert_locked; else knl->kl_assert_locked = kl_assert_locked; if (kl_assert_unlocked == NULL) knl->kl_assert_unlocked = knlist_mtx_assert_unlocked; else knl->kl_assert_unlocked = kl_assert_unlocked; knl->kl_autodestroy = 0; SLIST_INIT(&knl->kl_list); } void knlist_init_mtx(struct knlist *knl, struct mtx *lock) { knlist_init(knl, lock, NULL, NULL, NULL, NULL); } struct knlist * knlist_alloc(struct mtx *lock) { struct knlist *knl; knl = malloc(sizeof(struct knlist), M_KQUEUE, M_WAITOK); knlist_init_mtx(knl, lock); return (knl); } void knlist_init_rw_reader(struct knlist *knl, struct rwlock *lock) { knlist_init(knl, lock, knlist_rw_rlock, knlist_rw_runlock, knlist_rw_assert_locked, knlist_rw_assert_unlocked); } void knlist_destroy(struct knlist *knl) { KASSERT(KNLIST_EMPTY(knl), ("destroying knlist %p with knotes on it", knl)); } void knlist_detach(struct knlist *knl) { KNL_ASSERT_LOCKED(knl); knl->kl_autodestroy = 1; if (knlist_empty(knl)) { knlist_destroy(knl); free(knl, M_KQUEUE); } } /* * Even if we are locked, we may need to drop the lock to allow any influx * knotes time to "settle". */ void knlist_cleardel(struct knlist *knl, struct thread *td, int islocked, int killkn) { struct knote *kn, *kn2; struct kqueue *kq; KASSERT(!knl->kl_autodestroy, ("cleardel for autodestroy %p", knl)); if (islocked) KNL_ASSERT_LOCKED(knl); else { KNL_ASSERT_UNLOCKED(knl); again: /* need to reacquire lock since we have dropped it */ knl->kl_lock(knl->kl_lockarg); } SLIST_FOREACH_SAFE(kn, &knl->kl_list, kn_selnext, kn2) { kq = kn->kn_kq; KQ_LOCK(kq); if (kn_in_flux(kn)) { KQ_UNLOCK(kq); continue; } knlist_remove_kq(knl, kn, 1, 1); if (killkn) { kn_enter_flux(kn); KQ_UNLOCK(kq); knote_drop_detached(kn, td); } else { /* Make sure cleared knotes disappear soon */ kn->kn_flags |= EV_EOF | EV_ONESHOT; KQ_UNLOCK(kq); } kq = NULL; } if (!SLIST_EMPTY(&knl->kl_list)) { /* there are still in flux knotes remaining */ kn = SLIST_FIRST(&knl->kl_list); kq = kn->kn_kq; KQ_LOCK(kq); KASSERT(kn_in_flux(kn), ("knote removed w/o list lock")); knl->kl_unlock(knl->kl_lockarg); kq->kq_state |= KQ_FLUXWAIT; msleep(kq, &kq->kq_lock, PSOCK | PDROP, "kqkclr", 0); kq = NULL; goto again; } if (islocked) KNL_ASSERT_LOCKED(knl); else { knl->kl_unlock(knl->kl_lockarg); KNL_ASSERT_UNLOCKED(knl); } } /* * Remove all knotes referencing a specified fd must be called with FILEDESC * lock. This prevents a race where a new fd comes along and occupies the * entry and we attach a knote to the fd. */ void knote_fdclose(struct thread *td, int fd) { struct filedesc *fdp = td->td_proc->p_fd; struct kqueue *kq; struct knote *kn; int influx; FILEDESC_XLOCK_ASSERT(fdp); /* * We shouldn't have to worry about new kevents appearing on fd * since filedesc is locked. */ TAILQ_FOREACH(kq, &fdp->fd_kqlist, kq_list) { KQ_LOCK(kq); again: influx = 0; while (kq->kq_knlistsize > fd && (kn = SLIST_FIRST(&kq->kq_knlist[fd])) != NULL) { if (kn_in_flux(kn)) { /* someone else might be waiting on our knote */ if (influx) wakeup(kq); kq->kq_state |= KQ_FLUXWAIT; msleep(kq, &kq->kq_lock, PSOCK, "kqflxwt", 0); goto again; } kn_enter_flux(kn); KQ_UNLOCK(kq); influx = 1; knote_drop(kn, td); KQ_LOCK(kq); } KQ_UNLOCK_FLUX(kq); } } static int knote_attach(struct knote *kn, struct kqueue *kq) { struct klist *list; KASSERT(kn_in_flux(kn), ("knote %p not marked influx", kn)); KQ_OWNED(kq); if ((kq->kq_state & KQ_CLOSING) != 0) return (EBADF); if (kn->kn_fop->f_isfd) { if (kn->kn_id >= kq->kq_knlistsize) return (ENOMEM); list = &kq->kq_knlist[kn->kn_id]; } else { if (kq->kq_knhash == NULL) return (ENOMEM); list = &kq->kq_knhash[KN_HASH(kn->kn_id, kq->kq_knhashmask)]; } SLIST_INSERT_HEAD(list, kn, kn_link); return (0); } static void knote_drop(struct knote *kn, struct thread *td) { if ((kn->kn_status & KN_DETACHED) == 0) kn->kn_fop->f_detach(kn); knote_drop_detached(kn, td); } static void knote_drop_detached(struct knote *kn, struct thread *td) { struct kqueue *kq; struct klist *list; kq = kn->kn_kq; KASSERT((kn->kn_status & KN_DETACHED) != 0, ("knote %p still attached", kn)); KQ_NOTOWNED(kq); KQ_LOCK(kq); KASSERT(kn->kn_influx == 1, ("knote_drop called on %p with influx %d", kn, kn->kn_influx)); if (kn->kn_fop->f_isfd) list = &kq->kq_knlist[kn->kn_id]; else list = &kq->kq_knhash[KN_HASH(kn->kn_id, kq->kq_knhashmask)]; if (!SLIST_EMPTY(list)) SLIST_REMOVE(list, kn, knote, kn_link); if (kn->kn_status & KN_QUEUED) knote_dequeue(kn); KQ_UNLOCK_FLUX(kq); if (kn->kn_fop->f_isfd) { fdrop(kn->kn_fp, td); kn->kn_fp = NULL; } kqueue_fo_release(kn->kn_kevent.filter); kn->kn_fop = NULL; knote_free(kn); } static void knote_enqueue(struct knote *kn) { struct kqueue *kq = kn->kn_kq; KQ_OWNED(kn->kn_kq); KASSERT((kn->kn_status & KN_QUEUED) == 0, ("knote already queued")); TAILQ_INSERT_TAIL(&kq->kq_head, kn, kn_tqe); kn->kn_status |= KN_QUEUED; kq->kq_count++; kqueue_wakeup(kq); } static void knote_dequeue(struct knote *kn) { struct kqueue *kq = kn->kn_kq; KQ_OWNED(kn->kn_kq); KASSERT(kn->kn_status & KN_QUEUED, ("knote not queued")); TAILQ_REMOVE(&kq->kq_head, kn, kn_tqe); kn->kn_status &= ~KN_QUEUED; kq->kq_count--; } static void knote_init(void) { knote_zone = uma_zcreate("KNOTE", sizeof(struct knote), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); } SYSINIT(knote, SI_SUB_PSEUDO, SI_ORDER_ANY, knote_init, NULL); static struct knote * knote_alloc(int mflag) { return (uma_zalloc(knote_zone, mflag | M_ZERO)); } static void knote_free(struct knote *kn) { uma_zfree(knote_zone, kn); } /* * Register the kev w/ the kq specified by fd. */ int kqfd_register(int fd, struct kevent *kev, struct thread *td, int mflag) { struct kqueue *kq; struct file *fp; cap_rights_t rights; int error; error = fget(td, fd, cap_rights_init(&rights, CAP_KQUEUE_CHANGE), &fp); if (error != 0) return (error); if ((error = kqueue_acquire(fp, &kq)) != 0) goto noacquire; error = kqueue_register(kq, kev, td, mflag); kqueue_release(kq, 0); noacquire: fdrop(fp, td); return (error); } Index: projects/clang1000-import/sys/kern/kern_tc.c =================================================================== --- projects/clang1000-import/sys/kern/kern_tc.c (revision 357965) +++ projects/clang1000-import/sys/kern/kern_tc.c (revision 357966) @@ -1,2224 +1,2166 @@ /*- * SPDX-License-Identifier: Beerware * * ---------------------------------------------------------------------------- * "THE BEER-WARE LICENSE" (Revision 42): * wrote this file. As long as you retain this notice you * can do whatever you want with this stuff. If we meet some day, and you think * this stuff is worth it, you can buy me a beer in return. Poul-Henning Kamp * ---------------------------------------------------------------------------- * * Copyright (c) 2011, 2015, 2016 The FreeBSD Foundation * All rights reserved. * * Portions of this software were developed by Julien Ridoux at the University * of Melbourne under sponsorship from the FreeBSD Foundation. * * Portions of this software were developed by Konstantin Belousov * under sponsorship from the FreeBSD Foundation. */ #include __FBSDID("$FreeBSD$"); #include "opt_ntp.h" #include "opt_ffclock.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* * A large step happens on boot. This constant detects such steps. * It is relatively small so that ntp_update_second gets called enough * in the typical 'missed a couple of seconds' case, but doesn't loop * forever when the time step is large. */ #define LARGE_STEP 200 /* * Implement a dummy timecounter which we can use until we get a real one * in the air. This allows the console and other early stuff to use * time services. */ static u_int dummy_get_timecount(struct timecounter *tc) { static u_int now; return (++now); } static struct timecounter dummy_timecounter = { dummy_get_timecount, 0, ~0u, 1000000, "dummy", -1000000 }; struct timehands { /* These fields must be initialized by the driver. */ struct timecounter *th_counter; int64_t th_adjustment; uint64_t th_scale; + u_int th_large_delta; u_int th_offset_count; struct bintime th_offset; struct bintime th_bintime; struct timeval th_microtime; struct timespec th_nanotime; struct bintime th_boottime; /* Fields not to be copied in tc_windup start with th_generation. */ u_int th_generation; struct timehands *th_next; }; static struct timehands ths[16] = { [0] = { .th_counter = &dummy_timecounter, .th_scale = (uint64_t)-1 / 1000000, + .th_large_delta = 1000000, .th_offset = { .sec = 1 }, .th_generation = 1, }, }; static struct timehands *volatile timehands = &ths[0]; struct timecounter *timecounter = &dummy_timecounter; static struct timecounter *timecounters = &dummy_timecounter; int tc_min_ticktock_freq = 1; volatile time_t time_second = 1; volatile time_t time_uptime = 1; static int sysctl_kern_boottime(SYSCTL_HANDLER_ARGS); SYSCTL_PROC(_kern, KERN_BOOTTIME, boottime, CTLTYPE_STRUCT|CTLFLAG_RD, NULL, 0, sysctl_kern_boottime, "S,timeval", "System boottime"); SYSCTL_NODE(_kern, OID_AUTO, timecounter, CTLFLAG_RW, 0, ""); static SYSCTL_NODE(_kern_timecounter, OID_AUTO, tc, CTLFLAG_RW, 0, ""); static int timestepwarnings; SYSCTL_INT(_kern_timecounter, OID_AUTO, stepwarnings, CTLFLAG_RW, ×tepwarnings, 0, "Log time steps"); static int timehands_count = 2; SYSCTL_INT(_kern_timecounter, OID_AUTO, timehands_count, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, &timehands_count, 0, "Count of timehands in rotation"); struct bintime bt_timethreshold; struct bintime bt_tickthreshold; sbintime_t sbt_timethreshold; sbintime_t sbt_tickthreshold; struct bintime tc_tick_bt; sbintime_t tc_tick_sbt; int tc_precexp; int tc_timepercentage = TC_DEFAULTPERC; static int sysctl_kern_timecounter_adjprecision(SYSCTL_HANDLER_ARGS); SYSCTL_PROC(_kern_timecounter, OID_AUTO, alloweddeviation, CTLTYPE_INT | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, 0, 0, sysctl_kern_timecounter_adjprecision, "I", "Allowed time interval deviation in percents"); volatile int rtc_generation = 1; static int tc_chosen; /* Non-zero if a specific tc was chosen via sysctl. */ static void tc_windup(struct bintime *new_boottimebin); static void cpu_tick_calibrate(int); void dtrace_getnanotime(struct timespec *tsp); static int sysctl_kern_boottime(SYSCTL_HANDLER_ARGS) { struct timeval boottime; getboottime(&boottime); /* i386 is the only arch which uses a 32bits time_t */ #ifdef __amd64__ #ifdef SCTL_MASK32 int tv[2]; if (req->flags & SCTL_MASK32) { tv[0] = boottime.tv_sec; tv[1] = boottime.tv_usec; return (SYSCTL_OUT(req, tv, sizeof(tv))); } #endif #endif return (SYSCTL_OUT(req, &boottime, sizeof(boottime))); } static int sysctl_kern_timecounter_get(SYSCTL_HANDLER_ARGS) { u_int ncount; struct timecounter *tc = arg1; ncount = tc->tc_get_timecount(tc); return (sysctl_handle_int(oidp, &ncount, 0, req)); } static int sysctl_kern_timecounter_freq(SYSCTL_HANDLER_ARGS) { uint64_t freq; struct timecounter *tc = arg1; freq = tc->tc_frequency; return (sysctl_handle_64(oidp, &freq, 0, req)); } /* * Return the difference between the timehands' counter value now and what * was when we copied it to the timehands' offset_count. */ static __inline u_int tc_delta(struct timehands *th) { struct timecounter *tc; tc = th->th_counter; return ((tc->tc_get_timecount(tc) - th->th_offset_count) & tc->tc_counter_mask); } /* * Functions for reading the time. We have to loop until we are sure that * the timehands that we operated on was not updated under our feet. See * the comment in for a description of these 12 functions. */ -#ifdef FFCLOCK -void -fbclock_binuptime(struct bintime *bt) +static __inline void +bintime_off(struct bintime *bt, u_int off) { struct timehands *th; - unsigned int gen; + struct bintime *btp; + uint64_t scale, x; + u_int delta, gen, large_delta; do { th = timehands; gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_offset; - bintime_addx(bt, th->th_scale * tc_delta(th)); + btp = (struct bintime *)((vm_offset_t)th + off); + *bt = *btp; + scale = th->th_scale; + delta = tc_delta(th); + large_delta = th->th_large_delta; atomic_thread_fence_acq(); } while (gen == 0 || gen != th->th_generation); + + if (__predict_false(delta >= large_delta)) { + /* Avoid overflow for scale * delta. */ + x = (scale >> 32) * delta; + bt->sec += x >> 32; + bintime_addx(bt, x << 32); + bintime_addx(bt, (scale & 0xffffffff) * delta); + } else { + bintime_addx(bt, scale * delta); + } } +#define GETTHBINTIME(dst, member) \ +do { \ + _Static_assert(_Generic(((struct timehands *)NULL)->member, \ + struct bintime: 1, default: 0) == 1, \ + "struct timehands member is not of struct bintime type"); \ + bintime_off(dst, __offsetof(struct timehands, member)); \ +} while (0) +static __inline void +getthmember(void *out, size_t out_size, u_int off) +{ + struct timehands *th; + u_int gen; + + do { + th = timehands; + gen = atomic_load_acq_int(&th->th_generation); + memcpy(out, (char *)th + off, out_size); + atomic_thread_fence_acq(); + } while (gen == 0 || gen != th->th_generation); +} +#define GETTHMEMBER(dst, member) \ +do { \ + _Static_assert(_Generic(*dst, \ + __typeof(((struct timehands *)NULL)->member): 1, \ + default: 0) == 1, \ + "*dst and struct timehands member have different types"); \ + getthmember(dst, sizeof(*dst), __offsetof(struct timehands, \ + member)); \ +} while (0) + +#ifdef FFCLOCK void +fbclock_binuptime(struct bintime *bt) +{ + + GETTHBINTIME(bt, th_offset); +} + +void fbclock_nanouptime(struct timespec *tsp) { struct bintime bt; fbclock_binuptime(&bt); bintime2timespec(&bt, tsp); } void fbclock_microuptime(struct timeval *tvp) { struct bintime bt; fbclock_binuptime(&bt); bintime2timeval(&bt, tvp); } void fbclock_bintime(struct bintime *bt) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_bintime; - bintime_addx(bt, th->th_scale * tc_delta(th)); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHBINTIME(bt, th_bintime); } void fbclock_nanotime(struct timespec *tsp) { struct bintime bt; fbclock_bintime(&bt); bintime2timespec(&bt, tsp); } void fbclock_microtime(struct timeval *tvp) { struct bintime bt; fbclock_bintime(&bt); bintime2timeval(&bt, tvp); } void fbclock_getbinuptime(struct bintime *bt) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_offset; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(bt, th_offset); } void fbclock_getnanouptime(struct timespec *tsp) { - struct timehands *th; - unsigned int gen; + struct bintime bt; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - bintime2timespec(&th->th_offset, tsp); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(&bt, th_offset); + bintime2timespec(&bt, tsp); } void fbclock_getmicrouptime(struct timeval *tvp) { - struct timehands *th; - unsigned int gen; + struct bintime bt; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - bintime2timeval(&th->th_offset, tvp); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(&bt, th_offset); + bintime2timeval(&bt, tvp); } void fbclock_getbintime(struct bintime *bt) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_bintime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(bt, th_bintime); } void fbclock_getnanotime(struct timespec *tsp) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tsp = th->th_nanotime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tsp, th_nanotime); } void fbclock_getmicrotime(struct timeval *tvp) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tvp = th->th_microtime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tvp, th_microtime); } #else /* !FFCLOCK */ + void binuptime(struct bintime *bt) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_offset; - bintime_addx(bt, th->th_scale * tc_delta(th)); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHBINTIME(bt, th_offset); } void nanouptime(struct timespec *tsp) { struct bintime bt; binuptime(&bt); bintime2timespec(&bt, tsp); } void microuptime(struct timeval *tvp) { struct bintime bt; binuptime(&bt); bintime2timeval(&bt, tvp); } void bintime(struct bintime *bt) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_bintime; - bintime_addx(bt, th->th_scale * tc_delta(th)); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHBINTIME(bt, th_bintime); } void nanotime(struct timespec *tsp) { struct bintime bt; bintime(&bt); bintime2timespec(&bt, tsp); } void microtime(struct timeval *tvp) { struct bintime bt; bintime(&bt); bintime2timeval(&bt, tvp); } void getbinuptime(struct bintime *bt) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_offset; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(bt, th_offset); } void getnanouptime(struct timespec *tsp) { - struct timehands *th; - u_int gen; + struct bintime bt; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - bintime2timespec(&th->th_offset, tsp); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(&bt, th_offset); + bintime2timespec(&bt, tsp); } void getmicrouptime(struct timeval *tvp) { - struct timehands *th; - u_int gen; + struct bintime bt; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - bintime2timeval(&th->th_offset, tvp); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(&bt, th_offset); + bintime2timeval(&bt, tvp); } void getbintime(struct bintime *bt) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_bintime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(bt, th_bintime); } void getnanotime(struct timespec *tsp) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tsp = th->th_nanotime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tsp, th_nanotime); } void getmicrotime(struct timeval *tvp) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tvp = th->th_microtime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tvp, th_microtime); } #endif /* FFCLOCK */ void getboottime(struct timeval *boottime) { struct bintime boottimebin; getboottimebin(&boottimebin); bintime2timeval(&boottimebin, boottime); } void getboottimebin(struct bintime *boottimebin) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *boottimebin = th->th_boottime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(boottimebin, th_boottime); } #ifdef FFCLOCK /* * Support for feed-forward synchronization algorithms. This is heavily inspired * by the timehands mechanism but kept independent from it. *_windup() functions * have some connection to avoid accessing the timecounter hardware more than * necessary. */ /* Feed-forward clock estimates kept updated by the synchronization daemon. */ struct ffclock_estimate ffclock_estimate; struct bintime ffclock_boottime; /* Feed-forward boot time estimate. */ uint32_t ffclock_status; /* Feed-forward clock status. */ int8_t ffclock_updated; /* New estimates are available. */ struct mtx ffclock_mtx; /* Mutex on ffclock_estimate. */ struct fftimehands { struct ffclock_estimate cest; struct bintime tick_time; struct bintime tick_time_lerp; ffcounter tick_ffcount; uint64_t period_lerp; volatile uint8_t gen; struct fftimehands *next; }; #define NUM_ELEMENTS(x) (sizeof(x) / sizeof(*x)) static struct fftimehands ffth[10]; static struct fftimehands *volatile fftimehands = ffth; static void ffclock_init(void) { struct fftimehands *cur; struct fftimehands *last; memset(ffth, 0, sizeof(ffth)); last = ffth + NUM_ELEMENTS(ffth) - 1; for (cur = ffth; cur < last; cur++) cur->next = cur + 1; last->next = ffth; ffclock_updated = 0; ffclock_status = FFCLOCK_STA_UNSYNC; mtx_init(&ffclock_mtx, "ffclock lock", NULL, MTX_DEF); } /* * Reset the feed-forward clock estimates. Called from inittodr() to get things * kick started and uses the timecounter nominal frequency as a first period * estimate. Note: this function may be called several time just after boot. * Note: this is the only function that sets the value of boot time for the * monotonic (i.e. uptime) version of the feed-forward clock. */ void ffclock_reset_clock(struct timespec *ts) { struct timecounter *tc; struct ffclock_estimate cest; tc = timehands->th_counter; memset(&cest, 0, sizeof(struct ffclock_estimate)); timespec2bintime(ts, &ffclock_boottime); timespec2bintime(ts, &(cest.update_time)); ffclock_read_counter(&cest.update_ffcount); cest.leapsec_next = 0; cest.period = ((1ULL << 63) / tc->tc_frequency) << 1; cest.errb_abs = 0; cest.errb_rate = 0; cest.status = FFCLOCK_STA_UNSYNC; cest.leapsec_total = 0; cest.leapsec = 0; mtx_lock(&ffclock_mtx); bcopy(&cest, &ffclock_estimate, sizeof(struct ffclock_estimate)); ffclock_updated = INT8_MAX; mtx_unlock(&ffclock_mtx); printf("ffclock reset: %s (%llu Hz), time = %ld.%09lu\n", tc->tc_name, (unsigned long long)tc->tc_frequency, (long)ts->tv_sec, (unsigned long)ts->tv_nsec); } /* * Sub-routine to convert a time interval measured in RAW counter units to time * in seconds stored in bintime format. * NOTE: bintime_mul requires u_int, but the value of the ffcounter may be * larger than the max value of u_int (on 32 bit architecture). Loop to consume * extra cycles. */ static void ffclock_convert_delta(ffcounter ffdelta, uint64_t period, struct bintime *bt) { struct bintime bt2; ffcounter delta, delta_max; delta_max = (1ULL << (8 * sizeof(unsigned int))) - 1; bintime_clear(bt); do { if (ffdelta > delta_max) delta = delta_max; else delta = ffdelta; bt2.sec = 0; bt2.frac = period; bintime_mul(&bt2, (unsigned int)delta); bintime_add(bt, &bt2); ffdelta -= delta; } while (ffdelta > 0); } /* * Update the fftimehands. * Push the tick ffcount and time(s) forward based on current clock estimate. * The conversion from ffcounter to bintime relies on the difference clock * principle, whose accuracy relies on computing small time intervals. If a new * clock estimate has been passed by the synchronisation daemon, make it * current, and compute the linear interpolation for monotonic time if needed. */ static void ffclock_windup(unsigned int delta) { struct ffclock_estimate *cest; struct fftimehands *ffth; struct bintime bt, gap_lerp; ffcounter ffdelta; uint64_t frac; unsigned int polling; uint8_t forward_jump, ogen; /* * Pick the next timehand, copy current ffclock estimates and move tick * times and counter forward. */ forward_jump = 0; ffth = fftimehands->next; ogen = ffth->gen; ffth->gen = 0; cest = &ffth->cest; bcopy(&fftimehands->cest, cest, sizeof(struct ffclock_estimate)); ffdelta = (ffcounter)delta; ffth->period_lerp = fftimehands->period_lerp; ffth->tick_time = fftimehands->tick_time; ffclock_convert_delta(ffdelta, cest->period, &bt); bintime_add(&ffth->tick_time, &bt); ffth->tick_time_lerp = fftimehands->tick_time_lerp; ffclock_convert_delta(ffdelta, ffth->period_lerp, &bt); bintime_add(&ffth->tick_time_lerp, &bt); ffth->tick_ffcount = fftimehands->tick_ffcount + ffdelta; /* * Assess the status of the clock, if the last update is too old, it is * likely the synchronisation daemon is dead and the clock is free * running. */ if (ffclock_updated == 0) { ffdelta = ffth->tick_ffcount - cest->update_ffcount; ffclock_convert_delta(ffdelta, cest->period, &bt); if (bt.sec > 2 * FFCLOCK_SKM_SCALE) ffclock_status |= FFCLOCK_STA_UNSYNC; } /* * If available, grab updated clock estimates and make them current. * Recompute time at this tick using the updated estimates. The clock * estimates passed the feed-forward synchronisation daemon may result * in time conversion that is not monotonically increasing (just after * the update). time_lerp is a particular linear interpolation over the * synchronisation algo polling period that ensures monotonicity for the * clock ids requesting it. */ if (ffclock_updated > 0) { bcopy(&ffclock_estimate, cest, sizeof(struct ffclock_estimate)); ffdelta = ffth->tick_ffcount - cest->update_ffcount; ffth->tick_time = cest->update_time; ffclock_convert_delta(ffdelta, cest->period, &bt); bintime_add(&ffth->tick_time, &bt); /* ffclock_reset sets ffclock_updated to INT8_MAX */ if (ffclock_updated == INT8_MAX) ffth->tick_time_lerp = ffth->tick_time; if (bintime_cmp(&ffth->tick_time, &ffth->tick_time_lerp, >)) forward_jump = 1; else forward_jump = 0; bintime_clear(&gap_lerp); if (forward_jump) { gap_lerp = ffth->tick_time; bintime_sub(&gap_lerp, &ffth->tick_time_lerp); } else { gap_lerp = ffth->tick_time_lerp; bintime_sub(&gap_lerp, &ffth->tick_time); } /* * The reset from the RTC clock may be far from accurate, and * reducing the gap between real time and interpolated time * could take a very long time if the interpolated clock insists * on strict monotonicity. The clock is reset under very strict * conditions (kernel time is known to be wrong and * synchronization daemon has been restarted recently. * ffclock_boottime absorbs the jump to ensure boot time is * correct and uptime functions stay consistent. */ if (((ffclock_status & FFCLOCK_STA_UNSYNC) == FFCLOCK_STA_UNSYNC) && ((cest->status & FFCLOCK_STA_UNSYNC) == 0) && ((cest->status & FFCLOCK_STA_WARMUP) == FFCLOCK_STA_WARMUP)) { if (forward_jump) bintime_add(&ffclock_boottime, &gap_lerp); else bintime_sub(&ffclock_boottime, &gap_lerp); ffth->tick_time_lerp = ffth->tick_time; bintime_clear(&gap_lerp); } ffclock_status = cest->status; ffth->period_lerp = cest->period; /* * Compute corrected period used for the linear interpolation of * time. The rate of linear interpolation is capped to 5000PPM * (5ms/s). */ if (bintime_isset(&gap_lerp)) { ffdelta = cest->update_ffcount; ffdelta -= fftimehands->cest.update_ffcount; ffclock_convert_delta(ffdelta, cest->period, &bt); polling = bt.sec; bt.sec = 0; bt.frac = 5000000 * (uint64_t)18446744073LL; bintime_mul(&bt, polling); if (bintime_cmp(&gap_lerp, &bt, >)) gap_lerp = bt; /* Approximate 1 sec by 1-(1/2^64) to ease arithmetic */ frac = 0; if (gap_lerp.sec > 0) { frac -= 1; frac /= ffdelta / gap_lerp.sec; } frac += gap_lerp.frac / ffdelta; if (forward_jump) ffth->period_lerp += frac; else ffth->period_lerp -= frac; } ffclock_updated = 0; } if (++ogen == 0) ogen = 1; ffth->gen = ogen; fftimehands = ffth; } /* * Adjust the fftimehands when the timecounter is changed. Stating the obvious, * the old and new hardware counter cannot be read simultaneously. tc_windup() * does read the two counters 'back to back', but a few cycles are effectively * lost, and not accumulated in tick_ffcount. This is a fairly radical * operation for a feed-forward synchronization daemon, and it is its job to not * pushing irrelevant data to the kernel. Because there is no locking here, * simply force to ignore pending or next update to give daemon a chance to * realize the counter has changed. */ static void ffclock_change_tc(struct timehands *th) { struct fftimehands *ffth; struct ffclock_estimate *cest; struct timecounter *tc; uint8_t ogen; tc = th->th_counter; ffth = fftimehands->next; ogen = ffth->gen; ffth->gen = 0; cest = &ffth->cest; bcopy(&(fftimehands->cest), cest, sizeof(struct ffclock_estimate)); cest->period = ((1ULL << 63) / tc->tc_frequency ) << 1; cest->errb_abs = 0; cest->errb_rate = 0; cest->status |= FFCLOCK_STA_UNSYNC; ffth->tick_ffcount = fftimehands->tick_ffcount; ffth->tick_time_lerp = fftimehands->tick_time_lerp; ffth->tick_time = fftimehands->tick_time; ffth->period_lerp = cest->period; /* Do not lock but ignore next update from synchronization daemon. */ ffclock_updated--; if (++ogen == 0) ogen = 1; ffth->gen = ogen; fftimehands = ffth; } /* * Retrieve feed-forward counter and time of last kernel tick. */ void ffclock_last_tick(ffcounter *ffcount, struct bintime *bt, uint32_t flags) { struct fftimehands *ffth; uint8_t gen; /* * No locking but check generation has not changed. Also need to make * sure ffdelta is positive, i.e. ffcount > tick_ffcount. */ do { ffth = fftimehands; gen = ffth->gen; if ((flags & FFCLOCK_LERP) == FFCLOCK_LERP) *bt = ffth->tick_time_lerp; else *bt = ffth->tick_time; *ffcount = ffth->tick_ffcount; } while (gen == 0 || gen != ffth->gen); } /* * Absolute clock conversion. Low level function to convert ffcounter to * bintime. The ffcounter is converted using the current ffclock period estimate * or the "interpolated period" to ensure monotonicity. * NOTE: this conversion may have been deferred, and the clock updated since the * hardware counter has been read. */ void ffclock_convert_abs(ffcounter ffcount, struct bintime *bt, uint32_t flags) { struct fftimehands *ffth; struct bintime bt2; ffcounter ffdelta; uint8_t gen; /* * No locking but check generation has not changed. Also need to make * sure ffdelta is positive, i.e. ffcount > tick_ffcount. */ do { ffth = fftimehands; gen = ffth->gen; if (ffcount > ffth->tick_ffcount) ffdelta = ffcount - ffth->tick_ffcount; else ffdelta = ffth->tick_ffcount - ffcount; if ((flags & FFCLOCK_LERP) == FFCLOCK_LERP) { *bt = ffth->tick_time_lerp; ffclock_convert_delta(ffdelta, ffth->period_lerp, &bt2); } else { *bt = ffth->tick_time; ffclock_convert_delta(ffdelta, ffth->cest.period, &bt2); } if (ffcount > ffth->tick_ffcount) bintime_add(bt, &bt2); else bintime_sub(bt, &bt2); } while (gen == 0 || gen != ffth->gen); } /* * Difference clock conversion. * Low level function to Convert a time interval measured in RAW counter units * into bintime. The difference clock allows measuring small intervals much more * reliably than the absolute clock. */ void ffclock_convert_diff(ffcounter ffdelta, struct bintime *bt) { struct fftimehands *ffth; uint8_t gen; /* No locking but check generation has not changed. */ do { ffth = fftimehands; gen = ffth->gen; ffclock_convert_delta(ffdelta, ffth->cest.period, bt); } while (gen == 0 || gen != ffth->gen); } /* * Access to current ffcounter value. */ void ffclock_read_counter(ffcounter *ffcount) { struct timehands *th; struct fftimehands *ffth; unsigned int gen, delta; /* * ffclock_windup() called from tc_windup(), safe to rely on * th->th_generation only, for correct delta and ffcounter. */ do { th = timehands; gen = atomic_load_acq_int(&th->th_generation); ffth = fftimehands; delta = tc_delta(th); *ffcount = ffth->tick_ffcount; atomic_thread_fence_acq(); } while (gen == 0 || gen != th->th_generation); *ffcount += delta; } void binuptime(struct bintime *bt) { binuptime_fromclock(bt, sysclock_active); } void nanouptime(struct timespec *tsp) { nanouptime_fromclock(tsp, sysclock_active); } void microuptime(struct timeval *tvp) { microuptime_fromclock(tvp, sysclock_active); } void bintime(struct bintime *bt) { bintime_fromclock(bt, sysclock_active); } void nanotime(struct timespec *tsp) { nanotime_fromclock(tsp, sysclock_active); } void microtime(struct timeval *tvp) { microtime_fromclock(tvp, sysclock_active); } void getbinuptime(struct bintime *bt) { getbinuptime_fromclock(bt, sysclock_active); } void getnanouptime(struct timespec *tsp) { getnanouptime_fromclock(tsp, sysclock_active); } void getmicrouptime(struct timeval *tvp) { getmicrouptime_fromclock(tvp, sysclock_active); } void getbintime(struct bintime *bt) { getbintime_fromclock(bt, sysclock_active); } void getnanotime(struct timespec *tsp) { getnanotime_fromclock(tsp, sysclock_active); } void getmicrotime(struct timeval *tvp) { getmicrouptime_fromclock(tvp, sysclock_active); } #endif /* FFCLOCK */ /* * This is a clone of getnanotime and used for walltimestamps. * The dtrace_ prefix prevents fbt from creating probes for * it so walltimestamp can be safely used in all fbt probes. */ void dtrace_getnanotime(struct timespec *tsp) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tsp = th->th_nanotime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tsp, th_nanotime); } /* * System clock currently providing time to the system. Modifiable via sysctl * when the FFCLOCK option is defined. */ int sysclock_active = SYSCLOCK_FBCK; /* Internal NTP status and error estimates. */ extern int time_status; extern long time_esterror; /* * Take a snapshot of sysclock data which can be used to compare system clocks * and generate timestamps after the fact. */ void sysclock_getsnapshot(struct sysclock_snap *clock_snap, int fast) { struct fbclock_info *fbi; struct timehands *th; struct bintime bt; unsigned int delta, gen; #ifdef FFCLOCK ffcounter ffcount; struct fftimehands *ffth; struct ffclock_info *ffi; struct ffclock_estimate cest; ffi = &clock_snap->ff_info; #endif fbi = &clock_snap->fb_info; delta = 0; do { th = timehands; gen = atomic_load_acq_int(&th->th_generation); fbi->th_scale = th->th_scale; fbi->tick_time = th->th_offset; #ifdef FFCLOCK ffth = fftimehands; ffi->tick_time = ffth->tick_time_lerp; ffi->tick_time_lerp = ffth->tick_time_lerp; ffi->period = ffth->cest.period; ffi->period_lerp = ffth->period_lerp; clock_snap->ffcount = ffth->tick_ffcount; cest = ffth->cest; #endif if (!fast) delta = tc_delta(th); atomic_thread_fence_acq(); } while (gen == 0 || gen != th->th_generation); clock_snap->delta = delta; clock_snap->sysclock_active = sysclock_active; /* Record feedback clock status and error. */ clock_snap->fb_info.status = time_status; /* XXX: Very crude estimate of feedback clock error. */ bt.sec = time_esterror / 1000000; bt.frac = ((time_esterror - bt.sec) * 1000000) * (uint64_t)18446744073709ULL; clock_snap->fb_info.error = bt; #ifdef FFCLOCK if (!fast) clock_snap->ffcount += delta; /* Record feed-forward clock leap second adjustment. */ ffi->leapsec_adjustment = cest.leapsec_total; if (clock_snap->ffcount > cest.leapsec_next) ffi->leapsec_adjustment -= cest.leapsec; /* Record feed-forward clock status and error. */ clock_snap->ff_info.status = cest.status; ffcount = clock_snap->ffcount - cest.update_ffcount; ffclock_convert_delta(ffcount, cest.period, &bt); /* 18446744073709 = int(2^64/1e12), err_bound_rate in [ps/s]. */ bintime_mul(&bt, cest.errb_rate * (uint64_t)18446744073709ULL); /* 18446744073 = int(2^64 / 1e9), since err_abs in [ns]. */ bintime_addx(&bt, cest.errb_abs * (uint64_t)18446744073ULL); clock_snap->ff_info.error = bt; #endif } /* * Convert a sysclock snapshot into a struct bintime based on the specified * clock source and flags. */ int sysclock_snap2bintime(struct sysclock_snap *cs, struct bintime *bt, int whichclock, uint32_t flags) { struct bintime boottimebin; #ifdef FFCLOCK struct bintime bt2; uint64_t period; #endif switch (whichclock) { case SYSCLOCK_FBCK: *bt = cs->fb_info.tick_time; /* If snapshot was created with !fast, delta will be >0. */ if (cs->delta > 0) bintime_addx(bt, cs->fb_info.th_scale * cs->delta); if ((flags & FBCLOCK_UPTIME) == 0) { getboottimebin(&boottimebin); bintime_add(bt, &boottimebin); } break; #ifdef FFCLOCK case SYSCLOCK_FFWD: if (flags & FFCLOCK_LERP) { *bt = cs->ff_info.tick_time_lerp; period = cs->ff_info.period_lerp; } else { *bt = cs->ff_info.tick_time; period = cs->ff_info.period; } /* If snapshot was created with !fast, delta will be >0. */ if (cs->delta > 0) { ffclock_convert_delta(cs->delta, period, &bt2); bintime_add(bt, &bt2); } /* Leap second adjustment. */ if (flags & FFCLOCK_LEAPSEC) bt->sec -= cs->ff_info.leapsec_adjustment; /* Boot time adjustment, for uptime/monotonic clocks. */ if (flags & FFCLOCK_UPTIME) bintime_sub(bt, &ffclock_boottime); break; #endif default: return (EINVAL); break; } return (0); } /* * Initialize a new timecounter and possibly use it. */ void tc_init(struct timecounter *tc) { u_int u; struct sysctl_oid *tc_root; u = tc->tc_frequency / tc->tc_counter_mask; /* XXX: We need some margin here, 10% is a guess */ u *= 11; u /= 10; if (u > hz && tc->tc_quality >= 0) { tc->tc_quality = -2000; if (bootverbose) { printf("Timecounter \"%s\" frequency %ju Hz", tc->tc_name, (uintmax_t)tc->tc_frequency); printf(" -- Insufficient hz, needs at least %u\n", u); } } else if (tc->tc_quality >= 0 || bootverbose) { printf("Timecounter \"%s\" frequency %ju Hz quality %d\n", tc->tc_name, (uintmax_t)tc->tc_frequency, tc->tc_quality); } tc->tc_next = timecounters; timecounters = tc; /* * Set up sysctl tree for this counter. */ tc_root = SYSCTL_ADD_NODE_WITH_LABEL(NULL, SYSCTL_STATIC_CHILDREN(_kern_timecounter_tc), OID_AUTO, tc->tc_name, CTLFLAG_RW, 0, "timecounter description", "timecounter"); SYSCTL_ADD_UINT(NULL, SYSCTL_CHILDREN(tc_root), OID_AUTO, "mask", CTLFLAG_RD, &(tc->tc_counter_mask), 0, "mask for implemented bits"); SYSCTL_ADD_PROC(NULL, SYSCTL_CHILDREN(tc_root), OID_AUTO, "counter", CTLTYPE_UINT | CTLFLAG_RD, tc, sizeof(*tc), sysctl_kern_timecounter_get, "IU", "current timecounter value"); SYSCTL_ADD_PROC(NULL, SYSCTL_CHILDREN(tc_root), OID_AUTO, "frequency", CTLTYPE_U64 | CTLFLAG_RD, tc, sizeof(*tc), sysctl_kern_timecounter_freq, "QU", "timecounter frequency"); SYSCTL_ADD_INT(NULL, SYSCTL_CHILDREN(tc_root), OID_AUTO, "quality", CTLFLAG_RD, &(tc->tc_quality), 0, "goodness of time counter"); /* * Do not automatically switch if the current tc was specifically * chosen. Never automatically use a timecounter with negative quality. * Even though we run on the dummy counter, switching here may be * worse since this timecounter may not be monotonic. */ if (tc_chosen) return; if (tc->tc_quality < 0) return; if (tc->tc_quality < timecounter->tc_quality) return; if (tc->tc_quality == timecounter->tc_quality && tc->tc_frequency < timecounter->tc_frequency) return; (void)tc->tc_get_timecount(tc); (void)tc->tc_get_timecount(tc); timecounter = tc; } /* Report the frequency of the current timecounter. */ uint64_t tc_getfrequency(void) { return (timehands->th_counter->tc_frequency); } static bool sleeping_on_old_rtc(struct thread *td) { /* * td_rtcgen is modified by curthread when it is running, * and by other threads in this function. By finding the thread * on a sleepqueue and holding the lock on the sleepqueue * chain, we guarantee that the thread is not running and that * modifying td_rtcgen is safe. Setting td_rtcgen to zero informs * the thread that it was woken due to a real-time clock adjustment. * (The declaration of td_rtcgen refers to this comment.) */ if (td->td_rtcgen != 0 && td->td_rtcgen != rtc_generation) { td->td_rtcgen = 0; return (true); } return (false); } static struct mtx tc_setclock_mtx; MTX_SYSINIT(tc_setclock_init, &tc_setclock_mtx, "tcsetc", MTX_SPIN); /* * Step our concept of UTC. This is done by modifying our estimate of * when we booted. */ void tc_setclock(struct timespec *ts) { struct timespec tbef, taft; struct bintime bt, bt2; timespec2bintime(ts, &bt); nanotime(&tbef); mtx_lock_spin(&tc_setclock_mtx); cpu_tick_calibrate(1); binuptime(&bt2); bintime_sub(&bt, &bt2); /* XXX fiddle all the little crinkly bits around the fiords... */ tc_windup(&bt); mtx_unlock_spin(&tc_setclock_mtx); /* Avoid rtc_generation == 0, since td_rtcgen == 0 is special. */ atomic_add_rel_int(&rtc_generation, 2); sleepq_chains_remove_matching(sleeping_on_old_rtc); if (timestepwarnings) { nanotime(&taft); log(LOG_INFO, "Time stepped from %jd.%09ld to %jd.%09ld (%jd.%09ld)\n", (intmax_t)tbef.tv_sec, tbef.tv_nsec, (intmax_t)taft.tv_sec, taft.tv_nsec, (intmax_t)ts->tv_sec, ts->tv_nsec); } } /* * Initialize the next struct timehands in the ring and make * it the active timehands. Along the way we might switch to a different * timecounter and/or do seconds processing in NTP. Slightly magic. */ static void tc_windup(struct bintime *new_boottimebin) { struct bintime bt; struct timehands *th, *tho; uint64_t scale; u_int delta, ncount, ogen; int i; time_t t; /* * Make the next timehands a copy of the current one, but do * not overwrite the generation or next pointer. While we * update the contents, the generation must be zero. We need * to ensure that the zero generation is visible before the * data updates become visible, which requires release fence. * For similar reasons, re-reading of the generation after the * data is read should use acquire fence. */ tho = timehands; th = tho->th_next; ogen = th->th_generation; th->th_generation = 0; atomic_thread_fence_rel(); memcpy(th, tho, offsetof(struct timehands, th_generation)); if (new_boottimebin != NULL) th->th_boottime = *new_boottimebin; /* * Capture a timecounter delta on the current timecounter and if * changing timecounters, a counter value from the new timecounter. * Update the offset fields accordingly. */ delta = tc_delta(th); if (th->th_counter != timecounter) ncount = timecounter->tc_get_timecount(timecounter); else ncount = 0; #ifdef FFCLOCK ffclock_windup(delta); #endif th->th_offset_count += delta; th->th_offset_count &= th->th_counter->tc_counter_mask; while (delta > th->th_counter->tc_frequency) { /* Eat complete unadjusted seconds. */ delta -= th->th_counter->tc_frequency; th->th_offset.sec++; } if ((delta > th->th_counter->tc_frequency / 2) && (th->th_scale * delta < ((uint64_t)1 << 63))) { /* The product th_scale * delta just barely overflows. */ th->th_offset.sec++; } bintime_addx(&th->th_offset, th->th_scale * delta); /* * Hardware latching timecounters may not generate interrupts on * PPS events, so instead we poll them. There is a finite risk that * the hardware might capture a count which is later than the one we * got above, and therefore possibly in the next NTP second which might * have a different rate than the current NTP second. It doesn't * matter in practice. */ if (tho->th_counter->tc_poll_pps) tho->th_counter->tc_poll_pps(tho->th_counter); /* * Deal with NTP second processing. The for loop normally * iterates at most once, but in extreme situations it might * keep NTP sane if timeouts are not run for several seconds. * At boot, the time step can be large when the TOD hardware * has been read, so on really large steps, we call * ntp_update_second only twice. We need to call it twice in * case we missed a leap second. */ bt = th->th_offset; bintime_add(&bt, &th->th_boottime); i = bt.sec - tho->th_microtime.tv_sec; if (i > LARGE_STEP) i = 2; for (; i > 0; i--) { t = bt.sec; ntp_update_second(&th->th_adjustment, &bt.sec); if (bt.sec != t) th->th_boottime.sec += bt.sec - t; } /* Update the UTC timestamps used by the get*() functions. */ th->th_bintime = bt; bintime2timeval(&bt, &th->th_microtime); bintime2timespec(&bt, &th->th_nanotime); /* Now is a good time to change timecounters. */ if (th->th_counter != timecounter) { #ifndef __arm__ if ((timecounter->tc_flags & TC_FLAGS_C2STOP) != 0) cpu_disable_c2_sleep++; if ((th->th_counter->tc_flags & TC_FLAGS_C2STOP) != 0) cpu_disable_c2_sleep--; #endif th->th_counter = timecounter; th->th_offset_count = ncount; tc_min_ticktock_freq = max(1, timecounter->tc_frequency / (((uint64_t)timecounter->tc_counter_mask + 1) / 3)); #ifdef FFCLOCK ffclock_change_tc(th); #endif } /*- * Recalculate the scaling factor. We want the number of 1/2^64 * fractions of a second per period of the hardware counter, taking * into account the th_adjustment factor which the NTP PLL/adjtime(2) * processing provides us with. * * The th_adjustment is nanoseconds per second with 32 bit binary * fraction and we want 64 bit binary fraction of second: * * x = a * 2^32 / 10^9 = a * 4.294967296 * * The range of th_adjustment is +/- 5000PPM so inside a 64bit int * we can only multiply by about 850 without overflowing, that * leaves no suitably precise fractions for multiply before divide. * * Divide before multiply with a fraction of 2199/512 results in a * systematic undercompensation of 10PPM of th_adjustment. On a * 5000PPM adjustment this is a 0.05PPM error. This is acceptable. * * We happily sacrifice the lowest of the 64 bits of our result * to the goddess of code clarity. * */ scale = (uint64_t)1 << 63; scale += (th->th_adjustment / 1024) * 2199; scale /= th->th_counter->tc_frequency; th->th_scale = scale * 2; + th->th_large_delta = MIN(((uint64_t)1 << 63) / scale, UINT_MAX); /* * Now that the struct timehands is again consistent, set the new * generation number, making sure to not make it zero. */ if (++ogen == 0) ogen = 1; atomic_store_rel_int(&th->th_generation, ogen); /* Go live with the new struct timehands. */ #ifdef FFCLOCK switch (sysclock_active) { case SYSCLOCK_FBCK: #endif time_second = th->th_microtime.tv_sec; time_uptime = th->th_offset.sec; #ifdef FFCLOCK break; case SYSCLOCK_FFWD: time_second = fftimehands->tick_time_lerp.sec; time_uptime = fftimehands->tick_time_lerp.sec - ffclock_boottime.sec; break; } #endif timehands = th; timekeep_push_vdso(); } /* Report or change the active timecounter hardware. */ static int sysctl_kern_timecounter_hardware(SYSCTL_HANDLER_ARGS) { char newname[32]; struct timecounter *newtc, *tc; int error; tc = timecounter; strlcpy(newname, tc->tc_name, sizeof(newname)); error = sysctl_handle_string(oidp, &newname[0], sizeof(newname), req); if (error != 0 || req->newptr == NULL) return (error); /* Record that the tc in use now was specifically chosen. */ tc_chosen = 1; if (strcmp(newname, tc->tc_name) == 0) return (0); for (newtc = timecounters; newtc != NULL; newtc = newtc->tc_next) { if (strcmp(newname, newtc->tc_name) != 0) continue; /* Warm up new timecounter. */ (void)newtc->tc_get_timecount(newtc); (void)newtc->tc_get_timecount(newtc); timecounter = newtc; /* * The vdso timehands update is deferred until the next * 'tc_windup()'. * * This is prudent given that 'timekeep_push_vdso()' does not * use any locking and that it can be called in hard interrupt * context via 'tc_windup()'. */ return (0); } return (EINVAL); } SYSCTL_PROC(_kern_timecounter, OID_AUTO, hardware, CTLTYPE_STRING | CTLFLAG_RW, 0, 0, sysctl_kern_timecounter_hardware, "A", "Timecounter hardware selected"); /* Report the available timecounter hardware. */ static int sysctl_kern_timecounter_choice(SYSCTL_HANDLER_ARGS) { struct sbuf sb; struct timecounter *tc; int error; sbuf_new_for_sysctl(&sb, NULL, 0, req); for (tc = timecounters; tc != NULL; tc = tc->tc_next) { if (tc != timecounters) sbuf_putc(&sb, ' '); sbuf_printf(&sb, "%s(%d)", tc->tc_name, tc->tc_quality); } error = sbuf_finish(&sb); sbuf_delete(&sb); return (error); } SYSCTL_PROC(_kern_timecounter, OID_AUTO, choice, CTLTYPE_STRING | CTLFLAG_RD, 0, 0, sysctl_kern_timecounter_choice, "A", "Timecounter hardware detected"); /* * RFC 2783 PPS-API implementation. */ /* * Return true if the driver is aware of the abi version extensions in the * pps_state structure, and it supports at least the given abi version number. */ static inline int abi_aware(struct pps_state *pps, int vers) { return ((pps->kcmode & KCMODE_ABIFLAG) && pps->driver_abi >= vers); } static int pps_fetch(struct pps_fetch_args *fapi, struct pps_state *pps) { int err, timo; pps_seq_t aseq, cseq; struct timeval tv; if (fapi->tsformat && fapi->tsformat != PPS_TSFMT_TSPEC) return (EINVAL); /* * If no timeout is requested, immediately return whatever values were * most recently captured. If timeout seconds is -1, that's a request * to block without a timeout. WITNESS won't let us sleep forever * without a lock (we really don't need a lock), so just repeatedly * sleep a long time. */ if (fapi->timeout.tv_sec || fapi->timeout.tv_nsec) { if (fapi->timeout.tv_sec == -1) timo = 0x7fffffff; else { tv.tv_sec = fapi->timeout.tv_sec; tv.tv_usec = fapi->timeout.tv_nsec / 1000; timo = tvtohz(&tv); } aseq = atomic_load_int(&pps->ppsinfo.assert_sequence); cseq = atomic_load_int(&pps->ppsinfo.clear_sequence); while (aseq == atomic_load_int(&pps->ppsinfo.assert_sequence) && cseq == atomic_load_int(&pps->ppsinfo.clear_sequence)) { if (abi_aware(pps, 1) && pps->driver_mtx != NULL) { if (pps->flags & PPSFLAG_MTX_SPIN) { err = msleep_spin(pps, pps->driver_mtx, "ppsfch", timo); } else { err = msleep(pps, pps->driver_mtx, PCATCH, "ppsfch", timo); } } else { err = tsleep(pps, PCATCH, "ppsfch", timo); } if (err == EWOULDBLOCK) { if (fapi->timeout.tv_sec == -1) { continue; } else { return (ETIMEDOUT); } } else if (err != 0) { return (err); } } } pps->ppsinfo.current_mode = pps->ppsparam.mode; fapi->pps_info_buf = pps->ppsinfo; return (0); } int pps_ioctl(u_long cmd, caddr_t data, struct pps_state *pps) { pps_params_t *app; struct pps_fetch_args *fapi; #ifdef FFCLOCK struct pps_fetch_ffc_args *fapi_ffc; #endif #ifdef PPS_SYNC struct pps_kcbind_args *kapi; #endif KASSERT(pps != NULL, ("NULL pps pointer in pps_ioctl")); switch (cmd) { case PPS_IOC_CREATE: return (0); case PPS_IOC_DESTROY: return (0); case PPS_IOC_SETPARAMS: app = (pps_params_t *)data; if (app->mode & ~pps->ppscap) return (EINVAL); #ifdef FFCLOCK /* Ensure only a single clock is selected for ffc timestamp. */ if ((app->mode & PPS_TSCLK_MASK) == PPS_TSCLK_MASK) return (EINVAL); #endif pps->ppsparam = *app; return (0); case PPS_IOC_GETPARAMS: app = (pps_params_t *)data; *app = pps->ppsparam; app->api_version = PPS_API_VERS_1; return (0); case PPS_IOC_GETCAP: *(int*)data = pps->ppscap; return (0); case PPS_IOC_FETCH: fapi = (struct pps_fetch_args *)data; return (pps_fetch(fapi, pps)); #ifdef FFCLOCK case PPS_IOC_FETCH_FFCOUNTER: fapi_ffc = (struct pps_fetch_ffc_args *)data; if (fapi_ffc->tsformat && fapi_ffc->tsformat != PPS_TSFMT_TSPEC) return (EINVAL); if (fapi_ffc->timeout.tv_sec || fapi_ffc->timeout.tv_nsec) return (EOPNOTSUPP); pps->ppsinfo_ffc.current_mode = pps->ppsparam.mode; fapi_ffc->pps_info_buf_ffc = pps->ppsinfo_ffc; /* Overwrite timestamps if feedback clock selected. */ switch (pps->ppsparam.mode & PPS_TSCLK_MASK) { case PPS_TSCLK_FBCK: fapi_ffc->pps_info_buf_ffc.assert_timestamp = pps->ppsinfo.assert_timestamp; fapi_ffc->pps_info_buf_ffc.clear_timestamp = pps->ppsinfo.clear_timestamp; break; case PPS_TSCLK_FFWD: break; default: break; } return (0); #endif /* FFCLOCK */ case PPS_IOC_KCBIND: #ifdef PPS_SYNC kapi = (struct pps_kcbind_args *)data; /* XXX Only root should be able to do this */ if (kapi->tsformat && kapi->tsformat != PPS_TSFMT_TSPEC) return (EINVAL); if (kapi->kernel_consumer != PPS_KC_HARDPPS) return (EINVAL); if (kapi->edge & ~pps->ppscap) return (EINVAL); pps->kcmode = (kapi->edge & KCMODE_EDGEMASK) | (pps->kcmode & KCMODE_ABIFLAG); return (0); #else return (EOPNOTSUPP); #endif default: return (ENOIOCTL); } } void pps_init(struct pps_state *pps) { pps->ppscap |= PPS_TSFMT_TSPEC | PPS_CANWAIT; if (pps->ppscap & PPS_CAPTUREASSERT) pps->ppscap |= PPS_OFFSETASSERT; if (pps->ppscap & PPS_CAPTURECLEAR) pps->ppscap |= PPS_OFFSETCLEAR; #ifdef FFCLOCK pps->ppscap |= PPS_TSCLK_MASK; #endif pps->kcmode &= ~KCMODE_ABIFLAG; } void pps_init_abi(struct pps_state *pps) { pps_init(pps); if (pps->driver_abi > 0) { pps->kcmode |= KCMODE_ABIFLAG; pps->kernel_abi = PPS_ABI_VERSION; } } void pps_capture(struct pps_state *pps) { struct timehands *th; KASSERT(pps != NULL, ("NULL pps pointer in pps_capture")); th = timehands; pps->capgen = atomic_load_acq_int(&th->th_generation); pps->capth = th; #ifdef FFCLOCK pps->capffth = fftimehands; #endif pps->capcount = th->th_counter->tc_get_timecount(th->th_counter); atomic_thread_fence_acq(); if (pps->capgen != th->th_generation) pps->capgen = 0; } void pps_event(struct pps_state *pps, int event) { struct bintime bt; struct timespec ts, *tsp, *osp; u_int tcount, *pcount; int foff; pps_seq_t *pseq; #ifdef FFCLOCK struct timespec *tsp_ffc; pps_seq_t *pseq_ffc; ffcounter *ffcount; #endif #ifdef PPS_SYNC int fhard; #endif KASSERT(pps != NULL, ("NULL pps pointer in pps_event")); /* Nothing to do if not currently set to capture this event type. */ if ((event & pps->ppsparam.mode) == 0) return; /* If the timecounter was wound up underneath us, bail out. */ if (pps->capgen == 0 || pps->capgen != atomic_load_acq_int(&pps->capth->th_generation)) return; /* Things would be easier with arrays. */ if (event == PPS_CAPTUREASSERT) { tsp = &pps->ppsinfo.assert_timestamp; osp = &pps->ppsparam.assert_offset; foff = pps->ppsparam.mode & PPS_OFFSETASSERT; #ifdef PPS_SYNC fhard = pps->kcmode & PPS_CAPTUREASSERT; #endif pcount = &pps->ppscount[0]; pseq = &pps->ppsinfo.assert_sequence; #ifdef FFCLOCK ffcount = &pps->ppsinfo_ffc.assert_ffcount; tsp_ffc = &pps->ppsinfo_ffc.assert_timestamp; pseq_ffc = &pps->ppsinfo_ffc.assert_sequence; #endif } else { tsp = &pps->ppsinfo.clear_timestamp; osp = &pps->ppsparam.clear_offset; foff = pps->ppsparam.mode & PPS_OFFSETCLEAR; #ifdef PPS_SYNC fhard = pps->kcmode & PPS_CAPTURECLEAR; #endif pcount = &pps->ppscount[1]; pseq = &pps->ppsinfo.clear_sequence; #ifdef FFCLOCK ffcount = &pps->ppsinfo_ffc.clear_ffcount; tsp_ffc = &pps->ppsinfo_ffc.clear_timestamp; pseq_ffc = &pps->ppsinfo_ffc.clear_sequence; #endif } /* * If the timecounter changed, we cannot compare the count values, so * we have to drop the rest of the PPS-stuff until the next event. */ if (pps->ppstc != pps->capth->th_counter) { pps->ppstc = pps->capth->th_counter; *pcount = pps->capcount; pps->ppscount[2] = pps->capcount; return; } /* Convert the count to a timespec. */ tcount = pps->capcount - pps->capth->th_offset_count; tcount &= pps->capth->th_counter->tc_counter_mask; bt = pps->capth->th_bintime; bintime_addx(&bt, pps->capth->th_scale * tcount); bintime2timespec(&bt, &ts); /* If the timecounter was wound up underneath us, bail out. */ atomic_thread_fence_acq(); if (pps->capgen != pps->capth->th_generation) return; *pcount = pps->capcount; (*pseq)++; *tsp = ts; if (foff) { timespecadd(tsp, osp, tsp); if (tsp->tv_nsec < 0) { tsp->tv_nsec += 1000000000; tsp->tv_sec -= 1; } } #ifdef FFCLOCK *ffcount = pps->capffth->tick_ffcount + tcount; bt = pps->capffth->tick_time; ffclock_convert_delta(tcount, pps->capffth->cest.period, &bt); bintime_add(&bt, &pps->capffth->tick_time); bintime2timespec(&bt, &ts); (*pseq_ffc)++; *tsp_ffc = ts; #endif #ifdef PPS_SYNC if (fhard) { uint64_t scale; /* * Feed the NTP PLL/FLL. * The FLL wants to know how many (hardware) nanoseconds * elapsed since the previous event. */ tcount = pps->capcount - pps->ppscount[2]; pps->ppscount[2] = pps->capcount; tcount &= pps->capth->th_counter->tc_counter_mask; scale = (uint64_t)1 << 63; scale /= pps->capth->th_counter->tc_frequency; scale *= 2; bt.sec = 0; bt.frac = 0; bintime_addx(&bt, scale * tcount); bintime2timespec(&bt, &ts); hardpps(tsp, ts.tv_nsec + 1000000000 * ts.tv_sec); } #endif /* Wakeup anyone sleeping in pps_fetch(). */ wakeup(pps); } /* * Timecounters need to be updated every so often to prevent the hardware * counter from overflowing. Updating also recalculates the cached values * used by the get*() family of functions, so their precision depends on * the update frequency. */ static int tc_tick; SYSCTL_INT(_kern_timecounter, OID_AUTO, tick, CTLFLAG_RD, &tc_tick, 0, "Approximate number of hardclock ticks in a millisecond"); void tc_ticktock(int cnt) { static int count; if (mtx_trylock_spin(&tc_setclock_mtx)) { count += cnt; if (count >= tc_tick) { count = 0; tc_windup(NULL); } mtx_unlock_spin(&tc_setclock_mtx); } } static void __inline tc_adjprecision(void) { int t; if (tc_timepercentage > 0) { t = (99 + tc_timepercentage) / tc_timepercentage; tc_precexp = fls(t + (t >> 1)) - 1; FREQ2BT(hz / tc_tick, &bt_timethreshold); FREQ2BT(hz, &bt_tickthreshold); bintime_shift(&bt_timethreshold, tc_precexp); bintime_shift(&bt_tickthreshold, tc_precexp); } else { tc_precexp = 31; bt_timethreshold.sec = INT_MAX; bt_timethreshold.frac = ~(uint64_t)0; bt_tickthreshold = bt_timethreshold; } sbt_timethreshold = bttosbt(bt_timethreshold); sbt_tickthreshold = bttosbt(bt_tickthreshold); } static int sysctl_kern_timecounter_adjprecision(SYSCTL_HANDLER_ARGS) { int error, val; val = tc_timepercentage; error = sysctl_handle_int(oidp, &val, 0, req); if (error != 0 || req->newptr == NULL) return (error); tc_timepercentage = val; if (cold) goto done; tc_adjprecision(); done: return (0); } /* Set up the requested number of timehands. */ static void inittimehands(void *dummy) { struct timehands *thp; int i; TUNABLE_INT_FETCH("kern.timecounter.timehands_count", &timehands_count); if (timehands_count < 1) timehands_count = 1; if (timehands_count > nitems(ths)) timehands_count = nitems(ths); for (i = 1, thp = &ths[0]; i < timehands_count; thp = &ths[i++]) thp->th_next = &ths[i]; thp->th_next = &ths[0]; } SYSINIT(timehands, SI_SUB_TUNABLES, SI_ORDER_ANY, inittimehands, NULL); static void inittimecounter(void *dummy) { u_int p; int tick_rate; /* * Set the initial timeout to * max(1, ). * People should probably not use the sysctl to set the timeout * to smaller than its initial value, since that value is the * smallest reasonable one. If they want better timestamps they * should use the non-"get"* functions. */ if (hz > 1000) tc_tick = (hz + 500) / 1000; else tc_tick = 1; tc_adjprecision(); FREQ2BT(hz, &tick_bt); tick_sbt = bttosbt(tick_bt); tick_rate = hz / tc_tick; FREQ2BT(tick_rate, &tc_tick_bt); tc_tick_sbt = bttosbt(tc_tick_bt); p = (tc_tick * 1000000) / hz; printf("Timecounters tick every %d.%03u msec\n", p / 1000, p % 1000); #ifdef FFCLOCK ffclock_init(); #endif /* warm up new timecounter (again) and get rolling. */ (void)timecounter->tc_get_timecount(timecounter); (void)timecounter->tc_get_timecount(timecounter); mtx_lock_spin(&tc_setclock_mtx); tc_windup(NULL); mtx_unlock_spin(&tc_setclock_mtx); } SYSINIT(timecounter, SI_SUB_CLOCKS, SI_ORDER_SECOND, inittimecounter, NULL); /* Cpu tick handling -------------------------------------------------*/ static int cpu_tick_variable; static uint64_t cpu_tick_frequency; DPCPU_DEFINE_STATIC(uint64_t, tc_cpu_ticks_base); DPCPU_DEFINE_STATIC(unsigned, tc_cpu_ticks_last); static uint64_t tc_cpu_ticks(void) { struct timecounter *tc; uint64_t res, *base; unsigned u, *last; critical_enter(); base = DPCPU_PTR(tc_cpu_ticks_base); last = DPCPU_PTR(tc_cpu_ticks_last); tc = timehands->th_counter; u = tc->tc_get_timecount(tc) & tc->tc_counter_mask; if (u < *last) *base += (uint64_t)tc->tc_counter_mask + 1; *last = u; res = u + *base; critical_exit(); return (res); } void cpu_tick_calibration(void) { static time_t last_calib; if (time_uptime != last_calib && !(time_uptime & 0xf)) { cpu_tick_calibrate(0); last_calib = time_uptime; } } /* * This function gets called every 16 seconds on only one designated * CPU in the system from hardclock() via cpu_tick_calibration()(). * * Whenever the real time clock is stepped we get called with reset=1 * to make sure we handle suspend/resume and similar events correctly. */ static void cpu_tick_calibrate(int reset) { static uint64_t c_last; uint64_t c_this, c_delta; static struct bintime t_last; struct bintime t_this, t_delta; uint32_t divi; if (reset) { /* The clock was stepped, abort & reset */ t_last.sec = 0; return; } /* we don't calibrate fixed rate cputicks */ if (!cpu_tick_variable) return; getbinuptime(&t_this); c_this = cpu_ticks(); if (t_last.sec != 0) { c_delta = c_this - c_last; t_delta = t_this; bintime_sub(&t_delta, &t_last); /* * Headroom: * 2^(64-20) / 16[s] = * 2^(44) / 16[s] = * 17.592.186.044.416 / 16 = * 1.099.511.627.776 [Hz] */ divi = t_delta.sec << 20; divi |= t_delta.frac >> (64 - 20); c_delta <<= 20; c_delta /= divi; if (c_delta > cpu_tick_frequency) { if (0 && bootverbose) printf("cpu_tick increased to %ju Hz\n", c_delta); cpu_tick_frequency = c_delta; } } c_last = c_this; t_last = t_this; } void set_cputicker(cpu_tick_f *func, uint64_t freq, unsigned var) { if (func == NULL) { cpu_ticks = tc_cpu_ticks; } else { cpu_tick_frequency = freq; cpu_tick_variable = var; cpu_ticks = func; } } uint64_t cpu_tickrate(void) { if (cpu_ticks == tc_cpu_ticks) return (tc_getfrequency()); return (cpu_tick_frequency); } /* * We need to be slightly careful converting cputicks to microseconds. * There is plenty of margin in 64 bits of microseconds (half a million * years) and in 64 bits at 4 GHz (146 years), but if we do a multiply * before divide conversion (to retain precision) we find that the * margin shrinks to 1.5 hours (one millionth of 146y). * With a three prong approach we never lose significant bits, no * matter what the cputick rate and length of timeinterval is. */ uint64_t cputick2usec(uint64_t tick) { if (tick > 18446744073709551LL) /* floor(2^64 / 1000) */ return (tick / (cpu_tickrate() / 1000000LL)); else if (tick > 18446744073709LL) /* floor(2^64 / 1000000) */ return ((tick * 1000LL) / (cpu_tickrate() / 1000LL)); else return ((tick * 1000000LL) / cpu_tickrate()); } cpu_tick_f *cpu_ticks = tc_cpu_ticks; static int vdso_th_enable = 1; static int sysctl_fast_gettime(SYSCTL_HANDLER_ARGS) { int old_vdso_th_enable, error; old_vdso_th_enable = vdso_th_enable; error = sysctl_handle_int(oidp, &old_vdso_th_enable, 0, req); if (error != 0) return (error); vdso_th_enable = old_vdso_th_enable; return (0); } SYSCTL_PROC(_kern_timecounter, OID_AUTO, fast_gettime, CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, NULL, 0, sysctl_fast_gettime, "I", "Enable fast time of day"); uint32_t tc_fill_vdso_timehands(struct vdso_timehands *vdso_th) { struct timehands *th; uint32_t enabled; th = timehands; vdso_th->th_scale = th->th_scale; vdso_th->th_offset_count = th->th_offset_count; vdso_th->th_counter_mask = th->th_counter->tc_counter_mask; vdso_th->th_offset = th->th_offset; vdso_th->th_boottime = th->th_boottime; if (th->th_counter->tc_fill_vdso_timehands != NULL) { enabled = th->th_counter->tc_fill_vdso_timehands(vdso_th, th->th_counter); } else enabled = 0; if (!vdso_th_enable) enabled = 0; return (enabled); } #ifdef COMPAT_FREEBSD32 uint32_t tc_fill_vdso_timehands32(struct vdso_timehands32 *vdso_th32) { struct timehands *th; uint32_t enabled; th = timehands; *(uint64_t *)&vdso_th32->th_scale[0] = th->th_scale; vdso_th32->th_offset_count = th->th_offset_count; vdso_th32->th_counter_mask = th->th_counter->tc_counter_mask; vdso_th32->th_offset.sec = th->th_offset.sec; *(uint64_t *)&vdso_th32->th_offset.frac[0] = th->th_offset.frac; vdso_th32->th_boottime.sec = th->th_boottime.sec; *(uint64_t *)&vdso_th32->th_boottime.frac[0] = th->th_boottime.frac; if (th->th_counter->tc_fill_vdso_timehands32 != NULL) { enabled = th->th_counter->tc_fill_vdso_timehands32(vdso_th32, th->th_counter); } else enabled = 0; if (!vdso_th_enable) enabled = 0; return (enabled); } #endif Index: projects/clang1000-import/sys/kern/subr_capability.c =================================================================== --- projects/clang1000-import/sys/kern/subr_capability.c (revision 357965) +++ projects/clang1000-import/sys/kern/subr_capability.c (revision 357966) @@ -1,420 +1,420 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2013 FreeBSD Foundation * All rights reserved. * * This software was developed by Pawel Jakub Dawidek under sponsorship from * the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); /* * Note that this file is compiled into the kernel and into libc. */ #include #include #ifdef _KERNEL #include #include #include #else /* !_KERNEL */ #include #include #include #include #include #endif #ifdef _KERNEL #define assert(exp) KASSERT((exp), ("%s:%u", __func__, __LINE__)) __read_mostly cap_rights_t cap_accept_rights; __read_mostly cap_rights_t cap_bind_rights; __read_mostly cap_rights_t cap_chflags_rights; __read_mostly cap_rights_t cap_connect_rights; __read_mostly cap_rights_t cap_event_rights; __read_mostly cap_rights_t cap_fchdir_rights; __read_mostly cap_rights_t cap_fchflags_rights; __read_mostly cap_rights_t cap_fchmod_rights; __read_mostly cap_rights_t cap_fchown_rights; __read_mostly cap_rights_t cap_fcntl_rights; __read_mostly cap_rights_t cap_fexecve_rights; __read_mostly cap_rights_t cap_flock_rights; __read_mostly cap_rights_t cap_fpathconf_rights; __read_mostly cap_rights_t cap_fstat_rights; __read_mostly cap_rights_t cap_fstatfs_rights; __read_mostly cap_rights_t cap_fsync_rights; __read_mostly cap_rights_t cap_ftruncate_rights; __read_mostly cap_rights_t cap_futimes_rights; __read_mostly cap_rights_t cap_getpeername_rights; __read_mostly cap_rights_t cap_getsockopt_rights; __read_mostly cap_rights_t cap_getsockname_rights; __read_mostly cap_rights_t cap_ioctl_rights; __read_mostly cap_rights_t cap_listen_rights; __read_mostly cap_rights_t cap_linkat_source_rights; __read_mostly cap_rights_t cap_linkat_target_rights; __read_mostly cap_rights_t cap_mmap_rights; __read_mostly cap_rights_t cap_mkdirat_rights; __read_mostly cap_rights_t cap_mkfifoat_rights; __read_mostly cap_rights_t cap_mknodat_rights; __read_mostly cap_rights_t cap_pdgetpid_rights; __read_mostly cap_rights_t cap_pdkill_rights; __read_mostly cap_rights_t cap_pread_rights; __read_mostly cap_rights_t cap_pwrite_rights; __read_mostly cap_rights_t cap_read_rights; __read_mostly cap_rights_t cap_recv_rights; __read_mostly cap_rights_t cap_renameat_source_rights; __read_mostly cap_rights_t cap_renameat_target_rights; __read_mostly cap_rights_t cap_seek_rights; __read_mostly cap_rights_t cap_send_rights; __read_mostly cap_rights_t cap_send_connect_rights; __read_mostly cap_rights_t cap_setsockopt_rights; __read_mostly cap_rights_t cap_shutdown_rights; __read_mostly cap_rights_t cap_symlinkat_rights; __read_mostly cap_rights_t cap_unlinkat_rights; __read_mostly cap_rights_t cap_write_rights; __read_mostly cap_rights_t cap_no_rights; static void __cap_rights_sysinit1(void *arg) { - cap_rights_init(&cap_accept_rights, CAP_ACCEPT); - cap_rights_init(&cap_bind_rights, CAP_BIND); - cap_rights_init(&cap_connect_rights, CAP_CONNECT); - cap_rights_init(&cap_event_rights, CAP_EVENT); - cap_rights_init(&cap_fchdir_rights, CAP_FCHDIR); - cap_rights_init(&cap_fchflags_rights, CAP_FCHFLAGS); - cap_rights_init(&cap_fchmod_rights, CAP_FCHMOD); - cap_rights_init(&cap_fchown_rights, CAP_FCHOWN); - cap_rights_init(&cap_fcntl_rights, CAP_FCNTL); - cap_rights_init(&cap_fexecve_rights, CAP_FEXECVE); - cap_rights_init(&cap_flock_rights, CAP_FLOCK); - cap_rights_init(&cap_fpathconf_rights, CAP_FPATHCONF); - cap_rights_init(&cap_fstat_rights, CAP_FSTAT); - cap_rights_init(&cap_fstatfs_rights, CAP_FSTATFS); - cap_rights_init(&cap_fsync_rights, CAP_FSYNC); - cap_rights_init(&cap_ftruncate_rights, CAP_FTRUNCATE); - cap_rights_init(&cap_futimes_rights, CAP_FUTIMES); - cap_rights_init(&cap_getpeername_rights, CAP_GETPEERNAME); - cap_rights_init(&cap_getsockname_rights, CAP_GETSOCKNAME); - cap_rights_init(&cap_getsockopt_rights, CAP_GETSOCKOPT); - cap_rights_init(&cap_ioctl_rights, CAP_IOCTL); - cap_rights_init(&cap_linkat_source_rights, CAP_LINKAT_SOURCE); - cap_rights_init(&cap_linkat_target_rights, CAP_LINKAT_TARGET); - cap_rights_init(&cap_listen_rights, CAP_LISTEN); - cap_rights_init(&cap_mkdirat_rights, CAP_MKDIRAT); - cap_rights_init(&cap_mkfifoat_rights, CAP_MKFIFOAT); - cap_rights_init(&cap_mknodat_rights, CAP_MKNODAT); - cap_rights_init(&cap_mmap_rights, CAP_MMAP); - cap_rights_init(&cap_pdgetpid_rights, CAP_PDGETPID); - cap_rights_init(&cap_pdkill_rights, CAP_PDKILL); - cap_rights_init(&cap_pread_rights, CAP_PREAD); - cap_rights_init(&cap_pwrite_rights, CAP_PWRITE); - cap_rights_init(&cap_read_rights, CAP_READ); - cap_rights_init(&cap_recv_rights, CAP_RECV); - cap_rights_init(&cap_renameat_source_rights, CAP_RENAMEAT_SOURCE); - cap_rights_init(&cap_renameat_target_rights, CAP_RENAMEAT_TARGET); - cap_rights_init(&cap_seek_rights, CAP_SEEK); - cap_rights_init(&cap_send_rights, CAP_SEND); + cap_rights_init_one(&cap_accept_rights, CAP_ACCEPT); + cap_rights_init_one(&cap_bind_rights, CAP_BIND); + cap_rights_init_one(&cap_connect_rights, CAP_CONNECT); + cap_rights_init_one(&cap_event_rights, CAP_EVENT); + cap_rights_init_one(&cap_fchdir_rights, CAP_FCHDIR); + cap_rights_init_one(&cap_fchflags_rights, CAP_FCHFLAGS); + cap_rights_init_one(&cap_fchmod_rights, CAP_FCHMOD); + cap_rights_init_one(&cap_fchown_rights, CAP_FCHOWN); + cap_rights_init_one(&cap_fcntl_rights, CAP_FCNTL); + cap_rights_init_one(&cap_fexecve_rights, CAP_FEXECVE); + cap_rights_init_one(&cap_flock_rights, CAP_FLOCK); + cap_rights_init_one(&cap_fpathconf_rights, CAP_FPATHCONF); + cap_rights_init_one(&cap_fstat_rights, CAP_FSTAT); + cap_rights_init_one(&cap_fstatfs_rights, CAP_FSTATFS); + cap_rights_init_one(&cap_fsync_rights, CAP_FSYNC); + cap_rights_init_one(&cap_ftruncate_rights, CAP_FTRUNCATE); + cap_rights_init_one(&cap_futimes_rights, CAP_FUTIMES); + cap_rights_init_one(&cap_getpeername_rights, CAP_GETPEERNAME); + cap_rights_init_one(&cap_getsockname_rights, CAP_GETSOCKNAME); + cap_rights_init_one(&cap_getsockopt_rights, CAP_GETSOCKOPT); + cap_rights_init_one(&cap_ioctl_rights, CAP_IOCTL); + cap_rights_init_one(&cap_linkat_source_rights, CAP_LINKAT_SOURCE); + cap_rights_init_one(&cap_linkat_target_rights, CAP_LINKAT_TARGET); + cap_rights_init_one(&cap_listen_rights, CAP_LISTEN); + cap_rights_init_one(&cap_mkdirat_rights, CAP_MKDIRAT); + cap_rights_init_one(&cap_mkfifoat_rights, CAP_MKFIFOAT); + cap_rights_init_one(&cap_mknodat_rights, CAP_MKNODAT); + cap_rights_init_one(&cap_mmap_rights, CAP_MMAP); + cap_rights_init_one(&cap_pdgetpid_rights, CAP_PDGETPID); + cap_rights_init_one(&cap_pdkill_rights, CAP_PDKILL); + cap_rights_init_one(&cap_pread_rights, CAP_PREAD); + cap_rights_init_one(&cap_pwrite_rights, CAP_PWRITE); + cap_rights_init_one(&cap_read_rights, CAP_READ); + cap_rights_init_one(&cap_recv_rights, CAP_RECV); + cap_rights_init_one(&cap_renameat_source_rights, CAP_RENAMEAT_SOURCE); + cap_rights_init_one(&cap_renameat_target_rights, CAP_RENAMEAT_TARGET); + cap_rights_init_one(&cap_seek_rights, CAP_SEEK); + cap_rights_init_one(&cap_send_rights, CAP_SEND); cap_rights_init(&cap_send_connect_rights, CAP_SEND, CAP_CONNECT); - cap_rights_init(&cap_setsockopt_rights, CAP_SETSOCKOPT); - cap_rights_init(&cap_shutdown_rights, CAP_SHUTDOWN); - cap_rights_init(&cap_symlinkat_rights, CAP_SYMLINKAT); - cap_rights_init(&cap_unlinkat_rights, CAP_UNLINKAT); - cap_rights_init(&cap_write_rights, CAP_WRITE); + cap_rights_init_one(&cap_setsockopt_rights, CAP_SETSOCKOPT); + cap_rights_init_one(&cap_shutdown_rights, CAP_SHUTDOWN); + cap_rights_init_one(&cap_symlinkat_rights, CAP_SYMLINKAT); + cap_rights_init_one(&cap_unlinkat_rights, CAP_UNLINKAT); + cap_rights_init_one(&cap_write_rights, CAP_WRITE); cap_rights_init(&cap_no_rights); } SYSINIT(cap_rights1_sysinit, SI_SUB_COPYRIGHT, SI_ORDER_ANY, \ __cap_rights_sysinit1, NULL); #endif #define CAPARSIZE_MIN (CAP_RIGHTS_VERSION_00 + 2) #define CAPARSIZE_MAX (CAP_RIGHTS_VERSION + 2) static __inline int right_to_index(uint64_t right) { static const int bit2idx[] = { -1, 0, 1, -1, 2, -1, -1, -1, 3, -1, -1, -1, -1, -1, -1, -1, 4, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1 }; int idx; idx = CAPIDXBIT(right); assert(idx >= 0 && idx < sizeof(bit2idx) / sizeof(bit2idx[0])); return (bit2idx[idx]); } static void cap_rights_vset(cap_rights_t *rights, va_list ap) { uint64_t right; int i, n __unused; assert(CAPVER(rights) == CAP_RIGHTS_VERSION_00); n = CAPARSIZE(rights); assert(n >= CAPARSIZE_MIN && n <= CAPARSIZE_MAX); for (;;) { right = (uint64_t)va_arg(ap, unsigned long long); if (right == 0) break; assert(CAPRVER(right) == 0); i = right_to_index(right); assert(i >= 0); assert(i < n); assert(CAPIDXBIT(rights->cr_rights[i]) == CAPIDXBIT(right)); rights->cr_rights[i] |= right; assert(CAPIDXBIT(rights->cr_rights[i]) == CAPIDXBIT(right)); } } static void cap_rights_vclear(cap_rights_t *rights, va_list ap) { uint64_t right; int i, n __unused; assert(CAPVER(rights) == CAP_RIGHTS_VERSION_00); n = CAPARSIZE(rights); assert(n >= CAPARSIZE_MIN && n <= CAPARSIZE_MAX); for (;;) { right = (uint64_t)va_arg(ap, unsigned long long); if (right == 0) break; assert(CAPRVER(right) == 0); i = right_to_index(right); assert(i >= 0); assert(i < n); assert(CAPIDXBIT(rights->cr_rights[i]) == CAPIDXBIT(right)); rights->cr_rights[i] &= ~(right & 0x01FFFFFFFFFFFFFFULL); assert(CAPIDXBIT(rights->cr_rights[i]) == CAPIDXBIT(right)); } } static bool cap_rights_is_vset(const cap_rights_t *rights, va_list ap) { uint64_t right; int i, n __unused; assert(CAPVER(rights) == CAP_RIGHTS_VERSION_00); n = CAPARSIZE(rights); assert(n >= CAPARSIZE_MIN && n <= CAPARSIZE_MAX); for (;;) { right = (uint64_t)va_arg(ap, unsigned long long); if (right == 0) break; assert(CAPRVER(right) == 0); i = right_to_index(right); assert(i >= 0); assert(i < n); assert(CAPIDXBIT(rights->cr_rights[i]) == CAPIDXBIT(right)); if ((rights->cr_rights[i] & right) != right) return (false); } return (true); } void __cap_rights_sysinit(void *arg) { struct cap_rights_init_args *cria = arg; cap_rights_t *rights = cria->cria_rights; __cap_rights_init(CAP_RIGHTS_VERSION, rights, cria->cria_value1, cria->cria_value2, cria->cria_value3, cria->cria_value4, 0ULL); } cap_rights_t * __cap_rights_init(int version, cap_rights_t *rights, ...) { unsigned int n __unused; va_list ap; assert(version == CAP_RIGHTS_VERSION_00); n = version + 2; assert(n >= CAPARSIZE_MIN && n <= CAPARSIZE_MAX); CAP_NONE(rights); va_start(ap, rights); cap_rights_vset(rights, ap); va_end(ap); return (rights); } cap_rights_t * __cap_rights_set(cap_rights_t *rights, ...) { va_list ap; assert(CAPVER(rights) == CAP_RIGHTS_VERSION_00); va_start(ap, rights); cap_rights_vset(rights, ap); va_end(ap); return (rights); } cap_rights_t * __cap_rights_clear(cap_rights_t *rights, ...) { va_list ap; assert(CAPVER(rights) == CAP_RIGHTS_VERSION_00); va_start(ap, rights); cap_rights_vclear(rights, ap); va_end(ap); return (rights); } bool __cap_rights_is_set(const cap_rights_t *rights, ...) { va_list ap; bool ret; assert(CAPVER(rights) == CAP_RIGHTS_VERSION_00); va_start(ap, rights); ret = cap_rights_is_vset(rights, ap); va_end(ap); return (ret); } bool cap_rights_is_valid(const cap_rights_t *rights) { cap_rights_t allrights; int i, j; if (CAPVER(rights) != CAP_RIGHTS_VERSION_00) return (false); if (CAPARSIZE(rights) < CAPARSIZE_MIN || CAPARSIZE(rights) > CAPARSIZE_MAX) { return (false); } CAP_ALL(&allrights); if (!cap_rights_contains(&allrights, rights)) return (false); for (i = 0; i < CAPARSIZE(rights); i++) { j = right_to_index(rights->cr_rights[i]); if (i != j) return (false); if (i > 0) { if (CAPRVER(rights->cr_rights[i]) != 0) return (false); } } return (true); } cap_rights_t * cap_rights_merge(cap_rights_t *dst, const cap_rights_t *src) { unsigned int i, n; assert(CAPVER(dst) == CAP_RIGHTS_VERSION_00); assert(CAPVER(src) == CAP_RIGHTS_VERSION_00); assert(CAPVER(dst) == CAPVER(src)); assert(cap_rights_is_valid(src)); assert(cap_rights_is_valid(dst)); n = CAPARSIZE(dst); assert(n >= CAPARSIZE_MIN && n <= CAPARSIZE_MAX); for (i = 0; i < n; i++) dst->cr_rights[i] |= src->cr_rights[i]; assert(cap_rights_is_valid(src)); assert(cap_rights_is_valid(dst)); return (dst); } cap_rights_t * cap_rights_remove(cap_rights_t *dst, const cap_rights_t *src) { unsigned int i, n; assert(CAPVER(dst) == CAP_RIGHTS_VERSION_00); assert(CAPVER(src) == CAP_RIGHTS_VERSION_00); assert(CAPVER(dst) == CAPVER(src)); assert(cap_rights_is_valid(src)); assert(cap_rights_is_valid(dst)); n = CAPARSIZE(dst); assert(n >= CAPARSIZE_MIN && n <= CAPARSIZE_MAX); for (i = 0; i < n; i++) { dst->cr_rights[i] &= ~(src->cr_rights[i] & 0x01FFFFFFFFFFFFFFULL); } assert(cap_rights_is_valid(src)); assert(cap_rights_is_valid(dst)); return (dst); } #ifndef _KERNEL bool cap_rights_contains(const cap_rights_t *big, const cap_rights_t *little) { unsigned int i, n; assert(CAPVER(big) == CAP_RIGHTS_VERSION_00); assert(CAPVER(little) == CAP_RIGHTS_VERSION_00); assert(CAPVER(big) == CAPVER(little)); n = CAPARSIZE(big); assert(n >= CAPARSIZE_MIN && n <= CAPARSIZE_MAX); for (i = 0; i < n; i++) { if ((big->cr_rights[i] & little->cr_rights[i]) != little->cr_rights[i]) { return (false); } } return (true); } #endif Index: projects/clang1000-import/sys/kern/subr_coverage.c =================================================================== --- projects/clang1000-import/sys/kern/subr_coverage.c (revision 357965) +++ projects/clang1000-import/sys/kern/subr_coverage.c (revision 357966) @@ -1,237 +1,237 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (C) 2018 The FreeBSD Foundation. All rights reserved. * Copyright (C) 2018, 2019 Andrew Turner * * This software was developed by Mitchell Horne under sponsorship of * the FreeBSD Foundation. * * This software was developed by SRI International and the University of * Cambridge Computer Laboratory under DARPA/AFRL contract FA8750-10-C-0237 * ("CTSRD"), as part of the DARPA CRASH research programme. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #include __FBSDID("$FreeBSD$"); #include #include #include void __sanitizer_cov_trace_pc(void); void __sanitizer_cov_trace_cmp1(uint8_t, uint8_t); void __sanitizer_cov_trace_cmp2(uint16_t, uint16_t); void __sanitizer_cov_trace_cmp4(uint32_t, uint32_t); void __sanitizer_cov_trace_cmp8(uint64_t, uint64_t); void __sanitizer_cov_trace_const_cmp1(uint8_t, uint8_t); void __sanitizer_cov_trace_const_cmp2(uint16_t, uint16_t); void __sanitizer_cov_trace_const_cmp4(uint32_t, uint32_t); void __sanitizer_cov_trace_const_cmp8(uint64_t, uint64_t); void __sanitizer_cov_trace_switch(uint64_t, uint64_t *); static cov_trace_pc_t cov_trace_pc; static cov_trace_cmp_t cov_trace_cmp; void cov_register_pc(cov_trace_pc_t trace_pc) { atomic_store_ptr(&cov_trace_pc, trace_pc); } void cov_unregister_pc(void) { atomic_store_ptr(&cov_trace_pc, NULL); } void cov_register_cmp(cov_trace_cmp_t trace_cmp) { atomic_store_ptr(&cov_trace_cmp, trace_cmp); } void cov_unregister_cmp(void) { atomic_store_ptr(&cov_trace_cmp, NULL); } /* * Main entry point. A call to this function will be inserted * at every edge, and if coverage is enabled for the thread * this function will add the PC to the buffer. */ void __sanitizer_cov_trace_pc(void) { cov_trace_pc_t trace_pc; - trace_pc = (cov_trace_pc_t)atomic_load_ptr(&cov_trace_pc); + trace_pc = atomic_load_ptr(&cov_trace_pc); if (trace_pc != NULL) trace_pc((uint64_t)__builtin_return_address(0)); } /* * Comparison entry points. When the kernel performs a comparison * operation the compiler inserts a call to one of the following * functions to record the operation. */ void __sanitizer_cov_trace_cmp1(uint8_t arg1, uint8_t arg2) { cov_trace_cmp_t trace_cmp; - trace_cmp = (cov_trace_cmp_t)atomic_load_ptr(&cov_trace_cmp); + trace_cmp = atomic_load_ptr(&cov_trace_cmp); if (trace_cmp != NULL) trace_cmp(COV_CMP_SIZE(0), arg1, arg2, (uint64_t)__builtin_return_address(0)); } void __sanitizer_cov_trace_cmp2(uint16_t arg1, uint16_t arg2) { cov_trace_cmp_t trace_cmp; - trace_cmp = (cov_trace_cmp_t)atomic_load_ptr(&cov_trace_cmp); + trace_cmp = atomic_load_ptr(&cov_trace_cmp); if (trace_cmp != NULL) trace_cmp(COV_CMP_SIZE(1), arg1, arg2, (uint64_t)__builtin_return_address(0)); } void __sanitizer_cov_trace_cmp4(uint32_t arg1, uint32_t arg2) { cov_trace_cmp_t trace_cmp; - trace_cmp = (cov_trace_cmp_t)atomic_load_ptr(&cov_trace_cmp); + trace_cmp = atomic_load_ptr(&cov_trace_cmp); if (trace_cmp != NULL) trace_cmp(COV_CMP_SIZE(2), arg1, arg2, (uint64_t)__builtin_return_address(0)); } void __sanitizer_cov_trace_cmp8(uint64_t arg1, uint64_t arg2) { cov_trace_cmp_t trace_cmp; - trace_cmp = (cov_trace_cmp_t)atomic_load_ptr(&cov_trace_cmp); + trace_cmp = atomic_load_ptr(&cov_trace_cmp); if (trace_cmp != NULL) trace_cmp(COV_CMP_SIZE(3), arg1, arg2, (uint64_t)__builtin_return_address(0)); } void __sanitizer_cov_trace_const_cmp1(uint8_t arg1, uint8_t arg2) { cov_trace_cmp_t trace_cmp; - trace_cmp = (cov_trace_cmp_t)atomic_load_ptr(&cov_trace_cmp); + trace_cmp = atomic_load_ptr(&cov_trace_cmp); if (trace_cmp != NULL) trace_cmp(COV_CMP_SIZE(0) | COV_CMP_CONST, arg1, arg2, (uint64_t)__builtin_return_address(0)); } void __sanitizer_cov_trace_const_cmp2(uint16_t arg1, uint16_t arg2) { cov_trace_cmp_t trace_cmp; - trace_cmp = (cov_trace_cmp_t)atomic_load_ptr(&cov_trace_cmp); + trace_cmp = atomic_load_ptr(&cov_trace_cmp); if (trace_cmp != NULL) trace_cmp(COV_CMP_SIZE(1) | COV_CMP_CONST, arg1, arg2, (uint64_t)__builtin_return_address(0)); } void __sanitizer_cov_trace_const_cmp4(uint32_t arg1, uint32_t arg2) { cov_trace_cmp_t trace_cmp; - trace_cmp = (cov_trace_cmp_t)atomic_load_ptr(&cov_trace_cmp); + trace_cmp = atomic_load_ptr(&cov_trace_cmp); if (trace_cmp != NULL) trace_cmp(COV_CMP_SIZE(2) | COV_CMP_CONST, arg1, arg2, (uint64_t)__builtin_return_address(0)); } void __sanitizer_cov_trace_const_cmp8(uint64_t arg1, uint64_t arg2) { cov_trace_cmp_t trace_cmp; - trace_cmp = (cov_trace_cmp_t)atomic_load_ptr(&cov_trace_cmp); + trace_cmp = atomic_load_ptr(&cov_trace_cmp); if (trace_cmp != NULL) trace_cmp(COV_CMP_SIZE(3) | COV_CMP_CONST, arg1, arg2, (uint64_t)__builtin_return_address(0)); } /* * val is the switch operand * cases[0] is the number of case constants * cases[1] is the size of val in bits * cases[2..n] are the case constants */ void __sanitizer_cov_trace_switch(uint64_t val, uint64_t *cases) { uint64_t i, count, ret, type; cov_trace_cmp_t trace_cmp; - trace_cmp = (cov_trace_cmp_t)atomic_load_ptr(&cov_trace_cmp); + trace_cmp = atomic_load_ptr(&cov_trace_cmp); if (trace_cmp == NULL) return; count = cases[0]; ret = (uint64_t)__builtin_return_address(0); switch (cases[1]) { case 8: type = COV_CMP_SIZE(0); break; case 16: type = COV_CMP_SIZE(1); break; case 32: type = COV_CMP_SIZE(2); break; case 64: type = COV_CMP_SIZE(3); break; default: return; } val |= COV_CMP_CONST; for (i = 0; i < count; i++) if (!trace_cmp(type, val, cases[i + 2], ret)) return; } Index: projects/clang1000-import/sys/kern/sys_capability.c =================================================================== --- projects/clang1000-import/sys/kern/sys_capability.c (revision 357965) +++ projects/clang1000-import/sys/kern/sys_capability.c (revision 357966) @@ -1,680 +1,680 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2008-2011 Robert N. M. Watson * Copyright (c) 2010-2011 Jonathan Anderson * Copyright (c) 2012 FreeBSD Foundation * All rights reserved. * * This software was developed at the University of Cambridge Computer * Laboratory with support from a grant from Google, Inc. * * Portions of this software were developed by Pawel Jakub Dawidek under * sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * FreeBSD kernel capability facility. * * Two kernel features are implemented here: capability mode, a sandboxed mode * of execution for processes, and capabilities, a refinement on file * descriptors that allows fine-grained control over operations on the file * descriptor. Collectively, these allow processes to run in the style of a * historic "capability system" in which they can use only resources * explicitly delegated to them. This model is enforced by restricting access * to global namespaces in capability mode. * * Capabilities wrap other file descriptor types, binding them to a constant * rights mask set when the capability is created. New capabilities may be * derived from existing capabilities, but only if they have the same or a * strict subset of the rights on the original capability. * * System calls permitted in capability mode are defined in capabilities.conf; * calls must be carefully audited for safety to ensure that they don't allow * escape from a sandbox. Some calls permit only a subset of operations in * capability mode -- for example, shm_open(2) is limited to creating * anonymous, rather than named, POSIX shared memory objects. */ #include __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_ktrace.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include bool __read_frequently trap_enotcap; SYSCTL_BOOL(_kern, OID_AUTO, trap_enotcap, CTLFLAG_RWTUN, &trap_enotcap, 0, "Deliver SIGTRAP on ENOTCAPABLE"); #ifdef CAPABILITY_MODE #define IOCTLS_MAX_COUNT 256 /* XXX: Is 256 sane? */ FEATURE(security_capability_mode, "Capsicum Capability Mode"); /* * System call to enter capability mode for the process. */ int sys_cap_enter(struct thread *td, struct cap_enter_args *uap) { struct ucred *newcred, *oldcred; struct proc *p; if (IN_CAPABILITY_MODE(td)) return (0); newcred = crget(); p = td->td_proc; PROC_LOCK(p); oldcred = crcopysafe(p, newcred); newcred->cr_flags |= CRED_FLAG_CAPMODE; proc_set_cred(p, newcred); PROC_UNLOCK(p); crfree(oldcred); return (0); } /* * System call to query whether the process is in capability mode. */ int sys_cap_getmode(struct thread *td, struct cap_getmode_args *uap) { u_int i; i = IN_CAPABILITY_MODE(td) ? 1 : 0; return (copyout(&i, uap->modep, sizeof(i))); } #else /* !CAPABILITY_MODE */ int sys_cap_enter(struct thread *td, struct cap_enter_args *uap) { return (ENOSYS); } int sys_cap_getmode(struct thread *td, struct cap_getmode_args *uap) { return (ENOSYS); } #endif /* CAPABILITY_MODE */ #ifdef CAPABILITIES FEATURE(security_capabilities, "Capsicum Capabilities"); MALLOC_DECLARE(M_FILECAPS); static inline int _cap_check(const cap_rights_t *havep, const cap_rights_t *needp, enum ktr_cap_fail_type type) { if (!cap_rights_contains(havep, needp)) { #ifdef KTRACE if (KTRPOINT(curthread, KTR_CAPFAIL)) ktrcapfail(type, needp, havep); #endif return (ENOTCAPABLE); } return (0); } /* * Test whether a capability grants the requested rights. */ int cap_check(const cap_rights_t *havep, const cap_rights_t *needp) { return (_cap_check(havep, needp, CAPFAIL_NOTCAPABLE)); } int cap_check_failed_notcapable(const cap_rights_t *havep, const cap_rights_t *needp) { #ifdef KTRACE if (KTRPOINT(curthread, KTR_CAPFAIL)) ktrcapfail(CAPFAIL_NOTCAPABLE, needp, havep); #endif return (ENOTCAPABLE); } /* * Convert capability rights into VM access flags. */ vm_prot_t cap_rights_to_vmprot(const cap_rights_t *havep) { vm_prot_t maxprot; maxprot = VM_PROT_NONE; if (cap_rights_is_set(havep, CAP_MMAP_R)) maxprot |= VM_PROT_READ; if (cap_rights_is_set(havep, CAP_MMAP_W)) maxprot |= VM_PROT_WRITE; if (cap_rights_is_set(havep, CAP_MMAP_X)) maxprot |= VM_PROT_EXECUTE; return (maxprot); } /* * Extract rights from a capability for monitoring purposes -- not for use in * any other way, as we want to keep all capability permission evaluation in * this one file. */ const cap_rights_t * cap_rights_fde(const struct filedescent *fdep) { return (cap_rights_fde_inline(fdep)); } const cap_rights_t * cap_rights(struct filedesc *fdp, int fd) { return (cap_rights_fde(&fdp->fd_ofiles[fd])); } int kern_cap_rights_limit(struct thread *td, int fd, cap_rights_t *rights) { struct filedesc *fdp; struct filedescent *fdep; int error; fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); fdep = fdeget_locked(fdp, fd); if (fdep == NULL) { FILEDESC_XUNLOCK(fdp); return (EBADF); } error = _cap_check(cap_rights(fdp, fd), rights, CAPFAIL_INCREASE); if (error == 0) { fdep->fde_rights = *rights; if (!cap_rights_is_set(rights, CAP_IOCTL)) { free(fdep->fde_ioctls, M_FILECAPS); fdep->fde_ioctls = NULL; fdep->fde_nioctls = 0; } if (!cap_rights_is_set(rights, CAP_FCNTL)) fdep->fde_fcntls = 0; } FILEDESC_XUNLOCK(fdp); return (error); } /* * System call to limit rights of the given capability. */ int sys_cap_rights_limit(struct thread *td, struct cap_rights_limit_args *uap) { cap_rights_t rights; int error, version; - cap_rights_init(&rights); + cap_rights_init_zero(&rights); error = copyin(uap->rightsp, &rights, sizeof(rights.cr_rights[0])); if (error != 0) return (error); version = CAPVER(&rights); if (version != CAP_RIGHTS_VERSION_00) return (EINVAL); error = copyin(uap->rightsp, &rights, sizeof(rights.cr_rights[0]) * CAPARSIZE(&rights)); if (error != 0) return (error); /* Check for race. */ if (CAPVER(&rights) != version) return (EINVAL); if (!cap_rights_is_valid(&rights)) return (EINVAL); if (version != CAP_RIGHTS_VERSION) { rights.cr_rights[0] &= ~(0x3ULL << 62); rights.cr_rights[0] |= ((uint64_t)CAP_RIGHTS_VERSION << 62); } #ifdef KTRACE if (KTRPOINT(td, KTR_STRUCT)) ktrcaprights(&rights); #endif AUDIT_ARG_FD(uap->fd); AUDIT_ARG_RIGHTS(&rights); return (kern_cap_rights_limit(td, uap->fd, &rights)); } /* * System call to query the rights mask associated with a capability. */ int sys___cap_rights_get(struct thread *td, struct __cap_rights_get_args *uap) { struct filedesc *fdp; cap_rights_t rights; int error, fd, i, n; if (uap->version != CAP_RIGHTS_VERSION_00) return (EINVAL); fd = uap->fd; AUDIT_ARG_FD(fd); fdp = td->td_proc->p_fd; FILEDESC_SLOCK(fdp); if (fget_locked(fdp, fd) == NULL) { FILEDESC_SUNLOCK(fdp); return (EBADF); } rights = *cap_rights(fdp, fd); FILEDESC_SUNLOCK(fdp); n = uap->version + 2; if (uap->version != CAPVER(&rights)) { /* * For older versions we need to check if the descriptor * doesn't contain rights not understood by the caller. * If it does, we have to return an error. */ for (i = n; i < CAPARSIZE(&rights); i++) { if ((rights.cr_rights[i] & ~(0x7FULL << 57)) != 0) return (EINVAL); } } error = copyout(&rights, uap->rightsp, sizeof(rights.cr_rights[0]) * n); #ifdef KTRACE if (error == 0 && KTRPOINT(td, KTR_STRUCT)) ktrcaprights(&rights); #endif return (error); } /* * Test whether a capability grants the given ioctl command. * If descriptor doesn't have CAP_IOCTL, then ioctls list is empty and * ENOTCAPABLE will be returned. */ int cap_ioctl_check(struct filedesc *fdp, int fd, u_long cmd) { struct filedescent *fdep; u_long *cmds; ssize_t ncmds; long i; KASSERT(fd >= 0 && fd < fdp->fd_nfiles, ("%s: invalid fd=%d", __func__, fd)); fdep = fdeget_locked(fdp, fd); KASSERT(fdep != NULL, ("%s: invalid fd=%d", __func__, fd)); ncmds = fdep->fde_nioctls; if (ncmds == -1) return (0); cmds = fdep->fde_ioctls; for (i = 0; i < ncmds; i++) { if (cmds[i] == cmd) return (0); } return (ENOTCAPABLE); } /* * Check if the current ioctls list can be replaced by the new one. */ static int cap_ioctl_limit_check(struct filedescent *fdep, const u_long *cmds, size_t ncmds) { u_long *ocmds; ssize_t oncmds; u_long i; long j; oncmds = fdep->fde_nioctls; if (oncmds == -1) return (0); if (oncmds < (ssize_t)ncmds) return (ENOTCAPABLE); ocmds = fdep->fde_ioctls; for (i = 0; i < ncmds; i++) { for (j = 0; j < oncmds; j++) { if (cmds[i] == ocmds[j]) break; } if (j == oncmds) return (ENOTCAPABLE); } return (0); } int kern_cap_ioctls_limit(struct thread *td, int fd, u_long *cmds, size_t ncmds) { struct filedesc *fdp; struct filedescent *fdep; u_long *ocmds; int error; AUDIT_ARG_FD(fd); if (ncmds > IOCTLS_MAX_COUNT) { error = EINVAL; goto out_free; } fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); fdep = fdeget_locked(fdp, fd); if (fdep == NULL) { error = EBADF; goto out; } error = cap_ioctl_limit_check(fdep, cmds, ncmds); if (error != 0) goto out; ocmds = fdep->fde_ioctls; fdep->fde_ioctls = cmds; fdep->fde_nioctls = ncmds; cmds = ocmds; error = 0; out: FILEDESC_XUNLOCK(fdp); out_free: free(cmds, M_FILECAPS); return (error); } int sys_cap_ioctls_limit(struct thread *td, struct cap_ioctls_limit_args *uap) { u_long *cmds; size_t ncmds; int error; ncmds = uap->ncmds; if (ncmds > IOCTLS_MAX_COUNT) return (EINVAL); if (ncmds == 0) { cmds = NULL; } else { cmds = malloc(sizeof(cmds[0]) * ncmds, M_FILECAPS, M_WAITOK); error = copyin(uap->cmds, cmds, sizeof(cmds[0]) * ncmds); if (error != 0) { free(cmds, M_FILECAPS); return (error); } } return (kern_cap_ioctls_limit(td, uap->fd, cmds, ncmds)); } int sys_cap_ioctls_get(struct thread *td, struct cap_ioctls_get_args *uap) { struct filedesc *fdp; struct filedescent *fdep; u_long *cmdsp, *dstcmds; size_t maxcmds, ncmds; int16_t count; int error, fd; fd = uap->fd; dstcmds = uap->cmds; maxcmds = uap->maxcmds; AUDIT_ARG_FD(fd); fdp = td->td_proc->p_fd; cmdsp = NULL; if (dstcmds != NULL) { cmdsp = malloc(sizeof(cmdsp[0]) * IOCTLS_MAX_COUNT, M_FILECAPS, M_WAITOK | M_ZERO); } FILEDESC_SLOCK(fdp); fdep = fdeget_locked(fdp, fd); if (fdep == NULL) { error = EBADF; FILEDESC_SUNLOCK(fdp); goto out; } count = fdep->fde_nioctls; if (count != -1 && cmdsp != NULL) { ncmds = MIN(count, maxcmds); memcpy(cmdsp, fdep->fde_ioctls, sizeof(cmdsp[0]) * ncmds); } FILEDESC_SUNLOCK(fdp); /* * If all ioctls are allowed (fde_nioctls == -1 && fde_ioctls == NULL) * the only sane thing we can do is to not populate the given array and * return CAP_IOCTLS_ALL. */ if (count != -1) { if (cmdsp != NULL) { error = copyout(cmdsp, dstcmds, sizeof(cmdsp[0]) * ncmds); if (error != 0) goto out; } td->td_retval[0] = count; } else { td->td_retval[0] = CAP_IOCTLS_ALL; } error = 0; out: free(cmdsp, M_FILECAPS); return (error); } /* * Test whether a capability grants the given fcntl command. */ int cap_fcntl_check_fde(struct filedescent *fdep, int cmd) { uint32_t fcntlcap; fcntlcap = (1 << cmd); KASSERT((CAP_FCNTL_ALL & fcntlcap) != 0, ("Unsupported fcntl=%d.", cmd)); if ((fdep->fde_fcntls & fcntlcap) != 0) return (0); return (ENOTCAPABLE); } int cap_fcntl_check(struct filedesc *fdp, int fd, int cmd) { KASSERT(fd >= 0 && fd < fdp->fd_nfiles, ("%s: invalid fd=%d", __func__, fd)); return (cap_fcntl_check_fde(&fdp->fd_ofiles[fd], cmd)); } int sys_cap_fcntls_limit(struct thread *td, struct cap_fcntls_limit_args *uap) { struct filedesc *fdp; struct filedescent *fdep; uint32_t fcntlrights; int fd; fd = uap->fd; fcntlrights = uap->fcntlrights; AUDIT_ARG_FD(fd); AUDIT_ARG_FCNTL_RIGHTS(fcntlrights); if ((fcntlrights & ~CAP_FCNTL_ALL) != 0) return (EINVAL); fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); fdep = fdeget_locked(fdp, fd); if (fdep == NULL) { FILEDESC_XUNLOCK(fdp); return (EBADF); } if ((fcntlrights & ~fdep->fde_fcntls) != 0) { FILEDESC_XUNLOCK(fdp); return (ENOTCAPABLE); } fdep->fde_fcntls = fcntlrights; FILEDESC_XUNLOCK(fdp); return (0); } int sys_cap_fcntls_get(struct thread *td, struct cap_fcntls_get_args *uap) { struct filedesc *fdp; struct filedescent *fdep; uint32_t rights; int fd; fd = uap->fd; AUDIT_ARG_FD(fd); fdp = td->td_proc->p_fd; FILEDESC_SLOCK(fdp); fdep = fdeget_locked(fdp, fd); if (fdep == NULL) { FILEDESC_SUNLOCK(fdp); return (EBADF); } rights = fdep->fde_fcntls; FILEDESC_SUNLOCK(fdp); return (copyout(&rights, uap->fcntlrightsp, sizeof(rights))); } #else /* !CAPABILITIES */ /* * Stub Capability functions for when options CAPABILITIES isn't compiled * into the kernel. */ int sys_cap_rights_limit(struct thread *td, struct cap_rights_limit_args *uap) { return (ENOSYS); } int sys___cap_rights_get(struct thread *td, struct __cap_rights_get_args *uap) { return (ENOSYS); } int sys_cap_ioctls_limit(struct thread *td, struct cap_ioctls_limit_args *uap) { return (ENOSYS); } int sys_cap_ioctls_get(struct thread *td, struct cap_ioctls_get_args *uap) { return (ENOSYS); } int sys_cap_fcntls_limit(struct thread *td, struct cap_fcntls_limit_args *uap) { return (ENOSYS); } int sys_cap_fcntls_get(struct thread *td, struct cap_fcntls_get_args *uap) { return (ENOSYS); } #endif /* CAPABILITIES */ Index: projects/clang1000-import/sys/kern/vfs_acl.c =================================================================== --- projects/clang1000-import/sys/kern/vfs_acl.c (revision 357965) +++ projects/clang1000-import/sys/kern/vfs_acl.c (revision 357966) @@ -1,599 +1,599 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 1999-2006, 2016-2017 Robert N. M. Watson * All rights reserved. * * This software was developed by Robert Watson for the TrustedBSD Project. * * Portions of this software were developed by BAE Systems, the University of * Cambridge Computer Laboratory, and Memorial University under DARPA/AFRL * contract FA8650-15-C-7558 ("CADETS"), as part of the DARPA Transparent * Computing (TC) research program. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * Developed by the TrustedBSD Project. * * ACL system calls and other functions common across different ACL types. * Type-specific routines go into subr_acl_.c. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include CTASSERT(ACL_MAX_ENTRIES >= OLDACL_MAX_ENTRIES); MALLOC_DEFINE(M_ACL, "acl", "Access Control Lists"); static int kern___acl_aclcheck_path(struct thread *td, const char *path, acl_type_t type, struct acl *aclp, int follow); static int kern___acl_delete_path(struct thread *td, const char *path, acl_type_t type, int follow); static int kern___acl_get_path(struct thread *td, const char *path, acl_type_t type, struct acl *aclp, int follow); static int kern___acl_set_path(struct thread *td, const char *path, acl_type_t type, const struct acl *aclp, int follow); static int vacl_set_acl(struct thread *td, struct vnode *vp, acl_type_t type, const struct acl *aclp); static int vacl_get_acl(struct thread *td, struct vnode *vp, acl_type_t type, struct acl *aclp); static int vacl_aclcheck(struct thread *td, struct vnode *vp, acl_type_t type, const struct acl *aclp); int acl_copy_oldacl_into_acl(const struct oldacl *source, struct acl *dest) { int i; if (source->acl_cnt < 0 || source->acl_cnt > OLDACL_MAX_ENTRIES) return (EINVAL); bzero(dest, sizeof(*dest)); dest->acl_cnt = source->acl_cnt; dest->acl_maxcnt = ACL_MAX_ENTRIES; for (i = 0; i < dest->acl_cnt; i++) { dest->acl_entry[i].ae_tag = source->acl_entry[i].ae_tag; dest->acl_entry[i].ae_id = source->acl_entry[i].ae_id; dest->acl_entry[i].ae_perm = source->acl_entry[i].ae_perm; } return (0); } int acl_copy_acl_into_oldacl(const struct acl *source, struct oldacl *dest) { int i; if (source->acl_cnt > OLDACL_MAX_ENTRIES) return (EINVAL); bzero(dest, sizeof(*dest)); dest->acl_cnt = source->acl_cnt; for (i = 0; i < dest->acl_cnt; i++) { dest->acl_entry[i].ae_tag = source->acl_entry[i].ae_tag; dest->acl_entry[i].ae_id = source->acl_entry[i].ae_id; dest->acl_entry[i].ae_perm = source->acl_entry[i].ae_perm; } return (0); } /* * At one time, "struct ACL" was extended in order to add support for NFSv4 * ACLs. Instead of creating compatibility versions of all the ACL-related * syscalls, they were left intact. It's possible to find out what the code * calling these syscalls (libc) expects basing on "type" argument - if it's * either ACL_TYPE_ACCESS_OLD or ACL_TYPE_DEFAULT_OLD (which previously were * known as ACL_TYPE_ACCESS and ACL_TYPE_DEFAULT), then it's the "struct * oldacl". If it's something else, then it's the new "struct acl". In the * latter case, the routines below just copyin/copyout the contents. In the * former case, they copyin the "struct oldacl" and convert it to the new * format. */ static int acl_copyin(const void *user_acl, struct acl *kernel_acl, acl_type_t type) { int error; struct oldacl old; switch (type) { case ACL_TYPE_ACCESS_OLD: case ACL_TYPE_DEFAULT_OLD: error = copyin(user_acl, &old, sizeof(old)); if (error != 0) break; acl_copy_oldacl_into_acl(&old, kernel_acl); break; default: error = copyin(user_acl, kernel_acl, sizeof(*kernel_acl)); if (kernel_acl->acl_maxcnt != ACL_MAX_ENTRIES) return (EINVAL); } return (error); } static int acl_copyout(const struct acl *kernel_acl, void *user_acl, acl_type_t type) { uint32_t am; int error; struct oldacl old; switch (type) { case ACL_TYPE_ACCESS_OLD: case ACL_TYPE_DEFAULT_OLD: error = acl_copy_acl_into_oldacl(kernel_acl, &old); if (error != 0) break; error = copyout(&old, user_acl, sizeof(old)); break; default: error = fueword32((char *)user_acl + offsetof(struct acl, acl_maxcnt), &am); if (error == -1) return (EFAULT); if (am != ACL_MAX_ENTRIES) return (EINVAL); error = copyout(kernel_acl, user_acl, sizeof(*kernel_acl)); } return (error); } /* * Convert "old" type - ACL_TYPE_{ACCESS,DEFAULT}_OLD - into its "new" * counterpart. It's required for old (pre-NFSv4 ACLs) libc to work * with new kernel. Fixing 'type' for old binaries with new libc * is being done in lib/libc/posix1e/acl_support.c:_acl_type_unold(). */ static int acl_type_unold(int type) { switch (type) { case ACL_TYPE_ACCESS_OLD: return (ACL_TYPE_ACCESS); case ACL_TYPE_DEFAULT_OLD: return (ACL_TYPE_DEFAULT); default: return (type); } } /* * These calls wrap the real vnode operations, and are called by the syscall * code once the syscall has converted the path or file descriptor to a vnode * (unlocked). The aclp pointer is assumed still to point to userland, so * this should not be consumed within the kernel except by syscall code. * Other code should directly invoke VOP_{SET,GET}ACL. */ /* * Given a vnode, set its ACL. */ static int vacl_set_acl(struct thread *td, struct vnode *vp, acl_type_t type, const struct acl *aclp) { struct acl *inkernelacl; struct mount *mp; int error; AUDIT_ARG_VALUE(type); inkernelacl = acl_alloc(M_WAITOK); error = acl_copyin(aclp, inkernelacl, type); if (error != 0) goto out; error = vn_start_write(vp, &mp, V_WAIT | PCATCH); if (error != 0) goto out; vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); AUDIT_ARG_VNODE1(vp); #ifdef MAC error = mac_vnode_check_setacl(td->td_ucred, vp, type, inkernelacl); if (error != 0) goto out_unlock; #endif error = VOP_SETACL(vp, acl_type_unold(type), inkernelacl, td->td_ucred, td); #ifdef MAC out_unlock: #endif VOP_UNLOCK(vp); vn_finished_write(mp); out: acl_free(inkernelacl); return (error); } /* * Given a vnode, get its ACL. */ static int vacl_get_acl(struct thread *td, struct vnode *vp, acl_type_t type, struct acl *aclp) { struct acl *inkernelacl; int error; AUDIT_ARG_VALUE(type); inkernelacl = acl_alloc(M_WAITOK | M_ZERO); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); AUDIT_ARG_VNODE1(vp); #ifdef MAC error = mac_vnode_check_getacl(td->td_ucred, vp, type); if (error != 0) goto out; #endif error = VOP_GETACL(vp, acl_type_unold(type), inkernelacl, td->td_ucred, td); #ifdef MAC out: #endif VOP_UNLOCK(vp); if (error == 0) error = acl_copyout(inkernelacl, aclp, type); acl_free(inkernelacl); return (error); } /* * Given a vnode, delete its ACL. */ static int vacl_delete(struct thread *td, struct vnode *vp, acl_type_t type) { struct mount *mp; int error; AUDIT_ARG_VALUE(type); error = vn_start_write(vp, &mp, V_WAIT | PCATCH); if (error != 0) return (error); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); AUDIT_ARG_VNODE1(vp); #ifdef MAC error = mac_vnode_check_deleteacl(td->td_ucred, vp, type); if (error != 0) goto out; #endif error = VOP_SETACL(vp, acl_type_unold(type), 0, td->td_ucred, td); #ifdef MAC out: #endif VOP_UNLOCK(vp); vn_finished_write(mp); return (error); } /* * Given a vnode, check whether an ACL is appropriate for it * * XXXRW: No vnode lock held so can't audit vnode state...? */ static int vacl_aclcheck(struct thread *td, struct vnode *vp, acl_type_t type, const struct acl *aclp) { struct acl *inkernelacl; int error; inkernelacl = acl_alloc(M_WAITOK); error = acl_copyin(aclp, inkernelacl, type); if (error != 0) goto out; error = VOP_ACLCHECK(vp, acl_type_unold(type), inkernelacl, td->td_ucred, td); out: acl_free(inkernelacl); return (error); } /* * syscalls -- convert the path/fd to a vnode, and call vacl_whatever. Don't * need to lock, as the vacl_ code will get/release any locks required. */ /* * Given a file path, get an ACL for it */ int sys___acl_get_file(struct thread *td, struct __acl_get_file_args *uap) { return (kern___acl_get_path(td, uap->path, uap->type, uap->aclp, FOLLOW)); } /* * Given a file path, get an ACL for it; don't follow links. */ int sys___acl_get_link(struct thread *td, struct __acl_get_link_args *uap) { return(kern___acl_get_path(td, uap->path, uap->type, uap->aclp, NOFOLLOW)); } static int kern___acl_get_path(struct thread *td, const char *path, acl_type_t type, struct acl *aclp, int follow) { struct nameidata nd; int error; NDINIT(&nd, LOOKUP, follow | AUDITVNODE1, UIO_USERSPACE, path, td); error = namei(&nd); if (error == 0) { error = vacl_get_acl(td, nd.ni_vp, type, aclp); NDFREE(&nd, 0); } return (error); } /* * Given a file path, set an ACL for it. */ int sys___acl_set_file(struct thread *td, struct __acl_set_file_args *uap) { return(kern___acl_set_path(td, uap->path, uap->type, uap->aclp, FOLLOW)); } /* * Given a file path, set an ACL for it; don't follow links. */ int sys___acl_set_link(struct thread *td, struct __acl_set_link_args *uap) { return(kern___acl_set_path(td, uap->path, uap->type, uap->aclp, NOFOLLOW)); } static int kern___acl_set_path(struct thread *td, const char *path, acl_type_t type, const struct acl *aclp, int follow) { struct nameidata nd; int error; NDINIT(&nd, LOOKUP, follow | AUDITVNODE1, UIO_USERSPACE, path, td); error = namei(&nd); if (error == 0) { error = vacl_set_acl(td, nd.ni_vp, type, aclp); NDFREE(&nd, 0); } return (error); } /* * Given a file descriptor, get an ACL for it. */ int sys___acl_get_fd(struct thread *td, struct __acl_get_fd_args *uap) { struct file *fp; cap_rights_t rights; int error; AUDIT_ARG_FD(uap->filedes); error = getvnode(td, uap->filedes, - cap_rights_init(&rights, CAP_ACL_GET), &fp); + cap_rights_init_one(&rights, CAP_ACL_GET), &fp); if (error == 0) { error = vacl_get_acl(td, fp->f_vnode, uap->type, uap->aclp); fdrop(fp, td); } return (error); } /* * Given a file descriptor, set an ACL for it. */ int sys___acl_set_fd(struct thread *td, struct __acl_set_fd_args *uap) { struct file *fp; cap_rights_t rights; int error; AUDIT_ARG_FD(uap->filedes); error = getvnode(td, uap->filedes, - cap_rights_init(&rights, CAP_ACL_SET), &fp); + cap_rights_init_one(&rights, CAP_ACL_SET), &fp); if (error == 0) { error = vacl_set_acl(td, fp->f_vnode, uap->type, uap->aclp); fdrop(fp, td); } return (error); } /* * Given a file path, delete an ACL from it. */ int sys___acl_delete_file(struct thread *td, struct __acl_delete_file_args *uap) { return (kern___acl_delete_path(td, uap->path, uap->type, FOLLOW)); } /* * Given a file path, delete an ACL from it; don't follow links. */ int sys___acl_delete_link(struct thread *td, struct __acl_delete_link_args *uap) { return (kern___acl_delete_path(td, uap->path, uap->type, NOFOLLOW)); } static int kern___acl_delete_path(struct thread *td, const char *path, acl_type_t type, int follow) { struct nameidata nd; int error; NDINIT(&nd, LOOKUP, follow, UIO_USERSPACE, path, td); error = namei(&nd); if (error == 0) { error = vacl_delete(td, nd.ni_vp, type); NDFREE(&nd, 0); } return (error); } /* * Given a file path, delete an ACL from it. */ int sys___acl_delete_fd(struct thread *td, struct __acl_delete_fd_args *uap) { struct file *fp; cap_rights_t rights; int error; AUDIT_ARG_FD(uap->filedes); error = getvnode(td, uap->filedes, - cap_rights_init(&rights, CAP_ACL_DELETE), &fp); + cap_rights_init_one(&rights, CAP_ACL_DELETE), &fp); if (error == 0) { error = vacl_delete(td, fp->f_vnode, uap->type); fdrop(fp, td); } return (error); } /* * Given a file path, check an ACL for it. */ int sys___acl_aclcheck_file(struct thread *td, struct __acl_aclcheck_file_args *uap) { return (kern___acl_aclcheck_path(td, uap->path, uap->type, uap->aclp, FOLLOW)); } /* * Given a file path, check an ACL for it; don't follow links. */ int sys___acl_aclcheck_link(struct thread *td, struct __acl_aclcheck_link_args *uap) { return (kern___acl_aclcheck_path(td, uap->path, uap->type, uap->aclp, NOFOLLOW)); } static int kern___acl_aclcheck_path(struct thread *td, const char *path, acl_type_t type, struct acl *aclp, int follow) { struct nameidata nd; int error; NDINIT(&nd, LOOKUP, follow, UIO_USERSPACE, path, td); error = namei(&nd); if (error == 0) { error = vacl_aclcheck(td, nd.ni_vp, type, aclp); NDFREE(&nd, 0); } return (error); } /* * Given a file descriptor, check an ACL for it. */ int sys___acl_aclcheck_fd(struct thread *td, struct __acl_aclcheck_fd_args *uap) { struct file *fp; cap_rights_t rights; int error; AUDIT_ARG_FD(uap->filedes); error = getvnode(td, uap->filedes, - cap_rights_init(&rights, CAP_ACL_CHECK), &fp); + cap_rights_init_one(&rights, CAP_ACL_CHECK), &fp); if (error == 0) { error = vacl_aclcheck(td, fp->f_vnode, uap->type, uap->aclp); fdrop(fp, td); } return (error); } struct acl * acl_alloc(int flags) { struct acl *aclp; aclp = malloc(sizeof(*aclp), M_ACL, flags); if (aclp == NULL) return (NULL); aclp->acl_maxcnt = ACL_MAX_ENTRIES; return (aclp); } void acl_free(struct acl *aclp) { free(aclp, M_ACL); } Index: projects/clang1000-import/sys/kern/vfs_extattr.c =================================================================== --- projects/clang1000-import/sys/kern/vfs_extattr.c (revision 357965) +++ projects/clang1000-import/sys/kern/vfs_extattr.c (revision 357966) @@ -1,755 +1,755 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 1999-2001 Robert N. M. Watson * All rights reserved. * * This software was developed by Robert Watson for the TrustedBSD Project. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static int kern_extattr_set_path(struct thread *td, const char *path, int attrnamespace, const char *attrname, void *data, size_t nbytes, int follow); static int kern_extattr_get_path(struct thread *td, const char *path, int attrnamespace, const char *attrname, void *data, size_t nbytes, int follow); static int kern_extattr_delete_path(struct thread *td, const char *path, int attrnamespace, const char *attrname, int follow); static int kern_extattr_list_path(struct thread *td, const char *path, int attrnamespace, void *data, size_t nbytes, int follow); /* * Syscall to push extended attribute configuration information into the VFS. * Accepts a path, which it converts to a mountpoint, as well as a command * (int cmd), and attribute name and misc data. * * Currently this is used only by UFS1 extended attributes. */ #ifndef _SYS_SYSPROTO_H_ struct extattrctl_args { const char *path; int cmd; const char *filename; int attrnamespace; const char *attrname; }; #endif int sys_extattrctl(struct thread *td, struct extattrctl_args *uap) { struct vnode *filename_vp; struct nameidata nd; struct mount *mp, *mp_writable; char attrname[EXTATTR_MAXNAMELEN]; int error; AUDIT_ARG_CMD(uap->cmd); AUDIT_ARG_VALUE(uap->attrnamespace); /* * uap->attrname is not always defined. We check again later when we * invoke the VFS call so as to pass in NULL there if needed. */ if (uap->attrname != NULL) { error = copyinstr(uap->attrname, attrname, EXTATTR_MAXNAMELEN, NULL); if (error) return (error); } AUDIT_ARG_TEXT(attrname); mp = NULL; filename_vp = NULL; if (uap->filename != NULL) { NDINIT(&nd, LOOKUP, FOLLOW | AUDITVNODE2, UIO_USERSPACE, uap->filename, td); error = namei(&nd); if (error) return (error); filename_vp = nd.ni_vp; NDFREE(&nd, NDF_NO_VP_RELE); } /* uap->path is always defined. */ NDINIT(&nd, LOOKUP, FOLLOW | LOCKLEAF | AUDITVNODE1, UIO_USERSPACE, uap->path, td); error = namei(&nd); if (error) goto out; mp = nd.ni_vp->v_mount; error = vfs_busy(mp, 0); if (error) { NDFREE(&nd, 0); mp = NULL; goto out; } VOP_UNLOCK(nd.ni_vp); error = vn_start_write(nd.ni_vp, &mp_writable, V_WAIT | PCATCH); NDFREE(&nd, NDF_NO_VP_UNLOCK); if (error) goto out; if (filename_vp != NULL) { /* * uap->filename is not always defined. If it is, * grab a vnode lock, which VFS_EXTATTRCTL() will * later release. */ error = vn_lock(filename_vp, LK_EXCLUSIVE); if (error) { vn_finished_write(mp_writable); goto out; } } error = VFS_EXTATTRCTL(mp, uap->cmd, filename_vp, uap->attrnamespace, uap->attrname != NULL ? attrname : NULL); vn_finished_write(mp_writable); out: if (mp != NULL) vfs_unbusy(mp); /* * VFS_EXTATTRCTL will have unlocked, but not de-ref'd, filename_vp, * so vrele it if it is defined. */ if (filename_vp != NULL) vrele(filename_vp); return (error); } /*- * Set a named extended attribute on a file or directory * * Arguments: unlocked vnode "vp", attribute namespace "attrnamespace", * kernelspace string pointer "attrname", userspace buffer * pointer "data", buffer length "nbytes", thread "td". * Returns: 0 on success, an error number otherwise * Locks: none * References: vp must be a valid reference for the duration of the call */ static int extattr_set_vp(struct vnode *vp, int attrnamespace, const char *attrname, void *data, size_t nbytes, struct thread *td) { struct mount *mp; struct uio auio; struct iovec aiov; ssize_t cnt; int error; if (nbytes > IOSIZE_MAX) return (EINVAL); error = vn_start_write(vp, &mp, V_WAIT | PCATCH); if (error) return (error); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); aiov.iov_base = data; aiov.iov_len = nbytes; auio.uio_iov = &aiov; auio.uio_iovcnt = 1; auio.uio_offset = 0; auio.uio_resid = nbytes; auio.uio_rw = UIO_WRITE; auio.uio_segflg = UIO_USERSPACE; auio.uio_td = td; cnt = nbytes; #ifdef MAC error = mac_vnode_check_setextattr(td->td_ucred, vp, attrnamespace, attrname); if (error) goto done; #endif error = VOP_SETEXTATTR(vp, attrnamespace, attrname, &auio, td->td_ucred, td); cnt -= auio.uio_resid; td->td_retval[0] = cnt; #ifdef MAC done: #endif VOP_UNLOCK(vp); vn_finished_write(mp); return (error); } #ifndef _SYS_SYSPROTO_H_ struct extattr_set_fd_args { int fd; int attrnamespace; const char *attrname; void *data; size_t nbytes; }; #endif int sys_extattr_set_fd(struct thread *td, struct extattr_set_fd_args *uap) { struct file *fp; char attrname[EXTATTR_MAXNAMELEN]; cap_rights_t rights; int error; AUDIT_ARG_FD(uap->fd); AUDIT_ARG_VALUE(uap->attrnamespace); error = copyinstr(uap->attrname, attrname, EXTATTR_MAXNAMELEN, NULL); if (error) return (error); AUDIT_ARG_TEXT(attrname); error = getvnode(td, uap->fd, - cap_rights_init(&rights, CAP_EXTATTR_SET), &fp); + cap_rights_init_one(&rights, CAP_EXTATTR_SET), &fp); if (error) return (error); error = extattr_set_vp(fp->f_vnode, uap->attrnamespace, attrname, uap->data, uap->nbytes, td); fdrop(fp, td); return (error); } #ifndef _SYS_SYSPROTO_H_ struct extattr_set_file_args { const char *path; int attrnamespace; const char *attrname; void *data; size_t nbytes; }; #endif int sys_extattr_set_file(struct thread *td, struct extattr_set_file_args *uap) { return (kern_extattr_set_path(td, uap->path, uap->attrnamespace, uap->attrname, uap->data, uap->nbytes, FOLLOW)); } #ifndef _SYS_SYSPROTO_H_ struct extattr_set_link_args { const char *path; int attrnamespace; const char *attrname; void *data; size_t nbytes; }; #endif int sys_extattr_set_link(struct thread *td, struct extattr_set_link_args *uap) { return (kern_extattr_set_path(td, uap->path, uap->attrnamespace, uap->attrname, uap->data, uap->nbytes, NOFOLLOW)); } static int kern_extattr_set_path(struct thread *td, const char *path, int attrnamespace, const char *uattrname, void *data, size_t nbytes, int follow) { struct nameidata nd; char attrname[EXTATTR_MAXNAMELEN]; int error; AUDIT_ARG_VALUE(attrnamespace); error = copyinstr(uattrname, attrname, EXTATTR_MAXNAMELEN, NULL); if (error) return (error); AUDIT_ARG_TEXT(attrname); NDINIT(&nd, LOOKUP, follow | AUDITVNODE1, UIO_USERSPACE, path, td); error = namei(&nd); if (error) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = extattr_set_vp(nd.ni_vp, attrnamespace, attrname, data, nbytes, td); vrele(nd.ni_vp); return (error); } /*- * Get a named extended attribute on a file or directory * * Arguments: unlocked vnode "vp", attribute namespace "attrnamespace", * kernelspace string pointer "attrname", userspace buffer * pointer "data", buffer length "nbytes", thread "td". * Returns: 0 on success, an error number otherwise * Locks: none * References: vp must be a valid reference for the duration of the call */ static int extattr_get_vp(struct vnode *vp, int attrnamespace, const char *attrname, void *data, size_t nbytes, struct thread *td) { struct uio auio, *auiop; struct iovec aiov; ssize_t cnt; size_t size, *sizep; int error; if (nbytes > IOSIZE_MAX) return (EINVAL); vn_lock(vp, LK_SHARED | LK_RETRY); /* * Slightly unusual semantics: if the user provides a NULL data * pointer, they don't want to receive the data, just the maximum * read length. */ auiop = NULL; sizep = NULL; cnt = 0; if (data != NULL) { aiov.iov_base = data; aiov.iov_len = nbytes; auio.uio_iov = &aiov; auio.uio_iovcnt = 1; auio.uio_offset = 0; auio.uio_resid = nbytes; auio.uio_rw = UIO_READ; auio.uio_segflg = UIO_USERSPACE; auio.uio_td = td; auiop = &auio; cnt = nbytes; } else sizep = &size; #ifdef MAC error = mac_vnode_check_getextattr(td->td_ucred, vp, attrnamespace, attrname); if (error) goto done; #endif error = VOP_GETEXTATTR(vp, attrnamespace, attrname, auiop, sizep, td->td_ucred, td); if (auiop != NULL) { cnt -= auio.uio_resid; td->td_retval[0] = cnt; } else td->td_retval[0] = size; #ifdef MAC done: #endif VOP_UNLOCK(vp); return (error); } #ifndef _SYS_SYSPROTO_H_ struct extattr_get_fd_args { int fd; int attrnamespace; const char *attrname; void *data; size_t nbytes; }; #endif int sys_extattr_get_fd(struct thread *td, struct extattr_get_fd_args *uap) { struct file *fp; char attrname[EXTATTR_MAXNAMELEN]; cap_rights_t rights; int error; AUDIT_ARG_FD(uap->fd); AUDIT_ARG_VALUE(uap->attrnamespace); error = copyinstr(uap->attrname, attrname, EXTATTR_MAXNAMELEN, NULL); if (error) return (error); AUDIT_ARG_TEXT(attrname); error = getvnode(td, uap->fd, - cap_rights_init(&rights, CAP_EXTATTR_GET), &fp); + cap_rights_init_one(&rights, CAP_EXTATTR_GET), &fp); if (error) return (error); error = extattr_get_vp(fp->f_vnode, uap->attrnamespace, attrname, uap->data, uap->nbytes, td); fdrop(fp, td); return (error); } #ifndef _SYS_SYSPROTO_H_ struct extattr_get_file_args { const char *path; int attrnamespace; const char *attrname; void *data; size_t nbytes; }; #endif int sys_extattr_get_file(struct thread *td, struct extattr_get_file_args *uap) { return (kern_extattr_get_path(td, uap->path, uap->attrnamespace, uap->attrname, uap->data, uap->nbytes, FOLLOW)); } #ifndef _SYS_SYSPROTO_H_ struct extattr_get_link_args { const char *path; int attrnamespace; const char *attrname; void *data; size_t nbytes; }; #endif int sys_extattr_get_link(struct thread *td, struct extattr_get_link_args *uap) { return (kern_extattr_get_path(td, uap->path, uap->attrnamespace, uap->attrname, uap->data, uap->nbytes, NOFOLLOW)); } static int kern_extattr_get_path(struct thread *td, const char *path, int attrnamespace, const char *uattrname, void *data, size_t nbytes, int follow) { struct nameidata nd; char attrname[EXTATTR_MAXNAMELEN]; int error; AUDIT_ARG_VALUE(attrnamespace); error = copyinstr(uattrname, attrname, EXTATTR_MAXNAMELEN, NULL); if (error) return (error); AUDIT_ARG_TEXT(attrname); NDINIT(&nd, LOOKUP, follow | AUDITVNODE1, UIO_USERSPACE, path, td); error = namei(&nd); if (error) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = extattr_get_vp(nd.ni_vp, attrnamespace, attrname, data, nbytes, td); vrele(nd.ni_vp); return (error); } /* * extattr_delete_vp(): Delete a named extended attribute on a file or * directory * * Arguments: unlocked vnode "vp", attribute namespace "attrnamespace", * kernelspace string pointer "attrname", proc "p" * Returns: 0 on success, an error number otherwise * Locks: none * References: vp must be a valid reference for the duration of the call */ static int extattr_delete_vp(struct vnode *vp, int attrnamespace, const char *attrname, struct thread *td) { struct mount *mp; int error; error = vn_start_write(vp, &mp, V_WAIT | PCATCH); if (error) return (error); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); #ifdef MAC error = mac_vnode_check_deleteextattr(td->td_ucred, vp, attrnamespace, attrname); if (error) goto done; #endif error = VOP_DELETEEXTATTR(vp, attrnamespace, attrname, td->td_ucred, td); if (error == EOPNOTSUPP) error = VOP_SETEXTATTR(vp, attrnamespace, attrname, NULL, td->td_ucred, td); #ifdef MAC done: #endif VOP_UNLOCK(vp); vn_finished_write(mp); return (error); } #ifndef _SYS_SYSPROTO_H_ struct extattr_delete_fd_args { int fd; int attrnamespace; const char *attrname; }; #endif int sys_extattr_delete_fd(struct thread *td, struct extattr_delete_fd_args *uap) { struct file *fp; char attrname[EXTATTR_MAXNAMELEN]; cap_rights_t rights; int error; AUDIT_ARG_FD(uap->fd); AUDIT_ARG_VALUE(uap->attrnamespace); error = copyinstr(uap->attrname, attrname, EXTATTR_MAXNAMELEN, NULL); if (error) return (error); AUDIT_ARG_TEXT(attrname); error = getvnode(td, uap->fd, - cap_rights_init(&rights, CAP_EXTATTR_DELETE), &fp); + cap_rights_init_one(&rights, CAP_EXTATTR_DELETE), &fp); if (error) return (error); error = extattr_delete_vp(fp->f_vnode, uap->attrnamespace, attrname, td); fdrop(fp, td); return (error); } #ifndef _SYS_SYSPROTO_H_ struct extattr_delete_file_args { const char *path; int attrnamespace; const char *attrname; }; #endif int sys_extattr_delete_file(struct thread *td, struct extattr_delete_file_args *uap) { return (kern_extattr_delete_path(td, uap->path, uap->attrnamespace, uap->attrname, FOLLOW)); } #ifndef _SYS_SYSPROTO_H_ struct extattr_delete_link_args { const char *path; int attrnamespace; const char *attrname; }; #endif int sys_extattr_delete_link(struct thread *td, struct extattr_delete_link_args *uap) { return (kern_extattr_delete_path(td, uap->path, uap->attrnamespace, uap->attrname, NOFOLLOW)); } static int kern_extattr_delete_path(struct thread *td, const char *path, int attrnamespace, const char *uattrname, int follow) { struct nameidata nd; char attrname[EXTATTR_MAXNAMELEN]; int error; AUDIT_ARG_VALUE(attrnamespace); error = copyinstr(uattrname, attrname, EXTATTR_MAXNAMELEN, NULL); if (error) return(error); AUDIT_ARG_TEXT(attrname); NDINIT(&nd, LOOKUP, follow | AUDITVNODE1, UIO_USERSPACE, path, td); error = namei(&nd); if (error) return(error); NDFREE(&nd, NDF_ONLY_PNBUF); error = extattr_delete_vp(nd.ni_vp, attrnamespace, attrname, td); vrele(nd.ni_vp); return(error); } /*- * Retrieve a list of extended attributes on a file or directory. * * Arguments: unlocked vnode "vp", attribute namespace 'attrnamespace", * userspace buffer pointer "data", buffer length "nbytes", * thread "td". * Returns: 0 on success, an error number otherwise * Locks: none * References: vp must be a valid reference for the duration of the call */ static int extattr_list_vp(struct vnode *vp, int attrnamespace, void *data, size_t nbytes, struct thread *td) { struct uio auio, *auiop; size_t size, *sizep; struct iovec aiov; ssize_t cnt; int error; if (nbytes > IOSIZE_MAX) return (EINVAL); auiop = NULL; sizep = NULL; cnt = 0; if (data != NULL) { aiov.iov_base = data; aiov.iov_len = nbytes; auio.uio_iov = &aiov; auio.uio_iovcnt = 1; auio.uio_offset = 0; auio.uio_resid = nbytes; auio.uio_rw = UIO_READ; auio.uio_segflg = UIO_USERSPACE; auio.uio_td = td; auiop = &auio; cnt = nbytes; } else sizep = &size; vn_lock(vp, LK_SHARED | LK_RETRY); #ifdef MAC error = mac_vnode_check_listextattr(td->td_ucred, vp, attrnamespace); if (error) { VOP_UNLOCK(vp); return (error); } #endif error = VOP_LISTEXTATTR(vp, attrnamespace, auiop, sizep, td->td_ucred, td); VOP_UNLOCK(vp); if (auiop != NULL) { cnt -= auio.uio_resid; td->td_retval[0] = cnt; } else td->td_retval[0] = size; return (error); } #ifndef _SYS_SYSPROTO_H_ struct extattr_list_fd_args { int fd; int attrnamespace; void *data; size_t nbytes; }; #endif int sys_extattr_list_fd(struct thread *td, struct extattr_list_fd_args *uap) { struct file *fp; cap_rights_t rights; int error; AUDIT_ARG_FD(uap->fd); AUDIT_ARG_VALUE(uap->attrnamespace); error = getvnode(td, uap->fd, - cap_rights_init(&rights, CAP_EXTATTR_LIST), &fp); + cap_rights_init_one(&rights, CAP_EXTATTR_LIST), &fp); if (error) return (error); error = extattr_list_vp(fp->f_vnode, uap->attrnamespace, uap->data, uap->nbytes, td); fdrop(fp, td); return (error); } #ifndef _SYS_SYSPROTO_H_ struct extattr_list_file_args { const char *path; int attrnamespace; void *data; size_t nbytes; } #endif int sys_extattr_list_file(struct thread *td, struct extattr_list_file_args *uap) { return (kern_extattr_list_path(td, uap->path, uap->attrnamespace, uap->data, uap->nbytes, FOLLOW)); } #ifndef _SYS_SYSPROTO_H_ struct extattr_list_link_args { const char *path; int attrnamespace; void *data; size_t nbytes; }; #endif int sys_extattr_list_link(struct thread *td, struct extattr_list_link_args *uap) { return (kern_extattr_list_path(td, uap->path, uap->attrnamespace, uap->data, uap->nbytes, NOFOLLOW)); } static int kern_extattr_list_path(struct thread *td, const char *path, int attrnamespace, void *data, size_t nbytes, int follow) { struct nameidata nd; int error; AUDIT_ARG_VALUE(attrnamespace); NDINIT(&nd, LOOKUP, follow | AUDITVNODE1, UIO_USERSPACE, path, td); error = namei(&nd); if (error) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = extattr_list_vp(nd.ni_vp, attrnamespace, data, nbytes, td); vrele(nd.ni_vp); return (error); } Index: projects/clang1000-import/sys/kern/vfs_lookup.c =================================================================== --- projects/clang1000-import/sys/kern/vfs_lookup.c (revision 357965) +++ projects/clang1000-import/sys/kern/vfs_lookup.c (revision 357966) @@ -1,1515 +1,1515 @@ /*- * SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 1982, 1986, 1989, 1993 * The Regents of the University of California. All rights reserved. * (c) UNIX System Laboratories, Inc. * All or some portions of this file are derived from material licensed * to the University of California by American Telephone and Telegraph * Co. or Unix System Laboratories, Inc. and are reproduced herein with * the permission of UNIX System Laboratories, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)vfs_lookup.c 8.4 (Berkeley) 2/16/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_ktrace.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef KTRACE #include #endif #include #include #include #define NAMEI_DIAGNOSTIC 1 #undef NAMEI_DIAGNOSTIC SDT_PROVIDER_DECLARE(vfs); SDT_PROBE_DEFINE3(vfs, namei, lookup, entry, "struct vnode *", "char *", "unsigned long"); SDT_PROBE_DEFINE2(vfs, namei, lookup, return, "int", "struct vnode *"); /* Allocation zone for namei. */ uma_zone_t namei_zone; /* Placeholder vnode for mp traversal. */ static struct vnode *vp_crossmp; static int crossmp_vop_islocked(struct vop_islocked_args *ap) { return (LK_SHARED); } static int crossmp_vop_lock1(struct vop_lock1_args *ap) { struct vnode *vp; struct lock *lk __unused; const char *file __unused; int flags, line __unused; vp = ap->a_vp; lk = vp->v_vnlock; flags = ap->a_flags; file = ap->a_file; line = ap->a_line; if ((flags & LK_SHARED) == 0) panic("invalid lock request for crossmp"); WITNESS_CHECKORDER(&lk->lock_object, LOP_NEWORDER, file, line, flags & LK_INTERLOCK ? &VI_MTX(vp)->lock_object : NULL); WITNESS_LOCK(&lk->lock_object, 0, file, line); if ((flags & LK_INTERLOCK) != 0) VI_UNLOCK(vp); LOCK_LOG_LOCK("SLOCK", &lk->lock_object, 0, 0, ap->a_file, line); return (0); } static int crossmp_vop_unlock(struct vop_unlock_args *ap) { struct vnode *vp; struct lock *lk __unused; vp = ap->a_vp; lk = vp->v_vnlock; WITNESS_UNLOCK(&lk->lock_object, 0, LOCK_FILE, LOCK_LINE); LOCK_LOG_LOCK("SUNLOCK", &lk->lock_object, 0, 0, LOCK_FILE, LOCK_LINE); return (0); } static struct vop_vector crossmp_vnodeops = { .vop_default = &default_vnodeops, .vop_islocked = crossmp_vop_islocked, .vop_lock1 = crossmp_vop_lock1, .vop_unlock = crossmp_vop_unlock, }; /* * VFS_VOP_VECTOR_REGISTER(crossmp_vnodeops) is not used here since the vnode * gets allocated early. See nameiinit for the direct call below. */ struct nameicap_tracker { struct vnode *dp; TAILQ_ENTRY(nameicap_tracker) nm_link; }; /* Zone for cap mode tracker elements used for dotdot capability checks. */ static uma_zone_t nt_zone; static void nameiinit(void *dummy __unused) { namei_zone = uma_zcreate("NAMEI", MAXPATHLEN, NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); nt_zone = uma_zcreate("rentr", sizeof(struct nameicap_tracker), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); vfs_vector_op_register(&crossmp_vnodeops); getnewvnode("crossmp", NULL, &crossmp_vnodeops, &vp_crossmp); } SYSINIT(vfs, SI_SUB_VFS, SI_ORDER_SECOND, nameiinit, NULL); static int lookup_cap_dotdot = 1; SYSCTL_INT(_vfs, OID_AUTO, lookup_cap_dotdot, CTLFLAG_RWTUN, &lookup_cap_dotdot, 0, "enables \"..\" components in path lookup in capability mode"); static int lookup_cap_dotdot_nonlocal = 1; SYSCTL_INT(_vfs, OID_AUTO, lookup_cap_dotdot_nonlocal, CTLFLAG_RWTUN, &lookup_cap_dotdot_nonlocal, 0, "enables \"..\" components in path lookup in capability mode " "on non-local mount"); static void nameicap_tracker_add(struct nameidata *ndp, struct vnode *dp) { struct nameicap_tracker *nt; if ((ndp->ni_lcf & NI_LCF_CAP_DOTDOT) == 0 || dp->v_type != VDIR) return; if ((ndp->ni_lcf & (NI_LCF_BENEATH_ABS | NI_LCF_BENEATH_LATCHED)) == NI_LCF_BENEATH_ABS) { MPASS((ndp->ni_lcf & NI_LCF_LATCH) != 0); if (dp != ndp->ni_beneath_latch) return; ndp->ni_lcf |= NI_LCF_BENEATH_LATCHED; } nt = uma_zalloc(nt_zone, M_WAITOK); vhold(dp); nt->dp = dp; TAILQ_INSERT_TAIL(&ndp->ni_cap_tracker, nt, nm_link); } static void nameicap_cleanup(struct nameidata *ndp, bool clean_latch) { struct nameicap_tracker *nt, *nt1; KASSERT(TAILQ_EMPTY(&ndp->ni_cap_tracker) || (ndp->ni_lcf & NI_LCF_CAP_DOTDOT) != 0, ("not strictrelative")); TAILQ_FOREACH_SAFE(nt, &ndp->ni_cap_tracker, nm_link, nt1) { TAILQ_REMOVE(&ndp->ni_cap_tracker, nt, nm_link); vdrop(nt->dp); uma_zfree(nt_zone, nt); } if (clean_latch && (ndp->ni_lcf & NI_LCF_LATCH) != 0) { ndp->ni_lcf &= ~NI_LCF_LATCH; vrele(ndp->ni_beneath_latch); } } /* * For dotdot lookups in capability mode, only allow the component * lookup to succeed if the resulting directory was already traversed * during the operation. Also fail dotdot lookups for non-local * filesystems, where external agents might assist local lookups to * escape the compartment. */ static int nameicap_check_dotdot(struct nameidata *ndp, struct vnode *dp) { struct nameicap_tracker *nt; struct mount *mp; if ((ndp->ni_lcf & NI_LCF_CAP_DOTDOT) == 0 || dp == NULL || dp->v_type != VDIR) return (0); mp = dp->v_mount; if (lookup_cap_dotdot_nonlocal == 0 && mp != NULL && (mp->mnt_flag & MNT_LOCAL) == 0) return (ENOTCAPABLE); TAILQ_FOREACH_REVERSE(nt, &ndp->ni_cap_tracker, nameicap_tracker_head, nm_link) { if (dp == nt->dp) return (0); } if ((ndp->ni_lcf & NI_LCF_BENEATH_ABS) != 0) { ndp->ni_lcf &= ~NI_LCF_BENEATH_LATCHED; nameicap_cleanup(ndp, false); return (0); } return (ENOTCAPABLE); } static void namei_cleanup_cnp(struct componentname *cnp) { uma_zfree(namei_zone, cnp->cn_pnbuf); #ifdef DIAGNOSTIC cnp->cn_pnbuf = NULL; cnp->cn_nameptr = NULL; #endif } static int namei_handle_root(struct nameidata *ndp, struct vnode **dpp, u_int n) { struct componentname *cnp; cnp = &ndp->ni_cnd; if ((ndp->ni_lcf & NI_LCF_STRICTRELATIVE) != 0) { #ifdef KTRACE if (KTRPOINT(curthread, KTR_CAPFAIL)) ktrcapfail(CAPFAIL_LOOKUP, NULL, NULL); #endif return (ENOTCAPABLE); } if ((cnp->cn_flags & BENEATH) != 0) { ndp->ni_lcf |= NI_LCF_BENEATH_ABS; ndp->ni_lcf &= ~NI_LCF_BENEATH_LATCHED; nameicap_cleanup(ndp, false); } while (*(cnp->cn_nameptr) == '/') { cnp->cn_nameptr++; ndp->ni_pathlen--; } *dpp = ndp->ni_rootdir; vrefactn(*dpp, n); return (0); } /* * Convert a pathname into a pointer to a locked vnode. * * The FOLLOW flag is set when symbolic links are to be followed * when they occur at the end of the name translation process. * Symbolic links are always followed for all other pathname * components other than the last. * * The segflg defines whether the name is to be copied from user * space or kernel space. * * Overall outline of namei: * * copy in name * get starting directory * while (!done && !error) { * call lookup to search path. * if symbolic link, massage name in buffer and continue * } */ int namei(struct nameidata *ndp) { struct filedesc *fdp; /* pointer to file descriptor state */ char *cp; /* pointer into pathname argument */ struct vnode *dp; /* the directory we are searching */ struct iovec aiov; /* uio for reading symbolic links */ struct componentname *cnp; struct file *dfp; struct thread *td; struct proc *p; cap_rights_t rights; struct filecaps dirfd_caps; struct uio auio; int error, linklen, startdir_used; cnp = &ndp->ni_cnd; td = cnp->cn_thread; p = td->td_proc; ndp->ni_cnd.cn_cred = ndp->ni_cnd.cn_thread->td_ucred; KASSERT(cnp->cn_cred && p, ("namei: bad cred/proc")); KASSERT((cnp->cn_nameiop & (~OPMASK)) == 0, ("namei: nameiop contaminated with flags")); KASSERT((cnp->cn_flags & OPMASK) == 0, ("namei: flags contaminated with nameiops")); MPASS(ndp->ni_startdir == NULL || ndp->ni_startdir->v_type == VDIR || ndp->ni_startdir->v_type == VBAD); fdp = p->p_fd; TAILQ_INIT(&ndp->ni_cap_tracker); ndp->ni_lcf = 0; /* We will set this ourselves if we need it. */ cnp->cn_flags &= ~TRAILINGSLASH; /* * Get a buffer for the name to be translated, and copy the * name into the buffer. */ if ((cnp->cn_flags & HASBUF) == 0) cnp->cn_pnbuf = uma_zalloc(namei_zone, M_WAITOK); if (ndp->ni_segflg == UIO_SYSSPACE) error = copystr(ndp->ni_dirp, cnp->cn_pnbuf, MAXPATHLEN, &ndp->ni_pathlen); else error = copyinstr(ndp->ni_dirp, cnp->cn_pnbuf, MAXPATHLEN, &ndp->ni_pathlen); /* * Don't allow empty pathnames. */ if (error == 0 && *cnp->cn_pnbuf == '\0') error = ENOENT; #ifdef CAPABILITY_MODE /* * In capability mode, lookups must be restricted to happen in * the subtree with the root specified by the file descriptor: * - The root must be real file descriptor, not the pseudo-descriptor * AT_FDCWD. * - The passed path must be relative and not absolute. * - If lookup_cap_dotdot is disabled, path must not contain the * '..' components. * - If lookup_cap_dotdot is enabled, we verify that all '..' * components lookups result in the directories which were * previously walked by us, which prevents an escape from * the relative root. */ if (error == 0 && IN_CAPABILITY_MODE(td) && (cnp->cn_flags & NOCAPCHECK) == 0) { ndp->ni_lcf |= NI_LCF_STRICTRELATIVE; if (ndp->ni_dirfd == AT_FDCWD) { #ifdef KTRACE if (KTRPOINT(td, KTR_CAPFAIL)) ktrcapfail(CAPFAIL_LOOKUP, NULL, NULL); #endif error = ECAPMODE; } } #endif if (error != 0) { namei_cleanup_cnp(cnp); ndp->ni_vp = NULL; return (error); } ndp->ni_loopcnt = 0; #ifdef KTRACE if (KTRPOINT(td, KTR_NAMEI)) { KASSERT(cnp->cn_thread == curthread, ("namei not using curthread")); ktrnamei(cnp->cn_pnbuf); } #endif /* * Get starting point for the translation. */ FILEDESC_SLOCK(fdp); /* * The reference on ni_rootdir is acquired in the block below to avoid * back-to-back atomics for absolute lookups. */ ndp->ni_rootdir = fdp->fd_rdir; ndp->ni_topdir = fdp->fd_jdir; /* * If we are auditing the kernel pathname, save the user pathname. */ if (cnp->cn_flags & AUDITVNODE1) AUDIT_ARG_UPATH1(td, ndp->ni_dirfd, cnp->cn_pnbuf); if (cnp->cn_flags & AUDITVNODE2) AUDIT_ARG_UPATH2(td, ndp->ni_dirfd, cnp->cn_pnbuf); startdir_used = 0; dp = NULL; cnp->cn_nameptr = cnp->cn_pnbuf; if (cnp->cn_pnbuf[0] == '/') { ndp->ni_resflags |= NIRES_ABS; error = namei_handle_root(ndp, &dp, 2); if (error != 0) { /* * Simplify error handling, we should almost never be * here. */ vrefact(ndp->ni_rootdir); } } else { if (ndp->ni_startdir != NULL) { vrefact(ndp->ni_rootdir); dp = ndp->ni_startdir; startdir_used = 1; } else if (ndp->ni_dirfd == AT_FDCWD) { dp = fdp->fd_cdir; if (dp == ndp->ni_rootdir) { vrefactn(dp, 2); } else { vrefact(ndp->ni_rootdir); vrefact(dp); } } else { vrefact(ndp->ni_rootdir); rights = ndp->ni_rightsneeded; - cap_rights_set(&rights, CAP_LOOKUP); + cap_rights_set_one(&rights, CAP_LOOKUP); if (cnp->cn_flags & AUDITVNODE1) AUDIT_ARG_ATFD1(ndp->ni_dirfd); if (cnp->cn_flags & AUDITVNODE2) AUDIT_ARG_ATFD2(ndp->ni_dirfd); /* * Effectively inlined fgetvp_rights, because we need to * inspect the file as well as grabbing the vnode. */ error = fget_cap_locked(fdp, ndp->ni_dirfd, &rights, &dfp, &ndp->ni_filecaps); if (error != 0) { /* * Preserve the error; it should either be EBADF * or capability-related, both of which can be * safely returned to the caller. */ } else if (dfp->f_ops == &badfileops) { error = EBADF; } else if (dfp->f_vnode == NULL) { error = ENOTDIR; } else { dp = dfp->f_vnode; vrefact(dp); if ((dfp->f_flag & FSEARCH) != 0) cnp->cn_flags |= NOEXECCHECK; } #ifdef CAPABILITIES /* * If file descriptor doesn't have all rights, * all lookups relative to it must also be * strictly relative. */ CAP_ALL(&rights); if (!cap_rights_contains(&ndp->ni_filecaps.fc_rights, &rights) || ndp->ni_filecaps.fc_fcntls != CAP_FCNTL_ALL || ndp->ni_filecaps.fc_nioctls != -1) { ndp->ni_lcf |= NI_LCF_STRICTRELATIVE; } #endif } if (error == 0 && dp->v_type != VDIR) error = ENOTDIR; } if (error == 0 && (cnp->cn_flags & BENEATH) != 0) { if (ndp->ni_dirfd == AT_FDCWD) { ndp->ni_beneath_latch = fdp->fd_cdir; vrefact(ndp->ni_beneath_latch); } else { rights = ndp->ni_rightsneeded; - cap_rights_set(&rights, CAP_LOOKUP); + cap_rights_set_one(&rights, CAP_LOOKUP); error = fgetvp_rights(td, ndp->ni_dirfd, &rights, &dirfd_caps, &ndp->ni_beneath_latch); if (error == 0 && dp->v_type != VDIR) { vrele(ndp->ni_beneath_latch); error = ENOTDIR; } } if (error == 0) ndp->ni_lcf |= NI_LCF_LATCH; } FILEDESC_SUNLOCK(fdp); if (ndp->ni_startdir != NULL && !startdir_used) vrele(ndp->ni_startdir); if (error != 0) { if (dp != NULL) vrele(dp); goto out; } MPASS((ndp->ni_lcf & (NI_LCF_BENEATH_ABS | NI_LCF_LATCH)) != NI_LCF_BENEATH_ABS); if (((ndp->ni_lcf & NI_LCF_STRICTRELATIVE) != 0 && lookup_cap_dotdot != 0) || ((ndp->ni_lcf & NI_LCF_STRICTRELATIVE) == 0 && (cnp->cn_flags & BENEATH) != 0)) ndp->ni_lcf |= NI_LCF_CAP_DOTDOT; SDT_PROBE3(vfs, namei, lookup, entry, dp, cnp->cn_pnbuf, cnp->cn_flags); for (;;) { ndp->ni_startdir = dp; error = lookup(ndp); if (error != 0) goto out; /* * If not a symbolic link, we're done. */ if ((cnp->cn_flags & ISSYMLINK) == 0) { vrele(ndp->ni_rootdir); if ((cnp->cn_flags & (SAVENAME | SAVESTART)) == 0) { namei_cleanup_cnp(cnp); } else cnp->cn_flags |= HASBUF; if ((ndp->ni_lcf & (NI_LCF_BENEATH_ABS | NI_LCF_BENEATH_LATCHED)) == NI_LCF_BENEATH_ABS) { NDFREE(ndp, 0); error = ENOTCAPABLE; } nameicap_cleanup(ndp, true); SDT_PROBE2(vfs, namei, lookup, return, error, (error == 0 ? ndp->ni_vp : NULL)); return (error); } if (ndp->ni_loopcnt++ >= MAXSYMLINKS) { error = ELOOP; break; } #ifdef MAC if ((cnp->cn_flags & NOMACCHECK) == 0) { error = mac_vnode_check_readlink(td->td_ucred, ndp->ni_vp); if (error != 0) break; } #endif if (ndp->ni_pathlen > 1) cp = uma_zalloc(namei_zone, M_WAITOK); else cp = cnp->cn_pnbuf; aiov.iov_base = cp; aiov.iov_len = MAXPATHLEN; auio.uio_iov = &aiov; auio.uio_iovcnt = 1; auio.uio_offset = 0; auio.uio_rw = UIO_READ; auio.uio_segflg = UIO_SYSSPACE; auio.uio_td = td; auio.uio_resid = MAXPATHLEN; error = VOP_READLINK(ndp->ni_vp, &auio, cnp->cn_cred); if (error != 0) { if (ndp->ni_pathlen > 1) uma_zfree(namei_zone, cp); break; } linklen = MAXPATHLEN - auio.uio_resid; if (linklen == 0) { if (ndp->ni_pathlen > 1) uma_zfree(namei_zone, cp); error = ENOENT; break; } if (linklen + ndp->ni_pathlen > MAXPATHLEN) { if (ndp->ni_pathlen > 1) uma_zfree(namei_zone, cp); error = ENAMETOOLONG; break; } if (ndp->ni_pathlen > 1) { bcopy(ndp->ni_next, cp + linklen, ndp->ni_pathlen); uma_zfree(namei_zone, cnp->cn_pnbuf); cnp->cn_pnbuf = cp; } else cnp->cn_pnbuf[linklen] = '\0'; ndp->ni_pathlen += linklen; vput(ndp->ni_vp); dp = ndp->ni_dvp; /* * Check if root directory should replace current directory. */ cnp->cn_nameptr = cnp->cn_pnbuf; if (*(cnp->cn_nameptr) == '/') { vrele(dp); error = namei_handle_root(ndp, &dp, 1); if (error != 0) goto out; } } vput(ndp->ni_vp); ndp->ni_vp = NULL; vrele(ndp->ni_dvp); out: vrele(ndp->ni_rootdir); MPASS(error != 0); namei_cleanup_cnp(cnp); nameicap_cleanup(ndp, true); SDT_PROBE2(vfs, namei, lookup, return, error, NULL); return (error); } static int compute_cn_lkflags(struct mount *mp, int lkflags, int cnflags) { if (mp == NULL || ((lkflags & LK_SHARED) && (!(mp->mnt_kern_flag & MNTK_LOOKUP_SHARED) || ((cnflags & ISDOTDOT) && (mp->mnt_kern_flag & MNTK_LOOKUP_EXCL_DOTDOT))))) { lkflags &= ~LK_SHARED; lkflags |= LK_EXCLUSIVE; } lkflags |= LK_NODDLKTREAT; return (lkflags); } static __inline int needs_exclusive_leaf(struct mount *mp, int flags) { /* * Intermediate nodes can use shared locks, we only need to * force an exclusive lock for leaf nodes. */ if ((flags & (ISLASTCN | LOCKLEAF)) != (ISLASTCN | LOCKLEAF)) return (0); /* Always use exclusive locks if LOCKSHARED isn't set. */ if (!(flags & LOCKSHARED)) return (1); /* * For lookups during open(), if the mount point supports * extended shared operations, then use a shared lock for the * leaf node, otherwise use an exclusive lock. */ if ((flags & ISOPEN) != 0) return (!MNT_EXTENDED_SHARED(mp)); /* * Lookup requests outside of open() that specify LOCKSHARED * only need a shared lock on the leaf vnode. */ return (0); } /* * Search a pathname. * This is a very central and rather complicated routine. * * The pathname is pointed to by ni_ptr and is of length ni_pathlen. * The starting directory is taken from ni_startdir. The pathname is * descended until done, or a symbolic link is encountered. The variable * ni_more is clear if the path is completed; it is set to one if a * symbolic link needing interpretation is encountered. * * The flag argument is LOOKUP, CREATE, RENAME, or DELETE depending on * whether the name is to be looked up, created, renamed, or deleted. * When CREATE, RENAME, or DELETE is specified, information usable in * creating, renaming, or deleting a directory entry may be calculated. * If flag has LOCKPARENT or'ed into it, the parent directory is returned * locked. If flag has WANTPARENT or'ed into it, the parent directory is * returned unlocked. Otherwise the parent directory is not returned. If * the target of the pathname exists and LOCKLEAF is or'ed into the flag * the target is returned locked, otherwise it is returned unlocked. * When creating or renaming and LOCKPARENT is specified, the target may not * be ".". When deleting and LOCKPARENT is specified, the target may be ".". * * Overall outline of lookup: * * dirloop: * identify next component of name at ndp->ni_ptr * handle degenerate case where name is null string * if .. and crossing mount points and on mounted filesys, find parent * call VOP_LOOKUP routine for next component name * directory vnode returned in ni_dvp, unlocked unless LOCKPARENT set * component vnode returned in ni_vp (if it exists), locked. * if result vnode is mounted on and crossing mount points, * find mounted on vnode * if more components of name, do next level at dirloop * return the answer in ni_vp, locked if LOCKLEAF set * if LOCKPARENT set, return locked parent in ni_dvp * if WANTPARENT set, return unlocked parent in ni_dvp */ int lookup(struct nameidata *ndp) { char *cp; /* pointer into pathname argument */ char *prev_ni_next; /* saved ndp->ni_next */ struct vnode *dp = NULL; /* the directory we are searching */ struct vnode *tdp; /* saved dp */ struct mount *mp; /* mount table entry */ struct prison *pr; size_t prev_ni_pathlen; /* saved ndp->ni_pathlen */ int docache; /* == 0 do not cache last component */ int wantparent; /* 1 => wantparent or lockparent flag */ int rdonly; /* lookup read-only flag bit */ int error = 0; int dpunlocked = 0; /* dp has already been unlocked */ int relookup = 0; /* do not consume the path component */ struct componentname *cnp = &ndp->ni_cnd; int lkflags_save; int ni_dvp_unlocked; /* * Setup: break out flag bits into variables. */ ni_dvp_unlocked = 0; wantparent = cnp->cn_flags & (LOCKPARENT | WANTPARENT); KASSERT(cnp->cn_nameiop == LOOKUP || wantparent, ("CREATE, DELETE, RENAME require LOCKPARENT or WANTPARENT.")); docache = (cnp->cn_flags & NOCACHE) ^ NOCACHE; if (cnp->cn_nameiop == DELETE || (wantparent && cnp->cn_nameiop != CREATE && cnp->cn_nameiop != LOOKUP)) docache = 0; rdonly = cnp->cn_flags & RDONLY; cnp->cn_flags &= ~ISSYMLINK; ndp->ni_dvp = NULL; /* * We use shared locks until we hit the parent of the last cn then * we adjust based on the requesting flags. */ cnp->cn_lkflags = LK_SHARED; dp = ndp->ni_startdir; ndp->ni_startdir = NULLVP; vn_lock(dp, compute_cn_lkflags(dp->v_mount, cnp->cn_lkflags | LK_RETRY, cnp->cn_flags)); dirloop: /* * Search a new directory. * * The last component of the filename is left accessible via * cnp->cn_nameptr for callers that need the name. Callers needing * the name set the SAVENAME flag. When done, they assume * responsibility for freeing the pathname buffer. */ for (cp = cnp->cn_nameptr; *cp != 0 && *cp != '/'; cp++) continue; cnp->cn_namelen = cp - cnp->cn_nameptr; if (cnp->cn_namelen > NAME_MAX) { error = ENAMETOOLONG; goto bad; } #ifdef NAMEI_DIAGNOSTIC { char c = *cp; *cp = '\0'; printf("{%s}: ", cnp->cn_nameptr); *cp = c; } #endif prev_ni_pathlen = ndp->ni_pathlen; ndp->ni_pathlen -= cnp->cn_namelen; KASSERT(ndp->ni_pathlen <= PATH_MAX, ("%s: ni_pathlen underflow to %zd\n", __func__, ndp->ni_pathlen)); prev_ni_next = ndp->ni_next; ndp->ni_next = cp; /* * Replace multiple slashes by a single slash and trailing slashes * by a null. This must be done before VOP_LOOKUP() because some * fs's don't know about trailing slashes. Remember if there were * trailing slashes to handle symlinks, existing non-directories * and non-existing files that won't be directories specially later. */ while (*cp == '/' && (cp[1] == '/' || cp[1] == '\0')) { cp++; ndp->ni_pathlen--; if (*cp == '\0') { *ndp->ni_next = '\0'; cnp->cn_flags |= TRAILINGSLASH; } } ndp->ni_next = cp; cnp->cn_flags |= MAKEENTRY; if (*cp == '\0' && docache == 0) cnp->cn_flags &= ~MAKEENTRY; if (cnp->cn_namelen == 2 && cnp->cn_nameptr[1] == '.' && cnp->cn_nameptr[0] == '.') cnp->cn_flags |= ISDOTDOT; else cnp->cn_flags &= ~ISDOTDOT; if (*ndp->ni_next == 0) cnp->cn_flags |= ISLASTCN; else cnp->cn_flags &= ~ISLASTCN; if ((cnp->cn_flags & ISLASTCN) != 0 && cnp->cn_namelen == 1 && cnp->cn_nameptr[0] == '.' && (cnp->cn_nameiop == DELETE || cnp->cn_nameiop == RENAME)) { error = EINVAL; goto bad; } nameicap_tracker_add(ndp, dp); /* * Check for degenerate name (e.g. / or "") * which is a way of talking about a directory, * e.g. like "/." or ".". */ if (cnp->cn_nameptr[0] == '\0') { if (dp->v_type != VDIR) { error = ENOTDIR; goto bad; } if (cnp->cn_nameiop != LOOKUP) { error = EISDIR; goto bad; } if (wantparent) { ndp->ni_dvp = dp; VREF(dp); } ndp->ni_vp = dp; if (cnp->cn_flags & AUDITVNODE1) AUDIT_ARG_VNODE1(dp); else if (cnp->cn_flags & AUDITVNODE2) AUDIT_ARG_VNODE2(dp); if (!(cnp->cn_flags & (LOCKPARENT | LOCKLEAF))) VOP_UNLOCK(dp); /* XXX This should probably move to the top of function. */ if (cnp->cn_flags & SAVESTART) panic("lookup: SAVESTART"); goto success; } /* * Handle "..": five special cases. * 0. If doing a capability lookup and lookup_cap_dotdot is * disabled, return ENOTCAPABLE. * 1. Return an error if this is the last component of * the name and the operation is DELETE or RENAME. * 2. If at root directory (e.g. after chroot) * or at absolute root directory * then ignore it so can't get out. * 3. If this vnode is the root of a mounted * filesystem, then replace it with the * vnode which was mounted on so we take the * .. in the other filesystem. * 4. If the vnode is the top directory of * the jail or chroot, don't let them out. * 5. If doing a capability lookup and lookup_cap_dotdot is * enabled, return ENOTCAPABLE if the lookup would escape * from the initial file descriptor directory. Checks are * done by ensuring that namei() already traversed the * result of dotdot lookup. */ if (cnp->cn_flags & ISDOTDOT) { if ((ndp->ni_lcf & (NI_LCF_STRICTRELATIVE | NI_LCF_CAP_DOTDOT)) == NI_LCF_STRICTRELATIVE) { #ifdef KTRACE if (KTRPOINT(curthread, KTR_CAPFAIL)) ktrcapfail(CAPFAIL_LOOKUP, NULL, NULL); #endif error = ENOTCAPABLE; goto bad; } if ((cnp->cn_flags & ISLASTCN) != 0 && (cnp->cn_nameiop == DELETE || cnp->cn_nameiop == RENAME)) { error = EINVAL; goto bad; } for (;;) { for (pr = cnp->cn_cred->cr_prison; pr != NULL; pr = pr->pr_parent) if (dp == pr->pr_root) break; if (dp == ndp->ni_rootdir || dp == ndp->ni_topdir || dp == rootvnode || pr != NULL || ((dp->v_vflag & VV_ROOT) != 0 && (cnp->cn_flags & NOCROSSMOUNT) != 0)) { ndp->ni_dvp = dp; ndp->ni_vp = dp; VREF(dp); goto nextname; } if ((dp->v_vflag & VV_ROOT) == 0) break; if (VN_IS_DOOMED(dp)) { /* forced unmount */ error = ENOENT; goto bad; } tdp = dp; dp = dp->v_mount->mnt_vnodecovered; VREF(dp); vput(tdp); vn_lock(dp, compute_cn_lkflags(dp->v_mount, cnp->cn_lkflags | LK_RETRY, ISDOTDOT)); error = nameicap_check_dotdot(ndp, dp); if (error != 0) { #ifdef KTRACE if (KTRPOINT(curthread, KTR_CAPFAIL)) ktrcapfail(CAPFAIL_LOOKUP, NULL, NULL); #endif goto bad; } } } /* * We now have a segment name to search for, and a directory to search. */ unionlookup: #ifdef MAC error = mac_vnode_check_lookup(cnp->cn_thread->td_ucred, dp, cnp); if (error) goto bad; #endif ndp->ni_dvp = dp; ndp->ni_vp = NULL; ASSERT_VOP_LOCKED(dp, "lookup"); /* * If we have a shared lock we may need to upgrade the lock for the * last operation. */ if ((cnp->cn_flags & LOCKPARENT) && (cnp->cn_flags & ISLASTCN) && dp != vp_crossmp && VOP_ISLOCKED(dp) == LK_SHARED) vn_lock(dp, LK_UPGRADE|LK_RETRY); if (VN_IS_DOOMED(dp)) { error = ENOENT; goto bad; } /* * If we're looking up the last component and we need an exclusive * lock, adjust our lkflags. */ if (needs_exclusive_leaf(dp->v_mount, cnp->cn_flags)) cnp->cn_lkflags = LK_EXCLUSIVE; #ifdef NAMEI_DIAGNOSTIC vn_printf(dp, "lookup in "); #endif lkflags_save = cnp->cn_lkflags; cnp->cn_lkflags = compute_cn_lkflags(dp->v_mount, cnp->cn_lkflags, cnp->cn_flags); error = VOP_LOOKUP(dp, &ndp->ni_vp, cnp); cnp->cn_lkflags = lkflags_save; if (error != 0) { KASSERT(ndp->ni_vp == NULL, ("leaf should be empty")); #ifdef NAMEI_DIAGNOSTIC printf("not found\n"); #endif if ((error == ENOENT) && (dp->v_vflag & VV_ROOT) && (dp->v_mount != NULL) && (dp->v_mount->mnt_flag & MNT_UNION)) { tdp = dp; dp = dp->v_mount->mnt_vnodecovered; VREF(dp); vput(tdp); vn_lock(dp, compute_cn_lkflags(dp->v_mount, cnp->cn_lkflags | LK_RETRY, cnp->cn_flags)); nameicap_tracker_add(ndp, dp); goto unionlookup; } if (error == ERELOOKUP) { vref(dp); ndp->ni_vp = dp; error = 0; relookup = 1; goto good; } if (error != EJUSTRETURN) goto bad; /* * At this point, we know we're at the end of the * pathname. If creating / renaming, we can consider * allowing the file or directory to be created / renamed, * provided we're not on a read-only filesystem. */ if (rdonly) { error = EROFS; goto bad; } /* trailing slash only allowed for directories */ if ((cnp->cn_flags & TRAILINGSLASH) && !(cnp->cn_flags & WILLBEDIR)) { error = ENOENT; goto bad; } if ((cnp->cn_flags & LOCKPARENT) == 0) VOP_UNLOCK(dp); /* * We return with ni_vp NULL to indicate that the entry * doesn't currently exist, leaving a pointer to the * (possibly locked) directory vnode in ndp->ni_dvp. */ if (cnp->cn_flags & SAVESTART) { ndp->ni_startdir = ndp->ni_dvp; VREF(ndp->ni_startdir); } goto success; } good: #ifdef NAMEI_DIAGNOSTIC printf("found\n"); #endif dp = ndp->ni_vp; /* * Check to see if the vnode has been mounted on; * if so find the root of the mounted filesystem. */ while (dp->v_type == VDIR && (mp = dp->v_mountedhere) && (cnp->cn_flags & NOCROSSMOUNT) == 0) { if (vfs_busy(mp, 0)) continue; vput(dp); if (dp != ndp->ni_dvp) vput(ndp->ni_dvp); else vrele(ndp->ni_dvp); vrefact(vp_crossmp); ndp->ni_dvp = vp_crossmp; error = VFS_ROOT(mp, compute_cn_lkflags(mp, cnp->cn_lkflags, cnp->cn_flags), &tdp); vfs_unbusy(mp); if (vn_lock(vp_crossmp, LK_SHARED | LK_NOWAIT)) panic("vp_crossmp exclusively locked or reclaimed"); if (error) { dpunlocked = 1; goto bad2; } ndp->ni_vp = dp = tdp; } /* * Check for symbolic link */ if ((dp->v_type == VLNK) && ((cnp->cn_flags & FOLLOW) || (cnp->cn_flags & TRAILINGSLASH) || *ndp->ni_next == '/')) { cnp->cn_flags |= ISSYMLINK; if (VN_IS_DOOMED(dp)) { /* * We can't know whether the directory was mounted with * NOSYMFOLLOW, so we can't follow safely. */ error = ENOENT; goto bad2; } if (dp->v_mount->mnt_flag & MNT_NOSYMFOLLOW) { error = EACCES; goto bad2; } /* * Symlink code always expects an unlocked dvp. */ if (ndp->ni_dvp != ndp->ni_vp) { VOP_UNLOCK(ndp->ni_dvp); ni_dvp_unlocked = 1; } goto success; } nextname: /* * Not a symbolic link that we will follow. Continue with the * next component if there is any; otherwise, we're done. */ KASSERT((cnp->cn_flags & ISLASTCN) || *ndp->ni_next == '/', ("lookup: invalid path state.")); if (relookup) { relookup = 0; ndp->ni_pathlen = prev_ni_pathlen; ndp->ni_next = prev_ni_next; if (ndp->ni_dvp != dp) vput(ndp->ni_dvp); else vrele(ndp->ni_dvp); goto dirloop; } if (cnp->cn_flags & ISDOTDOT) { error = nameicap_check_dotdot(ndp, ndp->ni_vp); if (error != 0) { #ifdef KTRACE if (KTRPOINT(curthread, KTR_CAPFAIL)) ktrcapfail(CAPFAIL_LOOKUP, NULL, NULL); #endif goto bad2; } } if (*ndp->ni_next == '/') { cnp->cn_nameptr = ndp->ni_next; while (*cnp->cn_nameptr == '/') { cnp->cn_nameptr++; ndp->ni_pathlen--; } if (ndp->ni_dvp != dp) vput(ndp->ni_dvp); else vrele(ndp->ni_dvp); goto dirloop; } /* * If we're processing a path with a trailing slash, * check that the end result is a directory. */ if ((cnp->cn_flags & TRAILINGSLASH) && dp->v_type != VDIR) { error = ENOTDIR; goto bad2; } /* * Disallow directory write attempts on read-only filesystems. */ if (rdonly && (cnp->cn_nameiop == DELETE || cnp->cn_nameiop == RENAME)) { error = EROFS; goto bad2; } if (cnp->cn_flags & SAVESTART) { ndp->ni_startdir = ndp->ni_dvp; VREF(ndp->ni_startdir); } if (!wantparent) { ni_dvp_unlocked = 2; if (ndp->ni_dvp != dp) vput(ndp->ni_dvp); else vrele(ndp->ni_dvp); } else if ((cnp->cn_flags & LOCKPARENT) == 0 && ndp->ni_dvp != dp) { VOP_UNLOCK(ndp->ni_dvp); ni_dvp_unlocked = 1; } if (cnp->cn_flags & AUDITVNODE1) AUDIT_ARG_VNODE1(dp); else if (cnp->cn_flags & AUDITVNODE2) AUDIT_ARG_VNODE2(dp); if ((cnp->cn_flags & LOCKLEAF) == 0) VOP_UNLOCK(dp); success: /* * Because of shared lookup we may have the vnode shared locked, but * the caller may want it to be exclusively locked. */ if (needs_exclusive_leaf(dp->v_mount, cnp->cn_flags) && VOP_ISLOCKED(dp) != LK_EXCLUSIVE) { vn_lock(dp, LK_UPGRADE | LK_RETRY); if (VN_IS_DOOMED(dp)) { error = ENOENT; goto bad2; } } return (0); bad2: if (ni_dvp_unlocked != 2) { if (dp != ndp->ni_dvp && !ni_dvp_unlocked) vput(ndp->ni_dvp); else vrele(ndp->ni_dvp); } bad: if (!dpunlocked) vput(dp); ndp->ni_vp = NULL; return (error); } /* * relookup - lookup a path name component * Used by lookup to re-acquire things. */ int relookup(struct vnode *dvp, struct vnode **vpp, struct componentname *cnp) { struct vnode *dp = NULL; /* the directory we are searching */ int wantparent; /* 1 => wantparent or lockparent flag */ int rdonly; /* lookup read-only flag bit */ int error = 0; KASSERT(cnp->cn_flags & ISLASTCN, ("relookup: Not given last component.")); /* * Setup: break out flag bits into variables. */ wantparent = cnp->cn_flags & (LOCKPARENT|WANTPARENT); KASSERT(wantparent, ("relookup: parent not wanted.")); rdonly = cnp->cn_flags & RDONLY; cnp->cn_flags &= ~ISSYMLINK; dp = dvp; cnp->cn_lkflags = LK_EXCLUSIVE; vn_lock(dp, LK_EXCLUSIVE | LK_RETRY); /* * Search a new directory. * * The last component of the filename is left accessible via * cnp->cn_nameptr for callers that need the name. Callers needing * the name set the SAVENAME flag. When done, they assume * responsibility for freeing the pathname buffer. */ #ifdef NAMEI_DIAGNOSTIC printf("{%s}: ", cnp->cn_nameptr); #endif /* * Check for "" which represents the root directory after slash * removal. */ if (cnp->cn_nameptr[0] == '\0') { /* * Support only LOOKUP for "/" because lookup() * can't succeed for CREATE, DELETE and RENAME. */ KASSERT(cnp->cn_nameiop == LOOKUP, ("nameiop must be LOOKUP")); KASSERT(dp->v_type == VDIR, ("dp is not a directory")); if (!(cnp->cn_flags & LOCKLEAF)) VOP_UNLOCK(dp); *vpp = dp; /* XXX This should probably move to the top of function. */ if (cnp->cn_flags & SAVESTART) panic("lookup: SAVESTART"); return (0); } if (cnp->cn_flags & ISDOTDOT) panic ("relookup: lookup on dot-dot"); /* * We now have a segment name to search for, and a directory to search. */ #ifdef NAMEI_DIAGNOSTIC vn_printf(dp, "search in "); #endif if ((error = VOP_LOOKUP(dp, vpp, cnp)) != 0) { KASSERT(*vpp == NULL, ("leaf should be empty")); if (error != EJUSTRETURN) goto bad; /* * If creating and at end of pathname, then can consider * allowing file to be created. */ if (rdonly) { error = EROFS; goto bad; } /* ASSERT(dvp == ndp->ni_startdir) */ if (cnp->cn_flags & SAVESTART) VREF(dvp); if ((cnp->cn_flags & LOCKPARENT) == 0) VOP_UNLOCK(dp); /* * We return with ni_vp NULL to indicate that the entry * doesn't currently exist, leaving a pointer to the * (possibly locked) directory vnode in ndp->ni_dvp. */ return (0); } dp = *vpp; /* * Disallow directory write attempts on read-only filesystems. */ if (rdonly && (cnp->cn_nameiop == DELETE || cnp->cn_nameiop == RENAME)) { if (dvp == dp) vrele(dvp); else vput(dvp); error = EROFS; goto bad; } /* * Set the parent lock/ref state to the requested state. */ if ((cnp->cn_flags & LOCKPARENT) == 0 && dvp != dp) { if (wantparent) VOP_UNLOCK(dvp); else vput(dvp); } else if (!wantparent) vrele(dvp); /* * Check for symbolic link */ KASSERT(dp->v_type != VLNK || !(cnp->cn_flags & FOLLOW), ("relookup: symlink found.\n")); /* ASSERT(dvp == ndp->ni_startdir) */ if (cnp->cn_flags & SAVESTART) VREF(dvp); if ((cnp->cn_flags & LOCKLEAF) == 0) VOP_UNLOCK(dp); return (0); bad: vput(dp); *vpp = NULL; return (error); } void NDINIT_ALL(struct nameidata *ndp, u_long op, u_long flags, enum uio_seg segflg, const char *namep, int dirfd, struct vnode *startdir, cap_rights_t *rightsp, struct thread *td) { ndp->ni_cnd.cn_nameiop = op; ndp->ni_cnd.cn_flags = flags; ndp->ni_segflg = segflg; ndp->ni_dirp = namep; ndp->ni_dirfd = dirfd; ndp->ni_startdir = startdir; ndp->ni_resflags = 0; filecaps_init(&ndp->ni_filecaps); ndp->ni_cnd.cn_thread = td; if (rightsp != NULL) ndp->ni_rightsneeded = *rightsp; else - cap_rights_init(&ndp->ni_rightsneeded); + cap_rights_init_zero(&ndp->ni_rightsneeded); } /* * Free data allocated by namei(); see namei(9) for details. */ void NDFREE(struct nameidata *ndp, const u_int flags) { int unlock_dvp; int unlock_vp; unlock_dvp = 0; unlock_vp = 0; if (!(flags & NDF_NO_FREE_PNBUF) && (ndp->ni_cnd.cn_flags & HASBUF)) { uma_zfree(namei_zone, ndp->ni_cnd.cn_pnbuf); ndp->ni_cnd.cn_flags &= ~HASBUF; } if (!(flags & NDF_NO_VP_UNLOCK) && (ndp->ni_cnd.cn_flags & LOCKLEAF) && ndp->ni_vp) unlock_vp = 1; if (!(flags & NDF_NO_DVP_UNLOCK) && (ndp->ni_cnd.cn_flags & LOCKPARENT) && ndp->ni_dvp != ndp->ni_vp) unlock_dvp = 1; if (!(flags & NDF_NO_VP_RELE) && ndp->ni_vp) { if (unlock_vp) { vput(ndp->ni_vp); unlock_vp = 0; } else vrele(ndp->ni_vp); ndp->ni_vp = NULL; } if (unlock_vp) VOP_UNLOCK(ndp->ni_vp); if (!(flags & NDF_NO_DVP_RELE) && (ndp->ni_cnd.cn_flags & (LOCKPARENT|WANTPARENT))) { if (unlock_dvp) { vput(ndp->ni_dvp); unlock_dvp = 0; } else vrele(ndp->ni_dvp); ndp->ni_dvp = NULL; } if (unlock_dvp) VOP_UNLOCK(ndp->ni_dvp); if (!(flags & NDF_NO_STARTDIR_RELE) && (ndp->ni_cnd.cn_flags & SAVESTART)) { vrele(ndp->ni_startdir); ndp->ni_startdir = NULL; } } /* * Determine if there is a suitable alternate filename under the specified * prefix for the specified path. If the create flag is set, then the * alternate prefix will be used so long as the parent directory exists. * This is used by the various compatibility ABIs so that Linux binaries prefer * files under /compat/linux for example. The chosen path (whether under * the prefix or under /) is returned in a kernel malloc'd buffer pointed * to by pathbuf. The caller is responsible for free'ing the buffer from * the M_TEMP bucket if one is returned. */ int kern_alternate_path(struct thread *td, const char *prefix, const char *path, enum uio_seg pathseg, char **pathbuf, int create, int dirfd) { struct nameidata nd, ndroot; char *ptr, *buf, *cp; size_t len, sz; int error; buf = (char *) malloc(MAXPATHLEN, M_TEMP, M_WAITOK); *pathbuf = buf; /* Copy the prefix into the new pathname as a starting point. */ len = strlcpy(buf, prefix, MAXPATHLEN); if (len >= MAXPATHLEN) { *pathbuf = NULL; free(buf, M_TEMP); return (EINVAL); } sz = MAXPATHLEN - len; ptr = buf + len; /* Append the filename to the prefix. */ if (pathseg == UIO_SYSSPACE) error = copystr(path, ptr, sz, &len); else error = copyinstr(path, ptr, sz, &len); if (error) { *pathbuf = NULL; free(buf, M_TEMP); return (error); } /* Only use a prefix with absolute pathnames. */ if (*ptr != '/') { error = EINVAL; goto keeporig; } if (dirfd != AT_FDCWD) { /* * We want the original because the "prefix" is * included in the already opened dirfd. */ bcopy(ptr, buf, len); return (0); } /* * We know that there is a / somewhere in this pathname. * Search backwards for it, to find the file's parent dir * to see if it exists in the alternate tree. If it does, * and we want to create a file (cflag is set). We don't * need to worry about the root comparison in this case. */ if (create) { for (cp = &ptr[len] - 1; *cp != '/'; cp--); *cp = '\0'; NDINIT(&nd, LOOKUP, NOFOLLOW, UIO_SYSSPACE, buf, td); error = namei(&nd); *cp = '/'; if (error != 0) goto keeporig; } else { NDINIT(&nd, LOOKUP, NOFOLLOW, UIO_SYSSPACE, buf, td); error = namei(&nd); if (error != 0) goto keeporig; /* * We now compare the vnode of the prefix to the one * vnode asked. If they resolve to be the same, then we * ignore the match so that the real root gets used. * This avoids the problem of traversing "../.." to find the * root directory and never finding it, because "/" resolves * to the emulation root directory. This is expensive :-( */ NDINIT(&ndroot, LOOKUP, FOLLOW, UIO_SYSSPACE, prefix, td); /* We shouldn't ever get an error from this namei(). */ error = namei(&ndroot); if (error == 0) { if (nd.ni_vp == ndroot.ni_vp) error = ENOENT; NDFREE(&ndroot, NDF_ONLY_PNBUF); vrele(ndroot.ni_vp); } } NDFREE(&nd, NDF_ONLY_PNBUF); vrele(nd.ni_vp); keeporig: /* If there was an error, use the original path name. */ if (error) bcopy(ptr, buf, len); return (error); } Index: projects/clang1000-import/sys/kern/vfs_subr.c =================================================================== --- projects/clang1000-import/sys/kern/vfs_subr.c (revision 357965) +++ projects/clang1000-import/sys/kern/vfs_subr.c (revision 357966) @@ -1,6463 +1,6463 @@ /*- * SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 1989, 1993 * The Regents of the University of California. All rights reserved. * (c) UNIX System Laboratories, Inc. * All or some portions of this file are derived from material licensed * to the University of California by American Telephone and Telegraph * Co. or Unix System Laboratories, Inc. and are reproduced herein with * the permission of UNIX System Laboratories, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)vfs_subr.c 8.31 (Berkeley) 5/26/95 */ /* * External virtual filesystem routines */ #include __FBSDID("$FreeBSD$"); #include "opt_ddb.h" #include "opt_watchdog.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef DDB #include #endif static void delmntque(struct vnode *vp); static int flushbuflist(struct bufv *bufv, int flags, struct bufobj *bo, int slpflag, int slptimeo); static void syncer_shutdown(void *arg, int howto); static int vtryrecycle(struct vnode *vp); static void v_init_counters(struct vnode *); static void v_incr_devcount(struct vnode *); static void v_decr_devcount(struct vnode *); static void vgonel(struct vnode *); static void vfs_knllock(void *arg); static void vfs_knlunlock(void *arg); static void vfs_knl_assert_locked(void *arg); static void vfs_knl_assert_unlocked(void *arg); static void destroy_vpollinfo(struct vpollinfo *vi); static int v_inval_buf_range_locked(struct vnode *vp, struct bufobj *bo, daddr_t startlbn, daddr_t endlbn); static void vnlru_recalc(void); /* * These fences are intended for cases where some synchronization is * needed between access of v_iflags and lockless vnode refcount (v_holdcnt * and v_usecount) updates. Access to v_iflags is generally synchronized * by the interlock, but we have some internal assertions that check vnode * flags without acquiring the lock. Thus, these fences are INVARIANTS-only * for now. */ #ifdef INVARIANTS #define VNODE_REFCOUNT_FENCE_ACQ() atomic_thread_fence_acq() #define VNODE_REFCOUNT_FENCE_REL() atomic_thread_fence_rel() #else #define VNODE_REFCOUNT_FENCE_ACQ() #define VNODE_REFCOUNT_FENCE_REL() #endif /* * Number of vnodes in existence. Increased whenever getnewvnode() * allocates a new vnode, decreased in vdropl() for VIRF_DOOMED vnode. */ static u_long __exclusive_cache_line numvnodes; SYSCTL_ULONG(_vfs, OID_AUTO, numvnodes, CTLFLAG_RD, &numvnodes, 0, "Number of vnodes in existence"); static counter_u64_t vnodes_created; SYSCTL_COUNTER_U64(_vfs, OID_AUTO, vnodes_created, CTLFLAG_RD, &vnodes_created, "Number of vnodes created by getnewvnode"); /* * Conversion tables for conversion from vnode types to inode formats * and back. */ enum vtype iftovt_tab[16] = { VNON, VFIFO, VCHR, VNON, VDIR, VNON, VBLK, VNON, VREG, VNON, VLNK, VNON, VSOCK, VNON, VNON, VNON }; int vttoif_tab[10] = { 0, S_IFREG, S_IFDIR, S_IFBLK, S_IFCHR, S_IFLNK, S_IFSOCK, S_IFIFO, S_IFMT, S_IFMT }; /* * List of allocates vnodes in the system. */ static TAILQ_HEAD(freelst, vnode) vnode_list; static struct vnode *vnode_list_free_marker; static struct vnode *vnode_list_reclaim_marker; /* * "Free" vnode target. Free vnodes are rarely completely free, but are * just ones that are cheap to recycle. Usually they are for files which * have been stat'd but not read; these usually have inode and namecache * data attached to them. This target is the preferred minimum size of a * sub-cache consisting mostly of such files. The system balances the size * of this sub-cache with its complement to try to prevent either from * thrashing while the other is relatively inactive. The targets express * a preference for the best balance. * * "Above" this target there are 2 further targets (watermarks) related * to recyling of free vnodes. In the best-operating case, the cache is * exactly full, the free list has size between vlowat and vhiwat above the * free target, and recycling from it and normal use maintains this state. * Sometimes the free list is below vlowat or even empty, but this state * is even better for immediate use provided the cache is not full. * Otherwise, vnlru_proc() runs to reclaim enough vnodes (usually non-free * ones) to reach one of these states. The watermarks are currently hard- * coded as 4% and 9% of the available space higher. These and the default * of 25% for wantfreevnodes are too large if the memory size is large. * E.g., 9% of 75% of MAXVNODES is more than 566000 vnodes to reclaim * whenever vnlru_proc() becomes active. */ static long wantfreevnodes; static long __exclusive_cache_line freevnodes; SYSCTL_ULONG(_vfs, OID_AUTO, freevnodes, CTLFLAG_RD, &freevnodes, 0, "Number of \"free\" vnodes"); static long freevnodes_old; static counter_u64_t recycles_count; SYSCTL_COUNTER_U64(_vfs, OID_AUTO, recycles, CTLFLAG_RD, &recycles_count, "Number of vnodes recycled to meet vnode cache targets"); static counter_u64_t recycles_free_count; SYSCTL_COUNTER_U64(_vfs, OID_AUTO, recycles_free, CTLFLAG_RD, &recycles_free_count, "Number of free vnodes recycled to meet vnode cache targets"); /* * Various variables used for debugging the new implementation of * reassignbuf(). * XXX these are probably of (very) limited utility now. */ static int reassignbufcalls; SYSCTL_INT(_vfs, OID_AUTO, reassignbufcalls, CTLFLAG_RW | CTLFLAG_STATS, &reassignbufcalls, 0, "Number of calls to reassignbuf"); static counter_u64_t deferred_inact; SYSCTL_COUNTER_U64(_vfs, OID_AUTO, deferred_inact, CTLFLAG_RD, &deferred_inact, "Number of times inactive processing was deferred"); /* To keep more than one thread at a time from running vfs_getnewfsid */ static struct mtx mntid_mtx; /* * Lock for any access to the following: * vnode_list * numvnodes * freevnodes */ static struct mtx __exclusive_cache_line vnode_list_mtx; /* Publicly exported FS */ struct nfs_public nfs_pub; static uma_zone_t buf_trie_zone; /* Zone for allocation of new vnodes - used exclusively by getnewvnode() */ static uma_zone_t vnode_zone; static uma_zone_t vnodepoll_zone; /* * The workitem queue. * * It is useful to delay writes of file data and filesystem metadata * for tens of seconds so that quickly created and deleted files need * not waste disk bandwidth being created and removed. To realize this, * we append vnodes to a "workitem" queue. When running with a soft * updates implementation, most pending metadata dependencies should * not wait for more than a few seconds. Thus, mounted on block devices * are delayed only about a half the time that file data is delayed. * Similarly, directory updates are more critical, so are only delayed * about a third the time that file data is delayed. Thus, there are * SYNCER_MAXDELAY queues that are processed round-robin at a rate of * one each second (driven off the filesystem syncer process). The * syncer_delayno variable indicates the next queue that is to be processed. * Items that need to be processed soon are placed in this queue: * * syncer_workitem_pending[syncer_delayno] * * A delay of fifteen seconds is done by placing the request fifteen * entries later in the queue: * * syncer_workitem_pending[(syncer_delayno + 15) & syncer_mask] * */ static int syncer_delayno; static long syncer_mask; LIST_HEAD(synclist, bufobj); static struct synclist *syncer_workitem_pending; /* * The sync_mtx protects: * bo->bo_synclist * sync_vnode_count * syncer_delayno * syncer_state * syncer_workitem_pending * syncer_worklist_len * rushjob */ static struct mtx sync_mtx; static struct cv sync_wakeup; #define SYNCER_MAXDELAY 32 static int syncer_maxdelay = SYNCER_MAXDELAY; /* maximum delay time */ static int syncdelay = 30; /* max time to delay syncing data */ static int filedelay = 30; /* time to delay syncing files */ SYSCTL_INT(_kern, OID_AUTO, filedelay, CTLFLAG_RW, &filedelay, 0, "Time to delay syncing files (in seconds)"); static int dirdelay = 29; /* time to delay syncing directories */ SYSCTL_INT(_kern, OID_AUTO, dirdelay, CTLFLAG_RW, &dirdelay, 0, "Time to delay syncing directories (in seconds)"); static int metadelay = 28; /* time to delay syncing metadata */ SYSCTL_INT(_kern, OID_AUTO, metadelay, CTLFLAG_RW, &metadelay, 0, "Time to delay syncing metadata (in seconds)"); static int rushjob; /* number of slots to run ASAP */ static int stat_rush_requests; /* number of times I/O speeded up */ SYSCTL_INT(_debug, OID_AUTO, rush_requests, CTLFLAG_RW, &stat_rush_requests, 0, "Number of times I/O speeded up (rush requests)"); #define VDBATCH_SIZE 8 struct vdbatch { u_int index; long freevnodes; struct mtx lock; struct vnode *tab[VDBATCH_SIZE]; }; DPCPU_DEFINE_STATIC(struct vdbatch, vd); static void vdbatch_dequeue(struct vnode *vp); /* * When shutting down the syncer, run it at four times normal speed. */ #define SYNCER_SHUTDOWN_SPEEDUP 4 static int sync_vnode_count; static int syncer_worklist_len; static enum { SYNCER_RUNNING, SYNCER_SHUTTING_DOWN, SYNCER_FINAL_DELAY } syncer_state; /* Target for maximum number of vnodes. */ u_long desiredvnodes; static u_long gapvnodes; /* gap between wanted and desired */ static u_long vhiwat; /* enough extras after expansion */ static u_long vlowat; /* minimal extras before expansion */ static u_long vstir; /* nonzero to stir non-free vnodes */ static volatile int vsmalltrigger = 8; /* pref to keep if > this many pages */ static u_long vnlru_read_freevnodes(void); /* * Note that no attempt is made to sanitize these parameters. */ static int sysctl_maxvnodes(SYSCTL_HANDLER_ARGS) { u_long val; int error; val = desiredvnodes; error = sysctl_handle_long(oidp, &val, 0, req); if (error != 0 || req->newptr == NULL) return (error); if (val == desiredvnodes) return (0); mtx_lock(&vnode_list_mtx); desiredvnodes = val; wantfreevnodes = desiredvnodes / 4; vnlru_recalc(); mtx_unlock(&vnode_list_mtx); /* * XXX There is no protection against multiple threads changing * desiredvnodes at the same time. Locking above only helps vnlru and * getnewvnode. */ vfs_hash_changesize(desiredvnodes); cache_changesize(desiredvnodes); return (0); } SYSCTL_PROC(_kern, KERN_MAXVNODES, maxvnodes, CTLTYPE_ULONG | CTLFLAG_MPSAFE | CTLFLAG_RW, NULL, 0, sysctl_maxvnodes, "LU", "Target for maximum number of vnodes"); static int sysctl_wantfreevnodes(SYSCTL_HANDLER_ARGS) { u_long val; int error; val = wantfreevnodes; error = sysctl_handle_long(oidp, &val, 0, req); if (error != 0 || req->newptr == NULL) return (error); if (val == wantfreevnodes) return (0); mtx_lock(&vnode_list_mtx); wantfreevnodes = val; vnlru_recalc(); mtx_unlock(&vnode_list_mtx); return (0); } SYSCTL_PROC(_vfs, OID_AUTO, wantfreevnodes, CTLTYPE_ULONG | CTLFLAG_MPSAFE | CTLFLAG_RW, NULL, 0, sysctl_wantfreevnodes, "LU", "Target for minimum number of \"free\" vnodes"); SYSCTL_ULONG(_kern, OID_AUTO, minvnodes, CTLFLAG_RW, &wantfreevnodes, 0, "Old name for vfs.wantfreevnodes (legacy)"); static int vnlru_nowhere; SYSCTL_INT(_debug, OID_AUTO, vnlru_nowhere, CTLFLAG_RW, &vnlru_nowhere, 0, "Number of times the vnlru process ran without success"); static int sysctl_try_reclaim_vnode(SYSCTL_HANDLER_ARGS) { struct vnode *vp; struct nameidata nd; char *buf; unsigned long ndflags; int error; if (req->newptr == NULL) return (EINVAL); if (req->newlen >= PATH_MAX) return (E2BIG); buf = malloc(PATH_MAX, M_TEMP, M_WAITOK); error = SYSCTL_IN(req, buf, req->newlen); if (error != 0) goto out; buf[req->newlen] = '\0'; ndflags = LOCKLEAF | NOFOLLOW | AUDITVNODE1 | NOCACHE | SAVENAME; NDINIT(&nd, LOOKUP, ndflags, UIO_SYSSPACE, buf, curthread); if ((error = namei(&nd)) != 0) goto out; vp = nd.ni_vp; if (VN_IS_DOOMED(vp)) { /* * This vnode is being recycled. Return != 0 to let the caller * know that the sysctl had no effect. Return EAGAIN because a * subsequent call will likely succeed (since namei will create * a new vnode if necessary) */ error = EAGAIN; goto putvnode; } counter_u64_add(recycles_count, 1); vgone(vp); putvnode: NDFREE(&nd, 0); out: free(buf, M_TEMP); return (error); } static int sysctl_ftry_reclaim_vnode(SYSCTL_HANDLER_ARGS) { struct thread *td = curthread; struct vnode *vp; struct file *fp; int error; int fd; if (req->newptr == NULL) return (EBADF); error = sysctl_handle_int(oidp, &fd, 0, req); if (error != 0) return (error); error = getvnode(curthread, fd, &cap_fcntl_rights, &fp); if (error != 0) return (error); vp = fp->f_vnode; error = vn_lock(vp, LK_EXCLUSIVE); if (error != 0) goto drop; counter_u64_add(recycles_count, 1); vgone(vp); VOP_UNLOCK(vp); drop: fdrop(fp, td); return (error); } SYSCTL_PROC(_debug, OID_AUTO, try_reclaim_vnode, CTLTYPE_STRING | CTLFLAG_MPSAFE | CTLFLAG_WR, NULL, 0, sysctl_try_reclaim_vnode, "A", "Try to reclaim a vnode by its pathname"); SYSCTL_PROC(_debug, OID_AUTO, ftry_reclaim_vnode, CTLTYPE_INT | CTLFLAG_MPSAFE | CTLFLAG_WR, NULL, 0, sysctl_ftry_reclaim_vnode, "I", "Try to reclaim a vnode by its file descriptor"); /* Shift count for (uintptr_t)vp to initialize vp->v_hash. */ static int vnsz2log; /* * Support for the bufobj clean & dirty pctrie. */ static void * buf_trie_alloc(struct pctrie *ptree) { return uma_zalloc(buf_trie_zone, M_NOWAIT); } static void buf_trie_free(struct pctrie *ptree, void *node) { uma_zfree(buf_trie_zone, node); } PCTRIE_DEFINE(BUF, buf, b_lblkno, buf_trie_alloc, buf_trie_free); /* * Initialize the vnode management data structures. * * Reevaluate the following cap on the number of vnodes after the physical * memory size exceeds 512GB. In the limit, as the physical memory size * grows, the ratio of the memory size in KB to vnodes approaches 64:1. */ #ifndef MAXVNODES_MAX #define MAXVNODES_MAX (512UL * 1024 * 1024 / 64) /* 8M */ #endif static MALLOC_DEFINE(M_VNODE_MARKER, "vnodemarker", "vnode marker"); static struct vnode * vn_alloc_marker(struct mount *mp) { struct vnode *vp; vp = malloc(sizeof(struct vnode), M_VNODE_MARKER, M_WAITOK | M_ZERO); vp->v_type = VMARKER; vp->v_mount = mp; return (vp); } static void vn_free_marker(struct vnode *vp) { MPASS(vp->v_type == VMARKER); free(vp, M_VNODE_MARKER); } /* * Initialize a vnode as it first enters the zone. */ static int vnode_init(void *mem, int size, int flags) { struct vnode *vp; vp = mem; bzero(vp, size); /* * Setup locks. */ vp->v_vnlock = &vp->v_lock; mtx_init(&vp->v_interlock, "vnode interlock", NULL, MTX_DEF); /* * By default, don't allow shared locks unless filesystems opt-in. */ lockinit(vp->v_vnlock, PVFS, "vnode", VLKTIMEOUT, LK_NOSHARE | LK_IS_VNODE); /* * Initialize bufobj. */ bufobj_init(&vp->v_bufobj, vp); /* * Initialize namecache. */ LIST_INIT(&vp->v_cache_src); TAILQ_INIT(&vp->v_cache_dst); /* * Initialize rangelocks. */ rangelock_init(&vp->v_rl); vp->v_dbatchcpu = NOCPU; mtx_lock(&vnode_list_mtx); TAILQ_INSERT_BEFORE(vnode_list_free_marker, vp, v_vnodelist); mtx_unlock(&vnode_list_mtx); return (0); } /* * Free a vnode when it is cleared from the zone. */ static void vnode_fini(void *mem, int size) { struct vnode *vp; struct bufobj *bo; vp = mem; vdbatch_dequeue(vp); mtx_lock(&vnode_list_mtx); TAILQ_REMOVE(&vnode_list, vp, v_vnodelist); mtx_unlock(&vnode_list_mtx); rangelock_destroy(&vp->v_rl); lockdestroy(vp->v_vnlock); mtx_destroy(&vp->v_interlock); bo = &vp->v_bufobj; rw_destroy(BO_LOCKPTR(bo)); } /* * Provide the size of NFS nclnode and NFS fh for calculation of the * vnode memory consumption. The size is specified directly to * eliminate dependency on NFS-private header. * * Other filesystems may use bigger or smaller (like UFS and ZFS) * private inode data, but the NFS-based estimation is ample enough. * Still, we care about differences in the size between 64- and 32-bit * platforms. * * Namecache structure size is heuristically * sizeof(struct namecache_ts) + CACHE_PATH_CUTOFF + 1. */ #ifdef _LP64 #define NFS_NCLNODE_SZ (528 + 64) #define NC_SZ 148 #else #define NFS_NCLNODE_SZ (360 + 32) #define NC_SZ 92 #endif static void vntblinit(void *dummy __unused) { struct vdbatch *vd; int cpu, physvnodes, virtvnodes; u_int i; /* * Desiredvnodes is a function of the physical memory size and the * kernel's heap size. Generally speaking, it scales with the * physical memory size. The ratio of desiredvnodes to the physical * memory size is 1:16 until desiredvnodes exceeds 98,304. * Thereafter, the * marginal ratio of desiredvnodes to the physical memory size is * 1:64. However, desiredvnodes is limited by the kernel's heap * size. The memory required by desiredvnodes vnodes and vm objects * must not exceed 1/10th of the kernel's heap size. */ physvnodes = maxproc + pgtok(vm_cnt.v_page_count) / 64 + 3 * min(98304 * 16, pgtok(vm_cnt.v_page_count)) / 64; virtvnodes = vm_kmem_size / (10 * (sizeof(struct vm_object) + sizeof(struct vnode) + NC_SZ * ncsizefactor + NFS_NCLNODE_SZ)); desiredvnodes = min(physvnodes, virtvnodes); if (desiredvnodes > MAXVNODES_MAX) { if (bootverbose) printf("Reducing kern.maxvnodes %lu -> %lu\n", desiredvnodes, MAXVNODES_MAX); desiredvnodes = MAXVNODES_MAX; } wantfreevnodes = desiredvnodes / 4; mtx_init(&mntid_mtx, "mntid", NULL, MTX_DEF); TAILQ_INIT(&vnode_list); mtx_init(&vnode_list_mtx, "vnode_list", NULL, MTX_DEF); /* * The lock is taken to appease WITNESS. */ mtx_lock(&vnode_list_mtx); vnlru_recalc(); mtx_unlock(&vnode_list_mtx); vnode_list_free_marker = vn_alloc_marker(NULL); TAILQ_INSERT_HEAD(&vnode_list, vnode_list_free_marker, v_vnodelist); vnode_list_reclaim_marker = vn_alloc_marker(NULL); TAILQ_INSERT_HEAD(&vnode_list, vnode_list_reclaim_marker, v_vnodelist); vnode_zone = uma_zcreate("VNODE", sizeof (struct vnode), NULL, NULL, vnode_init, vnode_fini, UMA_ALIGN_PTR, 0); vnodepoll_zone = uma_zcreate("VNODEPOLL", sizeof (struct vpollinfo), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); /* * Preallocate enough nodes to support one-per buf so that * we can not fail an insert. reassignbuf() callers can not * tolerate the insertion failure. */ buf_trie_zone = uma_zcreate("BUF TRIE", pctrie_node_size(), NULL, NULL, pctrie_zone_init, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE | UMA_ZONE_VM); uma_prealloc(buf_trie_zone, nbuf); vnodes_created = counter_u64_alloc(M_WAITOK); recycles_count = counter_u64_alloc(M_WAITOK); recycles_free_count = counter_u64_alloc(M_WAITOK); deferred_inact = counter_u64_alloc(M_WAITOK); /* * Initialize the filesystem syncer. */ syncer_workitem_pending = hashinit(syncer_maxdelay, M_VNODE, &syncer_mask); syncer_maxdelay = syncer_mask + 1; mtx_init(&sync_mtx, "Syncer mtx", NULL, MTX_DEF); cv_init(&sync_wakeup, "syncer"); for (i = 1; i <= sizeof(struct vnode); i <<= 1) vnsz2log++; vnsz2log--; CPU_FOREACH(cpu) { vd = DPCPU_ID_PTR((cpu), vd); bzero(vd, sizeof(*vd)); mtx_init(&vd->lock, "vdbatch", NULL, MTX_DEF); } } SYSINIT(vfs, SI_SUB_VFS, SI_ORDER_FIRST, vntblinit, NULL); /* * Mark a mount point as busy. Used to synchronize access and to delay * unmounting. Eventually, mountlist_mtx is not released on failure. * * vfs_busy() is a custom lock, it can block the caller. * vfs_busy() only sleeps if the unmount is active on the mount point. * For a mountpoint mp, vfs_busy-enforced lock is before lock of any * vnode belonging to mp. * * Lookup uses vfs_busy() to traverse mount points. * root fs var fs * / vnode lock A / vnode lock (/var) D * /var vnode lock B /log vnode lock(/var/log) E * vfs_busy lock C vfs_busy lock F * * Within each file system, the lock order is C->A->B and F->D->E. * * When traversing across mounts, the system follows that lock order: * * C->A->B * | * +->F->D->E * * The lookup() process for namei("/var") illustrates the process: * VOP_LOOKUP() obtains B while A is held * vfs_busy() obtains a shared lock on F while A and B are held * vput() releases lock on B * vput() releases lock on A * VFS_ROOT() obtains lock on D while shared lock on F is held * vfs_unbusy() releases shared lock on F * vn_lock() obtains lock on deadfs vnode vp_crossmp instead of A. * Attempt to lock A (instead of vp_crossmp) while D is held would * violate the global order, causing deadlocks. * * dounmount() locks B while F is drained. */ int vfs_busy(struct mount *mp, int flags) { MPASS((flags & ~MBF_MASK) == 0); CTR3(KTR_VFS, "%s: mp %p with flags %d", __func__, mp, flags); if (vfs_op_thread_enter(mp)) { MPASS((mp->mnt_kern_flag & MNTK_DRAINING) == 0); MPASS((mp->mnt_kern_flag & MNTK_UNMOUNT) == 0); MPASS((mp->mnt_kern_flag & MNTK_REFEXPIRE) == 0); vfs_mp_count_add_pcpu(mp, ref, 1); vfs_mp_count_add_pcpu(mp, lockref, 1); vfs_op_thread_exit(mp); if (flags & MBF_MNTLSTLOCK) mtx_unlock(&mountlist_mtx); return (0); } MNT_ILOCK(mp); vfs_assert_mount_counters(mp); MNT_REF(mp); /* * If mount point is currently being unmounted, sleep until the * mount point fate is decided. If thread doing the unmounting fails, * it will clear MNTK_UNMOUNT flag before waking us up, indicating * that this mount point has survived the unmount attempt and vfs_busy * should retry. Otherwise the unmounter thread will set MNTK_REFEXPIRE * flag in addition to MNTK_UNMOUNT, indicating that mount point is * about to be really destroyed. vfs_busy needs to release its * reference on the mount point in this case and return with ENOENT, * telling the caller that mount mount it tried to busy is no longer * valid. */ while (mp->mnt_kern_flag & MNTK_UNMOUNT) { if (flags & MBF_NOWAIT || mp->mnt_kern_flag & MNTK_REFEXPIRE) { MNT_REL(mp); MNT_IUNLOCK(mp); CTR1(KTR_VFS, "%s: failed busying before sleeping", __func__); return (ENOENT); } if (flags & MBF_MNTLSTLOCK) mtx_unlock(&mountlist_mtx); mp->mnt_kern_flag |= MNTK_MWAIT; msleep(mp, MNT_MTX(mp), PVFS | PDROP, "vfs_busy", 0); if (flags & MBF_MNTLSTLOCK) mtx_lock(&mountlist_mtx); MNT_ILOCK(mp); } if (flags & MBF_MNTLSTLOCK) mtx_unlock(&mountlist_mtx); mp->mnt_lockref++; MNT_IUNLOCK(mp); return (0); } /* * Free a busy filesystem. */ void vfs_unbusy(struct mount *mp) { int c; CTR2(KTR_VFS, "%s: mp %p", __func__, mp); if (vfs_op_thread_enter(mp)) { MPASS((mp->mnt_kern_flag & MNTK_DRAINING) == 0); vfs_mp_count_sub_pcpu(mp, lockref, 1); vfs_mp_count_sub_pcpu(mp, ref, 1); vfs_op_thread_exit(mp); return; } MNT_ILOCK(mp); vfs_assert_mount_counters(mp); MNT_REL(mp); c = --mp->mnt_lockref; if (mp->mnt_vfs_ops == 0) { MPASS((mp->mnt_kern_flag & MNTK_DRAINING) == 0); MNT_IUNLOCK(mp); return; } if (c < 0) vfs_dump_mount_counters(mp); if (c == 0 && (mp->mnt_kern_flag & MNTK_DRAINING) != 0) { MPASS(mp->mnt_kern_flag & MNTK_UNMOUNT); CTR1(KTR_VFS, "%s: waking up waiters", __func__); mp->mnt_kern_flag &= ~MNTK_DRAINING; wakeup(&mp->mnt_lockref); } MNT_IUNLOCK(mp); } /* * Lookup a mount point by filesystem identifier. */ struct mount * vfs_getvfs(fsid_t *fsid) { struct mount *mp; CTR2(KTR_VFS, "%s: fsid %p", __func__, fsid); mtx_lock(&mountlist_mtx); TAILQ_FOREACH(mp, &mountlist, mnt_list) { if (mp->mnt_stat.f_fsid.val[0] == fsid->val[0] && mp->mnt_stat.f_fsid.val[1] == fsid->val[1]) { vfs_ref(mp); mtx_unlock(&mountlist_mtx); return (mp); } } mtx_unlock(&mountlist_mtx); CTR2(KTR_VFS, "%s: lookup failed for %p id", __func__, fsid); return ((struct mount *) 0); } /* * Lookup a mount point by filesystem identifier, busying it before * returning. * * To avoid congestion on mountlist_mtx, implement simple direct-mapped * cache for popular filesystem identifiers. The cache is lockess, using * the fact that struct mount's are never freed. In worst case we may * get pointer to unmounted or even different filesystem, so we have to * check what we got, and go slow way if so. */ struct mount * vfs_busyfs(fsid_t *fsid) { #define FSID_CACHE_SIZE 256 typedef struct mount * volatile vmp_t; static vmp_t cache[FSID_CACHE_SIZE]; struct mount *mp; int error; uint32_t hash; CTR2(KTR_VFS, "%s: fsid %p", __func__, fsid); hash = fsid->val[0] ^ fsid->val[1]; hash = (hash >> 16 ^ hash) & (FSID_CACHE_SIZE - 1); mp = cache[hash]; if (mp == NULL || mp->mnt_stat.f_fsid.val[0] != fsid->val[0] || mp->mnt_stat.f_fsid.val[1] != fsid->val[1]) goto slow; if (vfs_busy(mp, 0) != 0) { cache[hash] = NULL; goto slow; } if (mp->mnt_stat.f_fsid.val[0] == fsid->val[0] && mp->mnt_stat.f_fsid.val[1] == fsid->val[1]) return (mp); else vfs_unbusy(mp); slow: mtx_lock(&mountlist_mtx); TAILQ_FOREACH(mp, &mountlist, mnt_list) { if (mp->mnt_stat.f_fsid.val[0] == fsid->val[0] && mp->mnt_stat.f_fsid.val[1] == fsid->val[1]) { error = vfs_busy(mp, MBF_MNTLSTLOCK); if (error) { cache[hash] = NULL; mtx_unlock(&mountlist_mtx); return (NULL); } cache[hash] = mp; return (mp); } } CTR2(KTR_VFS, "%s: lookup failed for %p id", __func__, fsid); mtx_unlock(&mountlist_mtx); return ((struct mount *) 0); } /* * Check if a user can access privileged mount options. */ int vfs_suser(struct mount *mp, struct thread *td) { int error; if (jailed(td->td_ucred)) { /* * If the jail of the calling thread lacks permission for * this type of file system, deny immediately. */ if (!prison_allow(td->td_ucred, mp->mnt_vfc->vfc_prison_flag)) return (EPERM); /* * If the file system was mounted outside the jail of the * calling thread, deny immediately. */ if (prison_check(td->td_ucred, mp->mnt_cred) != 0) return (EPERM); } /* * If file system supports delegated administration, we don't check * for the PRIV_VFS_MOUNT_OWNER privilege - it will be better verified * by the file system itself. * If this is not the user that did original mount, we check for * the PRIV_VFS_MOUNT_OWNER privilege. */ if (!(mp->mnt_vfc->vfc_flags & VFCF_DELEGADMIN) && mp->mnt_cred->cr_uid != td->td_ucred->cr_uid) { if ((error = priv_check(td, PRIV_VFS_MOUNT_OWNER)) != 0) return (error); } return (0); } /* * Get a new unique fsid. Try to make its val[0] unique, since this value * will be used to create fake device numbers for stat(). Also try (but * not so hard) make its val[0] unique mod 2^16, since some emulators only * support 16-bit device numbers. We end up with unique val[0]'s for the * first 2^16 calls and unique val[0]'s mod 2^16 for the first 2^8 calls. * * Keep in mind that several mounts may be running in parallel. Starting * the search one past where the previous search terminated is both a * micro-optimization and a defense against returning the same fsid to * different mounts. */ void vfs_getnewfsid(struct mount *mp) { static uint16_t mntid_base; struct mount *nmp; fsid_t tfsid; int mtype; CTR2(KTR_VFS, "%s: mp %p", __func__, mp); mtx_lock(&mntid_mtx); mtype = mp->mnt_vfc->vfc_typenum; tfsid.val[1] = mtype; mtype = (mtype & 0xFF) << 24; for (;;) { tfsid.val[0] = makedev(255, mtype | ((mntid_base & 0xFF00) << 8) | (mntid_base & 0xFF)); mntid_base++; if ((nmp = vfs_getvfs(&tfsid)) == NULL) break; vfs_rel(nmp); } mp->mnt_stat.f_fsid.val[0] = tfsid.val[0]; mp->mnt_stat.f_fsid.val[1] = tfsid.val[1]; mtx_unlock(&mntid_mtx); } /* * Knob to control the precision of file timestamps: * * 0 = seconds only; nanoseconds zeroed. * 1 = seconds and nanoseconds, accurate within 1/HZ. * 2 = seconds and nanoseconds, truncated to microseconds. * >=3 = seconds and nanoseconds, maximum precision. */ enum { TSP_SEC, TSP_HZ, TSP_USEC, TSP_NSEC }; static int timestamp_precision = TSP_USEC; SYSCTL_INT(_vfs, OID_AUTO, timestamp_precision, CTLFLAG_RW, ×tamp_precision, 0, "File timestamp precision (0: seconds, " "1: sec + ns accurate to 1/HZ, 2: sec + ns truncated to us, " "3+: sec + ns (max. precision))"); /* * Get a current timestamp. */ void vfs_timestamp(struct timespec *tsp) { struct timeval tv; switch (timestamp_precision) { case TSP_SEC: tsp->tv_sec = time_second; tsp->tv_nsec = 0; break; case TSP_HZ: getnanotime(tsp); break; case TSP_USEC: microtime(&tv); TIMEVAL_TO_TIMESPEC(&tv, tsp); break; case TSP_NSEC: default: nanotime(tsp); break; } } /* * Set vnode attributes to VNOVAL */ void vattr_null(struct vattr *vap) { vap->va_type = VNON; vap->va_size = VNOVAL; vap->va_bytes = VNOVAL; vap->va_mode = VNOVAL; vap->va_nlink = VNOVAL; vap->va_uid = VNOVAL; vap->va_gid = VNOVAL; vap->va_fsid = VNOVAL; vap->va_fileid = VNOVAL; vap->va_blocksize = VNOVAL; vap->va_rdev = VNOVAL; vap->va_atime.tv_sec = VNOVAL; vap->va_atime.tv_nsec = VNOVAL; vap->va_mtime.tv_sec = VNOVAL; vap->va_mtime.tv_nsec = VNOVAL; vap->va_ctime.tv_sec = VNOVAL; vap->va_ctime.tv_nsec = VNOVAL; vap->va_birthtime.tv_sec = VNOVAL; vap->va_birthtime.tv_nsec = VNOVAL; vap->va_flags = VNOVAL; vap->va_gen = VNOVAL; vap->va_vaflags = 0; } /* * Try to reduce the total number of vnodes. * * This routine (and its user) are buggy in at least the following ways: * - all parameters were picked years ago when RAM sizes were significantly * smaller * - it can pick vnodes based on pages used by the vm object, but filesystems * like ZFS don't use it making the pick broken * - since ZFS has its own aging policy it gets partially combated by this one * - a dedicated method should be provided for filesystems to let them decide * whether the vnode should be recycled * * This routine is called when we have too many vnodes. It attempts * to free vnodes and will potentially free vnodes that still * have VM backing store (VM backing store is typically the cause * of a vnode blowout so we want to do this). Therefore, this operation * is not considered cheap. * * A number of conditions may prevent a vnode from being reclaimed. * the buffer cache may have references on the vnode, a directory * vnode may still have references due to the namei cache representing * underlying files, or the vnode may be in active use. It is not * desirable to reuse such vnodes. These conditions may cause the * number of vnodes to reach some minimum value regardless of what * you set kern.maxvnodes to. Do not set kern.maxvnodes too low. * * @param reclaim_nc_src Only reclaim directories with outgoing namecache * entries if this argument is strue * @param trigger Only reclaim vnodes with fewer than this many resident * pages. * @param target How many vnodes to reclaim. * @return The number of vnodes that were reclaimed. */ static int vlrureclaim(bool reclaim_nc_src, int trigger, u_long target) { struct vnode *vp, *mvp; struct mount *mp; u_long done; bool retried; mtx_assert(&vnode_list_mtx, MA_OWNED); retried = false; done = 0; mvp = vnode_list_reclaim_marker; restart: vp = mvp; while (done < target) { vp = TAILQ_NEXT(vp, v_vnodelist); if (__predict_false(vp == NULL)) break; if (__predict_false(vp->v_type == VMARKER)) continue; /* * If it's been deconstructed already, it's still * referenced, or it exceeds the trigger, skip it. * Also skip free vnodes. We are trying to make space * to expand the free list, not reduce it. */ if (vp->v_usecount > 0 || vp->v_holdcnt == 0 || (!reclaim_nc_src && !LIST_EMPTY(&vp->v_cache_src))) goto next_iter; if (vp->v_type == VBAD || vp->v_type == VNON) goto next_iter; if (!VI_TRYLOCK(vp)) goto next_iter; if (vp->v_usecount > 0 || vp->v_holdcnt == 0 || (!reclaim_nc_src && !LIST_EMPTY(&vp->v_cache_src)) || vp->v_type == VBAD || vp->v_type == VNON || (vp->v_object != NULL && vp->v_object->resident_page_count > trigger)) { VI_UNLOCK(vp); goto next_iter; } vholdl(vp); VI_UNLOCK(vp); TAILQ_REMOVE(&vnode_list, mvp, v_vnodelist); TAILQ_INSERT_AFTER(&vnode_list, vp, mvp, v_vnodelist); mtx_unlock(&vnode_list_mtx); if (vn_start_write(vp, &mp, V_NOWAIT) != 0) { vdrop(vp); goto next_iter_unlocked; } if (VOP_LOCK(vp, LK_EXCLUSIVE|LK_NOWAIT) != 0) { vdrop(vp); vn_finished_write(mp); goto next_iter_unlocked; } VI_LOCK(vp); if (vp->v_usecount > 0 || (!reclaim_nc_src && !LIST_EMPTY(&vp->v_cache_src)) || (vp->v_object != NULL && vp->v_object->resident_page_count > trigger)) { VOP_UNLOCK(vp); vdropl(vp); vn_finished_write(mp); goto next_iter_unlocked; } counter_u64_add(recycles_count, 1); vgonel(vp); VOP_UNLOCK(vp); vdropl(vp); vn_finished_write(mp); done++; next_iter_unlocked: if (should_yield()) kern_yield(PRI_USER); mtx_lock(&vnode_list_mtx); goto restart; next_iter: MPASS(vp->v_type != VMARKER); if (!should_yield()) continue; TAILQ_REMOVE(&vnode_list, mvp, v_vnodelist); TAILQ_INSERT_AFTER(&vnode_list, vp, mvp, v_vnodelist); mtx_unlock(&vnode_list_mtx); kern_yield(PRI_USER); mtx_lock(&vnode_list_mtx); goto restart; } if (done == 0 && !retried) { TAILQ_REMOVE(&vnode_list, mvp, v_vnodelist); TAILQ_INSERT_HEAD(&vnode_list, mvp, v_vnodelist); retried = true; goto restart; } return (done); } static int max_vnlru_free = 10000; /* limit on vnode free requests per call */ SYSCTL_INT(_debug, OID_AUTO, max_vnlru_free, CTLFLAG_RW, &max_vnlru_free, 0, "limit on vnode free requests per call to the vnlru_free routine"); /* * Attempt to reduce the free list by the requested amount. */ static int vnlru_free_locked(int count, struct vfsops *mnt_op) { struct vnode *vp, *mvp; struct mount *mp; int ocount; mtx_assert(&vnode_list_mtx, MA_OWNED); if (count > max_vnlru_free) count = max_vnlru_free; ocount = count; mvp = vnode_list_free_marker; restart: vp = mvp; while (count > 0) { vp = TAILQ_NEXT(vp, v_vnodelist); if (__predict_false(vp == NULL)) { TAILQ_REMOVE(&vnode_list, mvp, v_vnodelist); TAILQ_INSERT_TAIL(&vnode_list, mvp, v_vnodelist); break; } if (__predict_false(vp->v_type == VMARKER)) continue; /* * Don't recycle if our vnode is from different type * of mount point. Note that mp is type-safe, the * check does not reach unmapped address even if * vnode is reclaimed. * Don't recycle if we can't get the interlock without * blocking. */ if (vp->v_holdcnt > 0 || (mnt_op != NULL && (mp = vp->v_mount) != NULL && mp->mnt_op != mnt_op) || !VI_TRYLOCK(vp)) { continue; } TAILQ_REMOVE(&vnode_list, mvp, v_vnodelist); TAILQ_INSERT_AFTER(&vnode_list, vp, mvp, v_vnodelist); if (__predict_false(vp->v_type == VBAD || vp->v_type == VNON)) { VI_UNLOCK(vp); continue; } vholdl(vp); count--; mtx_unlock(&vnode_list_mtx); VI_UNLOCK(vp); vtryrecycle(vp); vdrop(vp); mtx_lock(&vnode_list_mtx); goto restart; } return (ocount - count); } void vnlru_free(int count, struct vfsops *mnt_op) { mtx_lock(&vnode_list_mtx); vnlru_free_locked(count, mnt_op); mtx_unlock(&vnode_list_mtx); } static void vnlru_recalc(void) { mtx_assert(&vnode_list_mtx, MA_OWNED); gapvnodes = imax(desiredvnodes - wantfreevnodes, 100); vhiwat = gapvnodes / 11; /* 9% -- just under the 10% in vlrureclaim() */ vlowat = vhiwat / 2; } /* * Attempt to recycle vnodes in a context that is always safe to block. * Calling vlrurecycle() from the bowels of filesystem code has some * interesting deadlock problems. */ static struct proc *vnlruproc; static int vnlruproc_sig; /* * The main freevnodes counter is only updated when threads requeue their vnode * batches. CPUs are conditionally walked to compute a more accurate total. * * Limit how much of a slop are we willing to tolerate. Note: the actual value * at any given moment can still exceed slop, but it should not be by significant * margin in practice. */ #define VNLRU_FREEVNODES_SLOP 128 static u_long vnlru_read_freevnodes(void) { struct vdbatch *vd; long slop; int cpu; mtx_assert(&vnode_list_mtx, MA_OWNED); if (freevnodes > freevnodes_old) slop = freevnodes - freevnodes_old; else slop = freevnodes_old - freevnodes; if (slop < VNLRU_FREEVNODES_SLOP) return (freevnodes >= 0 ? freevnodes : 0); freevnodes_old = freevnodes; CPU_FOREACH(cpu) { vd = DPCPU_ID_PTR((cpu), vd); freevnodes_old += vd->freevnodes; } return (freevnodes_old >= 0 ? freevnodes_old : 0); } static bool vnlru_under(u_long rnumvnodes, u_long limit) { u_long rfreevnodes, space; if (__predict_false(rnumvnodes > desiredvnodes)) return (true); space = desiredvnodes - rnumvnodes; if (space < limit) { rfreevnodes = vnlru_read_freevnodes(); if (rfreevnodes > wantfreevnodes) space += rfreevnodes - wantfreevnodes; } return (space < limit); } static bool vnlru_under_unlocked(u_long rnumvnodes, u_long limit) { long rfreevnodes, space; if (__predict_false(rnumvnodes > desiredvnodes)) return (true); space = desiredvnodes - rnumvnodes; if (space < limit) { rfreevnodes = atomic_load_long(&freevnodes); if (rfreevnodes > wantfreevnodes) space += rfreevnodes - wantfreevnodes; } return (space < limit); } static void vnlru_kick(void) { mtx_assert(&vnode_list_mtx, MA_OWNED); if (vnlruproc_sig == 0) { vnlruproc_sig = 1; wakeup(vnlruproc); } } static void vnlru_proc(void) { u_long rnumvnodes, rfreevnodes, target; unsigned long onumvnodes; int done, force, trigger, usevnodes; bool reclaim_nc_src, want_reread; EVENTHANDLER_REGISTER(shutdown_pre_sync, kproc_shutdown, vnlruproc, SHUTDOWN_PRI_FIRST); force = 0; want_reread = false; for (;;) { kproc_suspend_check(vnlruproc); mtx_lock(&vnode_list_mtx); rnumvnodes = atomic_load_long(&numvnodes); if (want_reread) { force = vnlru_under(numvnodes, vhiwat) ? 1 : 0; want_reread = false; } /* * If numvnodes is too large (due to desiredvnodes being * adjusted using its sysctl, or emergency growth), first * try to reduce it by discarding from the free list. */ if (rnumvnodes > desiredvnodes) { vnlru_free_locked(rnumvnodes - desiredvnodes, NULL); rnumvnodes = atomic_load_long(&numvnodes); } /* * Sleep if the vnode cache is in a good state. This is * when it is not over-full and has space for about a 4% * or 9% expansion (by growing its size or inexcessively * reducing its free list). Otherwise, try to reclaim * space for a 10% expansion. */ if (vstir && force == 0) { force = 1; vstir = 0; } if (force == 0 && !vnlru_under(rnumvnodes, vlowat)) { vnlruproc_sig = 0; wakeup(&vnlruproc_sig); msleep(vnlruproc, &vnode_list_mtx, PVFS|PDROP, "vlruwt", hz); continue; } rfreevnodes = vnlru_read_freevnodes(); onumvnodes = rnumvnodes; /* * Calculate parameters for recycling. These are the same * throughout the loop to give some semblance of fairness. * The trigger point is to avoid recycling vnodes with lots * of resident pages. We aren't trying to free memory; we * are trying to recycle or at least free vnodes. */ if (rnumvnodes <= desiredvnodes) usevnodes = rnumvnodes - rfreevnodes; else usevnodes = rnumvnodes; if (usevnodes <= 0) usevnodes = 1; /* * The trigger value is is chosen to give a conservatively * large value to ensure that it alone doesn't prevent * making progress. The value can easily be so large that * it is effectively infinite in some congested and * misconfigured cases, and this is necessary. Normally * it is about 8 to 100 (pages), which is quite large. */ trigger = vm_cnt.v_page_count * 2 / usevnodes; if (force < 2) trigger = vsmalltrigger; reclaim_nc_src = force >= 3; target = rnumvnodes * (int64_t)gapvnodes / imax(desiredvnodes, 1); target = target / 10 + 1; done = vlrureclaim(reclaim_nc_src, trigger, target); mtx_unlock(&vnode_list_mtx); if (onumvnodes > desiredvnodes && numvnodes <= desiredvnodes) uma_reclaim(UMA_RECLAIM_DRAIN); if (done == 0) { if (force == 0 || force == 1) { force = 2; continue; } if (force == 2) { force = 3; continue; } want_reread = true; force = 0; vnlru_nowhere++; tsleep(vnlruproc, PPAUSE, "vlrup", hz * 3); } else { want_reread = true; kern_yield(PRI_USER); } } } static struct kproc_desc vnlru_kp = { "vnlru", vnlru_proc, &vnlruproc }; SYSINIT(vnlru, SI_SUB_KTHREAD_UPDATE, SI_ORDER_FIRST, kproc_start, &vnlru_kp); /* * Routines having to do with the management of the vnode table. */ /* * Try to recycle a freed vnode. We abort if anyone picks up a reference * before we actually vgone(). This function must be called with the vnode * held to prevent the vnode from being returned to the free list midway * through vgone(). */ static int vtryrecycle(struct vnode *vp) { struct mount *vnmp; CTR2(KTR_VFS, "%s: vp %p", __func__, vp); VNASSERT(vp->v_holdcnt, vp, ("vtryrecycle: Recycling vp %p without a reference.", vp)); /* * This vnode may found and locked via some other list, if so we * can't recycle it yet. */ if (VOP_LOCK(vp, LK_EXCLUSIVE | LK_NOWAIT) != 0) { CTR2(KTR_VFS, "%s: impossible to recycle, vp %p lock is already held", __func__, vp); return (EWOULDBLOCK); } /* * Don't recycle if its filesystem is being suspended. */ if (vn_start_write(vp, &vnmp, V_NOWAIT) != 0) { VOP_UNLOCK(vp); CTR2(KTR_VFS, "%s: impossible to recycle, cannot start the write for %p", __func__, vp); return (EBUSY); } /* * If we got this far, we need to acquire the interlock and see if * anyone picked up this vnode from another list. If not, we will * mark it with DOOMED via vgonel() so that anyone who does find it * will skip over it. */ VI_LOCK(vp); if (vp->v_usecount) { VOP_UNLOCK(vp); VI_UNLOCK(vp); vn_finished_write(vnmp); CTR2(KTR_VFS, "%s: impossible to recycle, %p is already referenced", __func__, vp); return (EBUSY); } if (!VN_IS_DOOMED(vp)) { counter_u64_add(recycles_free_count, 1); vgonel(vp); } VOP_UNLOCK(vp); VI_UNLOCK(vp); vn_finished_write(vnmp); return (0); } /* * Allocate a new vnode. * * The operation never returns an error. Returning an error was disabled * in r145385 (dated 2005) with the following comment: * * XXX Not all VFS_VGET/ffs_vget callers check returns. * * Given the age of this commit (almost 15 years at the time of writing this * comment) restoring the ability to fail requires a significant audit of * all codepaths. * * The routine can try to free a vnode or stall for up to 1 second waiting for * vnlru to clear things up, but ultimately always performs a M_WAITOK allocation. */ static u_long vn_alloc_cyclecount; static struct vnode * __noinline vn_alloc_hard(struct mount *mp) { u_long rnumvnodes, rfreevnodes; mtx_lock(&vnode_list_mtx); rnumvnodes = atomic_load_long(&numvnodes); if (rnumvnodes + 1 < desiredvnodes) { vn_alloc_cyclecount = 0; goto alloc; } rfreevnodes = vnlru_read_freevnodes(); if (vn_alloc_cyclecount++ >= rfreevnodes) { vn_alloc_cyclecount = 0; vstir = 1; } /* * Grow the vnode cache if it will not be above its target max * after growing. Otherwise, if the free list is nonempty, try * to reclaim 1 item from it before growing the cache (possibly * above its target max if the reclamation failed or is delayed). * Otherwise, wait for some space. In all cases, schedule * vnlru_proc() if we are getting short of space. The watermarks * should be chosen so that we never wait or even reclaim from * the free list to below its target minimum. */ if (vnlru_free_locked(1, NULL) > 0) goto alloc; if (mp == NULL || (mp->mnt_kern_flag & MNTK_SUSPEND) == 0) { /* * Wait for space for a new vnode. */ vnlru_kick(); msleep(&vnlruproc_sig, &vnode_list_mtx, PVFS, "vlruwk", hz); if (atomic_load_long(&numvnodes) + 1 > desiredvnodes && vnlru_read_freevnodes() > 1) vnlru_free_locked(1, NULL); } alloc: rnumvnodes = atomic_fetchadd_long(&numvnodes, 1) + 1; if (vnlru_under(rnumvnodes, vlowat)) vnlru_kick(); mtx_unlock(&vnode_list_mtx); return (uma_zalloc(vnode_zone, M_WAITOK)); } static struct vnode * vn_alloc(struct mount *mp) { u_long rnumvnodes; if (__predict_false(vn_alloc_cyclecount != 0)) return (vn_alloc_hard(mp)); rnumvnodes = atomic_fetchadd_long(&numvnodes, 1) + 1; if (__predict_false(vnlru_under_unlocked(rnumvnodes, vlowat))) { atomic_subtract_long(&numvnodes, 1); return (vn_alloc_hard(mp)); } return (uma_zalloc(vnode_zone, M_WAITOK)); } static void vn_free(struct vnode *vp) { atomic_subtract_long(&numvnodes, 1); uma_zfree(vnode_zone, vp); } /* * Return the next vnode from the free list. */ int getnewvnode(const char *tag, struct mount *mp, struct vop_vector *vops, struct vnode **vpp) { struct vnode *vp; struct thread *td; struct lock_object *lo; CTR3(KTR_VFS, "%s: mp %p with tag %s", __func__, mp, tag); KASSERT(vops->registered, ("%s: not registered vector op %p\n", __func__, vops)); td = curthread; if (td->td_vp_reserved != NULL) { vp = td->td_vp_reserved; td->td_vp_reserved = NULL; } else { vp = vn_alloc(mp); } counter_u64_add(vnodes_created, 1); /* * Locks are given the generic name "vnode" when created. * Follow the historic practice of using the filesystem * name when they allocated, e.g., "zfs", "ufs", "nfs, etc. * * Locks live in a witness group keyed on their name. Thus, * when a lock is renamed, it must also move from the witness * group of its old name to the witness group of its new name. * * The change only needs to be made when the vnode moves * from one filesystem type to another. We ensure that each * filesystem use a single static name pointer for its tag so * that we can compare pointers rather than doing a strcmp(). */ lo = &vp->v_vnlock->lock_object; #ifdef WITNESS if (lo->lo_name != tag) { #endif lo->lo_name = tag; #ifdef WITNESS WITNESS_DESTROY(lo); WITNESS_INIT(lo, tag); } #endif /* * By default, don't allow shared locks unless filesystems opt-in. */ vp->v_vnlock->lock_object.lo_flags |= LK_NOSHARE; /* * Finalize various vnode identity bits. */ KASSERT(vp->v_object == NULL, ("stale v_object %p", vp)); KASSERT(vp->v_lockf == NULL, ("stale v_lockf %p", vp)); KASSERT(vp->v_pollinfo == NULL, ("stale v_pollinfo %p", vp)); vp->v_type = VNON; vp->v_op = vops; v_init_counters(vp); vp->v_bufobj.bo_ops = &buf_ops_bio; #ifdef DIAGNOSTIC if (mp == NULL && vops != &dead_vnodeops) printf("NULL mp in getnewvnode(9), tag %s\n", tag); #endif #ifdef MAC mac_vnode_init(vp); if (mp != NULL && (mp->mnt_flag & MNT_MULTILABEL) == 0) mac_vnode_associate_singlelabel(mp, vp); #endif if (mp != NULL) { vp->v_bufobj.bo_bsize = mp->mnt_stat.f_iosize; if ((mp->mnt_kern_flag & MNTK_NOKNOTE) != 0) vp->v_vflag |= VV_NOKNOTE; } /* * For the filesystems which do not use vfs_hash_insert(), * still initialize v_hash to have vfs_hash_index() useful. * E.g., nullfs uses vfs_hash_index() on the lower vnode for * its own hashing. */ vp->v_hash = (uintptr_t)vp >> vnsz2log; *vpp = vp; return (0); } void getnewvnode_reserve(void) { struct thread *td; td = curthread; MPASS(td->td_vp_reserved == NULL); td->td_vp_reserved = vn_alloc(NULL); } void getnewvnode_drop_reserve(void) { struct thread *td; td = curthread; if (td->td_vp_reserved != NULL) { vn_free(td->td_vp_reserved); td->td_vp_reserved = NULL; } } static void freevnode(struct vnode *vp) { struct bufobj *bo; /* * The vnode has been marked for destruction, so free it. * * The vnode will be returned to the zone where it will * normally remain until it is needed for another vnode. We * need to cleanup (or verify that the cleanup has already * been done) any residual data left from its current use * so as not to contaminate the freshly allocated vnode. */ CTR2(KTR_VFS, "%s: destroying the vnode %p", __func__, vp); bo = &vp->v_bufobj; VNASSERT(vp->v_data == NULL, vp, ("cleaned vnode isn't")); VNASSERT(vp->v_holdcnt == 0, vp, ("Non-zero hold count")); VNASSERT(vp->v_usecount == 0, vp, ("Non-zero use count")); VNASSERT(vp->v_writecount == 0, vp, ("Non-zero write count")); VNASSERT(bo->bo_numoutput == 0, vp, ("Clean vnode has pending I/O's")); VNASSERT(bo->bo_clean.bv_cnt == 0, vp, ("cleanbufcnt not 0")); VNASSERT(pctrie_is_empty(&bo->bo_clean.bv_root), vp, ("clean blk trie not empty")); VNASSERT(bo->bo_dirty.bv_cnt == 0, vp, ("dirtybufcnt not 0")); VNASSERT(pctrie_is_empty(&bo->bo_dirty.bv_root), vp, ("dirty blk trie not empty")); VNASSERT(TAILQ_EMPTY(&vp->v_cache_dst), vp, ("vp has namecache dst")); VNASSERT(LIST_EMPTY(&vp->v_cache_src), vp, ("vp has namecache src")); VNASSERT(vp->v_cache_dd == NULL, vp, ("vp has namecache for ..")); VNASSERT(TAILQ_EMPTY(&vp->v_rl.rl_waiters), vp, ("Dangling rangelock waiters")); VI_UNLOCK(vp); #ifdef MAC mac_vnode_destroy(vp); #endif if (vp->v_pollinfo != NULL) { destroy_vpollinfo(vp->v_pollinfo); vp->v_pollinfo = NULL; } #ifdef INVARIANTS /* XXX Elsewhere we detect an already freed vnode via NULL v_op. */ vp->v_op = NULL; #endif vp->v_mountedhere = NULL; vp->v_unpcb = NULL; vp->v_rdev = NULL; vp->v_fifoinfo = NULL; vp->v_lasta = vp->v_clen = vp->v_cstart = vp->v_lastw = 0; vp->v_irflag = 0; vp->v_iflag = 0; vp->v_vflag = 0; bo->bo_flag = 0; vn_free(vp); } /* * Delete from old mount point vnode list, if on one. */ static void delmntque(struct vnode *vp) { struct mount *mp; VNPASS((vp->v_mflag & VMP_LAZYLIST) == 0, vp); mp = vp->v_mount; if (mp == NULL) return; MNT_ILOCK(mp); VI_LOCK(vp); vp->v_mount = NULL; VI_UNLOCK(vp); VNASSERT(mp->mnt_nvnodelistsize > 0, vp, ("bad mount point vnode list size")); TAILQ_REMOVE(&mp->mnt_nvnodelist, vp, v_nmntvnodes); mp->mnt_nvnodelistsize--; MNT_REL(mp); MNT_IUNLOCK(mp); } static void insmntque_stddtr(struct vnode *vp, void *dtr_arg) { vp->v_data = NULL; vp->v_op = &dead_vnodeops; vgone(vp); vput(vp); } /* * Insert into list of vnodes for the new mount point, if available. */ int insmntque1(struct vnode *vp, struct mount *mp, void (*dtr)(struct vnode *, void *), void *dtr_arg) { KASSERT(vp->v_mount == NULL, ("insmntque: vnode already on per mount vnode list")); VNASSERT(mp != NULL, vp, ("Don't call insmntque(foo, NULL)")); ASSERT_VOP_ELOCKED(vp, "insmntque: non-locked vp"); /* * We acquire the vnode interlock early to ensure that the * vnode cannot be recycled by another process releasing a * holdcnt on it before we get it on both the vnode list * and the active vnode list. The mount mutex protects only * manipulation of the vnode list and the vnode freelist * mutex protects only manipulation of the active vnode list. * Hence the need to hold the vnode interlock throughout. */ MNT_ILOCK(mp); VI_LOCK(vp); if (((mp->mnt_kern_flag & MNTK_UNMOUNT) != 0 && ((mp->mnt_kern_flag & MNTK_UNMOUNTF) != 0 || mp->mnt_nvnodelistsize == 0)) && (vp->v_vflag & VV_FORCEINSMQ) == 0) { VI_UNLOCK(vp); MNT_IUNLOCK(mp); if (dtr != NULL) dtr(vp, dtr_arg); return (EBUSY); } vp->v_mount = mp; MNT_REF(mp); TAILQ_INSERT_TAIL(&mp->mnt_nvnodelist, vp, v_nmntvnodes); VNASSERT(mp->mnt_nvnodelistsize >= 0, vp, ("neg mount point vnode list size")); mp->mnt_nvnodelistsize++; VI_UNLOCK(vp); MNT_IUNLOCK(mp); return (0); } int insmntque(struct vnode *vp, struct mount *mp) { return (insmntque1(vp, mp, insmntque_stddtr, NULL)); } /* * Flush out and invalidate all buffers associated with a bufobj * Called with the underlying object locked. */ int bufobj_invalbuf(struct bufobj *bo, int flags, int slpflag, int slptimeo) { int error; BO_LOCK(bo); if (flags & V_SAVE) { error = bufobj_wwait(bo, slpflag, slptimeo); if (error) { BO_UNLOCK(bo); return (error); } if (bo->bo_dirty.bv_cnt > 0) { BO_UNLOCK(bo); if ((error = BO_SYNC(bo, MNT_WAIT)) != 0) return (error); /* * XXX We could save a lock/unlock if this was only * enabled under INVARIANTS */ BO_LOCK(bo); if (bo->bo_numoutput > 0 || bo->bo_dirty.bv_cnt > 0) panic("vinvalbuf: dirty bufs"); } } /* * If you alter this loop please notice that interlock is dropped and * reacquired in flushbuflist. Special care is needed to ensure that * no race conditions occur from this. */ do { error = flushbuflist(&bo->bo_clean, flags, bo, slpflag, slptimeo); if (error == 0 && !(flags & V_CLEANONLY)) error = flushbuflist(&bo->bo_dirty, flags, bo, slpflag, slptimeo); if (error != 0 && error != EAGAIN) { BO_UNLOCK(bo); return (error); } } while (error != 0); /* * Wait for I/O to complete. XXX needs cleaning up. The vnode can * have write I/O in-progress but if there is a VM object then the * VM object can also have read-I/O in-progress. */ do { bufobj_wwait(bo, 0, 0); if ((flags & V_VMIO) == 0 && bo->bo_object != NULL) { BO_UNLOCK(bo); vm_object_pip_wait_unlocked(bo->bo_object, "bovlbx"); BO_LOCK(bo); } } while (bo->bo_numoutput > 0); BO_UNLOCK(bo); /* * Destroy the copy in the VM cache, too. */ if (bo->bo_object != NULL && (flags & (V_ALT | V_NORMAL | V_CLEANONLY | V_VMIO)) == 0) { VM_OBJECT_WLOCK(bo->bo_object); vm_object_page_remove(bo->bo_object, 0, 0, (flags & V_SAVE) ? OBJPR_CLEANONLY : 0); VM_OBJECT_WUNLOCK(bo->bo_object); } #ifdef INVARIANTS BO_LOCK(bo); if ((flags & (V_ALT | V_NORMAL | V_CLEANONLY | V_VMIO | V_ALLOWCLEAN)) == 0 && (bo->bo_dirty.bv_cnt > 0 || bo->bo_clean.bv_cnt > 0)) panic("vinvalbuf: flush failed"); if ((flags & (V_ALT | V_NORMAL | V_CLEANONLY | V_VMIO)) == 0 && bo->bo_dirty.bv_cnt > 0) panic("vinvalbuf: flush dirty failed"); BO_UNLOCK(bo); #endif return (0); } /* * Flush out and invalidate all buffers associated with a vnode. * Called with the underlying object locked. */ int vinvalbuf(struct vnode *vp, int flags, int slpflag, int slptimeo) { CTR3(KTR_VFS, "%s: vp %p with flags %d", __func__, vp, flags); ASSERT_VOP_LOCKED(vp, "vinvalbuf"); if (vp->v_object != NULL && vp->v_object->handle != vp) return (0); return (bufobj_invalbuf(&vp->v_bufobj, flags, slpflag, slptimeo)); } /* * Flush out buffers on the specified list. * */ static int flushbuflist(struct bufv *bufv, int flags, struct bufobj *bo, int slpflag, int slptimeo) { struct buf *bp, *nbp; int retval, error; daddr_t lblkno; b_xflags_t xflags; ASSERT_BO_WLOCKED(bo); retval = 0; TAILQ_FOREACH_SAFE(bp, &bufv->bv_hd, b_bobufs, nbp) { /* * If we are flushing both V_NORMAL and V_ALT buffers then * do not skip any buffers. If we are flushing only V_NORMAL * buffers then skip buffers marked as BX_ALTDATA. If we are * flushing only V_ALT buffers then skip buffers not marked * as BX_ALTDATA. */ if (((flags & (V_NORMAL | V_ALT)) != (V_NORMAL | V_ALT)) && (((flags & V_NORMAL) && (bp->b_xflags & BX_ALTDATA) != 0) || ((flags & V_ALT) && (bp->b_xflags & BX_ALTDATA) == 0))) { continue; } if (nbp != NULL) { lblkno = nbp->b_lblkno; xflags = nbp->b_xflags & (BX_VNDIRTY | BX_VNCLEAN); } retval = EAGAIN; error = BUF_TIMELOCK(bp, LK_EXCLUSIVE | LK_SLEEPFAIL | LK_INTERLOCK, BO_LOCKPTR(bo), "flushbuf", slpflag, slptimeo); if (error) { BO_LOCK(bo); return (error != ENOLCK ? error : EAGAIN); } KASSERT(bp->b_bufobj == bo, ("bp %p wrong b_bufobj %p should be %p", bp, bp->b_bufobj, bo)); /* * XXX Since there are no node locks for NFS, I * believe there is a slight chance that a delayed * write will occur while sleeping just above, so * check for it. */ if (((bp->b_flags & (B_DELWRI | B_INVAL)) == B_DELWRI) && (flags & V_SAVE)) { bremfree(bp); bp->b_flags |= B_ASYNC; bwrite(bp); BO_LOCK(bo); return (EAGAIN); /* XXX: why not loop ? */ } bremfree(bp); bp->b_flags |= (B_INVAL | B_RELBUF); bp->b_flags &= ~B_ASYNC; brelse(bp); BO_LOCK(bo); if (nbp == NULL) break; nbp = gbincore(bo, lblkno); if (nbp == NULL || (nbp->b_xflags & (BX_VNDIRTY | BX_VNCLEAN)) != xflags) break; /* nbp invalid */ } return (retval); } int bnoreuselist(struct bufv *bufv, struct bufobj *bo, daddr_t startn, daddr_t endn) { struct buf *bp; int error; daddr_t lblkno; ASSERT_BO_LOCKED(bo); for (lblkno = startn;;) { again: bp = BUF_PCTRIE_LOOKUP_GE(&bufv->bv_root, lblkno); if (bp == NULL || bp->b_lblkno >= endn || bp->b_lblkno < startn) break; error = BUF_TIMELOCK(bp, LK_EXCLUSIVE | LK_SLEEPFAIL | LK_INTERLOCK, BO_LOCKPTR(bo), "brlsfl", 0, 0); if (error != 0) { BO_RLOCK(bo); if (error == ENOLCK) goto again; return (error); } KASSERT(bp->b_bufobj == bo, ("bp %p wrong b_bufobj %p should be %p", bp, bp->b_bufobj, bo)); lblkno = bp->b_lblkno + 1; if ((bp->b_flags & B_MANAGED) == 0) bremfree(bp); bp->b_flags |= B_RELBUF; /* * In the VMIO case, use the B_NOREUSE flag to hint that the * pages backing each buffer in the range are unlikely to be * reused. Dirty buffers will have the hint applied once * they've been written. */ if ((bp->b_flags & B_VMIO) != 0) bp->b_flags |= B_NOREUSE; brelse(bp); BO_RLOCK(bo); } return (0); } /* * Truncate a file's buffer and pages to a specified length. This * is in lieu of the old vinvalbuf mechanism, which performed unneeded * sync activity. */ int vtruncbuf(struct vnode *vp, off_t length, int blksize) { struct buf *bp, *nbp; struct bufobj *bo; daddr_t startlbn; CTR4(KTR_VFS, "%s: vp %p with block %d:%ju", __func__, vp, blksize, (uintmax_t)length); /* * Round up to the *next* lbn. */ startlbn = howmany(length, blksize); ASSERT_VOP_LOCKED(vp, "vtruncbuf"); bo = &vp->v_bufobj; restart_unlocked: BO_LOCK(bo); while (v_inval_buf_range_locked(vp, bo, startlbn, INT64_MAX) == EAGAIN) ; if (length > 0) { restartsync: TAILQ_FOREACH_SAFE(bp, &bo->bo_dirty.bv_hd, b_bobufs, nbp) { if (bp->b_lblkno > 0) continue; /* * Since we hold the vnode lock this should only * fail if we're racing with the buf daemon. */ if (BUF_LOCK(bp, LK_EXCLUSIVE | LK_SLEEPFAIL | LK_INTERLOCK, BO_LOCKPTR(bo)) == ENOLCK) goto restart_unlocked; VNASSERT((bp->b_flags & B_DELWRI), vp, ("buf(%p) on dirty queue without DELWRI", bp)); bremfree(bp); bawrite(bp); BO_LOCK(bo); goto restartsync; } } bufobj_wwait(bo, 0, 0); BO_UNLOCK(bo); vnode_pager_setsize(vp, length); return (0); } /* * Invalidate the cached pages of a file's buffer within the range of block * numbers [startlbn, endlbn). */ void v_inval_buf_range(struct vnode *vp, daddr_t startlbn, daddr_t endlbn, int blksize) { struct bufobj *bo; off_t start, end; ASSERT_VOP_LOCKED(vp, "v_inval_buf_range"); start = blksize * startlbn; end = blksize * endlbn; bo = &vp->v_bufobj; BO_LOCK(bo); MPASS(blksize == bo->bo_bsize); while (v_inval_buf_range_locked(vp, bo, startlbn, endlbn) == EAGAIN) ; BO_UNLOCK(bo); vn_pages_remove(vp, OFF_TO_IDX(start), OFF_TO_IDX(end + PAGE_SIZE - 1)); } static int v_inval_buf_range_locked(struct vnode *vp, struct bufobj *bo, daddr_t startlbn, daddr_t endlbn) { struct buf *bp, *nbp; bool anyfreed; ASSERT_VOP_LOCKED(vp, "v_inval_buf_range_locked"); ASSERT_BO_LOCKED(bo); do { anyfreed = false; TAILQ_FOREACH_SAFE(bp, &bo->bo_clean.bv_hd, b_bobufs, nbp) { if (bp->b_lblkno < startlbn || bp->b_lblkno >= endlbn) continue; if (BUF_LOCK(bp, LK_EXCLUSIVE | LK_SLEEPFAIL | LK_INTERLOCK, BO_LOCKPTR(bo)) == ENOLCK) { BO_LOCK(bo); return (EAGAIN); } bremfree(bp); bp->b_flags |= B_INVAL | B_RELBUF; bp->b_flags &= ~B_ASYNC; brelse(bp); anyfreed = true; BO_LOCK(bo); if (nbp != NULL && (((nbp->b_xflags & BX_VNCLEAN) == 0) || nbp->b_vp != vp || (nbp->b_flags & B_DELWRI) != 0)) return (EAGAIN); } TAILQ_FOREACH_SAFE(bp, &bo->bo_dirty.bv_hd, b_bobufs, nbp) { if (bp->b_lblkno < startlbn || bp->b_lblkno >= endlbn) continue; if (BUF_LOCK(bp, LK_EXCLUSIVE | LK_SLEEPFAIL | LK_INTERLOCK, BO_LOCKPTR(bo)) == ENOLCK) { BO_LOCK(bo); return (EAGAIN); } bremfree(bp); bp->b_flags |= B_INVAL | B_RELBUF; bp->b_flags &= ~B_ASYNC; brelse(bp); anyfreed = true; BO_LOCK(bo); if (nbp != NULL && (((nbp->b_xflags & BX_VNDIRTY) == 0) || (nbp->b_vp != vp) || (nbp->b_flags & B_DELWRI) == 0)) return (EAGAIN); } } while (anyfreed); return (0); } static void buf_vlist_remove(struct buf *bp) { struct bufv *bv; KASSERT(bp->b_bufobj != NULL, ("No b_bufobj %p", bp)); ASSERT_BO_WLOCKED(bp->b_bufobj); KASSERT((bp->b_xflags & (BX_VNDIRTY|BX_VNCLEAN)) != (BX_VNDIRTY|BX_VNCLEAN), ("buf_vlist_remove: Buf %p is on two lists", bp)); if (bp->b_xflags & BX_VNDIRTY) bv = &bp->b_bufobj->bo_dirty; else bv = &bp->b_bufobj->bo_clean; BUF_PCTRIE_REMOVE(&bv->bv_root, bp->b_lblkno); TAILQ_REMOVE(&bv->bv_hd, bp, b_bobufs); bv->bv_cnt--; bp->b_xflags &= ~(BX_VNDIRTY | BX_VNCLEAN); } /* * Add the buffer to the sorted clean or dirty block list. * * NOTE: xflags is passed as a constant, optimizing this inline function! */ static void buf_vlist_add(struct buf *bp, struct bufobj *bo, b_xflags_t xflags) { struct bufv *bv; struct buf *n; int error; ASSERT_BO_WLOCKED(bo); KASSERT((xflags & BX_VNDIRTY) == 0 || (bo->bo_flag & BO_DEAD) == 0, ("dead bo %p", bo)); KASSERT((bp->b_xflags & (BX_VNDIRTY|BX_VNCLEAN)) == 0, ("buf_vlist_add: Buf %p has existing xflags %d", bp, bp->b_xflags)); bp->b_xflags |= xflags; if (xflags & BX_VNDIRTY) bv = &bo->bo_dirty; else bv = &bo->bo_clean; /* * Keep the list ordered. Optimize empty list insertion. Assume * we tend to grow at the tail so lookup_le should usually be cheaper * than _ge. */ if (bv->bv_cnt == 0 || bp->b_lblkno > TAILQ_LAST(&bv->bv_hd, buflists)->b_lblkno) TAILQ_INSERT_TAIL(&bv->bv_hd, bp, b_bobufs); else if ((n = BUF_PCTRIE_LOOKUP_LE(&bv->bv_root, bp->b_lblkno)) == NULL) TAILQ_INSERT_HEAD(&bv->bv_hd, bp, b_bobufs); else TAILQ_INSERT_AFTER(&bv->bv_hd, n, bp, b_bobufs); error = BUF_PCTRIE_INSERT(&bv->bv_root, bp); if (error) panic("buf_vlist_add: Preallocated nodes insufficient."); bv->bv_cnt++; } /* * Look up a buffer using the buffer tries. */ struct buf * gbincore(struct bufobj *bo, daddr_t lblkno) { struct buf *bp; ASSERT_BO_LOCKED(bo); bp = BUF_PCTRIE_LOOKUP(&bo->bo_clean.bv_root, lblkno); if (bp != NULL) return (bp); return BUF_PCTRIE_LOOKUP(&bo->bo_dirty.bv_root, lblkno); } /* * Associate a buffer with a vnode. */ void bgetvp(struct vnode *vp, struct buf *bp) { struct bufobj *bo; bo = &vp->v_bufobj; ASSERT_BO_WLOCKED(bo); VNASSERT(bp->b_vp == NULL, bp->b_vp, ("bgetvp: not free")); CTR3(KTR_BUF, "bgetvp(%p) vp %p flags %X", bp, vp, bp->b_flags); VNASSERT((bp->b_xflags & (BX_VNDIRTY|BX_VNCLEAN)) == 0, vp, ("bgetvp: bp already attached! %p", bp)); vhold(vp); bp->b_vp = vp; bp->b_bufobj = bo; /* * Insert onto list for new vnode. */ buf_vlist_add(bp, bo, BX_VNCLEAN); } /* * Disassociate a buffer from a vnode. */ void brelvp(struct buf *bp) { struct bufobj *bo; struct vnode *vp; CTR3(KTR_BUF, "brelvp(%p) vp %p flags %X", bp, bp->b_vp, bp->b_flags); KASSERT(bp->b_vp != NULL, ("brelvp: NULL")); /* * Delete from old vnode list, if on one. */ vp = bp->b_vp; /* XXX */ bo = bp->b_bufobj; BO_LOCK(bo); if (bp->b_xflags & (BX_VNDIRTY | BX_VNCLEAN)) buf_vlist_remove(bp); else panic("brelvp: Buffer %p not on queue.", bp); if ((bo->bo_flag & BO_ONWORKLST) && bo->bo_dirty.bv_cnt == 0) { bo->bo_flag &= ~BO_ONWORKLST; mtx_lock(&sync_mtx); LIST_REMOVE(bo, bo_synclist); syncer_worklist_len--; mtx_unlock(&sync_mtx); } bp->b_vp = NULL; bp->b_bufobj = NULL; BO_UNLOCK(bo); vdrop(vp); } /* * Add an item to the syncer work queue. */ static void vn_syncer_add_to_worklist(struct bufobj *bo, int delay) { int slot; ASSERT_BO_WLOCKED(bo); mtx_lock(&sync_mtx); if (bo->bo_flag & BO_ONWORKLST) LIST_REMOVE(bo, bo_synclist); else { bo->bo_flag |= BO_ONWORKLST; syncer_worklist_len++; } if (delay > syncer_maxdelay - 2) delay = syncer_maxdelay - 2; slot = (syncer_delayno + delay) & syncer_mask; LIST_INSERT_HEAD(&syncer_workitem_pending[slot], bo, bo_synclist); mtx_unlock(&sync_mtx); } static int sysctl_vfs_worklist_len(SYSCTL_HANDLER_ARGS) { int error, len; mtx_lock(&sync_mtx); len = syncer_worklist_len - sync_vnode_count; mtx_unlock(&sync_mtx); error = SYSCTL_OUT(req, &len, sizeof(len)); return (error); } SYSCTL_PROC(_vfs, OID_AUTO, worklist_len, CTLTYPE_INT | CTLFLAG_MPSAFE| CTLFLAG_RD, NULL, 0, sysctl_vfs_worklist_len, "I", "Syncer thread worklist length"); static struct proc *updateproc; static void sched_sync(void); static struct kproc_desc up_kp = { "syncer", sched_sync, &updateproc }; SYSINIT(syncer, SI_SUB_KTHREAD_UPDATE, SI_ORDER_FIRST, kproc_start, &up_kp); static int sync_vnode(struct synclist *slp, struct bufobj **bo, struct thread *td) { struct vnode *vp; struct mount *mp; *bo = LIST_FIRST(slp); if (*bo == NULL) return (0); vp = bo2vnode(*bo); if (VOP_ISLOCKED(vp) != 0 || VI_TRYLOCK(vp) == 0) return (1); /* * We use vhold in case the vnode does not * successfully sync. vhold prevents the vnode from * going away when we unlock the sync_mtx so that * we can acquire the vnode interlock. */ vholdl(vp); mtx_unlock(&sync_mtx); VI_UNLOCK(vp); if (vn_start_write(vp, &mp, V_NOWAIT) != 0) { vdrop(vp); mtx_lock(&sync_mtx); return (*bo == LIST_FIRST(slp)); } vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); (void) VOP_FSYNC(vp, MNT_LAZY, td); VOP_UNLOCK(vp); vn_finished_write(mp); BO_LOCK(*bo); if (((*bo)->bo_flag & BO_ONWORKLST) != 0) { /* * Put us back on the worklist. The worklist * routine will remove us from our current * position and then add us back in at a later * position. */ vn_syncer_add_to_worklist(*bo, syncdelay); } BO_UNLOCK(*bo); vdrop(vp); mtx_lock(&sync_mtx); return (0); } static int first_printf = 1; /* * System filesystem synchronizer daemon. */ static void sched_sync(void) { struct synclist *next, *slp; struct bufobj *bo; long starttime; struct thread *td = curthread; int last_work_seen; int net_worklist_len; int syncer_final_iter; int error; last_work_seen = 0; syncer_final_iter = 0; syncer_state = SYNCER_RUNNING; starttime = time_uptime; td->td_pflags |= TDP_NORUNNINGBUF; EVENTHANDLER_REGISTER(shutdown_pre_sync, syncer_shutdown, td->td_proc, SHUTDOWN_PRI_LAST); mtx_lock(&sync_mtx); for (;;) { if (syncer_state == SYNCER_FINAL_DELAY && syncer_final_iter == 0) { mtx_unlock(&sync_mtx); kproc_suspend_check(td->td_proc); mtx_lock(&sync_mtx); } net_worklist_len = syncer_worklist_len - sync_vnode_count; if (syncer_state != SYNCER_RUNNING && starttime != time_uptime) { if (first_printf) { printf("\nSyncing disks, vnodes remaining... "); first_printf = 0; } printf("%d ", net_worklist_len); } starttime = time_uptime; /* * Push files whose dirty time has expired. Be careful * of interrupt race on slp queue. * * Skip over empty worklist slots when shutting down. */ do { slp = &syncer_workitem_pending[syncer_delayno]; syncer_delayno += 1; if (syncer_delayno == syncer_maxdelay) syncer_delayno = 0; next = &syncer_workitem_pending[syncer_delayno]; /* * If the worklist has wrapped since the * it was emptied of all but syncer vnodes, * switch to the FINAL_DELAY state and run * for one more second. */ if (syncer_state == SYNCER_SHUTTING_DOWN && net_worklist_len == 0 && last_work_seen == syncer_delayno) { syncer_state = SYNCER_FINAL_DELAY; syncer_final_iter = SYNCER_SHUTDOWN_SPEEDUP; } } while (syncer_state != SYNCER_RUNNING && LIST_EMPTY(slp) && syncer_worklist_len > 0); /* * Keep track of the last time there was anything * on the worklist other than syncer vnodes. * Return to the SHUTTING_DOWN state if any * new work appears. */ if (net_worklist_len > 0 || syncer_state == SYNCER_RUNNING) last_work_seen = syncer_delayno; if (net_worklist_len > 0 && syncer_state == SYNCER_FINAL_DELAY) syncer_state = SYNCER_SHUTTING_DOWN; while (!LIST_EMPTY(slp)) { error = sync_vnode(slp, &bo, td); if (error == 1) { LIST_REMOVE(bo, bo_synclist); LIST_INSERT_HEAD(next, bo, bo_synclist); continue; } if (first_printf == 0) { /* * Drop the sync mutex, because some watchdog * drivers need to sleep while patting */ mtx_unlock(&sync_mtx); wdog_kern_pat(WD_LASTVAL); mtx_lock(&sync_mtx); } } if (syncer_state == SYNCER_FINAL_DELAY && syncer_final_iter > 0) syncer_final_iter--; /* * The variable rushjob allows the kernel to speed up the * processing of the filesystem syncer process. A rushjob * value of N tells the filesystem syncer to process the next * N seconds worth of work on its queue ASAP. Currently rushjob * is used by the soft update code to speed up the filesystem * syncer process when the incore state is getting so far * ahead of the disk that the kernel memory pool is being * threatened with exhaustion. */ if (rushjob > 0) { rushjob -= 1; continue; } /* * Just sleep for a short period of time between * iterations when shutting down to allow some I/O * to happen. * * If it has taken us less than a second to process the * current work, then wait. Otherwise start right over * again. We can still lose time if any single round * takes more than two seconds, but it does not really * matter as we are just trying to generally pace the * filesystem activity. */ if (syncer_state != SYNCER_RUNNING || time_uptime == starttime) { thread_lock(td); sched_prio(td, PPAUSE); thread_unlock(td); } if (syncer_state != SYNCER_RUNNING) cv_timedwait(&sync_wakeup, &sync_mtx, hz / SYNCER_SHUTDOWN_SPEEDUP); else if (time_uptime == starttime) cv_timedwait(&sync_wakeup, &sync_mtx, hz); } } /* * Request the syncer daemon to speed up its work. * We never push it to speed up more than half of its * normal turn time, otherwise it could take over the cpu. */ int speedup_syncer(void) { int ret = 0; mtx_lock(&sync_mtx); if (rushjob < syncdelay / 2) { rushjob += 1; stat_rush_requests += 1; ret = 1; } mtx_unlock(&sync_mtx); cv_broadcast(&sync_wakeup); return (ret); } /* * Tell the syncer to speed up its work and run though its work * list several times, then tell it to shut down. */ static void syncer_shutdown(void *arg, int howto) { if (howto & RB_NOSYNC) return; mtx_lock(&sync_mtx); syncer_state = SYNCER_SHUTTING_DOWN; rushjob = 0; mtx_unlock(&sync_mtx); cv_broadcast(&sync_wakeup); kproc_shutdown(arg, howto); } void syncer_suspend(void) { syncer_shutdown(updateproc, 0); } void syncer_resume(void) { mtx_lock(&sync_mtx); first_printf = 1; syncer_state = SYNCER_RUNNING; mtx_unlock(&sync_mtx); cv_broadcast(&sync_wakeup); kproc_resume(updateproc); } /* * Reassign a buffer from one vnode to another. * Used to assign file specific control information * (indirect blocks) to the vnode to which they belong. */ void reassignbuf(struct buf *bp) { struct vnode *vp; struct bufobj *bo; int delay; #ifdef INVARIANTS struct bufv *bv; #endif vp = bp->b_vp; bo = bp->b_bufobj; ++reassignbufcalls; CTR3(KTR_BUF, "reassignbuf(%p) vp %p flags %X", bp, bp->b_vp, bp->b_flags); /* * B_PAGING flagged buffers cannot be reassigned because their vp * is not fully linked in. */ if (bp->b_flags & B_PAGING) panic("cannot reassign paging buffer"); /* * Delete from old vnode list, if on one. */ BO_LOCK(bo); if (bp->b_xflags & (BX_VNDIRTY | BX_VNCLEAN)) buf_vlist_remove(bp); else panic("reassignbuf: Buffer %p not on queue.", bp); /* * If dirty, put on list of dirty buffers; otherwise insert onto list * of clean buffers. */ if (bp->b_flags & B_DELWRI) { if ((bo->bo_flag & BO_ONWORKLST) == 0) { switch (vp->v_type) { case VDIR: delay = dirdelay; break; case VCHR: delay = metadelay; break; default: delay = filedelay; } vn_syncer_add_to_worklist(bo, delay); } buf_vlist_add(bp, bo, BX_VNDIRTY); } else { buf_vlist_add(bp, bo, BX_VNCLEAN); if ((bo->bo_flag & BO_ONWORKLST) && bo->bo_dirty.bv_cnt == 0) { mtx_lock(&sync_mtx); LIST_REMOVE(bo, bo_synclist); syncer_worklist_len--; mtx_unlock(&sync_mtx); bo->bo_flag &= ~BO_ONWORKLST; } } #ifdef INVARIANTS bv = &bo->bo_clean; bp = TAILQ_FIRST(&bv->bv_hd); KASSERT(bp == NULL || bp->b_bufobj == bo, ("bp %p wrong b_bufobj %p should be %p", bp, bp->b_bufobj, bo)); bp = TAILQ_LAST(&bv->bv_hd, buflists); KASSERT(bp == NULL || bp->b_bufobj == bo, ("bp %p wrong b_bufobj %p should be %p", bp, bp->b_bufobj, bo)); bv = &bo->bo_dirty; bp = TAILQ_FIRST(&bv->bv_hd); KASSERT(bp == NULL || bp->b_bufobj == bo, ("bp %p wrong b_bufobj %p should be %p", bp, bp->b_bufobj, bo)); bp = TAILQ_LAST(&bv->bv_hd, buflists); KASSERT(bp == NULL || bp->b_bufobj == bo, ("bp %p wrong b_bufobj %p should be %p", bp, bp->b_bufobj, bo)); #endif BO_UNLOCK(bo); } static void v_init_counters(struct vnode *vp) { VNASSERT(vp->v_type == VNON && vp->v_data == NULL && vp->v_iflag == 0, vp, ("%s called for an initialized vnode", __FUNCTION__)); ASSERT_VI_UNLOCKED(vp, __FUNCTION__); refcount_init(&vp->v_holdcnt, 1); refcount_init(&vp->v_usecount, 1); } /* * Increment si_usecount of the associated device, if any. */ static void v_incr_devcount(struct vnode *vp) { ASSERT_VI_LOCKED(vp, __FUNCTION__); if (vp->v_type == VCHR && vp->v_rdev != NULL) { dev_lock(); vp->v_rdev->si_usecount++; dev_unlock(); } } /* * Decrement si_usecount of the associated device, if any. * * The caller is required to hold the interlock when transitioning a VCHR use * count to zero. This prevents a race with devfs_reclaim_vchr() that would * leak a si_usecount reference. The vnode lock will also prevent this race * if it is held while dropping the last ref. * * The race is: * * CPU1 CPU2 * devfs_reclaim_vchr * make v_usecount == 0 * VI_LOCK * sees v_usecount == 0, no updates * vp->v_rdev = NULL; * ... * VI_UNLOCK * VI_LOCK * v_decr_devcount * sees v_rdev == NULL, no updates * * In this scenario si_devcount decrement is not performed. */ static void v_decr_devcount(struct vnode *vp) { ASSERT_VOP_LOCKED(vp, __func__); ASSERT_VI_LOCKED(vp, __FUNCTION__); if (vp->v_type == VCHR && vp->v_rdev != NULL) { dev_lock(); VNPASS(vp->v_rdev->si_usecount > 0, vp); vp->v_rdev->si_usecount--; dev_unlock(); } } /* * Grab a particular vnode from the free list, increment its * reference count and lock it. VIRF_DOOMED is set if the vnode * is being destroyed. Only callers who specify LK_RETRY will * see doomed vnodes. If inactive processing was delayed in * vput try to do it here. * * usecount is manipulated using atomics without holding any locks. * * holdcnt can be manipulated using atomics without holding any locks, * except when transitioning 1<->0, in which case the interlock is held. */ enum vgetstate vget_prep(struct vnode *vp) { enum vgetstate vs; if (refcount_acquire_if_not_zero(&vp->v_usecount)) { vs = VGET_USECOUNT; } else { vhold(vp); vs = VGET_HOLDCNT; } return (vs); } int vget(struct vnode *vp, int flags, struct thread *td) { enum vgetstate vs; MPASS(td == curthread); vs = vget_prep(vp); return (vget_finish(vp, flags, vs)); } static int __noinline vget_finish_vchr(struct vnode *vp) { VNASSERT(vp->v_type == VCHR, vp, ("type != VCHR)")); /* * See the comment in vget_finish before usecount bump. */ if (refcount_acquire_if_not_zero(&vp->v_usecount)) { #ifdef INVARIANTS int old = atomic_fetchadd_int(&vp->v_holdcnt, -1); VNASSERT(old > 0, vp, ("%s: wrong hold count %d", __func__, old)); #else refcount_release(&vp->v_holdcnt); #endif return (0); } VI_LOCK(vp); if (refcount_acquire_if_not_zero(&vp->v_usecount)) { #ifdef INVARIANTS int old = atomic_fetchadd_int(&vp->v_holdcnt, -1); VNASSERT(old > 1, vp, ("%s: wrong hold count %d", __func__, old)); #else refcount_release(&vp->v_holdcnt); #endif VI_UNLOCK(vp); return (0); } v_incr_devcount(vp); refcount_acquire(&vp->v_usecount); VI_UNLOCK(vp); return (0); } int vget_finish(struct vnode *vp, int flags, enum vgetstate vs) { int error, old; if ((flags & LK_INTERLOCK) != 0) ASSERT_VI_LOCKED(vp, __func__); else ASSERT_VI_UNLOCKED(vp, __func__); VNPASS(vp->v_holdcnt > 0, vp); VNPASS(vs == VGET_HOLDCNT || vp->v_usecount > 0, vp); error = vn_lock(vp, flags); if (__predict_false(error != 0)) { if (vs == VGET_USECOUNT) vrele(vp); else vdrop(vp); CTR2(KTR_VFS, "%s: impossible to lock vnode %p", __func__, vp); return (error); } if (vs == VGET_USECOUNT) return (0); if (__predict_false(vp->v_type == VCHR)) return (vget_finish_vchr(vp)); /* * We hold the vnode. If the usecount is 0 it will be utilized to keep * the vnode around. Otherwise someone else lended their hold count and * we have to drop ours. */ old = atomic_fetchadd_int(&vp->v_usecount, 1); VNASSERT(old >= 0, vp, ("%s: wrong use count %d", __func__, old)); if (old != 0) { #ifdef INVARIANTS old = atomic_fetchadd_int(&vp->v_holdcnt, -1); VNASSERT(old > 1, vp, ("%s: wrong hold count %d", __func__, old)); #else refcount_release(&vp->v_holdcnt); #endif } return (0); } /* * Increase the reference (use) and hold count of a vnode. * This will also remove the vnode from the free list if it is presently free. */ static void __noinline vref_vchr(struct vnode *vp, bool interlock) { /* * See the comment in vget_finish before usecount bump. */ if (!interlock) { if (refcount_acquire_if_not_zero(&vp->v_usecount)) { VNODE_REFCOUNT_FENCE_ACQ(); VNASSERT(vp->v_holdcnt > 0, vp, ("%s: active vnode not held", __func__)); return; } VI_LOCK(vp); /* * By the time we get here the vnode might have been doomed, at * which point the 0->1 use count transition is no longer * protected by the interlock. Since it can't bounce back to * VCHR and requires vref semantics, punt it back */ if (__predict_false(vp->v_type == VBAD)) { VI_UNLOCK(vp); vref(vp); return; } } VNASSERT(vp->v_type == VCHR, vp, ("type != VCHR)")); if (refcount_acquire_if_not_zero(&vp->v_usecount)) { VNODE_REFCOUNT_FENCE_ACQ(); VNASSERT(vp->v_holdcnt > 0, vp, ("%s: active vnode not held", __func__)); if (!interlock) VI_UNLOCK(vp); return; } vhold(vp); v_incr_devcount(vp); refcount_acquire(&vp->v_usecount); if (!interlock) VI_UNLOCK(vp); return; } void vref(struct vnode *vp) { int old; CTR2(KTR_VFS, "%s: vp %p", __func__, vp); if (__predict_false(vp->v_type == VCHR)) { vref_vchr(vp, false); return; } if (refcount_acquire_if_not_zero(&vp->v_usecount)) { VNODE_REFCOUNT_FENCE_ACQ(); VNASSERT(vp->v_holdcnt > 0, vp, ("%s: active vnode not held", __func__)); return; } vhold(vp); /* * See the comment in vget_finish. */ old = atomic_fetchadd_int(&vp->v_usecount, 1); VNASSERT(old >= 0, vp, ("%s: wrong use count %d", __func__, old)); if (old != 0) { #ifdef INVARIANTS old = atomic_fetchadd_int(&vp->v_holdcnt, -1); VNASSERT(old > 1, vp, ("%s: wrong hold count %d", __func__, old)); #else refcount_release(&vp->v_holdcnt); #endif } } void vrefl(struct vnode *vp) { ASSERT_VI_LOCKED(vp, __func__); CTR2(KTR_VFS, "%s: vp %p", __func__, vp); if (__predict_false(vp->v_type == VCHR)) { vref_vchr(vp, true); return; } vref(vp); } void vrefact(struct vnode *vp) { CTR2(KTR_VFS, "%s: vp %p", __func__, vp); #ifdef INVARIANTS int old = atomic_fetchadd_int(&vp->v_usecount, 1); VNASSERT(old > 0, vp, ("%s: wrong use count %d", __func__, old)); #else refcount_acquire(&vp->v_usecount); #endif } void vrefactn(struct vnode *vp, u_int n) { CTR2(KTR_VFS, "%s: vp %p", __func__, vp); #ifdef INVARIANTS int old = atomic_fetchadd_int(&vp->v_usecount, n); VNASSERT(old > 0, vp, ("%s: wrong use count %d", __func__, old)); #else atomic_add_int(&vp->v_usecount, n); #endif } /* * Return reference count of a vnode. * * The results of this call are only guaranteed when some mechanism is used to * stop other processes from gaining references to the vnode. This may be the * case if the caller holds the only reference. This is also useful when stale * data is acceptable as race conditions may be accounted for by some other * means. */ int vrefcnt(struct vnode *vp) { return (vp->v_usecount); } void vlazy(struct vnode *vp) { struct mount *mp; VNASSERT(vp->v_holdcnt > 0, vp, ("%s: vnode not held", __func__)); if ((vp->v_mflag & VMP_LAZYLIST) != 0) return; /* * We may get here for inactive routines after the vnode got doomed. */ if (VN_IS_DOOMED(vp)) return; mp = vp->v_mount; mtx_lock(&mp->mnt_listmtx); if ((vp->v_mflag & VMP_LAZYLIST) == 0) { vp->v_mflag |= VMP_LAZYLIST; TAILQ_INSERT_TAIL(&mp->mnt_lazyvnodelist, vp, v_lazylist); mp->mnt_lazyvnodelistsize++; } mtx_unlock(&mp->mnt_listmtx); } /* * This routine is only meant to be called from vgonel prior to dooming * the vnode. */ static void vunlazy_gone(struct vnode *vp) { struct mount *mp; ASSERT_VOP_ELOCKED(vp, __func__); ASSERT_VI_LOCKED(vp, __func__); VNPASS(!VN_IS_DOOMED(vp), vp); if (vp->v_mflag & VMP_LAZYLIST) { mp = vp->v_mount; mtx_lock(&mp->mnt_listmtx); VNPASS(vp->v_mflag & VMP_LAZYLIST, vp); vp->v_mflag &= ~VMP_LAZYLIST; TAILQ_REMOVE(&mp->mnt_lazyvnodelist, vp, v_lazylist); mp->mnt_lazyvnodelistsize--; mtx_unlock(&mp->mnt_listmtx); } } static void vdefer_inactive(struct vnode *vp) { ASSERT_VI_LOCKED(vp, __func__); VNASSERT(vp->v_holdcnt > 0, vp, ("%s: vnode without hold count", __func__)); if (VN_IS_DOOMED(vp)) { vdropl(vp); return; } if (vp->v_iflag & VI_DEFINACT) { VNASSERT(vp->v_holdcnt > 1, vp, ("lost hold count")); vdropl(vp); return; } if (vp->v_usecount > 0) { vp->v_iflag &= ~VI_OWEINACT; vdropl(vp); return; } vlazy(vp); vp->v_iflag |= VI_DEFINACT; VI_UNLOCK(vp); counter_u64_add(deferred_inact, 1); } static void vdefer_inactive_unlocked(struct vnode *vp) { VI_LOCK(vp); if ((vp->v_iflag & VI_OWEINACT) == 0) { vdropl(vp); return; } vdefer_inactive(vp); } enum vput_op { VRELE, VPUT, VUNREF }; /* * Handle ->v_usecount transitioning to 0. * * By releasing the last usecount we take ownership of the hold count which * provides liveness of the vnode, meaning we have to vdrop. * * If the vnode is of type VCHR we may need to decrement si_usecount, see * v_decr_devcount for details. * * For all vnodes we may need to perform inactive processing. It requires an * exclusive lock on the vnode, while it is legal to call here with only a * shared lock (or no locks). If locking the vnode in an expected manner fails, * inactive processing gets deferred to the syncer. * * XXX Some filesystems pass in an exclusively locked vnode and strongly depend * on the lock being held all the way until VOP_INACTIVE. This in particular * happens with UFS which adds half-constructed vnodes to the hash, where they * can be found by other code. */ static void vput_final(struct vnode *vp, enum vput_op func) { int error; bool want_unlock; CTR2(KTR_VFS, "%s: vp %p", __func__, vp); VNPASS(vp->v_holdcnt > 0, vp); VI_LOCK(vp); if (func != VRELE) v_decr_devcount(vp); /* * By the time we got here someone else might have transitioned * the count back to > 0. */ if (vp->v_usecount > 0) goto out; /* * If the vnode is doomed vgone already performed inactive processing * (if needed). */ if (VN_IS_DOOMED(vp)) goto out; if (__predict_true(VOP_NEED_INACTIVE(vp) == 0)) goto out; if (vp->v_iflag & VI_DOINGINACT) goto out; /* * Locking operations here will drop the interlock and possibly the * vnode lock, opening a window where the vnode can get doomed all the * while ->v_usecount is 0. Set VI_OWEINACT to let vgone know to * perform inactive. */ vp->v_iflag |= VI_OWEINACT; want_unlock = false; error = 0; switch (func) { case VRELE: switch (VOP_ISLOCKED(vp)) { case LK_EXCLUSIVE: break; case LK_EXCLOTHER: case 0: want_unlock = true; error = vn_lock(vp, LK_EXCLUSIVE | LK_INTERLOCK); VI_LOCK(vp); break; default: /* * The lock has at least one sharer, but we have no way * to conclude whether this is us. Play it safe and * defer processing. */ error = EAGAIN; break; } break; case VPUT: want_unlock = true; if (VOP_ISLOCKED(vp) != LK_EXCLUSIVE) { error = VOP_LOCK(vp, LK_UPGRADE | LK_INTERLOCK | LK_NOWAIT); VI_LOCK(vp); } break; case VUNREF: if (VOP_ISLOCKED(vp) != LK_EXCLUSIVE) { error = VOP_LOCK(vp, LK_TRYUPGRADE | LK_INTERLOCK); VI_LOCK(vp); } break; } if (error == 0) { vinactive(vp); if (want_unlock) VOP_UNLOCK(vp); vdropl(vp); } else { vdefer_inactive(vp); } return; out: if (func == VPUT) VOP_UNLOCK(vp); vdropl(vp); } /* * Decrement ->v_usecount for a vnode. * * Releasing the last use count requires additional processing, see vput_final * above for details. * * Note that releasing use count without the vnode lock requires special casing * for VCHR, see v_decr_devcount for details. * * Comment above each variant denotes lock state on entry and exit. */ static void __noinline vrele_vchr(struct vnode *vp) { if (refcount_release_if_not_last(&vp->v_usecount)) return; VI_LOCK(vp); if (!refcount_release(&vp->v_usecount)) { VI_UNLOCK(vp); return; } v_decr_devcount(vp); VI_UNLOCK(vp); vput_final(vp, VRELE); } /* * in: any * out: same as passed in */ void vrele(struct vnode *vp) { ASSERT_VI_UNLOCKED(vp, __func__); if (__predict_false(vp->v_type == VCHR)) { vrele_vchr(vp); return; } if (!refcount_release(&vp->v_usecount)) return; vput_final(vp, VRELE); } /* * in: locked * out: unlocked */ void vput(struct vnode *vp) { ASSERT_VOP_LOCKED(vp, __func__); ASSERT_VI_UNLOCKED(vp, __func__); if (!refcount_release(&vp->v_usecount)) { VOP_UNLOCK(vp); return; } vput_final(vp, VPUT); } /* * in: locked * out: locked */ void vunref(struct vnode *vp) { ASSERT_VOP_LOCKED(vp, __func__); ASSERT_VI_UNLOCKED(vp, __func__); if (!refcount_release(&vp->v_usecount)) return; vput_final(vp, VUNREF); } void vhold(struct vnode *vp) { struct vdbatch *vd; int old; CTR2(KTR_VFS, "%s: vp %p", __func__, vp); old = atomic_fetchadd_int(&vp->v_holdcnt, 1); VNASSERT(old >= 0, vp, ("%s: wrong hold count %d", __func__, old)); if (old != 0) return; critical_enter(); vd = DPCPU_PTR(vd); vd->freevnodes--; critical_exit(); } void vholdl(struct vnode *vp) { ASSERT_VI_LOCKED(vp, __func__); CTR2(KTR_VFS, "%s: vp %p", __func__, vp); vhold(vp); } void vholdnz(struct vnode *vp) { CTR2(KTR_VFS, "%s: vp %p", __func__, vp); #ifdef INVARIANTS int old = atomic_fetchadd_int(&vp->v_holdcnt, 1); VNASSERT(old > 0, vp, ("%s: wrong hold count %d", __func__, old)); #else atomic_add_int(&vp->v_holdcnt, 1); #endif } static void __noinline vdbatch_process(struct vdbatch *vd) { struct vnode *vp; int i; mtx_assert(&vd->lock, MA_OWNED); MPASS(curthread->td_pinned > 0); MPASS(vd->index == VDBATCH_SIZE); mtx_lock(&vnode_list_mtx); critical_enter(); freevnodes += vd->freevnodes; for (i = 0; i < VDBATCH_SIZE; i++) { vp = vd->tab[i]; TAILQ_REMOVE(&vnode_list, vp, v_vnodelist); TAILQ_INSERT_TAIL(&vnode_list, vp, v_vnodelist); MPASS(vp->v_dbatchcpu != NOCPU); vp->v_dbatchcpu = NOCPU; } mtx_unlock(&vnode_list_mtx); vd->freevnodes = 0; bzero(vd->tab, sizeof(vd->tab)); vd->index = 0; critical_exit(); } static void vdbatch_enqueue(struct vnode *vp) { struct vdbatch *vd; ASSERT_VI_LOCKED(vp, __func__); VNASSERT(!VN_IS_DOOMED(vp), vp, ("%s: deferring requeue of a doomed vnode", __func__)); critical_enter(); vd = DPCPU_PTR(vd); vd->freevnodes++; if (vp->v_dbatchcpu != NOCPU) { VI_UNLOCK(vp); critical_exit(); return; } sched_pin(); critical_exit(); mtx_lock(&vd->lock); MPASS(vd->index < VDBATCH_SIZE); MPASS(vd->tab[vd->index] == NULL); /* * A hack: we depend on being pinned so that we know what to put in * ->v_dbatchcpu. */ vp->v_dbatchcpu = curcpu; vd->tab[vd->index] = vp; vd->index++; VI_UNLOCK(vp); if (vd->index == VDBATCH_SIZE) vdbatch_process(vd); mtx_unlock(&vd->lock); sched_unpin(); } /* * This routine must only be called for vnodes which are about to be * deallocated. Supporting dequeue for arbitrary vndoes would require * validating that the locked batch matches. */ static void vdbatch_dequeue(struct vnode *vp) { struct vdbatch *vd; int i; short cpu; VNASSERT(vp->v_type == VBAD || vp->v_type == VNON, vp, ("%s: called for a used vnode\n", __func__)); cpu = vp->v_dbatchcpu; if (cpu == NOCPU) return; vd = DPCPU_ID_PTR(cpu, vd); mtx_lock(&vd->lock); for (i = 0; i < vd->index; i++) { if (vd->tab[i] != vp) continue; vp->v_dbatchcpu = NOCPU; vd->index--; vd->tab[i] = vd->tab[vd->index]; vd->tab[vd->index] = NULL; break; } mtx_unlock(&vd->lock); /* * Either we dequeued the vnode above or the target CPU beat us to it. */ MPASS(vp->v_dbatchcpu == NOCPU); } /* * Drop the hold count of the vnode. If this is the last reference to * the vnode we place it on the free list unless it has been vgone'd * (marked VIRF_DOOMED) in which case we will free it. * * Because the vnode vm object keeps a hold reference on the vnode if * there is at least one resident non-cached page, the vnode cannot * leave the active list without the page cleanup done. */ static void vdrop_deactivate(struct vnode *vp) { struct mount *mp; ASSERT_VI_LOCKED(vp, __func__); /* * Mark a vnode as free: remove it from its active list * and put it up for recycling on the freelist. */ VNASSERT(!VN_IS_DOOMED(vp), vp, ("vdrop: returning doomed vnode")); VNASSERT(vp->v_op != NULL, vp, ("vdrop: vnode already reclaimed.")); VNASSERT((vp->v_iflag & VI_OWEINACT) == 0, vp, ("vnode with VI_OWEINACT set")); VNASSERT((vp->v_iflag & VI_DEFINACT) == 0, vp, ("vnode with VI_DEFINACT set")); if (vp->v_mflag & VMP_LAZYLIST) { mp = vp->v_mount; mtx_lock(&mp->mnt_listmtx); VNASSERT(vp->v_mflag & VMP_LAZYLIST, vp, ("lost VMP_LAZYLIST")); /* * Don't remove the vnode from the lazy list if another thread * has increased the hold count. It may have re-enqueued the * vnode to the lazy list and is now responsible for its * removal. */ if (vp->v_holdcnt == 0) { vp->v_mflag &= ~VMP_LAZYLIST; TAILQ_REMOVE(&mp->mnt_lazyvnodelist, vp, v_lazylist); mp->mnt_lazyvnodelistsize--; } mtx_unlock(&mp->mnt_listmtx); } vdbatch_enqueue(vp); } void vdrop(struct vnode *vp) { ASSERT_VI_UNLOCKED(vp, __func__); CTR2(KTR_VFS, "%s: vp %p", __func__, vp); if (refcount_release_if_not_last(&vp->v_holdcnt)) return; VI_LOCK(vp); vdropl(vp); } void vdropl(struct vnode *vp) { ASSERT_VI_LOCKED(vp, __func__); CTR2(KTR_VFS, "%s: vp %p", __func__, vp); if (!refcount_release(&vp->v_holdcnt)) { VI_UNLOCK(vp); return; } if (VN_IS_DOOMED(vp)) { freevnode(vp); return; } vdrop_deactivate(vp); } /* * Call VOP_INACTIVE on the vnode and manage the DOINGINACT and OWEINACT * flags. DOINGINACT prevents us from recursing in calls to vinactive. */ static void vinactivef(struct vnode *vp) { struct vm_object *obj; ASSERT_VOP_ELOCKED(vp, "vinactive"); ASSERT_VI_LOCKED(vp, "vinactive"); VNASSERT((vp->v_iflag & VI_DOINGINACT) == 0, vp, ("vinactive: recursed on VI_DOINGINACT")); CTR2(KTR_VFS, "%s: vp %p", __func__, vp); vp->v_iflag |= VI_DOINGINACT; vp->v_iflag &= ~VI_OWEINACT; VI_UNLOCK(vp); /* * Before moving off the active list, we must be sure that any * modified pages are converted into the vnode's dirty * buffers, since these will no longer be checked once the * vnode is on the inactive list. * * The write-out of the dirty pages is asynchronous. At the * point that VOP_INACTIVE() is called, there could still be * pending I/O and dirty pages in the object. */ if ((obj = vp->v_object) != NULL && (vp->v_vflag & VV_NOSYNC) == 0 && vm_object_mightbedirty(obj)) { VM_OBJECT_WLOCK(obj); vm_object_page_clean(obj, 0, 0, 0); VM_OBJECT_WUNLOCK(obj); } VOP_INACTIVE(vp, curthread); VI_LOCK(vp); VNASSERT(vp->v_iflag & VI_DOINGINACT, vp, ("vinactive: lost VI_DOINGINACT")); vp->v_iflag &= ~VI_DOINGINACT; } void vinactive(struct vnode *vp) { ASSERT_VOP_ELOCKED(vp, "vinactive"); ASSERT_VI_LOCKED(vp, "vinactive"); CTR2(KTR_VFS, "%s: vp %p", __func__, vp); if ((vp->v_iflag & VI_OWEINACT) == 0) return; if (vp->v_iflag & VI_DOINGINACT) return; if (vp->v_usecount > 0) { vp->v_iflag &= ~VI_OWEINACT; return; } vinactivef(vp); } /* * Remove any vnodes in the vnode table belonging to mount point mp. * * If FORCECLOSE is not specified, there should not be any active ones, * return error if any are found (nb: this is a user error, not a * system error). If FORCECLOSE is specified, detach any active vnodes * that are found. * * If WRITECLOSE is set, only flush out regular file vnodes open for * writing. * * SKIPSYSTEM causes any vnodes marked VV_SYSTEM to be skipped. * * `rootrefs' specifies the base reference count for the root vnode * of this filesystem. The root vnode is considered busy if its * v_usecount exceeds this value. On a successful return, vflush(, td) * will call vrele() on the root vnode exactly rootrefs times. * If the SKIPSYSTEM or WRITECLOSE flags are specified, rootrefs must * be zero. */ #ifdef DIAGNOSTIC static int busyprt = 0; /* print out busy vnodes */ SYSCTL_INT(_debug, OID_AUTO, busyprt, CTLFLAG_RW, &busyprt, 0, "Print out busy vnodes"); #endif int vflush(struct mount *mp, int rootrefs, int flags, struct thread *td) { struct vnode *vp, *mvp, *rootvp = NULL; struct vattr vattr; int busy = 0, error; CTR4(KTR_VFS, "%s: mp %p with rootrefs %d and flags %d", __func__, mp, rootrefs, flags); if (rootrefs > 0) { KASSERT((flags & (SKIPSYSTEM | WRITECLOSE)) == 0, ("vflush: bad args")); /* * Get the filesystem root vnode. We can vput() it * immediately, since with rootrefs > 0, it won't go away. */ if ((error = VFS_ROOT(mp, LK_EXCLUSIVE, &rootvp)) != 0) { CTR2(KTR_VFS, "%s: vfs_root lookup failed with %d", __func__, error); return (error); } vput(rootvp); } loop: MNT_VNODE_FOREACH_ALL(vp, mp, mvp) { vholdl(vp); error = vn_lock(vp, LK_INTERLOCK | LK_EXCLUSIVE); if (error) { vdrop(vp); MNT_VNODE_FOREACH_ALL_ABORT(mp, mvp); goto loop; } /* * Skip over a vnodes marked VV_SYSTEM. */ if ((flags & SKIPSYSTEM) && (vp->v_vflag & VV_SYSTEM)) { VOP_UNLOCK(vp); vdrop(vp); continue; } /* * If WRITECLOSE is set, flush out unlinked but still open * files (even if open only for reading) and regular file * vnodes open for writing. */ if (flags & WRITECLOSE) { if (vp->v_object != NULL) { VM_OBJECT_WLOCK(vp->v_object); vm_object_page_clean(vp->v_object, 0, 0, 0); VM_OBJECT_WUNLOCK(vp->v_object); } error = VOP_FSYNC(vp, MNT_WAIT, td); if (error != 0) { VOP_UNLOCK(vp); vdrop(vp); MNT_VNODE_FOREACH_ALL_ABORT(mp, mvp); return (error); } error = VOP_GETATTR(vp, &vattr, td->td_ucred); VI_LOCK(vp); if ((vp->v_type == VNON || (error == 0 && vattr.va_nlink > 0)) && (vp->v_writecount <= 0 || vp->v_type != VREG)) { VOP_UNLOCK(vp); vdropl(vp); continue; } } else VI_LOCK(vp); /* * With v_usecount == 0, all we need to do is clear out the * vnode data structures and we are done. * * If FORCECLOSE is set, forcibly close the vnode. */ if (vp->v_usecount == 0 || (flags & FORCECLOSE)) { vgonel(vp); } else { busy++; #ifdef DIAGNOSTIC if (busyprt) vn_printf(vp, "vflush: busy vnode "); #endif } VOP_UNLOCK(vp); vdropl(vp); } if (rootrefs > 0 && (flags & FORCECLOSE) == 0) { /* * If just the root vnode is busy, and if its refcount * is equal to `rootrefs', then go ahead and kill it. */ VI_LOCK(rootvp); KASSERT(busy > 0, ("vflush: not busy")); VNASSERT(rootvp->v_usecount >= rootrefs, rootvp, ("vflush: usecount %d < rootrefs %d", rootvp->v_usecount, rootrefs)); if (busy == 1 && rootvp->v_usecount == rootrefs) { VOP_LOCK(rootvp, LK_EXCLUSIVE|LK_INTERLOCK); vgone(rootvp); VOP_UNLOCK(rootvp); busy = 0; } else VI_UNLOCK(rootvp); } if (busy) { CTR2(KTR_VFS, "%s: failing as %d vnodes are busy", __func__, busy); return (EBUSY); } for (; rootrefs > 0; rootrefs--) vrele(rootvp); return (0); } /* * Recycle an unused vnode to the front of the free list. */ int vrecycle(struct vnode *vp) { int recycled; VI_LOCK(vp); recycled = vrecyclel(vp); VI_UNLOCK(vp); return (recycled); } /* * vrecycle, with the vp interlock held. */ int vrecyclel(struct vnode *vp) { int recycled; ASSERT_VOP_ELOCKED(vp, __func__); ASSERT_VI_LOCKED(vp, __func__); CTR2(KTR_VFS, "%s: vp %p", __func__, vp); recycled = 0; if (vp->v_usecount == 0) { recycled = 1; vgonel(vp); } return (recycled); } /* * Eliminate all activity associated with a vnode * in preparation for reuse. */ void vgone(struct vnode *vp) { VI_LOCK(vp); vgonel(vp); VI_UNLOCK(vp); } static void notify_lowervp_vfs_dummy(struct mount *mp __unused, struct vnode *lowervp __unused) { } /* * Notify upper mounts about reclaimed or unlinked vnode. */ void vfs_notify_upper(struct vnode *vp, int event) { static struct vfsops vgonel_vfsops = { .vfs_reclaim_lowervp = notify_lowervp_vfs_dummy, .vfs_unlink_lowervp = notify_lowervp_vfs_dummy, }; struct mount *mp, *ump, *mmp; mp = vp->v_mount; if (mp == NULL) return; if (TAILQ_EMPTY(&mp->mnt_uppers)) return; mmp = malloc(sizeof(struct mount), M_TEMP, M_WAITOK | M_ZERO); mmp->mnt_op = &vgonel_vfsops; mmp->mnt_kern_flag |= MNTK_MARKER; MNT_ILOCK(mp); mp->mnt_kern_flag |= MNTK_VGONE_UPPER; for (ump = TAILQ_FIRST(&mp->mnt_uppers); ump != NULL;) { if ((ump->mnt_kern_flag & MNTK_MARKER) != 0) { ump = TAILQ_NEXT(ump, mnt_upper_link); continue; } TAILQ_INSERT_AFTER(&mp->mnt_uppers, ump, mmp, mnt_upper_link); MNT_IUNLOCK(mp); switch (event) { case VFS_NOTIFY_UPPER_RECLAIM: VFS_RECLAIM_LOWERVP(ump, vp); break; case VFS_NOTIFY_UPPER_UNLINK: VFS_UNLINK_LOWERVP(ump, vp); break; default: KASSERT(0, ("invalid event %d", event)); break; } MNT_ILOCK(mp); ump = TAILQ_NEXT(mmp, mnt_upper_link); TAILQ_REMOVE(&mp->mnt_uppers, mmp, mnt_upper_link); } free(mmp, M_TEMP); mp->mnt_kern_flag &= ~MNTK_VGONE_UPPER; if ((mp->mnt_kern_flag & MNTK_VGONE_WAITER) != 0) { mp->mnt_kern_flag &= ~MNTK_VGONE_WAITER; wakeup(&mp->mnt_uppers); } MNT_IUNLOCK(mp); } /* * vgone, with the vp interlock held. */ static void vgonel(struct vnode *vp) { struct thread *td; struct mount *mp; vm_object_t object; bool active, oweinact; ASSERT_VOP_ELOCKED(vp, "vgonel"); ASSERT_VI_LOCKED(vp, "vgonel"); VNASSERT(vp->v_holdcnt, vp, ("vgonel: vp %p has no reference.", vp)); CTR2(KTR_VFS, "%s: vp %p", __func__, vp); td = curthread; /* * Don't vgonel if we're already doomed. */ if (vp->v_irflag & VIRF_DOOMED) return; vunlazy_gone(vp); vp->v_irflag |= VIRF_DOOMED; /* * Check to see if the vnode is in use. If so, we have to call * VOP_CLOSE() and VOP_INACTIVE(). */ active = vp->v_usecount > 0; oweinact = (vp->v_iflag & VI_OWEINACT) != 0; /* * If we need to do inactive VI_OWEINACT will be set. */ if (vp->v_iflag & VI_DEFINACT) { VNASSERT(vp->v_holdcnt > 1, vp, ("lost hold count")); vp->v_iflag &= ~VI_DEFINACT; vdropl(vp); } else { VNASSERT(vp->v_holdcnt > 0, vp, ("vnode without hold count")); VI_UNLOCK(vp); } vfs_notify_upper(vp, VFS_NOTIFY_UPPER_RECLAIM); /* * If purging an active vnode, it must be closed and * deactivated before being reclaimed. */ if (active) VOP_CLOSE(vp, FNONBLOCK, NOCRED, td); if (oweinact || active) { VI_LOCK(vp); vinactivef(vp); VI_UNLOCK(vp); } if (vp->v_type == VSOCK) vfs_unp_reclaim(vp); /* * Clean out any buffers associated with the vnode. * If the flush fails, just toss the buffers. */ mp = NULL; if (!TAILQ_EMPTY(&vp->v_bufobj.bo_dirty.bv_hd)) (void) vn_start_secondary_write(vp, &mp, V_WAIT); if (vinvalbuf(vp, V_SAVE, 0, 0) != 0) { while (vinvalbuf(vp, 0, 0, 0) != 0) ; } BO_LOCK(&vp->v_bufobj); KASSERT(TAILQ_EMPTY(&vp->v_bufobj.bo_dirty.bv_hd) && vp->v_bufobj.bo_dirty.bv_cnt == 0 && TAILQ_EMPTY(&vp->v_bufobj.bo_clean.bv_hd) && vp->v_bufobj.bo_clean.bv_cnt == 0, ("vp %p bufobj not invalidated", vp)); /* * For VMIO bufobj, BO_DEAD is set later, or in * vm_object_terminate() after the object's page queue is * flushed. */ object = vp->v_bufobj.bo_object; if (object == NULL) vp->v_bufobj.bo_flag |= BO_DEAD; BO_UNLOCK(&vp->v_bufobj); /* * Handle the VM part. Tmpfs handles v_object on its own (the * OBJT_VNODE check). Nullfs or other bypassing filesystems * should not touch the object borrowed from the lower vnode * (the handle check). */ if (object != NULL && object->type == OBJT_VNODE && object->handle == vp) vnode_destroy_vobject(vp); /* * Reclaim the vnode. */ if (VOP_RECLAIM(vp, td)) panic("vgone: cannot reclaim"); if (mp != NULL) vn_finished_secondary_write(mp); VNASSERT(vp->v_object == NULL, vp, ("vop_reclaim left v_object vp=%p", vp)); /* * Clear the advisory locks and wake up waiting threads. */ (void)VOP_ADVLOCKPURGE(vp); vp->v_lockf = NULL; /* * Delete from old mount point vnode list. */ delmntque(vp); cache_purge(vp); /* * Done with purge, reset to the standard lock and invalidate * the vnode. */ VI_LOCK(vp); vp->v_vnlock = &vp->v_lock; vp->v_op = &dead_vnodeops; vp->v_type = VBAD; } /* * Calculate the total number of references to a special device. */ int vcount(struct vnode *vp) { int count; dev_lock(); count = vp->v_rdev->si_usecount; dev_unlock(); return (count); } /* * Print out a description of a vnode. */ static char *typename[] = {"VNON", "VREG", "VDIR", "VBLK", "VCHR", "VLNK", "VSOCK", "VFIFO", "VBAD", "VMARKER"}; void vn_printf(struct vnode *vp, const char *fmt, ...) { va_list ap; char buf[256], buf2[16]; u_long flags; va_start(ap, fmt); vprintf(fmt, ap); va_end(ap); printf("%p: ", (void *)vp); printf("type %s\n", typename[vp->v_type]); printf(" usecount %d, writecount %d, refcount %d", vp->v_usecount, vp->v_writecount, vp->v_holdcnt); switch (vp->v_type) { case VDIR: printf(" mountedhere %p\n", vp->v_mountedhere); break; case VCHR: printf(" rdev %p\n", vp->v_rdev); break; case VSOCK: printf(" socket %p\n", vp->v_unpcb); break; case VFIFO: printf(" fifoinfo %p\n", vp->v_fifoinfo); break; default: printf("\n"); break; } buf[0] = '\0'; buf[1] = '\0'; if (vp->v_irflag & VIRF_DOOMED) strlcat(buf, "|VIRF_DOOMED", sizeof(buf)); flags = vp->v_irflag & ~(VIRF_DOOMED); if (flags != 0) { snprintf(buf2, sizeof(buf2), "|VIRF(0x%lx)", flags); strlcat(buf, buf2, sizeof(buf)); } if (vp->v_vflag & VV_ROOT) strlcat(buf, "|VV_ROOT", sizeof(buf)); if (vp->v_vflag & VV_ISTTY) strlcat(buf, "|VV_ISTTY", sizeof(buf)); if (vp->v_vflag & VV_NOSYNC) strlcat(buf, "|VV_NOSYNC", sizeof(buf)); if (vp->v_vflag & VV_ETERNALDEV) strlcat(buf, "|VV_ETERNALDEV", sizeof(buf)); if (vp->v_vflag & VV_CACHEDLABEL) strlcat(buf, "|VV_CACHEDLABEL", sizeof(buf)); if (vp->v_vflag & VV_VMSIZEVNLOCK) strlcat(buf, "|VV_VMSIZEVNLOCK", sizeof(buf)); if (vp->v_vflag & VV_COPYONWRITE) strlcat(buf, "|VV_COPYONWRITE", sizeof(buf)); if (vp->v_vflag & VV_SYSTEM) strlcat(buf, "|VV_SYSTEM", sizeof(buf)); if (vp->v_vflag & VV_PROCDEP) strlcat(buf, "|VV_PROCDEP", sizeof(buf)); if (vp->v_vflag & VV_NOKNOTE) strlcat(buf, "|VV_NOKNOTE", sizeof(buf)); if (vp->v_vflag & VV_DELETED) strlcat(buf, "|VV_DELETED", sizeof(buf)); if (vp->v_vflag & VV_MD) strlcat(buf, "|VV_MD", sizeof(buf)); if (vp->v_vflag & VV_FORCEINSMQ) strlcat(buf, "|VV_FORCEINSMQ", sizeof(buf)); if (vp->v_vflag & VV_READLINK) strlcat(buf, "|VV_READLINK", sizeof(buf)); flags = vp->v_vflag & ~(VV_ROOT | VV_ISTTY | VV_NOSYNC | VV_ETERNALDEV | VV_CACHEDLABEL | VV_COPYONWRITE | VV_SYSTEM | VV_PROCDEP | VV_NOKNOTE | VV_DELETED | VV_MD | VV_FORCEINSMQ); if (flags != 0) { snprintf(buf2, sizeof(buf2), "|VV(0x%lx)", flags); strlcat(buf, buf2, sizeof(buf)); } if (vp->v_iflag & VI_TEXT_REF) strlcat(buf, "|VI_TEXT_REF", sizeof(buf)); if (vp->v_iflag & VI_MOUNT) strlcat(buf, "|VI_MOUNT", sizeof(buf)); if (vp->v_iflag & VI_DOINGINACT) strlcat(buf, "|VI_DOINGINACT", sizeof(buf)); if (vp->v_iflag & VI_OWEINACT) strlcat(buf, "|VI_OWEINACT", sizeof(buf)); if (vp->v_iflag & VI_DEFINACT) strlcat(buf, "|VI_DEFINACT", sizeof(buf)); flags = vp->v_iflag & ~(VI_TEXT_REF | VI_MOUNT | VI_DOINGINACT | VI_OWEINACT | VI_DEFINACT); if (flags != 0) { snprintf(buf2, sizeof(buf2), "|VI(0x%lx)", flags); strlcat(buf, buf2, sizeof(buf)); } if (vp->v_mflag & VMP_LAZYLIST) strlcat(buf, "|VMP_LAZYLIST", sizeof(buf)); flags = vp->v_mflag & ~(VMP_LAZYLIST); if (flags != 0) { snprintf(buf2, sizeof(buf2), "|VMP(0x%lx)", flags); strlcat(buf, buf2, sizeof(buf)); } printf(" flags (%s)\n", buf + 1); if (mtx_owned(VI_MTX(vp))) printf(" VI_LOCKed"); if (vp->v_object != NULL) printf(" v_object %p ref %d pages %d " "cleanbuf %d dirtybuf %d\n", vp->v_object, vp->v_object->ref_count, vp->v_object->resident_page_count, vp->v_bufobj.bo_clean.bv_cnt, vp->v_bufobj.bo_dirty.bv_cnt); printf(" "); lockmgr_printinfo(vp->v_vnlock); if (vp->v_data != NULL) VOP_PRINT(vp); } #ifdef DDB /* * List all of the locked vnodes in the system. * Called when debugging the kernel. */ DB_SHOW_COMMAND(lockedvnods, lockedvnodes) { struct mount *mp; struct vnode *vp; /* * Note: because this is DDB, we can't obey the locking semantics * for these structures, which means we could catch an inconsistent * state and dereference a nasty pointer. Not much to be done * about that. */ db_printf("Locked vnodes\n"); TAILQ_FOREACH(mp, &mountlist, mnt_list) { TAILQ_FOREACH(vp, &mp->mnt_nvnodelist, v_nmntvnodes) { if (vp->v_type != VMARKER && VOP_ISLOCKED(vp)) vn_printf(vp, "vnode "); } } } /* * Show details about the given vnode. */ DB_SHOW_COMMAND(vnode, db_show_vnode) { struct vnode *vp; if (!have_addr) return; vp = (struct vnode *)addr; vn_printf(vp, "vnode "); } /* * Show details about the given mount point. */ DB_SHOW_COMMAND(mount, db_show_mount) { struct mount *mp; struct vfsopt *opt; struct statfs *sp; struct vnode *vp; char buf[512]; uint64_t mflags; u_int flags; if (!have_addr) { /* No address given, print short info about all mount points. */ TAILQ_FOREACH(mp, &mountlist, mnt_list) { db_printf("%p %s on %s (%s)\n", mp, mp->mnt_stat.f_mntfromname, mp->mnt_stat.f_mntonname, mp->mnt_stat.f_fstypename); if (db_pager_quit) break; } db_printf("\nMore info: show mount \n"); return; } mp = (struct mount *)addr; db_printf("%p %s on %s (%s)\n", mp, mp->mnt_stat.f_mntfromname, mp->mnt_stat.f_mntonname, mp->mnt_stat.f_fstypename); buf[0] = '\0'; mflags = mp->mnt_flag; #define MNT_FLAG(flag) do { \ if (mflags & (flag)) { \ if (buf[0] != '\0') \ strlcat(buf, ", ", sizeof(buf)); \ strlcat(buf, (#flag) + 4, sizeof(buf)); \ mflags &= ~(flag); \ } \ } while (0) MNT_FLAG(MNT_RDONLY); MNT_FLAG(MNT_SYNCHRONOUS); MNT_FLAG(MNT_NOEXEC); MNT_FLAG(MNT_NOSUID); MNT_FLAG(MNT_NFS4ACLS); MNT_FLAG(MNT_UNION); MNT_FLAG(MNT_ASYNC); MNT_FLAG(MNT_SUIDDIR); MNT_FLAG(MNT_SOFTDEP); MNT_FLAG(MNT_NOSYMFOLLOW); MNT_FLAG(MNT_GJOURNAL); MNT_FLAG(MNT_MULTILABEL); MNT_FLAG(MNT_ACLS); MNT_FLAG(MNT_NOATIME); MNT_FLAG(MNT_NOCLUSTERR); MNT_FLAG(MNT_NOCLUSTERW); MNT_FLAG(MNT_SUJ); MNT_FLAG(MNT_EXRDONLY); MNT_FLAG(MNT_EXPORTED); MNT_FLAG(MNT_DEFEXPORTED); MNT_FLAG(MNT_EXPORTANON); MNT_FLAG(MNT_EXKERB); MNT_FLAG(MNT_EXPUBLIC); MNT_FLAG(MNT_LOCAL); MNT_FLAG(MNT_QUOTA); MNT_FLAG(MNT_ROOTFS); MNT_FLAG(MNT_USER); MNT_FLAG(MNT_IGNORE); MNT_FLAG(MNT_UPDATE); MNT_FLAG(MNT_DELEXPORT); MNT_FLAG(MNT_RELOAD); MNT_FLAG(MNT_FORCE); MNT_FLAG(MNT_SNAPSHOT); MNT_FLAG(MNT_BYFSID); #undef MNT_FLAG if (mflags != 0) { if (buf[0] != '\0') strlcat(buf, ", ", sizeof(buf)); snprintf(buf + strlen(buf), sizeof(buf) - strlen(buf), "0x%016jx", mflags); } db_printf(" mnt_flag = %s\n", buf); buf[0] = '\0'; flags = mp->mnt_kern_flag; #define MNT_KERN_FLAG(flag) do { \ if (flags & (flag)) { \ if (buf[0] != '\0') \ strlcat(buf, ", ", sizeof(buf)); \ strlcat(buf, (#flag) + 5, sizeof(buf)); \ flags &= ~(flag); \ } \ } while (0) MNT_KERN_FLAG(MNTK_UNMOUNTF); MNT_KERN_FLAG(MNTK_ASYNC); MNT_KERN_FLAG(MNTK_SOFTDEP); MNT_KERN_FLAG(MNTK_DRAINING); MNT_KERN_FLAG(MNTK_REFEXPIRE); MNT_KERN_FLAG(MNTK_EXTENDED_SHARED); MNT_KERN_FLAG(MNTK_SHARED_WRITES); MNT_KERN_FLAG(MNTK_NO_IOPF); MNT_KERN_FLAG(MNTK_VGONE_UPPER); MNT_KERN_FLAG(MNTK_VGONE_WAITER); MNT_KERN_FLAG(MNTK_LOOKUP_EXCL_DOTDOT); MNT_KERN_FLAG(MNTK_MARKER); MNT_KERN_FLAG(MNTK_USES_BCACHE); MNT_KERN_FLAG(MNTK_NOASYNC); MNT_KERN_FLAG(MNTK_UNMOUNT); MNT_KERN_FLAG(MNTK_MWAIT); MNT_KERN_FLAG(MNTK_SUSPEND); MNT_KERN_FLAG(MNTK_SUSPEND2); MNT_KERN_FLAG(MNTK_SUSPENDED); MNT_KERN_FLAG(MNTK_LOOKUP_SHARED); MNT_KERN_FLAG(MNTK_NOKNOTE); #undef MNT_KERN_FLAG if (flags != 0) { if (buf[0] != '\0') strlcat(buf, ", ", sizeof(buf)); snprintf(buf + strlen(buf), sizeof(buf) - strlen(buf), "0x%08x", flags); } db_printf(" mnt_kern_flag = %s\n", buf); db_printf(" mnt_opt = "); opt = TAILQ_FIRST(mp->mnt_opt); if (opt != NULL) { db_printf("%s", opt->name); opt = TAILQ_NEXT(opt, link); while (opt != NULL) { db_printf(", %s", opt->name); opt = TAILQ_NEXT(opt, link); } } db_printf("\n"); sp = &mp->mnt_stat; db_printf(" mnt_stat = { version=%u type=%u flags=0x%016jx " "bsize=%ju iosize=%ju blocks=%ju bfree=%ju bavail=%jd files=%ju " "ffree=%jd syncwrites=%ju asyncwrites=%ju syncreads=%ju " "asyncreads=%ju namemax=%u owner=%u fsid=[%d, %d] }\n", (u_int)sp->f_version, (u_int)sp->f_type, (uintmax_t)sp->f_flags, (uintmax_t)sp->f_bsize, (uintmax_t)sp->f_iosize, (uintmax_t)sp->f_blocks, (uintmax_t)sp->f_bfree, (intmax_t)sp->f_bavail, (uintmax_t)sp->f_files, (intmax_t)sp->f_ffree, (uintmax_t)sp->f_syncwrites, (uintmax_t)sp->f_asyncwrites, (uintmax_t)sp->f_syncreads, (uintmax_t)sp->f_asyncreads, (u_int)sp->f_namemax, (u_int)sp->f_owner, (int)sp->f_fsid.val[0], (int)sp->f_fsid.val[1]); db_printf(" mnt_cred = { uid=%u ruid=%u", (u_int)mp->mnt_cred->cr_uid, (u_int)mp->mnt_cred->cr_ruid); if (jailed(mp->mnt_cred)) db_printf(", jail=%d", mp->mnt_cred->cr_prison->pr_id); db_printf(" }\n"); db_printf(" mnt_ref = %d (with %d in the struct)\n", vfs_mount_fetch_counter(mp, MNT_COUNT_REF), mp->mnt_ref); db_printf(" mnt_gen = %d\n", mp->mnt_gen); db_printf(" mnt_nvnodelistsize = %d\n", mp->mnt_nvnodelistsize); db_printf(" mnt_lazyvnodelistsize = %d\n", mp->mnt_lazyvnodelistsize); db_printf(" mnt_writeopcount = %d (with %d in the struct)\n", vfs_mount_fetch_counter(mp, MNT_COUNT_WRITEOPCOUNT), mp->mnt_writeopcount); db_printf(" mnt_maxsymlinklen = %d\n", mp->mnt_maxsymlinklen); db_printf(" mnt_iosize_max = %d\n", mp->mnt_iosize_max); db_printf(" mnt_hashseed = %u\n", mp->mnt_hashseed); db_printf(" mnt_lockref = %d (with %d in the struct)\n", vfs_mount_fetch_counter(mp, MNT_COUNT_LOCKREF), mp->mnt_lockref); db_printf(" mnt_secondary_writes = %d\n", mp->mnt_secondary_writes); db_printf(" mnt_secondary_accwrites = %d\n", mp->mnt_secondary_accwrites); db_printf(" mnt_gjprovider = %s\n", mp->mnt_gjprovider != NULL ? mp->mnt_gjprovider : "NULL"); db_printf(" mnt_vfs_ops = %d\n", mp->mnt_vfs_ops); db_printf("\n\nList of active vnodes\n"); TAILQ_FOREACH(vp, &mp->mnt_nvnodelist, v_nmntvnodes) { if (vp->v_type != VMARKER && vp->v_holdcnt > 0) { vn_printf(vp, "vnode "); if (db_pager_quit) break; } } db_printf("\n\nList of inactive vnodes\n"); TAILQ_FOREACH(vp, &mp->mnt_nvnodelist, v_nmntvnodes) { if (vp->v_type != VMARKER && vp->v_holdcnt == 0) { vn_printf(vp, "vnode "); if (db_pager_quit) break; } } } #endif /* DDB */ /* * Fill in a struct xvfsconf based on a struct vfsconf. */ static int vfsconf2x(struct sysctl_req *req, struct vfsconf *vfsp) { struct xvfsconf xvfsp; bzero(&xvfsp, sizeof(xvfsp)); strcpy(xvfsp.vfc_name, vfsp->vfc_name); xvfsp.vfc_typenum = vfsp->vfc_typenum; xvfsp.vfc_refcount = vfsp->vfc_refcount; xvfsp.vfc_flags = vfsp->vfc_flags; /* * These are unused in userland, we keep them * to not break binary compatibility. */ xvfsp.vfc_vfsops = NULL; xvfsp.vfc_next = NULL; return (SYSCTL_OUT(req, &xvfsp, sizeof(xvfsp))); } #ifdef COMPAT_FREEBSD32 struct xvfsconf32 { uint32_t vfc_vfsops; char vfc_name[MFSNAMELEN]; int32_t vfc_typenum; int32_t vfc_refcount; int32_t vfc_flags; uint32_t vfc_next; }; static int vfsconf2x32(struct sysctl_req *req, struct vfsconf *vfsp) { struct xvfsconf32 xvfsp; bzero(&xvfsp, sizeof(xvfsp)); strcpy(xvfsp.vfc_name, vfsp->vfc_name); xvfsp.vfc_typenum = vfsp->vfc_typenum; xvfsp.vfc_refcount = vfsp->vfc_refcount; xvfsp.vfc_flags = vfsp->vfc_flags; return (SYSCTL_OUT(req, &xvfsp, sizeof(xvfsp))); } #endif /* * Top level filesystem related information gathering. */ static int sysctl_vfs_conflist(SYSCTL_HANDLER_ARGS) { struct vfsconf *vfsp; int error; error = 0; vfsconf_slock(); TAILQ_FOREACH(vfsp, &vfsconf, vfc_list) { #ifdef COMPAT_FREEBSD32 if (req->flags & SCTL_MASK32) error = vfsconf2x32(req, vfsp); else #endif error = vfsconf2x(req, vfsp); if (error) break; } vfsconf_sunlock(); return (error); } SYSCTL_PROC(_vfs, OID_AUTO, conflist, CTLTYPE_OPAQUE | CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, 0, sysctl_vfs_conflist, "S,xvfsconf", "List of all configured filesystems"); #ifndef BURN_BRIDGES static int sysctl_ovfs_conf(SYSCTL_HANDLER_ARGS); static int vfs_sysctl(SYSCTL_HANDLER_ARGS) { int *name = (int *)arg1 - 1; /* XXX */ u_int namelen = arg2 + 1; /* XXX */ struct vfsconf *vfsp; log(LOG_WARNING, "userland calling deprecated sysctl, " "please rebuild world\n"); #if 1 || defined(COMPAT_PRELITE2) /* Resolve ambiguity between VFS_VFSCONF and VFS_GENERIC. */ if (namelen == 1) return (sysctl_ovfs_conf(oidp, arg1, arg2, req)); #endif switch (name[1]) { case VFS_MAXTYPENUM: if (namelen != 2) return (ENOTDIR); return (SYSCTL_OUT(req, &maxvfsconf, sizeof(int))); case VFS_CONF: if (namelen != 3) return (ENOTDIR); /* overloaded */ vfsconf_slock(); TAILQ_FOREACH(vfsp, &vfsconf, vfc_list) { if (vfsp->vfc_typenum == name[2]) break; } vfsconf_sunlock(); if (vfsp == NULL) return (EOPNOTSUPP); #ifdef COMPAT_FREEBSD32 if (req->flags & SCTL_MASK32) return (vfsconf2x32(req, vfsp)); else #endif return (vfsconf2x(req, vfsp)); } return (EOPNOTSUPP); } static SYSCTL_NODE(_vfs, VFS_GENERIC, generic, CTLFLAG_RD | CTLFLAG_SKIP | CTLFLAG_MPSAFE, vfs_sysctl, "Generic filesystem"); #if 1 || defined(COMPAT_PRELITE2) static int sysctl_ovfs_conf(SYSCTL_HANDLER_ARGS) { int error; struct vfsconf *vfsp; struct ovfsconf ovfs; vfsconf_slock(); TAILQ_FOREACH(vfsp, &vfsconf, vfc_list) { bzero(&ovfs, sizeof(ovfs)); ovfs.vfc_vfsops = vfsp->vfc_vfsops; /* XXX used as flag */ strcpy(ovfs.vfc_name, vfsp->vfc_name); ovfs.vfc_index = vfsp->vfc_typenum; ovfs.vfc_refcount = vfsp->vfc_refcount; ovfs.vfc_flags = vfsp->vfc_flags; error = SYSCTL_OUT(req, &ovfs, sizeof ovfs); if (error != 0) { vfsconf_sunlock(); return (error); } } vfsconf_sunlock(); return (0); } #endif /* 1 || COMPAT_PRELITE2 */ #endif /* !BURN_BRIDGES */ #define KINFO_VNODESLOP 10 #ifdef notyet /* * Dump vnode list (via sysctl). */ /* ARGSUSED */ static int sysctl_vnode(SYSCTL_HANDLER_ARGS) { struct xvnode *xvn; struct mount *mp; struct vnode *vp; int error, len, n; /* * Stale numvnodes access is not fatal here. */ req->lock = 0; len = (numvnodes + KINFO_VNODESLOP) * sizeof *xvn; if (!req->oldptr) /* Make an estimate */ return (SYSCTL_OUT(req, 0, len)); error = sysctl_wire_old_buffer(req, 0); if (error != 0) return (error); xvn = malloc(len, M_TEMP, M_ZERO | M_WAITOK); n = 0; mtx_lock(&mountlist_mtx); TAILQ_FOREACH(mp, &mountlist, mnt_list) { if (vfs_busy(mp, MBF_NOWAIT | MBF_MNTLSTLOCK)) continue; MNT_ILOCK(mp); TAILQ_FOREACH(vp, &mp->mnt_nvnodelist, v_nmntvnodes) { if (n == len) break; vref(vp); xvn[n].xv_size = sizeof *xvn; xvn[n].xv_vnode = vp; xvn[n].xv_id = 0; /* XXX compat */ #define XV_COPY(field) xvn[n].xv_##field = vp->v_##field XV_COPY(usecount); XV_COPY(writecount); XV_COPY(holdcnt); XV_COPY(mount); XV_COPY(numoutput); XV_COPY(type); #undef XV_COPY xvn[n].xv_flag = vp->v_vflag; switch (vp->v_type) { case VREG: case VDIR: case VLNK: break; case VBLK: case VCHR: if (vp->v_rdev == NULL) { vrele(vp); continue; } xvn[n].xv_dev = dev2udev(vp->v_rdev); break; case VSOCK: xvn[n].xv_socket = vp->v_socket; break; case VFIFO: xvn[n].xv_fifo = vp->v_fifoinfo; break; case VNON: case VBAD: default: /* shouldn't happen? */ vrele(vp); continue; } vrele(vp); ++n; } MNT_IUNLOCK(mp); mtx_lock(&mountlist_mtx); vfs_unbusy(mp); if (n == len) break; } mtx_unlock(&mountlist_mtx); error = SYSCTL_OUT(req, xvn, n * sizeof *xvn); free(xvn, M_TEMP); return (error); } SYSCTL_PROC(_kern, KERN_VNODE, vnode, CTLTYPE_OPAQUE | CTLFLAG_RD | CTLFLAG_MPSAFE, 0, 0, sysctl_vnode, "S,xvnode", ""); #endif static void unmount_or_warn(struct mount *mp) { int error; error = dounmount(mp, MNT_FORCE, curthread); if (error != 0) { printf("unmount of %s failed (", mp->mnt_stat.f_mntonname); if (error == EBUSY) printf("BUSY)\n"); else printf("%d)\n", error); } } /* * Unmount all filesystems. The list is traversed in reverse order * of mounting to avoid dependencies. */ void vfs_unmountall(void) { struct mount *mp, *tmp; CTR1(KTR_VFS, "%s: unmounting all filesystems", __func__); /* * Since this only runs when rebooting, it is not interlocked. */ TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, tmp) { vfs_ref(mp); /* * Forcibly unmounting "/dev" before "/" would prevent clean * unmount of the latter. */ if (mp == rootdevmp) continue; unmount_or_warn(mp); } if (rootdevmp != NULL) unmount_or_warn(rootdevmp); } static void vfs_deferred_inactive(struct vnode *vp, int lkflags) { ASSERT_VI_LOCKED(vp, __func__); VNASSERT((vp->v_iflag & VI_DEFINACT) == 0, vp, ("VI_DEFINACT still set")); if ((vp->v_iflag & VI_OWEINACT) == 0) { vdropl(vp); return; } if (vn_lock(vp, lkflags) == 0) { VI_LOCK(vp); vinactive(vp); VOP_UNLOCK(vp); vdropl(vp); return; } vdefer_inactive_unlocked(vp); } static int vfs_periodic_inactive_filter(struct vnode *vp, void *arg) { return (vp->v_iflag & VI_DEFINACT); } static void __noinline vfs_periodic_inactive(struct mount *mp, int flags) { struct vnode *vp, *mvp; int lkflags; lkflags = LK_EXCLUSIVE | LK_INTERLOCK; if (flags != MNT_WAIT) lkflags |= LK_NOWAIT; MNT_VNODE_FOREACH_LAZY(vp, mp, mvp, vfs_periodic_inactive_filter, NULL) { if ((vp->v_iflag & VI_DEFINACT) == 0) { VI_UNLOCK(vp); continue; } vp->v_iflag &= ~VI_DEFINACT; vfs_deferred_inactive(vp, lkflags); } } static inline bool vfs_want_msync(struct vnode *vp) { struct vm_object *obj; /* * This test may be performed without any locks held. * We rely on vm_object's type stability. */ if (vp->v_vflag & VV_NOSYNC) return (false); obj = vp->v_object; return (obj != NULL && vm_object_mightbedirty(obj)); } static int vfs_periodic_msync_inactive_filter(struct vnode *vp, void *arg __unused) { if (vp->v_vflag & VV_NOSYNC) return (false); if (vp->v_iflag & VI_DEFINACT) return (true); return (vfs_want_msync(vp)); } static void __noinline vfs_periodic_msync_inactive(struct mount *mp, int flags) { struct vnode *vp, *mvp; struct vm_object *obj; struct thread *td; int lkflags, objflags; bool seen_defer; td = curthread; lkflags = LK_EXCLUSIVE | LK_INTERLOCK; if (flags != MNT_WAIT) { lkflags |= LK_NOWAIT; objflags = OBJPC_NOSYNC; } else { objflags = OBJPC_SYNC; } MNT_VNODE_FOREACH_LAZY(vp, mp, mvp, vfs_periodic_msync_inactive_filter, NULL) { seen_defer = false; if (vp->v_iflag & VI_DEFINACT) { vp->v_iflag &= ~VI_DEFINACT; seen_defer = true; } if (!vfs_want_msync(vp)) { if (seen_defer) vfs_deferred_inactive(vp, lkflags); else VI_UNLOCK(vp); continue; } if (vget(vp, lkflags, td) == 0) { obj = vp->v_object; if (obj != NULL && (vp->v_vflag & VV_NOSYNC) == 0) { VM_OBJECT_WLOCK(obj); vm_object_page_clean(obj, 0, 0, objflags); VM_OBJECT_WUNLOCK(obj); } vput(vp); if (seen_defer) vdrop(vp); } else { if (seen_defer) vdefer_inactive_unlocked(vp); } } } void vfs_periodic(struct mount *mp, int flags) { CTR2(KTR_VFS, "%s: mp %p", __func__, mp); if ((mp->mnt_kern_flag & MNTK_NOMSYNC) != 0) vfs_periodic_inactive(mp, flags); else vfs_periodic_msync_inactive(mp, flags); } static void destroy_vpollinfo_free(struct vpollinfo *vi) { knlist_destroy(&vi->vpi_selinfo.si_note); mtx_destroy(&vi->vpi_lock); uma_zfree(vnodepoll_zone, vi); } static void destroy_vpollinfo(struct vpollinfo *vi) { knlist_clear(&vi->vpi_selinfo.si_note, 1); seldrain(&vi->vpi_selinfo); destroy_vpollinfo_free(vi); } /* * Initialize per-vnode helper structure to hold poll-related state. */ void v_addpollinfo(struct vnode *vp) { struct vpollinfo *vi; if (vp->v_pollinfo != NULL) return; vi = uma_zalloc(vnodepoll_zone, M_WAITOK | M_ZERO); mtx_init(&vi->vpi_lock, "vnode pollinfo", NULL, MTX_DEF); knlist_init(&vi->vpi_selinfo.si_note, vp, vfs_knllock, vfs_knlunlock, vfs_knl_assert_locked, vfs_knl_assert_unlocked); VI_LOCK(vp); if (vp->v_pollinfo != NULL) { VI_UNLOCK(vp); destroy_vpollinfo_free(vi); return; } vp->v_pollinfo = vi; VI_UNLOCK(vp); } /* * Record a process's interest in events which might happen to * a vnode. Because poll uses the historic select-style interface * internally, this routine serves as both the ``check for any * pending events'' and the ``record my interest in future events'' * functions. (These are done together, while the lock is held, * to avoid race conditions.) */ int vn_pollrecord(struct vnode *vp, struct thread *td, int events) { v_addpollinfo(vp); mtx_lock(&vp->v_pollinfo->vpi_lock); if (vp->v_pollinfo->vpi_revents & events) { /* * This leaves events we are not interested * in available for the other process which * which presumably had requested them * (otherwise they would never have been * recorded). */ events &= vp->v_pollinfo->vpi_revents; vp->v_pollinfo->vpi_revents &= ~events; mtx_unlock(&vp->v_pollinfo->vpi_lock); return (events); } vp->v_pollinfo->vpi_events |= events; selrecord(td, &vp->v_pollinfo->vpi_selinfo); mtx_unlock(&vp->v_pollinfo->vpi_lock); return (0); } /* * Routine to create and manage a filesystem syncer vnode. */ #define sync_close ((int (*)(struct vop_close_args *))nullop) static int sync_fsync(struct vop_fsync_args *); static int sync_inactive(struct vop_inactive_args *); static int sync_reclaim(struct vop_reclaim_args *); static struct vop_vector sync_vnodeops = { .vop_bypass = VOP_EOPNOTSUPP, .vop_close = sync_close, /* close */ .vop_fsync = sync_fsync, /* fsync */ .vop_inactive = sync_inactive, /* inactive */ .vop_need_inactive = vop_stdneed_inactive, /* need_inactive */ .vop_reclaim = sync_reclaim, /* reclaim */ .vop_lock1 = vop_stdlock, /* lock */ .vop_unlock = vop_stdunlock, /* unlock */ .vop_islocked = vop_stdislocked, /* islocked */ }; VFS_VOP_VECTOR_REGISTER(sync_vnodeops); /* * Create a new filesystem syncer vnode for the specified mount point. */ void vfs_allocate_syncvnode(struct mount *mp) { struct vnode *vp; struct bufobj *bo; static long start, incr, next; int error; /* Allocate a new vnode */ error = getnewvnode("syncer", mp, &sync_vnodeops, &vp); if (error != 0) panic("vfs_allocate_syncvnode: getnewvnode() failed"); vp->v_type = VNON; vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); vp->v_vflag |= VV_FORCEINSMQ; error = insmntque(vp, mp); if (error != 0) panic("vfs_allocate_syncvnode: insmntque() failed"); vp->v_vflag &= ~VV_FORCEINSMQ; VOP_UNLOCK(vp); /* * Place the vnode onto the syncer worklist. We attempt to * scatter them about on the list so that they will go off * at evenly distributed times even if all the filesystems * are mounted at once. */ next += incr; if (next == 0 || next > syncer_maxdelay) { start /= 2; incr /= 2; if (start == 0) { start = syncer_maxdelay / 2; incr = syncer_maxdelay; } next = start; } bo = &vp->v_bufobj; BO_LOCK(bo); vn_syncer_add_to_worklist(bo, syncdelay > 0 ? next % syncdelay : 0); /* XXX - vn_syncer_add_to_worklist() also grabs and drops sync_mtx. */ mtx_lock(&sync_mtx); sync_vnode_count++; if (mp->mnt_syncer == NULL) { mp->mnt_syncer = vp; vp = NULL; } mtx_unlock(&sync_mtx); BO_UNLOCK(bo); if (vp != NULL) { vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); vgone(vp); vput(vp); } } void vfs_deallocate_syncvnode(struct mount *mp) { struct vnode *vp; mtx_lock(&sync_mtx); vp = mp->mnt_syncer; if (vp != NULL) mp->mnt_syncer = NULL; mtx_unlock(&sync_mtx); if (vp != NULL) vrele(vp); } /* * Do a lazy sync of the filesystem. */ static int sync_fsync(struct vop_fsync_args *ap) { struct vnode *syncvp = ap->a_vp; struct mount *mp = syncvp->v_mount; int error, save; struct bufobj *bo; /* * We only need to do something if this is a lazy evaluation. */ if (ap->a_waitfor != MNT_LAZY) return (0); /* * Move ourselves to the back of the sync list. */ bo = &syncvp->v_bufobj; BO_LOCK(bo); vn_syncer_add_to_worklist(bo, syncdelay); BO_UNLOCK(bo); /* * Walk the list of vnodes pushing all that are dirty and * not already on the sync list. */ if (vfs_busy(mp, MBF_NOWAIT) != 0) return (0); if (vn_start_write(NULL, &mp, V_NOWAIT) != 0) { vfs_unbusy(mp); return (0); } save = curthread_pflags_set(TDP_SYNCIO); /* * The filesystem at hand may be idle with free vnodes stored in the * batch. Return them instead of letting them stay there indefinitely. */ vfs_periodic(mp, MNT_NOWAIT); error = VFS_SYNC(mp, MNT_LAZY); curthread_pflags_restore(save); vn_finished_write(mp); vfs_unbusy(mp); return (error); } /* * The syncer vnode is no referenced. */ static int sync_inactive(struct vop_inactive_args *ap) { vgone(ap->a_vp); return (0); } /* * The syncer vnode is no longer needed and is being decommissioned. * * Modifications to the worklist must be protected by sync_mtx. */ static int sync_reclaim(struct vop_reclaim_args *ap) { struct vnode *vp = ap->a_vp; struct bufobj *bo; bo = &vp->v_bufobj; BO_LOCK(bo); mtx_lock(&sync_mtx); if (vp->v_mount->mnt_syncer == vp) vp->v_mount->mnt_syncer = NULL; if (bo->bo_flag & BO_ONWORKLST) { LIST_REMOVE(bo, bo_synclist); syncer_worklist_len--; sync_vnode_count--; bo->bo_flag &= ~BO_ONWORKLST; } mtx_unlock(&sync_mtx); BO_UNLOCK(bo); return (0); } int vn_need_pageq_flush(struct vnode *vp) { struct vm_object *obj; int need; MPASS(mtx_owned(VI_MTX(vp))); need = 0; if ((obj = vp->v_object) != NULL && (vp->v_vflag & VV_NOSYNC) == 0 && vm_object_mightbedirty(obj)) need = 1; return (need); } /* * Check if vnode represents a disk device */ int vn_isdisk(struct vnode *vp, int *errp) { int error; if (vp->v_type != VCHR) { error = ENOTBLK; goto out; } error = 0; dev_lock(); if (vp->v_rdev == NULL) error = ENXIO; else if (vp->v_rdev->si_devsw == NULL) error = ENXIO; else if (!(vp->v_rdev->si_devsw->d_flags & D_DISK)) error = ENOTBLK; dev_unlock(); out: if (errp != NULL) *errp = error; return (error == 0); } /* * Common filesystem object access control check routine. Accepts a * vnode's type, "mode", uid and gid, requested access mode, credentials, * and optional call-by-reference privused argument allowing vaccess() * to indicate to the caller whether privilege was used to satisfy the * request (obsoleted). Returns 0 on success, or an errno on failure. */ int vaccess(enum vtype type, mode_t file_mode, uid_t file_uid, gid_t file_gid, accmode_t accmode, struct ucred *cred, int *privused) { accmode_t dac_granted; accmode_t priv_granted; KASSERT((accmode & ~(VEXEC | VWRITE | VREAD | VADMIN | VAPPEND)) == 0, ("invalid bit in accmode")); KASSERT((accmode & VAPPEND) == 0 || (accmode & VWRITE), ("VAPPEND without VWRITE")); /* * Look for a normal, non-privileged way to access the file/directory * as requested. If it exists, go with that. */ if (privused != NULL) *privused = 0; dac_granted = 0; /* Check the owner. */ if (cred->cr_uid == file_uid) { dac_granted |= VADMIN; if (file_mode & S_IXUSR) dac_granted |= VEXEC; if (file_mode & S_IRUSR) dac_granted |= VREAD; if (file_mode & S_IWUSR) dac_granted |= (VWRITE | VAPPEND); if ((accmode & dac_granted) == accmode) return (0); goto privcheck; } /* Otherwise, check the groups (first match) */ if (groupmember(file_gid, cred)) { if (file_mode & S_IXGRP) dac_granted |= VEXEC; if (file_mode & S_IRGRP) dac_granted |= VREAD; if (file_mode & S_IWGRP) dac_granted |= (VWRITE | VAPPEND); if ((accmode & dac_granted) == accmode) return (0); goto privcheck; } /* Otherwise, check everyone else. */ if (file_mode & S_IXOTH) dac_granted |= VEXEC; if (file_mode & S_IROTH) dac_granted |= VREAD; if (file_mode & S_IWOTH) dac_granted |= (VWRITE | VAPPEND); if ((accmode & dac_granted) == accmode) return (0); privcheck: /* * Build a privilege mask to determine if the set of privileges * satisfies the requirements when combined with the granted mask * from above. For each privilege, if the privilege is required, * bitwise or the request type onto the priv_granted mask. */ priv_granted = 0; if (type == VDIR) { /* * For directories, use PRIV_VFS_LOOKUP to satisfy VEXEC * requests, instead of PRIV_VFS_EXEC. */ if ((accmode & VEXEC) && ((dac_granted & VEXEC) == 0) && !priv_check_cred(cred, PRIV_VFS_LOOKUP)) priv_granted |= VEXEC; } else { /* * Ensure that at least one execute bit is on. Otherwise, * a privileged user will always succeed, and we don't want * this to happen unless the file really is executable. */ if ((accmode & VEXEC) && ((dac_granted & VEXEC) == 0) && (file_mode & (S_IXUSR | S_IXGRP | S_IXOTH)) != 0 && !priv_check_cred(cred, PRIV_VFS_EXEC)) priv_granted |= VEXEC; } if ((accmode & VREAD) && ((dac_granted & VREAD) == 0) && !priv_check_cred(cred, PRIV_VFS_READ)) priv_granted |= VREAD; if ((accmode & VWRITE) && ((dac_granted & VWRITE) == 0) && !priv_check_cred(cred, PRIV_VFS_WRITE)) priv_granted |= (VWRITE | VAPPEND); if ((accmode & VADMIN) && ((dac_granted & VADMIN) == 0) && !priv_check_cred(cred, PRIV_VFS_ADMIN)) priv_granted |= VADMIN; if ((accmode & (priv_granted | dac_granted)) == accmode) { /* XXX audit: privilege used */ if (privused != NULL) *privused = 1; return (0); } return ((accmode & VADMIN) ? EPERM : EACCES); } /* * Credential check based on process requesting service, and per-attribute * permissions. */ int extattr_check_cred(struct vnode *vp, int attrnamespace, struct ucred *cred, struct thread *td, accmode_t accmode) { /* * Kernel-invoked always succeeds. */ if (cred == NOCRED) return (0); /* * Do not allow privileged processes in jail to directly manipulate * system attributes. */ switch (attrnamespace) { case EXTATTR_NAMESPACE_SYSTEM: /* Potentially should be: return (EPERM); */ return (priv_check_cred(cred, PRIV_VFS_EXTATTR_SYSTEM)); case EXTATTR_NAMESPACE_USER: return (VOP_ACCESS(vp, accmode, cred, td)); default: return (EPERM); } } #ifdef DEBUG_VFS_LOCKS /* * This only exists to suppress warnings from unlocked specfs accesses. It is * no longer ok to have an unlocked VFS. */ #define IGNORE_LOCK(vp) (KERNEL_PANICKED() || (vp) == NULL || \ (vp)->v_type == VCHR || (vp)->v_type == VBAD) int vfs_badlock_ddb = 1; /* Drop into debugger on violation. */ SYSCTL_INT(_debug, OID_AUTO, vfs_badlock_ddb, CTLFLAG_RW, &vfs_badlock_ddb, 0, "Drop into debugger on lock violation"); int vfs_badlock_mutex = 1; /* Check for interlock across VOPs. */ SYSCTL_INT(_debug, OID_AUTO, vfs_badlock_mutex, CTLFLAG_RW, &vfs_badlock_mutex, 0, "Check for interlock across VOPs"); int vfs_badlock_print = 1; /* Print lock violations. */ SYSCTL_INT(_debug, OID_AUTO, vfs_badlock_print, CTLFLAG_RW, &vfs_badlock_print, 0, "Print lock violations"); int vfs_badlock_vnode = 1; /* Print vnode details on lock violations. */ SYSCTL_INT(_debug, OID_AUTO, vfs_badlock_vnode, CTLFLAG_RW, &vfs_badlock_vnode, 0, "Print vnode details on lock violations"); #ifdef KDB int vfs_badlock_backtrace = 1; /* Print backtrace at lock violations. */ SYSCTL_INT(_debug, OID_AUTO, vfs_badlock_backtrace, CTLFLAG_RW, &vfs_badlock_backtrace, 0, "Print backtrace at lock violations"); #endif static void vfs_badlock(const char *msg, const char *str, struct vnode *vp) { #ifdef KDB if (vfs_badlock_backtrace) kdb_backtrace(); #endif if (vfs_badlock_vnode) vn_printf(vp, "vnode "); if (vfs_badlock_print) printf("%s: %p %s\n", str, (void *)vp, msg); if (vfs_badlock_ddb) kdb_enter(KDB_WHY_VFSLOCK, "lock violation"); } void assert_vi_locked(struct vnode *vp, const char *str) { if (vfs_badlock_mutex && !mtx_owned(VI_MTX(vp))) vfs_badlock("interlock is not locked but should be", str, vp); } void assert_vi_unlocked(struct vnode *vp, const char *str) { if (vfs_badlock_mutex && mtx_owned(VI_MTX(vp))) vfs_badlock("interlock is locked but should not be", str, vp); } void assert_vop_locked(struct vnode *vp, const char *str) { int locked; if (!IGNORE_LOCK(vp)) { locked = VOP_ISLOCKED(vp); if (locked == 0 || locked == LK_EXCLOTHER) vfs_badlock("is not locked but should be", str, vp); } } void assert_vop_unlocked(struct vnode *vp, const char *str) { if (!IGNORE_LOCK(vp) && VOP_ISLOCKED(vp) == LK_EXCLUSIVE) vfs_badlock("is locked but should not be", str, vp); } void assert_vop_elocked(struct vnode *vp, const char *str) { if (!IGNORE_LOCK(vp) && VOP_ISLOCKED(vp) != LK_EXCLUSIVE) vfs_badlock("is not exclusive locked but should be", str, vp); } #endif /* DEBUG_VFS_LOCKS */ void vop_rename_fail(struct vop_rename_args *ap) { if (ap->a_tvp != NULL) vput(ap->a_tvp); if (ap->a_tdvp == ap->a_tvp) vrele(ap->a_tdvp); else vput(ap->a_tdvp); vrele(ap->a_fdvp); vrele(ap->a_fvp); } void vop_rename_pre(void *ap) { struct vop_rename_args *a = ap; #ifdef DEBUG_VFS_LOCKS if (a->a_tvp) ASSERT_VI_UNLOCKED(a->a_tvp, "VOP_RENAME"); ASSERT_VI_UNLOCKED(a->a_tdvp, "VOP_RENAME"); ASSERT_VI_UNLOCKED(a->a_fvp, "VOP_RENAME"); ASSERT_VI_UNLOCKED(a->a_fdvp, "VOP_RENAME"); /* Check the source (from). */ if (a->a_tdvp->v_vnlock != a->a_fdvp->v_vnlock && (a->a_tvp == NULL || a->a_tvp->v_vnlock != a->a_fdvp->v_vnlock)) ASSERT_VOP_UNLOCKED(a->a_fdvp, "vop_rename: fdvp locked"); if (a->a_tvp == NULL || a->a_tvp->v_vnlock != a->a_fvp->v_vnlock) ASSERT_VOP_UNLOCKED(a->a_fvp, "vop_rename: fvp locked"); /* Check the target. */ if (a->a_tvp) ASSERT_VOP_LOCKED(a->a_tvp, "vop_rename: tvp not locked"); ASSERT_VOP_LOCKED(a->a_tdvp, "vop_rename: tdvp not locked"); #endif if (a->a_tdvp != a->a_fdvp) vhold(a->a_fdvp); if (a->a_tvp != a->a_fvp) vhold(a->a_fvp); vhold(a->a_tdvp); if (a->a_tvp) vhold(a->a_tvp); } #ifdef DEBUG_VFS_LOCKS void vop_strategy_pre(void *ap) { struct vop_strategy_args *a; struct buf *bp; a = ap; bp = a->a_bp; /* * Cluster ops lock their component buffers but not the IO container. */ if ((bp->b_flags & B_CLUSTER) != 0) return; if (!KERNEL_PANICKED() && !BUF_ISLOCKED(bp)) { if (vfs_badlock_print) printf( "VOP_STRATEGY: bp is not locked but should be\n"); if (vfs_badlock_ddb) kdb_enter(KDB_WHY_VFSLOCK, "lock violation"); } } void vop_lock_pre(void *ap) { struct vop_lock1_args *a = ap; if ((a->a_flags & LK_INTERLOCK) == 0) ASSERT_VI_UNLOCKED(a->a_vp, "VOP_LOCK"); else ASSERT_VI_LOCKED(a->a_vp, "VOP_LOCK"); } void vop_lock_post(void *ap, int rc) { struct vop_lock1_args *a = ap; ASSERT_VI_UNLOCKED(a->a_vp, "VOP_LOCK"); if (rc == 0 && (a->a_flags & LK_EXCLOTHER) == 0) ASSERT_VOP_LOCKED(a->a_vp, "VOP_LOCK"); } void vop_unlock_pre(void *ap) { struct vop_unlock_args *a = ap; ASSERT_VOP_LOCKED(a->a_vp, "VOP_UNLOCK"); } void vop_need_inactive_pre(void *ap) { struct vop_need_inactive_args *a = ap; ASSERT_VI_LOCKED(a->a_vp, "VOP_NEED_INACTIVE"); } void vop_need_inactive_post(void *ap, int rc) { struct vop_need_inactive_args *a = ap; ASSERT_VI_LOCKED(a->a_vp, "VOP_NEED_INACTIVE"); } #endif void vop_create_post(void *ap, int rc) { struct vop_create_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_dvp, NOTE_WRITE); } void vop_deleteextattr_post(void *ap, int rc) { struct vop_deleteextattr_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_vp, NOTE_ATTRIB); } void vop_link_post(void *ap, int rc) { struct vop_link_args *a = ap; if (!rc) { VFS_KNOTE_LOCKED(a->a_vp, NOTE_LINK); VFS_KNOTE_LOCKED(a->a_tdvp, NOTE_WRITE); } } void vop_mkdir_post(void *ap, int rc) { struct vop_mkdir_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_dvp, NOTE_WRITE | NOTE_LINK); } void vop_mknod_post(void *ap, int rc) { struct vop_mknod_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_dvp, NOTE_WRITE); } void vop_reclaim_post(void *ap, int rc) { struct vop_reclaim_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_vp, NOTE_REVOKE); } void vop_remove_post(void *ap, int rc) { struct vop_remove_args *a = ap; if (!rc) { VFS_KNOTE_LOCKED(a->a_dvp, NOTE_WRITE); VFS_KNOTE_LOCKED(a->a_vp, NOTE_DELETE); } } void vop_rename_post(void *ap, int rc) { struct vop_rename_args *a = ap; long hint; if (!rc) { hint = NOTE_WRITE; if (a->a_fdvp == a->a_tdvp) { if (a->a_tvp != NULL && a->a_tvp->v_type == VDIR) hint |= NOTE_LINK; VFS_KNOTE_UNLOCKED(a->a_fdvp, hint); VFS_KNOTE_UNLOCKED(a->a_tdvp, hint); } else { hint |= NOTE_EXTEND; if (a->a_fvp->v_type == VDIR) hint |= NOTE_LINK; VFS_KNOTE_UNLOCKED(a->a_fdvp, hint); if (a->a_fvp->v_type == VDIR && a->a_tvp != NULL && a->a_tvp->v_type == VDIR) hint &= ~NOTE_LINK; VFS_KNOTE_UNLOCKED(a->a_tdvp, hint); } VFS_KNOTE_UNLOCKED(a->a_fvp, NOTE_RENAME); if (a->a_tvp) VFS_KNOTE_UNLOCKED(a->a_tvp, NOTE_DELETE); } if (a->a_tdvp != a->a_fdvp) vdrop(a->a_fdvp); if (a->a_tvp != a->a_fvp) vdrop(a->a_fvp); vdrop(a->a_tdvp); if (a->a_tvp) vdrop(a->a_tvp); } void vop_rmdir_post(void *ap, int rc) { struct vop_rmdir_args *a = ap; if (!rc) { VFS_KNOTE_LOCKED(a->a_dvp, NOTE_WRITE | NOTE_LINK); VFS_KNOTE_LOCKED(a->a_vp, NOTE_DELETE); } } void vop_setattr_post(void *ap, int rc) { struct vop_setattr_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_vp, NOTE_ATTRIB); } void vop_setextattr_post(void *ap, int rc) { struct vop_setextattr_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_vp, NOTE_ATTRIB); } void vop_symlink_post(void *ap, int rc) { struct vop_symlink_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_dvp, NOTE_WRITE); } void vop_open_post(void *ap, int rc) { struct vop_open_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_vp, NOTE_OPEN); } void vop_close_post(void *ap, int rc) { struct vop_close_args *a = ap; if (!rc && (a->a_cred != NOCRED || /* filter out revokes */ !VN_IS_DOOMED(a->a_vp))) { VFS_KNOTE_LOCKED(a->a_vp, (a->a_fflag & FWRITE) != 0 ? NOTE_CLOSE_WRITE : NOTE_CLOSE); } } void vop_read_post(void *ap, int rc) { struct vop_read_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_vp, NOTE_READ); } void vop_readdir_post(void *ap, int rc) { struct vop_readdir_args *a = ap; if (!rc) VFS_KNOTE_LOCKED(a->a_vp, NOTE_READ); } static struct knlist fs_knlist; static void vfs_event_init(void *arg) { knlist_init_mtx(&fs_knlist, NULL); } /* XXX - correct order? */ SYSINIT(vfs_knlist, SI_SUB_VFS, SI_ORDER_ANY, vfs_event_init, NULL); void vfs_event_signal(fsid_t *fsid, uint32_t event, intptr_t data __unused) { KNOTE_UNLOCKED(&fs_knlist, event); } static int filt_fsattach(struct knote *kn); static void filt_fsdetach(struct knote *kn); static int filt_fsevent(struct knote *kn, long hint); struct filterops fs_filtops = { .f_isfd = 0, .f_attach = filt_fsattach, .f_detach = filt_fsdetach, .f_event = filt_fsevent }; static int filt_fsattach(struct knote *kn) { kn->kn_flags |= EV_CLEAR; knlist_add(&fs_knlist, kn, 0); return (0); } static void filt_fsdetach(struct knote *kn) { knlist_remove(&fs_knlist, kn, 0); } static int filt_fsevent(struct knote *kn, long hint) { kn->kn_fflags |= hint; return (kn->kn_fflags != 0); } static int sysctl_vfs_ctl(SYSCTL_HANDLER_ARGS) { struct vfsidctl vc; int error; struct mount *mp; error = SYSCTL_IN(req, &vc, sizeof(vc)); if (error) return (error); if (vc.vc_vers != VFS_CTL_VERS1) return (EINVAL); mp = vfs_getvfs(&vc.vc_fsid); if (mp == NULL) return (ENOENT); /* ensure that a specific sysctl goes to the right filesystem. */ if (strcmp(vc.vc_fstypename, "*") != 0 && strcmp(vc.vc_fstypename, mp->mnt_vfc->vfc_name) != 0) { vfs_rel(mp); return (EINVAL); } VCTLTOREQ(&vc, req); error = VFS_SYSCTL(mp, vc.vc_op, req); vfs_rel(mp); return (error); } SYSCTL_PROC(_vfs, OID_AUTO, ctl, CTLTYPE_OPAQUE | CTLFLAG_MPSAFE | CTLFLAG_WR, NULL, 0, sysctl_vfs_ctl, "", "Sysctl by fsid"); /* * Function to initialize a va_filerev field sensibly. * XXX: Wouldn't a random number make a lot more sense ?? */ u_quad_t init_va_filerev(void) { struct bintime bt; getbinuptime(&bt); return (((u_quad_t)bt.sec << 32LL) | (bt.frac >> 32LL)); } static int filt_vfsread(struct knote *kn, long hint); static int filt_vfswrite(struct knote *kn, long hint); static int filt_vfsvnode(struct knote *kn, long hint); static void filt_vfsdetach(struct knote *kn); static struct filterops vfsread_filtops = { .f_isfd = 1, .f_detach = filt_vfsdetach, .f_event = filt_vfsread }; static struct filterops vfswrite_filtops = { .f_isfd = 1, .f_detach = filt_vfsdetach, .f_event = filt_vfswrite }; static struct filterops vfsvnode_filtops = { .f_isfd = 1, .f_detach = filt_vfsdetach, .f_event = filt_vfsvnode }; static void vfs_knllock(void *arg) { struct vnode *vp = arg; vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); } static void vfs_knlunlock(void *arg) { struct vnode *vp = arg; VOP_UNLOCK(vp); } static void vfs_knl_assert_locked(void *arg) { #ifdef DEBUG_VFS_LOCKS struct vnode *vp = arg; ASSERT_VOP_LOCKED(vp, "vfs_knl_assert_locked"); #endif } static void vfs_knl_assert_unlocked(void *arg) { #ifdef DEBUG_VFS_LOCKS struct vnode *vp = arg; ASSERT_VOP_UNLOCKED(vp, "vfs_knl_assert_unlocked"); #endif } int vfs_kqfilter(struct vop_kqfilter_args *ap) { struct vnode *vp = ap->a_vp; struct knote *kn = ap->a_kn; struct knlist *knl; switch (kn->kn_filter) { case EVFILT_READ: kn->kn_fop = &vfsread_filtops; break; case EVFILT_WRITE: kn->kn_fop = &vfswrite_filtops; break; case EVFILT_VNODE: kn->kn_fop = &vfsvnode_filtops; break; default: return (EINVAL); } kn->kn_hook = (caddr_t)vp; v_addpollinfo(vp); if (vp->v_pollinfo == NULL) return (ENOMEM); knl = &vp->v_pollinfo->vpi_selinfo.si_note; vhold(vp); knlist_add(knl, kn, 0); return (0); } /* * Detach knote from vnode */ static void filt_vfsdetach(struct knote *kn) { struct vnode *vp = (struct vnode *)kn->kn_hook; KASSERT(vp->v_pollinfo != NULL, ("Missing v_pollinfo")); knlist_remove(&vp->v_pollinfo->vpi_selinfo.si_note, kn, 0); vdrop(vp); } /*ARGSUSED*/ static int filt_vfsread(struct knote *kn, long hint) { struct vnode *vp = (struct vnode *)kn->kn_hook; struct vattr va; int res; /* * filesystem is gone, so set the EOF flag and schedule * the knote for deletion. */ if (hint == NOTE_REVOKE || (hint == 0 && vp->v_type == VBAD)) { VI_LOCK(vp); kn->kn_flags |= (EV_EOF | EV_ONESHOT); VI_UNLOCK(vp); return (1); } if (VOP_GETATTR(vp, &va, curthread->td_ucred)) return (0); VI_LOCK(vp); kn->kn_data = va.va_size - kn->kn_fp->f_offset; res = (kn->kn_sfflags & NOTE_FILE_POLL) != 0 || kn->kn_data != 0; VI_UNLOCK(vp); return (res); } /*ARGSUSED*/ static int filt_vfswrite(struct knote *kn, long hint) { struct vnode *vp = (struct vnode *)kn->kn_hook; VI_LOCK(vp); /* * filesystem is gone, so set the EOF flag and schedule * the knote for deletion. */ if (hint == NOTE_REVOKE || (hint == 0 && vp->v_type == VBAD)) kn->kn_flags |= (EV_EOF | EV_ONESHOT); kn->kn_data = 0; VI_UNLOCK(vp); return (1); } static int filt_vfsvnode(struct knote *kn, long hint) { struct vnode *vp = (struct vnode *)kn->kn_hook; int res; VI_LOCK(vp); if (kn->kn_sfflags & hint) kn->kn_fflags |= hint; if (hint == NOTE_REVOKE || (hint == 0 && vp->v_type == VBAD)) { kn->kn_flags |= EV_EOF; VI_UNLOCK(vp); return (1); } res = (kn->kn_fflags != 0); VI_UNLOCK(vp); return (res); } /* * Returns whether the directory is empty or not. * If it is empty, the return value is 0; otherwise * the return value is an error value (which may * be ENOTEMPTY). */ int vfs_emptydir(struct vnode *vp) { struct uio uio; struct iovec iov; struct dirent *dirent, *dp, *endp; int error, eof; error = 0; eof = 0; ASSERT_VOP_LOCKED(vp, "vfs_emptydir"); dirent = malloc(sizeof(struct dirent), M_TEMP, M_WAITOK); iov.iov_base = dirent; iov.iov_len = sizeof(struct dirent); uio.uio_iov = &iov; uio.uio_iovcnt = 1; uio.uio_offset = 0; uio.uio_resid = sizeof(struct dirent); uio.uio_segflg = UIO_SYSSPACE; uio.uio_rw = UIO_READ; uio.uio_td = curthread; while (eof == 0 && error == 0) { error = VOP_READDIR(vp, &uio, curthread->td_ucred, &eof, NULL, NULL); if (error != 0) break; endp = (void *)((uint8_t *)dirent + sizeof(struct dirent) - uio.uio_resid); for (dp = dirent; dp < endp; dp = (void *)((uint8_t *)dp + GENERIC_DIRSIZ(dp))) { if (dp->d_type == DT_WHT) continue; if (dp->d_namlen == 0) continue; if (dp->d_type != DT_DIR && dp->d_type != DT_UNKNOWN) { error = ENOTEMPTY; break; } if (dp->d_namlen > 2) { error = ENOTEMPTY; break; } if (dp->d_namlen == 1 && dp->d_name[0] != '.') { error = ENOTEMPTY; break; } if (dp->d_namlen == 2 && dp->d_name[1] != '.') { error = ENOTEMPTY; break; } uio.uio_resid = sizeof(struct dirent); } } free(dirent, M_TEMP); return (error); } int vfs_read_dirent(struct vop_readdir_args *ap, struct dirent *dp, off_t off) { int error; if (dp->d_reclen > ap->a_uio->uio_resid) return (ENAMETOOLONG); error = uiomove(dp, dp->d_reclen, ap->a_uio); if (error) { if (ap->a_ncookies != NULL) { if (ap->a_cookies != NULL) free(ap->a_cookies, M_TEMP); ap->a_cookies = NULL; *ap->a_ncookies = 0; } return (error); } if (ap->a_ncookies == NULL) return (0); KASSERT(ap->a_cookies, ("NULL ap->a_cookies value with non-NULL ap->a_ncookies!")); *ap->a_cookies = realloc(*ap->a_cookies, (*ap->a_ncookies + 1) * sizeof(u_long), M_TEMP, M_WAITOK | M_ZERO); (*ap->a_cookies)[*ap->a_ncookies] = off; *ap->a_ncookies += 1; return (0); } /* * The purpose of this routine is to remove granularity from accmode_t, * reducing it into standard unix access bits - VEXEC, VREAD, VWRITE, * VADMIN and VAPPEND. * * If it returns 0, the caller is supposed to continue with the usual * access checks using 'accmode' as modified by this routine. If it * returns nonzero value, the caller is supposed to return that value * as errno. * * Note that after this routine runs, accmode may be zero. */ int vfs_unixify_accmode(accmode_t *accmode) { /* * There is no way to specify explicit "deny" rule using * file mode or POSIX.1e ACLs. */ if (*accmode & VEXPLICIT_DENY) { *accmode = 0; return (0); } /* * None of these can be translated into usual access bits. * Also, the common case for NFSv4 ACLs is to not contain * either of these bits. Caller should check for VWRITE * on the containing directory instead. */ if (*accmode & (VDELETE_CHILD | VDELETE)) return (EPERM); if (*accmode & VADMIN_PERMS) { *accmode &= ~VADMIN_PERMS; *accmode |= VADMIN; } /* * There is no way to deny VREAD_ATTRIBUTES, VREAD_ACL * or VSYNCHRONIZE using file mode or POSIX.1e ACL. */ *accmode &= ~(VSTAT_PERMS | VSYNCHRONIZE); return (0); } /* * Clear out a doomed vnode (if any) and replace it with a new one as long * as the fs is not being unmounted. Return the root vnode to the caller. */ static int __noinline vfs_cache_root_fallback(struct mount *mp, int flags, struct vnode **vpp) { struct vnode *vp; int error; restart: if (mp->mnt_rootvnode != NULL) { MNT_ILOCK(mp); vp = mp->mnt_rootvnode; if (vp != NULL) { if (!VN_IS_DOOMED(vp)) { vrefact(vp); MNT_IUNLOCK(mp); error = vn_lock(vp, flags); if (error == 0) { *vpp = vp; return (0); } vrele(vp); goto restart; } /* * Clear the old one. */ mp->mnt_rootvnode = NULL; } MNT_IUNLOCK(mp); if (vp != NULL) { vfs_op_barrier_wait(mp); vrele(vp); } } error = VFS_CACHEDROOT(mp, flags, vpp); if (error != 0) return (error); if (mp->mnt_vfs_ops == 0) { MNT_ILOCK(mp); if (mp->mnt_vfs_ops != 0) { MNT_IUNLOCK(mp); return (0); } if (mp->mnt_rootvnode == NULL) { vrefact(*vpp); mp->mnt_rootvnode = *vpp; } else { if (mp->mnt_rootvnode != *vpp) { if (!VN_IS_DOOMED(mp->mnt_rootvnode)) { panic("%s: mismatch between vnode returned " " by VFS_CACHEDROOT and the one cached " " (%p != %p)", __func__, *vpp, mp->mnt_rootvnode); } } } MNT_IUNLOCK(mp); } return (0); } int vfs_cache_root(struct mount *mp, int flags, struct vnode **vpp) { struct vnode *vp; int error; if (!vfs_op_thread_enter(mp)) return (vfs_cache_root_fallback(mp, flags, vpp)); - vp = (struct vnode *)atomic_load_ptr(&mp->mnt_rootvnode); + vp = atomic_load_ptr(&mp->mnt_rootvnode); if (vp == NULL || VN_IS_DOOMED(vp)) { vfs_op_thread_exit(mp); return (vfs_cache_root_fallback(mp, flags, vpp)); } vrefact(vp); vfs_op_thread_exit(mp); error = vn_lock(vp, flags); if (error != 0) { vrele(vp); return (vfs_cache_root_fallback(mp, flags, vpp)); } *vpp = vp; return (0); } struct vnode * vfs_cache_root_clear(struct mount *mp) { struct vnode *vp; /* * ops > 0 guarantees there is nobody who can see this vnode */ MPASS(mp->mnt_vfs_ops > 0); vp = mp->mnt_rootvnode; mp->mnt_rootvnode = NULL; return (vp); } void vfs_cache_root_set(struct mount *mp, struct vnode *vp) { MPASS(mp->mnt_vfs_ops > 0); vrefact(vp); mp->mnt_rootvnode = vp; } /* * These are helper functions for filesystems to traverse all * their vnodes. See MNT_VNODE_FOREACH_ALL() in sys/mount.h. * * This interface replaces MNT_VNODE_FOREACH. */ struct vnode * __mnt_vnode_next_all(struct vnode **mvp, struct mount *mp) { struct vnode *vp; if (should_yield()) kern_yield(PRI_USER); MNT_ILOCK(mp); KASSERT((*mvp)->v_mount == mp, ("marker vnode mount list mismatch")); for (vp = TAILQ_NEXT(*mvp, v_nmntvnodes); vp != NULL; vp = TAILQ_NEXT(vp, v_nmntvnodes)) { /* Allow a racy peek at VIRF_DOOMED to save a lock acquisition. */ if (vp->v_type == VMARKER || VN_IS_DOOMED(vp)) continue; VI_LOCK(vp); if (VN_IS_DOOMED(vp)) { VI_UNLOCK(vp); continue; } break; } if (vp == NULL) { __mnt_vnode_markerfree_all(mvp, mp); /* MNT_IUNLOCK(mp); -- done in above function */ mtx_assert(MNT_MTX(mp), MA_NOTOWNED); return (NULL); } TAILQ_REMOVE(&mp->mnt_nvnodelist, *mvp, v_nmntvnodes); TAILQ_INSERT_AFTER(&mp->mnt_nvnodelist, vp, *mvp, v_nmntvnodes); MNT_IUNLOCK(mp); return (vp); } struct vnode * __mnt_vnode_first_all(struct vnode **mvp, struct mount *mp) { struct vnode *vp; *mvp = vn_alloc_marker(mp); MNT_ILOCK(mp); MNT_REF(mp); TAILQ_FOREACH(vp, &mp->mnt_nvnodelist, v_nmntvnodes) { /* Allow a racy peek at VIRF_DOOMED to save a lock acquisition. */ if (vp->v_type == VMARKER || VN_IS_DOOMED(vp)) continue; VI_LOCK(vp); if (VN_IS_DOOMED(vp)) { VI_UNLOCK(vp); continue; } break; } if (vp == NULL) { MNT_REL(mp); MNT_IUNLOCK(mp); vn_free_marker(*mvp); *mvp = NULL; return (NULL); } TAILQ_INSERT_AFTER(&mp->mnt_nvnodelist, vp, *mvp, v_nmntvnodes); MNT_IUNLOCK(mp); return (vp); } void __mnt_vnode_markerfree_all(struct vnode **mvp, struct mount *mp) { if (*mvp == NULL) { MNT_IUNLOCK(mp); return; } mtx_assert(MNT_MTX(mp), MA_OWNED); KASSERT((*mvp)->v_mount == mp, ("marker vnode mount list mismatch")); TAILQ_REMOVE(&mp->mnt_nvnodelist, *mvp, v_nmntvnodes); MNT_REL(mp); MNT_IUNLOCK(mp); vn_free_marker(*mvp); *mvp = NULL; } /* * These are helper functions for filesystems to traverse their * lazy vnodes. See MNT_VNODE_FOREACH_LAZY() in sys/mount.h */ static void mnt_vnode_markerfree_lazy(struct vnode **mvp, struct mount *mp) { KASSERT((*mvp)->v_mount == mp, ("marker vnode mount list mismatch")); MNT_ILOCK(mp); MNT_REL(mp); MNT_IUNLOCK(mp); vn_free_marker(*mvp); *mvp = NULL; } /* * Relock the mp mount vnode list lock with the vp vnode interlock in the * conventional lock order during mnt_vnode_next_lazy iteration. * * On entry, the mount vnode list lock is held and the vnode interlock is not. * The list lock is dropped and reacquired. On success, both locks are held. * On failure, the mount vnode list lock is held but the vnode interlock is * not, and the procedure may have yielded. */ static bool mnt_vnode_next_lazy_relock(struct vnode *mvp, struct mount *mp, struct vnode *vp) { VNASSERT(mvp->v_mount == mp && mvp->v_type == VMARKER && TAILQ_NEXT(mvp, v_lazylist) != NULL, mvp, ("%s: bad marker", __func__)); VNASSERT(vp->v_mount == mp && vp->v_type != VMARKER, vp, ("%s: inappropriate vnode", __func__)); ASSERT_VI_UNLOCKED(vp, __func__); mtx_assert(&mp->mnt_listmtx, MA_OWNED); TAILQ_REMOVE(&mp->mnt_lazyvnodelist, mvp, v_lazylist); TAILQ_INSERT_BEFORE(vp, mvp, v_lazylist); /* * Note we may be racing against vdrop which transitioned the hold * count to 0 and now waits for the ->mnt_listmtx lock. This is fine, * if we are the only user after we get the interlock we will just * vdrop. */ vhold(vp); mtx_unlock(&mp->mnt_listmtx); VI_LOCK(vp); if (VN_IS_DOOMED(vp)) { VNPASS((vp->v_mflag & VMP_LAZYLIST) == 0, vp); goto out_lost; } VNPASS(vp->v_mflag & VMP_LAZYLIST, vp); /* * There is nothing to do if we are the last user. */ if (!refcount_release_if_not_last(&vp->v_holdcnt)) goto out_lost; mtx_lock(&mp->mnt_listmtx); return (true); out_lost: vdropl(vp); maybe_yield(); mtx_lock(&mp->mnt_listmtx); return (false); } static struct vnode * mnt_vnode_next_lazy(struct vnode **mvp, struct mount *mp, mnt_lazy_cb_t *cb, void *cbarg) { struct vnode *vp; mtx_assert(&mp->mnt_listmtx, MA_OWNED); KASSERT((*mvp)->v_mount == mp, ("marker vnode mount list mismatch")); restart: vp = TAILQ_NEXT(*mvp, v_lazylist); while (vp != NULL) { if (vp->v_type == VMARKER) { vp = TAILQ_NEXT(vp, v_lazylist); continue; } /* * See if we want to process the vnode. Note we may encounter a * long string of vnodes we don't care about and hog the list * as a result. Check for it and requeue the marker. */ VNPASS(!VN_IS_DOOMED(vp), vp); if (!cb(vp, cbarg)) { if (!should_yield()) { vp = TAILQ_NEXT(vp, v_lazylist); continue; } TAILQ_REMOVE(&mp->mnt_lazyvnodelist, *mvp, v_lazylist); TAILQ_INSERT_AFTER(&mp->mnt_lazyvnodelist, vp, *mvp, v_lazylist); mtx_unlock(&mp->mnt_listmtx); kern_yield(PRI_USER); mtx_lock(&mp->mnt_listmtx); goto restart; } /* * Try-lock because this is the wrong lock order. */ if (!VI_TRYLOCK(vp) && !mnt_vnode_next_lazy_relock(*mvp, mp, vp)) goto restart; KASSERT(vp->v_type != VMARKER, ("locked marker %p", vp)); KASSERT(vp->v_mount == mp || vp->v_mount == NULL, ("alien vnode on the lazy list %p %p", vp, mp)); VNPASS(vp->v_mount == mp, vp); VNPASS(!VN_IS_DOOMED(vp), vp); break; } TAILQ_REMOVE(&mp->mnt_lazyvnodelist, *mvp, v_lazylist); /* Check if we are done */ if (vp == NULL) { mtx_unlock(&mp->mnt_listmtx); mnt_vnode_markerfree_lazy(mvp, mp); return (NULL); } TAILQ_INSERT_AFTER(&mp->mnt_lazyvnodelist, vp, *mvp, v_lazylist); mtx_unlock(&mp->mnt_listmtx); ASSERT_VI_LOCKED(vp, "lazy iter"); return (vp); } struct vnode * __mnt_vnode_next_lazy(struct vnode **mvp, struct mount *mp, mnt_lazy_cb_t *cb, void *cbarg) { if (should_yield()) kern_yield(PRI_USER); mtx_lock(&mp->mnt_listmtx); return (mnt_vnode_next_lazy(mvp, mp, cb, cbarg)); } struct vnode * __mnt_vnode_first_lazy(struct vnode **mvp, struct mount *mp, mnt_lazy_cb_t *cb, void *cbarg) { struct vnode *vp; if (TAILQ_EMPTY(&mp->mnt_lazyvnodelist)) return (NULL); *mvp = vn_alloc_marker(mp); MNT_ILOCK(mp); MNT_REF(mp); MNT_IUNLOCK(mp); mtx_lock(&mp->mnt_listmtx); vp = TAILQ_FIRST(&mp->mnt_lazyvnodelist); if (vp == NULL) { mtx_unlock(&mp->mnt_listmtx); mnt_vnode_markerfree_lazy(mvp, mp); return (NULL); } TAILQ_INSERT_BEFORE(vp, *mvp, v_lazylist); return (mnt_vnode_next_lazy(mvp, mp, cb, cbarg)); } void __mnt_vnode_markerfree_lazy(struct vnode **mvp, struct mount *mp) { if (*mvp == NULL) return; mtx_lock(&mp->mnt_listmtx); TAILQ_REMOVE(&mp->mnt_lazyvnodelist, *mvp, v_lazylist); mtx_unlock(&mp->mnt_listmtx); mnt_vnode_markerfree_lazy(mvp, mp); } int vn_dir_check_exec(struct vnode *vp, struct componentname *cnp) { if ((cnp->cn_flags & NOEXECCHECK) != 0) { cnp->cn_flags &= ~NOEXECCHECK; return (0); } return (VOP_ACCESS(vp, VEXEC, cnp->cn_cred, cnp->cn_thread)); } Index: projects/clang1000-import/sys/kern/vfs_syscalls.c =================================================================== --- projects/clang1000-import/sys/kern/vfs_syscalls.c (revision 357965) +++ projects/clang1000-import/sys/kern/vfs_syscalls.c (revision 357966) @@ -1,4833 +1,4833 @@ /*- * SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 1989, 1993 * The Regents of the University of California. All rights reserved. * (c) UNIX System Laboratories, Inc. * All or some portions of this file are derived from material licensed * to the University of California by American Telephone and Telegraph * Co. or Unix System Laboratories, Inc. and are reproduced herein with * the permission of UNIX System Laboratories, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)vfs_syscalls.c 8.13 (Berkeley) 4/15/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_ktrace.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef KTRACE #include #endif #include #include #include #include #include #include #include #include MALLOC_DEFINE(M_FADVISE, "fadvise", "posix_fadvise(2) information"); SDT_PROVIDER_DEFINE(vfs); SDT_PROBE_DEFINE2(vfs, , stat, mode, "char *", "int"); SDT_PROBE_DEFINE2(vfs, , stat, reg, "char *", "int"); static int kern_chflagsat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, u_long flags, int atflag); static int setfflags(struct thread *td, struct vnode *, u_long); static int getutimes(const struct timeval *, enum uio_seg, struct timespec *); static int getutimens(const struct timespec *, enum uio_seg, struct timespec *, int *); static int setutimes(struct thread *td, struct vnode *, const struct timespec *, int, int); static int vn_access(struct vnode *vp, int user_flags, struct ucred *cred, struct thread *td); static int kern_fhlinkat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, fhandle_t *fhp); static int kern_getfhat(struct thread *td, int flags, int fd, const char *path, enum uio_seg pathseg, fhandle_t *fhp); static int kern_readlink_vp(struct vnode *vp, char *buf, enum uio_seg bufseg, size_t count, struct thread *td); static int kern_linkat_vp(struct thread *td, struct vnode *vp, int fd, const char *path, enum uio_seg segflag); int kern_sync(struct thread *td) { struct mount *mp, *nmp; int save; mtx_lock(&mountlist_mtx); for (mp = TAILQ_FIRST(&mountlist); mp != NULL; mp = nmp) { if (vfs_busy(mp, MBF_NOWAIT | MBF_MNTLSTLOCK)) { nmp = TAILQ_NEXT(mp, mnt_list); continue; } if ((mp->mnt_flag & MNT_RDONLY) == 0 && vn_start_write(NULL, &mp, V_NOWAIT) == 0) { save = curthread_pflags_set(TDP_SYNCIO); vfs_periodic(mp, MNT_NOWAIT); VFS_SYNC(mp, MNT_NOWAIT); curthread_pflags_restore(save); vn_finished_write(mp); } mtx_lock(&mountlist_mtx); nmp = TAILQ_NEXT(mp, mnt_list); vfs_unbusy(mp); } mtx_unlock(&mountlist_mtx); return (0); } /* * Sync each mounted filesystem. */ #ifndef _SYS_SYSPROTO_H_ struct sync_args { int dummy; }; #endif /* ARGSUSED */ int sys_sync(struct thread *td, struct sync_args *uap) { return (kern_sync(td)); } /* * Change filesystem quotas. */ #ifndef _SYS_SYSPROTO_H_ struct quotactl_args { char *path; int cmd; int uid; caddr_t arg; }; #endif int sys_quotactl(struct thread *td, struct quotactl_args *uap) { struct mount *mp; struct nameidata nd; int error; AUDIT_ARG_CMD(uap->cmd); AUDIT_ARG_UID(uap->uid); if (!prison_allow(td->td_ucred, PR_ALLOW_QUOTAS)) return (EPERM); NDINIT(&nd, LOOKUP, FOLLOW | LOCKLEAF | AUDITVNODE1, UIO_USERSPACE, uap->path, td); if ((error = namei(&nd)) != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); mp = nd.ni_vp->v_mount; vfs_ref(mp); vput(nd.ni_vp); error = vfs_busy(mp, 0); if (error != 0) { vfs_rel(mp); return (error); } error = VFS_QUOTACTL(mp, uap->cmd, uap->uid, uap->arg); /* * Since quota on operation typically needs to open quota * file, the Q_QUOTAON handler needs to unbusy the mount point * before calling into namei. Otherwise, unmount might be * started between two vfs_busy() invocations (first is our, * second is from mount point cross-walk code in lookup()), * causing deadlock. * * Require that Q_QUOTAON handles the vfs_busy() reference on * its own, always returning with ubusied mount point. */ if ((uap->cmd >> SUBCMDSHIFT) != Q_QUOTAON && (uap->cmd >> SUBCMDSHIFT) != Q_QUOTAOFF) vfs_unbusy(mp); vfs_rel(mp); return (error); } /* * Used by statfs conversion routines to scale the block size up if * necessary so that all of the block counts are <= 'max_size'. Note * that 'max_size' should be a bitmask, i.e. 2^n - 1 for some non-zero * value of 'n'. */ void statfs_scale_blocks(struct statfs *sf, long max_size) { uint64_t count; int shift; KASSERT(powerof2(max_size + 1), ("%s: invalid max_size", __func__)); /* * Attempt to scale the block counts to give a more accurate * overview to userland of the ratio of free space to used * space. To do this, find the largest block count and compute * a divisor that lets it fit into a signed integer <= max_size. */ if (sf->f_bavail < 0) count = -sf->f_bavail; else count = sf->f_bavail; count = MAX(sf->f_blocks, MAX(sf->f_bfree, count)); if (count <= max_size) return; count >>= flsl(max_size); shift = 0; while (count > 0) { shift++; count >>=1; } sf->f_bsize <<= shift; sf->f_blocks >>= shift; sf->f_bfree >>= shift; sf->f_bavail >>= shift; } static int kern_do_statfs(struct thread *td, struct mount *mp, struct statfs *buf) { int error; if (mp == NULL) return (EBADF); error = vfs_busy(mp, 0); vfs_rel(mp); if (error != 0) return (error); #ifdef MAC error = mac_mount_check_stat(td->td_ucred, mp); if (error != 0) goto out; #endif error = VFS_STATFS(mp, buf); if (error != 0) goto out; if (priv_check_cred_vfs_generation(td->td_ucred)) { buf->f_fsid.val[0] = buf->f_fsid.val[1] = 0; prison_enforce_statfs(td->td_ucred, mp, buf); } out: vfs_unbusy(mp); return (error); } /* * Get filesystem statistics. */ #ifndef _SYS_SYSPROTO_H_ struct statfs_args { char *path; struct statfs *buf; }; #endif int sys_statfs(struct thread *td, struct statfs_args *uap) { struct statfs *sfp; int error; sfp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); error = kern_statfs(td, uap->path, UIO_USERSPACE, sfp); if (error == 0) error = copyout(sfp, uap->buf, sizeof(struct statfs)); free(sfp, M_STATFS); return (error); } int kern_statfs(struct thread *td, const char *path, enum uio_seg pathseg, struct statfs *buf) { struct mount *mp; struct nameidata nd; int error; NDINIT(&nd, LOOKUP, FOLLOW | LOCKSHARED | LOCKLEAF | AUDITVNODE1, pathseg, path, td); error = namei(&nd); if (error != 0) return (error); mp = nd.ni_vp->v_mount; vfs_ref(mp); NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_vp); return (kern_do_statfs(td, mp, buf)); } /* * Get filesystem statistics. */ #ifndef _SYS_SYSPROTO_H_ struct fstatfs_args { int fd; struct statfs *buf; }; #endif int sys_fstatfs(struct thread *td, struct fstatfs_args *uap) { struct statfs *sfp; int error; sfp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); error = kern_fstatfs(td, uap->fd, sfp); if (error == 0) error = copyout(sfp, uap->buf, sizeof(struct statfs)); free(sfp, M_STATFS); return (error); } int kern_fstatfs(struct thread *td, int fd, struct statfs *buf) { struct file *fp; struct mount *mp; struct vnode *vp; int error; AUDIT_ARG_FD(fd); error = getvnode(td, fd, &cap_fstatfs_rights, &fp); if (error != 0) return (error); vp = fp->f_vnode; vn_lock(vp, LK_SHARED | LK_RETRY); #ifdef AUDIT AUDIT_ARG_VNODE1(vp); #endif mp = vp->v_mount; if (mp != NULL) vfs_ref(mp); VOP_UNLOCK(vp); fdrop(fp, td); return (kern_do_statfs(td, mp, buf)); } /* * Get statistics on all filesystems. */ #ifndef _SYS_SYSPROTO_H_ struct getfsstat_args { struct statfs *buf; long bufsize; int mode; }; #endif int sys_getfsstat(struct thread *td, struct getfsstat_args *uap) { size_t count; int error; if (uap->bufsize < 0 || uap->bufsize > SIZE_MAX) return (EINVAL); error = kern_getfsstat(td, &uap->buf, uap->bufsize, &count, UIO_USERSPACE, uap->mode); if (error == 0) td->td_retval[0] = count; return (error); } /* * If (bufsize > 0 && bufseg == UIO_SYSSPACE) * The caller is responsible for freeing memory which will be allocated * in '*buf'. */ int kern_getfsstat(struct thread *td, struct statfs **buf, size_t bufsize, size_t *countp, enum uio_seg bufseg, int mode) { struct mount *mp, *nmp; struct statfs *sfsp, *sp, *sptmp, *tofree; size_t count, maxcount; int error; switch (mode) { case MNT_WAIT: case MNT_NOWAIT: break; default: if (bufseg == UIO_SYSSPACE) *buf = NULL; return (EINVAL); } restart: maxcount = bufsize / sizeof(struct statfs); if (bufsize == 0) { sfsp = NULL; tofree = NULL; } else if (bufseg == UIO_USERSPACE) { sfsp = *buf; tofree = NULL; } else /* if (bufseg == UIO_SYSSPACE) */ { count = 0; mtx_lock(&mountlist_mtx); TAILQ_FOREACH(mp, &mountlist, mnt_list) { count++; } mtx_unlock(&mountlist_mtx); if (maxcount > count) maxcount = count; tofree = sfsp = *buf = malloc(maxcount * sizeof(struct statfs), M_STATFS, M_WAITOK); } count = 0; mtx_lock(&mountlist_mtx); for (mp = TAILQ_FIRST(&mountlist); mp != NULL; mp = nmp) { if (prison_canseemount(td->td_ucred, mp) != 0) { nmp = TAILQ_NEXT(mp, mnt_list); continue; } #ifdef MAC if (mac_mount_check_stat(td->td_ucred, mp) != 0) { nmp = TAILQ_NEXT(mp, mnt_list); continue; } #endif if (mode == MNT_WAIT) { if (vfs_busy(mp, MBF_MNTLSTLOCK) != 0) { /* * If vfs_busy() failed, and MBF_NOWAIT * wasn't passed, then the mp is gone. * Furthermore, because of MBF_MNTLSTLOCK, * the mountlist_mtx was dropped. We have * no other choice than to start over. */ mtx_unlock(&mountlist_mtx); free(tofree, M_STATFS); goto restart; } } else { if (vfs_busy(mp, MBF_NOWAIT | MBF_MNTLSTLOCK) != 0) { nmp = TAILQ_NEXT(mp, mnt_list); continue; } } if (sfsp != NULL && count < maxcount) { sp = &mp->mnt_stat; /* * If MNT_NOWAIT is specified, do not refresh * the fsstat cache. */ if (mode != MNT_NOWAIT) { error = VFS_STATFS(mp, sp); if (error != 0) { mtx_lock(&mountlist_mtx); nmp = TAILQ_NEXT(mp, mnt_list); vfs_unbusy(mp); continue; } } if (priv_check_cred_vfs_generation(td->td_ucred)) { sptmp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); *sptmp = *sp; sptmp->f_fsid.val[0] = sptmp->f_fsid.val[1] = 0; prison_enforce_statfs(td->td_ucred, mp, sptmp); sp = sptmp; } else sptmp = NULL; if (bufseg == UIO_SYSSPACE) { bcopy(sp, sfsp, sizeof(*sp)); free(sptmp, M_STATFS); } else /* if (bufseg == UIO_USERSPACE) */ { error = copyout(sp, sfsp, sizeof(*sp)); free(sptmp, M_STATFS); if (error != 0) { vfs_unbusy(mp); return (error); } } sfsp++; } count++; mtx_lock(&mountlist_mtx); nmp = TAILQ_NEXT(mp, mnt_list); vfs_unbusy(mp); } mtx_unlock(&mountlist_mtx); if (sfsp != NULL && count > maxcount) *countp = maxcount; else *countp = count; return (0); } #ifdef COMPAT_FREEBSD4 /* * Get old format filesystem statistics. */ static void freebsd4_cvtstatfs(struct statfs *, struct ostatfs *); #ifndef _SYS_SYSPROTO_H_ struct freebsd4_statfs_args { char *path; struct ostatfs *buf; }; #endif int freebsd4_statfs(struct thread *td, struct freebsd4_statfs_args *uap) { struct ostatfs osb; struct statfs *sfp; int error; sfp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); error = kern_statfs(td, uap->path, UIO_USERSPACE, sfp); if (error == 0) { freebsd4_cvtstatfs(sfp, &osb); error = copyout(&osb, uap->buf, sizeof(osb)); } free(sfp, M_STATFS); return (error); } /* * Get filesystem statistics. */ #ifndef _SYS_SYSPROTO_H_ struct freebsd4_fstatfs_args { int fd; struct ostatfs *buf; }; #endif int freebsd4_fstatfs(struct thread *td, struct freebsd4_fstatfs_args *uap) { struct ostatfs osb; struct statfs *sfp; int error; sfp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); error = kern_fstatfs(td, uap->fd, sfp); if (error == 0) { freebsd4_cvtstatfs(sfp, &osb); error = copyout(&osb, uap->buf, sizeof(osb)); } free(sfp, M_STATFS); return (error); } /* * Get statistics on all filesystems. */ #ifndef _SYS_SYSPROTO_H_ struct freebsd4_getfsstat_args { struct ostatfs *buf; long bufsize; int mode; }; #endif int freebsd4_getfsstat(struct thread *td, struct freebsd4_getfsstat_args *uap) { struct statfs *buf, *sp; struct ostatfs osb; size_t count, size; int error; if (uap->bufsize < 0) return (EINVAL); count = uap->bufsize / sizeof(struct ostatfs); if (count > SIZE_MAX / sizeof(struct statfs)) return (EINVAL); size = count * sizeof(struct statfs); error = kern_getfsstat(td, &buf, size, &count, UIO_SYSSPACE, uap->mode); if (error == 0) td->td_retval[0] = count; if (size != 0) { sp = buf; while (count != 0 && error == 0) { freebsd4_cvtstatfs(sp, &osb); error = copyout(&osb, uap->buf, sizeof(osb)); sp++; uap->buf++; count--; } free(buf, M_STATFS); } return (error); } /* * Implement fstatfs() for (NFS) file handles. */ #ifndef _SYS_SYSPROTO_H_ struct freebsd4_fhstatfs_args { struct fhandle *u_fhp; struct ostatfs *buf; }; #endif int freebsd4_fhstatfs(struct thread *td, struct freebsd4_fhstatfs_args *uap) { struct ostatfs osb; struct statfs *sfp; fhandle_t fh; int error; error = copyin(uap->u_fhp, &fh, sizeof(fhandle_t)); if (error != 0) return (error); sfp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); error = kern_fhstatfs(td, fh, sfp); if (error == 0) { freebsd4_cvtstatfs(sfp, &osb); error = copyout(&osb, uap->buf, sizeof(osb)); } free(sfp, M_STATFS); return (error); } /* * Convert a new format statfs structure to an old format statfs structure. */ static void freebsd4_cvtstatfs(struct statfs *nsp, struct ostatfs *osp) { statfs_scale_blocks(nsp, LONG_MAX); bzero(osp, sizeof(*osp)); osp->f_bsize = nsp->f_bsize; osp->f_iosize = MIN(nsp->f_iosize, LONG_MAX); osp->f_blocks = nsp->f_blocks; osp->f_bfree = nsp->f_bfree; osp->f_bavail = nsp->f_bavail; osp->f_files = MIN(nsp->f_files, LONG_MAX); osp->f_ffree = MIN(nsp->f_ffree, LONG_MAX); osp->f_owner = nsp->f_owner; osp->f_type = nsp->f_type; osp->f_flags = nsp->f_flags; osp->f_syncwrites = MIN(nsp->f_syncwrites, LONG_MAX); osp->f_asyncwrites = MIN(nsp->f_asyncwrites, LONG_MAX); osp->f_syncreads = MIN(nsp->f_syncreads, LONG_MAX); osp->f_asyncreads = MIN(nsp->f_asyncreads, LONG_MAX); strlcpy(osp->f_fstypename, nsp->f_fstypename, MIN(MFSNAMELEN, OMFSNAMELEN)); strlcpy(osp->f_mntonname, nsp->f_mntonname, MIN(MNAMELEN, OMNAMELEN)); strlcpy(osp->f_mntfromname, nsp->f_mntfromname, MIN(MNAMELEN, OMNAMELEN)); osp->f_fsid = nsp->f_fsid; } #endif /* COMPAT_FREEBSD4 */ #if defined(COMPAT_FREEBSD11) /* * Get old format filesystem statistics. */ static void freebsd11_cvtstatfs(struct statfs *, struct freebsd11_statfs *); int freebsd11_statfs(struct thread *td, struct freebsd11_statfs_args *uap) { struct freebsd11_statfs osb; struct statfs *sfp; int error; sfp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); error = kern_statfs(td, uap->path, UIO_USERSPACE, sfp); if (error == 0) { freebsd11_cvtstatfs(sfp, &osb); error = copyout(&osb, uap->buf, sizeof(osb)); } free(sfp, M_STATFS); return (error); } /* * Get filesystem statistics. */ int freebsd11_fstatfs(struct thread *td, struct freebsd11_fstatfs_args *uap) { struct freebsd11_statfs osb; struct statfs *sfp; int error; sfp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); error = kern_fstatfs(td, uap->fd, sfp); if (error == 0) { freebsd11_cvtstatfs(sfp, &osb); error = copyout(&osb, uap->buf, sizeof(osb)); } free(sfp, M_STATFS); return (error); } /* * Get statistics on all filesystems. */ int freebsd11_getfsstat(struct thread *td, struct freebsd11_getfsstat_args *uap) { struct freebsd11_statfs osb; struct statfs *buf, *sp; size_t count, size; int error; count = uap->bufsize / sizeof(struct ostatfs); size = count * sizeof(struct statfs); error = kern_getfsstat(td, &buf, size, &count, UIO_SYSSPACE, uap->mode); if (error == 0) td->td_retval[0] = count; if (size > 0) { sp = buf; while (count > 0 && error == 0) { freebsd11_cvtstatfs(sp, &osb); error = copyout(&osb, uap->buf, sizeof(osb)); sp++; uap->buf++; count--; } free(buf, M_STATFS); } return (error); } /* * Implement fstatfs() for (NFS) file handles. */ int freebsd11_fhstatfs(struct thread *td, struct freebsd11_fhstatfs_args *uap) { struct freebsd11_statfs osb; struct statfs *sfp; fhandle_t fh; int error; error = copyin(uap->u_fhp, &fh, sizeof(fhandle_t)); if (error) return (error); sfp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); error = kern_fhstatfs(td, fh, sfp); if (error == 0) { freebsd11_cvtstatfs(sfp, &osb); error = copyout(&osb, uap->buf, sizeof(osb)); } free(sfp, M_STATFS); return (error); } /* * Convert a new format statfs structure to an old format statfs structure. */ static void freebsd11_cvtstatfs(struct statfs *nsp, struct freebsd11_statfs *osp) { bzero(osp, sizeof(*osp)); osp->f_version = FREEBSD11_STATFS_VERSION; osp->f_type = nsp->f_type; osp->f_flags = nsp->f_flags; osp->f_bsize = nsp->f_bsize; osp->f_iosize = nsp->f_iosize; osp->f_blocks = nsp->f_blocks; osp->f_bfree = nsp->f_bfree; osp->f_bavail = nsp->f_bavail; osp->f_files = nsp->f_files; osp->f_ffree = nsp->f_ffree; osp->f_syncwrites = nsp->f_syncwrites; osp->f_asyncwrites = nsp->f_asyncwrites; osp->f_syncreads = nsp->f_syncreads; osp->f_asyncreads = nsp->f_asyncreads; osp->f_namemax = nsp->f_namemax; osp->f_owner = nsp->f_owner; osp->f_fsid = nsp->f_fsid; strlcpy(osp->f_fstypename, nsp->f_fstypename, MIN(MFSNAMELEN, sizeof(osp->f_fstypename))); strlcpy(osp->f_mntonname, nsp->f_mntonname, MIN(MNAMELEN, sizeof(osp->f_mntonname))); strlcpy(osp->f_mntfromname, nsp->f_mntfromname, MIN(MNAMELEN, sizeof(osp->f_mntfromname))); } #endif /* COMPAT_FREEBSD11 */ /* * Change current working directory to a given file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct fchdir_args { int fd; }; #endif int sys_fchdir(struct thread *td, struct fchdir_args *uap) { struct vnode *vp, *tdp; struct mount *mp; struct file *fp; int error; AUDIT_ARG_FD(uap->fd); error = getvnode(td, uap->fd, &cap_fchdir_rights, &fp); if (error != 0) return (error); vp = fp->f_vnode; vrefact(vp); fdrop(fp, td); vn_lock(vp, LK_SHARED | LK_RETRY); AUDIT_ARG_VNODE1(vp); error = change_dir(vp, td); while (!error && (mp = vp->v_mountedhere) != NULL) { if (vfs_busy(mp, 0)) continue; error = VFS_ROOT(mp, LK_SHARED, &tdp); vfs_unbusy(mp); if (error != 0) break; vput(vp); vp = tdp; } if (error != 0) { vput(vp); return (error); } VOP_UNLOCK(vp); pwd_chdir(td, vp); return (0); } /* * Change current working directory (``.''). */ #ifndef _SYS_SYSPROTO_H_ struct chdir_args { char *path; }; #endif int sys_chdir(struct thread *td, struct chdir_args *uap) { return (kern_chdir(td, uap->path, UIO_USERSPACE)); } int kern_chdir(struct thread *td, const char *path, enum uio_seg pathseg) { struct nameidata nd; int error; NDINIT(&nd, LOOKUP, FOLLOW | LOCKSHARED | LOCKLEAF | AUDITVNODE1, pathseg, path, td); if ((error = namei(&nd)) != 0) return (error); if ((error = change_dir(nd.ni_vp, td)) != 0) { vput(nd.ni_vp); NDFREE(&nd, NDF_ONLY_PNBUF); return (error); } VOP_UNLOCK(nd.ni_vp); NDFREE(&nd, NDF_ONLY_PNBUF); pwd_chdir(td, nd.ni_vp); return (0); } /* * Change notion of root (``/'') directory. */ #ifndef _SYS_SYSPROTO_H_ struct chroot_args { char *path; }; #endif int sys_chroot(struct thread *td, struct chroot_args *uap) { struct nameidata nd; int error; error = priv_check(td, PRIV_VFS_CHROOT); if (error != 0) return (error); NDINIT(&nd, LOOKUP, FOLLOW | LOCKSHARED | LOCKLEAF | AUDITVNODE1, UIO_USERSPACE, uap->path, td); error = namei(&nd); if (error != 0) goto error; error = change_dir(nd.ni_vp, td); if (error != 0) goto e_vunlock; #ifdef MAC error = mac_vnode_check_chroot(td->td_ucred, nd.ni_vp); if (error != 0) goto e_vunlock; #endif VOP_UNLOCK(nd.ni_vp); error = pwd_chroot(td, nd.ni_vp); vrele(nd.ni_vp); NDFREE(&nd, NDF_ONLY_PNBUF); return (error); e_vunlock: vput(nd.ni_vp); error: NDFREE(&nd, NDF_ONLY_PNBUF); return (error); } /* * Common routine for chroot and chdir. Callers must provide a locked vnode * instance. */ int change_dir(struct vnode *vp, struct thread *td) { #ifdef MAC int error; #endif ASSERT_VOP_LOCKED(vp, "change_dir(): vp not locked"); if (vp->v_type != VDIR) return (ENOTDIR); #ifdef MAC error = mac_vnode_check_chdir(td->td_ucred, vp); if (error != 0) return (error); #endif return (VOP_ACCESS(vp, VEXEC, td->td_ucred, td)); } static __inline void flags_to_rights(int flags, cap_rights_t *rightsp) { if (flags & O_EXEC) { - cap_rights_set(rightsp, CAP_FEXECVE); + cap_rights_set_one(rightsp, CAP_FEXECVE); } else { switch ((flags & O_ACCMODE)) { case O_RDONLY: - cap_rights_set(rightsp, CAP_READ); + cap_rights_set_one(rightsp, CAP_READ); break; case O_RDWR: - cap_rights_set(rightsp, CAP_READ); + cap_rights_set_one(rightsp, CAP_READ); /* FALLTHROUGH */ case O_WRONLY: - cap_rights_set(rightsp, CAP_WRITE); + cap_rights_set_one(rightsp, CAP_WRITE); if (!(flags & (O_APPEND | O_TRUNC))) - cap_rights_set(rightsp, CAP_SEEK); + cap_rights_set_one(rightsp, CAP_SEEK); break; } } if (flags & O_CREAT) - cap_rights_set(rightsp, CAP_CREATE); + cap_rights_set_one(rightsp, CAP_CREATE); if (flags & O_TRUNC) - cap_rights_set(rightsp, CAP_FTRUNCATE); + cap_rights_set_one(rightsp, CAP_FTRUNCATE); if (flags & (O_SYNC | O_FSYNC)) - cap_rights_set(rightsp, CAP_FSYNC); + cap_rights_set_one(rightsp, CAP_FSYNC); if (flags & (O_EXLOCK | O_SHLOCK)) - cap_rights_set(rightsp, CAP_FLOCK); + cap_rights_set_one(rightsp, CAP_FLOCK); } /* * Check permissions, allocate an open file structure, and call the device * open routine if any. */ #ifndef _SYS_SYSPROTO_H_ struct open_args { char *path; int flags; int mode; }; #endif int sys_open(struct thread *td, struct open_args *uap) { return (kern_openat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->flags, uap->mode)); } #ifndef _SYS_SYSPROTO_H_ struct openat_args { int fd; char *path; int flag; int mode; }; #endif int sys_openat(struct thread *td, struct openat_args *uap) { AUDIT_ARG_FD(uap->fd); return (kern_openat(td, uap->fd, uap->path, UIO_USERSPACE, uap->flag, uap->mode)); } int kern_openat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, int flags, int mode) { struct proc *p = td->td_proc; struct filedesc *fdp = p->p_fd; struct file *fp; struct vnode *vp; struct nameidata nd; cap_rights_t rights; int cmode, error, indx; indx = -1; AUDIT_ARG_FFLAGS(flags); AUDIT_ARG_MODE(mode); - cap_rights_init(&rights, CAP_LOOKUP); + cap_rights_init_one(&rights, CAP_LOOKUP); flags_to_rights(flags, &rights); /* * Only one of the O_EXEC, O_RDONLY, O_WRONLY and O_RDWR flags * may be specified. */ if (flags & O_EXEC) { if (flags & O_ACCMODE) return (EINVAL); } else if ((flags & O_ACCMODE) == O_ACCMODE) { return (EINVAL); } else { flags = FFLAGS(flags); } /* * Allocate a file structure. The descriptor to reference it * is allocated and set by finstall() below. */ error = falloc_noinstall(td, &fp); if (error != 0) return (error); /* * An extra reference on `fp' has been held for us by * falloc_noinstall(). */ /* Set the flags early so the finit in devfs can pick them up. */ fp->f_flag = flags & FMASK; cmode = ((mode & ~fdp->fd_cmask) & ALLPERMS) & ~S_ISTXT; NDINIT_ATRIGHTS(&nd, LOOKUP, FOLLOW | AUDITVNODE1, pathseg, path, fd, &rights, td); td->td_dupfd = -1; /* XXX check for fdopen */ error = vn_open(&nd, &flags, cmode, fp); if (error != 0) { /* * If the vn_open replaced the method vector, something * wonderous happened deep below and we just pass it up * pretending we know what we do. */ if (error == ENXIO && fp->f_ops != &badfileops) goto success; /* * Handle special fdopen() case. bleh. * * Don't do this for relative (capability) lookups; we don't * understand exactly what would happen, and we don't think * that it ever should. */ if ((nd.ni_lcf & NI_LCF_STRICTRELATIVE) == 0 && (error == ENODEV || error == ENXIO) && td->td_dupfd >= 0) { error = dupfdopen(td, fdp, td->td_dupfd, flags, error, &indx); if (error == 0) goto success; } goto bad; } td->td_dupfd = 0; NDFREE(&nd, NDF_ONLY_PNBUF); vp = nd.ni_vp; /* * Store the vnode, for any f_type. Typically, the vnode use * count is decremented by direct call to vn_closefile() for * files that switched type in the cdevsw fdopen() method. */ fp->f_vnode = vp; /* * If the file wasn't claimed by devfs bind it to the normal * vnode operations here. */ if (fp->f_ops == &badfileops) { KASSERT(vp->v_type != VFIFO, ("Unexpected fifo.")); fp->f_seqcount = 1; finit(fp, (flags & FMASK) | (fp->f_flag & FHASLOCK), DTYPE_VNODE, vp, &vnops); } VOP_UNLOCK(vp); if (flags & O_TRUNC) { error = fo_truncate(fp, 0, td->td_ucred, td); if (error != 0) goto bad; } success: /* * If we haven't already installed the FD (for dupfdopen), do so now. */ if (indx == -1) { struct filecaps *fcaps; #ifdef CAPABILITIES if ((nd.ni_lcf & NI_LCF_STRICTRELATIVE) != 0) fcaps = &nd.ni_filecaps; else #endif fcaps = NULL; error = finstall(td, fp, &indx, flags, fcaps); /* On success finstall() consumes fcaps. */ if (error != 0) { filecaps_free(&nd.ni_filecaps); goto bad; } } else { filecaps_free(&nd.ni_filecaps); } /* * Release our private reference, leaving the one associated with * the descriptor table intact. */ fdrop(fp, td); td->td_retval[0] = indx; return (0); bad: KASSERT(indx == -1, ("indx=%d, should be -1", indx)); fdrop(fp, td); return (error); } #ifdef COMPAT_43 /* * Create a file. */ #ifndef _SYS_SYSPROTO_H_ struct ocreat_args { char *path; int mode; }; #endif int ocreat(struct thread *td, struct ocreat_args *uap) { return (kern_openat(td, AT_FDCWD, uap->path, UIO_USERSPACE, O_WRONLY | O_CREAT | O_TRUNC, uap->mode)); } #endif /* COMPAT_43 */ /* * Create a special file. */ #ifndef _SYS_SYSPROTO_H_ struct mknodat_args { int fd; char *path; mode_t mode; dev_t dev; }; #endif int sys_mknodat(struct thread *td, struct mknodat_args *uap) { return (kern_mknodat(td, uap->fd, uap->path, UIO_USERSPACE, uap->mode, uap->dev)); } #if defined(COMPAT_FREEBSD11) int freebsd11_mknod(struct thread *td, struct freebsd11_mknod_args *uap) { return (kern_mknodat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->mode, uap->dev)); } int freebsd11_mknodat(struct thread *td, struct freebsd11_mknodat_args *uap) { return (kern_mknodat(td, uap->fd, uap->path, UIO_USERSPACE, uap->mode, uap->dev)); } #endif /* COMPAT_FREEBSD11 */ int kern_mknodat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, int mode, dev_t dev) { struct vnode *vp; struct mount *mp; struct vattr vattr; struct nameidata nd; int error, whiteout = 0; AUDIT_ARG_MODE(mode); AUDIT_ARG_DEV(dev); switch (mode & S_IFMT) { case S_IFCHR: case S_IFBLK: error = priv_check(td, PRIV_VFS_MKNOD_DEV); if (error == 0 && dev == VNOVAL) error = EINVAL; break; case S_IFWHT: error = priv_check(td, PRIV_VFS_MKNOD_WHT); break; case S_IFIFO: if (dev == 0) return (kern_mkfifoat(td, fd, path, pathseg, mode)); /* FALLTHROUGH */ default: error = EINVAL; break; } if (error != 0) return (error); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, CREATE, LOCKPARENT | SAVENAME | AUDITVNODE1 | NOCACHE, pathseg, path, fd, &cap_mknodat_rights, td); if ((error = namei(&nd)) != 0) return (error); vp = nd.ni_vp; if (vp != NULL) { NDFREE(&nd, NDF_ONLY_PNBUF); if (vp == nd.ni_dvp) vrele(nd.ni_dvp); else vput(nd.ni_dvp); vrele(vp); return (EEXIST); } else { VATTR_NULL(&vattr); vattr.va_mode = (mode & ALLPERMS) & ~td->td_proc->p_fd->fd_cmask; vattr.va_rdev = dev; whiteout = 0; switch (mode & S_IFMT) { case S_IFCHR: vattr.va_type = VCHR; break; case S_IFBLK: vattr.va_type = VBLK; break; case S_IFWHT: whiteout = 1; break; default: panic("kern_mknod: invalid mode"); } } if (vn_start_write(nd.ni_dvp, &mp, V_NOWAIT) != 0) { NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); if ((error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH)) != 0) return (error); goto restart; } #ifdef MAC if (error == 0 && !whiteout) error = mac_vnode_check_create(td->td_ucred, nd.ni_dvp, &nd.ni_cnd, &vattr); #endif if (error == 0) { if (whiteout) error = VOP_WHITEOUT(nd.ni_dvp, &nd.ni_cnd, CREATE); else { error = VOP_MKNOD(nd.ni_dvp, &nd.ni_vp, &nd.ni_cnd, &vattr); if (error == 0) vput(nd.ni_vp); } } NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); vn_finished_write(mp); return (error); } /* * Create a named pipe. */ #ifndef _SYS_SYSPROTO_H_ struct mkfifo_args { char *path; int mode; }; #endif int sys_mkfifo(struct thread *td, struct mkfifo_args *uap) { return (kern_mkfifoat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->mode)); } #ifndef _SYS_SYSPROTO_H_ struct mkfifoat_args { int fd; char *path; mode_t mode; }; #endif int sys_mkfifoat(struct thread *td, struct mkfifoat_args *uap) { return (kern_mkfifoat(td, uap->fd, uap->path, UIO_USERSPACE, uap->mode)); } int kern_mkfifoat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, int mode) { struct mount *mp; struct vattr vattr; struct nameidata nd; int error; AUDIT_ARG_MODE(mode); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, CREATE, LOCKPARENT | SAVENAME | AUDITVNODE1 | NOCACHE, pathseg, path, fd, &cap_mkfifoat_rights, td); if ((error = namei(&nd)) != 0) return (error); if (nd.ni_vp != NULL) { NDFREE(&nd, NDF_ONLY_PNBUF); if (nd.ni_vp == nd.ni_dvp) vrele(nd.ni_dvp); else vput(nd.ni_dvp); vrele(nd.ni_vp); return (EEXIST); } if (vn_start_write(nd.ni_dvp, &mp, V_NOWAIT) != 0) { NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); if ((error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH)) != 0) return (error); goto restart; } VATTR_NULL(&vattr); vattr.va_type = VFIFO; vattr.va_mode = (mode & ALLPERMS) & ~td->td_proc->p_fd->fd_cmask; #ifdef MAC error = mac_vnode_check_create(td->td_ucred, nd.ni_dvp, &nd.ni_cnd, &vattr); if (error != 0) goto out; #endif error = VOP_MKNOD(nd.ni_dvp, &nd.ni_vp, &nd.ni_cnd, &vattr); if (error == 0) vput(nd.ni_vp); #ifdef MAC out: #endif vput(nd.ni_dvp); vn_finished_write(mp); NDFREE(&nd, NDF_ONLY_PNBUF); return (error); } /* * Make a hard file link. */ #ifndef _SYS_SYSPROTO_H_ struct link_args { char *path; char *link; }; #endif int sys_link(struct thread *td, struct link_args *uap) { return (kern_linkat(td, AT_FDCWD, AT_FDCWD, uap->path, uap->link, UIO_USERSPACE, FOLLOW)); } #ifndef _SYS_SYSPROTO_H_ struct linkat_args { int fd1; char *path1; int fd2; char *path2; int flag; }; #endif int sys_linkat(struct thread *td, struct linkat_args *uap) { int flag; flag = uap->flag; if ((flag & ~(AT_SYMLINK_FOLLOW | AT_BENEATH)) != 0) return (EINVAL); return (kern_linkat(td, uap->fd1, uap->fd2, uap->path1, uap->path2, UIO_USERSPACE, ((flag & AT_SYMLINK_FOLLOW) != 0 ? FOLLOW : NOFOLLOW) | ((flag & AT_BENEATH) != 0 ? BENEATH : 0))); } int hardlink_check_uid = 0; SYSCTL_INT(_security_bsd, OID_AUTO, hardlink_check_uid, CTLFLAG_RW, &hardlink_check_uid, 0, "Unprivileged processes cannot create hard links to files owned by other " "users"); static int hardlink_check_gid = 0; SYSCTL_INT(_security_bsd, OID_AUTO, hardlink_check_gid, CTLFLAG_RW, &hardlink_check_gid, 0, "Unprivileged processes cannot create hard links to files owned by other " "groups"); static int can_hardlink(struct vnode *vp, struct ucred *cred) { struct vattr va; int error; if (!hardlink_check_uid && !hardlink_check_gid) return (0); error = VOP_GETATTR(vp, &va, cred); if (error != 0) return (error); if (hardlink_check_uid && cred->cr_uid != va.va_uid) { error = priv_check_cred(cred, PRIV_VFS_LINK); if (error != 0) return (error); } if (hardlink_check_gid && !groupmember(va.va_gid, cred)) { error = priv_check_cred(cred, PRIV_VFS_LINK); if (error != 0) return (error); } return (0); } int kern_linkat(struct thread *td, int fd1, int fd2, const char *path1, const char *path2, enum uio_seg segflag, int follow) { struct nameidata nd; int error; do { bwillwrite(); NDINIT_ATRIGHTS(&nd, LOOKUP, follow | AUDITVNODE1, segflag, path1, fd1, &cap_linkat_source_rights, td); if ((error = namei(&nd)) != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = kern_linkat_vp(td, nd.ni_vp, fd2, path2, segflag); } while (error == EAGAIN); return (error); } static int kern_linkat_vp(struct thread *td, struct vnode *vp, int fd, const char *path, enum uio_seg segflag) { struct nameidata nd; struct mount *mp; int error; if (vp->v_type == VDIR) { vrele(vp); return (EPERM); /* POSIX */ } NDINIT_ATRIGHTS(&nd, CREATE, LOCKPARENT | SAVENAME | AUDITVNODE2 | NOCACHE, segflag, path, fd, &cap_linkat_target_rights, td); if ((error = namei(&nd)) == 0) { if (nd.ni_vp != NULL) { NDFREE(&nd, NDF_ONLY_PNBUF); if (nd.ni_dvp == nd.ni_vp) vrele(nd.ni_dvp); else vput(nd.ni_dvp); vrele(nd.ni_vp); vrele(vp); return (EEXIST); } else if (nd.ni_dvp->v_mount != vp->v_mount) { /* * Cross-device link. No need to recheck * vp->v_type, since it cannot change, except * to VBAD. */ NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); vrele(vp); return (EXDEV); } else if ((error = vn_lock(vp, LK_EXCLUSIVE)) == 0) { error = can_hardlink(vp, td->td_ucred); #ifdef MAC if (error == 0) error = mac_vnode_check_link(td->td_ucred, nd.ni_dvp, vp, &nd.ni_cnd); #endif if (error != 0) { vput(vp); vput(nd.ni_dvp); NDFREE(&nd, NDF_ONLY_PNBUF); return (error); } error = vn_start_write(vp, &mp, V_NOWAIT); if (error != 0) { vput(vp); vput(nd.ni_dvp); NDFREE(&nd, NDF_ONLY_PNBUF); error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH); if (error != 0) return (error); return (EAGAIN); } error = VOP_LINK(nd.ni_dvp, vp, &nd.ni_cnd); VOP_UNLOCK(vp); vput(nd.ni_dvp); vn_finished_write(mp); NDFREE(&nd, NDF_ONLY_PNBUF); } else { vput(nd.ni_dvp); NDFREE(&nd, NDF_ONLY_PNBUF); vrele(vp); return (EAGAIN); } } vrele(vp); return (error); } /* * Make a symbolic link. */ #ifndef _SYS_SYSPROTO_H_ struct symlink_args { char *path; char *link; }; #endif int sys_symlink(struct thread *td, struct symlink_args *uap) { return (kern_symlinkat(td, uap->path, AT_FDCWD, uap->link, UIO_USERSPACE)); } #ifndef _SYS_SYSPROTO_H_ struct symlinkat_args { char *path; int fd; char *path2; }; #endif int sys_symlinkat(struct thread *td, struct symlinkat_args *uap) { return (kern_symlinkat(td, uap->path1, uap->fd, uap->path2, UIO_USERSPACE)); } int kern_symlinkat(struct thread *td, const char *path1, int fd, const char *path2, enum uio_seg segflg) { struct mount *mp; struct vattr vattr; const char *syspath; char *tmppath; struct nameidata nd; int error; if (segflg == UIO_SYSSPACE) { syspath = path1; } else { tmppath = uma_zalloc(namei_zone, M_WAITOK); if ((error = copyinstr(path1, tmppath, MAXPATHLEN, NULL)) != 0) goto out; syspath = tmppath; } AUDIT_ARG_TEXT(syspath); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, CREATE, LOCKPARENT | SAVENAME | AUDITVNODE1 | NOCACHE, segflg, path2, fd, &cap_symlinkat_rights, td); if ((error = namei(&nd)) != 0) goto out; if (nd.ni_vp) { NDFREE(&nd, NDF_ONLY_PNBUF); if (nd.ni_vp == nd.ni_dvp) vrele(nd.ni_dvp); else vput(nd.ni_dvp); vrele(nd.ni_vp); error = EEXIST; goto out; } if (vn_start_write(nd.ni_dvp, &mp, V_NOWAIT) != 0) { NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); if ((error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH)) != 0) goto out; goto restart; } VATTR_NULL(&vattr); vattr.va_mode = ACCESSPERMS &~ td->td_proc->p_fd->fd_cmask; #ifdef MAC vattr.va_type = VLNK; error = mac_vnode_check_create(td->td_ucred, nd.ni_dvp, &nd.ni_cnd, &vattr); if (error != 0) goto out2; #endif error = VOP_SYMLINK(nd.ni_dvp, &nd.ni_vp, &nd.ni_cnd, &vattr, syspath); if (error == 0) vput(nd.ni_vp); #ifdef MAC out2: #endif NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); vn_finished_write(mp); out: if (segflg != UIO_SYSSPACE) uma_zfree(namei_zone, tmppath); return (error); } /* * Delete a whiteout from the filesystem. */ #ifndef _SYS_SYSPROTO_H_ struct undelete_args { char *path; }; #endif int sys_undelete(struct thread *td, struct undelete_args *uap) { struct mount *mp; struct nameidata nd; int error; restart: bwillwrite(); NDINIT(&nd, DELETE, LOCKPARENT | DOWHITEOUT | AUDITVNODE1, UIO_USERSPACE, uap->path, td); error = namei(&nd); if (error != 0) return (error); if (nd.ni_vp != NULLVP || !(nd.ni_cnd.cn_flags & ISWHITEOUT)) { NDFREE(&nd, NDF_ONLY_PNBUF); if (nd.ni_vp == nd.ni_dvp) vrele(nd.ni_dvp); else vput(nd.ni_dvp); if (nd.ni_vp) vrele(nd.ni_vp); return (EEXIST); } if (vn_start_write(nd.ni_dvp, &mp, V_NOWAIT) != 0) { NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); if ((error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH)) != 0) return (error); goto restart; } error = VOP_WHITEOUT(nd.ni_dvp, &nd.ni_cnd, DELETE); NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); vn_finished_write(mp); return (error); } /* * Delete a name from the filesystem. */ #ifndef _SYS_SYSPROTO_H_ struct unlink_args { char *path; }; #endif int sys_unlink(struct thread *td, struct unlink_args *uap) { return (kern_funlinkat(td, AT_FDCWD, uap->path, FD_NONE, UIO_USERSPACE, 0, 0)); } static int kern_funlinkat_ex(struct thread *td, int dfd, const char *path, int fd, int flag, enum uio_seg pathseg, ino_t oldinum) { if ((flag & ~AT_REMOVEDIR) != 0) return (EINVAL); if ((flag & AT_REMOVEDIR) != 0) return (kern_frmdirat(td, dfd, path, fd, UIO_USERSPACE, 0)); return (kern_funlinkat(td, dfd, path, fd, UIO_USERSPACE, 0, 0)); } #ifndef _SYS_SYSPROTO_H_ struct unlinkat_args { int fd; char *path; int flag; }; #endif int sys_unlinkat(struct thread *td, struct unlinkat_args *uap) { return (kern_funlinkat_ex(td, uap->fd, uap->path, FD_NONE, uap->flag, UIO_USERSPACE, 0)); } #ifndef _SYS_SYSPROTO_H_ struct funlinkat_args { int dfd; const char *path; int fd; int flag; }; #endif int sys_funlinkat(struct thread *td, struct funlinkat_args *uap) { return (kern_funlinkat_ex(td, uap->dfd, uap->path, uap->fd, uap->flag, UIO_USERSPACE, 0)); } int kern_funlinkat(struct thread *td, int dfd, const char *path, int fd, enum uio_seg pathseg, int flag, ino_t oldinum) { struct mount *mp; struct file *fp; struct vnode *vp; struct nameidata nd; struct stat sb; int error; fp = NULL; if (fd != FD_NONE) { error = getvnode(td, fd, &cap_no_rights, &fp); if (error != 0) return (error); } restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, DELETE, LOCKPARENT | LOCKLEAF | AUDITVNODE1 | ((flag & AT_BENEATH) != 0 ? BENEATH : 0), pathseg, path, dfd, &cap_unlinkat_rights, td); if ((error = namei(&nd)) != 0) { if (error == EINVAL) error = EPERM; goto fdout; } vp = nd.ni_vp; if (vp->v_type == VDIR && oldinum == 0) { error = EPERM; /* POSIX */ } else if (oldinum != 0 && ((error = vn_stat(vp, &sb, td->td_ucred, NOCRED, td)) == 0) && sb.st_ino != oldinum) { error = EIDRM; /* Identifier removed */ } else if (fp != NULL && fp->f_vnode != vp) { if (VN_IS_DOOMED(fp->f_vnode)) error = EBADF; else error = EDEADLK; } else { /* * The root of a mounted filesystem cannot be deleted. * * XXX: can this only be a VDIR case? */ if (vp->v_vflag & VV_ROOT) error = EBUSY; } if (error == 0) { if (vn_start_write(nd.ni_dvp, &mp, V_NOWAIT) != 0) { NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); if (vp == nd.ni_dvp) vrele(vp); else vput(vp); if ((error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH)) != 0) { goto fdout; } goto restart; } #ifdef MAC error = mac_vnode_check_unlink(td->td_ucred, nd.ni_dvp, vp, &nd.ni_cnd); if (error != 0) goto out; #endif vfs_notify_upper(vp, VFS_NOTIFY_UPPER_UNLINK); error = VOP_REMOVE(nd.ni_dvp, vp, &nd.ni_cnd); #ifdef MAC out: #endif vn_finished_write(mp); } NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); if (vp == nd.ni_dvp) vrele(vp); else vput(vp); fdout: if (fp != NULL) fdrop(fp, td); return (error); } /* * Reposition read/write file offset. */ #ifndef _SYS_SYSPROTO_H_ struct lseek_args { int fd; int pad; off_t offset; int whence; }; #endif int sys_lseek(struct thread *td, struct lseek_args *uap) { return (kern_lseek(td, uap->fd, uap->offset, uap->whence)); } int kern_lseek(struct thread *td, int fd, off_t offset, int whence) { struct file *fp; int error; AUDIT_ARG_FD(fd); error = fget(td, fd, &cap_seek_rights, &fp); if (error != 0) return (error); error = (fp->f_ops->fo_flags & DFLAG_SEEKABLE) != 0 ? fo_seek(fp, offset, whence, td) : ESPIPE; fdrop(fp, td); return (error); } #if defined(COMPAT_43) /* * Reposition read/write file offset. */ #ifndef _SYS_SYSPROTO_H_ struct olseek_args { int fd; long offset; int whence; }; #endif int olseek(struct thread *td, struct olseek_args *uap) { return (kern_lseek(td, uap->fd, uap->offset, uap->whence)); } #endif /* COMPAT_43 */ #if defined(COMPAT_FREEBSD6) /* Version with the 'pad' argument */ int freebsd6_lseek(struct thread *td, struct freebsd6_lseek_args *uap) { return (kern_lseek(td, uap->fd, uap->offset, uap->whence)); } #endif /* * Check access permissions using passed credentials. */ static int vn_access(struct vnode *vp, int user_flags, struct ucred *cred, struct thread *td) { accmode_t accmode; int error; /* Flags == 0 means only check for existence. */ if (user_flags == 0) return (0); accmode = 0; if (user_flags & R_OK) accmode |= VREAD; if (user_flags & W_OK) accmode |= VWRITE; if (user_flags & X_OK) accmode |= VEXEC; #ifdef MAC error = mac_vnode_check_access(cred, vp, accmode); if (error != 0) return (error); #endif if ((accmode & VWRITE) == 0 || (error = vn_writechk(vp)) == 0) error = VOP_ACCESS(vp, accmode, cred, td); return (error); } /* * Check access permissions using "real" credentials. */ #ifndef _SYS_SYSPROTO_H_ struct access_args { char *path; int amode; }; #endif int sys_access(struct thread *td, struct access_args *uap) { return (kern_accessat(td, AT_FDCWD, uap->path, UIO_USERSPACE, 0, uap->amode)); } #ifndef _SYS_SYSPROTO_H_ struct faccessat_args { int dirfd; char *path; int amode; int flag; } #endif int sys_faccessat(struct thread *td, struct faccessat_args *uap) { return (kern_accessat(td, uap->fd, uap->path, UIO_USERSPACE, uap->flag, uap->amode)); } int kern_accessat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, int flag, int amode) { struct ucred *cred, *usecred; struct vnode *vp; struct nameidata nd; int error; if ((flag & ~(AT_EACCESS | AT_BENEATH)) != 0) return (EINVAL); if (amode != F_OK && (amode & ~(R_OK | W_OK | X_OK)) != 0) return (EINVAL); /* * Create and modify a temporary credential instead of one that * is potentially shared (if we need one). */ cred = td->td_ucred; if ((flag & AT_EACCESS) == 0 && ((cred->cr_uid != cred->cr_ruid || cred->cr_rgid != cred->cr_groups[0]))) { usecred = crdup(cred); usecred->cr_uid = cred->cr_ruid; usecred->cr_groups[0] = cred->cr_rgid; td->td_ucred = usecred; } else usecred = cred; AUDIT_ARG_VALUE(amode); NDINIT_ATRIGHTS(&nd, LOOKUP, FOLLOW | LOCKSHARED | LOCKLEAF | AUDITVNODE1 | ((flag & AT_BENEATH) != 0 ? BENEATH : 0), pathseg, path, fd, &cap_fstat_rights, td); if ((error = namei(&nd)) != 0) goto out; vp = nd.ni_vp; error = vn_access(vp, amode, usecred, td); NDFREE(&nd, NDF_ONLY_PNBUF); vput(vp); out: if (usecred != cred) { td->td_ucred = cred; crfree(usecred); } return (error); } /* * Check access permissions using "effective" credentials. */ #ifndef _SYS_SYSPROTO_H_ struct eaccess_args { char *path; int amode; }; #endif int sys_eaccess(struct thread *td, struct eaccess_args *uap) { return (kern_accessat(td, AT_FDCWD, uap->path, UIO_USERSPACE, AT_EACCESS, uap->amode)); } #if defined(COMPAT_43) /* * Get file status; this version follows links. */ #ifndef _SYS_SYSPROTO_H_ struct ostat_args { char *path; struct ostat *ub; }; #endif int ostat(struct thread *td, struct ostat_args *uap) { struct stat sb; struct ostat osb; int error; error = kern_statat(td, 0, AT_FDCWD, uap->path, UIO_USERSPACE, &sb, NULL); if (error != 0) return (error); cvtstat(&sb, &osb); return (copyout(&osb, uap->ub, sizeof (osb))); } /* * Get file status; this version does not follow links. */ #ifndef _SYS_SYSPROTO_H_ struct olstat_args { char *path; struct ostat *ub; }; #endif int olstat(struct thread *td, struct olstat_args *uap) { struct stat sb; struct ostat osb; int error; error = kern_statat(td, AT_SYMLINK_NOFOLLOW, AT_FDCWD, uap->path, UIO_USERSPACE, &sb, NULL); if (error != 0) return (error); cvtstat(&sb, &osb); return (copyout(&osb, uap->ub, sizeof (osb))); } /* * Convert from an old to a new stat structure. * XXX: many values are blindly truncated. */ void cvtstat(struct stat *st, struct ostat *ost) { bzero(ost, sizeof(*ost)); ost->st_dev = st->st_dev; ost->st_ino = st->st_ino; ost->st_mode = st->st_mode; ost->st_nlink = st->st_nlink; ost->st_uid = st->st_uid; ost->st_gid = st->st_gid; ost->st_rdev = st->st_rdev; ost->st_size = MIN(st->st_size, INT32_MAX); ost->st_atim = st->st_atim; ost->st_mtim = st->st_mtim; ost->st_ctim = st->st_ctim; ost->st_blksize = st->st_blksize; ost->st_blocks = st->st_blocks; ost->st_flags = st->st_flags; ost->st_gen = st->st_gen; } #endif /* COMPAT_43 */ #if defined(COMPAT_43) || defined(COMPAT_FREEBSD11) int ino64_trunc_error; SYSCTL_INT(_vfs, OID_AUTO, ino64_trunc_error, CTLFLAG_RW, &ino64_trunc_error, 0, "Error on truncation of device, file or inode number, or link count"); int freebsd11_cvtstat(struct stat *st, struct freebsd11_stat *ost) { ost->st_dev = st->st_dev; if (ost->st_dev != st->st_dev) { switch (ino64_trunc_error) { default: /* * Since dev_t is almost raw, don't clamp to the * maximum for case 2, but ignore the error. */ break; case 1: return (EOVERFLOW); } } ost->st_ino = st->st_ino; if (ost->st_ino != st->st_ino) { switch (ino64_trunc_error) { default: case 0: break; case 1: return (EOVERFLOW); case 2: ost->st_ino = UINT32_MAX; break; } } ost->st_mode = st->st_mode; ost->st_nlink = st->st_nlink; if (ost->st_nlink != st->st_nlink) { switch (ino64_trunc_error) { default: case 0: break; case 1: return (EOVERFLOW); case 2: ost->st_nlink = UINT16_MAX; break; } } ost->st_uid = st->st_uid; ost->st_gid = st->st_gid; ost->st_rdev = st->st_rdev; if (ost->st_rdev != st->st_rdev) { switch (ino64_trunc_error) { default: break; case 1: return (EOVERFLOW); } } ost->st_atim = st->st_atim; ost->st_mtim = st->st_mtim; ost->st_ctim = st->st_ctim; ost->st_size = st->st_size; ost->st_blocks = st->st_blocks; ost->st_blksize = st->st_blksize; ost->st_flags = st->st_flags; ost->st_gen = st->st_gen; ost->st_lspare = 0; ost->st_birthtim = st->st_birthtim; bzero((char *)&ost->st_birthtim + sizeof(ost->st_birthtim), sizeof(*ost) - offsetof(struct freebsd11_stat, st_birthtim) - sizeof(ost->st_birthtim)); return (0); } int freebsd11_stat(struct thread *td, struct freebsd11_stat_args* uap) { struct stat sb; struct freebsd11_stat osb; int error; error = kern_statat(td, 0, AT_FDCWD, uap->path, UIO_USERSPACE, &sb, NULL); if (error != 0) return (error); error = freebsd11_cvtstat(&sb, &osb); if (error == 0) error = copyout(&osb, uap->ub, sizeof(osb)); return (error); } int freebsd11_lstat(struct thread *td, struct freebsd11_lstat_args* uap) { struct stat sb; struct freebsd11_stat osb; int error; error = kern_statat(td, AT_SYMLINK_NOFOLLOW, AT_FDCWD, uap->path, UIO_USERSPACE, &sb, NULL); if (error != 0) return (error); error = freebsd11_cvtstat(&sb, &osb); if (error == 0) error = copyout(&osb, uap->ub, sizeof(osb)); return (error); } int freebsd11_fhstat(struct thread *td, struct freebsd11_fhstat_args* uap) { struct fhandle fh; struct stat sb; struct freebsd11_stat osb; int error; error = copyin(uap->u_fhp, &fh, sizeof(fhandle_t)); if (error != 0) return (error); error = kern_fhstat(td, fh, &sb); if (error != 0) return (error); error = freebsd11_cvtstat(&sb, &osb); if (error == 0) error = copyout(&osb, uap->sb, sizeof(osb)); return (error); } int freebsd11_fstatat(struct thread *td, struct freebsd11_fstatat_args* uap) { struct stat sb; struct freebsd11_stat osb; int error; error = kern_statat(td, uap->flag, uap->fd, uap->path, UIO_USERSPACE, &sb, NULL); if (error != 0) return (error); error = freebsd11_cvtstat(&sb, &osb); if (error == 0) error = copyout(&osb, uap->buf, sizeof(osb)); return (error); } #endif /* COMPAT_FREEBSD11 */ /* * Get file status */ #ifndef _SYS_SYSPROTO_H_ struct fstatat_args { int fd; char *path; struct stat *buf; int flag; } #endif int sys_fstatat(struct thread *td, struct fstatat_args *uap) { struct stat sb; int error; error = kern_statat(td, uap->flag, uap->fd, uap->path, UIO_USERSPACE, &sb, NULL); if (error == 0) error = copyout(&sb, uap->buf, sizeof (sb)); return (error); } int kern_statat(struct thread *td, int flag, int fd, const char *path, enum uio_seg pathseg, struct stat *sbp, void (*hook)(struct vnode *vp, struct stat *sbp)) { struct nameidata nd; int error; if ((flag & ~(AT_SYMLINK_NOFOLLOW | AT_BENEATH)) != 0) return (EINVAL); NDINIT_ATRIGHTS(&nd, LOOKUP, ((flag & AT_SYMLINK_NOFOLLOW) != 0 ? NOFOLLOW : FOLLOW) | ((flag & AT_BENEATH) != 0 ? BENEATH : 0) | LOCKSHARED | LOCKLEAF | AUDITVNODE1, pathseg, path, fd, &cap_fstat_rights, td); if ((error = namei(&nd)) != 0) return (error); error = vn_stat(nd.ni_vp, sbp, td->td_ucred, NOCRED, td); if (error == 0) { SDT_PROBE2(vfs, , stat, mode, path, sbp->st_mode); if (S_ISREG(sbp->st_mode)) SDT_PROBE2(vfs, , stat, reg, path, pathseg); if (__predict_false(hook != NULL)) hook(nd.ni_vp, sbp); } NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_vp); #ifdef __STAT_TIME_T_EXT sbp->st_atim_ext = 0; sbp->st_mtim_ext = 0; sbp->st_ctim_ext = 0; sbp->st_btim_ext = 0; #endif #ifdef KTRACE if (KTRPOINT(td, KTR_STRUCT)) ktrstat_error(sbp, error); #endif return (error); } #if defined(COMPAT_FREEBSD11) /* * Implementation of the NetBSD [l]stat() functions. */ void freebsd11_cvtnstat(struct stat *sb, struct nstat *nsb) { bzero(nsb, sizeof(*nsb)); nsb->st_dev = sb->st_dev; nsb->st_ino = sb->st_ino; nsb->st_mode = sb->st_mode; nsb->st_nlink = sb->st_nlink; nsb->st_uid = sb->st_uid; nsb->st_gid = sb->st_gid; nsb->st_rdev = sb->st_rdev; nsb->st_atim = sb->st_atim; nsb->st_mtim = sb->st_mtim; nsb->st_ctim = sb->st_ctim; nsb->st_size = sb->st_size; nsb->st_blocks = sb->st_blocks; nsb->st_blksize = sb->st_blksize; nsb->st_flags = sb->st_flags; nsb->st_gen = sb->st_gen; nsb->st_birthtim = sb->st_birthtim; } #ifndef _SYS_SYSPROTO_H_ struct freebsd11_nstat_args { char *path; struct nstat *ub; }; #endif int freebsd11_nstat(struct thread *td, struct freebsd11_nstat_args *uap) { struct stat sb; struct nstat nsb; int error; error = kern_statat(td, 0, AT_FDCWD, uap->path, UIO_USERSPACE, &sb, NULL); if (error != 0) return (error); freebsd11_cvtnstat(&sb, &nsb); return (copyout(&nsb, uap->ub, sizeof (nsb))); } /* * NetBSD lstat. Get file status; this version does not follow links. */ #ifndef _SYS_SYSPROTO_H_ struct freebsd11_nlstat_args { char *path; struct nstat *ub; }; #endif int freebsd11_nlstat(struct thread *td, struct freebsd11_nlstat_args *uap) { struct stat sb; struct nstat nsb; int error; error = kern_statat(td, AT_SYMLINK_NOFOLLOW, AT_FDCWD, uap->path, UIO_USERSPACE, &sb, NULL); if (error != 0) return (error); freebsd11_cvtnstat(&sb, &nsb); return (copyout(&nsb, uap->ub, sizeof (nsb))); } #endif /* COMPAT_FREEBSD11 */ /* * Get configurable pathname variables. */ #ifndef _SYS_SYSPROTO_H_ struct pathconf_args { char *path; int name; }; #endif int sys_pathconf(struct thread *td, struct pathconf_args *uap) { long value; int error; error = kern_pathconf(td, uap->path, UIO_USERSPACE, uap->name, FOLLOW, &value); if (error == 0) td->td_retval[0] = value; return (error); } #ifndef _SYS_SYSPROTO_H_ struct lpathconf_args { char *path; int name; }; #endif int sys_lpathconf(struct thread *td, struct lpathconf_args *uap) { long value; int error; error = kern_pathconf(td, uap->path, UIO_USERSPACE, uap->name, NOFOLLOW, &value); if (error == 0) td->td_retval[0] = value; return (error); } int kern_pathconf(struct thread *td, const char *path, enum uio_seg pathseg, int name, u_long flags, long *valuep) { struct nameidata nd; int error; NDINIT(&nd, LOOKUP, LOCKSHARED | LOCKLEAF | AUDITVNODE1 | flags, pathseg, path, td); if ((error = namei(&nd)) != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = VOP_PATHCONF(nd.ni_vp, name, valuep); vput(nd.ni_vp); return (error); } /* * Return target name of a symbolic link. */ #ifndef _SYS_SYSPROTO_H_ struct readlink_args { char *path; char *buf; size_t count; }; #endif int sys_readlink(struct thread *td, struct readlink_args *uap) { return (kern_readlinkat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->buf, UIO_USERSPACE, uap->count)); } #ifndef _SYS_SYSPROTO_H_ struct readlinkat_args { int fd; char *path; char *buf; size_t bufsize; }; #endif int sys_readlinkat(struct thread *td, struct readlinkat_args *uap) { return (kern_readlinkat(td, uap->fd, uap->path, UIO_USERSPACE, uap->buf, UIO_USERSPACE, uap->bufsize)); } int kern_readlinkat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, char *buf, enum uio_seg bufseg, size_t count) { struct vnode *vp; struct nameidata nd; int error; if (count > IOSIZE_MAX) return (EINVAL); NDINIT_AT(&nd, LOOKUP, NOFOLLOW | LOCKSHARED | LOCKLEAF | AUDITVNODE1, pathseg, path, fd, td); if ((error = namei(&nd)) != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); vp = nd.ni_vp; error = kern_readlink_vp(vp, buf, bufseg, count, td); vput(vp); return (error); } /* * Helper function to readlink from a vnode */ static int kern_readlink_vp(struct vnode *vp, char *buf, enum uio_seg bufseg, size_t count, struct thread *td) { struct iovec aiov; struct uio auio; int error; ASSERT_VOP_LOCKED(vp, "kern_readlink_vp(): vp not locked"); #ifdef MAC error = mac_vnode_check_readlink(td->td_ucred, vp); if (error != 0) return (error); #endif if (vp->v_type != VLNK && (vp->v_vflag & VV_READLINK) == 0) return (EINVAL); aiov.iov_base = buf; aiov.iov_len = count; auio.uio_iov = &aiov; auio.uio_iovcnt = 1; auio.uio_offset = 0; auio.uio_rw = UIO_READ; auio.uio_segflg = bufseg; auio.uio_td = td; auio.uio_resid = count; error = VOP_READLINK(vp, &auio, td->td_ucred); td->td_retval[0] = count - auio.uio_resid; return (error); } /* * Common implementation code for chflags() and fchflags(). */ static int setfflags(struct thread *td, struct vnode *vp, u_long flags) { struct mount *mp; struct vattr vattr; int error; /* We can't support the value matching VNOVAL. */ if (flags == VNOVAL) return (EOPNOTSUPP); /* * Prevent non-root users from setting flags on devices. When * a device is reused, users can retain ownership of the device * if they are allowed to set flags and programs assume that * chown can't fail when done as root. */ if (vp->v_type == VCHR || vp->v_type == VBLK) { error = priv_check(td, PRIV_VFS_CHFLAGS_DEV); if (error != 0) return (error); } if ((error = vn_start_write(vp, &mp, V_WAIT | PCATCH)) != 0) return (error); VATTR_NULL(&vattr); vattr.va_flags = flags; vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); #ifdef MAC error = mac_vnode_check_setflags(td->td_ucred, vp, vattr.va_flags); if (error == 0) #endif error = VOP_SETATTR(vp, &vattr, td->td_ucred); VOP_UNLOCK(vp); vn_finished_write(mp); return (error); } /* * Change flags of a file given a path name. */ #ifndef _SYS_SYSPROTO_H_ struct chflags_args { const char *path; u_long flags; }; #endif int sys_chflags(struct thread *td, struct chflags_args *uap) { return (kern_chflagsat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->flags, 0)); } #ifndef _SYS_SYSPROTO_H_ struct chflagsat_args { int fd; const char *path; u_long flags; int atflag; } #endif int sys_chflagsat(struct thread *td, struct chflagsat_args *uap) { if ((uap->atflag & ~(AT_SYMLINK_NOFOLLOW | AT_BENEATH)) != 0) return (EINVAL); return (kern_chflagsat(td, uap->fd, uap->path, UIO_USERSPACE, uap->flags, uap->atflag)); } /* * Same as chflags() but doesn't follow symlinks. */ #ifndef _SYS_SYSPROTO_H_ struct lchflags_args { const char *path; u_long flags; }; #endif int sys_lchflags(struct thread *td, struct lchflags_args *uap) { return (kern_chflagsat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->flags, AT_SYMLINK_NOFOLLOW)); } static int kern_chflagsat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, u_long flags, int atflag) { struct nameidata nd; int error, follow; AUDIT_ARG_FFLAGS(flags); follow = (atflag & AT_SYMLINK_NOFOLLOW) ? NOFOLLOW : FOLLOW; follow |= (atflag & AT_BENEATH) != 0 ? BENEATH : 0; NDINIT_ATRIGHTS(&nd, LOOKUP, follow | AUDITVNODE1, pathseg, path, fd, &cap_fchflags_rights, td); if ((error = namei(&nd)) != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = setfflags(td, nd.ni_vp, flags); vrele(nd.ni_vp); return (error); } /* * Change flags of a file given a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct fchflags_args { int fd; u_long flags; }; #endif int sys_fchflags(struct thread *td, struct fchflags_args *uap) { struct file *fp; int error; AUDIT_ARG_FD(uap->fd); AUDIT_ARG_FFLAGS(uap->flags); error = getvnode(td, uap->fd, &cap_fchflags_rights, &fp); if (error != 0) return (error); #ifdef AUDIT vn_lock(fp->f_vnode, LK_SHARED | LK_RETRY); AUDIT_ARG_VNODE1(fp->f_vnode); VOP_UNLOCK(fp->f_vnode); #endif error = setfflags(td, fp->f_vnode, uap->flags); fdrop(fp, td); return (error); } /* * Common implementation code for chmod(), lchmod() and fchmod(). */ int setfmode(struct thread *td, struct ucred *cred, struct vnode *vp, int mode) { struct mount *mp; struct vattr vattr; int error; if ((error = vn_start_write(vp, &mp, V_WAIT | PCATCH)) != 0) return (error); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); VATTR_NULL(&vattr); vattr.va_mode = mode & ALLPERMS; #ifdef MAC error = mac_vnode_check_setmode(cred, vp, vattr.va_mode); if (error == 0) #endif error = VOP_SETATTR(vp, &vattr, cred); VOP_UNLOCK(vp); vn_finished_write(mp); return (error); } /* * Change mode of a file given path name. */ #ifndef _SYS_SYSPROTO_H_ struct chmod_args { char *path; int mode; }; #endif int sys_chmod(struct thread *td, struct chmod_args *uap) { return (kern_fchmodat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->mode, 0)); } #ifndef _SYS_SYSPROTO_H_ struct fchmodat_args { int dirfd; char *path; mode_t mode; int flag; } #endif int sys_fchmodat(struct thread *td, struct fchmodat_args *uap) { if ((uap->flag & ~(AT_SYMLINK_NOFOLLOW | AT_BENEATH)) != 0) return (EINVAL); return (kern_fchmodat(td, uap->fd, uap->path, UIO_USERSPACE, uap->mode, uap->flag)); } /* * Change mode of a file given path name (don't follow links.) */ #ifndef _SYS_SYSPROTO_H_ struct lchmod_args { char *path; int mode; }; #endif int sys_lchmod(struct thread *td, struct lchmod_args *uap) { return (kern_fchmodat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->mode, AT_SYMLINK_NOFOLLOW)); } int kern_fchmodat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, mode_t mode, int flag) { struct nameidata nd; int error, follow; AUDIT_ARG_MODE(mode); follow = (flag & AT_SYMLINK_NOFOLLOW) != 0 ? NOFOLLOW : FOLLOW; follow |= (flag & AT_BENEATH) != 0 ? BENEATH : 0; NDINIT_ATRIGHTS(&nd, LOOKUP, follow | AUDITVNODE1, pathseg, path, fd, &cap_fchmod_rights, td); if ((error = namei(&nd)) != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = setfmode(td, td->td_ucred, nd.ni_vp, mode); vrele(nd.ni_vp); return (error); } /* * Change mode of a file given a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct fchmod_args { int fd; int mode; }; #endif int sys_fchmod(struct thread *td, struct fchmod_args *uap) { struct file *fp; int error; AUDIT_ARG_FD(uap->fd); AUDIT_ARG_MODE(uap->mode); error = fget(td, uap->fd, &cap_fchmod_rights, &fp); if (error != 0) return (error); error = fo_chmod(fp, uap->mode, td->td_ucred, td); fdrop(fp, td); return (error); } /* * Common implementation for chown(), lchown(), and fchown() */ int setfown(struct thread *td, struct ucred *cred, struct vnode *vp, uid_t uid, gid_t gid) { struct mount *mp; struct vattr vattr; int error; if ((error = vn_start_write(vp, &mp, V_WAIT | PCATCH)) != 0) return (error); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); VATTR_NULL(&vattr); vattr.va_uid = uid; vattr.va_gid = gid; #ifdef MAC error = mac_vnode_check_setowner(cred, vp, vattr.va_uid, vattr.va_gid); if (error == 0) #endif error = VOP_SETATTR(vp, &vattr, cred); VOP_UNLOCK(vp); vn_finished_write(mp); return (error); } /* * Set ownership given a path name. */ #ifndef _SYS_SYSPROTO_H_ struct chown_args { char *path; int uid; int gid; }; #endif int sys_chown(struct thread *td, struct chown_args *uap) { return (kern_fchownat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->uid, uap->gid, 0)); } #ifndef _SYS_SYSPROTO_H_ struct fchownat_args { int fd; const char * path; uid_t uid; gid_t gid; int flag; }; #endif int sys_fchownat(struct thread *td, struct fchownat_args *uap) { if ((uap->flag & ~(AT_SYMLINK_NOFOLLOW | AT_BENEATH)) != 0) return (EINVAL); return (kern_fchownat(td, uap->fd, uap->path, UIO_USERSPACE, uap->uid, uap->gid, uap->flag)); } int kern_fchownat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, int uid, int gid, int flag) { struct nameidata nd; int error, follow; AUDIT_ARG_OWNER(uid, gid); follow = (flag & AT_SYMLINK_NOFOLLOW) ? NOFOLLOW : FOLLOW; follow |= (flag & AT_BENEATH) != 0 ? BENEATH : 0; NDINIT_ATRIGHTS(&nd, LOOKUP, follow | AUDITVNODE1, pathseg, path, fd, &cap_fchown_rights, td); if ((error = namei(&nd)) != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = setfown(td, td->td_ucred, nd.ni_vp, uid, gid); vrele(nd.ni_vp); return (error); } /* * Set ownership given a path name, do not cross symlinks. */ #ifndef _SYS_SYSPROTO_H_ struct lchown_args { char *path; int uid; int gid; }; #endif int sys_lchown(struct thread *td, struct lchown_args *uap) { return (kern_fchownat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->uid, uap->gid, AT_SYMLINK_NOFOLLOW)); } /* * Set ownership given a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct fchown_args { int fd; int uid; int gid; }; #endif int sys_fchown(struct thread *td, struct fchown_args *uap) { struct file *fp; int error; AUDIT_ARG_FD(uap->fd); AUDIT_ARG_OWNER(uap->uid, uap->gid); error = fget(td, uap->fd, &cap_fchown_rights, &fp); if (error != 0) return (error); error = fo_chown(fp, uap->uid, uap->gid, td->td_ucred, td); fdrop(fp, td); return (error); } /* * Common implementation code for utimes(), lutimes(), and futimes(). */ static int getutimes(const struct timeval *usrtvp, enum uio_seg tvpseg, struct timespec *tsp) { struct timeval tv[2]; const struct timeval *tvp; int error; if (usrtvp == NULL) { vfs_timestamp(&tsp[0]); tsp[1] = tsp[0]; } else { if (tvpseg == UIO_SYSSPACE) { tvp = usrtvp; } else { if ((error = copyin(usrtvp, tv, sizeof(tv))) != 0) return (error); tvp = tv; } if (tvp[0].tv_usec < 0 || tvp[0].tv_usec >= 1000000 || tvp[1].tv_usec < 0 || tvp[1].tv_usec >= 1000000) return (EINVAL); TIMEVAL_TO_TIMESPEC(&tvp[0], &tsp[0]); TIMEVAL_TO_TIMESPEC(&tvp[1], &tsp[1]); } return (0); } /* * Common implementation code for futimens(), utimensat(). */ #define UTIMENS_NULL 0x1 #define UTIMENS_EXIT 0x2 static int getutimens(const struct timespec *usrtsp, enum uio_seg tspseg, struct timespec *tsp, int *retflags) { struct timespec tsnow; int error; vfs_timestamp(&tsnow); *retflags = 0; if (usrtsp == NULL) { tsp[0] = tsnow; tsp[1] = tsnow; *retflags |= UTIMENS_NULL; return (0); } if (tspseg == UIO_SYSSPACE) { tsp[0] = usrtsp[0]; tsp[1] = usrtsp[1]; } else if ((error = copyin(usrtsp, tsp, sizeof(*tsp) * 2)) != 0) return (error); if (tsp[0].tv_nsec == UTIME_OMIT && tsp[1].tv_nsec == UTIME_OMIT) *retflags |= UTIMENS_EXIT; if (tsp[0].tv_nsec == UTIME_NOW && tsp[1].tv_nsec == UTIME_NOW) *retflags |= UTIMENS_NULL; if (tsp[0].tv_nsec == UTIME_OMIT) tsp[0].tv_sec = VNOVAL; else if (tsp[0].tv_nsec == UTIME_NOW) tsp[0] = tsnow; else if (tsp[0].tv_nsec < 0 || tsp[0].tv_nsec >= 1000000000L) return (EINVAL); if (tsp[1].tv_nsec == UTIME_OMIT) tsp[1].tv_sec = VNOVAL; else if (tsp[1].tv_nsec == UTIME_NOW) tsp[1] = tsnow; else if (tsp[1].tv_nsec < 0 || tsp[1].tv_nsec >= 1000000000L) return (EINVAL); return (0); } /* * Common implementation code for utimes(), lutimes(), futimes(), futimens(), * and utimensat(). */ static int setutimes(struct thread *td, struct vnode *vp, const struct timespec *ts, int numtimes, int nullflag) { struct mount *mp; struct vattr vattr; int error, setbirthtime; if ((error = vn_start_write(vp, &mp, V_WAIT | PCATCH)) != 0) return (error); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); setbirthtime = 0; if (numtimes < 3 && !VOP_GETATTR(vp, &vattr, td->td_ucred) && timespeccmp(&ts[1], &vattr.va_birthtime, < )) setbirthtime = 1; VATTR_NULL(&vattr); vattr.va_atime = ts[0]; vattr.va_mtime = ts[1]; if (setbirthtime) vattr.va_birthtime = ts[1]; if (numtimes > 2) vattr.va_birthtime = ts[2]; if (nullflag) vattr.va_vaflags |= VA_UTIMES_NULL; #ifdef MAC error = mac_vnode_check_setutimes(td->td_ucred, vp, vattr.va_atime, vattr.va_mtime); #endif if (error == 0) error = VOP_SETATTR(vp, &vattr, td->td_ucred); VOP_UNLOCK(vp); vn_finished_write(mp); return (error); } /* * Set the access and modification times of a file. */ #ifndef _SYS_SYSPROTO_H_ struct utimes_args { char *path; struct timeval *tptr; }; #endif int sys_utimes(struct thread *td, struct utimes_args *uap) { return (kern_utimesat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->tptr, UIO_USERSPACE)); } #ifndef _SYS_SYSPROTO_H_ struct futimesat_args { int fd; const char * path; const struct timeval * times; }; #endif int sys_futimesat(struct thread *td, struct futimesat_args *uap) { return (kern_utimesat(td, uap->fd, uap->path, UIO_USERSPACE, uap->times, UIO_USERSPACE)); } int kern_utimesat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, struct timeval *tptr, enum uio_seg tptrseg) { struct nameidata nd; struct timespec ts[2]; int error; if ((error = getutimes(tptr, tptrseg, ts)) != 0) return (error); NDINIT_ATRIGHTS(&nd, LOOKUP, FOLLOW | AUDITVNODE1, pathseg, path, fd, &cap_futimes_rights, td); if ((error = namei(&nd)) != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = setutimes(td, nd.ni_vp, ts, 2, tptr == NULL); vrele(nd.ni_vp); return (error); } /* * Set the access and modification times of a file. */ #ifndef _SYS_SYSPROTO_H_ struct lutimes_args { char *path; struct timeval *tptr; }; #endif int sys_lutimes(struct thread *td, struct lutimes_args *uap) { return (kern_lutimes(td, uap->path, UIO_USERSPACE, uap->tptr, UIO_USERSPACE)); } int kern_lutimes(struct thread *td, const char *path, enum uio_seg pathseg, struct timeval *tptr, enum uio_seg tptrseg) { struct timespec ts[2]; struct nameidata nd; int error; if ((error = getutimes(tptr, tptrseg, ts)) != 0) return (error); NDINIT(&nd, LOOKUP, NOFOLLOW | AUDITVNODE1, pathseg, path, td); if ((error = namei(&nd)) != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); error = setutimes(td, nd.ni_vp, ts, 2, tptr == NULL); vrele(nd.ni_vp); return (error); } /* * Set the access and modification times of a file. */ #ifndef _SYS_SYSPROTO_H_ struct futimes_args { int fd; struct timeval *tptr; }; #endif int sys_futimes(struct thread *td, struct futimes_args *uap) { return (kern_futimes(td, uap->fd, uap->tptr, UIO_USERSPACE)); } int kern_futimes(struct thread *td, int fd, struct timeval *tptr, enum uio_seg tptrseg) { struct timespec ts[2]; struct file *fp; int error; AUDIT_ARG_FD(fd); error = getutimes(tptr, tptrseg, ts); if (error != 0) return (error); error = getvnode(td, fd, &cap_futimes_rights, &fp); if (error != 0) return (error); #ifdef AUDIT vn_lock(fp->f_vnode, LK_SHARED | LK_RETRY); AUDIT_ARG_VNODE1(fp->f_vnode); VOP_UNLOCK(fp->f_vnode); #endif error = setutimes(td, fp->f_vnode, ts, 2, tptr == NULL); fdrop(fp, td); return (error); } int sys_futimens(struct thread *td, struct futimens_args *uap) { return (kern_futimens(td, uap->fd, uap->times, UIO_USERSPACE)); } int kern_futimens(struct thread *td, int fd, struct timespec *tptr, enum uio_seg tptrseg) { struct timespec ts[2]; struct file *fp; int error, flags; AUDIT_ARG_FD(fd); error = getutimens(tptr, tptrseg, ts, &flags); if (error != 0) return (error); if (flags & UTIMENS_EXIT) return (0); error = getvnode(td, fd, &cap_futimes_rights, &fp); if (error != 0) return (error); #ifdef AUDIT vn_lock(fp->f_vnode, LK_SHARED | LK_RETRY); AUDIT_ARG_VNODE1(fp->f_vnode); VOP_UNLOCK(fp->f_vnode); #endif error = setutimes(td, fp->f_vnode, ts, 2, flags & UTIMENS_NULL); fdrop(fp, td); return (error); } int sys_utimensat(struct thread *td, struct utimensat_args *uap) { return (kern_utimensat(td, uap->fd, uap->path, UIO_USERSPACE, uap->times, UIO_USERSPACE, uap->flag)); } int kern_utimensat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, struct timespec *tptr, enum uio_seg tptrseg, int flag) { struct nameidata nd; struct timespec ts[2]; int error, flags; if ((flag & ~(AT_SYMLINK_NOFOLLOW | AT_BENEATH)) != 0) return (EINVAL); if ((error = getutimens(tptr, tptrseg, ts, &flags)) != 0) return (error); NDINIT_ATRIGHTS(&nd, LOOKUP, ((flag & AT_SYMLINK_NOFOLLOW) ? NOFOLLOW : FOLLOW) | ((flag & AT_BENEATH) != 0 ? BENEATH : 0) | AUDITVNODE1, pathseg, path, fd, &cap_futimes_rights, td); if ((error = namei(&nd)) != 0) return (error); /* * We are allowed to call namei() regardless of 2xUTIME_OMIT. * POSIX states: * "If both tv_nsec fields are UTIME_OMIT... EACCESS may be detected." * "Search permission is denied by a component of the path prefix." */ NDFREE(&nd, NDF_ONLY_PNBUF); if ((flags & UTIMENS_EXIT) == 0) error = setutimes(td, nd.ni_vp, ts, 2, flags & UTIMENS_NULL); vrele(nd.ni_vp); return (error); } /* * Truncate a file given its path name. */ #ifndef _SYS_SYSPROTO_H_ struct truncate_args { char *path; int pad; off_t length; }; #endif int sys_truncate(struct thread *td, struct truncate_args *uap) { return (kern_truncate(td, uap->path, UIO_USERSPACE, uap->length)); } int kern_truncate(struct thread *td, const char *path, enum uio_seg pathseg, off_t length) { struct mount *mp; struct vnode *vp; void *rl_cookie; struct vattr vattr; struct nameidata nd; int error; if (length < 0) return(EINVAL); NDINIT(&nd, LOOKUP, FOLLOW | AUDITVNODE1, pathseg, path, td); if ((error = namei(&nd)) != 0) return (error); vp = nd.ni_vp; rl_cookie = vn_rangelock_wlock(vp, 0, OFF_MAX); if ((error = vn_start_write(vp, &mp, V_WAIT | PCATCH)) != 0) { vn_rangelock_unlock(vp, rl_cookie); vrele(vp); return (error); } NDFREE(&nd, NDF_ONLY_PNBUF); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); if (vp->v_type == VDIR) error = EISDIR; #ifdef MAC else if ((error = mac_vnode_check_write(td->td_ucred, NOCRED, vp))) { } #endif else if ((error = vn_writechk(vp)) == 0 && (error = VOP_ACCESS(vp, VWRITE, td->td_ucred, td)) == 0) { VATTR_NULL(&vattr); vattr.va_size = length; error = VOP_SETATTR(vp, &vattr, td->td_ucred); } VOP_UNLOCK(vp); vn_finished_write(mp); vn_rangelock_unlock(vp, rl_cookie); vrele(vp); return (error); } #if defined(COMPAT_43) /* * Truncate a file given its path name. */ #ifndef _SYS_SYSPROTO_H_ struct otruncate_args { char *path; long length; }; #endif int otruncate(struct thread *td, struct otruncate_args *uap) { return (kern_truncate(td, uap->path, UIO_USERSPACE, uap->length)); } #endif /* COMPAT_43 */ #if defined(COMPAT_FREEBSD6) /* Versions with the pad argument */ int freebsd6_truncate(struct thread *td, struct freebsd6_truncate_args *uap) { return (kern_truncate(td, uap->path, UIO_USERSPACE, uap->length)); } int freebsd6_ftruncate(struct thread *td, struct freebsd6_ftruncate_args *uap) { return (kern_ftruncate(td, uap->fd, uap->length)); } #endif int kern_fsync(struct thread *td, int fd, bool fullsync) { struct vnode *vp; struct mount *mp; struct file *fp; int error, lock_flags; AUDIT_ARG_FD(fd); error = getvnode(td, fd, &cap_fsync_rights, &fp); if (error != 0) return (error); vp = fp->f_vnode; #if 0 if (!fullsync) /* XXXKIB: compete outstanding aio writes */; #endif error = vn_start_write(vp, &mp, V_WAIT | PCATCH); if (error != 0) goto drop; if (MNT_SHARED_WRITES(mp) || ((mp == NULL) && MNT_SHARED_WRITES(vp->v_mount))) { lock_flags = LK_SHARED; } else { lock_flags = LK_EXCLUSIVE; } vn_lock(vp, lock_flags | LK_RETRY); AUDIT_ARG_VNODE1(vp); if (vp->v_object != NULL) { VM_OBJECT_WLOCK(vp->v_object); vm_object_page_clean(vp->v_object, 0, 0, 0); VM_OBJECT_WUNLOCK(vp->v_object); } error = fullsync ? VOP_FSYNC(vp, MNT_WAIT, td) : VOP_FDATASYNC(vp, td); VOP_UNLOCK(vp); vn_finished_write(mp); drop: fdrop(fp, td); return (error); } /* * Sync an open file. */ #ifndef _SYS_SYSPROTO_H_ struct fsync_args { int fd; }; #endif int sys_fsync(struct thread *td, struct fsync_args *uap) { return (kern_fsync(td, uap->fd, true)); } int sys_fdatasync(struct thread *td, struct fdatasync_args *uap) { return (kern_fsync(td, uap->fd, false)); } /* * Rename files. Source and destination must either both be directories, or * both not be directories. If target is a directory, it must be empty. */ #ifndef _SYS_SYSPROTO_H_ struct rename_args { char *from; char *to; }; #endif int sys_rename(struct thread *td, struct rename_args *uap) { return (kern_renameat(td, AT_FDCWD, uap->from, AT_FDCWD, uap->to, UIO_USERSPACE)); } #ifndef _SYS_SYSPROTO_H_ struct renameat_args { int oldfd; char *old; int newfd; char *new; }; #endif int sys_renameat(struct thread *td, struct renameat_args *uap) { return (kern_renameat(td, uap->oldfd, uap->old, uap->newfd, uap->new, UIO_USERSPACE)); } int kern_renameat(struct thread *td, int oldfd, const char *old, int newfd, const char *new, enum uio_seg pathseg) { struct mount *mp = NULL; struct vnode *tvp, *fvp, *tdvp; struct nameidata fromnd, tond; int error; again: bwillwrite(); #ifdef MAC NDINIT_ATRIGHTS(&fromnd, DELETE, LOCKPARENT | LOCKLEAF | SAVESTART | AUDITVNODE1, pathseg, old, oldfd, &cap_renameat_source_rights, td); #else NDINIT_ATRIGHTS(&fromnd, DELETE, WANTPARENT | SAVESTART | AUDITVNODE1, pathseg, old, oldfd, &cap_renameat_source_rights, td); #endif if ((error = namei(&fromnd)) != 0) return (error); #ifdef MAC error = mac_vnode_check_rename_from(td->td_ucred, fromnd.ni_dvp, fromnd.ni_vp, &fromnd.ni_cnd); VOP_UNLOCK(fromnd.ni_dvp); if (fromnd.ni_dvp != fromnd.ni_vp) VOP_UNLOCK(fromnd.ni_vp); #endif fvp = fromnd.ni_vp; NDINIT_ATRIGHTS(&tond, RENAME, LOCKPARENT | LOCKLEAF | NOCACHE | SAVESTART | AUDITVNODE2, pathseg, new, newfd, &cap_renameat_target_rights, td); if (fromnd.ni_vp->v_type == VDIR) tond.ni_cnd.cn_flags |= WILLBEDIR; if ((error = namei(&tond)) != 0) { /* Translate error code for rename("dir1", "dir2/."). */ if (error == EISDIR && fvp->v_type == VDIR) error = EINVAL; NDFREE(&fromnd, NDF_ONLY_PNBUF); vrele(fromnd.ni_dvp); vrele(fvp); goto out1; } tdvp = tond.ni_dvp; tvp = tond.ni_vp; error = vn_start_write(fvp, &mp, V_NOWAIT); if (error != 0) { NDFREE(&fromnd, NDF_ONLY_PNBUF); NDFREE(&tond, NDF_ONLY_PNBUF); if (tvp != NULL) vput(tvp); if (tdvp == tvp) vrele(tdvp); else vput(tdvp); vrele(fromnd.ni_dvp); vrele(fvp); vrele(tond.ni_startdir); if (fromnd.ni_startdir != NULL) vrele(fromnd.ni_startdir); error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH); if (error != 0) return (error); goto again; } if (tvp != NULL) { if (fvp->v_type == VDIR && tvp->v_type != VDIR) { error = ENOTDIR; goto out; } else if (fvp->v_type != VDIR && tvp->v_type == VDIR) { error = EISDIR; goto out; } #ifdef CAPABILITIES if (newfd != AT_FDCWD && (tond.ni_resflags & NIRES_ABS) == 0) { /* * If the target already exists we require CAP_UNLINKAT * from 'newfd', when newfd was used for the lookup. */ error = cap_check(&tond.ni_filecaps.fc_rights, &cap_unlinkat_rights); if (error != 0) goto out; } #endif } if (fvp == tdvp) { error = EINVAL; goto out; } /* * If the source is the same as the destination (that is, if they * are links to the same vnode), then there is nothing to do. */ if (fvp == tvp) error = -1; #ifdef MAC else error = mac_vnode_check_rename_to(td->td_ucred, tdvp, tond.ni_vp, fromnd.ni_dvp == tdvp, &tond.ni_cnd); #endif out: if (error == 0) { error = VOP_RENAME(fromnd.ni_dvp, fromnd.ni_vp, &fromnd.ni_cnd, tond.ni_dvp, tond.ni_vp, &tond.ni_cnd); NDFREE(&fromnd, NDF_ONLY_PNBUF); NDFREE(&tond, NDF_ONLY_PNBUF); } else { NDFREE(&fromnd, NDF_ONLY_PNBUF); NDFREE(&tond, NDF_ONLY_PNBUF); if (tvp != NULL) vput(tvp); if (tdvp == tvp) vrele(tdvp); else vput(tdvp); vrele(fromnd.ni_dvp); vrele(fvp); } vrele(tond.ni_startdir); vn_finished_write(mp); out1: if (fromnd.ni_startdir) vrele(fromnd.ni_startdir); if (error == -1) return (0); return (error); } /* * Make a directory file. */ #ifndef _SYS_SYSPROTO_H_ struct mkdir_args { char *path; int mode; }; #endif int sys_mkdir(struct thread *td, struct mkdir_args *uap) { return (kern_mkdirat(td, AT_FDCWD, uap->path, UIO_USERSPACE, uap->mode)); } #ifndef _SYS_SYSPROTO_H_ struct mkdirat_args { int fd; char *path; mode_t mode; }; #endif int sys_mkdirat(struct thread *td, struct mkdirat_args *uap) { return (kern_mkdirat(td, uap->fd, uap->path, UIO_USERSPACE, uap->mode)); } int kern_mkdirat(struct thread *td, int fd, const char *path, enum uio_seg segflg, int mode) { struct mount *mp; struct vnode *vp; struct vattr vattr; struct nameidata nd; int error; AUDIT_ARG_MODE(mode); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, CREATE, LOCKPARENT | SAVENAME | AUDITVNODE1 | NOCACHE, segflg, path, fd, &cap_mkdirat_rights, td); nd.ni_cnd.cn_flags |= WILLBEDIR; if ((error = namei(&nd)) != 0) return (error); vp = nd.ni_vp; if (vp != NULL) { NDFREE(&nd, NDF_ONLY_PNBUF); /* * XXX namei called with LOCKPARENT but not LOCKLEAF has * the strange behaviour of leaving the vnode unlocked * if the target is the same vnode as the parent. */ if (vp == nd.ni_dvp) vrele(nd.ni_dvp); else vput(nd.ni_dvp); vrele(vp); return (EEXIST); } if (vn_start_write(nd.ni_dvp, &mp, V_NOWAIT) != 0) { NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); if ((error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH)) != 0) return (error); goto restart; } VATTR_NULL(&vattr); vattr.va_type = VDIR; vattr.va_mode = (mode & ACCESSPERMS) &~ td->td_proc->p_fd->fd_cmask; #ifdef MAC error = mac_vnode_check_create(td->td_ucred, nd.ni_dvp, &nd.ni_cnd, &vattr); if (error != 0) goto out; #endif error = VOP_MKDIR(nd.ni_dvp, &nd.ni_vp, &nd.ni_cnd, &vattr); #ifdef MAC out: #endif NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); if (error == 0) vput(nd.ni_vp); vn_finished_write(mp); return (error); } /* * Remove a directory file. */ #ifndef _SYS_SYSPROTO_H_ struct rmdir_args { char *path; }; #endif int sys_rmdir(struct thread *td, struct rmdir_args *uap) { return (kern_frmdirat(td, AT_FDCWD, uap->path, FD_NONE, UIO_USERSPACE, 0)); } int kern_frmdirat(struct thread *td, int dfd, const char *path, int fd, enum uio_seg pathseg, int flag) { struct mount *mp; struct vnode *vp; struct file *fp; struct nameidata nd; cap_rights_t rights; int error; fp = NULL; if (fd != FD_NONE) { - error = getvnode(td, fd, cap_rights_init(&rights, CAP_LOOKUP), + error = getvnode(td, fd, cap_rights_init_one(&rights, CAP_LOOKUP), &fp); if (error != 0) return (error); } restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, DELETE, LOCKPARENT | LOCKLEAF | AUDITVNODE1 | ((flag & AT_BENEATH) != 0 ? BENEATH : 0), pathseg, path, dfd, &cap_unlinkat_rights, td); if ((error = namei(&nd)) != 0) goto fdout; vp = nd.ni_vp; if (vp->v_type != VDIR) { error = ENOTDIR; goto out; } /* * No rmdir "." please. */ if (nd.ni_dvp == vp) { error = EINVAL; goto out; } /* * The root of a mounted filesystem cannot be deleted. */ if (vp->v_vflag & VV_ROOT) { error = EBUSY; goto out; } if (fp != NULL && fp->f_vnode != vp) { if (VN_IS_DOOMED(fp->f_vnode)) error = EBADF; else error = EDEADLK; goto out; } #ifdef MAC error = mac_vnode_check_unlink(td->td_ucred, nd.ni_dvp, vp, &nd.ni_cnd); if (error != 0) goto out; #endif if (vn_start_write(nd.ni_dvp, &mp, V_NOWAIT) != 0) { NDFREE(&nd, NDF_ONLY_PNBUF); vput(vp); if (nd.ni_dvp == vp) vrele(nd.ni_dvp); else vput(nd.ni_dvp); if ((error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH)) != 0) goto fdout; goto restart; } vfs_notify_upper(vp, VFS_NOTIFY_UPPER_UNLINK); error = VOP_RMDIR(nd.ni_dvp, nd.ni_vp, &nd.ni_cnd); vn_finished_write(mp); out: NDFREE(&nd, NDF_ONLY_PNBUF); vput(vp); if (nd.ni_dvp == vp) vrele(nd.ni_dvp); else vput(nd.ni_dvp); fdout: if (fp != NULL) fdrop(fp, td); return (error); } #if defined(COMPAT_43) || defined(COMPAT_FREEBSD11) int freebsd11_kern_getdirentries(struct thread *td, int fd, char *ubuf, u_int count, long *basep, void (*func)(struct freebsd11_dirent *)) { struct freebsd11_dirent dstdp; struct dirent *dp, *edp; char *dirbuf; off_t base; ssize_t resid, ucount; int error; /* XXX arbitrary sanity limit on `count'. */ count = min(count, 64 * 1024); dirbuf = malloc(count, M_TEMP, M_WAITOK); error = kern_getdirentries(td, fd, dirbuf, count, &base, &resid, UIO_SYSSPACE); if (error != 0) goto done; if (basep != NULL) *basep = base; ucount = 0; for (dp = (struct dirent *)dirbuf, edp = (struct dirent *)&dirbuf[count - resid]; ucount < count && dp < edp; ) { if (dp->d_reclen == 0) break; MPASS(dp->d_reclen >= _GENERIC_DIRLEN(0)); if (dp->d_namlen >= sizeof(dstdp.d_name)) continue; dstdp.d_type = dp->d_type; dstdp.d_namlen = dp->d_namlen; dstdp.d_fileno = dp->d_fileno; /* truncate */ if (dstdp.d_fileno != dp->d_fileno) { switch (ino64_trunc_error) { default: case 0: break; case 1: error = EOVERFLOW; goto done; case 2: dstdp.d_fileno = UINT32_MAX; break; } } dstdp.d_reclen = sizeof(dstdp) - sizeof(dstdp.d_name) + ((dp->d_namlen + 1 + 3) &~ 3); bcopy(dp->d_name, dstdp.d_name, dstdp.d_namlen); bzero(dstdp.d_name + dstdp.d_namlen, dstdp.d_reclen - offsetof(struct freebsd11_dirent, d_name) - dstdp.d_namlen); MPASS(dstdp.d_reclen <= dp->d_reclen); MPASS(ucount + dstdp.d_reclen <= count); if (func != NULL) func(&dstdp); error = copyout(&dstdp, ubuf + ucount, dstdp.d_reclen); if (error != 0) break; dp = (struct dirent *)((char *)dp + dp->d_reclen); ucount += dstdp.d_reclen; } done: free(dirbuf, M_TEMP); if (error == 0) td->td_retval[0] = ucount; return (error); } #endif /* COMPAT */ #ifdef COMPAT_43 static void ogetdirentries_cvt(struct freebsd11_dirent *dp) { #if (BYTE_ORDER == LITTLE_ENDIAN) /* * The expected low byte of dp->d_namlen is our dp->d_type. * The high MBZ byte of dp->d_namlen is our dp->d_namlen. */ dp->d_type = dp->d_namlen; dp->d_namlen = 0; #else /* * The dp->d_type is the high byte of the expected dp->d_namlen, * so must be zero'ed. */ dp->d_type = 0; #endif } /* * Read a block of directory entries in a filesystem independent format. */ #ifndef _SYS_SYSPROTO_H_ struct ogetdirentries_args { int fd; char *buf; u_int count; long *basep; }; #endif int ogetdirentries(struct thread *td, struct ogetdirentries_args *uap) { long loff; int error; error = kern_ogetdirentries(td, uap, &loff); if (error == 0) error = copyout(&loff, uap->basep, sizeof(long)); return (error); } int kern_ogetdirentries(struct thread *td, struct ogetdirentries_args *uap, long *ploff) { long base; int error; /* XXX arbitrary sanity limit on `count'. */ if (uap->count > 64 * 1024) return (EINVAL); error = freebsd11_kern_getdirentries(td, uap->fd, uap->buf, uap->count, &base, ogetdirentries_cvt); if (error == 0 && uap->basep != NULL) error = copyout(&base, uap->basep, sizeof(long)); return (error); } #endif /* COMPAT_43 */ #if defined(COMPAT_FREEBSD11) #ifndef _SYS_SYSPROTO_H_ struct freebsd11_getdirentries_args { int fd; char *buf; u_int count; long *basep; }; #endif int freebsd11_getdirentries(struct thread *td, struct freebsd11_getdirentries_args *uap) { long base; int error; error = freebsd11_kern_getdirentries(td, uap->fd, uap->buf, uap->count, &base, NULL); if (error == 0 && uap->basep != NULL) error = copyout(&base, uap->basep, sizeof(long)); return (error); } int freebsd11_getdents(struct thread *td, struct freebsd11_getdents_args *uap) { struct freebsd11_getdirentries_args ap; ap.fd = uap->fd; ap.buf = uap->buf; ap.count = uap->count; ap.basep = NULL; return (freebsd11_getdirentries(td, &ap)); } #endif /* COMPAT_FREEBSD11 */ /* * Read a block of directory entries in a filesystem independent format. */ int sys_getdirentries(struct thread *td, struct getdirentries_args *uap) { off_t base; int error; error = kern_getdirentries(td, uap->fd, uap->buf, uap->count, &base, NULL, UIO_USERSPACE); if (error != 0) return (error); if (uap->basep != NULL) error = copyout(&base, uap->basep, sizeof(off_t)); return (error); } int kern_getdirentries(struct thread *td, int fd, char *buf, size_t count, off_t *basep, ssize_t *residp, enum uio_seg bufseg) { struct vnode *vp; struct file *fp; struct uio auio; struct iovec aiov; off_t loff; int error, eofflag; off_t foffset; AUDIT_ARG_FD(fd); if (count > IOSIZE_MAX) return (EINVAL); auio.uio_resid = count; error = getvnode(td, fd, &cap_read_rights, &fp); if (error != 0) return (error); if ((fp->f_flag & FREAD) == 0) { fdrop(fp, td); return (EBADF); } vp = fp->f_vnode; foffset = foffset_lock(fp, 0); unionread: if (vp->v_type != VDIR) { error = EINVAL; goto fail; } aiov.iov_base = buf; aiov.iov_len = count; auio.uio_iov = &aiov; auio.uio_iovcnt = 1; auio.uio_rw = UIO_READ; auio.uio_segflg = bufseg; auio.uio_td = td; vn_lock(vp, LK_SHARED | LK_RETRY); AUDIT_ARG_VNODE1(vp); loff = auio.uio_offset = foffset; #ifdef MAC error = mac_vnode_check_readdir(td->td_ucred, vp); if (error == 0) #endif error = VOP_READDIR(vp, &auio, fp->f_cred, &eofflag, NULL, NULL); foffset = auio.uio_offset; if (error != 0) { VOP_UNLOCK(vp); goto fail; } if (count == auio.uio_resid && (vp->v_vflag & VV_ROOT) && (vp->v_mount->mnt_flag & MNT_UNION)) { struct vnode *tvp = vp; vp = vp->v_mount->mnt_vnodecovered; VREF(vp); fp->f_vnode = vp; fp->f_data = vp; foffset = 0; vput(tvp); goto unionread; } VOP_UNLOCK(vp); *basep = loff; if (residp != NULL) *residp = auio.uio_resid; td->td_retval[0] = count - auio.uio_resid; fail: foffset_unlock(fp, foffset, 0); fdrop(fp, td); return (error); } /* * Set the mode mask for creation of filesystem nodes. */ #ifndef _SYS_SYSPROTO_H_ struct umask_args { int newmask; }; #endif int sys_umask(struct thread *td, struct umask_args *uap) { struct filedesc *fdp; fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); td->td_retval[0] = fdp->fd_cmask; fdp->fd_cmask = uap->newmask & ALLPERMS; FILEDESC_XUNLOCK(fdp); return (0); } /* * Void all references to file by ripping underlying filesystem away from * vnode. */ #ifndef _SYS_SYSPROTO_H_ struct revoke_args { char *path; }; #endif int sys_revoke(struct thread *td, struct revoke_args *uap) { struct vnode *vp; struct vattr vattr; struct nameidata nd; int error; NDINIT(&nd, LOOKUP, FOLLOW | LOCKLEAF | AUDITVNODE1, UIO_USERSPACE, uap->path, td); if ((error = namei(&nd)) != 0) return (error); vp = nd.ni_vp; NDFREE(&nd, NDF_ONLY_PNBUF); if (vp->v_type != VCHR || vp->v_rdev == NULL) { error = EINVAL; goto out; } #ifdef MAC error = mac_vnode_check_revoke(td->td_ucred, vp); if (error != 0) goto out; #endif error = VOP_GETATTR(vp, &vattr, td->td_ucred); if (error != 0) goto out; if (td->td_ucred->cr_uid != vattr.va_uid) { error = priv_check(td, PRIV_VFS_ADMIN); if (error != 0) goto out; } if (vp->v_usecount > 1 || vcount(vp) > 1) VOP_REVOKE(vp, REVOKEALL); out: vput(vp); return (error); } /* * Convert a user file descriptor to a kernel file entry and check that, if it * is a capability, the correct rights are present. A reference on the file * entry is held upon returning. */ int getvnode(struct thread *td, int fd, cap_rights_t *rightsp, struct file **fpp) { struct file *fp; int error; error = fget_unlocked(td->td_proc->p_fd, fd, rightsp, &fp); if (error != 0) return (error); /* * The file could be not of the vnode type, or it may be not * yet fully initialized, in which case the f_vnode pointer * may be set, but f_ops is still badfileops. E.g., * devfs_open() transiently create such situation to * facilitate csw d_fdopen(). * * Dupfdopen() handling in kern_openat() installs the * half-baked file into the process descriptor table, allowing * other thread to dereference it. Guard against the race by * checking f_ops. */ if (fp->f_vnode == NULL || fp->f_ops == &badfileops) { fdrop(fp, td); return (EINVAL); } *fpp = fp; return (0); } /* * Get an (NFS) file handle. */ #ifndef _SYS_SYSPROTO_H_ struct lgetfh_args { char *fname; fhandle_t *fhp; }; #endif int sys_lgetfh(struct thread *td, struct lgetfh_args *uap) { return (kern_getfhat(td, AT_SYMLINK_NOFOLLOW, AT_FDCWD, uap->fname, UIO_USERSPACE, uap->fhp)); } #ifndef _SYS_SYSPROTO_H_ struct getfh_args { char *fname; fhandle_t *fhp; }; #endif int sys_getfh(struct thread *td, struct getfh_args *uap) { return (kern_getfhat(td, 0, AT_FDCWD, uap->fname, UIO_USERSPACE, uap->fhp)); } /* * syscall for the rpc.lockd to use to translate an open descriptor into * a NFS file handle. * * warning: do not remove the priv_check() call or this becomes one giant * security hole. */ #ifndef _SYS_SYSPROTO_H_ struct getfhat_args { int fd; char *path; fhandle_t *fhp; int flags; }; #endif int sys_getfhat(struct thread *td, struct getfhat_args *uap) { if ((uap->flags & ~(AT_SYMLINK_NOFOLLOW | AT_BENEATH)) != 0) return (EINVAL); return (kern_getfhat(td, uap->flags, uap->fd, uap->path, UIO_USERSPACE, uap->fhp)); } static int kern_getfhat(struct thread *td, int flags, int fd, const char *path, enum uio_seg pathseg, fhandle_t *fhp) { struct nameidata nd; fhandle_t fh; struct vnode *vp; int error; error = priv_check(td, PRIV_VFS_GETFH); if (error != 0) return (error); NDINIT_AT(&nd, LOOKUP, ((flags & AT_SYMLINK_NOFOLLOW) != 0 ? NOFOLLOW : FOLLOW) | ((flags & AT_BENEATH) != 0 ? BENEATH : 0) | LOCKLEAF | AUDITVNODE1, pathseg, path, fd, td); error = namei(&nd); if (error != 0) return (error); NDFREE(&nd, NDF_ONLY_PNBUF); vp = nd.ni_vp; bzero(&fh, sizeof(fh)); fh.fh_fsid = vp->v_mount->mnt_stat.f_fsid; error = VOP_VPTOFH(vp, &fh.fh_fid); vput(vp); if (error == 0) error = copyout(&fh, fhp, sizeof (fh)); return (error); } #ifndef _SYS_SYSPROTO_H_ struct fhlink_args { fhandle_t *fhp; const char *to; }; #endif int sys_fhlink(struct thread *td, struct fhlink_args *uap) { return (kern_fhlinkat(td, AT_FDCWD, uap->to, UIO_USERSPACE, uap->fhp)); } #ifndef _SYS_SYSPROTO_H_ struct fhlinkat_args { fhandle_t *fhp; int tofd; const char *to; }; #endif int sys_fhlinkat(struct thread *td, struct fhlinkat_args *uap) { return (kern_fhlinkat(td, uap->tofd, uap->to, UIO_USERSPACE, uap->fhp)); } static int kern_fhlinkat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, fhandle_t *fhp) { fhandle_t fh; struct mount *mp; struct vnode *vp; int error; error = priv_check(td, PRIV_VFS_GETFH); if (error != 0) return (error); error = copyin(fhp, &fh, sizeof(fh)); if (error != 0) return (error); do { bwillwrite(); if ((mp = vfs_busyfs(&fh.fh_fsid)) == NULL) return (ESTALE); error = VFS_FHTOVP(mp, &fh.fh_fid, LK_SHARED, &vp); vfs_unbusy(mp); if (error != 0) return (error); VOP_UNLOCK(vp); } while ((error = kern_linkat_vp(td, vp, fd, path, pathseg)) == EAGAIN); return (error); } #ifndef _SYS_SYSPROTO_H_ struct fhreadlink_args { fhandle_t *fhp; char *buf; size_t bufsize; }; #endif int sys_fhreadlink(struct thread *td, struct fhreadlink_args *uap) { fhandle_t fh; struct mount *mp; struct vnode *vp; int error; error = priv_check(td, PRIV_VFS_GETFH); if (error != 0) return (error); if (uap->bufsize > IOSIZE_MAX) return (EINVAL); error = copyin(uap->fhp, &fh, sizeof(fh)); if (error != 0) return (error); if ((mp = vfs_busyfs(&fh.fh_fsid)) == NULL) return (ESTALE); error = VFS_FHTOVP(mp, &fh.fh_fid, LK_SHARED, &vp); vfs_unbusy(mp); if (error != 0) return (error); error = kern_readlink_vp(vp, uap->buf, UIO_USERSPACE, uap->bufsize, td); vput(vp); return (error); } /* * syscall for the rpc.lockd to use to translate a NFS file handle into an * open descriptor. * * warning: do not remove the priv_check() call or this becomes one giant * security hole. */ #ifndef _SYS_SYSPROTO_H_ struct fhopen_args { const struct fhandle *u_fhp; int flags; }; #endif int sys_fhopen(struct thread *td, struct fhopen_args *uap) { struct mount *mp; struct vnode *vp; struct fhandle fhp; struct file *fp; int fmode, error; int indx; error = priv_check(td, PRIV_VFS_FHOPEN); if (error != 0) return (error); indx = -1; fmode = FFLAGS(uap->flags); /* why not allow a non-read/write open for our lockd? */ if (((fmode & (FREAD | FWRITE)) == 0) || (fmode & O_CREAT)) return (EINVAL); error = copyin(uap->u_fhp, &fhp, sizeof(fhp)); if (error != 0) return(error); /* find the mount point */ mp = vfs_busyfs(&fhp.fh_fsid); if (mp == NULL) return (ESTALE); /* now give me my vnode, it gets returned to me locked */ error = VFS_FHTOVP(mp, &fhp.fh_fid, LK_EXCLUSIVE, &vp); vfs_unbusy(mp); if (error != 0) return (error); error = falloc_noinstall(td, &fp); if (error != 0) { vput(vp); return (error); } /* * An extra reference on `fp' has been held for us by * falloc_noinstall(). */ #ifdef INVARIANTS td->td_dupfd = -1; #endif error = vn_open_vnode(vp, fmode, td->td_ucred, td, fp); if (error != 0) { KASSERT(fp->f_ops == &badfileops, ("VOP_OPEN in fhopen() set f_ops")); KASSERT(td->td_dupfd < 0, ("fhopen() encountered fdopen()")); vput(vp); goto bad; } #ifdef INVARIANTS td->td_dupfd = 0; #endif fp->f_vnode = vp; fp->f_seqcount = 1; finit(fp, (fmode & FMASK) | (fp->f_flag & FHASLOCK), DTYPE_VNODE, vp, &vnops); VOP_UNLOCK(vp); if ((fmode & O_TRUNC) != 0) { error = fo_truncate(fp, 0, td->td_ucred, td); if (error != 0) goto bad; } error = finstall(td, fp, &indx, fmode, NULL); bad: fdrop(fp, td); td->td_retval[0] = indx; return (error); } /* * Stat an (NFS) file handle. */ #ifndef _SYS_SYSPROTO_H_ struct fhstat_args { struct fhandle *u_fhp; struct stat *sb; }; #endif int sys_fhstat(struct thread *td, struct fhstat_args *uap) { struct stat sb; struct fhandle fh; int error; error = copyin(uap->u_fhp, &fh, sizeof(fh)); if (error != 0) return (error); error = kern_fhstat(td, fh, &sb); if (error == 0) error = copyout(&sb, uap->sb, sizeof(sb)); return (error); } int kern_fhstat(struct thread *td, struct fhandle fh, struct stat *sb) { struct mount *mp; struct vnode *vp; int error; error = priv_check(td, PRIV_VFS_FHSTAT); if (error != 0) return (error); if ((mp = vfs_busyfs(&fh.fh_fsid)) == NULL) return (ESTALE); error = VFS_FHTOVP(mp, &fh.fh_fid, LK_EXCLUSIVE, &vp); vfs_unbusy(mp); if (error != 0) return (error); error = vn_stat(vp, sb, td->td_ucred, NOCRED, td); vput(vp); return (error); } /* * Implement fstatfs() for (NFS) file handles. */ #ifndef _SYS_SYSPROTO_H_ struct fhstatfs_args { struct fhandle *u_fhp; struct statfs *buf; }; #endif int sys_fhstatfs(struct thread *td, struct fhstatfs_args *uap) { struct statfs *sfp; fhandle_t fh; int error; error = copyin(uap->u_fhp, &fh, sizeof(fhandle_t)); if (error != 0) return (error); sfp = malloc(sizeof(struct statfs), M_STATFS, M_WAITOK); error = kern_fhstatfs(td, fh, sfp); if (error == 0) error = copyout(sfp, uap->buf, sizeof(*sfp)); free(sfp, M_STATFS); return (error); } int kern_fhstatfs(struct thread *td, fhandle_t fh, struct statfs *buf) { struct mount *mp; struct vnode *vp; int error; error = priv_check(td, PRIV_VFS_FHSTATFS); if (error != 0) return (error); if ((mp = vfs_busyfs(&fh.fh_fsid)) == NULL) return (ESTALE); error = VFS_FHTOVP(mp, &fh.fh_fid, LK_EXCLUSIVE, &vp); if (error != 0) { vfs_unbusy(mp); return (error); } vput(vp); error = prison_canseemount(td->td_ucred, mp); if (error != 0) goto out; #ifdef MAC error = mac_mount_check_stat(td->td_ucred, mp); if (error != 0) goto out; #endif error = VFS_STATFS(mp, buf); out: vfs_unbusy(mp); return (error); } /* * Unlike madvise(2), we do not make a best effort to remember every * possible caching hint. Instead, we remember the last setting with * the exception that we will allow POSIX_FADV_NORMAL to adjust the * region of any current setting. */ int kern_posix_fadvise(struct thread *td, int fd, off_t offset, off_t len, int advice) { struct fadvise_info *fa, *new; struct file *fp; struct vnode *vp; off_t end; int error; if (offset < 0 || len < 0 || offset > OFF_MAX - len) return (EINVAL); AUDIT_ARG_VALUE(advice); switch (advice) { case POSIX_FADV_SEQUENTIAL: case POSIX_FADV_RANDOM: case POSIX_FADV_NOREUSE: new = malloc(sizeof(*fa), M_FADVISE, M_WAITOK); break; case POSIX_FADV_NORMAL: case POSIX_FADV_WILLNEED: case POSIX_FADV_DONTNEED: new = NULL; break; default: return (EINVAL); } /* XXX: CAP_POSIX_FADVISE? */ AUDIT_ARG_FD(fd); error = fget(td, fd, &cap_no_rights, &fp); if (error != 0) goto out; AUDIT_ARG_FILE(td->td_proc, fp); if ((fp->f_ops->fo_flags & DFLAG_SEEKABLE) == 0) { error = ESPIPE; goto out; } if (fp->f_type != DTYPE_VNODE) { error = ENODEV; goto out; } vp = fp->f_vnode; if (vp->v_type != VREG) { error = ENODEV; goto out; } if (len == 0) end = OFF_MAX; else end = offset + len - 1; switch (advice) { case POSIX_FADV_SEQUENTIAL: case POSIX_FADV_RANDOM: case POSIX_FADV_NOREUSE: /* * Try to merge any existing non-standard region with * this new region if possible, otherwise create a new * non-standard region for this request. */ mtx_pool_lock(mtxpool_sleep, fp); fa = fp->f_advice; if (fa != NULL && fa->fa_advice == advice && ((fa->fa_start <= end && fa->fa_end >= offset) || (end != OFF_MAX && fa->fa_start == end + 1) || (fa->fa_end != OFF_MAX && fa->fa_end + 1 == offset))) { if (offset < fa->fa_start) fa->fa_start = offset; if (end > fa->fa_end) fa->fa_end = end; } else { new->fa_advice = advice; new->fa_start = offset; new->fa_end = end; fp->f_advice = new; new = fa; } mtx_pool_unlock(mtxpool_sleep, fp); break; case POSIX_FADV_NORMAL: /* * If a the "normal" region overlaps with an existing * non-standard region, trim or remove the * non-standard region. */ mtx_pool_lock(mtxpool_sleep, fp); fa = fp->f_advice; if (fa != NULL) { if (offset <= fa->fa_start && end >= fa->fa_end) { new = fa; fp->f_advice = NULL; } else if (offset <= fa->fa_start && end >= fa->fa_start) fa->fa_start = end + 1; else if (offset <= fa->fa_end && end >= fa->fa_end) fa->fa_end = offset - 1; else if (offset >= fa->fa_start && end <= fa->fa_end) { /* * If the "normal" region is a middle * portion of the existing * non-standard region, just remove * the whole thing rather than picking * one side or the other to * preserve. */ new = fa; fp->f_advice = NULL; } } mtx_pool_unlock(mtxpool_sleep, fp); break; case POSIX_FADV_WILLNEED: case POSIX_FADV_DONTNEED: error = VOP_ADVISE(vp, offset, end, advice); break; } out: if (fp != NULL) fdrop(fp, td); free(new, M_FADVISE); return (error); } int sys_posix_fadvise(struct thread *td, struct posix_fadvise_args *uap) { int error; error = kern_posix_fadvise(td, uap->fd, uap->offset, uap->len, uap->advice); return (kern_posix_error(td, error)); } int kern_copy_file_range(struct thread *td, int infd, off_t *inoffp, int outfd, off_t *outoffp, size_t len, unsigned int flags) { struct file *infp, *outfp; struct vnode *invp, *outvp; int error; size_t retlen; void *rl_rcookie, *rl_wcookie; off_t savinoff, savoutoff; infp = outfp = NULL; rl_rcookie = rl_wcookie = NULL; savinoff = -1; error = 0; retlen = 0; if (flags != 0) { error = EINVAL; goto out; } if (len > SSIZE_MAX) /* * Although the len argument is size_t, the return argument * is ssize_t (which is signed). Therefore a size that won't * fit in ssize_t can't be returned. */ len = SSIZE_MAX; /* Get the file structures for the file descriptors. */ error = fget_read(td, infd, &cap_read_rights, &infp); if (error != 0) goto out; error = fget_write(td, outfd, &cap_write_rights, &outfp); if (error != 0) goto out; /* Set the offset pointers to the correct place. */ if (inoffp == NULL) inoffp = &infp->f_offset; if (outoffp == NULL) outoffp = &outfp->f_offset; savinoff = *inoffp; savoutoff = *outoffp; invp = infp->f_vnode; outvp = outfp->f_vnode; /* Sanity check the f_flag bits. */ if ((outfp->f_flag & (FWRITE | FAPPEND)) != FWRITE || (infp->f_flag & FREAD) == 0) { error = EBADF; goto out; } /* If len == 0, just return 0. */ if (len == 0) goto out; /* * If infp and outfp refer to the same file, the byte ranges cannot * overlap. */ if (invp == outvp && ((savinoff <= savoutoff && savinoff + len > savoutoff) || (savinoff > savoutoff && savoutoff + len > savinoff))) { error = EINVAL; goto out; } /* Range lock the byte ranges for both invp and outvp. */ for (;;) { rl_wcookie = vn_rangelock_wlock(outvp, *outoffp, *outoffp + len); rl_rcookie = vn_rangelock_tryrlock(invp, *inoffp, *inoffp + len); if (rl_rcookie != NULL) break; vn_rangelock_unlock(outvp, rl_wcookie); rl_rcookie = vn_rangelock_rlock(invp, *inoffp, *inoffp + len); vn_rangelock_unlock(invp, rl_rcookie); } retlen = len; error = vn_copy_file_range(invp, inoffp, outvp, outoffp, &retlen, flags, infp->f_cred, outfp->f_cred, td); out: if (rl_rcookie != NULL) vn_rangelock_unlock(invp, rl_rcookie); if (rl_wcookie != NULL) vn_rangelock_unlock(outvp, rl_wcookie); if (savinoff != -1 && (error == EINTR || error == ERESTART)) { *inoffp = savinoff; *outoffp = savoutoff; } if (outfp != NULL) fdrop(outfp, td); if (infp != NULL) fdrop(infp, td); td->td_retval[0] = retlen; return (error); } int sys_copy_file_range(struct thread *td, struct copy_file_range_args *uap) { off_t inoff, outoff, *inoffp, *outoffp; int error; inoffp = outoffp = NULL; if (uap->inoffp != NULL) { error = copyin(uap->inoffp, &inoff, sizeof(off_t)); if (error != 0) return (error); inoffp = &inoff; } if (uap->outoffp != NULL) { error = copyin(uap->outoffp, &outoff, sizeof(off_t)); if (error != 0) return (error); outoffp = &outoff; } error = kern_copy_file_range(td, uap->infd, inoffp, uap->outfd, outoffp, uap->len, uap->flags); if (error == 0 && uap->inoffp != NULL) error = copyout(inoffp, uap->inoffp, sizeof(off_t)); if (error == 0 && uap->outoffp != NULL) error = copyout(outoffp, uap->outoffp, sizeof(off_t)); return (error); } Index: projects/clang1000-import/sys/kern/vfs_vnops.c =================================================================== --- projects/clang1000-import/sys/kern/vfs_vnops.c (revision 357965) +++ projects/clang1000-import/sys/kern/vfs_vnops.c (revision 357966) @@ -1,3208 +1,3184 @@ /*- * SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 1982, 1986, 1989, 1993 * The Regents of the University of California. All rights reserved. * (c) UNIX System Laboratories, Inc. * All or some portions of this file are derived from material licensed * to the University of California by American Telephone and Telegraph * Co. or Unix System Laboratories, Inc. and are reproduced herein with * the permission of UNIX System Laboratories, Inc. * * Copyright (c) 2012 Konstantin Belousov * Copyright (c) 2013, 2014 The FreeBSD Foundation * * Portions of this software were developed by Konstantin Belousov * under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)vfs_vnops.c 8.2 (Berkeley) 1/21/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_hwpmc_hooks.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef HWPMC_HOOKS #include #endif static fo_rdwr_t vn_read; static fo_rdwr_t vn_write; static fo_rdwr_t vn_io_fault; static fo_truncate_t vn_truncate; static fo_ioctl_t vn_ioctl; static fo_poll_t vn_poll; static fo_kqfilter_t vn_kqfilter; static fo_stat_t vn_statfile; static fo_close_t vn_closefile; static fo_mmap_t vn_mmap; static fo_fallocate_t vn_fallocate; struct fileops vnops = { .fo_read = vn_io_fault, .fo_write = vn_io_fault, .fo_truncate = vn_truncate, .fo_ioctl = vn_ioctl, .fo_poll = vn_poll, .fo_kqfilter = vn_kqfilter, .fo_stat = vn_statfile, .fo_close = vn_closefile, .fo_chmod = vn_chmod, .fo_chown = vn_chown, .fo_sendfile = vn_sendfile, .fo_seek = vn_seek, .fo_fill_kinfo = vn_fill_kinfo, .fo_mmap = vn_mmap, .fo_fallocate = vn_fallocate, .fo_flags = DFLAG_PASSABLE | DFLAG_SEEKABLE }; static const int io_hold_cnt = 16; static int vn_io_fault_enable = 1; SYSCTL_INT(_debug, OID_AUTO, vn_io_fault_enable, CTLFLAG_RW, &vn_io_fault_enable, 0, "Enable vn_io_fault lock avoidance"); static int vn_io_fault_prefault = 0; SYSCTL_INT(_debug, OID_AUTO, vn_io_fault_prefault, CTLFLAG_RW, &vn_io_fault_prefault, 0, "Enable vn_io_fault prefaulting"); static u_long vn_io_faults_cnt; SYSCTL_ULONG(_debug, OID_AUTO, vn_io_faults, CTLFLAG_RD, &vn_io_faults_cnt, 0, "Count of vn_io_fault lock avoidance triggers"); /* * Returns true if vn_io_fault mode of handling the i/o request should * be used. */ static bool do_vn_io_fault(struct vnode *vp, struct uio *uio) { struct mount *mp; return (uio->uio_segflg == UIO_USERSPACE && vp->v_type == VREG && (mp = vp->v_mount) != NULL && (mp->mnt_kern_flag & MNTK_NO_IOPF) != 0 && vn_io_fault_enable); } /* * Structure used to pass arguments to vn_io_fault1(), to do either * file- or vnode-based I/O calls. */ struct vn_io_fault_args { enum { VN_IO_FAULT_FOP, VN_IO_FAULT_VOP } kind; struct ucred *cred; int flags; union { struct fop_args_tag { struct file *fp; fo_rdwr_t *doio; } fop_args; struct vop_args_tag { struct vnode *vp; } vop_args; } args; }; static int vn_io_fault1(struct vnode *vp, struct uio *uio, struct vn_io_fault_args *args, struct thread *td); int vn_open(struct nameidata *ndp, int *flagp, int cmode, struct file *fp) { struct thread *td = ndp->ni_cnd.cn_thread; return (vn_open_cred(ndp, flagp, cmode, 0, td->td_ucred, fp)); } /* * Common code for vnode open operations via a name lookup. * Lookup the vnode and invoke VOP_CREATE if needed. * Check permissions, and call the VOP_OPEN or VOP_CREATE routine. * * Note that this does NOT free nameidata for the successful case, * due to the NDINIT being done elsewhere. */ int vn_open_cred(struct nameidata *ndp, int *flagp, int cmode, u_int vn_open_flags, struct ucred *cred, struct file *fp) { struct vnode *vp; struct mount *mp; struct thread *td = ndp->ni_cnd.cn_thread; struct vattr vat; struct vattr *vap = &vat; int fmode, error; restart: fmode = *flagp; if ((fmode & (O_CREAT | O_EXCL | O_DIRECTORY)) == (O_CREAT | O_EXCL | O_DIRECTORY)) return (EINVAL); else if ((fmode & (O_CREAT | O_DIRECTORY)) == O_CREAT) { ndp->ni_cnd.cn_nameiop = CREATE; /* * Set NOCACHE to avoid flushing the cache when * rolling in many files at once. */ ndp->ni_cnd.cn_flags = ISOPEN | LOCKPARENT | LOCKLEAF | NOCACHE; if ((fmode & O_EXCL) == 0 && (fmode & O_NOFOLLOW) == 0) ndp->ni_cnd.cn_flags |= FOLLOW; if ((fmode & O_BENEATH) != 0) ndp->ni_cnd.cn_flags |= BENEATH; if (!(vn_open_flags & VN_OPEN_NOAUDIT)) ndp->ni_cnd.cn_flags |= AUDITVNODE1; if (vn_open_flags & VN_OPEN_NOCAPCHECK) ndp->ni_cnd.cn_flags |= NOCAPCHECK; if ((vn_open_flags & VN_OPEN_INVFS) == 0) bwillwrite(); if ((error = namei(ndp)) != 0) return (error); if (ndp->ni_vp == NULL) { VATTR_NULL(vap); vap->va_type = VREG; vap->va_mode = cmode; if (fmode & O_EXCL) vap->va_vaflags |= VA_EXCLUSIVE; if (vn_start_write(ndp->ni_dvp, &mp, V_NOWAIT) != 0) { NDFREE(ndp, NDF_ONLY_PNBUF); vput(ndp->ni_dvp); if ((error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH)) != 0) return (error); goto restart; } if ((vn_open_flags & VN_OPEN_NAMECACHE) != 0) ndp->ni_cnd.cn_flags |= MAKEENTRY; #ifdef MAC error = mac_vnode_check_create(cred, ndp->ni_dvp, &ndp->ni_cnd, vap); if (error == 0) #endif error = VOP_CREATE(ndp->ni_dvp, &ndp->ni_vp, &ndp->ni_cnd, vap); vput(ndp->ni_dvp); vn_finished_write(mp); if (error) { NDFREE(ndp, NDF_ONLY_PNBUF); return (error); } fmode &= ~O_TRUNC; vp = ndp->ni_vp; } else { if (ndp->ni_dvp == ndp->ni_vp) vrele(ndp->ni_dvp); else vput(ndp->ni_dvp); ndp->ni_dvp = NULL; vp = ndp->ni_vp; if (fmode & O_EXCL) { error = EEXIST; goto bad; } if (vp->v_type == VDIR) { error = EISDIR; goto bad; } fmode &= ~O_CREAT; } } else { ndp->ni_cnd.cn_nameiop = LOOKUP; ndp->ni_cnd.cn_flags = ISOPEN | ((fmode & O_NOFOLLOW) ? NOFOLLOW : FOLLOW) | LOCKLEAF; if (!(fmode & FWRITE)) ndp->ni_cnd.cn_flags |= LOCKSHARED; if ((fmode & O_BENEATH) != 0) ndp->ni_cnd.cn_flags |= BENEATH; if (!(vn_open_flags & VN_OPEN_NOAUDIT)) ndp->ni_cnd.cn_flags |= AUDITVNODE1; if (vn_open_flags & VN_OPEN_NOCAPCHECK) ndp->ni_cnd.cn_flags |= NOCAPCHECK; if ((error = namei(ndp)) != 0) return (error); vp = ndp->ni_vp; } error = vn_open_vnode(vp, fmode, cred, td, fp); if (error) goto bad; *flagp = fmode; return (0); bad: NDFREE(ndp, NDF_ONLY_PNBUF); vput(vp); *flagp = fmode; ndp->ni_vp = NULL; return (error); } static int vn_open_vnode_advlock(struct vnode *vp, int fmode, struct file *fp) { struct flock lf; int error, lock_flags, type; ASSERT_VOP_LOCKED(vp, "vn_open_vnode_advlock"); if ((fmode & (O_EXLOCK | O_SHLOCK)) == 0) return (0); KASSERT(fp != NULL, ("open with flock requires fp")); if (fp->f_type != DTYPE_NONE && fp->f_type != DTYPE_VNODE) return (EOPNOTSUPP); lock_flags = VOP_ISLOCKED(vp); VOP_UNLOCK(vp); lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; lf.l_type = (fmode & O_EXLOCK) != 0 ? F_WRLCK : F_RDLCK; type = F_FLOCK; if ((fmode & FNONBLOCK) == 0) type |= F_WAIT; error = VOP_ADVLOCK(vp, (caddr_t)fp, F_SETLK, &lf, type); if (error == 0) fp->f_flag |= FHASLOCK; vn_lock(vp, lock_flags | LK_RETRY); if (error == 0 && VN_IS_DOOMED(vp)) error = ENOENT; return (error); } /* * Common code for vnode open operations once a vnode is located. * Check permissions, and call the VOP_OPEN routine. */ int vn_open_vnode(struct vnode *vp, int fmode, struct ucred *cred, struct thread *td, struct file *fp) { accmode_t accmode; int error; if (vp->v_type == VLNK) return (EMLINK); if (vp->v_type == VSOCK) return (EOPNOTSUPP); if (vp->v_type != VDIR && fmode & O_DIRECTORY) return (ENOTDIR); accmode = 0; if (fmode & (FWRITE | O_TRUNC)) { if (vp->v_type == VDIR) return (EISDIR); accmode |= VWRITE; } if (fmode & FREAD) accmode |= VREAD; if (fmode & FEXEC) accmode |= VEXEC; if ((fmode & O_APPEND) && (fmode & FWRITE)) accmode |= VAPPEND; #ifdef MAC if (fmode & O_CREAT) accmode |= VCREAT; if (fmode & O_VERIFY) accmode |= VVERIFY; error = mac_vnode_check_open(cred, vp, accmode); if (error) return (error); accmode &= ~(VCREAT | VVERIFY); #endif if ((fmode & O_CREAT) == 0 && accmode != 0) { error = VOP_ACCESS(vp, accmode, cred, td); if (error != 0) return (error); } if (vp->v_type == VFIFO && VOP_ISLOCKED(vp) != LK_EXCLUSIVE) vn_lock(vp, LK_UPGRADE | LK_RETRY); error = VOP_OPEN(vp, fmode, cred, td, fp); if (error != 0) return (error); error = vn_open_vnode_advlock(vp, fmode, fp); if (error == 0 && (fmode & FWRITE) != 0) { error = VOP_ADD_WRITECOUNT(vp, 1); if (error == 0) { CTR3(KTR_VFS, "%s: vp %p v_writecount increased to %d", __func__, vp, vp->v_writecount); } } /* * Error from advlock or VOP_ADD_WRITECOUNT() still requires * calling VOP_CLOSE() to pair with earlier VOP_OPEN(). * Arrange for that by having fdrop() to use vn_closefile(). */ if (error != 0) { fp->f_flag |= FOPENFAILED; fp->f_vnode = vp; if (fp->f_ops == &badfileops) { fp->f_type = DTYPE_VNODE; fp->f_ops = &vnops; } vref(vp); } ASSERT_VOP_LOCKED(vp, "vn_open_vnode"); return (error); } /* * Check for write permissions on the specified vnode. * Prototype text segments cannot be written. * It is racy. */ int vn_writechk(struct vnode *vp) { ASSERT_VOP_LOCKED(vp, "vn_writechk"); /* * If there's shared text associated with * the vnode, try to free it up once. If * we fail, we can't allow writing. */ if (VOP_IS_TEXT(vp)) return (ETXTBSY); return (0); } /* * Vnode close call */ static int vn_close1(struct vnode *vp, int flags, struct ucred *file_cred, struct thread *td, bool keep_ref) { struct mount *mp; int error, lock_flags; if (vp->v_type != VFIFO && (flags & FWRITE) == 0 && MNT_EXTENDED_SHARED(vp->v_mount)) lock_flags = LK_SHARED; else lock_flags = LK_EXCLUSIVE; vn_start_write(vp, &mp, V_WAIT); vn_lock(vp, lock_flags | LK_RETRY); AUDIT_ARG_VNODE1(vp); if ((flags & (FWRITE | FOPENFAILED)) == FWRITE) { VOP_ADD_WRITECOUNT_CHECKED(vp, -1); CTR3(KTR_VFS, "%s: vp %p v_writecount decreased to %d", __func__, vp, vp->v_writecount); } error = VOP_CLOSE(vp, flags, file_cred, td); if (keep_ref) VOP_UNLOCK(vp); else vput(vp); vn_finished_write(mp); return (error); } int vn_close(struct vnode *vp, int flags, struct ucred *file_cred, struct thread *td) { return (vn_close1(vp, flags, file_cred, td, false)); } /* * Heuristic to detect sequential operation. */ static int sequential_heuristic(struct uio *uio, struct file *fp) { ASSERT_VOP_LOCKED(fp->f_vnode, __func__); if (fp->f_flag & FRDAHEAD) return (fp->f_seqcount << IO_SEQSHIFT); /* * Offset 0 is handled specially. open() sets f_seqcount to 1 so * that the first I/O is normally considered to be slightly * sequential. Seeking to offset 0 doesn't change sequentiality * unless previous seeks have reduced f_seqcount to 0, in which * case offset 0 is not special. */ if ((uio->uio_offset == 0 && fp->f_seqcount > 0) || uio->uio_offset == fp->f_nextoff) { /* * f_seqcount is in units of fixed-size blocks so that it * depends mainly on the amount of sequential I/O and not * much on the number of sequential I/O's. The fixed size * of 16384 is hard-coded here since it is (not quite) just * a magic size that works well here. This size is more * closely related to the best I/O size for real disks than * to any block size used by software. */ if (uio->uio_resid >= IO_SEQMAX * 16384) fp->f_seqcount = IO_SEQMAX; else { fp->f_seqcount += howmany(uio->uio_resid, 16384); if (fp->f_seqcount > IO_SEQMAX) fp->f_seqcount = IO_SEQMAX; } return (fp->f_seqcount << IO_SEQSHIFT); } /* Not sequential. Quickly draw-down sequentiality. */ if (fp->f_seqcount > 1) fp->f_seqcount = 1; else fp->f_seqcount = 0; return (0); } /* * Package up an I/O request on a vnode into a uio and do it. */ int vn_rdwr(enum uio_rw rw, struct vnode *vp, void *base, int len, off_t offset, enum uio_seg segflg, int ioflg, struct ucred *active_cred, struct ucred *file_cred, ssize_t *aresid, struct thread *td) { struct uio auio; struct iovec aiov; struct mount *mp; struct ucred *cred; void *rl_cookie; struct vn_io_fault_args args; int error, lock_flags; if (offset < 0 && vp->v_type != VCHR) return (EINVAL); auio.uio_iov = &aiov; auio.uio_iovcnt = 1; aiov.iov_base = base; aiov.iov_len = len; auio.uio_resid = len; auio.uio_offset = offset; auio.uio_segflg = segflg; auio.uio_rw = rw; auio.uio_td = td; error = 0; if ((ioflg & IO_NODELOCKED) == 0) { if ((ioflg & IO_RANGELOCKED) == 0) { if (rw == UIO_READ) { rl_cookie = vn_rangelock_rlock(vp, offset, offset + len); } else { rl_cookie = vn_rangelock_wlock(vp, offset, offset + len); } } else rl_cookie = NULL; mp = NULL; if (rw == UIO_WRITE) { if (vp->v_type != VCHR && (error = vn_start_write(vp, &mp, V_WAIT | PCATCH)) != 0) goto out; if (MNT_SHARED_WRITES(mp) || ((mp == NULL) && MNT_SHARED_WRITES(vp->v_mount))) lock_flags = LK_SHARED; else lock_flags = LK_EXCLUSIVE; } else lock_flags = LK_SHARED; vn_lock(vp, lock_flags | LK_RETRY); } else rl_cookie = NULL; ASSERT_VOP_LOCKED(vp, "IO_NODELOCKED with no vp lock held"); #ifdef MAC if ((ioflg & IO_NOMACCHECK) == 0) { if (rw == UIO_READ) error = mac_vnode_check_read(active_cred, file_cred, vp); else error = mac_vnode_check_write(active_cred, file_cred, vp); } #endif if (error == 0) { if (file_cred != NULL) cred = file_cred; else cred = active_cred; if (do_vn_io_fault(vp, &auio)) { args.kind = VN_IO_FAULT_VOP; args.cred = cred; args.flags = ioflg; args.args.vop_args.vp = vp; error = vn_io_fault1(vp, &auio, &args, td); } else if (rw == UIO_READ) { error = VOP_READ(vp, &auio, ioflg, cred); } else /* if (rw == UIO_WRITE) */ { error = VOP_WRITE(vp, &auio, ioflg, cred); } } if (aresid) *aresid = auio.uio_resid; else if (auio.uio_resid && error == 0) error = EIO; if ((ioflg & IO_NODELOCKED) == 0) { VOP_UNLOCK(vp); if (mp != NULL) vn_finished_write(mp); } out: if (rl_cookie != NULL) vn_rangelock_unlock(vp, rl_cookie); return (error); } /* * Package up an I/O request on a vnode into a uio and do it. The I/O * request is split up into smaller chunks and we try to avoid saturating * the buffer cache while potentially holding a vnode locked, so we * check bwillwrite() before calling vn_rdwr(). We also call kern_yield() * to give other processes a chance to lock the vnode (either other processes * core'ing the same binary, or unrelated processes scanning the directory). */ int vn_rdwr_inchunks(enum uio_rw rw, struct vnode *vp, void *base, size_t len, off_t offset, enum uio_seg segflg, int ioflg, struct ucred *active_cred, struct ucred *file_cred, size_t *aresid, struct thread *td) { int error = 0; ssize_t iaresid; do { int chunk; /* * Force `offset' to a multiple of MAXBSIZE except possibly * for the first chunk, so that filesystems only need to * write full blocks except possibly for the first and last * chunks. */ chunk = MAXBSIZE - (uoff_t)offset % MAXBSIZE; if (chunk > len) chunk = len; if (rw != UIO_READ && vp->v_type == VREG) bwillwrite(); iaresid = 0; error = vn_rdwr(rw, vp, base, chunk, offset, segflg, ioflg, active_cred, file_cred, &iaresid, td); len -= chunk; /* aresid calc already includes length */ if (error) break; offset += chunk; base = (char *)base + chunk; kern_yield(PRI_USER); } while (len); if (aresid) *aresid = len + iaresid; return (error); } off_t foffset_lock(struct file *fp, int flags) { struct mtx *mtxp; off_t res; KASSERT((flags & FOF_OFFSET) == 0, ("FOF_OFFSET passed")); #if OFF_MAX <= LONG_MAX /* * Caller only wants the current f_offset value. Assume that * the long and shorter integer types reads are atomic. */ if ((flags & FOF_NOLOCK) != 0) return (fp->f_offset); #endif /* * According to McKusick the vn lock was protecting f_offset here. * It is now protected by the FOFFSET_LOCKED flag. */ mtxp = mtx_pool_find(mtxpool_sleep, fp); mtx_lock(mtxp); if ((flags & FOF_NOLOCK) == 0) { while (fp->f_vnread_flags & FOFFSET_LOCKED) { fp->f_vnread_flags |= FOFFSET_LOCK_WAITING; msleep(&fp->f_vnread_flags, mtxp, PUSER -1, "vofflock", 0); } fp->f_vnread_flags |= FOFFSET_LOCKED; } res = fp->f_offset; mtx_unlock(mtxp); return (res); } void foffset_unlock(struct file *fp, off_t val, int flags) { struct mtx *mtxp; KASSERT((flags & FOF_OFFSET) == 0, ("FOF_OFFSET passed")); #if OFF_MAX <= LONG_MAX if ((flags & FOF_NOLOCK) != 0) { if ((flags & FOF_NOUPDATE) == 0) fp->f_offset = val; if ((flags & FOF_NEXTOFF) != 0) fp->f_nextoff = val; return; } #endif mtxp = mtx_pool_find(mtxpool_sleep, fp); mtx_lock(mtxp); if ((flags & FOF_NOUPDATE) == 0) fp->f_offset = val; if ((flags & FOF_NEXTOFF) != 0) fp->f_nextoff = val; if ((flags & FOF_NOLOCK) == 0) { KASSERT((fp->f_vnread_flags & FOFFSET_LOCKED) != 0, ("Lost FOFFSET_LOCKED")); if (fp->f_vnread_flags & FOFFSET_LOCK_WAITING) wakeup(&fp->f_vnread_flags); fp->f_vnread_flags = 0; } mtx_unlock(mtxp); } void foffset_lock_uio(struct file *fp, struct uio *uio, int flags) { if ((flags & FOF_OFFSET) == 0) uio->uio_offset = foffset_lock(fp, flags); } void foffset_unlock_uio(struct file *fp, struct uio *uio, int flags) { if ((flags & FOF_OFFSET) == 0) foffset_unlock(fp, uio->uio_offset, flags); } static int get_advice(struct file *fp, struct uio *uio) { struct mtx *mtxp; int ret; ret = POSIX_FADV_NORMAL; if (fp->f_advice == NULL || fp->f_vnode->v_type != VREG) return (ret); mtxp = mtx_pool_find(mtxpool_sleep, fp); mtx_lock(mtxp); if (fp->f_advice != NULL && uio->uio_offset >= fp->f_advice->fa_start && uio->uio_offset + uio->uio_resid <= fp->f_advice->fa_end) ret = fp->f_advice->fa_advice; mtx_unlock(mtxp); return (ret); } /* * File table vnode read routine. */ static int vn_read(struct file *fp, struct uio *uio, struct ucred *active_cred, int flags, struct thread *td) { struct vnode *vp; off_t orig_offset; int error, ioflag; int advice; KASSERT(uio->uio_td == td, ("uio_td %p is not td %p", uio->uio_td, td)); KASSERT(flags & FOF_OFFSET, ("No FOF_OFFSET")); vp = fp->f_vnode; ioflag = 0; if (fp->f_flag & FNONBLOCK) ioflag |= IO_NDELAY; if (fp->f_flag & O_DIRECT) ioflag |= IO_DIRECT; advice = get_advice(fp, uio); vn_lock(vp, LK_SHARED | LK_RETRY); switch (advice) { case POSIX_FADV_NORMAL: case POSIX_FADV_SEQUENTIAL: case POSIX_FADV_NOREUSE: ioflag |= sequential_heuristic(uio, fp); break; case POSIX_FADV_RANDOM: /* Disable read-ahead for random I/O. */ break; } orig_offset = uio->uio_offset; #ifdef MAC error = mac_vnode_check_read(active_cred, fp->f_cred, vp); if (error == 0) #endif error = VOP_READ(vp, uio, ioflag, fp->f_cred); fp->f_nextoff = uio->uio_offset; VOP_UNLOCK(vp); if (error == 0 && advice == POSIX_FADV_NOREUSE && orig_offset != uio->uio_offset) /* * Use POSIX_FADV_DONTNEED to flush pages and buffers * for the backing file after a POSIX_FADV_NOREUSE * read(2). */ error = VOP_ADVISE(vp, orig_offset, uio->uio_offset - 1, POSIX_FADV_DONTNEED); return (error); } /* * File table vnode write routine. */ static int vn_write(struct file *fp, struct uio *uio, struct ucred *active_cred, int flags, struct thread *td) { struct vnode *vp; struct mount *mp; off_t orig_offset; int error, ioflag, lock_flags; int advice; KASSERT(uio->uio_td == td, ("uio_td %p is not td %p", uio->uio_td, td)); KASSERT(flags & FOF_OFFSET, ("No FOF_OFFSET")); vp = fp->f_vnode; if (vp->v_type == VREG) bwillwrite(); ioflag = IO_UNIT; if (vp->v_type == VREG && (fp->f_flag & O_APPEND)) ioflag |= IO_APPEND; if (fp->f_flag & FNONBLOCK) ioflag |= IO_NDELAY; if (fp->f_flag & O_DIRECT) ioflag |= IO_DIRECT; if ((fp->f_flag & O_FSYNC) || (vp->v_mount && (vp->v_mount->mnt_flag & MNT_SYNCHRONOUS))) ioflag |= IO_SYNC; mp = NULL; if (vp->v_type != VCHR && (error = vn_start_write(vp, &mp, V_WAIT | PCATCH)) != 0) goto unlock; advice = get_advice(fp, uio); if (MNT_SHARED_WRITES(mp) || (mp == NULL && MNT_SHARED_WRITES(vp->v_mount))) { lock_flags = LK_SHARED; } else { lock_flags = LK_EXCLUSIVE; } vn_lock(vp, lock_flags | LK_RETRY); switch (advice) { case POSIX_FADV_NORMAL: case POSIX_FADV_SEQUENTIAL: case POSIX_FADV_NOREUSE: ioflag |= sequential_heuristic(uio, fp); break; case POSIX_FADV_RANDOM: /* XXX: Is this correct? */ break; } orig_offset = uio->uio_offset; #ifdef MAC error = mac_vnode_check_write(active_cred, fp->f_cred, vp); if (error == 0) #endif error = VOP_WRITE(vp, uio, ioflag, fp->f_cred); fp->f_nextoff = uio->uio_offset; VOP_UNLOCK(vp); if (vp->v_type != VCHR) vn_finished_write(mp); if (error == 0 && advice == POSIX_FADV_NOREUSE && orig_offset != uio->uio_offset) /* * Use POSIX_FADV_DONTNEED to flush pages and buffers * for the backing file after a POSIX_FADV_NOREUSE * write(2). */ error = VOP_ADVISE(vp, orig_offset, uio->uio_offset - 1, POSIX_FADV_DONTNEED); unlock: return (error); } /* * The vn_io_fault() is a wrapper around vn_read() and vn_write() to * prevent the following deadlock: * * Assume that the thread A reads from the vnode vp1 into userspace * buffer buf1 backed by the pages of vnode vp2. If a page in buf1 is * currently not resident, then system ends up with the call chain * vn_read() -> VOP_READ(vp1) -> uiomove() -> [Page Fault] -> * vm_fault(buf1) -> vnode_pager_getpages(vp2) -> VOP_GETPAGES(vp2) * which establishes lock order vp1->vn_lock, then vp2->vn_lock. * If, at the same time, thread B reads from vnode vp2 into buffer buf2 * backed by the pages of vnode vp1, and some page in buf2 is not * resident, we get a reversed order vp2->vn_lock, then vp1->vn_lock. * * To prevent the lock order reversal and deadlock, vn_io_fault() does * not allow page faults to happen during VOP_READ() or VOP_WRITE(). * Instead, it first tries to do the whole range i/o with pagefaults * disabled. If all pages in the i/o buffer are resident and mapped, * VOP will succeed (ignoring the genuine filesystem errors). * Otherwise, we get back EFAULT, and vn_io_fault() falls back to do * i/o in chunks, with all pages in the chunk prefaulted and held * using vm_fault_quick_hold_pages(). * * Filesystems using this deadlock avoidance scheme should use the * array of the held pages from uio, saved in the curthread->td_ma, * instead of doing uiomove(). A helper function * vn_io_fault_uiomove() converts uiomove request into * uiomove_fromphys() over td_ma array. * * Since vnode locks do not cover the whole i/o anymore, rangelocks * make the current i/o request atomic with respect to other i/os and * truncations. */ /* * Decode vn_io_fault_args and perform the corresponding i/o. */ static int vn_io_fault_doio(struct vn_io_fault_args *args, struct uio *uio, struct thread *td) { int error, save; error = 0; save = vm_fault_disable_pagefaults(); switch (args->kind) { case VN_IO_FAULT_FOP: error = (args->args.fop_args.doio)(args->args.fop_args.fp, uio, args->cred, args->flags, td); break; case VN_IO_FAULT_VOP: if (uio->uio_rw == UIO_READ) { error = VOP_READ(args->args.vop_args.vp, uio, args->flags, args->cred); } else if (uio->uio_rw == UIO_WRITE) { error = VOP_WRITE(args->args.vop_args.vp, uio, args->flags, args->cred); } break; default: panic("vn_io_fault_doio: unknown kind of io %d %d", args->kind, uio->uio_rw); } vm_fault_enable_pagefaults(save); return (error); } static int vn_io_fault_touch(char *base, const struct uio *uio) { int r; r = fubyte(base); if (r == -1 || (uio->uio_rw == UIO_READ && subyte(base, r) == -1)) return (EFAULT); return (0); } static int vn_io_fault_prefault_user(const struct uio *uio) { char *base; const struct iovec *iov; size_t len; ssize_t resid; int error, i; KASSERT(uio->uio_segflg == UIO_USERSPACE, ("vn_io_fault_prefault userspace")); error = i = 0; iov = uio->uio_iov; resid = uio->uio_resid; base = iov->iov_base; len = iov->iov_len; while (resid > 0) { error = vn_io_fault_touch(base, uio); if (error != 0) break; if (len < PAGE_SIZE) { if (len != 0) { error = vn_io_fault_touch(base + len - 1, uio); if (error != 0) break; resid -= len; } if (++i >= uio->uio_iovcnt) break; iov = uio->uio_iov + i; base = iov->iov_base; len = iov->iov_len; } else { len -= PAGE_SIZE; base += PAGE_SIZE; resid -= PAGE_SIZE; } } return (error); } /* * Common code for vn_io_fault(), agnostic to the kind of i/o request. * Uses vn_io_fault_doio() to make the call to an actual i/o function. * Used from vn_rdwr() and vn_io_fault(), which encode the i/o request * into args and call vn_io_fault1() to handle faults during the user * mode buffer accesses. */ static int vn_io_fault1(struct vnode *vp, struct uio *uio, struct vn_io_fault_args *args, struct thread *td) { vm_page_t ma[io_hold_cnt + 2]; struct uio *uio_clone, short_uio; struct iovec short_iovec[1]; vm_page_t *prev_td_ma; vm_prot_t prot; vm_offset_t addr, end; size_t len, resid; ssize_t adv; int error, cnt, saveheld, prev_td_ma_cnt; if (vn_io_fault_prefault) { error = vn_io_fault_prefault_user(uio); if (error != 0) return (error); /* Or ignore ? */ } prot = uio->uio_rw == UIO_READ ? VM_PROT_WRITE : VM_PROT_READ; /* * The UFS follows IO_UNIT directive and replays back both * uio_offset and uio_resid if an error is encountered during the * operation. But, since the iovec may be already advanced, * uio is still in an inconsistent state. * * Cache a copy of the original uio, which is advanced to the redo * point using UIO_NOCOPY below. */ uio_clone = cloneuio(uio); resid = uio->uio_resid; short_uio.uio_segflg = UIO_USERSPACE; short_uio.uio_rw = uio->uio_rw; short_uio.uio_td = uio->uio_td; error = vn_io_fault_doio(args, uio, td); if (error != EFAULT) goto out; atomic_add_long(&vn_io_faults_cnt, 1); uio_clone->uio_segflg = UIO_NOCOPY; uiomove(NULL, resid - uio->uio_resid, uio_clone); uio_clone->uio_segflg = uio->uio_segflg; saveheld = curthread_pflags_set(TDP_UIOHELD); prev_td_ma = td->td_ma; prev_td_ma_cnt = td->td_ma_cnt; while (uio_clone->uio_resid != 0) { len = uio_clone->uio_iov->iov_len; if (len == 0) { KASSERT(uio_clone->uio_iovcnt >= 1, ("iovcnt underflow")); uio_clone->uio_iov++; uio_clone->uio_iovcnt--; continue; } if (len > io_hold_cnt * PAGE_SIZE) len = io_hold_cnt * PAGE_SIZE; addr = (uintptr_t)uio_clone->uio_iov->iov_base; end = round_page(addr + len); if (end < addr) { error = EFAULT; break; } cnt = atop(end - trunc_page(addr)); /* * A perfectly misaligned address and length could cause * both the start and the end of the chunk to use partial * page. +2 accounts for such a situation. */ cnt = vm_fault_quick_hold_pages(&td->td_proc->p_vmspace->vm_map, addr, len, prot, ma, io_hold_cnt + 2); if (cnt == -1) { error = EFAULT; break; } short_uio.uio_iov = &short_iovec[0]; short_iovec[0].iov_base = (void *)addr; short_uio.uio_iovcnt = 1; short_uio.uio_resid = short_iovec[0].iov_len = len; short_uio.uio_offset = uio_clone->uio_offset; td->td_ma = ma; td->td_ma_cnt = cnt; error = vn_io_fault_doio(args, &short_uio, td); vm_page_unhold_pages(ma, cnt); adv = len - short_uio.uio_resid; uio_clone->uio_iov->iov_base = (char *)uio_clone->uio_iov->iov_base + adv; uio_clone->uio_iov->iov_len -= adv; uio_clone->uio_resid -= adv; uio_clone->uio_offset += adv; uio->uio_resid -= adv; uio->uio_offset += adv; if (error != 0 || adv == 0) break; } td->td_ma = prev_td_ma; td->td_ma_cnt = prev_td_ma_cnt; curthread_pflags_restore(saveheld); out: free(uio_clone, M_IOV); return (error); } static int vn_io_fault(struct file *fp, struct uio *uio, struct ucred *active_cred, int flags, struct thread *td) { fo_rdwr_t *doio; struct vnode *vp; void *rl_cookie; struct vn_io_fault_args args; int error; doio = uio->uio_rw == UIO_READ ? vn_read : vn_write; vp = fp->f_vnode; foffset_lock_uio(fp, uio, flags); if (do_vn_io_fault(vp, uio)) { args.kind = VN_IO_FAULT_FOP; args.args.fop_args.fp = fp; args.args.fop_args.doio = doio; args.cred = active_cred; args.flags = flags | FOF_OFFSET; if (uio->uio_rw == UIO_READ) { rl_cookie = vn_rangelock_rlock(vp, uio->uio_offset, uio->uio_offset + uio->uio_resid); } else if ((fp->f_flag & O_APPEND) != 0 || (flags & FOF_OFFSET) == 0) { /* For appenders, punt and lock the whole range. */ rl_cookie = vn_rangelock_wlock(vp, 0, OFF_MAX); } else { rl_cookie = vn_rangelock_wlock(vp, uio->uio_offset, uio->uio_offset + uio->uio_resid); } error = vn_io_fault1(vp, uio, &args, td); vn_rangelock_unlock(vp, rl_cookie); } else { error = doio(fp, uio, active_cred, flags | FOF_OFFSET, td); } foffset_unlock_uio(fp, uio, flags); return (error); } /* * Helper function to perform the requested uiomove operation using * the held pages for io->uio_iov[0].iov_base buffer instead of * copyin/copyout. Access to the pages with uiomove_fromphys() * instead of iov_base prevents page faults that could occur due to * pmap_collect() invalidating the mapping created by * vm_fault_quick_hold_pages(), or pageout daemon, page laundry or * object cleanup revoking the write access from page mappings. * * Filesystems specified MNTK_NO_IOPF shall use vn_io_fault_uiomove() * instead of plain uiomove(). */ int vn_io_fault_uiomove(char *data, int xfersize, struct uio *uio) { struct uio transp_uio; struct iovec transp_iov[1]; struct thread *td; size_t adv; int error, pgadv; td = curthread; if ((td->td_pflags & TDP_UIOHELD) == 0 || uio->uio_segflg != UIO_USERSPACE) return (uiomove(data, xfersize, uio)); KASSERT(uio->uio_iovcnt == 1, ("uio_iovcnt %d", uio->uio_iovcnt)); transp_iov[0].iov_base = data; transp_uio.uio_iov = &transp_iov[0]; transp_uio.uio_iovcnt = 1; if (xfersize > uio->uio_resid) xfersize = uio->uio_resid; transp_uio.uio_resid = transp_iov[0].iov_len = xfersize; transp_uio.uio_offset = 0; transp_uio.uio_segflg = UIO_SYSSPACE; /* * Since transp_iov points to data, and td_ma page array * corresponds to original uio->uio_iov, we need to invert the * direction of the i/o operation as passed to * uiomove_fromphys(). */ switch (uio->uio_rw) { case UIO_WRITE: transp_uio.uio_rw = UIO_READ; break; case UIO_READ: transp_uio.uio_rw = UIO_WRITE; break; } transp_uio.uio_td = uio->uio_td; error = uiomove_fromphys(td->td_ma, ((vm_offset_t)uio->uio_iov->iov_base) & PAGE_MASK, xfersize, &transp_uio); adv = xfersize - transp_uio.uio_resid; pgadv = (((vm_offset_t)uio->uio_iov->iov_base + adv) >> PAGE_SHIFT) - (((vm_offset_t)uio->uio_iov->iov_base) >> PAGE_SHIFT); td->td_ma += pgadv; KASSERT(td->td_ma_cnt >= pgadv, ("consumed pages %d %d", td->td_ma_cnt, pgadv)); td->td_ma_cnt -= pgadv; uio->uio_iov->iov_base = (char *)uio->uio_iov->iov_base + adv; uio->uio_iov->iov_len -= adv; uio->uio_resid -= adv; uio->uio_offset += adv; return (error); } int vn_io_fault_pgmove(vm_page_t ma[], vm_offset_t offset, int xfersize, struct uio *uio) { struct thread *td; vm_offset_t iov_base; int cnt, pgadv; td = curthread; if ((td->td_pflags & TDP_UIOHELD) == 0 || uio->uio_segflg != UIO_USERSPACE) return (uiomove_fromphys(ma, offset, xfersize, uio)); KASSERT(uio->uio_iovcnt == 1, ("uio_iovcnt %d", uio->uio_iovcnt)); cnt = xfersize > uio->uio_resid ? uio->uio_resid : xfersize; iov_base = (vm_offset_t)uio->uio_iov->iov_base; switch (uio->uio_rw) { case UIO_WRITE: pmap_copy_pages(td->td_ma, iov_base & PAGE_MASK, ma, offset, cnt); break; case UIO_READ: pmap_copy_pages(ma, offset, td->td_ma, iov_base & PAGE_MASK, cnt); break; } pgadv = ((iov_base + cnt) >> PAGE_SHIFT) - (iov_base >> PAGE_SHIFT); td->td_ma += pgadv; KASSERT(td->td_ma_cnt >= pgadv, ("consumed pages %d %d", td->td_ma_cnt, pgadv)); td->td_ma_cnt -= pgadv; uio->uio_iov->iov_base = (char *)(iov_base + cnt); uio->uio_iov->iov_len -= cnt; uio->uio_resid -= cnt; uio->uio_offset += cnt; return (0); } /* * File table truncate routine. */ static int vn_truncate(struct file *fp, off_t length, struct ucred *active_cred, struct thread *td) { struct mount *mp; struct vnode *vp; void *rl_cookie; int error; vp = fp->f_vnode; /* * Lock the whole range for truncation. Otherwise split i/o * might happen partly before and partly after the truncation. */ rl_cookie = vn_rangelock_wlock(vp, 0, OFF_MAX); error = vn_start_write(vp, &mp, V_WAIT | PCATCH); if (error) goto out1; vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); AUDIT_ARG_VNODE1(vp); if (vp->v_type == VDIR) { error = EISDIR; goto out; } #ifdef MAC error = mac_vnode_check_write(active_cred, fp->f_cred, vp); if (error) goto out; #endif error = vn_truncate_locked(vp, length, (fp->f_flag & O_FSYNC) != 0, fp->f_cred); out: VOP_UNLOCK(vp); vn_finished_write(mp); out1: vn_rangelock_unlock(vp, rl_cookie); return (error); } /* * Truncate a file that is already locked. */ int vn_truncate_locked(struct vnode *vp, off_t length, bool sync, struct ucred *cred) { struct vattr vattr; int error; error = VOP_ADD_WRITECOUNT(vp, 1); if (error == 0) { VATTR_NULL(&vattr); vattr.va_size = length; if (sync) vattr.va_vaflags |= VA_SYNC; error = VOP_SETATTR(vp, &vattr, cred); VOP_ADD_WRITECOUNT_CHECKED(vp, -1); } return (error); } /* * File table vnode stat routine. */ static int vn_statfile(struct file *fp, struct stat *sb, struct ucred *active_cred, struct thread *td) { struct vnode *vp = fp->f_vnode; int error; vn_lock(vp, LK_SHARED | LK_RETRY); error = vn_stat(vp, sb, active_cred, fp->f_cred, td); VOP_UNLOCK(vp); return (error); } /* * Stat a vnode; implementation for the stat syscall */ int vn_stat(struct vnode *vp, struct stat *sb, struct ucred *active_cred, struct ucred *file_cred, struct thread *td) { struct vattr vattr; struct vattr *vap; int error; u_short mode; AUDIT_ARG_VNODE1(vp); #ifdef MAC error = mac_vnode_check_stat(active_cred, file_cred, vp); if (error) return (error); #endif vap = &vattr; /* * Initialize defaults for new and unusual fields, so that file * systems which don't support these fields don't need to know * about them. */ vap->va_birthtime.tv_sec = -1; vap->va_birthtime.tv_nsec = 0; vap->va_fsid = VNOVAL; vap->va_rdev = NODEV; error = VOP_GETATTR(vp, vap, active_cred); if (error) return (error); /* * Zero the spare stat fields */ bzero(sb, sizeof *sb); /* * Copy from vattr table */ if (vap->va_fsid != VNOVAL) sb->st_dev = vap->va_fsid; else sb->st_dev = vp->v_mount->mnt_stat.f_fsid.val[0]; sb->st_ino = vap->va_fileid; mode = vap->va_mode; switch (vap->va_type) { case VREG: mode |= S_IFREG; break; case VDIR: mode |= S_IFDIR; break; case VBLK: mode |= S_IFBLK; break; case VCHR: mode |= S_IFCHR; break; case VLNK: mode |= S_IFLNK; break; case VSOCK: mode |= S_IFSOCK; break; case VFIFO: mode |= S_IFIFO; break; default: return (EBADF); } sb->st_mode = mode; sb->st_nlink = vap->va_nlink; sb->st_uid = vap->va_uid; sb->st_gid = vap->va_gid; sb->st_rdev = vap->va_rdev; if (vap->va_size > OFF_MAX) return (EOVERFLOW); sb->st_size = vap->va_size; sb->st_atim.tv_sec = vap->va_atime.tv_sec; sb->st_atim.tv_nsec = vap->va_atime.tv_nsec; sb->st_mtim.tv_sec = vap->va_mtime.tv_sec; sb->st_mtim.tv_nsec = vap->va_mtime.tv_nsec; sb->st_ctim.tv_sec = vap->va_ctime.tv_sec; sb->st_ctim.tv_nsec = vap->va_ctime.tv_nsec; sb->st_birthtim.tv_sec = vap->va_birthtime.tv_sec; sb->st_birthtim.tv_nsec = vap->va_birthtime.tv_nsec; /* * According to www.opengroup.org, the meaning of st_blksize is * "a filesystem-specific preferred I/O block size for this * object. In some filesystem types, this may vary from file * to file" * Use miminum/default of PAGE_SIZE (e.g. for VCHR). */ sb->st_blksize = max(PAGE_SIZE, vap->va_blocksize); sb->st_flags = vap->va_flags; if (priv_check_cred_vfs_generation(td->td_ucred)) sb->st_gen = 0; else sb->st_gen = vap->va_gen; sb->st_blocks = vap->va_bytes / S_BLKSIZE; return (0); } /* * File table vnode ioctl routine. */ static int vn_ioctl(struct file *fp, u_long com, void *data, struct ucred *active_cred, struct thread *td) { struct vattr vattr; struct vnode *vp; struct fiobmap2_arg *bmarg; int error; vp = fp->f_vnode; switch (vp->v_type) { case VDIR: case VREG: switch (com) { case FIONREAD: vn_lock(vp, LK_SHARED | LK_RETRY); error = VOP_GETATTR(vp, &vattr, active_cred); VOP_UNLOCK(vp); if (error == 0) *(int *)data = vattr.va_size - fp->f_offset; return (error); case FIOBMAP2: bmarg = (struct fiobmap2_arg *)data; vn_lock(vp, LK_SHARED | LK_RETRY); #ifdef MAC error = mac_vnode_check_read(active_cred, fp->f_cred, vp); if (error == 0) #endif error = VOP_BMAP(vp, bmarg->bn, NULL, &bmarg->bn, &bmarg->runp, &bmarg->runb); VOP_UNLOCK(vp); return (error); case FIONBIO: case FIOASYNC: return (0); default: return (VOP_IOCTL(vp, com, data, fp->f_flag, active_cred, td)); } break; case VCHR: return (VOP_IOCTL(vp, com, data, fp->f_flag, active_cred, td)); default: return (ENOTTY); } } /* * File table vnode poll routine. */ static int vn_poll(struct file *fp, int events, struct ucred *active_cred, struct thread *td) { struct vnode *vp; int error; vp = fp->f_vnode; #ifdef MAC vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); AUDIT_ARG_VNODE1(vp); error = mac_vnode_check_poll(active_cred, fp->f_cred, vp); VOP_UNLOCK(vp); if (!error) #endif error = VOP_POLL(vp, events, fp->f_cred, td); return (error); } /* * Acquire the requested lock and then check for validity. LK_RETRY * permits vn_lock to return doomed vnodes. */ static int __noinline _vn_lock_fallback(struct vnode *vp, int flags, char *file, int line, int error) { KASSERT((flags & LK_RETRY) == 0 || error == 0, ("vn_lock: error %d incompatible with flags %#x", error, flags)); if (error == 0) VNASSERT(VN_IS_DOOMED(vp), vp, ("vnode not doomed")); if ((flags & LK_RETRY) == 0) { if (error == 0) { VOP_UNLOCK(vp); error = ENOENT; } return (error); } /* * LK_RETRY case. * * Nothing to do if we got the lock. */ if (error == 0) return (0); /* * Interlock was dropped by the call in _vn_lock. */ flags &= ~LK_INTERLOCK; do { error = VOP_LOCK1(vp, flags, file, line); } while (error != 0); return (0); } int _vn_lock(struct vnode *vp, int flags, char *file, int line) { int error; VNASSERT((flags & LK_TYPE_MASK) != 0, vp, ("vn_lock: no locktype (%d passed)", flags)); VNPASS(vp->v_holdcnt > 0, vp); error = VOP_LOCK1(vp, flags, file, line); if (__predict_false(error != 0 || VN_IS_DOOMED(vp))) return (_vn_lock_fallback(vp, flags, file, line, error)); return (0); } /* * File table vnode close routine. */ static int vn_closefile(struct file *fp, struct thread *td) { struct vnode *vp; struct flock lf; int error; bool ref; vp = fp->f_vnode; fp->f_ops = &badfileops; ref= (fp->f_flag & FHASLOCK) != 0 && fp->f_type == DTYPE_VNODE; error = vn_close1(vp, fp->f_flag, fp->f_cred, td, ref); if (__predict_false(ref)) { lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; lf.l_type = F_UNLCK; (void) VOP_ADVLOCK(vp, fp, F_UNLCK, &lf, F_FLOCK); vrele(vp); } return (error); } -static bool -vn_suspendable(struct mount *mp) -{ - - return (mp->mnt_op->vfs_susp_clean != NULL); -} - /* * Preparing to start a filesystem write operation. If the operation is * permitted, then we bump the count of operations in progress and * proceed. If a suspend request is in progress, we wait until the * suspension is over, and then proceed. */ static int vn_start_write_refed(struct mount *mp, int flags, bool mplocked) { int error, mflags; if (__predict_true(!mplocked) && (flags & V_XSLEEP) == 0 && vfs_op_thread_enter(mp)) { MPASS((mp->mnt_kern_flag & MNTK_SUSPEND) == 0); vfs_mp_count_add_pcpu(mp, writeopcount, 1); vfs_op_thread_exit(mp); return (0); } if (mplocked) mtx_assert(MNT_MTX(mp), MA_OWNED); else MNT_ILOCK(mp); error = 0; /* * Check on status of suspension. */ if ((curthread->td_pflags & TDP_IGNSUSP) == 0 || mp->mnt_susp_owner != curthread) { mflags = ((mp->mnt_vfc->vfc_flags & VFCF_SBDRY) != 0 ? (flags & PCATCH) : 0) | (PUSER - 1); while ((mp->mnt_kern_flag & MNTK_SUSPEND) != 0) { if (flags & V_NOWAIT) { error = EWOULDBLOCK; goto unlock; } error = msleep(&mp->mnt_flag, MNT_MTX(mp), mflags, "suspfs", 0); if (error) goto unlock; } } if (flags & V_XSLEEP) goto unlock; mp->mnt_writeopcount++; unlock: if (error != 0 || (flags & V_XSLEEP) != 0) MNT_REL(mp); MNT_IUNLOCK(mp); return (error); } int vn_start_write(struct vnode *vp, struct mount **mpp, int flags) { struct mount *mp; int error; KASSERT((flags & V_MNTREF) == 0 || (*mpp != NULL && vp == NULL), ("V_MNTREF requires mp")); error = 0; /* * If a vnode is provided, get and return the mount point that * to which it will write. */ if (vp != NULL) { if ((error = VOP_GETWRITEMOUNT(vp, mpp)) != 0) { *mpp = NULL; if (error != EOPNOTSUPP) return (error); return (0); } } if ((mp = *mpp) == NULL) return (0); - if (!vn_suspendable(mp)) { - if (vp != NULL || (flags & V_MNTREF) != 0) - vfs_rel(mp); - return (0); - } - /* * VOP_GETWRITEMOUNT() returns with the mp refcount held through * a vfs_ref(). * As long as a vnode is not provided we need to acquire a * refcount for the provided mountpoint too, in order to * emulate a vfs_ref(). */ if (vp == NULL && (flags & V_MNTREF) == 0) vfs_ref(mp); return (vn_start_write_refed(mp, flags, false)); } /* * Secondary suspension. Used by operations such as vop_inactive * routines that are needed by the higher level functions. These * are allowed to proceed until all the higher level functions have * completed (indicated by mnt_writeopcount dropping to zero). At that * time, these operations are halted until the suspension is over. */ int vn_start_secondary_write(struct vnode *vp, struct mount **mpp, int flags) { struct mount *mp; int error; KASSERT((flags & V_MNTREF) == 0 || (*mpp != NULL && vp == NULL), ("V_MNTREF requires mp")); retry: if (vp != NULL) { if ((error = VOP_GETWRITEMOUNT(vp, mpp)) != 0) { *mpp = NULL; if (error != EOPNOTSUPP) return (error); return (0); } } /* * If we are not suspended or have not yet reached suspended * mode, then let the operation proceed. */ if ((mp = *mpp) == NULL) return (0); - if (!vn_suspendable(mp)) { - if (vp != NULL || (flags & V_MNTREF) != 0) - vfs_rel(mp); - return (0); - } - /* * VOP_GETWRITEMOUNT() returns with the mp refcount held through * a vfs_ref(). * As long as a vnode is not provided we need to acquire a * refcount for the provided mountpoint too, in order to * emulate a vfs_ref(). */ MNT_ILOCK(mp); if (vp == NULL && (flags & V_MNTREF) == 0) MNT_REF(mp); if ((mp->mnt_kern_flag & (MNTK_SUSPENDED | MNTK_SUSPEND2)) == 0) { mp->mnt_secondary_writes++; mp->mnt_secondary_accwrites++; MNT_IUNLOCK(mp); return (0); } if (flags & V_NOWAIT) { MNT_REL(mp); MNT_IUNLOCK(mp); return (EWOULDBLOCK); } /* * Wait for the suspension to finish. */ error = msleep(&mp->mnt_flag, MNT_MTX(mp), (PUSER - 1) | PDROP | ((mp->mnt_vfc->vfc_flags & VFCF_SBDRY) != 0 ? (flags & PCATCH) : 0), "suspfs", 0); vfs_rel(mp); if (error == 0) goto retry; return (error); } /* * Filesystem write operation has completed. If we are suspending and this * operation is the last one, notify the suspender that the suspension is * now in effect. */ void vn_finished_write(struct mount *mp) { int c; - if (mp == NULL || !vn_suspendable(mp)) + if (mp == NULL) return; if (vfs_op_thread_enter(mp)) { vfs_mp_count_sub_pcpu(mp, writeopcount, 1); vfs_mp_count_sub_pcpu(mp, ref, 1); vfs_op_thread_exit(mp); return; } MNT_ILOCK(mp); vfs_assert_mount_counters(mp); MNT_REL(mp); c = --mp->mnt_writeopcount; if (mp->mnt_vfs_ops == 0) { MPASS((mp->mnt_kern_flag & MNTK_SUSPEND) == 0); MNT_IUNLOCK(mp); return; } if (c < 0) vfs_dump_mount_counters(mp); if ((mp->mnt_kern_flag & MNTK_SUSPEND) != 0 && c == 0) wakeup(&mp->mnt_writeopcount); MNT_IUNLOCK(mp); } /* * Filesystem secondary write operation has completed. If we are * suspending and this operation is the last one, notify the suspender * that the suspension is now in effect. */ void vn_finished_secondary_write(struct mount *mp) { - if (mp == NULL || !vn_suspendable(mp)) + if (mp == NULL) return; MNT_ILOCK(mp); MNT_REL(mp); mp->mnt_secondary_writes--; if (mp->mnt_secondary_writes < 0) panic("vn_finished_secondary_write: neg cnt"); if ((mp->mnt_kern_flag & MNTK_SUSPEND) != 0 && mp->mnt_secondary_writes <= 0) wakeup(&mp->mnt_secondary_writes); MNT_IUNLOCK(mp); } /* * Request a filesystem to suspend write operations. */ int vfs_write_suspend(struct mount *mp, int flags) { int error; - MPASS(vn_suspendable(mp)); - vfs_op_enter(mp); MNT_ILOCK(mp); vfs_assert_mount_counters(mp); if (mp->mnt_susp_owner == curthread) { vfs_op_exit_locked(mp); MNT_IUNLOCK(mp); return (EALREADY); } while (mp->mnt_kern_flag & MNTK_SUSPEND) msleep(&mp->mnt_flag, MNT_MTX(mp), PUSER - 1, "wsuspfs", 0); /* * Unmount holds a write reference on the mount point. If we * own busy reference and drain for writers, we deadlock with * the reference draining in the unmount path. Callers of * vfs_write_suspend() must specify VS_SKIP_UNMOUNT if * vfs_busy() reference is owned and caller is not in the * unmount context. */ if ((flags & VS_SKIP_UNMOUNT) != 0 && (mp->mnt_kern_flag & MNTK_UNMOUNT) != 0) { vfs_op_exit_locked(mp); MNT_IUNLOCK(mp); return (EBUSY); } mp->mnt_kern_flag |= MNTK_SUSPEND; mp->mnt_susp_owner = curthread; if (mp->mnt_writeopcount > 0) (void) msleep(&mp->mnt_writeopcount, MNT_MTX(mp), (PUSER - 1)|PDROP, "suspwt", 0); else MNT_IUNLOCK(mp); if ((error = VFS_SYNC(mp, MNT_SUSPEND)) != 0) { vfs_write_resume(mp, 0); vfs_op_exit(mp); } return (error); } /* * Request a filesystem to resume write operations. */ void vfs_write_resume(struct mount *mp, int flags) { - MPASS(vn_suspendable(mp)); - MNT_ILOCK(mp); if ((mp->mnt_kern_flag & MNTK_SUSPEND) != 0) { KASSERT(mp->mnt_susp_owner == curthread, ("mnt_susp_owner")); mp->mnt_kern_flag &= ~(MNTK_SUSPEND | MNTK_SUSPEND2 | MNTK_SUSPENDED); mp->mnt_susp_owner = NULL; wakeup(&mp->mnt_writeopcount); wakeup(&mp->mnt_flag); curthread->td_pflags &= ~TDP_IGNSUSP; if ((flags & VR_START_WRITE) != 0) { MNT_REF(mp); mp->mnt_writeopcount++; } MNT_IUNLOCK(mp); if ((flags & VR_NO_SUSPCLR) == 0) VFS_SUSP_CLEAN(mp); vfs_op_exit(mp); } else if ((flags & VR_START_WRITE) != 0) { MNT_REF(mp); vn_start_write_refed(mp, 0, true); } else { MNT_IUNLOCK(mp); } } /* * Helper loop around vfs_write_suspend() for filesystem unmount VFS * methods. */ int vfs_write_suspend_umnt(struct mount *mp) { int error; - MPASS(vn_suspendable(mp)); KASSERT((curthread->td_pflags & TDP_IGNSUSP) == 0, ("vfs_write_suspend_umnt: recursed")); /* dounmount() already called vn_start_write(). */ for (;;) { vn_finished_write(mp); error = vfs_write_suspend(mp, 0); if (error != 0) { vn_start_write(NULL, &mp, V_WAIT); return (error); } MNT_ILOCK(mp); if ((mp->mnt_kern_flag & MNTK_SUSPENDED) != 0) break; MNT_IUNLOCK(mp); vn_start_write(NULL, &mp, V_WAIT); } mp->mnt_kern_flag &= ~(MNTK_SUSPENDED | MNTK_SUSPEND2); wakeup(&mp->mnt_flag); MNT_IUNLOCK(mp); curthread->td_pflags |= TDP_IGNSUSP; return (0); } /* * Implement kqueues for files by translating it to vnode operation. */ static int vn_kqfilter(struct file *fp, struct knote *kn) { return (VOP_KQFILTER(fp->f_vnode, kn)); } /* * Simplified in-kernel wrapper calls for extended attribute access. * Both calls pass in a NULL credential, authorizing as "kernel" access. * Set IO_NODELOCKED in ioflg if the vnode is already locked. */ int vn_extattr_get(struct vnode *vp, int ioflg, int attrnamespace, const char *attrname, int *buflen, char *buf, struct thread *td) { struct uio auio; struct iovec iov; int error; iov.iov_len = *buflen; iov.iov_base = buf; auio.uio_iov = &iov; auio.uio_iovcnt = 1; auio.uio_rw = UIO_READ; auio.uio_segflg = UIO_SYSSPACE; auio.uio_td = td; auio.uio_offset = 0; auio.uio_resid = *buflen; if ((ioflg & IO_NODELOCKED) == 0) vn_lock(vp, LK_SHARED | LK_RETRY); ASSERT_VOP_LOCKED(vp, "IO_NODELOCKED with no vp lock held"); /* authorize attribute retrieval as kernel */ error = VOP_GETEXTATTR(vp, attrnamespace, attrname, &auio, NULL, NULL, td); if ((ioflg & IO_NODELOCKED) == 0) VOP_UNLOCK(vp); if (error == 0) { *buflen = *buflen - auio.uio_resid; } return (error); } /* * XXX failure mode if partially written? */ int vn_extattr_set(struct vnode *vp, int ioflg, int attrnamespace, const char *attrname, int buflen, char *buf, struct thread *td) { struct uio auio; struct iovec iov; struct mount *mp; int error; iov.iov_len = buflen; iov.iov_base = buf; auio.uio_iov = &iov; auio.uio_iovcnt = 1; auio.uio_rw = UIO_WRITE; auio.uio_segflg = UIO_SYSSPACE; auio.uio_td = td; auio.uio_offset = 0; auio.uio_resid = buflen; if ((ioflg & IO_NODELOCKED) == 0) { if ((error = vn_start_write(vp, &mp, V_WAIT)) != 0) return (error); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); } ASSERT_VOP_LOCKED(vp, "IO_NODELOCKED with no vp lock held"); /* authorize attribute setting as kernel */ error = VOP_SETEXTATTR(vp, attrnamespace, attrname, &auio, NULL, td); if ((ioflg & IO_NODELOCKED) == 0) { vn_finished_write(mp); VOP_UNLOCK(vp); } return (error); } int vn_extattr_rm(struct vnode *vp, int ioflg, int attrnamespace, const char *attrname, struct thread *td) { struct mount *mp; int error; if ((ioflg & IO_NODELOCKED) == 0) { if ((error = vn_start_write(vp, &mp, V_WAIT)) != 0) return (error); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); } ASSERT_VOP_LOCKED(vp, "IO_NODELOCKED with no vp lock held"); /* authorize attribute removal as kernel */ error = VOP_DELETEEXTATTR(vp, attrnamespace, attrname, NULL, td); if (error == EOPNOTSUPP) error = VOP_SETEXTATTR(vp, attrnamespace, attrname, NULL, NULL, td); if ((ioflg & IO_NODELOCKED) == 0) { vn_finished_write(mp); VOP_UNLOCK(vp); } return (error); } static int vn_get_ino_alloc_vget(struct mount *mp, void *arg, int lkflags, struct vnode **rvp) { return (VFS_VGET(mp, *(ino_t *)arg, lkflags, rvp)); } int vn_vget_ino(struct vnode *vp, ino_t ino, int lkflags, struct vnode **rvp) { return (vn_vget_ino_gen(vp, vn_get_ino_alloc_vget, &ino, lkflags, rvp)); } int vn_vget_ino_gen(struct vnode *vp, vn_get_ino_t alloc, void *alloc_arg, int lkflags, struct vnode **rvp) { struct mount *mp; int ltype, error; ASSERT_VOP_LOCKED(vp, "vn_vget_ino_get"); mp = vp->v_mount; ltype = VOP_ISLOCKED(vp); KASSERT(ltype == LK_EXCLUSIVE || ltype == LK_SHARED, ("vn_vget_ino: vp not locked")); error = vfs_busy(mp, MBF_NOWAIT); if (error != 0) { vfs_ref(mp); VOP_UNLOCK(vp); error = vfs_busy(mp, 0); vn_lock(vp, ltype | LK_RETRY); vfs_rel(mp); if (error != 0) return (ENOENT); if (VN_IS_DOOMED(vp)) { vfs_unbusy(mp); return (ENOENT); } } VOP_UNLOCK(vp); error = alloc(mp, alloc_arg, lkflags, rvp); vfs_unbusy(mp); if (error != 0 || *rvp != vp) vn_lock(vp, ltype | LK_RETRY); if (VN_IS_DOOMED(vp)) { if (error == 0) { if (*rvp == vp) vunref(vp); else vput(*rvp); } error = ENOENT; } return (error); } int vn_rlimit_fsize(const struct vnode *vp, const struct uio *uio, struct thread *td) { if (vp->v_type != VREG || td == NULL) return (0); if ((uoff_t)uio->uio_offset + uio->uio_resid > lim_cur(td, RLIMIT_FSIZE)) { PROC_LOCK(td->td_proc); kern_psignal(td->td_proc, SIGXFSZ); PROC_UNLOCK(td->td_proc); return (EFBIG); } return (0); } int vn_chmod(struct file *fp, mode_t mode, struct ucred *active_cred, struct thread *td) { struct vnode *vp; vp = fp->f_vnode; #ifdef AUDIT vn_lock(vp, LK_SHARED | LK_RETRY); AUDIT_ARG_VNODE1(vp); VOP_UNLOCK(vp); #endif return (setfmode(td, active_cred, vp, mode)); } int vn_chown(struct file *fp, uid_t uid, gid_t gid, struct ucred *active_cred, struct thread *td) { struct vnode *vp; vp = fp->f_vnode; #ifdef AUDIT vn_lock(vp, LK_SHARED | LK_RETRY); AUDIT_ARG_VNODE1(vp); VOP_UNLOCK(vp); #endif return (setfown(td, active_cred, vp, uid, gid)); } void vn_pages_remove(struct vnode *vp, vm_pindex_t start, vm_pindex_t end) { vm_object_t object; if ((object = vp->v_object) == NULL) return; VM_OBJECT_WLOCK(object); vm_object_page_remove(object, start, end, 0); VM_OBJECT_WUNLOCK(object); } int vn_bmap_seekhole(struct vnode *vp, u_long cmd, off_t *off, struct ucred *cred) { struct vattr va; daddr_t bn, bnp; uint64_t bsize; off_t noff; int error; KASSERT(cmd == FIOSEEKHOLE || cmd == FIOSEEKDATA, ("Wrong command %lu", cmd)); if (vn_lock(vp, LK_SHARED) != 0) return (EBADF); if (vp->v_type != VREG) { error = ENOTTY; goto unlock; } error = VOP_GETATTR(vp, &va, cred); if (error != 0) goto unlock; noff = *off; if (noff >= va.va_size) { error = ENXIO; goto unlock; } bsize = vp->v_mount->mnt_stat.f_iosize; for (bn = noff / bsize; noff < va.va_size; bn++, noff += bsize - noff % bsize) { error = VOP_BMAP(vp, bn, NULL, &bnp, NULL, NULL); if (error == EOPNOTSUPP) { error = ENOTTY; goto unlock; } if ((bnp == -1 && cmd == FIOSEEKHOLE) || (bnp != -1 && cmd == FIOSEEKDATA)) { noff = bn * bsize; if (noff < *off) noff = *off; goto unlock; } } if (noff > va.va_size) noff = va.va_size; /* noff == va.va_size. There is an implicit hole at the end of file. */ if (cmd == FIOSEEKDATA) error = ENXIO; unlock: VOP_UNLOCK(vp); if (error == 0) *off = noff; return (error); } int vn_seek(struct file *fp, off_t offset, int whence, struct thread *td) { struct ucred *cred; struct vnode *vp; struct vattr vattr; off_t foffset, size; int error, noneg; cred = td->td_ucred; vp = fp->f_vnode; foffset = foffset_lock(fp, 0); noneg = (vp->v_type != VCHR); error = 0; switch (whence) { case L_INCR: if (noneg && (foffset < 0 || (offset > 0 && foffset > OFF_MAX - offset))) { error = EOVERFLOW; break; } offset += foffset; break; case L_XTND: vn_lock(vp, LK_SHARED | LK_RETRY); error = VOP_GETATTR(vp, &vattr, cred); VOP_UNLOCK(vp); if (error) break; /* * If the file references a disk device, then fetch * the media size and use that to determine the ending * offset. */ if (vattr.va_size == 0 && vp->v_type == VCHR && fo_ioctl(fp, DIOCGMEDIASIZE, &size, cred, td) == 0) vattr.va_size = size; if (noneg && (vattr.va_size > OFF_MAX || (offset > 0 && vattr.va_size > OFF_MAX - offset))) { error = EOVERFLOW; break; } offset += vattr.va_size; break; case L_SET: break; case SEEK_DATA: error = fo_ioctl(fp, FIOSEEKDATA, &offset, cred, td); if (error == ENOTTY) error = EINVAL; break; case SEEK_HOLE: error = fo_ioctl(fp, FIOSEEKHOLE, &offset, cred, td); if (error == ENOTTY) error = EINVAL; break; default: error = EINVAL; } if (error == 0 && noneg && offset < 0) error = EINVAL; if (error != 0) goto drop; VFS_KNOTE_UNLOCKED(vp, 0); td->td_uretoff.tdu_off = offset; drop: foffset_unlock(fp, offset, error != 0 ? FOF_NOUPDATE : 0); return (error); } int vn_utimes_perm(struct vnode *vp, struct vattr *vap, struct ucred *cred, struct thread *td) { int error; /* * Grant permission if the caller is the owner of the file, or * the super-user, or has ACL_WRITE_ATTRIBUTES permission on * on the file. If the time pointer is null, then write * permission on the file is also sufficient. * * From NFSv4.1, draft 21, 6.2.1.3.1, Discussion of Mask Attributes: * A user having ACL_WRITE_DATA or ACL_WRITE_ATTRIBUTES * will be allowed to set the times [..] to the current * server time. */ error = VOP_ACCESSX(vp, VWRITE_ATTRIBUTES, cred, td); if (error != 0 && (vap->va_vaflags & VA_UTIMES_NULL) != 0) error = VOP_ACCESS(vp, VWRITE, cred, td); return (error); } int vn_fill_kinfo(struct file *fp, struct kinfo_file *kif, struct filedesc *fdp) { struct vnode *vp; int error; if (fp->f_type == DTYPE_FIFO) kif->kf_type = KF_TYPE_FIFO; else kif->kf_type = KF_TYPE_VNODE; vp = fp->f_vnode; vref(vp); FILEDESC_SUNLOCK(fdp); error = vn_fill_kinfo_vnode(vp, kif); vrele(vp); FILEDESC_SLOCK(fdp); return (error); } static inline void vn_fill_junk(struct kinfo_file *kif) { size_t len, olen; /* * Simulate vn_fullpath returning changing values for a given * vp during e.g. coredump. */ len = (arc4random() % (sizeof(kif->kf_path) - 2)) + 1; olen = strlen(kif->kf_path); if (len < olen) strcpy(&kif->kf_path[len - 1], "$"); else for (; olen < len; olen++) strcpy(&kif->kf_path[olen], "A"); } int vn_fill_kinfo_vnode(struct vnode *vp, struct kinfo_file *kif) { struct vattr va; char *fullpath, *freepath; int error; kif->kf_un.kf_file.kf_file_type = vntype_to_kinfo(vp->v_type); freepath = NULL; fullpath = "-"; error = vn_fullpath(curthread, vp, &fullpath, &freepath); if (error == 0) { strlcpy(kif->kf_path, fullpath, sizeof(kif->kf_path)); } if (freepath != NULL) free(freepath, M_TEMP); KFAIL_POINT_CODE(DEBUG_FP, fill_kinfo_vnode__random_path, vn_fill_junk(kif); ); /* * Retrieve vnode attributes. */ va.va_fsid = VNOVAL; va.va_rdev = NODEV; vn_lock(vp, LK_SHARED | LK_RETRY); error = VOP_GETATTR(vp, &va, curthread->td_ucred); VOP_UNLOCK(vp); if (error != 0) return (error); if (va.va_fsid != VNOVAL) kif->kf_un.kf_file.kf_file_fsid = va.va_fsid; else kif->kf_un.kf_file.kf_file_fsid = vp->v_mount->mnt_stat.f_fsid.val[0]; kif->kf_un.kf_file.kf_file_fsid_freebsd11 = kif->kf_un.kf_file.kf_file_fsid; /* truncate */ kif->kf_un.kf_file.kf_file_fileid = va.va_fileid; kif->kf_un.kf_file.kf_file_mode = MAKEIMODE(va.va_type, va.va_mode); kif->kf_un.kf_file.kf_file_size = va.va_size; kif->kf_un.kf_file.kf_file_rdev = va.va_rdev; kif->kf_un.kf_file.kf_file_rdev_freebsd11 = kif->kf_un.kf_file.kf_file_rdev; /* truncate */ return (0); } int vn_mmap(struct file *fp, vm_map_t map, vm_offset_t *addr, vm_size_t size, vm_prot_t prot, vm_prot_t cap_maxprot, int flags, vm_ooffset_t foff, struct thread *td) { #ifdef HWPMC_HOOKS struct pmckern_map_in pkm; #endif struct mount *mp; struct vnode *vp; vm_object_t object; vm_prot_t maxprot; boolean_t writecounted; int error; #if defined(COMPAT_FREEBSD7) || defined(COMPAT_FREEBSD6) || \ defined(COMPAT_FREEBSD5) || defined(COMPAT_FREEBSD4) /* * POSIX shared-memory objects are defined to have * kernel persistence, and are not defined to support * read(2)/write(2) -- or even open(2). Thus, we can * use MAP_ASYNC to trade on-disk coherence for speed. * The shm_open(3) library routine turns on the FPOSIXSHM * flag to request this behavior. */ if ((fp->f_flag & FPOSIXSHM) != 0) flags |= MAP_NOSYNC; #endif vp = fp->f_vnode; /* * Ensure that file and memory protections are * compatible. Note that we only worry about * writability if mapping is shared; in this case, * current and max prot are dictated by the open file. * XXX use the vnode instead? Problem is: what * credentials do we use for determination? What if * proc does a setuid? */ mp = vp->v_mount; if (mp != NULL && (mp->mnt_flag & MNT_NOEXEC) != 0) { maxprot = VM_PROT_NONE; if ((prot & VM_PROT_EXECUTE) != 0) return (EACCES); } else maxprot = VM_PROT_EXECUTE; if ((fp->f_flag & FREAD) != 0) maxprot |= VM_PROT_READ; else if ((prot & VM_PROT_READ) != 0) return (EACCES); /* * If we are sharing potential changes via MAP_SHARED and we * are trying to get write permission although we opened it * without asking for it, bail out. */ if ((flags & MAP_SHARED) != 0) { if ((fp->f_flag & FWRITE) != 0) maxprot |= VM_PROT_WRITE; else if ((prot & VM_PROT_WRITE) != 0) return (EACCES); } else { maxprot |= VM_PROT_WRITE; cap_maxprot |= VM_PROT_WRITE; } maxprot &= cap_maxprot; /* * For regular files and shared memory, POSIX requires that * the value of foff be a legitimate offset within the data * object. In particular, negative offsets are invalid. * Blocking negative offsets and overflows here avoids * possible wraparound or user-level access into reserved * ranges of the data object later. In contrast, POSIX does * not dictate how offsets are used by device drivers, so in * the case of a device mapping a negative offset is passed * on. */ if ( #ifdef _LP64 size > OFF_MAX || #endif foff < 0 || foff > OFF_MAX - size) return (EINVAL); writecounted = FALSE; error = vm_mmap_vnode(td, size, prot, &maxprot, &flags, vp, &foff, &object, &writecounted); if (error != 0) return (error); error = vm_mmap_object(map, addr, size, prot, maxprot, flags, object, foff, writecounted, td); if (error != 0) { /* * If this mapping was accounted for in the vnode's * writecount, then undo that now. */ if (writecounted) vm_pager_release_writecount(object, 0, size); vm_object_deallocate(object); } #ifdef HWPMC_HOOKS /* Inform hwpmc(4) if an executable is being mapped. */ if (PMC_HOOK_INSTALLED(PMC_FN_MMAP)) { if ((prot & VM_PROT_EXECUTE) != 0 && error == 0) { pkm.pm_file = vp; pkm.pm_address = (uintptr_t) *addr; PMC_CALL_HOOK_UNLOCKED(td, PMC_FN_MMAP, (void *) &pkm); } } #endif return (error); } void vn_fsid(struct vnode *vp, struct vattr *va) { fsid_t *f; f = &vp->v_mount->mnt_stat.f_fsid; va->va_fsid = (uint32_t)f->val[1]; va->va_fsid <<= sizeof(f->val[1]) * NBBY; va->va_fsid += (uint32_t)f->val[0]; } int vn_fsync_buf(struct vnode *vp, int waitfor) { struct buf *bp, *nbp; struct bufobj *bo; struct mount *mp; int error, maxretry; error = 0; maxretry = 10000; /* large, arbitrarily chosen */ mp = NULL; if (vp->v_type == VCHR) { VI_LOCK(vp); mp = vp->v_rdev->si_mountpt; VI_UNLOCK(vp); } bo = &vp->v_bufobj; BO_LOCK(bo); loop1: /* * MARK/SCAN initialization to avoid infinite loops. */ TAILQ_FOREACH(bp, &bo->bo_dirty.bv_hd, b_bobufs) { bp->b_vflags &= ~BV_SCANNED; bp->b_error = 0; } /* * Flush all dirty buffers associated with a vnode. */ loop2: TAILQ_FOREACH_SAFE(bp, &bo->bo_dirty.bv_hd, b_bobufs, nbp) { if ((bp->b_vflags & BV_SCANNED) != 0) continue; bp->b_vflags |= BV_SCANNED; if (BUF_LOCK(bp, LK_EXCLUSIVE | LK_NOWAIT, NULL)) { if (waitfor != MNT_WAIT) continue; if (BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK | LK_SLEEPFAIL, BO_LOCKPTR(bo)) != 0) { BO_LOCK(bo); goto loop1; } BO_LOCK(bo); } BO_UNLOCK(bo); KASSERT(bp->b_bufobj == bo, ("bp %p wrong b_bufobj %p should be %p", bp, bp->b_bufobj, bo)); if ((bp->b_flags & B_DELWRI) == 0) panic("fsync: not dirty"); if ((vp->v_object != NULL) && (bp->b_flags & B_CLUSTEROK)) { vfs_bio_awrite(bp); } else { bremfree(bp); bawrite(bp); } if (maxretry < 1000) pause("dirty", hz < 1000 ? 1 : hz / 1000); BO_LOCK(bo); goto loop2; } /* * If synchronous the caller expects us to completely resolve all * dirty buffers in the system. Wait for in-progress I/O to * complete (which could include background bitmap writes), then * retry if dirty blocks still exist. */ if (waitfor == MNT_WAIT) { bufobj_wwait(bo, 0, 0); if (bo->bo_dirty.bv_cnt > 0) { /* * If we are unable to write any of these buffers * then we fail now rather than trying endlessly * to write them out. */ TAILQ_FOREACH(bp, &bo->bo_dirty.bv_hd, b_bobufs) if ((error = bp->b_error) != 0) break; if ((mp != NULL && mp->mnt_secondary_writes > 0) || (error == 0 && --maxretry >= 0)) goto loop1; if (error == 0) error = EAGAIN; } } BO_UNLOCK(bo); if (error != 0) vn_printf(vp, "fsync: giving up on dirty (error = %d) ", error); return (error); } /* * Copies a byte range from invp to outvp. Calls VOP_COPY_FILE_RANGE() * or vn_generic_copy_file_range() after rangelocking the byte ranges, * to do the actual copy. * vn_generic_copy_file_range() is factored out, so it can be called * from a VOP_COPY_FILE_RANGE() call as well, but handles vnodes from * different file systems. */ int vn_copy_file_range(struct vnode *invp, off_t *inoffp, struct vnode *outvp, off_t *outoffp, size_t *lenp, unsigned int flags, struct ucred *incred, struct ucred *outcred, struct thread *fsize_td) { int error; size_t len; uint64_t uvalin, uvalout; len = *lenp; *lenp = 0; /* For error returns. */ error = 0; /* Do some sanity checks on the arguments. */ uvalin = *inoffp; uvalin += len; uvalout = *outoffp; uvalout += len; if (invp->v_type == VDIR || outvp->v_type == VDIR) error = EISDIR; else if (*inoffp < 0 || uvalin > INT64_MAX || uvalin < (uint64_t)*inoffp || *outoffp < 0 || uvalout > INT64_MAX || uvalout < (uint64_t)*outoffp || invp->v_type != VREG || outvp->v_type != VREG) error = EINVAL; if (error != 0) goto out; /* * If the two vnode are for the same file system, call * VOP_COPY_FILE_RANGE(), otherwise call vn_generic_copy_file_range() * which can handle copies across multiple file systems. */ *lenp = len; if (invp->v_mount == outvp->v_mount) error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, lenp, flags, incred, outcred, fsize_td); else error = vn_generic_copy_file_range(invp, inoffp, outvp, outoffp, lenp, flags, incred, outcred, fsize_td); out: return (error); } /* * Test len bytes of data starting at dat for all bytes == 0. * Return true if all bytes are zero, false otherwise. * Expects dat to be well aligned. */ static bool mem_iszero(void *dat, int len) { int i; const u_int *p; const char *cp; for (p = dat; len > 0; len -= sizeof(*p), p++) { if (len >= sizeof(*p)) { if (*p != 0) return (false); } else { cp = (const char *)p; for (i = 0; i < len; i++, cp++) if (*cp != '\0') return (false); } } return (true); } /* * Look for a hole in the output file and, if found, adjust *outoffp * and *xferp to skip past the hole. * *xferp is the entire hole length to be written and xfer2 is how many bytes * to be written as 0's upon return. */ static off_t vn_skip_hole(struct vnode *outvp, off_t xfer2, off_t *outoffp, off_t *xferp, off_t *dataoffp, off_t *holeoffp, struct ucred *cred) { int error; off_t delta; if (*holeoffp == 0 || *holeoffp <= *outoffp) { *dataoffp = *outoffp; error = VOP_IOCTL(outvp, FIOSEEKDATA, dataoffp, 0, cred, curthread); if (error == 0) { *holeoffp = *dataoffp; error = VOP_IOCTL(outvp, FIOSEEKHOLE, holeoffp, 0, cred, curthread); } if (error != 0 || *holeoffp == *dataoffp) { /* * Since outvp is unlocked, it may be possible for * another thread to do a truncate(), lseek(), write() * creating a hole at startoff between the above * VOP_IOCTL() calls, if the other thread does not do * rangelocking. * If that happens, *holeoffp == *dataoffp and finding * the hole has failed, so disable vn_skip_hole(). */ *holeoffp = -1; /* Disable use of vn_skip_hole(). */ return (xfer2); } KASSERT(*dataoffp >= *outoffp, ("vn_skip_hole: dataoff=%jd < outoff=%jd", (intmax_t)*dataoffp, (intmax_t)*outoffp)); KASSERT(*holeoffp > *dataoffp, ("vn_skip_hole: holeoff=%jd <= dataoff=%jd", (intmax_t)*holeoffp, (intmax_t)*dataoffp)); } /* * If there is a hole before the data starts, advance *outoffp and * *xferp past the hole. */ if (*dataoffp > *outoffp) { delta = *dataoffp - *outoffp; if (delta >= *xferp) { /* Entire *xferp is a hole. */ *outoffp += *xferp; *xferp = 0; return (0); } *xferp -= delta; *outoffp += delta; xfer2 = MIN(xfer2, *xferp); } /* * If a hole starts before the end of this xfer2, reduce this xfer2 so * that the write ends at the start of the hole. * *holeoffp should always be greater than *outoffp, but for the * non-INVARIANTS case, check this to make sure xfer2 remains a sane * value. */ if (*holeoffp > *outoffp && *holeoffp < *outoffp + xfer2) xfer2 = *holeoffp - *outoffp; return (xfer2); } /* * Write an xfer sized chunk to outvp in blksize blocks from dat. * dat is a maximum of blksize in length and can be written repeatedly in * the chunk. * If growfile == true, just grow the file via vn_truncate_locked() instead * of doing actual writes. * If checkhole == true, a hole is being punched, so skip over any hole * already in the output file. */ static int vn_write_outvp(struct vnode *outvp, char *dat, off_t outoff, off_t xfer, u_long blksize, bool growfile, bool checkhole, struct ucred *cred) { struct mount *mp; off_t dataoff, holeoff, xfer2; int error, lckf; /* * Loop around doing writes of blksize until write has been completed. * Lock/unlock on each loop iteration so that a bwillwrite() can be * done for each iteration, since the xfer argument can be very * large if there is a large hole to punch in the output file. */ error = 0; holeoff = 0; do { xfer2 = MIN(xfer, blksize); if (checkhole) { /* * Punching a hole. Skip writing if there is * already a hole in the output file. */ xfer2 = vn_skip_hole(outvp, xfer2, &outoff, &xfer, &dataoff, &holeoff, cred); if (xfer == 0) break; if (holeoff < 0) checkhole = false; KASSERT(xfer2 > 0, ("vn_write_outvp: xfer2=%jd", (intmax_t)xfer2)); } bwillwrite(); mp = NULL; error = vn_start_write(outvp, &mp, V_WAIT); if (error == 0) { if (MNT_SHARED_WRITES(mp)) lckf = LK_SHARED; else lckf = LK_EXCLUSIVE; error = vn_lock(outvp, lckf); } if (error == 0) { if (growfile) error = vn_truncate_locked(outvp, outoff + xfer, false, cred); else { error = vn_rdwr(UIO_WRITE, outvp, dat, xfer2, outoff, UIO_SYSSPACE, IO_NODELOCKED, curthread->td_ucred, cred, NULL, curthread); outoff += xfer2; xfer -= xfer2; } VOP_UNLOCK(outvp); } if (mp != NULL) vn_finished_write(mp); } while (!growfile && xfer > 0 && error == 0); return (error); } /* * Copy a byte range of one file to another. This function can handle the * case where invp and outvp are on different file systems. * It can also be called by a VOP_COPY_FILE_RANGE() to do the work, if there * is no better file system specific way to do it. */ int vn_generic_copy_file_range(struct vnode *invp, off_t *inoffp, struct vnode *outvp, off_t *outoffp, size_t *lenp, unsigned int flags, struct ucred *incred, struct ucred *outcred, struct thread *fsize_td) { struct vattr va; struct mount *mp; struct uio io; off_t startoff, endoff, xfer, xfer2; u_long blksize; int error; bool cantseek, readzeros, eof, lastblock; ssize_t aresid; size_t copylen, len, savlen; char *dat; long holein, holeout; holein = holeout = 0; savlen = len = *lenp; error = 0; dat = NULL; error = vn_lock(invp, LK_SHARED); if (error != 0) goto out; if (VOP_PATHCONF(invp, _PC_MIN_HOLE_SIZE, &holein) != 0) holein = 0; VOP_UNLOCK(invp); mp = NULL; error = vn_start_write(outvp, &mp, V_WAIT); if (error == 0) error = vn_lock(outvp, LK_EXCLUSIVE); if (error == 0) { /* * If fsize_td != NULL, do a vn_rlimit_fsize() call, * now that outvp is locked. */ if (fsize_td != NULL) { io.uio_offset = *outoffp; io.uio_resid = len; error = vn_rlimit_fsize(outvp, &io, fsize_td); if (error != 0) error = EFBIG; } if (VOP_PATHCONF(outvp, _PC_MIN_HOLE_SIZE, &holeout) != 0) holeout = 0; /* * Holes that are past EOF do not need to be written as a block * of zero bytes. So, truncate the output file as far as * possible and then use va.va_size to decide if writing 0 * bytes is necessary in the loop below. */ if (error == 0) error = VOP_GETATTR(outvp, &va, outcred); if (error == 0 && va.va_size > *outoffp && va.va_size <= *outoffp + len) { #ifdef MAC error = mac_vnode_check_write(curthread->td_ucred, outcred, outvp); if (error == 0) #endif error = vn_truncate_locked(outvp, *outoffp, false, outcred); if (error == 0) va.va_size = *outoffp; } VOP_UNLOCK(outvp); } if (mp != NULL) vn_finished_write(mp); if (error != 0) goto out; /* * Set the blksize to the larger of the hole sizes for invp and outvp. * If hole sizes aren't available, set the blksize to the larger * f_iosize of invp and outvp. * This code expects the hole sizes and f_iosizes to be powers of 2. * This value is clipped at 4Kbytes and 1Mbyte. */ blksize = MAX(holein, holeout); if (blksize == 0) blksize = MAX(invp->v_mount->mnt_stat.f_iosize, outvp->v_mount->mnt_stat.f_iosize); if (blksize < 4096) blksize = 4096; else if (blksize > 1024 * 1024) blksize = 1024 * 1024; dat = malloc(blksize, M_TEMP, M_WAITOK); /* * If VOP_IOCTL(FIOSEEKHOLE) works for invp, use it and FIOSEEKDATA * to find holes. Otherwise, just scan the read block for all 0s * in the inner loop where the data copying is done. * Note that some file systems such as NFSv3, NFSv4.0 and NFSv4.1 may * support holes on the server, but do not support FIOSEEKHOLE. */ eof = false; while (len > 0 && error == 0 && !eof) { endoff = 0; /* To shut up compilers. */ cantseek = true; startoff = *inoffp; copylen = len; /* * Find the next data area. If there is just a hole to EOF, * FIOSEEKDATA should fail and then we drop down into the * inner loop and create the hole on the outvp file. * (I do not know if any file system will report a hole to * EOF via FIOSEEKHOLE, but I am pretty sure FIOSEEKDATA * will fail for those file systems.) * * For input files that don't support FIOSEEKDATA/FIOSEEKHOLE, * the code just falls through to the inner copy loop. */ error = EINVAL; if (holein > 0) error = VOP_IOCTL(invp, FIOSEEKDATA, &startoff, 0, incred, curthread); if (error == 0) { endoff = startoff; error = VOP_IOCTL(invp, FIOSEEKHOLE, &endoff, 0, incred, curthread); /* * Since invp is unlocked, it may be possible for * another thread to do a truncate(), lseek(), write() * creating a hole at startoff between the above * VOP_IOCTL() calls, if the other thread does not do * rangelocking. * If that happens, startoff == endoff and finding * the hole has failed, so set an error. */ if (error == 0 && startoff == endoff) error = EINVAL; /* Any error. Reset to 0. */ } if (error == 0) { if (startoff > *inoffp) { /* Found hole before data block. */ xfer = MIN(startoff - *inoffp, len); if (*outoffp < va.va_size) { /* Must write 0s to punch hole. */ xfer2 = MIN(va.va_size - *outoffp, xfer); memset(dat, 0, MIN(xfer2, blksize)); error = vn_write_outvp(outvp, dat, *outoffp, xfer2, blksize, false, holeout > 0, outcred); } if (error == 0 && *outoffp + xfer > va.va_size && xfer == len) /* Grow last block. */ error = vn_write_outvp(outvp, dat, *outoffp, xfer, blksize, true, false, outcred); if (error == 0) { *inoffp += xfer; *outoffp += xfer; len -= xfer; } } copylen = MIN(len, endoff - startoff); cantseek = false; } else { cantseek = true; startoff = *inoffp; copylen = len; error = 0; } xfer = blksize; if (cantseek) { /* * Set first xfer to end at a block boundary, so that * holes are more likely detected in the loop below via * the for all bytes 0 method. */ xfer -= (*inoffp % blksize); } /* Loop copying the data block. */ while (copylen > 0 && error == 0 && !eof) { if (copylen < xfer) xfer = copylen; error = vn_lock(invp, LK_SHARED); if (error != 0) goto out; error = vn_rdwr(UIO_READ, invp, dat, xfer, startoff, UIO_SYSSPACE, IO_NODELOCKED, curthread->td_ucred, incred, &aresid, curthread); VOP_UNLOCK(invp); lastblock = false; if (error == 0 && aresid > 0) { /* Stop the copy at EOF on the input file. */ xfer -= aresid; eof = true; lastblock = true; } if (error == 0) { /* * Skip the write for holes past the initial EOF * of the output file, unless this is the last * write of the output file at EOF. */ readzeros = cantseek ? mem_iszero(dat, xfer) : false; if (xfer == len) lastblock = true; if (!cantseek || *outoffp < va.va_size || lastblock || !readzeros) error = vn_write_outvp(outvp, dat, *outoffp, xfer, blksize, readzeros && lastblock && *outoffp >= va.va_size, false, outcred); if (error == 0) { *inoffp += xfer; startoff += xfer; *outoffp += xfer; copylen -= xfer; len -= xfer; } } xfer = blksize; } } out: *lenp = savlen - len; free(dat, M_TEMP); return (error); } static int vn_fallocate(struct file *fp, off_t offset, off_t len, struct thread *td) { struct mount *mp; struct vnode *vp; off_t olen, ooffset; int error; #ifdef AUDIT int audited_vnode1 = 0; #endif vp = fp->f_vnode; if (vp->v_type != VREG) return (ENODEV); /* Allocating blocks may take a long time, so iterate. */ for (;;) { olen = len; ooffset = offset; bwillwrite(); mp = NULL; error = vn_start_write(vp, &mp, V_WAIT | PCATCH); if (error != 0) break; error = vn_lock(vp, LK_EXCLUSIVE); if (error != 0) { vn_finished_write(mp); break; } #ifdef AUDIT if (!audited_vnode1) { AUDIT_ARG_VNODE1(vp); audited_vnode1 = 1; } #endif #ifdef MAC error = mac_vnode_check_write(td->td_ucred, fp->f_cred, vp); if (error == 0) #endif error = VOP_ALLOCATE(vp, &offset, &len); VOP_UNLOCK(vp); vn_finished_write(mp); if (olen + ooffset != offset + len) { panic("offset + len changed from %jx/%jx to %jx/%jx", ooffset, olen, offset, len); } if (error != 0 || len == 0) break; KASSERT(olen > len, ("Iteration did not make progress?")); maybe_yield(); } return (error); } Index: projects/clang1000-import/sys/mips/include/pcpu.h =================================================================== --- projects/clang1000-import/sys/mips/include/pcpu.h (revision 357965) +++ projects/clang1000-import/sys/mips/include/pcpu.h (revision 357966) @@ -1,100 +1,100 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 1999 Luoqi Chen * Copyright (c) Peter Wemm * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * from: src/sys/alpha/include/pcpu.h,v 1.15 2004/11/05 19:16:44 jhb * $FreeBSD$ */ #ifndef _MACHINE_PCPU_H_ #define _MACHINE_PCPU_H_ #include #include #define PCPU_MD_COMMON_FIELDS \ pd_entry_t *pc_segbase; /* curthread segbase */ \ struct pmap *pc_curpmap; /* pmap of curthread */ \ u_int32_t pc_next_asid; /* next ASID to alloc */ \ u_int32_t pc_asid_generation; /* current ASID generation */ \ u_int pc_pending_ipis; /* IPIs pending to this CPU */ \ struct pcpu *pc_self; /* globally-uniqe self pointer */ #ifdef __mips_n64 #define PCPU_MD_MIPS64_FIELDS \ PCPU_MD_COMMON_FIELDS \ char __pad[245] #else #define PCPU_MD_MIPS32_FIELDS \ PCPU_MD_COMMON_FIELDS \ pt_entry_t *pc_cmap1_ptep; /* PTE for copy window 1 KVA */ \ pt_entry_t *pc_cmap2_ptep; /* PTE for copy window 2 KVA */ \ vm_offset_t pc_cmap1_addr; /* KVA page for copy window 1 */ \ vm_offset_t pc_cmap2_addr; /* KVA page for copy window 2 */ \ vm_offset_t pc_qmap_addr; /* KVA page for temporary mappings */ \ pt_entry_t *pc_qmap_ptep; /* PTE for temporary mapping KVA */ \ - char __pad[101] + char __pad[97] #endif #ifdef __mips_n64 #define PCPU_MD_FIELDS PCPU_MD_MIPS64_FIELDS #else #define PCPU_MD_FIELDS PCPU_MD_MIPS32_FIELDS #endif #ifdef _KERNEL extern char pcpu_space[MAXCPU][PAGE_SIZE * 2]; #define PCPU_ADDR(cpu) (struct pcpu *)(pcpu_space[(cpu)]) extern struct pcpu *pcpup; #define PCPUP pcpup /* * Since we use a wired TLB entry to map the same VA to a different * physical page for each CPU, get_pcpu() must use the pc_self * field to obtain a globally-unique pointer. */ #define get_pcpu() (PCPUP->pc_self) #define PCPU_ADD(member, value) (PCPUP->pc_ ## member += (value)) #define PCPU_GET(member) (PCPUP->pc_ ## member) #define PCPU_INC(member) PCPU_ADD(member, 1) #define PCPU_PTR(member) (&PCPUP->pc_ ## member) #define PCPU_SET(member,value) (PCPUP->pc_ ## member = (value)) #define PCPU_LAZY_INC(member) (++PCPUP->pc_ ## member) #ifdef SMP /* * Instantiate the wired TLB entry at PCPU_TLB_ENTRY to map 'pcpu' at 'pcpup'. */ void mips_pcpu_tlb_init(struct pcpu *pcpu); #endif #endif /* _KERNEL */ #endif /* !_MACHINE_PCPU_H_ */ Index: projects/clang1000-import/sys/netinet/sctp_syscalls.c =================================================================== --- projects/clang1000-import/sys/netinet/sctp_syscalls.c (revision 357965) +++ projects/clang1000-import/sys/netinet/sctp_syscalls.c (revision 357966) @@ -1,578 +1,578 @@ /*- * Copyright (c) 1982, 1986, 1989, 1990, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #include __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_inet.h" #include "opt_inet6.h" #include "opt_sctp.h" #include "opt_ktrace.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef KTRACE #include #endif #ifdef COMPAT_FREEBSD32 #include #endif #include #include #include #include #include static struct syscall_helper_data sctp_syscalls[] = { SYSCALL_INIT_HELPER_F(sctp_peeloff, SYF_CAPENABLED), SYSCALL_INIT_HELPER_F(sctp_generic_sendmsg, SYF_CAPENABLED), SYSCALL_INIT_HELPER_F(sctp_generic_sendmsg_iov, SYF_CAPENABLED), SYSCALL_INIT_HELPER_F(sctp_generic_recvmsg, SYF_CAPENABLED), SYSCALL_INIT_LAST }; static void sctp_syscalls_init(void *unused __unused) { int error __unused; error = syscall_helper_register(sctp_syscalls, SY_THR_STATIC); KASSERT((error == 0), ("%s: syscall_helper_register failed for sctp syscalls", __func__)); #ifdef COMPAT_FREEBSD32 error = syscall32_helper_register(sctp_syscalls, SY_THR_STATIC); KASSERT((error == 0), ("%s: syscall32_helper_register failed for sctp syscalls", __func__)); #endif } SYSINIT(sctp_syscalls, SI_SUB_SYSCALLS, SI_ORDER_ANY, sctp_syscalls_init, NULL); /* * SCTP syscalls. * Functionality only compiled in if SCTP is defined in the kernel Makefile, * otherwise all return EOPNOTSUPP. * XXX: We should make this loadable one day. */ int sys_sctp_peeloff(td, uap) struct thread *td; struct sctp_peeloff_args /* { int sd; caddr_t name; } */ *uap; { #if (defined(INET) || defined(INET6)) && defined(SCTP) struct file *headfp, *nfp = NULL; struct socket *head, *so; cap_rights_t rights; u_int fflag; int error, fd; AUDIT_ARG_FD(uap->sd); error = getsock_cap(td, uap->sd, cap_rights_init(&rights, CAP_PEELOFF), &headfp, &fflag, NULL); if (error != 0) goto done2; head = headfp->f_data; if (head->so_proto->pr_protocol != IPPROTO_SCTP) { error = EOPNOTSUPP; goto done; } error = sctp_can_peel_off(head, (sctp_assoc_t)uap->name); if (error != 0) goto done; /* * At this point we know we do have a assoc to pull * we proceed to get the fd setup. This may block * but that is ok. */ error = falloc(td, &nfp, &fd, 0); if (error != 0) goto done; td->td_retval[0] = fd; CURVNET_SET(head->so_vnet); so = sopeeloff(head); if (so == NULL) { error = ENOMEM; goto noconnection; } finit(nfp, fflag, DTYPE_SOCKET, so, &socketops); error = sctp_do_peeloff(head, so, (sctp_assoc_t)uap->name); if (error != 0) goto noconnection; if (head->so_sigio != NULL) fsetown(fgetown(&head->so_sigio), &so->so_sigio); noconnection: /* * close the new descriptor, assuming someone hasn't ripped it * out from under us. */ if (error != 0) fdclose(td, nfp, fd); /* * Release explicitly held references before returning. */ CURVNET_RESTORE(); done: if (nfp != NULL) fdrop(nfp, td); fdrop(headfp, td); done2: return (error); #else /* SCTP */ return (EOPNOTSUPP); #endif /* SCTP */ } int sys_sctp_generic_sendmsg (td, uap) struct thread *td; struct sctp_generic_sendmsg_args /* { int sd, caddr_t msg, int mlen, caddr_t to, __socklen_t tolen, struct sctp_sndrcvinfo *sinfo, int flags } */ *uap; { #if (defined(INET) || defined(INET6)) && defined(SCTP) struct sctp_sndrcvinfo sinfo, *u_sinfo = NULL; struct socket *so; struct file *fp = NULL; struct sockaddr *to = NULL; #ifdef KTRACE struct uio *ktruio = NULL; #endif struct uio auio; struct iovec iov[1]; cap_rights_t rights; int error = 0, len; if (uap->sinfo != NULL) { error = copyin(uap->sinfo, &sinfo, sizeof (sinfo)); if (error != 0) return (error); u_sinfo = &sinfo; } - cap_rights_init(&rights, CAP_SEND); + cap_rights_init_one(&rights, CAP_SEND); if (uap->tolen != 0) { error = getsockaddr(&to, uap->to, uap->tolen); if (error != 0) { to = NULL; goto sctp_bad2; } - cap_rights_set(&rights, CAP_CONNECT); + cap_rights_set_one(&rights, CAP_CONNECT); } AUDIT_ARG_FD(uap->sd); error = getsock_cap(td, uap->sd, &rights, &fp, NULL, NULL); if (error != 0) goto sctp_bad; #ifdef KTRACE if (to && (KTRPOINT(td, KTR_STRUCT))) ktrsockaddr(to); #endif iov[0].iov_base = uap->msg; iov[0].iov_len = uap->mlen; so = (struct socket *)fp->f_data; if (so->so_proto->pr_protocol != IPPROTO_SCTP) { error = EOPNOTSUPP; goto sctp_bad; } #ifdef MAC error = mac_socket_check_send(td->td_ucred, so); if (error != 0) goto sctp_bad; #endif /* MAC */ auio.uio_iov = iov; auio.uio_iovcnt = 1; auio.uio_segflg = UIO_USERSPACE; auio.uio_rw = UIO_WRITE; auio.uio_td = td; auio.uio_offset = 0; /* XXX */ auio.uio_resid = 0; #ifdef KTRACE if (KTRPOINT(td, KTR_GENIO)) ktruio = cloneuio(&auio); #endif /* KTRACE */ len = auio.uio_resid = uap->mlen; CURVNET_SET(so->so_vnet); error = sctp_lower_sosend(so, to, &auio, (struct mbuf *)NULL, (struct mbuf *)NULL, uap->flags, u_sinfo, td); CURVNET_RESTORE(); if (error != 0) { if (auio.uio_resid != len && (error == ERESTART || error == EINTR || error == EWOULDBLOCK)) error = 0; /* Generation of SIGPIPE can be controlled per socket. */ if (error == EPIPE && !(so->so_options & SO_NOSIGPIPE) && !(uap->flags & MSG_NOSIGNAL)) { PROC_LOCK(td->td_proc); tdsignal(td, SIGPIPE); PROC_UNLOCK(td->td_proc); } } if (error == 0) td->td_retval[0] = len - auio.uio_resid; #ifdef KTRACE if (ktruio != NULL) { ktruio->uio_resid = td->td_retval[0]; ktrgenio(uap->sd, UIO_WRITE, ktruio, error); } #endif /* KTRACE */ sctp_bad: if (fp != NULL) fdrop(fp, td); sctp_bad2: free(to, M_SONAME); return (error); #else /* SCTP */ return (EOPNOTSUPP); #endif /* SCTP */ } int sys_sctp_generic_sendmsg_iov(td, uap) struct thread *td; struct sctp_generic_sendmsg_iov_args /* { int sd, struct iovec *iov, int iovlen, caddr_t to, __socklen_t tolen, struct sctp_sndrcvinfo *sinfo, int flags } */ *uap; { #if (defined(INET) || defined(INET6)) && defined(SCTP) struct sctp_sndrcvinfo sinfo, *u_sinfo = NULL; struct socket *so; struct file *fp = NULL; struct sockaddr *to = NULL; #ifdef KTRACE struct uio *ktruio = NULL; #endif struct uio auio; struct iovec *iov, *tiov; cap_rights_t rights; ssize_t len; int error, i; if (uap->sinfo != NULL) { error = copyin(uap->sinfo, &sinfo, sizeof (sinfo)); if (error != 0) return (error); u_sinfo = &sinfo; } - cap_rights_init(&rights, CAP_SEND); + cap_rights_init_one(&rights, CAP_SEND); if (uap->tolen != 0) { error = getsockaddr(&to, uap->to, uap->tolen); if (error != 0) { to = NULL; goto sctp_bad2; } - cap_rights_set(&rights, CAP_CONNECT); + cap_rights_set_one(&rights, CAP_CONNECT); } AUDIT_ARG_FD(uap->sd); error = getsock_cap(td, uap->sd, &rights, &fp, NULL, NULL); if (error != 0) goto sctp_bad1; #ifdef COMPAT_FREEBSD32 if (SV_CURPROC_FLAG(SV_ILP32)) error = freebsd32_copyiniov((struct iovec32 *)uap->iov, uap->iovlen, &iov, EMSGSIZE); else #endif error = copyiniov(uap->iov, uap->iovlen, &iov, EMSGSIZE); if (error != 0) goto sctp_bad1; #ifdef KTRACE if (to && (KTRPOINT(td, KTR_STRUCT))) ktrsockaddr(to); #endif so = (struct socket *)fp->f_data; if (so->so_proto->pr_protocol != IPPROTO_SCTP) { error = EOPNOTSUPP; goto sctp_bad; } #ifdef MAC error = mac_socket_check_send(td->td_ucred, so); if (error != 0) goto sctp_bad; #endif /* MAC */ auio.uio_iov = iov; auio.uio_iovcnt = uap->iovlen; auio.uio_segflg = UIO_USERSPACE; auio.uio_rw = UIO_WRITE; auio.uio_td = td; auio.uio_offset = 0; /* XXX */ auio.uio_resid = 0; tiov = iov; for (i = 0; i iovlen; i++, tiov++) { if ((auio.uio_resid += tiov->iov_len) < 0) { error = EINVAL; goto sctp_bad; } } #ifdef KTRACE if (KTRPOINT(td, KTR_GENIO)) ktruio = cloneuio(&auio); #endif /* KTRACE */ len = auio.uio_resid; CURVNET_SET(so->so_vnet); error = sctp_lower_sosend(so, to, &auio, (struct mbuf *)NULL, (struct mbuf *)NULL, uap->flags, u_sinfo, td); CURVNET_RESTORE(); if (error != 0) { if (auio.uio_resid != len && (error == ERESTART || error == EINTR || error == EWOULDBLOCK)) error = 0; /* Generation of SIGPIPE can be controlled per socket */ if (error == EPIPE && !(so->so_options & SO_NOSIGPIPE) && !(uap->flags & MSG_NOSIGNAL)) { PROC_LOCK(td->td_proc); tdsignal(td, SIGPIPE); PROC_UNLOCK(td->td_proc); } } if (error == 0) td->td_retval[0] = len - auio.uio_resid; #ifdef KTRACE if (ktruio != NULL) { ktruio->uio_resid = td->td_retval[0]; ktrgenio(uap->sd, UIO_WRITE, ktruio, error); } #endif /* KTRACE */ sctp_bad: free(iov, M_IOV); sctp_bad1: if (fp != NULL) fdrop(fp, td); sctp_bad2: free(to, M_SONAME); return (error); #else /* SCTP */ return (EOPNOTSUPP); #endif /* SCTP */ } int sys_sctp_generic_recvmsg(td, uap) struct thread *td; struct sctp_generic_recvmsg_args /* { int sd, struct iovec *iov, int iovlen, struct sockaddr *from, __socklen_t *fromlenaddr, struct sctp_sndrcvinfo *sinfo, int *msg_flags } */ *uap; { #if (defined(INET) || defined(INET6)) && defined(SCTP) uint8_t sockbufstore[256]; struct uio auio; struct iovec *iov, *tiov; struct sctp_sndrcvinfo sinfo; struct socket *so; struct file *fp = NULL; struct sockaddr *fromsa; cap_rights_t rights; #ifdef KTRACE struct uio *ktruio = NULL; #endif ssize_t len; int error, fromlen, i, msg_flags; AUDIT_ARG_FD(uap->sd); error = getsock_cap(td, uap->sd, cap_rights_init(&rights, CAP_RECV), &fp, NULL, NULL); if (error != 0) return (error); #ifdef COMPAT_FREEBSD32 if (SV_CURPROC_FLAG(SV_ILP32)) error = freebsd32_copyiniov((struct iovec32 *)uap->iov, uap->iovlen, &iov, EMSGSIZE); else #endif error = copyiniov(uap->iov, uap->iovlen, &iov, EMSGSIZE); if (error != 0) goto out1; so = fp->f_data; if (so->so_proto->pr_protocol != IPPROTO_SCTP) { error = EOPNOTSUPP; goto out; } #ifdef MAC error = mac_socket_check_receive(td->td_ucred, so); if (error != 0) goto out; #endif /* MAC */ if (uap->fromlenaddr != NULL) { error = copyin(uap->fromlenaddr, &fromlen, sizeof (fromlen)); if (error != 0) goto out; } else { fromlen = 0; } if (uap->msg_flags) { error = copyin(uap->msg_flags, &msg_flags, sizeof (int)); if (error != 0) goto out; } else { msg_flags = 0; } auio.uio_iov = iov; auio.uio_iovcnt = uap->iovlen; auio.uio_segflg = UIO_USERSPACE; auio.uio_rw = UIO_READ; auio.uio_td = td; auio.uio_offset = 0; /* XXX */ auio.uio_resid = 0; tiov = iov; for (i = 0; i iovlen; i++, tiov++) { if ((auio.uio_resid += tiov->iov_len) < 0) { error = EINVAL; goto out; } } len = auio.uio_resid; fromsa = (struct sockaddr *)sockbufstore; #ifdef KTRACE if (KTRPOINT(td, KTR_GENIO)) ktruio = cloneuio(&auio); #endif /* KTRACE */ memset(&sinfo, 0, sizeof(struct sctp_sndrcvinfo)); CURVNET_SET(so->so_vnet); error = sctp_sorecvmsg(so, &auio, (struct mbuf **)NULL, fromsa, fromlen, &msg_flags, (struct sctp_sndrcvinfo *)&sinfo, 1); CURVNET_RESTORE(); if (error != 0) { if (auio.uio_resid != len && (error == ERESTART || error == EINTR || error == EWOULDBLOCK)) error = 0; } else { if (uap->sinfo) error = copyout(&sinfo, uap->sinfo, sizeof (sinfo)); } #ifdef KTRACE if (ktruio != NULL) { ktruio->uio_resid = len - auio.uio_resid; ktrgenio(uap->sd, UIO_READ, ktruio, error); } #endif /* KTRACE */ if (error != 0) goto out; td->td_retval[0] = len - auio.uio_resid; if (fromlen && uap->from) { len = fromlen; if (len <= 0 || fromsa == NULL) len = 0; else { len = MIN(len, fromsa->sa_len); error = copyout(fromsa, uap->from, (size_t)len); if (error != 0) goto out; } error = copyout(&len, uap->fromlenaddr, sizeof (socklen_t)); if (error != 0) goto out; } #ifdef KTRACE if (KTRPOINT(td, KTR_STRUCT)) ktrsockaddr(fromsa); #endif if (uap->msg_flags) { error = copyout(&msg_flags, uap->msg_flags, sizeof (int)); if (error != 0) goto out; } out: free(iov, M_IOV); out1: if (fp != NULL) fdrop(fp, td); return (error); #else /* SCTP */ return (EOPNOTSUPP); #endif /* SCTP */ } Index: projects/clang1000-import/sys/sys/_cscan_atomic.h =================================================================== --- projects/clang1000-import/sys/sys/_cscan_atomic.h (revision 357965) +++ projects/clang1000-import/sys/sys/_cscan_atomic.h (revision 357966) @@ -1,307 +1,311 @@ /*- * SPDX-License-Identifier: BSD-2-Clause * * Copyright (c) 2019 Andrew Turner * * This software was developed by SRI International and the University of * Cambridge Computer Laboratory (Department of Computer Science and * Technology) under DARPA contract HR0011-18-C-0016 ("ECATS"), as part of the * DARPA SSITH research programme. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _SYS__CSAN_ATOMIC_H_ #define _SYS__CSAN_ATOMIC_H_ #ifndef _MACHINE_ATOMIC_H_ #error do not include this header, use machine/atomic.h #endif #define KCSAN_ATOMIC_FUNC_1(op, name, type) \ void kcsan_atomic_##op##_##name(volatile type *, type); \ void kcsan_atomic_##op##_acq_##name(volatile type *, type); \ void kcsan_atomic_##op##_rel_##name(volatile type *, type) #define KCSAN_ATOMIC_CMPSET(name, type) \ int kcsan_atomic_cmpset_##name(volatile type *, type, type); \ int kcsan_atomic_cmpset_acq_##name(volatile type *, type, type); \ int kcsan_atomic_cmpset_rel_##name(volatile type *, type, type) #define KCSAN_ATOMIC_FCMPSET(name, type) \ int kcsan_atomic_fcmpset_##name(volatile type *, type *, type); \ int kcsan_atomic_fcmpset_acq_##name(volatile type *, type *, type); \ int kcsan_atomic_fcmpset_rel_##name(volatile type *, type *, type) #define KCSAN_ATOMIC_READ(op, name, type) \ type kcsan_atomic_##op##_##name(volatile type *, type) #define KCSAN_ATOMIC_READANDCLEAR(name, type) \ type kcsan_atomic_readandclear_##name(volatile type *) #define KCSAN_ATOMIC_LOAD(name, type) \ type kcsan_atomic_load_##name(volatile type *); \ type kcsan_atomic_load_acq_##name(volatile type *) #define KCSAN_ATOMIC_STORE(name, type) \ void kcsan_atomic_store_##name(volatile type *, type); \ void kcsan_atomic_store_rel_##name(volatile type *, type) #define KCSAN_ATOMIC_TEST(op, name, type) \ int kcsan_atomic_##op##_##name(volatile type *, u_int) #define KCSAN_ATOMIC_FUNCS(name, type) \ KCSAN_ATOMIC_FUNC_1(add, name, type); \ KCSAN_ATOMIC_FUNC_1(clear, name, type); \ KCSAN_ATOMIC_CMPSET(name, type); \ KCSAN_ATOMIC_FCMPSET(name, type); \ KCSAN_ATOMIC_READ(fetchadd, name, type); \ KCSAN_ATOMIC_LOAD(name, type); \ KCSAN_ATOMIC_READANDCLEAR(name, type); \ KCSAN_ATOMIC_FUNC_1(set, name, type); \ KCSAN_ATOMIC_FUNC_1(subtract, name, type); \ KCSAN_ATOMIC_STORE(name, type); \ KCSAN_ATOMIC_READ(swap, name, type); \ KCSAN_ATOMIC_TEST(testandclear, name, type); \ KCSAN_ATOMIC_TEST(testandset, name, type) KCSAN_ATOMIC_FUNCS(int, u_int); KCSAN_ATOMIC_FUNCS(long, u_long); KCSAN_ATOMIC_FUNCS(ptr, uintptr_t); KCSAN_ATOMIC_FUNCS(8, uint8_t); KCSAN_ATOMIC_FUNCS(16, uint16_t); KCSAN_ATOMIC_FUNCS(32, uint32_t); KCSAN_ATOMIC_FUNCS(64, uint64_t); void kcsan_atomic_thread_fence_acq(void); void kcsan_atomic_thread_fence_acq_rel(void); void kcsan_atomic_thread_fence_rel(void); void kcsan_atomic_thread_fence_seq_cst(void); #ifndef KCSAN_RUNTIME #define atomic_add_int kcsan_atomic_add_int #define atomic_add_acq_int kcsan_atomic_add_acq_int #define atomic_add_rel_int kcsan_atomic_add_rel_int #define atomic_clear_int kcsan_atomic_clear_int #define atomic_clear_acq_int kcsan_atomic_clear_acq_int #define atomic_clear_rel_int kcsan_atomic_clear_rel_int #define atomic_cmpset_int kcsan_atomic_cmpset_int #define atomic_cmpset_acq_int kcsan_atomic_cmpset_acq_int #define atomic_cmpset_rel_int kcsan_atomic_cmpset_rel_int #define atomic_fcmpset_int kcsan_atomic_fcmpset_int #define atomic_fcmpset_acq_int kcsan_atomic_fcmpset_acq_int #define atomic_fcmpset_rel_int kcsan_atomic_fcmpset_rel_int #define atomic_fetchadd_int kcsan_atomic_fetchadd_int #define atomic_load_int kcsan_atomic_load_int #define atomic_load_acq_int kcsan_atomic_load_acq_int #define atomic_readandclear_int kcsan_atomic_readandclear_int #define atomic_set_int kcsan_atomic_set_int #define atomic_set_acq_int kcsan_atomic_set_acq_int #define atomic_set_rel_int kcsan_atomic_set_rel_int #define atomic_subtract_int kcsan_atomic_subtract_int #define atomic_subtract_acq_int kcsan_atomic_subtract_acq_int #define atomic_subtract_rel_int kcsan_atomic_subtract_rel_int #define atomic_store_int kcsan_atomic_store_int #define atomic_store_rel_int kcsan_atomic_store_rel_int #define atomic_swap_int kcsan_atomic_swap_int #define atomic_testandclear_int kcsan_atomic_testandclear_int #define atomic_testandset_int kcsan_atomic_testandset_int #define atomic_add_long kcsan_atomic_add_long #define atomic_add_acq_long kcsan_atomic_add_acq_long #define atomic_add_rel_long kcsan_atomic_add_rel_long #define atomic_clear_long kcsan_atomic_clear_long #define atomic_clear_acq_long kcsan_atomic_clear_acq_long #define atomic_clear_rel_long kcsan_atomic_clear_rel_long #define atomic_cmpset_long kcsan_atomic_cmpset_long #define atomic_cmpset_acq_long kcsan_atomic_cmpset_acq_long #define atomic_cmpset_rel_long kcsan_atomic_cmpset_rel_long #define atomic_fcmpset_long kcsan_atomic_fcmpset_long #define atomic_fcmpset_acq_long kcsan_atomic_fcmpset_acq_long #define atomic_fcmpset_rel_long kcsan_atomic_fcmpset_rel_long #define atomic_fetchadd_long kcsan_atomic_fetchadd_long #define atomic_load_long kcsan_atomic_load_long #define atomic_load_acq_long kcsan_atomic_load_acq_long #define atomic_readandclear_long kcsan_atomic_readandclear_long #define atomic_set_long kcsan_atomic_set_long #define atomic_set_acq_long kcsan_atomic_set_acq_long #define atomic_set_rel_long kcsan_atomic_set_rel_long #define atomic_subtract_long kcsan_atomic_subtract_long #define atomic_subtract_acq_long kcsan_atomic_subtract_acq_long #define atomic_subtract_rel_long kcsan_atomic_subtract_rel_long #define atomic_store_long kcsan_atomic_store_long #define atomic_store_rel_long kcsan_atomic_store_rel_long #define atomic_swap_long kcsan_atomic_swap_long #define atomic_testandclear_long kcsan_atomic_testandclear_long #define atomic_testandset_long kcsan_atomic_testandset_long #define atomic_add_ptr kcsan_atomic_add_ptr #define atomic_add_acq_ptr kcsan_atomic_add_acq_ptr #define atomic_add_rel_ptr kcsan_atomic_add_rel_ptr #define atomic_clear_ptr kcsan_atomic_clear_ptr #define atomic_clear_acq_ptr kcsan_atomic_clear_acq_ptr #define atomic_clear_rel_ptr kcsan_atomic_clear_rel_ptr #define atomic_cmpset_ptr kcsan_atomic_cmpset_ptr #define atomic_cmpset_acq_ptr kcsan_atomic_cmpset_acq_ptr #define atomic_cmpset_rel_ptr kcsan_atomic_cmpset_rel_ptr #define atomic_fcmpset_ptr kcsan_atomic_fcmpset_ptr #define atomic_fcmpset_acq_ptr kcsan_atomic_fcmpset_acq_ptr #define atomic_fcmpset_rel_ptr kcsan_atomic_fcmpset_rel_ptr #define atomic_fetchadd_ptr kcsan_atomic_fetchadd_ptr -#define atomic_load_ptr(x) kcsan_atomic_load_ptr((volatile uintptr_t *)(x)) +#define atomic_load_ptr(x) ({ \ + __typeof(*x) __retptr; \ + __retptr = (void *)kcsan_atomic_load_ptr((volatile uintptr_t *)(x)); \ + __retptr; \ +}) #define atomic_load_acq_ptr kcsan_atomic_load_acq_ptr #define atomic_readandclear_ptr kcsan_atomic_readandclear_ptr #define atomic_set_ptr kcsan_atomic_set_ptr #define atomic_set_acq_ptr kcsan_atomic_set_acq_ptr #define atomic_set_rel_ptr kcsan_atomic_set_rel_ptr #define atomic_subtract_ptr kcsan_atomic_subtract_ptr #define atomic_subtract_acq_ptr kcsan_atomic_subtract_acq_ptr #define atomic_subtract_rel_ptr kcsan_atomic_subtract_rel_ptr #define atomic_store_ptr kcsan_atomic_store_ptr #define atomic_store_rel_ptr kcsan_atomic_store_rel_ptr #define atomic_swap_ptr kcsan_atomic_swap_ptr #define atomic_testandclear_ptr kcsan_atomic_testandclear_ptr #define atomic_testandset_ptr kcsan_atomic_testandset_ptr #define atomic_add_8 kcsan_atomic_add_8 #define atomic_add_acq_8 kcsan_atomic_add_acq_8 #define atomic_add_rel_8 kcsan_atomic_add_rel_8 #define atomic_clear_8 kcsan_atomic_clear_8 #define atomic_clear_acq_8 kcsan_atomic_clear_acq_8 #define atomic_clear_rel_8 kcsan_atomic_clear_rel_8 #define atomic_cmpset_8 kcsan_atomic_cmpset_8 #define atomic_cmpset_acq_8 kcsan_atomic_cmpset_acq_8 #define atomic_cmpset_rel_8 kcsan_atomic_cmpset_rel_8 #define atomic_fcmpset_8 kcsan_atomic_fcmpset_8 #define atomic_fcmpset_acq_8 kcsan_atomic_fcmpset_acq_8 #define atomic_fcmpset_rel_8 kcsan_atomic_fcmpset_rel_8 #define atomic_fetchadd_8 kcsan_atomic_fetchadd_8 #define atomic_load_8 kcsan_atomic_load_8 #define atomic_load_acq_8 kcsan_atomic_load_acq_8 #define atomic_readandclear_8 kcsan_atomic_readandclear_8 #define atomic_set_8 kcsan_atomic_set_8 #define atomic_set_acq_8 kcsan_atomic_set_acq_8 #define atomic_set_rel_8 kcsan_atomic_set_rel_8 #define atomic_subtract_8 kcsan_atomic_subtract_8 #define atomic_subtract_acq_8 kcsan_atomic_subtract_acq_8 #define atomic_subtract_rel_8 kcsan_atomic_subtract_rel_8 #define atomic_store_8 kcsan_atomic_store_8 #define atomic_store_rel_8 kcsan_atomic_store_rel_8 #define atomic_swap_8 kcsan_atomic_swap_8 #define atomic_testandclear_8 kcsan_atomic_testandclear_8 #define atomic_testandset_8 kcsan_atomic_testandset_8 #define atomic_add_16 kcsan_atomic_add_16 #define atomic_add_acq_16 kcsan_atomic_add_acq_16 #define atomic_add_rel_16 kcsan_atomic_add_rel_16 #define atomic_clear_16 kcsan_atomic_clear_16 #define atomic_clear_acq_16 kcsan_atomic_clear_acq_16 #define atomic_clear_rel_16 kcsan_atomic_clear_rel_16 #define atomic_cmpset_16 kcsan_atomic_cmpset_16 #define atomic_cmpset_acq_16 kcsan_atomic_cmpset_acq_16 #define atomic_cmpset_rel_16 kcsan_atomic_cmpset_rel_16 #define atomic_fcmpset_16 kcsan_atomic_fcmpset_16 #define atomic_fcmpset_acq_16 kcsan_atomic_fcmpset_acq_16 #define atomic_fcmpset_rel_16 kcsan_atomic_fcmpset_rel_16 #define atomic_fetchadd_16 kcsan_atomic_fetchadd_16 #define atomic_load_16 kcsan_atomic_load_16 #define atomic_load_acq_16 kcsan_atomic_load_acq_16 #define atomic_readandclear_16 kcsan_atomic_readandclear_16 #define atomic_set_16 kcsan_atomic_set_16 #define atomic_set_acq_16 kcsan_atomic_set_acq_16 #define atomic_set_rel_16 kcsan_atomic_set_rel_16 #define atomic_subtract_16 kcsan_atomic_subtract_16 #define atomic_subtract_acq_16 kcsan_atomic_subtract_acq_16 #define atomic_subtract_rel_16 kcsan_atomic_subtract_rel_16 #define atomic_store_16 kcsan_atomic_store_16 #define atomic_store_rel_16 kcsan_atomic_store_rel_16 #define atomic_swap_16 kcsan_atomic_swap_16 #define atomic_testandclear_16 kcsan_atomic_testandclear_16 #define atomic_testandset_16 kcsan_atomic_testandset_16 #define atomic_add_32 kcsan_atomic_add_32 #define atomic_add_acq_32 kcsan_atomic_add_acq_32 #define atomic_add_rel_32 kcsan_atomic_add_rel_32 #define atomic_clear_32 kcsan_atomic_clear_32 #define atomic_clear_acq_32 kcsan_atomic_clear_acq_32 #define atomic_clear_rel_32 kcsan_atomic_clear_rel_32 #define atomic_cmpset_32 kcsan_atomic_cmpset_32 #define atomic_cmpset_acq_32 kcsan_atomic_cmpset_acq_32 #define atomic_cmpset_rel_32 kcsan_atomic_cmpset_rel_32 #define atomic_fcmpset_32 kcsan_atomic_fcmpset_32 #define atomic_fcmpset_acq_32 kcsan_atomic_fcmpset_acq_32 #define atomic_fcmpset_rel_32 kcsan_atomic_fcmpset_rel_32 #define atomic_fetchadd_32 kcsan_atomic_fetchadd_32 #define atomic_load_32 kcsan_atomic_load_32 #define atomic_load_acq_32 kcsan_atomic_load_acq_32 #define atomic_readandclear_32 kcsan_atomic_readandclear_32 #define atomic_set_32 kcsan_atomic_set_32 #define atomic_set_acq_32 kcsan_atomic_set_acq_32 #define atomic_set_rel_32 kcsan_atomic_set_rel_32 #define atomic_subtract_32 kcsan_atomic_subtract_32 #define atomic_subtract_acq_32 kcsan_atomic_subtract_acq_32 #define atomic_subtract_rel_32 kcsan_atomic_subtract_rel_32 #define atomic_store_32 kcsan_atomic_store_32 #define atomic_store_rel_32 kcsan_atomic_store_rel_32 #define atomic_swap_32 kcsan_atomic_swap_32 #define atomic_testandclear_32 kcsan_atomic_testandclear_32 #define atomic_testandset_32 kcsan_atomic_testandset_32 #define atomic_add_64 kcsan_atomic_add_64 #define atomic_add_acq_64 kcsan_atomic_add_acq_64 #define atomic_add_rel_64 kcsan_atomic_add_rel_64 #define atomic_clear_64 kcsan_atomic_clear_64 #define atomic_clear_acq_64 kcsan_atomic_clear_acq_64 #define atomic_clear_rel_64 kcsan_atomic_clear_rel_64 #define atomic_cmpset_64 kcsan_atomic_cmpset_64 #define atomic_cmpset_acq_64 kcsan_atomic_cmpset_acq_64 #define atomic_cmpset_rel_64 kcsan_atomic_cmpset_rel_64 #define atomic_fcmpset_64 kcsan_atomic_fcmpset_64 #define atomic_fcmpset_acq_64 kcsan_atomic_fcmpset_acq_64 #define atomic_fcmpset_rel_64 kcsan_atomic_fcmpset_rel_64 #define atomic_fetchadd_64 kcsan_atomic_fetchadd_64 #define atomic_load_64 kcsan_atomic_load_64 #define atomic_load_acq_64 kcsan_atomic_load_acq_64 #define atomic_readandclear_64 kcsan_atomic_readandclear_64 #define atomic_set_64 kcsan_atomic_set_64 #define atomic_set_acq_64 kcsan_atomic_set_acq_64 #define atomic_set_rel_64 kcsan_atomic_set_rel_64 #define atomic_subtract_64 kcsan_atomic_subtract_64 #define atomic_subtract_acq_64 kcsan_atomic_subtract_acq_64 #define atomic_subtract_rel_64 kcsan_atomic_subtract_rel_64 #define atomic_store_64 kcsan_atomic_store_64 #define atomic_store_rel_64 kcsan_atomic_store_rel_64 #define atomic_swap_64 kcsan_atomic_swap_64 #define atomic_testandclear_64 kcsan_atomic_testandclear_64 #define atomic_testandset_64 kcsan_atomic_testandset_64 #define atomic_thread_fence_acq kcsan_atomic_thread_fence_acq #define atomic_thread_fence_acq_rel kcsan_atomic_thread_fence_acq_rel #define atomic_thread_fence_rel kcsan_atomic_thread_fence_rel #define atomic_thread_fence_seq_cst kcsan_atomic_thread_fence_seq_cst #endif /* !KCSAN_RUNTIME */ #endif /* !_SYS__CSAN_ATOMIC_H_ */ Index: projects/clang1000-import/sys/sys/atomic_common.h =================================================================== --- projects/clang1000-import/sys/sys/atomic_common.h (revision 357965) +++ projects/clang1000-import/sys/sys/atomic_common.h (revision 357966) @@ -1,73 +1,73 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2017 The FreeBSD Foundation * All rights reserved. * * This software was developed by Konstantin Belousov * under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _SYS_ATOMIC_COMMON_H_ #define _SYS_ATOMIC_COMMON_H_ #ifndef _MACHINE_ATOMIC_H_ #error do not include this header, use machine/atomic.h #endif #define atomic_load_char(p) (*(volatile u_char *)(p)) #define atomic_load_short(p) (*(volatile u_short *)(p)) #define atomic_load_int(p) (*(volatile u_int *)(p)) #define atomic_load_long(p) (*(volatile u_long *)(p)) -#define atomic_load_ptr(p) (*(volatile uintptr_t*)(p)) +#define atomic_load_ptr(p) (*(volatile __typeof(p))(p)) #define atomic_load_8(p) (*(volatile uint8_t *)(p)) #define atomic_load_16(p) (*(volatile uint16_t *)(p)) #define atomic_load_32(p) (*(volatile uint32_t *)(p)) #ifdef _LP64 #define atomic_load_64(p) (*(volatile uint64_t *)(p)) #endif #define atomic_store_char(p, v) \ (*(volatile u_char *)(p) = (u_char)(v)) #define atomic_store_short(p, v) \ (*(volatile u_short *)(p) = (u_short)(v)) #define atomic_store_int(p, v) \ (*(volatile u_int *)(p) = (u_int)(v)) #define atomic_store_long(p, v) \ (*(volatile u_long *)(p) = (u_long)(v)) #define atomic_store_ptr(p, v) \ (*(volatile uintptr_t *)(p) = (uintptr_t)(v)) #define atomic_store_8(p, v) \ (*(volatile uint8_t *)(p) = (uint8_t)(v)) #define atomic_store_16(p, v) \ (*(volatile uint16_t *)(p) = (uint16_t)(v)) #define atomic_store_32(p, v) \ (*(volatile uint32_t *)(p) = (uint32_t)(v)) #ifdef _LP64 #define atomic_store_64(p, v) \ (*(volatile uint64_t *)(p) = (uint64_t)(v)) #endif #endif Index: projects/clang1000-import/sys/sys/capsicum.h =================================================================== --- projects/clang1000-import/sys/sys/capsicum.h (revision 357965) +++ projects/clang1000-import/sys/sys/capsicum.h (revision 357966) @@ -1,593 +1,614 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2008-2010, 2015 Robert N. M. Watson * Copyright (c) 2012 FreeBSD Foundation * All rights reserved. * * This software was developed at the University of Cambridge Computer * Laboratory with support from a grant from Google, Inc. * * Portions of this software were developed by Pawel Jakub Dawidek under * sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * Definitions for FreeBSD capabilities facility. */ #ifndef _SYS_CAPSICUM_H_ #define _SYS_CAPSICUM_H_ #include #include #include #include #include #ifndef _KERNEL #include #endif #define CAPRIGHT(idx, bit) ((1ULL << (57 + (idx))) | (bit)) /* * Possible rights on capabilities. * * Notes: * Some system calls don't require a capability in order to perform an * operation on an fd. These include: close, dup, dup2. * * sendfile is authorized using CAP_READ on the file and CAP_WRITE on the * socket. * * mmap() and aio*() system calls will need special attention as they may * involve reads or writes depending a great deal on context. */ /* INDEX 0 */ /* * General file I/O. */ /* Allows for openat(O_RDONLY), read(2), readv(2). */ #define CAP_READ CAPRIGHT(0, 0x0000000000000001ULL) /* Allows for openat(O_WRONLY | O_APPEND), write(2), writev(2). */ #define CAP_WRITE CAPRIGHT(0, 0x0000000000000002ULL) /* Allows for lseek(fd, 0, SEEK_CUR). */ #define CAP_SEEK_TELL CAPRIGHT(0, 0x0000000000000004ULL) /* Allows for lseek(2). */ #define CAP_SEEK (CAP_SEEK_TELL | 0x0000000000000008ULL) /* Allows for aio_read(2), pread(2), preadv(2). */ #define CAP_PREAD (CAP_SEEK | CAP_READ) /* * Allows for aio_write(2), openat(O_WRONLY) (without O_APPEND), pwrite(2), * pwritev(2). */ #define CAP_PWRITE (CAP_SEEK | CAP_WRITE) /* Allows for mmap(PROT_NONE). */ #define CAP_MMAP CAPRIGHT(0, 0x0000000000000010ULL) /* Allows for mmap(PROT_READ). */ #define CAP_MMAP_R (CAP_MMAP | CAP_SEEK | CAP_READ) /* Allows for mmap(PROT_WRITE). */ #define CAP_MMAP_W (CAP_MMAP | CAP_SEEK | CAP_WRITE) /* Allows for mmap(PROT_EXEC). */ #define CAP_MMAP_X (CAP_MMAP | CAP_SEEK | 0x0000000000000020ULL) /* Allows for mmap(PROT_READ | PROT_WRITE). */ #define CAP_MMAP_RW (CAP_MMAP_R | CAP_MMAP_W) /* Allows for mmap(PROT_READ | PROT_EXEC). */ #define CAP_MMAP_RX (CAP_MMAP_R | CAP_MMAP_X) /* Allows for mmap(PROT_WRITE | PROT_EXEC). */ #define CAP_MMAP_WX (CAP_MMAP_W | CAP_MMAP_X) /* Allows for mmap(PROT_READ | PROT_WRITE | PROT_EXEC). */ #define CAP_MMAP_RWX (CAP_MMAP_R | CAP_MMAP_W | CAP_MMAP_X) /* Allows for openat(O_CREAT). */ #define CAP_CREATE CAPRIGHT(0, 0x0000000000000040ULL) /* Allows for openat(O_EXEC) and fexecve(2) in turn. */ #define CAP_FEXECVE CAPRIGHT(0, 0x0000000000000080ULL) /* Allows for openat(O_SYNC), openat(O_FSYNC), fsync(2), aio_fsync(2). */ #define CAP_FSYNC CAPRIGHT(0, 0x0000000000000100ULL) /* Allows for openat(O_TRUNC), ftruncate(2). */ #define CAP_FTRUNCATE CAPRIGHT(0, 0x0000000000000200ULL) /* Lookups - used to constrain *at() calls. */ #define CAP_LOOKUP CAPRIGHT(0, 0x0000000000000400ULL) /* VFS methods. */ /* Allows for fchdir(2). */ #define CAP_FCHDIR CAPRIGHT(0, 0x0000000000000800ULL) /* Allows for fchflags(2). */ #define CAP_FCHFLAGS CAPRIGHT(0, 0x0000000000001000ULL) /* Allows for fchflags(2) and chflagsat(2). */ #define CAP_CHFLAGSAT (CAP_FCHFLAGS | CAP_LOOKUP) /* Allows for fchmod(2). */ #define CAP_FCHMOD CAPRIGHT(0, 0x0000000000002000ULL) /* Allows for fchmod(2) and fchmodat(2). */ #define CAP_FCHMODAT (CAP_FCHMOD | CAP_LOOKUP) /* Allows for fchown(2). */ #define CAP_FCHOWN CAPRIGHT(0, 0x0000000000004000ULL) /* Allows for fchown(2) and fchownat(2). */ #define CAP_FCHOWNAT (CAP_FCHOWN | CAP_LOOKUP) /* Allows for fcntl(2). */ #define CAP_FCNTL CAPRIGHT(0, 0x0000000000008000ULL) /* * Allows for flock(2), openat(O_SHLOCK), openat(O_EXLOCK), * fcntl(F_SETLK_REMOTE), fcntl(F_SETLKW), fcntl(F_SETLK), fcntl(F_GETLK). */ #define CAP_FLOCK CAPRIGHT(0, 0x0000000000010000ULL) /* Allows for fpathconf(2). */ #define CAP_FPATHCONF CAPRIGHT(0, 0x0000000000020000ULL) /* Allows for UFS background-fsck operations. */ #define CAP_FSCK CAPRIGHT(0, 0x0000000000040000ULL) /* Allows for fstat(2). */ #define CAP_FSTAT CAPRIGHT(0, 0x0000000000080000ULL) /* Allows for fstat(2), fstatat(2) and faccessat(2). */ #define CAP_FSTATAT (CAP_FSTAT | CAP_LOOKUP) /* Allows for fstatfs(2). */ #define CAP_FSTATFS CAPRIGHT(0, 0x0000000000100000ULL) /* Allows for futimens(2) and futimes(2). */ #define CAP_FUTIMES CAPRIGHT(0, 0x0000000000200000ULL) /* Allows for futimens(2), futimes(2), futimesat(2) and utimensat(2). */ #define CAP_FUTIMESAT (CAP_FUTIMES | CAP_LOOKUP) /* Allows for linkat(2) (target directory descriptor). */ #define CAP_LINKAT_TARGET (CAP_LOOKUP | 0x0000000000400000ULL) /* Allows for mkdirat(2). */ #define CAP_MKDIRAT (CAP_LOOKUP | 0x0000000000800000ULL) /* Allows for mkfifoat(2). */ #define CAP_MKFIFOAT (CAP_LOOKUP | 0x0000000001000000ULL) /* Allows for mknodat(2). */ #define CAP_MKNODAT (CAP_LOOKUP | 0x0000000002000000ULL) /* Allows for renameat(2) (source directory descriptor). */ #define CAP_RENAMEAT_SOURCE (CAP_LOOKUP | 0x0000000004000000ULL) /* Allows for symlinkat(2). */ #define CAP_SYMLINKAT (CAP_LOOKUP | 0x0000000008000000ULL) /* * Allows for unlinkat(2) and renameat(2) if destination object exists and * will be removed. */ #define CAP_UNLINKAT (CAP_LOOKUP | 0x0000000010000000ULL) /* Socket operations. */ /* Allows for accept(2) and accept4(2). */ #define CAP_ACCEPT CAPRIGHT(0, 0x0000000020000000ULL) /* Allows for bind(2). */ #define CAP_BIND CAPRIGHT(0, 0x0000000040000000ULL) /* Allows for connect(2). */ #define CAP_CONNECT CAPRIGHT(0, 0x0000000080000000ULL) /* Allows for getpeername(2). */ #define CAP_GETPEERNAME CAPRIGHT(0, 0x0000000100000000ULL) /* Allows for getsockname(2). */ #define CAP_GETSOCKNAME CAPRIGHT(0, 0x0000000200000000ULL) /* Allows for getsockopt(2). */ #define CAP_GETSOCKOPT CAPRIGHT(0, 0x0000000400000000ULL) /* Allows for listen(2). */ #define CAP_LISTEN CAPRIGHT(0, 0x0000000800000000ULL) /* Allows for sctp_peeloff(2). */ #define CAP_PEELOFF CAPRIGHT(0, 0x0000001000000000ULL) #define CAP_RECV CAP_READ #define CAP_SEND CAP_WRITE /* Allows for setsockopt(2). */ #define CAP_SETSOCKOPT CAPRIGHT(0, 0x0000002000000000ULL) /* Allows for shutdown(2). */ #define CAP_SHUTDOWN CAPRIGHT(0, 0x0000004000000000ULL) /* Allows for bindat(2) on a directory descriptor. */ #define CAP_BINDAT (CAP_LOOKUP | 0x0000008000000000ULL) /* Allows for connectat(2) on a directory descriptor. */ #define CAP_CONNECTAT (CAP_LOOKUP | 0x0000010000000000ULL) /* Allows for linkat(2) (source directory descriptor). */ #define CAP_LINKAT_SOURCE (CAP_LOOKUP | 0x0000020000000000ULL) /* Allows for renameat(2) (target directory descriptor). */ #define CAP_RENAMEAT_TARGET (CAP_LOOKUP | 0x0000040000000000ULL) #define CAP_SOCK_CLIENT \ (CAP_CONNECT | CAP_GETPEERNAME | CAP_GETSOCKNAME | CAP_GETSOCKOPT | \ CAP_PEELOFF | CAP_RECV | CAP_SEND | CAP_SETSOCKOPT | CAP_SHUTDOWN) #define CAP_SOCK_SERVER \ (CAP_ACCEPT | CAP_BIND | CAP_GETPEERNAME | CAP_GETSOCKNAME | \ CAP_GETSOCKOPT | CAP_LISTEN | CAP_PEELOFF | CAP_RECV | CAP_SEND | \ CAP_SETSOCKOPT | CAP_SHUTDOWN) /* All used bits for index 0. */ #define CAP_ALL0 CAPRIGHT(0, 0x000007FFFFFFFFFFULL) /* Available bits for index 0. */ #define CAP_UNUSED0_44 CAPRIGHT(0, 0x0000080000000000ULL) /* ... */ #define CAP_UNUSED0_57 CAPRIGHT(0, 0x0100000000000000ULL) /* INDEX 1 */ /* Mandatory Access Control. */ /* Allows for mac_get_fd(3). */ #define CAP_MAC_GET CAPRIGHT(1, 0x0000000000000001ULL) /* Allows for mac_set_fd(3). */ #define CAP_MAC_SET CAPRIGHT(1, 0x0000000000000002ULL) /* Methods on semaphores. */ #define CAP_SEM_GETVALUE CAPRIGHT(1, 0x0000000000000004ULL) #define CAP_SEM_POST CAPRIGHT(1, 0x0000000000000008ULL) #define CAP_SEM_WAIT CAPRIGHT(1, 0x0000000000000010ULL) /* Allows select(2) and poll(2) on descriptor. */ #define CAP_EVENT CAPRIGHT(1, 0x0000000000000020ULL) /* Allows for kevent(2) on kqueue descriptor with eventlist != NULL. */ #define CAP_KQUEUE_EVENT CAPRIGHT(1, 0x0000000000000040ULL) /* Strange and powerful rights that should not be given lightly. */ /* Allows for ioctl(2). */ #define CAP_IOCTL CAPRIGHT(1, 0x0000000000000080ULL) #define CAP_TTYHOOK CAPRIGHT(1, 0x0000000000000100ULL) /* Process management via process descriptors. */ /* Allows for pdgetpid(2). */ #define CAP_PDGETPID CAPRIGHT(1, 0x0000000000000200ULL) /* * Allows for pdwait4(2). * * XXX: this constant was imported unused, but is targeted to be implemented * in the future (bug 235871). */ #define CAP_PDWAIT CAPRIGHT(1, 0x0000000000000400ULL) /* Allows for pdkill(2). */ #define CAP_PDKILL CAPRIGHT(1, 0x0000000000000800ULL) /* Extended attributes. */ /* Allows for extattr_delete_fd(2). */ #define CAP_EXTATTR_DELETE CAPRIGHT(1, 0x0000000000001000ULL) /* Allows for extattr_get_fd(2). */ #define CAP_EXTATTR_GET CAPRIGHT(1, 0x0000000000002000ULL) /* Allows for extattr_list_fd(2). */ #define CAP_EXTATTR_LIST CAPRIGHT(1, 0x0000000000004000ULL) /* Allows for extattr_set_fd(2). */ #define CAP_EXTATTR_SET CAPRIGHT(1, 0x0000000000008000ULL) /* Access Control Lists. */ /* Allows for acl_valid_fd_np(3). */ #define CAP_ACL_CHECK CAPRIGHT(1, 0x0000000000010000ULL) /* Allows for acl_delete_fd_np(3). */ #define CAP_ACL_DELETE CAPRIGHT(1, 0x0000000000020000ULL) /* Allows for acl_get_fd(3) and acl_get_fd_np(3). */ #define CAP_ACL_GET CAPRIGHT(1, 0x0000000000040000ULL) /* Allows for acl_set_fd(3) and acl_set_fd_np(3). */ #define CAP_ACL_SET CAPRIGHT(1, 0x0000000000080000ULL) /* Allows for kevent(2) on kqueue descriptor with changelist != NULL. */ #define CAP_KQUEUE_CHANGE CAPRIGHT(1, 0x0000000000100000ULL) #define CAP_KQUEUE (CAP_KQUEUE_EVENT | CAP_KQUEUE_CHANGE) /* All used bits for index 1. */ #define CAP_ALL1 CAPRIGHT(1, 0x00000000001FFFFFULL) /* Available bits for index 1. */ #define CAP_UNUSED1_22 CAPRIGHT(1, 0x0000000000200000ULL) /* ... */ #define CAP_UNUSED1_57 CAPRIGHT(1, 0x0100000000000000ULL) /* Backward compatibility. */ #define CAP_POLL_EVENT CAP_EVENT #define CAP_ALL(rights) do { \ (rights)->cr_rights[0] = \ ((uint64_t)CAP_RIGHTS_VERSION << 62) | CAP_ALL0; \ (rights)->cr_rights[1] = CAP_ALL1; \ } while (0) #define CAP_NONE(rights) do { \ (rights)->cr_rights[0] = \ ((uint64_t)CAP_RIGHTS_VERSION << 62) | CAPRIGHT(0, 0ULL); \ (rights)->cr_rights[1] = CAPRIGHT(1, 0ULL); \ } while (0) #define CAPRVER(right) ((int)((right) >> 62)) #define CAPVER(rights) CAPRVER((rights)->cr_rights[0]) #define CAPARSIZE(rights) (CAPVER(rights) + 2) #define CAPIDXBIT(right) ((int)(((right) >> 57) & 0x1F)) /* * Allowed fcntl(2) commands. */ #define CAP_FCNTL_GETFL (1 << F_GETFL) #define CAP_FCNTL_SETFL (1 << F_SETFL) #define CAP_FCNTL_GETOWN (1 << F_GETOWN) #define CAP_FCNTL_SETOWN (1 << F_SETOWN) #define CAP_FCNTL_ALL (CAP_FCNTL_GETFL | CAP_FCNTL_SETFL | \ CAP_FCNTL_GETOWN | CAP_FCNTL_SETOWN) #define CAP_IOCTLS_ALL SSIZE_MAX __BEGIN_DECLS #define cap_rights_init(...) \ __cap_rights_init(CAP_RIGHTS_VERSION, __VA_ARGS__, 0ULL) cap_rights_t *__cap_rights_init(int version, cap_rights_t *rights, ...); #define cap_rights_set(...) \ __cap_rights_set(__VA_ARGS__, 0ULL) cap_rights_t *__cap_rights_set(cap_rights_t *rights, ...); #define cap_rights_clear(...) \ __cap_rights_clear(__VA_ARGS__, 0ULL) cap_rights_t *__cap_rights_clear(cap_rights_t *rights, ...); #define cap_rights_is_set(...) \ __cap_rights_is_set(__VA_ARGS__, 0ULL) bool __cap_rights_is_set(const cap_rights_t *rights, ...); bool cap_rights_is_valid(const cap_rights_t *rights); cap_rights_t *cap_rights_merge(cap_rights_t *dst, const cap_rights_t *src); cap_rights_t *cap_rights_remove(cap_rights_t *dst, const cap_rights_t *src); void __cap_rights_sysinit(void *arg); #ifdef _KERNEL /* * We only support one size to reduce branching. */ _Static_assert(CAP_RIGHTS_VERSION == CAP_RIGHTS_VERSION_00, "unsupported version of capsicum rights"); +#define cap_rights_init_zero(r) ({ \ + cap_rights_t *_r = (r); \ + CAP_NONE(_r); \ + _r; \ +}) + +#define cap_rights_init_one(r, right) ({ \ + CTASSERT(CAPRVER(right) == CAP_RIGHTS_VERSION); \ + cap_rights_t *_r = (r); \ + CAP_NONE(_r); \ + _r->cr_rights[CAPIDXBIT(right) - 1] |= right; \ + _r; \ +}) + +#define cap_rights_set_one(r, right) ({ \ + CTASSERT(CAPRVER(right) == CAP_RIGHTS_VERSION); \ + cap_rights_t *_r = (r); \ + _r->cr_rights[CAPIDXBIT(right) - 1] |= right; \ + _r; \ +}) + /* * Allow checking caps which are possibly getting modified at the same time. * The caller is expected to determine whether the result is legitimate via * other means, see fget_unlocked for an example. */ static inline bool cap_rights_contains_transient(const cap_rights_t *big, const cap_rights_t *little) { if (__predict_true( (big->cr_rights[0] & little->cr_rights[0]) == little->cr_rights[0] && (big->cr_rights[1] & little->cr_rights[1]) == little->cr_rights[1])) return (true); return (false); } #define cap_rights_contains cap_rights_contains_transient int cap_check_failed_notcapable(const cap_rights_t *havep, const cap_rights_t *needp); static inline int cap_check_inline(const cap_rights_t *havep, const cap_rights_t *needp) { if (__predict_false(!cap_rights_contains(havep, needp))) return (cap_check_failed_notcapable(havep, needp)); return (0); } static inline int cap_check_inline_transient(const cap_rights_t *havep, const cap_rights_t *needp) { if (__predict_false(!cap_rights_contains(havep, needp))) return (1); return (0); } #else bool cap_rights_contains(const cap_rights_t *big, const cap_rights_t *little); #endif __END_DECLS struct cap_rights_init_args { cap_rights_t *cria_rights; uint64_t cria_value1; uint64_t cria_value2; uint64_t cria_value3; uint64_t cria_value4; uint64_t cria_value5; }; #define CAP_RIGHTS_SYSINIT0(name, rights) \ static struct cap_rights_init_args name##_args = { \ &(rights) \ }; \ SYSINIT(name##_cap_rights_sysinit, SI_SUB_COPYRIGHT+1, SI_ORDER_ANY, \ __cap_rights_sysinit, &name##_args); #define CAP_RIGHTS_SYSINIT1(name, rights, value1) \ static struct cap_rights_init_args name##_args = { \ &(rights), \ (value1) \ }; \ SYSINIT(name##_cap_rights_sysinit, SI_SUB_COPYRIGHT+1, SI_ORDER_ANY, \ __cap_rights_sysinit, &name##_args); #define CAP_RIGHTS_SYSINIT2(name, rights, value1, value2) \ static struct cap_rights_init_args name##_args = { \ &(rights), \ (value1), \ (value2) \ }; \ SYSINIT(name##_cap_rights_sysinit, SI_SUB_COPYRIGHT, SI_ORDER_ANY, \ __cap_rights_sysinit, &name##_args); #define CAP_RIGHTS_SYSINIT3(name, rights, value1, value2, value3) \ static struct cap_rights_init_args name##_args = { \ &(rights), \ (value1), \ (value2), \ (value3) \ }; \ SYSINIT(name##_cap_rights_sysinit, SI_SUB_COPYRIGHT, SI_ORDER_ANY, \ __cap_rights_sysinit, &name##_args); #define CAP_RIGHTS_SYSINIT4(name, rights, value1, value2, value3, value4) \ static struct cap_rights_init_args name##_args = { \ &(rights), \ (value1), \ (value2), \ (value3), \ (value4) \ }; \ SYSINIT(name##_cap_rights_sysinit, SI_SUB_COPYRIGHT, SI_ORDER_ANY, \ __cap_rights_sysinit, &name##_args); #define CAP_RIGHTS_DEFINE1(name, value) \ __read_mostly cap_rights_t name; \ CAP_RIGHTS_SYSINIT1(name, name, value); #ifdef _KERNEL #include extern cap_rights_t cap_accept_rights; extern cap_rights_t cap_bind_rights; extern cap_rights_t cap_connect_rights; extern cap_rights_t cap_event_rights; extern cap_rights_t cap_fchdir_rights; extern cap_rights_t cap_fchflags_rights; extern cap_rights_t cap_fchmod_rights; extern cap_rights_t cap_fchown_rights; extern cap_rights_t cap_fcntl_rights; extern cap_rights_t cap_fexecve_rights; extern cap_rights_t cap_flock_rights; extern cap_rights_t cap_fpathconf_rights; extern cap_rights_t cap_fstat_rights; extern cap_rights_t cap_fstatfs_rights; extern cap_rights_t cap_fsync_rights; extern cap_rights_t cap_ftruncate_rights; extern cap_rights_t cap_futimes_rights; extern cap_rights_t cap_getpeername_rights; extern cap_rights_t cap_getsockopt_rights; extern cap_rights_t cap_getsockname_rights; extern cap_rights_t cap_ioctl_rights; extern cap_rights_t cap_linkat_source_rights; extern cap_rights_t cap_linkat_target_rights; extern cap_rights_t cap_listen_rights; extern cap_rights_t cap_mkdirat_rights; extern cap_rights_t cap_mkfifoat_rights; extern cap_rights_t cap_mknodat_rights; extern cap_rights_t cap_mmap_rights; extern cap_rights_t cap_no_rights; extern cap_rights_t cap_pdgetpid_rights; extern cap_rights_t cap_pdkill_rights; extern cap_rights_t cap_pread_rights; extern cap_rights_t cap_pwrite_rights; extern cap_rights_t cap_read_rights; extern cap_rights_t cap_recv_rights; extern cap_rights_t cap_renameat_source_rights; extern cap_rights_t cap_renameat_target_rights; extern cap_rights_t cap_seek_rights; extern cap_rights_t cap_send_rights; extern cap_rights_t cap_send_connect_rights; extern cap_rights_t cap_setsockopt_rights; extern cap_rights_t cap_shutdown_rights; extern cap_rights_t cap_symlinkat_rights; extern cap_rights_t cap_unlinkat_rights; extern cap_rights_t cap_write_rights; #define IN_CAPABILITY_MODE(td) (((td)->td_ucred->cr_flags & CRED_FLAG_CAPMODE) != 0) struct filedesc; struct filedescent; /* * Test whether a capability grants the requested rights. */ int cap_check(const cap_rights_t *havep, const cap_rights_t *needp); /* * Convert capability rights into VM access flags. */ vm_prot_t cap_rights_to_vmprot(const cap_rights_t *havep); /* * For the purposes of procstat(1) and similar tools, allow kern_descrip.c to * extract the rights from a capability. * * Dereferencing fdep requires filedesc.h, but including it would cause * significant pollution. Instead add a macro for consumers which want it, * most notably kern_descrip.c. */ #define cap_rights_fde_inline(fdep) (&(fdep)->fde_rights) const cap_rights_t *cap_rights_fde(const struct filedescent *fde); const cap_rights_t *cap_rights(struct filedesc *fdp, int fd); int cap_ioctl_check(struct filedesc *fdp, int fd, u_long cmd); int cap_fcntl_check_fde(struct filedescent *fde, int cmd); int cap_fcntl_check(struct filedesc *fdp, int fd, int cmd); extern bool trap_enotcap; #else /* !_KERNEL */ __BEGIN_DECLS /* * cap_enter(): Cause the process to enter capability mode, which will * prevent it from directly accessing global namespaces. System calls will * be limited to process-local, process-inherited, or file descriptor * operations. If already in capability mode, a no-op. */ int cap_enter(void); /* * Are we sandboxed (in capability mode)? * This is a libc wrapper around the cap_getmode(2) system call. */ bool cap_sandboxed(void); /* * cap_getmode(): Are we in capability mode? */ int cap_getmode(u_int *modep); /* * Limits capability rights for the given descriptor (CAP_*). */ int cap_rights_limit(int fd, const cap_rights_t *rights); /* * Returns capability rights for the given descriptor. */ #define cap_rights_get(fd, rights) \ __cap_rights_get(CAP_RIGHTS_VERSION, (fd), (rights)) int __cap_rights_get(int version, int fd, cap_rights_t *rights); /* * Limits allowed ioctls for the given descriptor. */ int cap_ioctls_limit(int fd, const cap_ioctl_t *cmds, size_t ncmds); /* * Returns array of allowed ioctls for the given descriptor. * If all ioctls are allowed, the cmds array is not populated and * the function returns CAP_IOCTLS_ALL. */ ssize_t cap_ioctls_get(int fd, cap_ioctl_t *cmds, size_t maxcmds); /* * Limits allowed fcntls for the given descriptor (CAP_FCNTL_*). */ int cap_fcntls_limit(int fd, uint32_t fcntlrights); /* * Returns bitmask of allowed fcntls for the given descriptor. */ int cap_fcntls_get(int fd, uint32_t *fcntlrightsp); __END_DECLS #endif /* !_KERNEL */ #endif /* !_SYS_CAPSICUM_H_ */ Index: projects/clang1000-import/sys/vm/vm_mmap.c =================================================================== --- projects/clang1000-import/sys/vm/vm_mmap.c (revision 357965) +++ projects/clang1000-import/sys/vm/vm_mmap.c (revision 357966) @@ -1,1661 +1,1660 @@ /*- * SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 1988 University of Utah. * Copyright (c) 1991, 1993 * The Regents of the University of California. All rights reserved. * * This code is derived from software contributed to Berkeley by * the Systems Programming Group of the University of Utah Computer * Science Department. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * from: Utah $Hdr: vm_mmap.c 1.6 91/10/21$ * * @(#)vm_mmap.c 8.4 (Berkeley) 1/12/94 */ /* * Mapped file (mmap) interface to VM */ #include __FBSDID("$FreeBSD$"); #include "opt_hwpmc_hooks.h" #include "opt_vm.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #if defined(__amd64__) || defined(__i386__) /* for i386_read_exec */ #include #endif #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef HWPMC_HOOKS #include #endif int old_mlock = 0; SYSCTL_INT(_vm, OID_AUTO, old_mlock, CTLFLAG_RWTUN, &old_mlock, 0, "Do not apply RLIMIT_MEMLOCK on mlockall"); static int mincore_mapped = 1; SYSCTL_INT(_vm, OID_AUTO, mincore_mapped, CTLFLAG_RWTUN, &mincore_mapped, 0, "mincore reports mappings, not residency"); static int imply_prot_max = 0; SYSCTL_INT(_vm, OID_AUTO, imply_prot_max, CTLFLAG_RWTUN, &imply_prot_max, 0, "Imply maximum page permissions in mmap() when none are specified"); #ifdef MAP_32BIT #define MAP_32BIT_MAX_ADDR ((vm_offset_t)1 << 31) #endif #ifndef _SYS_SYSPROTO_H_ struct sbrk_args { int incr; }; #endif int sys_sbrk(struct thread *td, struct sbrk_args *uap) { /* Not yet implemented */ return (EOPNOTSUPP); } #ifndef _SYS_SYSPROTO_H_ struct sstk_args { int incr; }; #endif int sys_sstk(struct thread *td, struct sstk_args *uap) { /* Not yet implemented */ return (EOPNOTSUPP); } #if defined(COMPAT_43) int ogetpagesize(struct thread *td, struct ogetpagesize_args *uap) { td->td_retval[0] = PAGE_SIZE; return (0); } #endif /* COMPAT_43 */ /* * Memory Map (mmap) system call. Note that the file offset * and address are allowed to be NOT page aligned, though if * the MAP_FIXED flag it set, both must have the same remainder * modulo the PAGE_SIZE (POSIX 1003.1b). If the address is not * page-aligned, the actual mapping starts at trunc_page(addr) * and the return value is adjusted up by the page offset. * * Generally speaking, only character devices which are themselves * memory-based, such as a video framebuffer, can be mmap'd. Otherwise * there would be no cache coherency between a descriptor and a VM mapping * both to the same character device. */ #ifndef _SYS_SYSPROTO_H_ struct mmap_args { void *addr; size_t len; int prot; int flags; int fd; long pad; off_t pos; }; #endif int sys_mmap(struct thread *td, struct mmap_args *uap) { return (kern_mmap(td, (uintptr_t)uap->addr, uap->len, uap->prot, uap->flags, uap->fd, uap->pos)); } int kern_mmap_maxprot(struct proc *p, int prot) { if ((p->p_flag2 & P2_PROTMAX_DISABLE) != 0 || (p->p_fctl0 & NT_FREEBSD_FCTL_PROTMAX_DISABLE) != 0) return (_PROT_ALL); if (((p->p_flag2 & P2_PROTMAX_ENABLE) != 0 || imply_prot_max) && prot != PROT_NONE) return (prot); return (_PROT_ALL); } int kern_mmap(struct thread *td, uintptr_t addr0, size_t len, int prot, int flags, int fd, off_t pos) { return (kern_mmap_fpcheck(td, addr0, len, prot, flags, fd, pos, NULL)); } /* * When mmap'ing a file, check_fp_fn may be used for the caller to do any * last-minute validation based on the referenced file in a non-racy way. */ int kern_mmap_fpcheck(struct thread *td, uintptr_t addr0, size_t len, int prot, int flags, int fd, off_t pos, mmap_check_fp_fn check_fp_fn) { struct vmspace *vms; struct file *fp; struct proc *p; vm_offset_t addr; vm_size_t pageoff, size; vm_prot_t cap_maxprot; int align, error, max_prot; cap_rights_t rights; if ((prot & ~(_PROT_ALL | PROT_MAX(_PROT_ALL))) != 0) return (EINVAL); max_prot = PROT_MAX_EXTRACT(prot); prot = PROT_EXTRACT(prot); if (max_prot != 0 && (max_prot & prot) != prot) return (EINVAL); p = td->td_proc; /* * Always honor PROT_MAX if set. If not, default to all * permissions unless we're implying maximum permissions. */ if (max_prot == 0) max_prot = kern_mmap_maxprot(p, prot); vms = p->p_vmspace; fp = NULL; AUDIT_ARG_FD(fd); addr = addr0; /* * Ignore old flags that used to be defined but did not do anything. */ flags &= ~(MAP_RESERVED0020 | MAP_RESERVED0040); /* * Enforce the constraints. * Mapping of length 0 is only allowed for old binaries. * Anonymous mapping shall specify -1 as filedescriptor and * zero position for new code. Be nice to ancient a.out * binaries and correct pos for anonymous mapping, since old * ld.so sometimes issues anonymous map requests with non-zero * pos. */ if (!SV_CURPROC_FLAG(SV_AOUT)) { if ((len == 0 && p->p_osrel >= P_OSREL_MAP_ANON) || ((flags & MAP_ANON) != 0 && (fd != -1 || pos != 0))) return (EINVAL); } else { if ((flags & MAP_ANON) != 0) pos = 0; } if (flags & MAP_STACK) { if ((fd != -1) || ((prot & (PROT_READ | PROT_WRITE)) != (PROT_READ | PROT_WRITE))) return (EINVAL); flags |= MAP_ANON; pos = 0; } if ((flags & ~(MAP_SHARED | MAP_PRIVATE | MAP_FIXED | MAP_HASSEMAPHORE | MAP_STACK | MAP_NOSYNC | MAP_ANON | MAP_EXCL | MAP_NOCORE | MAP_PREFAULT_READ | MAP_GUARD | #ifdef MAP_32BIT MAP_32BIT | #endif MAP_ALIGNMENT_MASK)) != 0) return (EINVAL); if ((flags & (MAP_EXCL | MAP_FIXED)) == MAP_EXCL) return (EINVAL); if ((flags & (MAP_SHARED | MAP_PRIVATE)) == (MAP_SHARED | MAP_PRIVATE)) return (EINVAL); if (prot != PROT_NONE && (prot & ~(PROT_READ | PROT_WRITE | PROT_EXEC)) != 0) return (EINVAL); if ((flags & MAP_GUARD) != 0 && (prot != PROT_NONE || fd != -1 || pos != 0 || (flags & ~(MAP_FIXED | MAP_GUARD | MAP_EXCL | #ifdef MAP_32BIT MAP_32BIT | #endif MAP_ALIGNMENT_MASK)) != 0)) return (EINVAL); /* * Align the file position to a page boundary, * and save its page offset component. */ pageoff = (pos & PAGE_MASK); pos -= pageoff; /* Compute size from len by rounding (on both ends). */ size = len + pageoff; /* low end... */ size = round_page(size); /* hi end */ /* Check for rounding up to zero. */ if (len > size) return (ENOMEM); /* Ensure alignment is at least a page and fits in a pointer. */ align = flags & MAP_ALIGNMENT_MASK; if (align != 0 && align != MAP_ALIGNED_SUPER && (align >> MAP_ALIGNMENT_SHIFT >= sizeof(void *) * NBBY || align >> MAP_ALIGNMENT_SHIFT < PAGE_SHIFT)) return (EINVAL); /* * Check for illegal addresses. Watch out for address wrap... Note * that VM_*_ADDRESS are not constants due to casts (argh). */ if (flags & MAP_FIXED) { /* * The specified address must have the same remainder * as the file offset taken modulo PAGE_SIZE, so it * should be aligned after adjustment by pageoff. */ addr -= pageoff; if (addr & PAGE_MASK) return (EINVAL); /* Address range must be all in user VM space. */ if (addr < vm_map_min(&vms->vm_map) || addr + size > vm_map_max(&vms->vm_map)) return (EINVAL); if (addr + size < addr) return (EINVAL); #ifdef MAP_32BIT if (flags & MAP_32BIT && addr + size > MAP_32BIT_MAX_ADDR) return (EINVAL); } else if (flags & MAP_32BIT) { /* * For MAP_32BIT, override the hint if it is too high and * do not bother moving the mapping past the heap (since * the heap is usually above 2GB). */ if (addr + size > MAP_32BIT_MAX_ADDR) addr = 0; #endif } else { /* * XXX for non-fixed mappings where no hint is provided or * the hint would fall in the potential heap space, * place it after the end of the largest possible heap. * * There should really be a pmap call to determine a reasonable * location. */ if (addr == 0 || (addr >= round_page((vm_offset_t)vms->vm_taddr) && addr < round_page((vm_offset_t)vms->vm_daddr + lim_max(td, RLIMIT_DATA)))) addr = round_page((vm_offset_t)vms->vm_daddr + lim_max(td, RLIMIT_DATA)); } if (len == 0) { /* * Return success without mapping anything for old * binaries that request a page-aligned mapping of * length 0. For modern binaries, this function * returns an error earlier. */ error = 0; } else if ((flags & MAP_GUARD) != 0) { error = vm_mmap_object(&vms->vm_map, &addr, size, VM_PROT_NONE, VM_PROT_NONE, flags, NULL, pos, FALSE, td); } else if ((flags & MAP_ANON) != 0) { /* * Mapping blank space is trivial. * * This relies on VM_PROT_* matching PROT_*. */ error = vm_mmap_object(&vms->vm_map, &addr, size, prot, max_prot, flags, NULL, pos, FALSE, td); } else { /* * Mapping file, get fp for validation and don't let the * descriptor disappear on us if we block. Check capability * rights, but also return the maximum rights to be combined * with maxprot later. */ - cap_rights_init(&rights, CAP_MMAP); + cap_rights_init_one(&rights, CAP_MMAP); if (prot & PROT_READ) - cap_rights_set(&rights, CAP_MMAP_R); + cap_rights_set_one(&rights, CAP_MMAP_R); if ((flags & MAP_SHARED) != 0) { if (prot & PROT_WRITE) - cap_rights_set(&rights, CAP_MMAP_W); + cap_rights_set_one(&rights, CAP_MMAP_W); } if (prot & PROT_EXEC) - cap_rights_set(&rights, CAP_MMAP_X); + cap_rights_set_one(&rights, CAP_MMAP_X); error = fget_mmap(td, fd, &rights, &cap_maxprot, &fp); if (error != 0) goto done; if ((flags & (MAP_SHARED | MAP_PRIVATE)) == 0 && p->p_osrel >= P_OSREL_MAP_FSTRICT) { error = EINVAL; goto done; } if (check_fp_fn != NULL) { error = check_fp_fn(fp, prot, max_prot & cap_maxprot, flags); if (error != 0) goto done; } /* This relies on VM_PROT_* matching PROT_*. */ error = fo_mmap(fp, &vms->vm_map, &addr, size, prot, max_prot & cap_maxprot, flags, pos, td); } if (error == 0) td->td_retval[0] = (register_t) (addr + pageoff); done: if (fp) fdrop(fp, td); return (error); } #if defined(COMPAT_FREEBSD6) int freebsd6_mmap(struct thread *td, struct freebsd6_mmap_args *uap) { return (kern_mmap(td, (uintptr_t)uap->addr, uap->len, uap->prot, uap->flags, uap->fd, uap->pos)); } #endif #ifdef COMPAT_43 #ifndef _SYS_SYSPROTO_H_ struct ommap_args { caddr_t addr; int len; int prot; int flags; int fd; long pos; }; #endif int ommap(struct thread *td, struct ommap_args *uap) { static const char cvtbsdprot[8] = { 0, PROT_EXEC, PROT_WRITE, PROT_EXEC | PROT_WRITE, PROT_READ, PROT_EXEC | PROT_READ, PROT_WRITE | PROT_READ, PROT_EXEC | PROT_WRITE | PROT_READ, }; int flags, prot; #define OMAP_ANON 0x0002 #define OMAP_COPY 0x0020 #define OMAP_SHARED 0x0010 #define OMAP_FIXED 0x0100 prot = cvtbsdprot[uap->prot & 0x7]; #if (defined(COMPAT_FREEBSD32) && defined(__amd64__)) || defined(__i386__) if (i386_read_exec && SV_PROC_FLAG(td->td_proc, SV_ILP32) && prot != 0) prot |= PROT_EXEC; #endif flags = 0; if (uap->flags & OMAP_ANON) flags |= MAP_ANON; if (uap->flags & OMAP_COPY) flags |= MAP_COPY; if (uap->flags & OMAP_SHARED) flags |= MAP_SHARED; else flags |= MAP_PRIVATE; if (uap->flags & OMAP_FIXED) flags |= MAP_FIXED; return (kern_mmap(td, (uintptr_t)uap->addr, uap->len, prot, flags, uap->fd, uap->pos)); } #endif /* COMPAT_43 */ #ifndef _SYS_SYSPROTO_H_ struct msync_args { void *addr; size_t len; int flags; }; #endif int sys_msync(struct thread *td, struct msync_args *uap) { return (kern_msync(td, (uintptr_t)uap->addr, uap->len, uap->flags)); } int kern_msync(struct thread *td, uintptr_t addr0, size_t size, int flags) { vm_offset_t addr; vm_size_t pageoff; vm_map_t map; int rv; addr = addr0; pageoff = (addr & PAGE_MASK); addr -= pageoff; size += pageoff; size = (vm_size_t) round_page(size); if (addr + size < addr) return (EINVAL); if ((flags & (MS_ASYNC|MS_INVALIDATE)) == (MS_ASYNC|MS_INVALIDATE)) return (EINVAL); map = &td->td_proc->p_vmspace->vm_map; /* * Clean the pages and interpret the return value. */ rv = vm_map_sync(map, addr, addr + size, (flags & MS_ASYNC) == 0, (flags & MS_INVALIDATE) != 0); switch (rv) { case KERN_SUCCESS: return (0); case KERN_INVALID_ADDRESS: return (ENOMEM); case KERN_INVALID_ARGUMENT: return (EBUSY); case KERN_FAILURE: return (EIO); default: return (EINVAL); } } #ifndef _SYS_SYSPROTO_H_ struct munmap_args { void *addr; size_t len; }; #endif int sys_munmap(struct thread *td, struct munmap_args *uap) { return (kern_munmap(td, (uintptr_t)uap->addr, uap->len)); } int kern_munmap(struct thread *td, uintptr_t addr0, size_t size) { #ifdef HWPMC_HOOKS struct pmckern_map_out pkm; vm_map_entry_t entry; bool pmc_handled; #endif vm_offset_t addr; vm_size_t pageoff; vm_map_t map; if (size == 0) return (EINVAL); addr = addr0; pageoff = (addr & PAGE_MASK); addr -= pageoff; size += pageoff; size = (vm_size_t) round_page(size); if (addr + size < addr) return (EINVAL); /* * Check for illegal addresses. Watch out for address wrap... */ map = &td->td_proc->p_vmspace->vm_map; if (addr < vm_map_min(map) || addr + size > vm_map_max(map)) return (EINVAL); vm_map_lock(map); #ifdef HWPMC_HOOKS pmc_handled = false; if (PMC_HOOK_INSTALLED(PMC_FN_MUNMAP)) { pmc_handled = true; /* * Inform hwpmc if the address range being unmapped contains * an executable region. */ pkm.pm_address = (uintptr_t) NULL; if (vm_map_lookup_entry(map, addr, &entry)) { for (; entry->start < addr + size; entry = vm_map_entry_succ(entry)) { if (vm_map_check_protection(map, entry->start, entry->end, VM_PROT_EXECUTE) == TRUE) { pkm.pm_address = (uintptr_t) addr; pkm.pm_size = (size_t) size; break; } } } } #endif vm_map_delete(map, addr, addr + size); #ifdef HWPMC_HOOKS if (__predict_false(pmc_handled)) { /* downgrade the lock to prevent a LOR with the pmc-sx lock */ vm_map_lock_downgrade(map); if (pkm.pm_address != (uintptr_t) NULL) PMC_CALL_HOOK(td, PMC_FN_MUNMAP, (void *) &pkm); vm_map_unlock_read(map); } else #endif vm_map_unlock(map); /* vm_map_delete returns nothing but KERN_SUCCESS anyway */ return (0); } #ifndef _SYS_SYSPROTO_H_ struct mprotect_args { const void *addr; size_t len; int prot; }; #endif int sys_mprotect(struct thread *td, struct mprotect_args *uap) { return (kern_mprotect(td, (uintptr_t)uap->addr, uap->len, uap->prot)); } int kern_mprotect(struct thread *td, uintptr_t addr0, size_t size, int prot) { vm_offset_t addr; vm_size_t pageoff; int vm_error, max_prot; addr = addr0; if ((prot & ~(_PROT_ALL | PROT_MAX(_PROT_ALL))) != 0) return (EINVAL); max_prot = PROT_MAX_EXTRACT(prot); prot = PROT_EXTRACT(prot); pageoff = (addr & PAGE_MASK); addr -= pageoff; size += pageoff; size = (vm_size_t) round_page(size); #ifdef COMPAT_FREEBSD32 if (SV_PROC_FLAG(td->td_proc, SV_ILP32)) { if (((addr + size) & 0xffffffff) < addr) return (EINVAL); } else #endif if (addr + size < addr) return (EINVAL); vm_error = KERN_SUCCESS; if (max_prot != 0) { if ((max_prot & prot) != prot) return (EINVAL); vm_error = vm_map_protect(&td->td_proc->p_vmspace->vm_map, addr, addr + size, max_prot, TRUE); } if (vm_error == KERN_SUCCESS) vm_error = vm_map_protect(&td->td_proc->p_vmspace->vm_map, addr, addr + size, prot, FALSE); switch (vm_error) { case KERN_SUCCESS: return (0); case KERN_PROTECTION_FAILURE: return (EACCES); case KERN_RESOURCE_SHORTAGE: return (ENOMEM); } return (EINVAL); } #ifndef _SYS_SYSPROTO_H_ struct minherit_args { void *addr; size_t len; int inherit; }; #endif int sys_minherit(struct thread *td, struct minherit_args *uap) { vm_offset_t addr; vm_size_t size, pageoff; vm_inherit_t inherit; addr = (vm_offset_t)uap->addr; size = uap->len; inherit = uap->inherit; pageoff = (addr & PAGE_MASK); addr -= pageoff; size += pageoff; size = (vm_size_t) round_page(size); if (addr + size < addr) return (EINVAL); switch (vm_map_inherit(&td->td_proc->p_vmspace->vm_map, addr, addr + size, inherit)) { case KERN_SUCCESS: return (0); case KERN_PROTECTION_FAILURE: return (EACCES); } return (EINVAL); } #ifndef _SYS_SYSPROTO_H_ struct madvise_args { void *addr; size_t len; int behav; }; #endif int sys_madvise(struct thread *td, struct madvise_args *uap) { return (kern_madvise(td, (uintptr_t)uap->addr, uap->len, uap->behav)); } int kern_madvise(struct thread *td, uintptr_t addr0, size_t len, int behav) { vm_map_t map; vm_offset_t addr, end, start; int flags; /* * Check for our special case, advising the swap pager we are * "immortal." */ if (behav == MADV_PROTECT) { flags = PPROT_SET; return (kern_procctl(td, P_PID, td->td_proc->p_pid, PROC_SPROTECT, &flags)); } /* * Check for illegal addresses. Watch out for address wrap... Note * that VM_*_ADDRESS are not constants due to casts (argh). */ map = &td->td_proc->p_vmspace->vm_map; addr = addr0; if (addr < vm_map_min(map) || addr + len > vm_map_max(map)) return (EINVAL); if ((addr + len) < addr) return (EINVAL); /* * Since this routine is only advisory, we default to conservative * behavior. */ start = trunc_page(addr); end = round_page(addr + len); /* * vm_map_madvise() checks for illegal values of behav. */ return (vm_map_madvise(map, start, end, behav)); } #ifndef _SYS_SYSPROTO_H_ struct mincore_args { const void *addr; size_t len; char *vec; }; #endif int sys_mincore(struct thread *td, struct mincore_args *uap) { return (kern_mincore(td, (uintptr_t)uap->addr, uap->len, uap->vec)); } int kern_mincore(struct thread *td, uintptr_t addr0, size_t len, char *vec) { pmap_t pmap; vm_map_t map; vm_map_entry_t current, entry; vm_object_t object; vm_offset_t addr, cend, end, first_addr; vm_paddr_t pa; vm_page_t m; vm_pindex_t pindex; int error, lastvecindex, mincoreinfo, vecindex; unsigned int timestamp; /* * Make sure that the addresses presented are valid for user * mode. */ first_addr = addr = trunc_page(addr0); end = round_page(addr0 + len); map = &td->td_proc->p_vmspace->vm_map; if (end > vm_map_max(map) || end < addr) return (ENOMEM); pmap = vmspace_pmap(td->td_proc->p_vmspace); vm_map_lock_read(map); RestartScan: timestamp = map->timestamp; if (!vm_map_lookup_entry(map, addr, &entry)) { vm_map_unlock_read(map); return (ENOMEM); } /* * Do this on a map entry basis so that if the pages are not * in the current processes address space, we can easily look * up the pages elsewhere. */ lastvecindex = -1; while (entry->start < end) { /* * check for contiguity */ current = entry; entry = vm_map_entry_succ(current); if (current->end < end && entry->start > current->end) { vm_map_unlock_read(map); return (ENOMEM); } /* * ignore submaps (for now) or null objects */ if ((current->eflags & MAP_ENTRY_IS_SUB_MAP) || current->object.vm_object == NULL) continue; /* * limit this scan to the current map entry and the * limits for the mincore call */ if (addr < current->start) addr = current->start; cend = current->end; if (cend > end) cend = end; for (; addr < cend; addr += PAGE_SIZE) { /* * Check pmap first, it is likely faster, also * it can provide info as to whether we are the * one referencing or modifying the page. */ m = NULL; object = NULL; retry: pa = 0; mincoreinfo = pmap_mincore(pmap, addr, &pa); if (mincore_mapped) { /* * We only care about this pmap's * mapping of the page, if any. */ ; } else if (pa != 0) { /* * The page is mapped by this process but not * both accessed and modified. It is also * managed. Acquire the object lock so that * other mappings might be examined. The page's * identity may change at any point before its * object lock is acquired, so re-validate if * necessary. */ m = PHYS_TO_VM_PAGE(pa); while (object == NULL || m->object != object) { if (object != NULL) VM_OBJECT_WUNLOCK(object); - object = (vm_object_t)atomic_load_ptr( - &m->object); + object = atomic_load_ptr(&m->object); if (object == NULL) goto retry; VM_OBJECT_WLOCK(object); } if (pa != pmap_extract(pmap, addr)) goto retry; KASSERT(vm_page_all_valid(m), ("mincore: page %p is mapped but invalid", m)); } else if (mincoreinfo == 0) { /* * The page is not mapped by this process. If * the object implements managed pages, then * determine if the page is resident so that * the mappings might be examined. */ if (current->object.vm_object != object) { if (object != NULL) VM_OBJECT_WUNLOCK(object); object = current->object.vm_object; VM_OBJECT_WLOCK(object); } if (object->type == OBJT_DEFAULT || object->type == OBJT_SWAP || object->type == OBJT_VNODE) { pindex = OFF_TO_IDX(current->offset + (addr - current->start)); m = vm_page_lookup(object, pindex); if (m != NULL && vm_page_none_valid(m)) m = NULL; if (m != NULL) mincoreinfo = MINCORE_INCORE; } } if (m != NULL) { VM_OBJECT_ASSERT_WLOCKED(m->object); /* Examine other mappings of the page. */ if (m->dirty == 0 && pmap_is_modified(m)) vm_page_dirty(m); if (m->dirty != 0) mincoreinfo |= MINCORE_MODIFIED_OTHER; /* * The first test for PGA_REFERENCED is an * optimization. The second test is * required because a concurrent pmap * operation could clear the last reference * and set PGA_REFERENCED before the call to * pmap_is_referenced(). */ if ((m->a.flags & PGA_REFERENCED) != 0 || pmap_is_referenced(m) || (m->a.flags & PGA_REFERENCED) != 0) mincoreinfo |= MINCORE_REFERENCED_OTHER; } if (object != NULL) VM_OBJECT_WUNLOCK(object); /* * subyte may page fault. In case it needs to modify * the map, we release the lock. */ vm_map_unlock_read(map); /* * calculate index into user supplied byte vector */ vecindex = atop(addr - first_addr); /* * If we have skipped map entries, we need to make sure that * the byte vector is zeroed for those skipped entries. */ while ((lastvecindex + 1) < vecindex) { ++lastvecindex; error = subyte(vec + lastvecindex, 0); if (error) { error = EFAULT; goto done2; } } /* * Pass the page information to the user */ error = subyte(vec + vecindex, mincoreinfo); if (error) { error = EFAULT; goto done2; } /* * If the map has changed, due to the subyte, the previous * output may be invalid. */ vm_map_lock_read(map); if (timestamp != map->timestamp) goto RestartScan; lastvecindex = vecindex; } } /* * subyte may page fault. In case it needs to modify * the map, we release the lock. */ vm_map_unlock_read(map); /* * Zero the last entries in the byte vector. */ vecindex = atop(end - first_addr); while ((lastvecindex + 1) < vecindex) { ++lastvecindex; error = subyte(vec + lastvecindex, 0); if (error) { error = EFAULT; goto done2; } } /* * If the map has changed, due to the subyte, the previous * output may be invalid. */ vm_map_lock_read(map); if (timestamp != map->timestamp) goto RestartScan; vm_map_unlock_read(map); done2: return (error); } #ifndef _SYS_SYSPROTO_H_ struct mlock_args { const void *addr; size_t len; }; #endif int sys_mlock(struct thread *td, struct mlock_args *uap) { return (kern_mlock(td->td_proc, td->td_ucred, __DECONST(uintptr_t, uap->addr), uap->len)); } int kern_mlock(struct proc *proc, struct ucred *cred, uintptr_t addr0, size_t len) { vm_offset_t addr, end, last, start; vm_size_t npages, size; vm_map_t map; unsigned long nsize; int error; error = priv_check_cred(cred, PRIV_VM_MLOCK); if (error) return (error); addr = addr0; size = len; last = addr + size; start = trunc_page(addr); end = round_page(last); if (last < addr || end < addr) return (EINVAL); npages = atop(end - start); if (npages > vm_page_max_user_wired) return (ENOMEM); map = &proc->p_vmspace->vm_map; PROC_LOCK(proc); nsize = ptoa(npages + pmap_wired_count(map->pmap)); if (nsize > lim_cur_proc(proc, RLIMIT_MEMLOCK)) { PROC_UNLOCK(proc); return (ENOMEM); } PROC_UNLOCK(proc); #ifdef RACCT if (racct_enable) { PROC_LOCK(proc); error = racct_set(proc, RACCT_MEMLOCK, nsize); PROC_UNLOCK(proc); if (error != 0) return (ENOMEM); } #endif error = vm_map_wire(map, start, end, VM_MAP_WIRE_USER | VM_MAP_WIRE_NOHOLES); #ifdef RACCT if (racct_enable && error != KERN_SUCCESS) { PROC_LOCK(proc); racct_set(proc, RACCT_MEMLOCK, ptoa(pmap_wired_count(map->pmap))); PROC_UNLOCK(proc); } #endif return (error == KERN_SUCCESS ? 0 : ENOMEM); } #ifndef _SYS_SYSPROTO_H_ struct mlockall_args { int how; }; #endif int sys_mlockall(struct thread *td, struct mlockall_args *uap) { vm_map_t map; int error; map = &td->td_proc->p_vmspace->vm_map; error = priv_check(td, PRIV_VM_MLOCK); if (error) return (error); if ((uap->how == 0) || ((uap->how & ~(MCL_CURRENT|MCL_FUTURE)) != 0)) return (EINVAL); /* * If wiring all pages in the process would cause it to exceed * a hard resource limit, return ENOMEM. */ if (!old_mlock && uap->how & MCL_CURRENT) { if (map->size > lim_cur(td, RLIMIT_MEMLOCK)) return (ENOMEM); } #ifdef RACCT if (racct_enable) { PROC_LOCK(td->td_proc); error = racct_set(td->td_proc, RACCT_MEMLOCK, map->size); PROC_UNLOCK(td->td_proc); if (error != 0) return (ENOMEM); } #endif if (uap->how & MCL_FUTURE) { vm_map_lock(map); vm_map_modflags(map, MAP_WIREFUTURE, 0); vm_map_unlock(map); error = 0; } if (uap->how & MCL_CURRENT) { /* * P1003.1-2001 mandates that all currently mapped pages * will be memory resident and locked (wired) upon return * from mlockall(). vm_map_wire() will wire pages, by * calling vm_fault_wire() for each page in the region. */ error = vm_map_wire(map, vm_map_min(map), vm_map_max(map), VM_MAP_WIRE_USER|VM_MAP_WIRE_HOLESOK); if (error == KERN_SUCCESS) error = 0; else if (error == KERN_RESOURCE_SHORTAGE) error = ENOMEM; else error = EAGAIN; } #ifdef RACCT if (racct_enable && error != KERN_SUCCESS) { PROC_LOCK(td->td_proc); racct_set(td->td_proc, RACCT_MEMLOCK, ptoa(pmap_wired_count(map->pmap))); PROC_UNLOCK(td->td_proc); } #endif return (error); } #ifndef _SYS_SYSPROTO_H_ struct munlockall_args { register_t dummy; }; #endif int sys_munlockall(struct thread *td, struct munlockall_args *uap) { vm_map_t map; int error; map = &td->td_proc->p_vmspace->vm_map; error = priv_check(td, PRIV_VM_MUNLOCK); if (error) return (error); /* Clear the MAP_WIREFUTURE flag from this vm_map. */ vm_map_lock(map); vm_map_modflags(map, 0, MAP_WIREFUTURE); vm_map_unlock(map); /* Forcibly unwire all pages. */ error = vm_map_unwire(map, vm_map_min(map), vm_map_max(map), VM_MAP_WIRE_USER|VM_MAP_WIRE_HOLESOK); #ifdef RACCT if (racct_enable && error == KERN_SUCCESS) { PROC_LOCK(td->td_proc); racct_set(td->td_proc, RACCT_MEMLOCK, 0); PROC_UNLOCK(td->td_proc); } #endif return (error); } #ifndef _SYS_SYSPROTO_H_ struct munlock_args { const void *addr; size_t len; }; #endif int sys_munlock(struct thread *td, struct munlock_args *uap) { return (kern_munlock(td, (uintptr_t)uap->addr, uap->len)); } int kern_munlock(struct thread *td, uintptr_t addr0, size_t size) { vm_offset_t addr, end, last, start; #ifdef RACCT vm_map_t map; #endif int error; error = priv_check(td, PRIV_VM_MUNLOCK); if (error) return (error); addr = addr0; last = addr + size; start = trunc_page(addr); end = round_page(last); if (last < addr || end < addr) return (EINVAL); error = vm_map_unwire(&td->td_proc->p_vmspace->vm_map, start, end, VM_MAP_WIRE_USER | VM_MAP_WIRE_NOHOLES); #ifdef RACCT if (racct_enable && error == KERN_SUCCESS) { PROC_LOCK(td->td_proc); map = &td->td_proc->p_vmspace->vm_map; racct_set(td->td_proc, RACCT_MEMLOCK, ptoa(pmap_wired_count(map->pmap))); PROC_UNLOCK(td->td_proc); } #endif return (error == KERN_SUCCESS ? 0 : ENOMEM); } /* * vm_mmap_vnode() * * Helper function for vm_mmap. Perform sanity check specific for mmap * operations on vnodes. */ int vm_mmap_vnode(struct thread *td, vm_size_t objsize, vm_prot_t prot, vm_prot_t *maxprotp, int *flagsp, struct vnode *vp, vm_ooffset_t *foffp, vm_object_t *objp, boolean_t *writecounted) { struct vattr va; vm_object_t obj; vm_ooffset_t foff; struct ucred *cred; int error, flags; bool writex; cred = td->td_ucred; writex = (*maxprotp & VM_PROT_WRITE) != 0 && (*flagsp & MAP_SHARED) != 0; if ((error = vget(vp, LK_SHARED, td)) != 0) return (error); AUDIT_ARG_VNODE1(vp); foff = *foffp; flags = *flagsp; obj = vp->v_object; if (vp->v_type == VREG) { /* * Get the proper underlying object */ if (obj == NULL) { error = EINVAL; goto done; } if (obj->type == OBJT_VNODE && obj->handle != vp) { vput(vp); vp = (struct vnode *)obj->handle; /* * Bypass filesystems obey the mpsafety of the * underlying fs. Tmpfs never bypasses. */ error = vget(vp, LK_SHARED, td); if (error != 0) return (error); } if (writex) { *writecounted = TRUE; vm_pager_update_writecount(obj, 0, objsize); } } else { error = EINVAL; goto done; } if ((error = VOP_GETATTR(vp, &va, cred))) goto done; #ifdef MAC /* This relies on VM_PROT_* matching PROT_*. */ error = mac_vnode_check_mmap(cred, vp, (int)prot, flags); if (error != 0) goto done; #endif if ((flags & MAP_SHARED) != 0) { if ((va.va_flags & (SF_SNAPSHOT|IMMUTABLE|APPEND)) != 0) { if (prot & VM_PROT_WRITE) { error = EPERM; goto done; } *maxprotp &= ~VM_PROT_WRITE; } } /* * If it is a regular file without any references * we do not need to sync it. * Adjust object size to be the size of actual file. */ objsize = round_page(va.va_size); if (va.va_nlink == 0) flags |= MAP_NOSYNC; if (obj->type == OBJT_VNODE) { obj = vm_pager_allocate(OBJT_VNODE, vp, objsize, prot, foff, cred); if (obj == NULL) { error = ENOMEM; goto done; } } else { KASSERT(obj->type == OBJT_DEFAULT || obj->type == OBJT_SWAP, ("wrong object type")); vm_object_reference(obj); #if VM_NRESERVLEVEL > 0 if ((obj->flags & OBJ_COLORED) == 0) { VM_OBJECT_WLOCK(obj); vm_object_color(obj, 0); VM_OBJECT_WUNLOCK(obj); } #endif } *objp = obj; *flagsp = flags; VOP_MMAPPED(vp); done: if (error != 0 && *writecounted) { *writecounted = FALSE; vm_pager_update_writecount(obj, objsize, 0); } vput(vp); return (error); } /* * vm_mmap_cdev() * * Helper function for vm_mmap. Perform sanity check specific for mmap * operations on cdevs. */ int vm_mmap_cdev(struct thread *td, vm_size_t objsize, vm_prot_t prot, vm_prot_t *maxprotp, int *flagsp, struct cdev *cdev, struct cdevsw *dsw, vm_ooffset_t *foff, vm_object_t *objp) { vm_object_t obj; int error, flags; flags = *flagsp; if (dsw->d_flags & D_MMAP_ANON) { *objp = NULL; *foff = 0; *maxprotp = VM_PROT_ALL; *flagsp |= MAP_ANON; return (0); } /* * cdevs do not provide private mappings of any kind. */ if ((*maxprotp & VM_PROT_WRITE) == 0 && (prot & VM_PROT_WRITE) != 0) return (EACCES); if (flags & (MAP_PRIVATE|MAP_COPY)) return (EINVAL); /* * Force device mappings to be shared. */ flags |= MAP_SHARED; #ifdef MAC_XXX error = mac_cdev_check_mmap(td->td_ucred, cdev, (int)prot); if (error != 0) return (error); #endif /* * First, try d_mmap_single(). If that is not implemented * (returns ENODEV), fall back to using the device pager. * Note that d_mmap_single() must return a reference to the * object (it needs to bump the reference count of the object * it returns somehow). * * XXX assumes VM_PROT_* == PROT_* */ error = dsw->d_mmap_single(cdev, foff, objsize, objp, (int)prot); if (error != ENODEV) return (error); obj = vm_pager_allocate(OBJT_DEVICE, cdev, objsize, prot, *foff, td->td_ucred); if (obj == NULL) return (EINVAL); *objp = obj; *flagsp = flags; return (0); } /* * vm_mmap() * * Internal version of mmap used by exec, sys5 shared memory, and * various device drivers. Handle is either a vnode pointer, a * character device, or NULL for MAP_ANON. */ int vm_mmap(vm_map_t map, vm_offset_t *addr, vm_size_t size, vm_prot_t prot, vm_prot_t maxprot, int flags, objtype_t handle_type, void *handle, vm_ooffset_t foff) { vm_object_t object; struct thread *td = curthread; int error; boolean_t writecounted; if (size == 0) return (EINVAL); size = round_page(size); object = NULL; writecounted = FALSE; /* * Lookup/allocate object. */ switch (handle_type) { case OBJT_DEVICE: { struct cdevsw *dsw; struct cdev *cdev; int ref; cdev = handle; dsw = dev_refthread(cdev, &ref); if (dsw == NULL) return (ENXIO); error = vm_mmap_cdev(td, size, prot, &maxprot, &flags, cdev, dsw, &foff, &object); dev_relthread(cdev, ref); break; } case OBJT_VNODE: error = vm_mmap_vnode(td, size, prot, &maxprot, &flags, handle, &foff, &object, &writecounted); break; case OBJT_DEFAULT: if (handle == NULL) { error = 0; break; } /* FALLTHROUGH */ default: error = EINVAL; break; } if (error) return (error); error = vm_mmap_object(map, addr, size, prot, maxprot, flags, object, foff, writecounted, td); if (error != 0 && object != NULL) { /* * If this mapping was accounted for in the vnode's * writecount, then undo that now. */ if (writecounted) vm_pager_release_writecount(object, 0, size); vm_object_deallocate(object); } return (error); } /* * Internal version of mmap that maps a specific VM object into an * map. Called by mmap for MAP_ANON, vm_mmap, shm_mmap, and vn_mmap. */ int vm_mmap_object(vm_map_t map, vm_offset_t *addr, vm_size_t size, vm_prot_t prot, vm_prot_t maxprot, int flags, vm_object_t object, vm_ooffset_t foff, boolean_t writecounted, struct thread *td) { boolean_t curmap, fitit; vm_offset_t max_addr; int docow, error, findspace, rv; curmap = map == &td->td_proc->p_vmspace->vm_map; if (curmap) { RACCT_PROC_LOCK(td->td_proc); if (map->size + size > lim_cur(td, RLIMIT_VMEM)) { RACCT_PROC_UNLOCK(td->td_proc); return (ENOMEM); } if (racct_set(td->td_proc, RACCT_VMEM, map->size + size)) { RACCT_PROC_UNLOCK(td->td_proc); return (ENOMEM); } if (!old_mlock && map->flags & MAP_WIREFUTURE) { if (ptoa(pmap_wired_count(map->pmap)) + size > lim_cur(td, RLIMIT_MEMLOCK)) { racct_set_force(td->td_proc, RACCT_VMEM, map->size); RACCT_PROC_UNLOCK(td->td_proc); return (ENOMEM); } error = racct_set(td->td_proc, RACCT_MEMLOCK, ptoa(pmap_wired_count(map->pmap)) + size); if (error != 0) { racct_set_force(td->td_proc, RACCT_VMEM, map->size); RACCT_PROC_UNLOCK(td->td_proc); return (error); } } RACCT_PROC_UNLOCK(td->td_proc); } /* * We currently can only deal with page aligned file offsets. * The mmap() system call already enforces this by subtracting * the page offset from the file offset, but checking here * catches errors in device drivers (e.g. d_single_mmap() * callbacks) and other internal mapping requests (such as in * exec). */ if (foff & PAGE_MASK) return (EINVAL); if ((flags & MAP_FIXED) == 0) { fitit = TRUE; *addr = round_page(*addr); } else { if (*addr != trunc_page(*addr)) return (EINVAL); fitit = FALSE; } if (flags & MAP_ANON) { if (object != NULL || foff != 0) return (EINVAL); docow = 0; } else if (flags & MAP_PREFAULT_READ) docow = MAP_PREFAULT; else docow = MAP_PREFAULT_PARTIAL; if ((flags & (MAP_ANON|MAP_SHARED)) == 0) docow |= MAP_COPY_ON_WRITE; if (flags & MAP_NOSYNC) docow |= MAP_DISABLE_SYNCER; if (flags & MAP_NOCORE) docow |= MAP_DISABLE_COREDUMP; /* Shared memory is also shared with children. */ if (flags & MAP_SHARED) docow |= MAP_INHERIT_SHARE; if (writecounted) docow |= MAP_WRITECOUNT; if (flags & MAP_STACK) { if (object != NULL) return (EINVAL); docow |= MAP_STACK_GROWS_DOWN; } if ((flags & MAP_EXCL) != 0) docow |= MAP_CHECK_EXCL; if ((flags & MAP_GUARD) != 0) docow |= MAP_CREATE_GUARD; if (fitit) { if ((flags & MAP_ALIGNMENT_MASK) == MAP_ALIGNED_SUPER) findspace = VMFS_SUPER_SPACE; else if ((flags & MAP_ALIGNMENT_MASK) != 0) findspace = VMFS_ALIGNED_SPACE(flags >> MAP_ALIGNMENT_SHIFT); else findspace = VMFS_OPTIMAL_SPACE; max_addr = 0; #ifdef MAP_32BIT if ((flags & MAP_32BIT) != 0) max_addr = MAP_32BIT_MAX_ADDR; #endif if (curmap) { rv = vm_map_find_min(map, object, foff, addr, size, round_page((vm_offset_t)td->td_proc->p_vmspace-> vm_daddr + lim_max(td, RLIMIT_DATA)), max_addr, findspace, prot, maxprot, docow); } else { rv = vm_map_find(map, object, foff, addr, size, max_addr, findspace, prot, maxprot, docow); } } else { rv = vm_map_fixed(map, object, foff, *addr, size, prot, maxprot, docow); } if (rv == KERN_SUCCESS) { /* * If the process has requested that all future mappings * be wired, then heed this. */ if ((map->flags & MAP_WIREFUTURE) != 0) { vm_map_lock(map); if ((map->flags & MAP_WIREFUTURE) != 0) (void)vm_map_wire_locked(map, *addr, *addr + size, VM_MAP_WIRE_USER | ((flags & MAP_STACK) ? VM_MAP_WIRE_HOLESOK : VM_MAP_WIRE_NOHOLES)); vm_map_unlock(map); } } return (vm_mmap_to_errno(rv)); } /* * Translate a Mach VM return code to zero on success or the appropriate errno * on failure. */ int vm_mmap_to_errno(int rv) { switch (rv) { case KERN_SUCCESS: return (0); case KERN_INVALID_ADDRESS: case KERN_NO_SPACE: return (ENOMEM); case KERN_PROTECTION_FAILURE: return (EACCES); default: return (EINVAL); } } Index: projects/clang1000-import/sys/vm/vm_page.c =================================================================== --- projects/clang1000-import/sys/vm/vm_page.c (revision 357965) +++ projects/clang1000-import/sys/vm/vm_page.c (revision 357966) @@ -1,5165 +1,5163 @@ /*- * SPDX-License-Identifier: (BSD-3-Clause AND MIT-CMU) * * Copyright (c) 1991 Regents of the University of California. * All rights reserved. * Copyright (c) 1998 Matthew Dillon. All Rights Reserved. * * This code is derived from software contributed to Berkeley by * The Mach Operating System project at Carnegie-Mellon University. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * from: @(#)vm_page.c 7.4 (Berkeley) 5/7/91 */ /*- * Copyright (c) 1987, 1990 Carnegie-Mellon University. * All rights reserved. * * Authors: Avadis Tevanian, Jr., Michael Wayne Young * * Permission to use, copy, modify and distribute this software and * its documentation is hereby granted, provided that both the copyright * notice and this permission notice appear in all copies of the * software, derivative works or modified versions, and any portions * thereof, and that both notices appear in supporting documentation. * * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. * * Carnegie Mellon requests users of this software to return to * * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU * School of Computer Science * Carnegie Mellon University * Pittsburgh PA 15213-3890 * * any improvements or extensions that they make and grant Carnegie the * rights to redistribute these changes. */ /* * Resident memory management module. */ #include __FBSDID("$FreeBSD$"); #include "opt_vm.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include struct vm_domain vm_dom[MAXMEMDOM]; DPCPU_DEFINE_STATIC(struct vm_batchqueue, pqbatch[MAXMEMDOM][PQ_COUNT]); struct mtx_padalign __exclusive_cache_line pa_lock[PA_LOCK_COUNT]; struct mtx_padalign __exclusive_cache_line vm_domainset_lock; /* The following fields are protected by the domainset lock. */ domainset_t __exclusive_cache_line vm_min_domains; domainset_t __exclusive_cache_line vm_severe_domains; static int vm_min_waiters; static int vm_severe_waiters; static int vm_pageproc_waiters; static SYSCTL_NODE(_vm_stats, OID_AUTO, page, CTLFLAG_RD, 0, "VM page statistics"); static counter_u64_t pqstate_commit_retries = EARLY_COUNTER; SYSCTL_COUNTER_U64(_vm_stats_page, OID_AUTO, pqstate_commit_retries, CTLFLAG_RD, &pqstate_commit_retries, "Number of failed per-page atomic queue state updates"); static counter_u64_t queue_ops = EARLY_COUNTER; SYSCTL_COUNTER_U64(_vm_stats_page, OID_AUTO, queue_ops, CTLFLAG_RD, &queue_ops, "Number of batched queue operations"); static counter_u64_t queue_nops = EARLY_COUNTER; SYSCTL_COUNTER_U64(_vm_stats_page, OID_AUTO, queue_nops, CTLFLAG_RD, &queue_nops, "Number of batched queue operations with no effects"); static void counter_startup(void) { pqstate_commit_retries = counter_u64_alloc(M_WAITOK); queue_ops = counter_u64_alloc(M_WAITOK); queue_nops = counter_u64_alloc(M_WAITOK); } SYSINIT(page_counters, SI_SUB_CPU, SI_ORDER_ANY, counter_startup, NULL); /* * bogus page -- for I/O to/from partially complete buffers, * or for paging into sparsely invalid regions. */ vm_page_t bogus_page; vm_page_t vm_page_array; long vm_page_array_size; long first_page; static TAILQ_HEAD(, vm_page) blacklist_head; static int sysctl_vm_page_blacklist(SYSCTL_HANDLER_ARGS); SYSCTL_PROC(_vm, OID_AUTO, page_blacklist, CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, 0, sysctl_vm_page_blacklist, "A", "Blacklist pages"); static uma_zone_t fakepg_zone; static void vm_page_alloc_check(vm_page_t m); static bool _vm_page_busy_sleep(vm_object_t obj, vm_page_t m, const char *wmesg, bool nonshared, bool locked); static void vm_page_clear_dirty_mask(vm_page_t m, vm_page_bits_t pagebits); static void vm_page_enqueue(vm_page_t m, uint8_t queue); static bool vm_page_free_prep(vm_page_t m); static void vm_page_free_toq(vm_page_t m); static void vm_page_init(void *dummy); static int vm_page_insert_after(vm_page_t m, vm_object_t object, vm_pindex_t pindex, vm_page_t mpred); static void vm_page_insert_radixdone(vm_page_t m, vm_object_t object, vm_page_t mpred); static void vm_page_mvqueue(vm_page_t m, const uint8_t queue, const uint16_t nflag); static int vm_page_reclaim_run(int req_class, int domain, u_long npages, vm_page_t m_run, vm_paddr_t high); static void vm_page_release_toq(vm_page_t m, uint8_t nqueue, bool noreuse); static int vm_domain_alloc_fail(struct vm_domain *vmd, vm_object_t object, int req); static int vm_page_zone_import(void *arg, void **store, int cnt, int domain, int flags); static void vm_page_zone_release(void *arg, void **store, int cnt); SYSINIT(vm_page, SI_SUB_VM, SI_ORDER_SECOND, vm_page_init, NULL); static void vm_page_init(void *dummy) { fakepg_zone = uma_zcreate("fakepg", sizeof(struct vm_page), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE | UMA_ZONE_VM); bogus_page = vm_page_alloc(NULL, 0, VM_ALLOC_NOOBJ | VM_ALLOC_NORMAL | VM_ALLOC_WIRED); } /* * The cache page zone is initialized later since we need to be able to allocate * pages before UMA is fully initialized. */ static void vm_page_init_cache_zones(void *dummy __unused) { struct vm_domain *vmd; struct vm_pgcache *pgcache; int cache, domain, maxcache, pool; maxcache = 0; TUNABLE_INT_FETCH("vm.pgcache_zone_max_pcpu", &maxcache); maxcache *= mp_ncpus; for (domain = 0; domain < vm_ndomains; domain++) { vmd = VM_DOMAIN(domain); for (pool = 0; pool < VM_NFREEPOOL; pool++) { pgcache = &vmd->vmd_pgcache[pool]; pgcache->domain = domain; pgcache->pool = pool; pgcache->zone = uma_zcache_create("vm pgcache", PAGE_SIZE, NULL, NULL, NULL, NULL, vm_page_zone_import, vm_page_zone_release, pgcache, UMA_ZONE_VM); /* * Limit each pool's zone to 0.1% of the pages in the * domain. */ cache = maxcache != 0 ? maxcache : vmd->vmd_page_count / 1000; uma_zone_set_maxcache(pgcache->zone, cache); } } } SYSINIT(vm_page2, SI_SUB_VM_CONF, SI_ORDER_ANY, vm_page_init_cache_zones, NULL); /* Make sure that u_long is at least 64 bits when PAGE_SIZE is 32K. */ #if PAGE_SIZE == 32768 #ifdef CTASSERT CTASSERT(sizeof(u_long) >= 8); #endif #endif /* * vm_set_page_size: * * Sets the page size, perhaps based upon the memory * size. Must be called before any use of page-size * dependent functions. */ void vm_set_page_size(void) { if (vm_cnt.v_page_size == 0) vm_cnt.v_page_size = PAGE_SIZE; if (((vm_cnt.v_page_size - 1) & vm_cnt.v_page_size) != 0) panic("vm_set_page_size: page size not a power of two"); } /* * vm_page_blacklist_next: * * Find the next entry in the provided string of blacklist * addresses. Entries are separated by space, comma, or newline. * If an invalid integer is encountered then the rest of the * string is skipped. Updates the list pointer to the next * character, or NULL if the string is exhausted or invalid. */ static vm_paddr_t vm_page_blacklist_next(char **list, char *end) { vm_paddr_t bad; char *cp, *pos; if (list == NULL || *list == NULL) return (0); if (**list =='\0') { *list = NULL; return (0); } /* * If there's no end pointer then the buffer is coming from * the kenv and we know it's null-terminated. */ if (end == NULL) end = *list + strlen(*list); /* Ensure that strtoq() won't walk off the end */ if (*end != '\0') { if (*end == '\n' || *end == ' ' || *end == ',') *end = '\0'; else { printf("Blacklist not terminated, skipping\n"); *list = NULL; return (0); } } for (pos = *list; *pos != '\0'; pos = cp) { bad = strtoq(pos, &cp, 0); if (*cp == '\0' || *cp == ' ' || *cp == ',' || *cp == '\n') { if (bad == 0) { if (++cp < end) continue; else break; } } else break; if (*cp == '\0' || ++cp >= end) *list = NULL; else *list = cp; return (trunc_page(bad)); } printf("Garbage in RAM blacklist, skipping\n"); *list = NULL; return (0); } bool vm_page_blacklist_add(vm_paddr_t pa, bool verbose) { struct vm_domain *vmd; vm_page_t m; int ret; m = vm_phys_paddr_to_vm_page(pa); if (m == NULL) return (true); /* page does not exist, no failure */ vmd = vm_pagequeue_domain(m); vm_domain_free_lock(vmd); ret = vm_phys_unfree_page(m); vm_domain_free_unlock(vmd); if (ret != 0) { vm_domain_freecnt_inc(vmd, -1); TAILQ_INSERT_TAIL(&blacklist_head, m, listq); if (verbose) printf("Skipping page with pa 0x%jx\n", (uintmax_t)pa); } return (ret); } /* * vm_page_blacklist_check: * * Iterate through the provided string of blacklist addresses, pulling * each entry out of the physical allocator free list and putting it * onto a list for reporting via the vm.page_blacklist sysctl. */ static void vm_page_blacklist_check(char *list, char *end) { vm_paddr_t pa; char *next; next = list; while (next != NULL) { if ((pa = vm_page_blacklist_next(&next, end)) == 0) continue; vm_page_blacklist_add(pa, bootverbose); } } /* * vm_page_blacklist_load: * * Search for a special module named "ram_blacklist". It'll be a * plain text file provided by the user via the loader directive * of the same name. */ static void vm_page_blacklist_load(char **list, char **end) { void *mod; u_char *ptr; u_int len; mod = NULL; ptr = NULL; mod = preload_search_by_type("ram_blacklist"); if (mod != NULL) { ptr = preload_fetch_addr(mod); len = preload_fetch_size(mod); } *list = ptr; if (ptr != NULL) *end = ptr + len; else *end = NULL; return; } static int sysctl_vm_page_blacklist(SYSCTL_HANDLER_ARGS) { vm_page_t m; struct sbuf sbuf; int error, first; first = 1; error = sysctl_wire_old_buffer(req, 0); if (error != 0) return (error); sbuf_new_for_sysctl(&sbuf, NULL, 128, req); TAILQ_FOREACH(m, &blacklist_head, listq) { sbuf_printf(&sbuf, "%s%#jx", first ? "" : ",", (uintmax_t)m->phys_addr); first = 0; } error = sbuf_finish(&sbuf); sbuf_delete(&sbuf); return (error); } /* * Initialize a dummy page for use in scans of the specified paging queue. * In principle, this function only needs to set the flag PG_MARKER. * Nonetheless, it write busies the page as a safety precaution. */ static void vm_page_init_marker(vm_page_t marker, int queue, uint16_t aflags) { bzero(marker, sizeof(*marker)); marker->flags = PG_MARKER; marker->a.flags = aflags; marker->busy_lock = VPB_CURTHREAD_EXCLUSIVE; marker->a.queue = queue; } static void vm_page_domain_init(int domain) { struct vm_domain *vmd; struct vm_pagequeue *pq; int i; vmd = VM_DOMAIN(domain); bzero(vmd, sizeof(*vmd)); *__DECONST(char **, &vmd->vmd_pagequeues[PQ_INACTIVE].pq_name) = "vm inactive pagequeue"; *__DECONST(char **, &vmd->vmd_pagequeues[PQ_ACTIVE].pq_name) = "vm active pagequeue"; *__DECONST(char **, &vmd->vmd_pagequeues[PQ_LAUNDRY].pq_name) = "vm laundry pagequeue"; *__DECONST(char **, &vmd->vmd_pagequeues[PQ_UNSWAPPABLE].pq_name) = "vm unswappable pagequeue"; vmd->vmd_domain = domain; vmd->vmd_page_count = 0; vmd->vmd_free_count = 0; vmd->vmd_segs = 0; vmd->vmd_oom = FALSE; for (i = 0; i < PQ_COUNT; i++) { pq = &vmd->vmd_pagequeues[i]; TAILQ_INIT(&pq->pq_pl); mtx_init(&pq->pq_mutex, pq->pq_name, "vm pagequeue", MTX_DEF | MTX_DUPOK); pq->pq_pdpages = 0; vm_page_init_marker(&vmd->vmd_markers[i], i, 0); } mtx_init(&vmd->vmd_free_mtx, "vm page free queue", NULL, MTX_DEF); mtx_init(&vmd->vmd_pageout_mtx, "vm pageout lock", NULL, MTX_DEF); snprintf(vmd->vmd_name, sizeof(vmd->vmd_name), "%d", domain); /* * inacthead is used to provide FIFO ordering for LRU-bypassing * insertions. */ vm_page_init_marker(&vmd->vmd_inacthead, PQ_INACTIVE, PGA_ENQUEUED); TAILQ_INSERT_HEAD(&vmd->vmd_pagequeues[PQ_INACTIVE].pq_pl, &vmd->vmd_inacthead, plinks.q); /* * The clock pages are used to implement active queue scanning without * requeues. Scans start at clock[0], which is advanced after the scan * ends. When the two clock hands meet, they are reset and scanning * resumes from the head of the queue. */ vm_page_init_marker(&vmd->vmd_clock[0], PQ_ACTIVE, PGA_ENQUEUED); vm_page_init_marker(&vmd->vmd_clock[1], PQ_ACTIVE, PGA_ENQUEUED); TAILQ_INSERT_HEAD(&vmd->vmd_pagequeues[PQ_ACTIVE].pq_pl, &vmd->vmd_clock[0], plinks.q); TAILQ_INSERT_TAIL(&vmd->vmd_pagequeues[PQ_ACTIVE].pq_pl, &vmd->vmd_clock[1], plinks.q); } /* * Initialize a physical page in preparation for adding it to the free * lists. */ static void vm_page_init_page(vm_page_t m, vm_paddr_t pa, int segind) { m->object = NULL; m->ref_count = 0; m->busy_lock = VPB_FREED; m->flags = m->a.flags = 0; m->phys_addr = pa; m->a.queue = PQ_NONE; m->psind = 0; m->segind = segind; m->order = VM_NFREEORDER; m->pool = VM_FREEPOOL_DEFAULT; m->valid = m->dirty = 0; pmap_page_init(m); } #ifndef PMAP_HAS_PAGE_ARRAY static vm_paddr_t vm_page_array_alloc(vm_offset_t *vaddr, vm_paddr_t end, vm_paddr_t page_range) { vm_paddr_t new_end; /* * Reserve an unmapped guard page to trap access to vm_page_array[-1]. * However, because this page is allocated from KVM, out-of-bounds * accesses using the direct map will not be trapped. */ *vaddr += PAGE_SIZE; /* * Allocate physical memory for the page structures, and map it. */ new_end = trunc_page(end - page_range * sizeof(struct vm_page)); vm_page_array = (vm_page_t)pmap_map(vaddr, new_end, end, VM_PROT_READ | VM_PROT_WRITE); vm_page_array_size = page_range; return (new_end); } #endif /* * vm_page_startup: * * Initializes the resident memory module. Allocates physical memory for * bootstrapping UMA and some data structures that are used to manage * physical pages. Initializes these structures, and populates the free * page queues. */ vm_offset_t vm_page_startup(vm_offset_t vaddr) { struct vm_phys_seg *seg; vm_page_t m; char *list, *listend; vm_paddr_t end, high_avail, low_avail, new_end, size; vm_paddr_t page_range __unused; vm_paddr_t last_pa, pa; u_long pagecount; int biggestone, i, segind; #ifdef WITNESS vm_offset_t mapped; int witness_size; #endif #if defined(__i386__) && defined(VM_PHYSSEG_DENSE) long ii; #endif vaddr = round_page(vaddr); vm_phys_early_startup(); biggestone = vm_phys_avail_largest(); end = phys_avail[biggestone+1]; /* * Initialize the page and queue locks. */ mtx_init(&vm_domainset_lock, "vm domainset lock", NULL, MTX_DEF); for (i = 0; i < PA_LOCK_COUNT; i++) mtx_init(&pa_lock[i], "vm page", NULL, MTX_DEF); for (i = 0; i < vm_ndomains; i++) vm_page_domain_init(i); new_end = end; #ifdef WITNESS witness_size = round_page(witness_startup_count()); new_end -= witness_size; mapped = pmap_map(&vaddr, new_end, new_end + witness_size, VM_PROT_READ | VM_PROT_WRITE); bzero((void *)mapped, witness_size); witness_startup((void *)mapped); #endif #if defined(__aarch64__) || defined(__amd64__) || defined(__arm__) || \ defined(__i386__) || defined(__mips__) || defined(__riscv) || \ defined(__powerpc64__) /* * Allocate a bitmap to indicate that a random physical page * needs to be included in a minidump. * * The amd64 port needs this to indicate which direct map pages * need to be dumped, via calls to dump_add_page()/dump_drop_page(). * * However, i386 still needs this workspace internally within the * minidump code. In theory, they are not needed on i386, but are * included should the sf_buf code decide to use them. */ last_pa = 0; for (i = 0; dump_avail[i + 1] != 0; i += 2) if (dump_avail[i + 1] > last_pa) last_pa = dump_avail[i + 1]; page_range = last_pa / PAGE_SIZE; vm_page_dump_size = round_page(roundup2(page_range, NBBY) / NBBY); new_end -= vm_page_dump_size; vm_page_dump = (void *)(uintptr_t)pmap_map(&vaddr, new_end, new_end + vm_page_dump_size, VM_PROT_READ | VM_PROT_WRITE); bzero((void *)vm_page_dump, vm_page_dump_size); #else (void)last_pa; #endif #if defined(__aarch64__) || defined(__amd64__) || defined(__mips__) || \ defined(__riscv) || defined(__powerpc64__) /* * Include the UMA bootstrap pages, witness pages and vm_page_dump * in a crash dump. When pmap_map() uses the direct map, they are * not automatically included. */ for (pa = new_end; pa < end; pa += PAGE_SIZE) dump_add_page(pa); #endif phys_avail[biggestone + 1] = new_end; #ifdef __amd64__ /* * Request that the physical pages underlying the message buffer be * included in a crash dump. Since the message buffer is accessed * through the direct map, they are not automatically included. */ pa = DMAP_TO_PHYS((vm_offset_t)msgbufp->msg_ptr); last_pa = pa + round_page(msgbufsize); while (pa < last_pa) { dump_add_page(pa); pa += PAGE_SIZE; } #endif /* * Compute the number of pages of memory that will be available for * use, taking into account the overhead of a page structure per page. * In other words, solve * "available physical memory" - round_page(page_range * * sizeof(struct vm_page)) = page_range * PAGE_SIZE * for page_range. */ low_avail = phys_avail[0]; high_avail = phys_avail[1]; for (i = 0; i < vm_phys_nsegs; i++) { if (vm_phys_segs[i].start < low_avail) low_avail = vm_phys_segs[i].start; if (vm_phys_segs[i].end > high_avail) high_avail = vm_phys_segs[i].end; } /* Skip the first chunk. It is already accounted for. */ for (i = 2; phys_avail[i + 1] != 0; i += 2) { if (phys_avail[i] < low_avail) low_avail = phys_avail[i]; if (phys_avail[i + 1] > high_avail) high_avail = phys_avail[i + 1]; } first_page = low_avail / PAGE_SIZE; #ifdef VM_PHYSSEG_SPARSE size = 0; for (i = 0; i < vm_phys_nsegs; i++) size += vm_phys_segs[i].end - vm_phys_segs[i].start; for (i = 0; phys_avail[i + 1] != 0; i += 2) size += phys_avail[i + 1] - phys_avail[i]; #elif defined(VM_PHYSSEG_DENSE) size = high_avail - low_avail; #else #error "Either VM_PHYSSEG_DENSE or VM_PHYSSEG_SPARSE must be defined." #endif #ifdef PMAP_HAS_PAGE_ARRAY pmap_page_array_startup(size / PAGE_SIZE); biggestone = vm_phys_avail_largest(); end = new_end = phys_avail[biggestone + 1]; #else #ifdef VM_PHYSSEG_DENSE /* * In the VM_PHYSSEG_DENSE case, the number of pages can account for * the overhead of a page structure per page only if vm_page_array is * allocated from the last physical memory chunk. Otherwise, we must * allocate page structures representing the physical memory * underlying vm_page_array, even though they will not be used. */ if (new_end != high_avail) page_range = size / PAGE_SIZE; else #endif { page_range = size / (PAGE_SIZE + sizeof(struct vm_page)); /* * If the partial bytes remaining are large enough for * a page (PAGE_SIZE) without a corresponding * 'struct vm_page', then new_end will contain an * extra page after subtracting the length of the VM * page array. Compensate by subtracting an extra * page from new_end. */ if (size % (PAGE_SIZE + sizeof(struct vm_page)) >= PAGE_SIZE) { if (new_end == high_avail) high_avail -= PAGE_SIZE; new_end -= PAGE_SIZE; } } end = new_end; new_end = vm_page_array_alloc(&vaddr, end, page_range); #endif #if VM_NRESERVLEVEL > 0 /* * Allocate physical memory for the reservation management system's * data structures, and map it. */ new_end = vm_reserv_startup(&vaddr, new_end); #endif #if defined(__aarch64__) || defined(__amd64__) || defined(__mips__) || \ defined(__riscv) || defined(__powerpc64__) /* * Include vm_page_array and vm_reserv_array in a crash dump. */ for (pa = new_end; pa < end; pa += PAGE_SIZE) dump_add_page(pa); #endif phys_avail[biggestone + 1] = new_end; /* * Add physical memory segments corresponding to the available * physical pages. */ for (i = 0; phys_avail[i + 1] != 0; i += 2) if (vm_phys_avail_size(i) != 0) vm_phys_add_seg(phys_avail[i], phys_avail[i + 1]); /* * Initialize the physical memory allocator. */ vm_phys_init(); /* * Initialize the page structures and add every available page to the * physical memory allocator's free lists. */ #if defined(__i386__) && defined(VM_PHYSSEG_DENSE) for (ii = 0; ii < vm_page_array_size; ii++) { m = &vm_page_array[ii]; vm_page_init_page(m, (first_page + ii) << PAGE_SHIFT, 0); m->flags = PG_FICTITIOUS; } #endif vm_cnt.v_page_count = 0; for (segind = 0; segind < vm_phys_nsegs; segind++) { seg = &vm_phys_segs[segind]; for (m = seg->first_page, pa = seg->start; pa < seg->end; m++, pa += PAGE_SIZE) vm_page_init_page(m, pa, segind); /* * Add the segment to the free lists only if it is covered by * one of the ranges in phys_avail. Because we've added the * ranges to the vm_phys_segs array, we can assume that each * segment is either entirely contained in one of the ranges, * or doesn't overlap any of them. */ for (i = 0; phys_avail[i + 1] != 0; i += 2) { struct vm_domain *vmd; if (seg->start < phys_avail[i] || seg->end > phys_avail[i + 1]) continue; m = seg->first_page; pagecount = (u_long)atop(seg->end - seg->start); vmd = VM_DOMAIN(seg->domain); vm_domain_free_lock(vmd); vm_phys_enqueue_contig(m, pagecount); vm_domain_free_unlock(vmd); vm_domain_freecnt_inc(vmd, pagecount); vm_cnt.v_page_count += (u_int)pagecount; vmd = VM_DOMAIN(seg->domain); vmd->vmd_page_count += (u_int)pagecount; vmd->vmd_segs |= 1UL << m->segind; break; } } /* * Remove blacklisted pages from the physical memory allocator. */ TAILQ_INIT(&blacklist_head); vm_page_blacklist_load(&list, &listend); vm_page_blacklist_check(list, listend); list = kern_getenv("vm.blacklist"); vm_page_blacklist_check(list, NULL); freeenv(list); #if VM_NRESERVLEVEL > 0 /* * Initialize the reservation management system. */ vm_reserv_init(); #endif return (vaddr); } void vm_page_reference(vm_page_t m) { vm_page_aflag_set(m, PGA_REFERENCED); } static bool vm_page_acquire_flags(vm_page_t m, int allocflags) { bool locked; if ((allocflags & (VM_ALLOC_SBUSY | VM_ALLOC_IGN_SBUSY)) != 0) locked = vm_page_trysbusy(m); else locked = vm_page_tryxbusy(m); if (locked && (allocflags & VM_ALLOC_WIRED) != 0) vm_page_wire(m); return (locked); } /* * vm_page_busy_sleep_flags * * Sleep for busy according to VM_ALLOC_ parameters. */ static bool vm_page_busy_sleep_flags(vm_object_t object, vm_page_t m, const char *wmesg, int allocflags) { if ((allocflags & VM_ALLOC_NOWAIT) != 0) return (false); /* * Reference the page before unlocking and * sleeping so that the page daemon is less * likely to reclaim it. */ if ((allocflags & VM_ALLOC_NOCREAT) == 0) vm_page_aflag_set(m, PGA_REFERENCED); if (_vm_page_busy_sleep(object, m, wmesg, (allocflags & VM_ALLOC_IGN_SBUSY) != 0, true)) VM_OBJECT_WLOCK(object); if ((allocflags & VM_ALLOC_WAITFAIL) != 0) return (false); return (true); } /* * vm_page_busy_acquire: * * Acquire the busy lock as described by VM_ALLOC_* flags. Will loop * and drop the object lock if necessary. */ bool vm_page_busy_acquire(vm_page_t m, int allocflags) { vm_object_t obj; bool locked; /* * The page-specific object must be cached because page * identity can change during the sleep, causing the * re-lock of a different object. * It is assumed that a reference to the object is already * held by the callers. */ obj = m->object; for (;;) { if (vm_page_acquire_flags(m, allocflags)) return (true); if ((allocflags & VM_ALLOC_NOWAIT) != 0) return (false); if (obj != NULL) locked = VM_OBJECT_WOWNED(obj); else locked = false; MPASS(locked || vm_page_wired(m)); if (_vm_page_busy_sleep(obj, m, "vmpba", (allocflags & VM_ALLOC_SBUSY) != 0, locked)) VM_OBJECT_WLOCK(obj); if ((allocflags & VM_ALLOC_WAITFAIL) != 0) return (false); KASSERT(m->object == obj || m->object == NULL, ("vm_page_busy_acquire: page %p does not belong to %p", m, obj)); } } /* * vm_page_busy_downgrade: * * Downgrade an exclusive busy page into a single shared busy page. */ void vm_page_busy_downgrade(vm_page_t m) { u_int x; vm_page_assert_xbusied(m); x = m->busy_lock; for (;;) { if (atomic_fcmpset_rel_int(&m->busy_lock, &x, VPB_SHARERS_WORD(1))) break; } if ((x & VPB_BIT_WAITERS) != 0) wakeup(m); } /* * * vm_page_busy_tryupgrade: * * Attempt to upgrade a single shared busy into an exclusive busy. */ int vm_page_busy_tryupgrade(vm_page_t m) { u_int ce, x; vm_page_assert_sbusied(m); x = m->busy_lock; ce = VPB_CURTHREAD_EXCLUSIVE; for (;;) { if (VPB_SHARERS(x) > 1) return (0); KASSERT((x & ~VPB_BIT_WAITERS) == VPB_SHARERS_WORD(1), ("vm_page_busy_tryupgrade: invalid lock state")); if (!atomic_fcmpset_acq_int(&m->busy_lock, &x, ce | (x & VPB_BIT_WAITERS))) continue; return (1); } } /* * vm_page_sbusied: * * Return a positive value if the page is shared busied, 0 otherwise. */ int vm_page_sbusied(vm_page_t m) { u_int x; x = m->busy_lock; return ((x & VPB_BIT_SHARED) != 0 && x != VPB_UNBUSIED); } /* * vm_page_sunbusy: * * Shared unbusy a page. */ void vm_page_sunbusy(vm_page_t m) { u_int x; vm_page_assert_sbusied(m); x = m->busy_lock; for (;;) { KASSERT(x != VPB_FREED, ("vm_page_sunbusy: Unlocking freed page.")); if (VPB_SHARERS(x) > 1) { if (atomic_fcmpset_int(&m->busy_lock, &x, x - VPB_ONE_SHARER)) break; continue; } KASSERT((x & ~VPB_BIT_WAITERS) == VPB_SHARERS_WORD(1), ("vm_page_sunbusy: invalid lock state")); if (!atomic_fcmpset_rel_int(&m->busy_lock, &x, VPB_UNBUSIED)) continue; if ((x & VPB_BIT_WAITERS) == 0) break; wakeup(m); break; } } /* * vm_page_busy_sleep: * * Sleep if the page is busy, using the page pointer as wchan. * This is used to implement the hard-path of busying mechanism. * * If nonshared is true, sleep only if the page is xbusy. * * The object lock must be held on entry and will be released on exit. */ void vm_page_busy_sleep(vm_page_t m, const char *wmesg, bool nonshared) { vm_object_t obj; obj = m->object; VM_OBJECT_ASSERT_LOCKED(obj); vm_page_lock_assert(m, MA_NOTOWNED); if (!_vm_page_busy_sleep(obj, m, wmesg, nonshared, true)) VM_OBJECT_DROP(obj); } /* * _vm_page_busy_sleep: * * Internal busy sleep function. */ static bool _vm_page_busy_sleep(vm_object_t obj, vm_page_t m, const char *wmesg, bool nonshared, bool locked) { u_int x; /* * If the object is busy we must wait for that to drain to zero * before trying the page again. */ if (obj != NULL && vm_object_busied(obj)) { if (locked) VM_OBJECT_DROP(obj); vm_object_busy_wait(obj, wmesg); return (locked); } sleepq_lock(m); x = m->busy_lock; if (x == VPB_UNBUSIED || (nonshared && (x & VPB_BIT_SHARED) != 0) || ((x & VPB_BIT_WAITERS) == 0 && !atomic_cmpset_int(&m->busy_lock, x, x | VPB_BIT_WAITERS))) { sleepq_release(m); return (false); } if (locked) VM_OBJECT_DROP(obj); DROP_GIANT(); sleepq_add(m, NULL, wmesg, 0, 0); sleepq_wait(m, PVM); PICKUP_GIANT(); return (locked); } /* * vm_page_trysbusy: * * Try to shared busy a page. * If the operation succeeds 1 is returned otherwise 0. * The operation never sleeps. */ int vm_page_trysbusy(vm_page_t m) { vm_object_t obj; u_int x; obj = m->object; x = m->busy_lock; for (;;) { if ((x & VPB_BIT_SHARED) == 0) return (0); /* * Reduce the window for transient busies that will trigger * false negatives in vm_page_ps_test(). */ if (obj != NULL && vm_object_busied(obj)) return (0); if (atomic_fcmpset_acq_int(&m->busy_lock, &x, x + VPB_ONE_SHARER)) break; } /* Refetch the object now that we're guaranteed that it is stable. */ obj = m->object; if (obj != NULL && vm_object_busied(obj)) { vm_page_sunbusy(m); return (0); } return (1); } /* * vm_page_tryxbusy: * * Try to exclusive busy a page. * If the operation succeeds 1 is returned otherwise 0. * The operation never sleeps. */ int vm_page_tryxbusy(vm_page_t m) { vm_object_t obj; if (atomic_cmpset_acq_int(&(m)->busy_lock, VPB_UNBUSIED, VPB_CURTHREAD_EXCLUSIVE) == 0) return (0); obj = m->object; if (obj != NULL && vm_object_busied(obj)) { vm_page_xunbusy(m); return (0); } return (1); } static void vm_page_xunbusy_hard_tail(vm_page_t m) { atomic_store_rel_int(&m->busy_lock, VPB_UNBUSIED); /* Wake the waiter. */ wakeup(m); } /* * vm_page_xunbusy_hard: * * Called when unbusy has failed because there is a waiter. */ void vm_page_xunbusy_hard(vm_page_t m) { vm_page_assert_xbusied(m); vm_page_xunbusy_hard_tail(m); } void vm_page_xunbusy_hard_unchecked(vm_page_t m) { vm_page_assert_xbusied_unchecked(m); vm_page_xunbusy_hard_tail(m); } static void vm_page_busy_free(vm_page_t m) { u_int x; atomic_thread_fence_rel(); x = atomic_swap_int(&m->busy_lock, VPB_FREED); if ((x & VPB_BIT_WAITERS) != 0) wakeup(m); } /* * vm_page_unhold_pages: * * Unhold each of the pages that is referenced by the given array. */ void vm_page_unhold_pages(vm_page_t *ma, int count) { for (; count != 0; count--) { vm_page_unwire(*ma, PQ_ACTIVE); ma++; } } vm_page_t PHYS_TO_VM_PAGE(vm_paddr_t pa) { vm_page_t m; #ifdef VM_PHYSSEG_SPARSE m = vm_phys_paddr_to_vm_page(pa); if (m == NULL) m = vm_phys_fictitious_to_vm_page(pa); return (m); #elif defined(VM_PHYSSEG_DENSE) long pi; pi = atop(pa); if (pi >= first_page && (pi - first_page) < vm_page_array_size) { m = &vm_page_array[pi - first_page]; return (m); } return (vm_phys_fictitious_to_vm_page(pa)); #else #error "Either VM_PHYSSEG_DENSE or VM_PHYSSEG_SPARSE must be defined." #endif } /* * vm_page_getfake: * * Create a fictitious page with the specified physical address and * memory attribute. The memory attribute is the only the machine- * dependent aspect of a fictitious page that must be initialized. */ vm_page_t vm_page_getfake(vm_paddr_t paddr, vm_memattr_t memattr) { vm_page_t m; m = uma_zalloc(fakepg_zone, M_WAITOK | M_ZERO); vm_page_initfake(m, paddr, memattr); return (m); } void vm_page_initfake(vm_page_t m, vm_paddr_t paddr, vm_memattr_t memattr) { if ((m->flags & PG_FICTITIOUS) != 0) { /* * The page's memattr might have changed since the * previous initialization. Update the pmap to the * new memattr. */ goto memattr; } m->phys_addr = paddr; m->a.queue = PQ_NONE; /* Fictitious pages don't use "segind". */ m->flags = PG_FICTITIOUS; /* Fictitious pages don't use "order" or "pool". */ m->oflags = VPO_UNMANAGED; m->busy_lock = VPB_CURTHREAD_EXCLUSIVE; /* Fictitious pages are unevictable. */ m->ref_count = 1; pmap_page_init(m); memattr: pmap_page_set_memattr(m, memattr); } /* * vm_page_putfake: * * Release a fictitious page. */ void vm_page_putfake(vm_page_t m) { KASSERT((m->oflags & VPO_UNMANAGED) != 0, ("managed %p", m)); KASSERT((m->flags & PG_FICTITIOUS) != 0, ("vm_page_putfake: bad page %p", m)); vm_page_assert_xbusied(m); vm_page_busy_free(m); uma_zfree(fakepg_zone, m); } /* * vm_page_updatefake: * * Update the given fictitious page to the specified physical address and * memory attribute. */ void vm_page_updatefake(vm_page_t m, vm_paddr_t paddr, vm_memattr_t memattr) { KASSERT((m->flags & PG_FICTITIOUS) != 0, ("vm_page_updatefake: bad page %p", m)); m->phys_addr = paddr; pmap_page_set_memattr(m, memattr); } /* * vm_page_free: * * Free a page. */ void vm_page_free(vm_page_t m) { m->flags &= ~PG_ZERO; vm_page_free_toq(m); } /* * vm_page_free_zero: * * Free a page to the zerod-pages queue */ void vm_page_free_zero(vm_page_t m) { m->flags |= PG_ZERO; vm_page_free_toq(m); } /* * Unbusy and handle the page queueing for a page from a getpages request that * was optionally read ahead or behind. */ void vm_page_readahead_finish(vm_page_t m) { /* We shouldn't put invalid pages on queues. */ KASSERT(!vm_page_none_valid(m), ("%s: %p is invalid", __func__, m)); /* * Since the page is not the actually needed one, whether it should * be activated or deactivated is not obvious. Empirical results * have shown that deactivating the page is usually the best choice, * unless the page is wanted by another thread. */ if ((m->busy_lock & VPB_BIT_WAITERS) != 0) vm_page_activate(m); else vm_page_deactivate(m); vm_page_xunbusy_unchecked(m); } /* * vm_page_sleep_if_busy: * * Sleep and release the object lock if the page is busied. * Returns TRUE if the thread slept. * * The given page must be unlocked and object containing it must * be locked. */ int vm_page_sleep_if_busy(vm_page_t m, const char *msg) { vm_object_t obj; vm_page_lock_assert(m, MA_NOTOWNED); VM_OBJECT_ASSERT_WLOCKED(m->object); /* * The page-specific object must be cached because page * identity can change during the sleep, causing the * re-lock of a different object. * It is assumed that a reference to the object is already * held by the callers. */ obj = m->object; if (vm_page_busied(m) || (obj != NULL && obj->busy)) { vm_page_busy_sleep(m, msg, false); VM_OBJECT_WLOCK(obj); return (TRUE); } return (FALSE); } /* * vm_page_sleep_if_xbusy: * * Sleep and release the object lock if the page is xbusied. * Returns TRUE if the thread slept. * * The given page must be unlocked and object containing it must * be locked. */ int vm_page_sleep_if_xbusy(vm_page_t m, const char *msg) { vm_object_t obj; vm_page_lock_assert(m, MA_NOTOWNED); VM_OBJECT_ASSERT_WLOCKED(m->object); /* * The page-specific object must be cached because page * identity can change during the sleep, causing the * re-lock of a different object. * It is assumed that a reference to the object is already * held by the callers. */ obj = m->object; if (vm_page_xbusied(m) || (obj != NULL && obj->busy)) { vm_page_busy_sleep(m, msg, true); VM_OBJECT_WLOCK(obj); return (TRUE); } return (FALSE); } /* * vm_page_dirty_KBI: [ internal use only ] * * Set all bits in the page's dirty field. * * The object containing the specified page must be locked if the * call is made from the machine-independent layer. * * See vm_page_clear_dirty_mask(). * * This function should only be called by vm_page_dirty(). */ void vm_page_dirty_KBI(vm_page_t m) { /* Refer to this operation by its public name. */ KASSERT(vm_page_all_valid(m), ("vm_page_dirty: page is invalid!")); m->dirty = VM_PAGE_BITS_ALL; } /* * vm_page_insert: [ internal use only ] * * Inserts the given mem entry into the object and object list. * * The object must be locked. */ int vm_page_insert(vm_page_t m, vm_object_t object, vm_pindex_t pindex) { vm_page_t mpred; VM_OBJECT_ASSERT_WLOCKED(object); mpred = vm_radix_lookup_le(&object->rtree, pindex); return (vm_page_insert_after(m, object, pindex, mpred)); } /* * vm_page_insert_after: * * Inserts the page "m" into the specified object at offset "pindex". * * The page "mpred" must immediately precede the offset "pindex" within * the specified object. * * The object must be locked. */ static int vm_page_insert_after(vm_page_t m, vm_object_t object, vm_pindex_t pindex, vm_page_t mpred) { vm_page_t msucc; VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(m->object == NULL, ("vm_page_insert_after: page already inserted")); if (mpred != NULL) { KASSERT(mpred->object == object, ("vm_page_insert_after: object doesn't contain mpred")); KASSERT(mpred->pindex < pindex, ("vm_page_insert_after: mpred doesn't precede pindex")); msucc = TAILQ_NEXT(mpred, listq); } else msucc = TAILQ_FIRST(&object->memq); if (msucc != NULL) KASSERT(msucc->pindex > pindex, ("vm_page_insert_after: msucc doesn't succeed pindex")); /* * Record the object/offset pair in this page. */ m->object = object; m->pindex = pindex; m->ref_count |= VPRC_OBJREF; /* * Now link into the object's ordered list of backed pages. */ if (vm_radix_insert(&object->rtree, m)) { m->object = NULL; m->pindex = 0; m->ref_count &= ~VPRC_OBJREF; return (1); } vm_page_insert_radixdone(m, object, mpred); return (0); } /* * vm_page_insert_radixdone: * * Complete page "m" insertion into the specified object after the * radix trie hooking. * * The page "mpred" must precede the offset "m->pindex" within the * specified object. * * The object must be locked. */ static void vm_page_insert_radixdone(vm_page_t m, vm_object_t object, vm_page_t mpred) { VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(object != NULL && m->object == object, ("vm_page_insert_radixdone: page %p has inconsistent object", m)); KASSERT((m->ref_count & VPRC_OBJREF) != 0, ("vm_page_insert_radixdone: page %p is missing object ref", m)); if (mpred != NULL) { KASSERT(mpred->object == object, ("vm_page_insert_radixdone: object doesn't contain mpred")); KASSERT(mpred->pindex < m->pindex, ("vm_page_insert_radixdone: mpred doesn't precede pindex")); } if (mpred != NULL) TAILQ_INSERT_AFTER(&object->memq, mpred, m, listq); else TAILQ_INSERT_HEAD(&object->memq, m, listq); /* * Show that the object has one more resident page. */ object->resident_page_count++; /* * Hold the vnode until the last page is released. */ if (object->resident_page_count == 1 && object->type == OBJT_VNODE) vhold(object->handle); /* * Since we are inserting a new and possibly dirty page, * update the object's generation count. */ if (pmap_page_is_write_mapped(m)) vm_object_set_writeable_dirty(object); } /* * Do the work to remove a page from its object. The caller is responsible for * updating the page's fields to reflect this removal. */ static void vm_page_object_remove(vm_page_t m) { vm_object_t object; vm_page_t mrem; vm_page_assert_xbusied(m); object = m->object; VM_OBJECT_ASSERT_WLOCKED(object); KASSERT((m->ref_count & VPRC_OBJREF) != 0, ("page %p is missing its object ref", m)); /* Deferred free of swap space. */ if ((m->a.flags & PGA_SWAP_FREE) != 0) vm_pager_page_unswapped(m); mrem = vm_radix_remove(&object->rtree, m->pindex); KASSERT(mrem == m, ("removed page %p, expected page %p", mrem, m)); /* * Now remove from the object's list of backed pages. */ TAILQ_REMOVE(&object->memq, m, listq); /* * And show that the object has one fewer resident page. */ object->resident_page_count--; /* * The vnode may now be recycled. */ if (object->resident_page_count == 0 && object->type == OBJT_VNODE) vdrop(object->handle); } /* * vm_page_remove: * * Removes the specified page from its containing object, but does not * invalidate any backing storage. Returns true if the object's reference * was the last reference to the page, and false otherwise. * * The object must be locked and the page must be exclusively busied. * The exclusive busy will be released on return. If this is not the * final ref and the caller does not hold a wire reference it may not * continue to access the page. */ bool vm_page_remove(vm_page_t m) { bool dropped; dropped = vm_page_remove_xbusy(m); vm_page_xunbusy(m); return (dropped); } /* * vm_page_remove_xbusy * * Removes the page but leaves the xbusy held. Returns true if this * removed the final ref and false otherwise. */ bool vm_page_remove_xbusy(vm_page_t m) { vm_page_object_remove(m); m->object = NULL; return (vm_page_drop(m, VPRC_OBJREF) == VPRC_OBJREF); } /* * vm_page_lookup: * * Returns the page associated with the object/offset * pair specified; if none is found, NULL is returned. * * The object must be locked. */ vm_page_t vm_page_lookup(vm_object_t object, vm_pindex_t pindex) { VM_OBJECT_ASSERT_LOCKED(object); return (vm_radix_lookup(&object->rtree, pindex)); } /* * vm_page_find_least: * * Returns the page associated with the object with least pindex * greater than or equal to the parameter pindex, or NULL. * * The object must be locked. */ vm_page_t vm_page_find_least(vm_object_t object, vm_pindex_t pindex) { vm_page_t m; VM_OBJECT_ASSERT_LOCKED(object); if ((m = TAILQ_FIRST(&object->memq)) != NULL && m->pindex < pindex) m = vm_radix_lookup_ge(&object->rtree, pindex); return (m); } /* * Returns the given page's successor (by pindex) within the object if it is * resident; if none is found, NULL is returned. * * The object must be locked. */ vm_page_t vm_page_next(vm_page_t m) { vm_page_t next; VM_OBJECT_ASSERT_LOCKED(m->object); if ((next = TAILQ_NEXT(m, listq)) != NULL) { MPASS(next->object == m->object); if (next->pindex != m->pindex + 1) next = NULL; } return (next); } /* * Returns the given page's predecessor (by pindex) within the object if it is * resident; if none is found, NULL is returned. * * The object must be locked. */ vm_page_t vm_page_prev(vm_page_t m) { vm_page_t prev; VM_OBJECT_ASSERT_LOCKED(m->object); if ((prev = TAILQ_PREV(m, pglist, listq)) != NULL) { MPASS(prev->object == m->object); if (prev->pindex != m->pindex - 1) prev = NULL; } return (prev); } /* * Uses the page mnew as a replacement for an existing page at index * pindex which must be already present in the object. * * Both pages must be exclusively busied on enter. The old page is * unbusied on exit. * * A return value of true means mold is now free. If this is not the * final ref and the caller does not hold a wire reference it may not * continue to access the page. */ static bool vm_page_replace_hold(vm_page_t mnew, vm_object_t object, vm_pindex_t pindex, vm_page_t mold) { vm_page_t mret; bool dropped; VM_OBJECT_ASSERT_WLOCKED(object); vm_page_assert_xbusied(mold); KASSERT(mnew->object == NULL && (mnew->ref_count & VPRC_OBJREF) == 0, ("vm_page_replace: page %p already in object", mnew)); /* * This function mostly follows vm_page_insert() and * vm_page_remove() without the radix, object count and vnode * dance. Double check such functions for more comments. */ mnew->object = object; mnew->pindex = pindex; atomic_set_int(&mnew->ref_count, VPRC_OBJREF); mret = vm_radix_replace(&object->rtree, mnew); KASSERT(mret == mold, ("invalid page replacement, mold=%p, mret=%p", mold, mret)); KASSERT((mold->oflags & VPO_UNMANAGED) == (mnew->oflags & VPO_UNMANAGED), ("vm_page_replace: mismatched VPO_UNMANAGED")); /* Keep the resident page list in sorted order. */ TAILQ_INSERT_AFTER(&object->memq, mold, mnew, listq); TAILQ_REMOVE(&object->memq, mold, listq); mold->object = NULL; /* * The object's resident_page_count does not change because we have * swapped one page for another, but the generation count should * change if the page is dirty. */ if (pmap_page_is_write_mapped(mnew)) vm_object_set_writeable_dirty(object); dropped = vm_page_drop(mold, VPRC_OBJREF) == VPRC_OBJREF; vm_page_xunbusy(mold); return (dropped); } void vm_page_replace(vm_page_t mnew, vm_object_t object, vm_pindex_t pindex, vm_page_t mold) { vm_page_assert_xbusied(mnew); if (vm_page_replace_hold(mnew, object, pindex, mold)) vm_page_free(mold); } /* * vm_page_rename: * * Move the given memory entry from its * current object to the specified target object/offset. * * Note: swap associated with the page must be invalidated by the move. We * have to do this for several reasons: (1) we aren't freeing the * page, (2) we are dirtying the page, (3) the VM system is probably * moving the page from object A to B, and will then later move * the backing store from A to B and we can't have a conflict. * * Note: we *always* dirty the page. It is necessary both for the * fact that we moved it, and because we may be invalidating * swap. * * The objects must be locked. */ int vm_page_rename(vm_page_t m, vm_object_t new_object, vm_pindex_t new_pindex) { vm_page_t mpred; vm_pindex_t opidx; VM_OBJECT_ASSERT_WLOCKED(new_object); KASSERT(m->ref_count != 0, ("vm_page_rename: page %p has no refs", m)); mpred = vm_radix_lookup_le(&new_object->rtree, new_pindex); KASSERT(mpred == NULL || mpred->pindex != new_pindex, ("vm_page_rename: pindex already renamed")); /* * Create a custom version of vm_page_insert() which does not depend * by m_prev and can cheat on the implementation aspects of the * function. */ opidx = m->pindex; m->pindex = new_pindex; if (vm_radix_insert(&new_object->rtree, m)) { m->pindex = opidx; return (1); } /* * The operation cannot fail anymore. The removal must happen before * the listq iterator is tainted. */ m->pindex = opidx; vm_page_object_remove(m); /* Return back to the new pindex to complete vm_page_insert(). */ m->pindex = new_pindex; m->object = new_object; vm_page_insert_radixdone(m, new_object, mpred); vm_page_dirty(m); return (0); } /* * vm_page_alloc: * * Allocate and return a page that is associated with the specified * object and offset pair. By default, this page is exclusive busied. * * The caller must always specify an allocation class. * * allocation classes: * VM_ALLOC_NORMAL normal process request * VM_ALLOC_SYSTEM system *really* needs a page * VM_ALLOC_INTERRUPT interrupt time request * * optional allocation flags: * VM_ALLOC_COUNT(number) the number of additional pages that the caller * intends to allocate * VM_ALLOC_NOBUSY do not exclusive busy the page * VM_ALLOC_NODUMP do not include the page in a kernel core dump * VM_ALLOC_NOOBJ page is not associated with an object and * should not be exclusive busy * VM_ALLOC_SBUSY shared busy the allocated page * VM_ALLOC_WIRED wire the allocated page * VM_ALLOC_ZERO prefer a zeroed page */ vm_page_t vm_page_alloc(vm_object_t object, vm_pindex_t pindex, int req) { return (vm_page_alloc_after(object, pindex, req, object != NULL ? vm_radix_lookup_le(&object->rtree, pindex) : NULL)); } vm_page_t vm_page_alloc_domain(vm_object_t object, vm_pindex_t pindex, int domain, int req) { return (vm_page_alloc_domain_after(object, pindex, domain, req, object != NULL ? vm_radix_lookup_le(&object->rtree, pindex) : NULL)); } /* * Allocate a page in the specified object with the given page index. To * optimize insertion of the page into the object, the caller must also specifiy * the resident page in the object with largest index smaller than the given * page index, or NULL if no such page exists. */ vm_page_t vm_page_alloc_after(vm_object_t object, vm_pindex_t pindex, int req, vm_page_t mpred) { struct vm_domainset_iter di; vm_page_t m; int domain; vm_domainset_iter_page_init(&di, object, pindex, &domain, &req); do { m = vm_page_alloc_domain_after(object, pindex, domain, req, mpred); if (m != NULL) break; } while (vm_domainset_iter_page(&di, object, &domain) == 0); return (m); } /* * Returns true if the number of free pages exceeds the minimum * for the request class and false otherwise. */ static int _vm_domain_allocate(struct vm_domain *vmd, int req_class, int npages) { u_int limit, old, new; if (req_class == VM_ALLOC_INTERRUPT) limit = 0; else if (req_class == VM_ALLOC_SYSTEM) limit = vmd->vmd_interrupt_free_min; else limit = vmd->vmd_free_reserved; /* * Attempt to reserve the pages. Fail if we're below the limit. */ limit += npages; old = vmd->vmd_free_count; do { if (old < limit) return (0); new = old - npages; } while (atomic_fcmpset_int(&vmd->vmd_free_count, &old, new) == 0); /* Wake the page daemon if we've crossed the threshold. */ if (vm_paging_needed(vmd, new) && !vm_paging_needed(vmd, old)) pagedaemon_wakeup(vmd->vmd_domain); /* Only update bitsets on transitions. */ if ((old >= vmd->vmd_free_min && new < vmd->vmd_free_min) || (old >= vmd->vmd_free_severe && new < vmd->vmd_free_severe)) vm_domain_set(vmd); return (1); } int vm_domain_allocate(struct vm_domain *vmd, int req, int npages) { int req_class; /* * The page daemon is allowed to dig deeper into the free page list. */ req_class = req & VM_ALLOC_CLASS_MASK; if (curproc == pageproc && req_class != VM_ALLOC_INTERRUPT) req_class = VM_ALLOC_SYSTEM; return (_vm_domain_allocate(vmd, req_class, npages)); } vm_page_t vm_page_alloc_domain_after(vm_object_t object, vm_pindex_t pindex, int domain, int req, vm_page_t mpred) { struct vm_domain *vmd; vm_page_t m; int flags, pool; KASSERT((object != NULL) == ((req & VM_ALLOC_NOOBJ) == 0) && (object != NULL || (req & VM_ALLOC_SBUSY) == 0) && ((req & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) != (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)), ("inconsistent object(%p)/req(%x)", object, req)); KASSERT(object == NULL || (req & VM_ALLOC_WAITOK) == 0, ("Can't sleep and retry object insertion.")); KASSERT(mpred == NULL || mpred->pindex < pindex, ("mpred %p doesn't precede pindex 0x%jx", mpred, (uintmax_t)pindex)); if (object != NULL) VM_OBJECT_ASSERT_WLOCKED(object); flags = 0; m = NULL; pool = object != NULL ? VM_FREEPOOL_DEFAULT : VM_FREEPOOL_DIRECT; again: #if VM_NRESERVLEVEL > 0 /* * Can we allocate the page from a reservation? */ if (vm_object_reserv(object) && (m = vm_reserv_alloc_page(object, pindex, domain, req, mpred)) != NULL) { domain = vm_phys_domain(m); vmd = VM_DOMAIN(domain); goto found; } #endif vmd = VM_DOMAIN(domain); if (vmd->vmd_pgcache[pool].zone != NULL) { m = uma_zalloc(vmd->vmd_pgcache[pool].zone, M_NOWAIT); if (m != NULL) { flags |= PG_PCPU_CACHE; goto found; } } if (vm_domain_allocate(vmd, req, 1)) { /* * If not, allocate it from the free page queues. */ vm_domain_free_lock(vmd); m = vm_phys_alloc_pages(domain, pool, 0); vm_domain_free_unlock(vmd); if (m == NULL) { vm_domain_freecnt_inc(vmd, 1); #if VM_NRESERVLEVEL > 0 if (vm_reserv_reclaim_inactive(domain)) goto again; #endif } } if (m == NULL) { /* * Not allocatable, give up. */ if (vm_domain_alloc_fail(vmd, object, req)) goto again; return (NULL); } /* * At this point we had better have found a good page. */ found: vm_page_dequeue(m); vm_page_alloc_check(m); /* * Initialize the page. Only the PG_ZERO flag is inherited. */ if ((req & VM_ALLOC_ZERO) != 0) flags |= (m->flags & PG_ZERO); if ((req & VM_ALLOC_NODUMP) != 0) flags |= PG_NODUMP; m->flags = flags; m->a.flags = 0; m->oflags = object == NULL || (object->flags & OBJ_UNMANAGED) != 0 ? VPO_UNMANAGED : 0; if ((req & (VM_ALLOC_NOBUSY | VM_ALLOC_NOOBJ | VM_ALLOC_SBUSY)) == 0) m->busy_lock = VPB_CURTHREAD_EXCLUSIVE; else if ((req & VM_ALLOC_SBUSY) != 0) m->busy_lock = VPB_SHARERS_WORD(1); else m->busy_lock = VPB_UNBUSIED; if (req & VM_ALLOC_WIRED) { vm_wire_add(1); m->ref_count = 1; } m->a.act_count = 0; if (object != NULL) { if (vm_page_insert_after(m, object, pindex, mpred)) { if (req & VM_ALLOC_WIRED) { vm_wire_sub(1); m->ref_count = 0; } KASSERT(m->object == NULL, ("page %p has object", m)); m->oflags = VPO_UNMANAGED; m->busy_lock = VPB_UNBUSIED; /* Don't change PG_ZERO. */ vm_page_free_toq(m); if (req & VM_ALLOC_WAITFAIL) { VM_OBJECT_WUNLOCK(object); vm_radix_wait(); VM_OBJECT_WLOCK(object); } return (NULL); } /* Ignore device objects; the pager sets "memattr" for them. */ if (object->memattr != VM_MEMATTR_DEFAULT && (object->flags & OBJ_FICTITIOUS) == 0) pmap_page_set_memattr(m, object->memattr); } else m->pindex = pindex; return (m); } /* * vm_page_alloc_contig: * * Allocate a contiguous set of physical pages of the given size "npages" * from the free lists. All of the physical pages must be at or above * the given physical address "low" and below the given physical address * "high". The given value "alignment" determines the alignment of the * first physical page in the set. If the given value "boundary" is * non-zero, then the set of physical pages cannot cross any physical * address boundary that is a multiple of that value. Both "alignment" * and "boundary" must be a power of two. * * If the specified memory attribute, "memattr", is VM_MEMATTR_DEFAULT, * then the memory attribute setting for the physical pages is configured * to the object's memory attribute setting. Otherwise, the memory * attribute setting for the physical pages is configured to "memattr", * overriding the object's memory attribute setting. However, if the * object's memory attribute setting is not VM_MEMATTR_DEFAULT, then the * memory attribute setting for the physical pages cannot be configured * to VM_MEMATTR_DEFAULT. * * The specified object may not contain fictitious pages. * * The caller must always specify an allocation class. * * allocation classes: * VM_ALLOC_NORMAL normal process request * VM_ALLOC_SYSTEM system *really* needs a page * VM_ALLOC_INTERRUPT interrupt time request * * optional allocation flags: * VM_ALLOC_NOBUSY do not exclusive busy the page * VM_ALLOC_NODUMP do not include the page in a kernel core dump * VM_ALLOC_NOOBJ page is not associated with an object and * should not be exclusive busy * VM_ALLOC_SBUSY shared busy the allocated page * VM_ALLOC_WIRED wire the allocated page * VM_ALLOC_ZERO prefer a zeroed page */ vm_page_t vm_page_alloc_contig(vm_object_t object, vm_pindex_t pindex, int req, u_long npages, vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary, vm_memattr_t memattr) { struct vm_domainset_iter di; vm_page_t m; int domain; vm_domainset_iter_page_init(&di, object, pindex, &domain, &req); do { m = vm_page_alloc_contig_domain(object, pindex, domain, req, npages, low, high, alignment, boundary, memattr); if (m != NULL) break; } while (vm_domainset_iter_page(&di, object, &domain) == 0); return (m); } vm_page_t vm_page_alloc_contig_domain(vm_object_t object, vm_pindex_t pindex, int domain, int req, u_long npages, vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary, vm_memattr_t memattr) { struct vm_domain *vmd; vm_page_t m, m_ret, mpred; u_int busy_lock, flags, oflags; mpred = NULL; /* XXX: pacify gcc */ KASSERT((object != NULL) == ((req & VM_ALLOC_NOOBJ) == 0) && (object != NULL || (req & VM_ALLOC_SBUSY) == 0) && ((req & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) != (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)), ("vm_page_alloc_contig: inconsistent object(%p)/req(%x)", object, req)); KASSERT(object == NULL || (req & VM_ALLOC_WAITOK) == 0, ("Can't sleep and retry object insertion.")); if (object != NULL) { VM_OBJECT_ASSERT_WLOCKED(object); KASSERT((object->flags & OBJ_FICTITIOUS) == 0, ("vm_page_alloc_contig: object %p has fictitious pages", object)); } KASSERT(npages > 0, ("vm_page_alloc_contig: npages is zero")); if (object != NULL) { mpred = vm_radix_lookup_le(&object->rtree, pindex); KASSERT(mpred == NULL || mpred->pindex != pindex, ("vm_page_alloc_contig: pindex already allocated")); } /* * Can we allocate the pages without the number of free pages falling * below the lower bound for the allocation class? */ m_ret = NULL; again: #if VM_NRESERVLEVEL > 0 /* * Can we allocate the pages from a reservation? */ if (vm_object_reserv(object) && (m_ret = vm_reserv_alloc_contig(object, pindex, domain, req, mpred, npages, low, high, alignment, boundary)) != NULL) { domain = vm_phys_domain(m_ret); vmd = VM_DOMAIN(domain); goto found; } #endif vmd = VM_DOMAIN(domain); if (vm_domain_allocate(vmd, req, npages)) { /* * allocate them from the free page queues. */ vm_domain_free_lock(vmd); m_ret = vm_phys_alloc_contig(domain, npages, low, high, alignment, boundary); vm_domain_free_unlock(vmd); if (m_ret == NULL) { vm_domain_freecnt_inc(vmd, npages); #if VM_NRESERVLEVEL > 0 if (vm_reserv_reclaim_contig(domain, npages, low, high, alignment, boundary)) goto again; #endif } } if (m_ret == NULL) { if (vm_domain_alloc_fail(vmd, object, req)) goto again; return (NULL); } #if VM_NRESERVLEVEL > 0 found: #endif for (m = m_ret; m < &m_ret[npages]; m++) { vm_page_dequeue(m); vm_page_alloc_check(m); } /* * Initialize the pages. Only the PG_ZERO flag is inherited. */ flags = 0; if ((req & VM_ALLOC_ZERO) != 0) flags = PG_ZERO; if ((req & VM_ALLOC_NODUMP) != 0) flags |= PG_NODUMP; oflags = object == NULL || (object->flags & OBJ_UNMANAGED) != 0 ? VPO_UNMANAGED : 0; if ((req & (VM_ALLOC_NOBUSY | VM_ALLOC_NOOBJ | VM_ALLOC_SBUSY)) == 0) busy_lock = VPB_CURTHREAD_EXCLUSIVE; else if ((req & VM_ALLOC_SBUSY) != 0) busy_lock = VPB_SHARERS_WORD(1); else busy_lock = VPB_UNBUSIED; if ((req & VM_ALLOC_WIRED) != 0) vm_wire_add(npages); if (object != NULL) { if (object->memattr != VM_MEMATTR_DEFAULT && memattr == VM_MEMATTR_DEFAULT) memattr = object->memattr; } for (m = m_ret; m < &m_ret[npages]; m++) { m->a.flags = 0; m->flags = (m->flags | PG_NODUMP) & flags; m->busy_lock = busy_lock; if ((req & VM_ALLOC_WIRED) != 0) m->ref_count = 1; m->a.act_count = 0; m->oflags = oflags; if (object != NULL) { if (vm_page_insert_after(m, object, pindex, mpred)) { if ((req & VM_ALLOC_WIRED) != 0) vm_wire_sub(npages); KASSERT(m->object == NULL, ("page %p has object", m)); mpred = m; for (m = m_ret; m < &m_ret[npages]; m++) { if (m <= mpred && (req & VM_ALLOC_WIRED) != 0) m->ref_count = 0; m->oflags = VPO_UNMANAGED; m->busy_lock = VPB_UNBUSIED; /* Don't change PG_ZERO. */ vm_page_free_toq(m); } if (req & VM_ALLOC_WAITFAIL) { VM_OBJECT_WUNLOCK(object); vm_radix_wait(); VM_OBJECT_WLOCK(object); } return (NULL); } mpred = m; } else m->pindex = pindex; if (memattr != VM_MEMATTR_DEFAULT) pmap_page_set_memattr(m, memattr); pindex++; } return (m_ret); } /* * Check a page that has been freshly dequeued from a freelist. */ static void vm_page_alloc_check(vm_page_t m) { KASSERT(m->object == NULL, ("page %p has object", m)); KASSERT(m->a.queue == PQ_NONE && (m->a.flags & PGA_QUEUE_STATE_MASK) == 0, ("page %p has unexpected queue %d, flags %#x", m, m->a.queue, (m->a.flags & PGA_QUEUE_STATE_MASK))); KASSERT(m->ref_count == 0, ("page %p has references", m)); KASSERT(vm_page_busy_freed(m), ("page %p is not freed", m)); KASSERT(m->dirty == 0, ("page %p is dirty", m)); KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, ("page %p has unexpected memattr %d", m, pmap_page_get_memattr(m))); KASSERT(m->valid == 0, ("free page %p is valid", m)); } /* * vm_page_alloc_freelist: * * Allocate a physical page from the specified free page list. * * The caller must always specify an allocation class. * * allocation classes: * VM_ALLOC_NORMAL normal process request * VM_ALLOC_SYSTEM system *really* needs a page * VM_ALLOC_INTERRUPT interrupt time request * * optional allocation flags: * VM_ALLOC_COUNT(number) the number of additional pages that the caller * intends to allocate * VM_ALLOC_WIRED wire the allocated page * VM_ALLOC_ZERO prefer a zeroed page */ vm_page_t vm_page_alloc_freelist(int freelist, int req) { struct vm_domainset_iter di; vm_page_t m; int domain; vm_domainset_iter_page_init(&di, NULL, 0, &domain, &req); do { m = vm_page_alloc_freelist_domain(domain, freelist, req); if (m != NULL) break; } while (vm_domainset_iter_page(&di, NULL, &domain) == 0); return (m); } vm_page_t vm_page_alloc_freelist_domain(int domain, int freelist, int req) { struct vm_domain *vmd; vm_page_t m; u_int flags; m = NULL; vmd = VM_DOMAIN(domain); again: if (vm_domain_allocate(vmd, req, 1)) { vm_domain_free_lock(vmd); m = vm_phys_alloc_freelist_pages(domain, freelist, VM_FREEPOOL_DIRECT, 0); vm_domain_free_unlock(vmd); if (m == NULL) vm_domain_freecnt_inc(vmd, 1); } if (m == NULL) { if (vm_domain_alloc_fail(vmd, NULL, req)) goto again; return (NULL); } vm_page_dequeue(m); vm_page_alloc_check(m); /* * Initialize the page. Only the PG_ZERO flag is inherited. */ m->a.flags = 0; flags = 0; if ((req & VM_ALLOC_ZERO) != 0) flags = PG_ZERO; m->flags &= flags; if ((req & VM_ALLOC_WIRED) != 0) { vm_wire_add(1); m->ref_count = 1; } /* Unmanaged pages don't use "act_count". */ m->oflags = VPO_UNMANAGED; return (m); } static int vm_page_zone_import(void *arg, void **store, int cnt, int domain, int flags) { struct vm_domain *vmd; struct vm_pgcache *pgcache; int i; pgcache = arg; vmd = VM_DOMAIN(pgcache->domain); /* * The page daemon should avoid creating extra memory pressure since its * main purpose is to replenish the store of free pages. */ if (vmd->vmd_severeset || curproc == pageproc || !_vm_domain_allocate(vmd, VM_ALLOC_NORMAL, cnt)) return (0); domain = vmd->vmd_domain; vm_domain_free_lock(vmd); i = vm_phys_alloc_npages(domain, pgcache->pool, cnt, (vm_page_t *)store); vm_domain_free_unlock(vmd); if (cnt != i) vm_domain_freecnt_inc(vmd, cnt - i); return (i); } static void vm_page_zone_release(void *arg, void **store, int cnt) { struct vm_domain *vmd; struct vm_pgcache *pgcache; vm_page_t m; int i; pgcache = arg; vmd = VM_DOMAIN(pgcache->domain); vm_domain_free_lock(vmd); for (i = 0; i < cnt; i++) { m = (vm_page_t)store[i]; vm_phys_free_pages(m, 0); } vm_domain_free_unlock(vmd); vm_domain_freecnt_inc(vmd, cnt); } #define VPSC_ANY 0 /* No restrictions. */ #define VPSC_NORESERV 1 /* Skip reservations; implies VPSC_NOSUPER. */ #define VPSC_NOSUPER 2 /* Skip superpages. */ /* * vm_page_scan_contig: * * Scan vm_page_array[] between the specified entries "m_start" and * "m_end" for a run of contiguous physical pages that satisfy the * specified conditions, and return the lowest page in the run. The * specified "alignment" determines the alignment of the lowest physical * page in the run. If the specified "boundary" is non-zero, then the * run of physical pages cannot span a physical address that is a * multiple of "boundary". * * "m_end" is never dereferenced, so it need not point to a vm_page * structure within vm_page_array[]. * * "npages" must be greater than zero. "m_start" and "m_end" must not * span a hole (or discontiguity) in the physical address space. Both * "alignment" and "boundary" must be a power of two. */ vm_page_t vm_page_scan_contig(u_long npages, vm_page_t m_start, vm_page_t m_end, u_long alignment, vm_paddr_t boundary, int options) { vm_object_t object; vm_paddr_t pa; vm_page_t m, m_run; #if VM_NRESERVLEVEL > 0 int level; #endif int m_inc, order, run_ext, run_len; KASSERT(npages > 0, ("npages is 0")); KASSERT(powerof2(alignment), ("alignment is not a power of 2")); KASSERT(powerof2(boundary), ("boundary is not a power of 2")); m_run = NULL; run_len = 0; for (m = m_start; m < m_end && run_len < npages; m += m_inc) { KASSERT((m->flags & PG_MARKER) == 0, ("page %p is PG_MARKER", m)); KASSERT((m->flags & PG_FICTITIOUS) == 0 || m->ref_count >= 1, ("fictitious page %p has invalid ref count", m)); /* * If the current page would be the start of a run, check its * physical address against the end, alignment, and boundary * conditions. If it doesn't satisfy these conditions, either * terminate the scan or advance to the next page that * satisfies the failed condition. */ if (run_len == 0) { KASSERT(m_run == NULL, ("m_run != NULL")); if (m + npages > m_end) break; pa = VM_PAGE_TO_PHYS(m); if ((pa & (alignment - 1)) != 0) { m_inc = atop(roundup2(pa, alignment) - pa); continue; } if (rounddown2(pa ^ (pa + ptoa(npages) - 1), boundary) != 0) { m_inc = atop(roundup2(pa, boundary) - pa); continue; } } else KASSERT(m_run != NULL, ("m_run == NULL")); retry: m_inc = 1; if (vm_page_wired(m)) run_ext = 0; #if VM_NRESERVLEVEL > 0 else if ((level = vm_reserv_level(m)) >= 0 && (options & VPSC_NORESERV) != 0) { run_ext = 0; /* Advance to the end of the reservation. */ pa = VM_PAGE_TO_PHYS(m); m_inc = atop(roundup2(pa + 1, vm_reserv_size(level)) - pa); } #endif - else if ((object = - (vm_object_t)atomic_load_ptr(&m->object)) != NULL) { + else if ((object = atomic_load_ptr(&m->object)) != NULL) { /* * The page is considered eligible for relocation if * and only if it could be laundered or reclaimed by * the page daemon. */ VM_OBJECT_RLOCK(object); if (object != m->object) { VM_OBJECT_RUNLOCK(object); goto retry; } /* Don't care: PG_NODUMP, PG_ZERO. */ if (object->type != OBJT_DEFAULT && object->type != OBJT_SWAP && object->type != OBJT_VNODE) { run_ext = 0; #if VM_NRESERVLEVEL > 0 } else if ((options & VPSC_NOSUPER) != 0 && (level = vm_reserv_level_iffullpop(m)) >= 0) { run_ext = 0; /* Advance to the end of the superpage. */ pa = VM_PAGE_TO_PHYS(m); m_inc = atop(roundup2(pa + 1, vm_reserv_size(level)) - pa); #endif } else if (object->memattr == VM_MEMATTR_DEFAULT && vm_page_queue(m) != PQ_NONE && !vm_page_busied(m)) { /* * The page is allocated but eligible for * relocation. Extend the current run by one * page. */ KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, ("page %p has an unexpected memattr", m)); KASSERT((m->oflags & (VPO_SWAPINPROG | VPO_SWAPSLEEP | VPO_UNMANAGED)) == 0, ("page %p has unexpected oflags", m)); /* Don't care: PGA_NOSYNC. */ run_ext = 1; } else run_ext = 0; VM_OBJECT_RUNLOCK(object); #if VM_NRESERVLEVEL > 0 } else if (level >= 0) { /* * The page is reserved but not yet allocated. In * other words, it is still free. Extend the current * run by one page. */ run_ext = 1; #endif } else if ((order = m->order) < VM_NFREEORDER) { /* * The page is enqueued in the physical memory * allocator's free page queues. Moreover, it is the * first page in a power-of-two-sized run of * contiguous free pages. Add these pages to the end * of the current run, and jump ahead. */ run_ext = 1 << order; m_inc = 1 << order; } else { /* * Skip the page for one of the following reasons: (1) * It is enqueued in the physical memory allocator's * free page queues. However, it is not the first * page in a run of contiguous free pages. (This case * rarely occurs because the scan is performed in * ascending order.) (2) It is not reserved, and it is * transitioning from free to allocated. (Conversely, * the transition from allocated to free for managed * pages is blocked by the page lock.) (3) It is * allocated but not contained by an object and not * wired, e.g., allocated by Xen's balloon driver. */ run_ext = 0; } /* * Extend or reset the current run of pages. */ if (run_ext > 0) { if (run_len == 0) m_run = m; run_len += run_ext; } else { if (run_len > 0) { m_run = NULL; run_len = 0; } } } if (run_len >= npages) return (m_run); return (NULL); } /* * vm_page_reclaim_run: * * Try to relocate each of the allocated virtual pages within the * specified run of physical pages to a new physical address. Free the * physical pages underlying the relocated virtual pages. A virtual page * is relocatable if and only if it could be laundered or reclaimed by * the page daemon. Whenever possible, a virtual page is relocated to a * physical address above "high". * * Returns 0 if every physical page within the run was already free or * just freed by a successful relocation. Otherwise, returns a non-zero * value indicating why the last attempt to relocate a virtual page was * unsuccessful. * * "req_class" must be an allocation class. */ static int vm_page_reclaim_run(int req_class, int domain, u_long npages, vm_page_t m_run, vm_paddr_t high) { struct vm_domain *vmd; struct spglist free; vm_object_t object; vm_paddr_t pa; vm_page_t m, m_end, m_new; int error, order, req; KASSERT((req_class & VM_ALLOC_CLASS_MASK) == req_class, ("req_class is not an allocation class")); SLIST_INIT(&free); error = 0; m = m_run; m_end = m_run + npages; for (; error == 0 && m < m_end; m++) { KASSERT((m->flags & (PG_FICTITIOUS | PG_MARKER)) == 0, ("page %p is PG_FICTITIOUS or PG_MARKER", m)); /* * Racily check for wirings. Races are handled once the object * lock is held and the page is unmapped. */ if (vm_page_wired(m)) error = EBUSY; - else if ((object = - (vm_object_t)atomic_load_ptr(&m->object)) != NULL) { + else if ((object = atomic_load_ptr(&m->object)) != NULL) { /* * The page is relocated if and only if it could be * laundered or reclaimed by the page daemon. */ VM_OBJECT_WLOCK(object); /* Don't care: PG_NODUMP, PG_ZERO. */ if (m->object != object || (object->type != OBJT_DEFAULT && object->type != OBJT_SWAP && object->type != OBJT_VNODE)) error = EINVAL; else if (object->memattr != VM_MEMATTR_DEFAULT) error = EINVAL; else if (vm_page_queue(m) != PQ_NONE && vm_page_tryxbusy(m) != 0) { if (vm_page_wired(m)) { vm_page_xunbusy(m); error = EBUSY; goto unlock; } KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, ("page %p has an unexpected memattr", m)); KASSERT(m->oflags == 0, ("page %p has unexpected oflags", m)); /* Don't care: PGA_NOSYNC. */ if (!vm_page_none_valid(m)) { /* * First, try to allocate a new page * that is above "high". Failing * that, try to allocate a new page * that is below "m_run". Allocate * the new page between the end of * "m_run" and "high" only as a last * resort. */ req = req_class | VM_ALLOC_NOOBJ; if ((m->flags & PG_NODUMP) != 0) req |= VM_ALLOC_NODUMP; if (trunc_page(high) != ~(vm_paddr_t)PAGE_MASK) { m_new = vm_page_alloc_contig( NULL, 0, req, 1, round_page(high), ~(vm_paddr_t)0, PAGE_SIZE, 0, VM_MEMATTR_DEFAULT); } else m_new = NULL; if (m_new == NULL) { pa = VM_PAGE_TO_PHYS(m_run); m_new = vm_page_alloc_contig( NULL, 0, req, 1, 0, pa - 1, PAGE_SIZE, 0, VM_MEMATTR_DEFAULT); } if (m_new == NULL) { pa += ptoa(npages); m_new = vm_page_alloc_contig( NULL, 0, req, 1, pa, high, PAGE_SIZE, 0, VM_MEMATTR_DEFAULT); } if (m_new == NULL) { vm_page_xunbusy(m); error = ENOMEM; goto unlock; } /* * Unmap the page and check for new * wirings that may have been acquired * through a pmap lookup. */ if (object->ref_count != 0 && !vm_page_try_remove_all(m)) { vm_page_xunbusy(m); vm_page_free(m_new); error = EBUSY; goto unlock; } /* * Replace "m" with the new page. For * vm_page_replace(), "m" must be busy * and dequeued. Finally, change "m" * as if vm_page_free() was called. */ m_new->a.flags = m->a.flags & ~PGA_QUEUE_STATE_MASK; KASSERT(m_new->oflags == VPO_UNMANAGED, ("page %p is managed", m_new)); m_new->oflags = 0; pmap_copy_page(m, m_new); m_new->valid = m->valid; m_new->dirty = m->dirty; m->flags &= ~PG_ZERO; vm_page_dequeue(m); if (vm_page_replace_hold(m_new, object, m->pindex, m) && vm_page_free_prep(m)) SLIST_INSERT_HEAD(&free, m, plinks.s.ss); /* * The new page must be deactivated * before the object is unlocked. */ vm_page_deactivate(m_new); } else { m->flags &= ~PG_ZERO; vm_page_dequeue(m); if (vm_page_free_prep(m)) SLIST_INSERT_HEAD(&free, m, plinks.s.ss); KASSERT(m->dirty == 0, ("page %p is dirty", m)); } } else error = EBUSY; unlock: VM_OBJECT_WUNLOCK(object); } else { MPASS(vm_phys_domain(m) == domain); vmd = VM_DOMAIN(domain); vm_domain_free_lock(vmd); order = m->order; if (order < VM_NFREEORDER) { /* * The page is enqueued in the physical memory * allocator's free page queues. Moreover, it * is the first page in a power-of-two-sized * run of contiguous free pages. Jump ahead * to the last page within that run, and * continue from there. */ m += (1 << order) - 1; } #if VM_NRESERVLEVEL > 0 else if (vm_reserv_is_page_free(m)) order = 0; #endif vm_domain_free_unlock(vmd); if (order == VM_NFREEORDER) error = EINVAL; } } if ((m = SLIST_FIRST(&free)) != NULL) { int cnt; vmd = VM_DOMAIN(domain); cnt = 0; vm_domain_free_lock(vmd); do { MPASS(vm_phys_domain(m) == domain); SLIST_REMOVE_HEAD(&free, plinks.s.ss); vm_phys_free_pages(m, 0); cnt++; } while ((m = SLIST_FIRST(&free)) != NULL); vm_domain_free_unlock(vmd); vm_domain_freecnt_inc(vmd, cnt); } return (error); } #define NRUNS 16 CTASSERT(powerof2(NRUNS)); #define RUN_INDEX(count) ((count) & (NRUNS - 1)) #define MIN_RECLAIM 8 /* * vm_page_reclaim_contig: * * Reclaim allocated, contiguous physical memory satisfying the specified * conditions by relocating the virtual pages using that physical memory. * Returns true if reclamation is successful and false otherwise. Since * relocation requires the allocation of physical pages, reclamation may * fail due to a shortage of free pages. When reclamation fails, callers * are expected to perform vm_wait() before retrying a failed allocation * operation, e.g., vm_page_alloc_contig(). * * The caller must always specify an allocation class through "req". * * allocation classes: * VM_ALLOC_NORMAL normal process request * VM_ALLOC_SYSTEM system *really* needs a page * VM_ALLOC_INTERRUPT interrupt time request * * The optional allocation flags are ignored. * * "npages" must be greater than zero. Both "alignment" and "boundary" * must be a power of two. */ bool vm_page_reclaim_contig_domain(int domain, int req, u_long npages, vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary) { struct vm_domain *vmd; vm_paddr_t curr_low; vm_page_t m_run, m_runs[NRUNS]; u_long count, reclaimed; int error, i, options, req_class; KASSERT(npages > 0, ("npages is 0")); KASSERT(powerof2(alignment), ("alignment is not a power of 2")); KASSERT(powerof2(boundary), ("boundary is not a power of 2")); req_class = req & VM_ALLOC_CLASS_MASK; /* * The page daemon is allowed to dig deeper into the free page list. */ if (curproc == pageproc && req_class != VM_ALLOC_INTERRUPT) req_class = VM_ALLOC_SYSTEM; /* * Return if the number of free pages cannot satisfy the requested * allocation. */ vmd = VM_DOMAIN(domain); count = vmd->vmd_free_count; if (count < npages + vmd->vmd_free_reserved || (count < npages + vmd->vmd_interrupt_free_min && req_class == VM_ALLOC_SYSTEM) || (count < npages && req_class == VM_ALLOC_INTERRUPT)) return (false); /* * Scan up to three times, relaxing the restrictions ("options") on * the reclamation of reservations and superpages each time. */ for (options = VPSC_NORESERV;;) { /* * Find the highest runs that satisfy the given constraints * and restrictions, and record them in "m_runs". */ curr_low = low; count = 0; for (;;) { m_run = vm_phys_scan_contig(domain, npages, curr_low, high, alignment, boundary, options); if (m_run == NULL) break; curr_low = VM_PAGE_TO_PHYS(m_run) + ptoa(npages); m_runs[RUN_INDEX(count)] = m_run; count++; } /* * Reclaim the highest runs in LIFO (descending) order until * the number of reclaimed pages, "reclaimed", is at least * MIN_RECLAIM. Reset "reclaimed" each time because each * reclamation is idempotent, and runs will (likely) recur * from one scan to the next as restrictions are relaxed. */ reclaimed = 0; for (i = 0; count > 0 && i < NRUNS; i++) { count--; m_run = m_runs[RUN_INDEX(count)]; error = vm_page_reclaim_run(req_class, domain, npages, m_run, high); if (error == 0) { reclaimed += npages; if (reclaimed >= MIN_RECLAIM) return (true); } } /* * Either relax the restrictions on the next scan or return if * the last scan had no restrictions. */ if (options == VPSC_NORESERV) options = VPSC_NOSUPER; else if (options == VPSC_NOSUPER) options = VPSC_ANY; else if (options == VPSC_ANY) return (reclaimed != 0); } } bool vm_page_reclaim_contig(int req, u_long npages, vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary) { struct vm_domainset_iter di; int domain; bool ret; vm_domainset_iter_page_init(&di, NULL, 0, &domain, &req); do { ret = vm_page_reclaim_contig_domain(domain, req, npages, low, high, alignment, boundary); if (ret) break; } while (vm_domainset_iter_page(&di, NULL, &domain) == 0); return (ret); } /* * Set the domain in the appropriate page level domainset. */ void vm_domain_set(struct vm_domain *vmd) { mtx_lock(&vm_domainset_lock); if (!vmd->vmd_minset && vm_paging_min(vmd)) { vmd->vmd_minset = 1; DOMAINSET_SET(vmd->vmd_domain, &vm_min_domains); } if (!vmd->vmd_severeset && vm_paging_severe(vmd)) { vmd->vmd_severeset = 1; DOMAINSET_SET(vmd->vmd_domain, &vm_severe_domains); } mtx_unlock(&vm_domainset_lock); } /* * Clear the domain from the appropriate page level domainset. */ void vm_domain_clear(struct vm_domain *vmd) { mtx_lock(&vm_domainset_lock); if (vmd->vmd_minset && !vm_paging_min(vmd)) { vmd->vmd_minset = 0; DOMAINSET_CLR(vmd->vmd_domain, &vm_min_domains); if (vm_min_waiters != 0) { vm_min_waiters = 0; wakeup(&vm_min_domains); } } if (vmd->vmd_severeset && !vm_paging_severe(vmd)) { vmd->vmd_severeset = 0; DOMAINSET_CLR(vmd->vmd_domain, &vm_severe_domains); if (vm_severe_waiters != 0) { vm_severe_waiters = 0; wakeup(&vm_severe_domains); } } /* * If pageout daemon needs pages, then tell it that there are * some free. */ if (vmd->vmd_pageout_pages_needed && vmd->vmd_free_count >= vmd->vmd_pageout_free_min) { wakeup(&vmd->vmd_pageout_pages_needed); vmd->vmd_pageout_pages_needed = 0; } /* See comments in vm_wait_doms(). */ if (vm_pageproc_waiters) { vm_pageproc_waiters = 0; wakeup(&vm_pageproc_waiters); } mtx_unlock(&vm_domainset_lock); } /* * Wait for free pages to exceed the min threshold globally. */ void vm_wait_min(void) { mtx_lock(&vm_domainset_lock); while (vm_page_count_min()) { vm_min_waiters++; msleep(&vm_min_domains, &vm_domainset_lock, PVM, "vmwait", 0); } mtx_unlock(&vm_domainset_lock); } /* * Wait for free pages to exceed the severe threshold globally. */ void vm_wait_severe(void) { mtx_lock(&vm_domainset_lock); while (vm_page_count_severe()) { vm_severe_waiters++; msleep(&vm_severe_domains, &vm_domainset_lock, PVM, "vmwait", 0); } mtx_unlock(&vm_domainset_lock); } u_int vm_wait_count(void) { return (vm_severe_waiters + vm_min_waiters + vm_pageproc_waiters); } void vm_wait_doms(const domainset_t *wdoms) { /* * We use racey wakeup synchronization to avoid expensive global * locking for the pageproc when sleeping with a non-specific vm_wait. * To handle this, we only sleep for one tick in this instance. It * is expected that most allocations for the pageproc will come from * kmem or vm_page_grab* which will use the more specific and * race-free vm_wait_domain(). */ if (curproc == pageproc) { mtx_lock(&vm_domainset_lock); vm_pageproc_waiters++; msleep(&vm_pageproc_waiters, &vm_domainset_lock, PVM | PDROP, "pageprocwait", 1); } else { /* * XXX Ideally we would wait only until the allocation could * be satisfied. This condition can cause new allocators to * consume all freed pages while old allocators wait. */ mtx_lock(&vm_domainset_lock); if (vm_page_count_min_set(wdoms)) { vm_min_waiters++; msleep(&vm_min_domains, &vm_domainset_lock, PVM | PDROP, "vmwait", 0); } else mtx_unlock(&vm_domainset_lock); } } /* * vm_wait_domain: * * Sleep until free pages are available for allocation. * - Called in various places after failed memory allocations. */ void vm_wait_domain(int domain) { struct vm_domain *vmd; domainset_t wdom; vmd = VM_DOMAIN(domain); vm_domain_free_assert_unlocked(vmd); if (curproc == pageproc) { mtx_lock(&vm_domainset_lock); if (vmd->vmd_free_count < vmd->vmd_pageout_free_min) { vmd->vmd_pageout_pages_needed = 1; msleep(&vmd->vmd_pageout_pages_needed, &vm_domainset_lock, PDROP | PSWP, "VMWait", 0); } else mtx_unlock(&vm_domainset_lock); } else { if (pageproc == NULL) panic("vm_wait in early boot"); DOMAINSET_ZERO(&wdom); DOMAINSET_SET(vmd->vmd_domain, &wdom); vm_wait_doms(&wdom); } } /* * vm_wait: * * Sleep until free pages are available for allocation in the * affinity domains of the obj. If obj is NULL, the domain set * for the calling thread is used. * Called in various places after failed memory allocations. */ void vm_wait(vm_object_t obj) { struct domainset *d; d = NULL; /* * Carefully fetch pointers only once: the struct domainset * itself is ummutable but the pointer might change. */ if (obj != NULL) d = obj->domain.dr_policy; if (d == NULL) d = curthread->td_domain.dr_policy; vm_wait_doms(&d->ds_mask); } /* * vm_domain_alloc_fail: * * Called when a page allocation function fails. Informs the * pagedaemon and performs the requested wait. Requires the * domain_free and object lock on entry. Returns with the * object lock held and free lock released. Returns an error when * retry is necessary. * */ static int vm_domain_alloc_fail(struct vm_domain *vmd, vm_object_t object, int req) { vm_domain_free_assert_unlocked(vmd); atomic_add_int(&vmd->vmd_pageout_deficit, max((u_int)req >> VM_ALLOC_COUNT_SHIFT, 1)); if (req & (VM_ALLOC_WAITOK | VM_ALLOC_WAITFAIL)) { if (object != NULL) VM_OBJECT_WUNLOCK(object); vm_wait_domain(vmd->vmd_domain); if (object != NULL) VM_OBJECT_WLOCK(object); if (req & VM_ALLOC_WAITOK) return (EAGAIN); } return (0); } /* * vm_waitpfault: * * Sleep until free pages are available for allocation. * - Called only in vm_fault so that processes page faulting * can be easily tracked. * - Sleeps at a lower priority than vm_wait() so that vm_wait()ing * processes will be able to grab memory first. Do not change * this balance without careful testing first. */ void vm_waitpfault(struct domainset *dset, int timo) { /* * XXX Ideally we would wait only until the allocation could * be satisfied. This condition can cause new allocators to * consume all freed pages while old allocators wait. */ mtx_lock(&vm_domainset_lock); if (vm_page_count_min_set(&dset->ds_mask)) { vm_min_waiters++; msleep(&vm_min_domains, &vm_domainset_lock, PUSER | PDROP, "pfault", timo); } else mtx_unlock(&vm_domainset_lock); } static struct vm_pagequeue * _vm_page_pagequeue(vm_page_t m, uint8_t queue) { return (&vm_pagequeue_domain(m)->vmd_pagequeues[queue]); } #ifdef INVARIANTS static struct vm_pagequeue * vm_page_pagequeue(vm_page_t m) { return (_vm_page_pagequeue(m, vm_page_astate_load(m).queue)); } #endif static __always_inline bool vm_page_pqstate_fcmpset(vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new) { vm_page_astate_t tmp; tmp = *old; do { if (__predict_true(vm_page_astate_fcmpset(m, old, new))) return (true); counter_u64_add(pqstate_commit_retries, 1); } while (old->_bits == tmp._bits); return (false); } /* * Do the work of committing a queue state update that moves the page out of * its current queue. */ static bool _vm_page_pqstate_commit_dequeue(struct vm_pagequeue *pq, vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new) { vm_page_t next; vm_pagequeue_assert_locked(pq); KASSERT(vm_page_pagequeue(m) == pq, ("%s: queue %p does not match page %p", __func__, pq, m)); KASSERT(old->queue != PQ_NONE && new.queue != old->queue, ("%s: invalid queue indices %d %d", __func__, old->queue, new.queue)); /* * Once the queue index of the page changes there is nothing * synchronizing with further updates to the page's physical * queue state. Therefore we must speculatively remove the page * from the queue now and be prepared to roll back if the queue * state update fails. If the page is not physically enqueued then * we just update its queue index. */ if ((old->flags & PGA_ENQUEUED) != 0) { new.flags &= ~PGA_ENQUEUED; next = TAILQ_NEXT(m, plinks.q); TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); vm_pagequeue_cnt_dec(pq); if (!vm_page_pqstate_fcmpset(m, old, new)) { if (next == NULL) TAILQ_INSERT_TAIL(&pq->pq_pl, m, plinks.q); else TAILQ_INSERT_BEFORE(next, m, plinks.q); vm_pagequeue_cnt_inc(pq); return (false); } else { return (true); } } else { return (vm_page_pqstate_fcmpset(m, old, new)); } } static bool vm_page_pqstate_commit_dequeue(vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new) { struct vm_pagequeue *pq; vm_page_astate_t as; bool ret; pq = _vm_page_pagequeue(m, old->queue); /* * The queue field and PGA_ENQUEUED flag are stable only so long as the * corresponding page queue lock is held. */ vm_pagequeue_lock(pq); as = vm_page_astate_load(m); if (__predict_false(as._bits != old->_bits)) { *old = as; ret = false; } else { ret = _vm_page_pqstate_commit_dequeue(pq, m, old, new); } vm_pagequeue_unlock(pq); return (ret); } /* * Commit a queue state update that enqueues or requeues a page. */ static bool _vm_page_pqstate_commit_requeue(struct vm_pagequeue *pq, vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new) { struct vm_domain *vmd; vm_pagequeue_assert_locked(pq); KASSERT(old->queue != PQ_NONE && new.queue == old->queue, ("%s: invalid queue indices %d %d", __func__, old->queue, new.queue)); new.flags |= PGA_ENQUEUED; if (!vm_page_pqstate_fcmpset(m, old, new)) return (false); if ((old->flags & PGA_ENQUEUED) != 0) TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); else vm_pagequeue_cnt_inc(pq); /* * Give PGA_REQUEUE_HEAD precedence over PGA_REQUEUE. In particular, if * both flags are set in close succession, only PGA_REQUEUE_HEAD will be * applied, even if it was set first. */ if ((old->flags & PGA_REQUEUE_HEAD) != 0) { vmd = vm_pagequeue_domain(m); KASSERT(pq == &vmd->vmd_pagequeues[PQ_INACTIVE], ("%s: invalid page queue for page %p", __func__, m)); TAILQ_INSERT_BEFORE(&vmd->vmd_inacthead, m, plinks.q); } else { TAILQ_INSERT_TAIL(&pq->pq_pl, m, plinks.q); } return (true); } /* * Commit a queue state update that encodes a request for a deferred queue * operation. */ static bool vm_page_pqstate_commit_request(vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new) { KASSERT(old->queue == new.queue || new.queue != PQ_NONE, ("%s: invalid state, queue %d flags %x", __func__, new.queue, new.flags)); if (old->_bits != new._bits && !vm_page_pqstate_fcmpset(m, old, new)) return (false); vm_page_pqbatch_submit(m, new.queue); return (true); } /* * A generic queue state update function. This handles more cases than the * specialized functions above. */ bool vm_page_pqstate_commit(vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new) { if (old->_bits == new._bits) return (true); if (old->queue != PQ_NONE && new.queue != old->queue) { if (!vm_page_pqstate_commit_dequeue(m, old, new)) return (false); if (new.queue != PQ_NONE) vm_page_pqbatch_submit(m, new.queue); } else { if (!vm_page_pqstate_fcmpset(m, old, new)) return (false); if (new.queue != PQ_NONE && ((new.flags & ~old->flags) & PGA_QUEUE_OP_MASK) != 0) vm_page_pqbatch_submit(m, new.queue); } return (true); } /* * Apply deferred queue state updates to a page. */ static inline void vm_pqbatch_process_page(struct vm_pagequeue *pq, vm_page_t m, uint8_t queue) { vm_page_astate_t new, old; CRITICAL_ASSERT(curthread); vm_pagequeue_assert_locked(pq); KASSERT(queue < PQ_COUNT, ("%s: invalid queue index %d", __func__, queue)); KASSERT(pq == _vm_page_pagequeue(m, queue), ("%s: page %p does not belong to queue %p", __func__, m, pq)); for (old = vm_page_astate_load(m);;) { if (__predict_false(old.queue != queue || (old.flags & PGA_QUEUE_OP_MASK) == 0)) { counter_u64_add(queue_nops, 1); break; } KASSERT(old.queue != PQ_NONE || (old.flags & PGA_QUEUE_STATE_MASK) == 0, ("%s: page %p has unexpected queue state", __func__, m)); new = old; if ((old.flags & PGA_DEQUEUE) != 0) { new.flags &= ~PGA_QUEUE_OP_MASK; new.queue = PQ_NONE; if (__predict_true(_vm_page_pqstate_commit_dequeue(pq, m, &old, new))) { counter_u64_add(queue_ops, 1); break; } } else { new.flags &= ~(PGA_REQUEUE | PGA_REQUEUE_HEAD); if (__predict_true(_vm_page_pqstate_commit_requeue(pq, m, &old, new))) { counter_u64_add(queue_ops, 1); break; } } } } static void vm_pqbatch_process(struct vm_pagequeue *pq, struct vm_batchqueue *bq, uint8_t queue) { int i; for (i = 0; i < bq->bq_cnt; i++) vm_pqbatch_process_page(pq, bq->bq_pa[i], queue); vm_batchqueue_init(bq); } /* * vm_page_pqbatch_submit: [ internal use only ] * * Enqueue a page in the specified page queue's batched work queue. * The caller must have encoded the requested operation in the page * structure's a.flags field. */ void vm_page_pqbatch_submit(vm_page_t m, uint8_t queue) { struct vm_batchqueue *bq; struct vm_pagequeue *pq; int domain; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("page %p is unmanaged", m)); KASSERT(queue < PQ_COUNT, ("invalid queue %d", queue)); domain = vm_phys_domain(m); pq = &vm_pagequeue_domain(m)->vmd_pagequeues[queue]; critical_enter(); bq = DPCPU_PTR(pqbatch[domain][queue]); if (vm_batchqueue_insert(bq, m)) { critical_exit(); return; } critical_exit(); vm_pagequeue_lock(pq); critical_enter(); bq = DPCPU_PTR(pqbatch[domain][queue]); vm_pqbatch_process(pq, bq, queue); vm_pqbatch_process_page(pq, m, queue); vm_pagequeue_unlock(pq); critical_exit(); } /* * vm_page_pqbatch_drain: [ internal use only ] * * Force all per-CPU page queue batch queues to be drained. This is * intended for use in severe memory shortages, to ensure that pages * do not remain stuck in the batch queues. */ void vm_page_pqbatch_drain(void) { struct thread *td; struct vm_domain *vmd; struct vm_pagequeue *pq; int cpu, domain, queue; td = curthread; CPU_FOREACH(cpu) { thread_lock(td); sched_bind(td, cpu); thread_unlock(td); for (domain = 0; domain < vm_ndomains; domain++) { vmd = VM_DOMAIN(domain); for (queue = 0; queue < PQ_COUNT; queue++) { pq = &vmd->vmd_pagequeues[queue]; vm_pagequeue_lock(pq); critical_enter(); vm_pqbatch_process(pq, DPCPU_PTR(pqbatch[domain][queue]), queue); critical_exit(); vm_pagequeue_unlock(pq); } } } thread_lock(td); sched_unbind(td); thread_unlock(td); } /* * vm_page_dequeue_deferred: [ internal use only ] * * Request removal of the given page from its current page * queue. Physical removal from the queue may be deferred * indefinitely. * * The page must be locked. */ void vm_page_dequeue_deferred(vm_page_t m) { vm_page_astate_t new, old; old = vm_page_astate_load(m); do { if (old.queue == PQ_NONE) { KASSERT((old.flags & PGA_QUEUE_STATE_MASK) == 0, ("%s: page %p has unexpected queue state", __func__, m)); break; } new = old; new.flags |= PGA_DEQUEUE; } while (!vm_page_pqstate_commit_request(m, &old, new)); } /* * vm_page_dequeue: * * Remove the page from whichever page queue it's in, if any, before * returning. */ void vm_page_dequeue(vm_page_t m) { vm_page_astate_t new, old; old = vm_page_astate_load(m); do { if (old.queue == PQ_NONE) { KASSERT((old.flags & PGA_QUEUE_STATE_MASK) == 0, ("%s: page %p has unexpected queue state", __func__, m)); break; } new = old; new.flags &= ~PGA_QUEUE_OP_MASK; new.queue = PQ_NONE; } while (!vm_page_pqstate_commit_dequeue(m, &old, new)); } /* * Schedule the given page for insertion into the specified page queue. * Physical insertion of the page may be deferred indefinitely. */ static void vm_page_enqueue(vm_page_t m, uint8_t queue) { KASSERT(m->a.queue == PQ_NONE && (m->a.flags & PGA_QUEUE_STATE_MASK) == 0, ("%s: page %p is already enqueued", __func__, m)); KASSERT(m->ref_count > 0, ("%s: page %p does not carry any references", __func__, m)); m->a.queue = queue; if ((m->a.flags & PGA_REQUEUE) == 0) vm_page_aflag_set(m, PGA_REQUEUE); vm_page_pqbatch_submit(m, queue); } /* * vm_page_free_prep: * * Prepares the given page to be put on the free list, * disassociating it from any VM object. The caller may return * the page to the free list only if this function returns true. * * The object must be locked. The page must be locked if it is * managed. */ static bool vm_page_free_prep(vm_page_t m) { /* * Synchronize with threads that have dropped a reference to this * page. */ atomic_thread_fence_acq(); #if defined(DIAGNOSTIC) && defined(PHYS_TO_DMAP) if (PMAP_HAS_DMAP && (m->flags & PG_ZERO) != 0) { uint64_t *p; int i; p = (uint64_t *)PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m)); for (i = 0; i < PAGE_SIZE / sizeof(uint64_t); i++, p++) KASSERT(*p == 0, ("vm_page_free_prep %p PG_ZERO %d %jx", m, i, (uintmax_t)*p)); } #endif if ((m->oflags & VPO_UNMANAGED) == 0) { KASSERT(!pmap_page_is_mapped(m), ("vm_page_free_prep: freeing mapped page %p", m)); KASSERT((m->a.flags & (PGA_EXECUTABLE | PGA_WRITEABLE)) == 0, ("vm_page_free_prep: mapping flags set in page %p", m)); } else { KASSERT(m->a.queue == PQ_NONE, ("vm_page_free_prep: unmanaged page %p is queued", m)); } VM_CNT_INC(v_tfree); if (m->object != NULL) { KASSERT(((m->oflags & VPO_UNMANAGED) != 0) == ((m->object->flags & OBJ_UNMANAGED) != 0), ("vm_page_free_prep: managed flag mismatch for page %p", m)); vm_page_assert_xbusied(m); /* * The object reference can be released without an atomic * operation. */ KASSERT((m->flags & PG_FICTITIOUS) != 0 || m->ref_count == VPRC_OBJREF, ("vm_page_free_prep: page %p has unexpected ref_count %u", m, m->ref_count)); vm_page_object_remove(m); m->object = NULL; m->ref_count -= VPRC_OBJREF; } else vm_page_assert_unbusied(m); vm_page_busy_free(m); /* * If fictitious remove object association and * return. */ if ((m->flags & PG_FICTITIOUS) != 0) { KASSERT(m->ref_count == 1, ("fictitious page %p is referenced", m)); KASSERT(m->a.queue == PQ_NONE, ("fictitious page %p is queued", m)); return (false); } /* * Pages need not be dequeued before they are returned to the physical * memory allocator, but they must at least be marked for a deferred * dequeue. */ if ((m->oflags & VPO_UNMANAGED) == 0) vm_page_dequeue_deferred(m); m->valid = 0; vm_page_undirty(m); if (m->ref_count != 0) panic("vm_page_free_prep: page %p has references", m); /* * Restore the default memory attribute to the page. */ if (pmap_page_get_memattr(m) != VM_MEMATTR_DEFAULT) pmap_page_set_memattr(m, VM_MEMATTR_DEFAULT); #if VM_NRESERVLEVEL > 0 /* * Determine whether the page belongs to a reservation. If the page was * allocated from a per-CPU cache, it cannot belong to a reservation, so * as an optimization, we avoid the check in that case. */ if ((m->flags & PG_PCPU_CACHE) == 0 && vm_reserv_free_page(m)) return (false); #endif return (true); } /* * vm_page_free_toq: * * Returns the given page to the free list, disassociating it * from any VM object. * * The object must be locked. The page must be locked if it is * managed. */ static void vm_page_free_toq(vm_page_t m) { struct vm_domain *vmd; uma_zone_t zone; if (!vm_page_free_prep(m)) return; vmd = vm_pagequeue_domain(m); zone = vmd->vmd_pgcache[m->pool].zone; if ((m->flags & PG_PCPU_CACHE) != 0 && zone != NULL) { uma_zfree(zone, m); return; } vm_domain_free_lock(vmd); vm_phys_free_pages(m, 0); vm_domain_free_unlock(vmd); vm_domain_freecnt_inc(vmd, 1); } /* * vm_page_free_pages_toq: * * Returns a list of pages to the free list, disassociating it * from any VM object. In other words, this is equivalent to * calling vm_page_free_toq() for each page of a list of VM objects. * * The objects must be locked. The pages must be locked if it is * managed. */ void vm_page_free_pages_toq(struct spglist *free, bool update_wire_count) { vm_page_t m; int count; if (SLIST_EMPTY(free)) return; count = 0; while ((m = SLIST_FIRST(free)) != NULL) { count++; SLIST_REMOVE_HEAD(free, plinks.s.ss); vm_page_free_toq(m); } if (update_wire_count) vm_wire_sub(count); } /* * Mark this page as wired down, preventing reclamation by the page daemon * or when the containing object is destroyed. */ void vm_page_wire(vm_page_t m) { u_int old; KASSERT(m->object != NULL, ("vm_page_wire: page %p does not belong to an object", m)); if (!vm_page_busied(m) && !vm_object_busied(m->object)) VM_OBJECT_ASSERT_LOCKED(m->object); KASSERT((m->flags & PG_FICTITIOUS) == 0 || VPRC_WIRE_COUNT(m->ref_count) >= 1, ("vm_page_wire: fictitious page %p has zero wirings", m)); old = atomic_fetchadd_int(&m->ref_count, 1); KASSERT(VPRC_WIRE_COUNT(old) != VPRC_WIRE_COUNT_MAX, ("vm_page_wire: counter overflow for page %p", m)); if (VPRC_WIRE_COUNT(old) == 0) { if ((m->oflags & VPO_UNMANAGED) == 0) vm_page_aflag_set(m, PGA_DEQUEUE); vm_wire_add(1); } } /* * Attempt to wire a mapped page following a pmap lookup of that page. * This may fail if a thread is concurrently tearing down mappings of the page. * The transient failure is acceptable because it translates to the * failure of the caller pmap_extract_and_hold(), which should be then * followed by the vm_fault() fallback, see e.g. vm_fault_quick_hold_pages(). */ bool vm_page_wire_mapped(vm_page_t m) { u_int old; old = m->ref_count; do { KASSERT(old > 0, ("vm_page_wire_mapped: wiring unreferenced page %p", m)); if ((old & VPRC_BLOCKED) != 0) return (false); } while (!atomic_fcmpset_int(&m->ref_count, &old, old + 1)); if (VPRC_WIRE_COUNT(old) == 0) { if ((m->oflags & VPO_UNMANAGED) == 0) vm_page_aflag_set(m, PGA_DEQUEUE); vm_wire_add(1); } return (true); } /* * Release a wiring reference to a managed page. If the page still belongs to * an object, update its position in the page queues to reflect the reference. * If the wiring was the last reference to the page, free the page. */ static void vm_page_unwire_managed(vm_page_t m, uint8_t nqueue, bool noreuse) { u_int old; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("%s: page %p is unmanaged", __func__, m)); /* * Update LRU state before releasing the wiring reference. * Use a release store when updating the reference count to * synchronize with vm_page_free_prep(). */ old = m->ref_count; do { KASSERT(VPRC_WIRE_COUNT(old) > 0, ("vm_page_unwire: wire count underflow for page %p", m)); if (old > VPRC_OBJREF + 1) { /* * The page has at least one other wiring reference. An * earlier iteration of this loop may have called * vm_page_release_toq() and cleared PGA_DEQUEUE, so * re-set it if necessary. */ if ((vm_page_astate_load(m).flags & PGA_DEQUEUE) == 0) vm_page_aflag_set(m, PGA_DEQUEUE); } else if (old == VPRC_OBJREF + 1) { /* * This is the last wiring. Clear PGA_DEQUEUE and * update the page's queue state to reflect the * reference. If the page does not belong to an object * (i.e., the VPRC_OBJREF bit is clear), we only need to * clear leftover queue state. */ vm_page_release_toq(m, nqueue, false); } else if (old == 1) { vm_page_aflag_clear(m, PGA_DEQUEUE); } } while (!atomic_fcmpset_rel_int(&m->ref_count, &old, old - 1)); if (VPRC_WIRE_COUNT(old) == 1) { vm_wire_sub(1); if (old == 1) vm_page_free(m); } } /* * Release one wiring of the specified page, potentially allowing it to be * paged out. * * Only managed pages belonging to an object can be paged out. If the number * of wirings transitions to zero and the page is eligible for page out, then * the page is added to the specified paging queue. If the released wiring * represented the last reference to the page, the page is freed. * * A managed page must be locked. */ void vm_page_unwire(vm_page_t m, uint8_t nqueue) { KASSERT(nqueue < PQ_COUNT, ("vm_page_unwire: invalid queue %u request for page %p", nqueue, m)); if ((m->oflags & VPO_UNMANAGED) != 0) { if (vm_page_unwire_noq(m) && m->ref_count == 0) vm_page_free(m); return; } vm_page_unwire_managed(m, nqueue, false); } /* * Unwire a page without (re-)inserting it into a page queue. It is up * to the caller to enqueue, requeue, or free the page as appropriate. * In most cases involving managed pages, vm_page_unwire() should be used * instead. */ bool vm_page_unwire_noq(vm_page_t m) { u_int old; old = vm_page_drop(m, 1); KASSERT(VPRC_WIRE_COUNT(old) != 0, ("vm_page_unref: counter underflow for page %p", m)); KASSERT((m->flags & PG_FICTITIOUS) == 0 || VPRC_WIRE_COUNT(old) > 1, ("vm_page_unref: missing ref on fictitious page %p", m)); if (VPRC_WIRE_COUNT(old) > 1) return (false); if ((m->oflags & VPO_UNMANAGED) == 0) vm_page_aflag_clear(m, PGA_DEQUEUE); vm_wire_sub(1); return (true); } /* * Ensure that the page ends up in the specified page queue. If the page is * active or being moved to the active queue, ensure that its act_count is * at least ACT_INIT but do not otherwise mess with it. * * A managed page must be locked. */ static __always_inline void vm_page_mvqueue(vm_page_t m, const uint8_t nqueue, const uint16_t nflag) { vm_page_astate_t old, new; KASSERT(m->ref_count > 0, ("%s: page %p does not carry any references", __func__, m)); KASSERT(nflag == PGA_REQUEUE || nflag == PGA_REQUEUE_HEAD, ("%s: invalid flags %x", __func__, nflag)); if ((m->oflags & VPO_UNMANAGED) != 0 || vm_page_wired(m)) return; old = vm_page_astate_load(m); do { if ((old.flags & PGA_DEQUEUE) != 0) break; new = old; new.flags &= ~PGA_QUEUE_OP_MASK; if (nqueue == PQ_ACTIVE) new.act_count = max(old.act_count, ACT_INIT); if (old.queue == nqueue) { if (nqueue != PQ_ACTIVE) new.flags |= nflag; } else { new.flags |= nflag; new.queue = nqueue; } } while (!vm_page_pqstate_commit(m, &old, new)); } /* * Put the specified page on the active list (if appropriate). */ void vm_page_activate(vm_page_t m) { vm_page_mvqueue(m, PQ_ACTIVE, PGA_REQUEUE); } /* * Move the specified page to the tail of the inactive queue, or requeue * the page if it is already in the inactive queue. */ void vm_page_deactivate(vm_page_t m) { vm_page_mvqueue(m, PQ_INACTIVE, PGA_REQUEUE); } void vm_page_deactivate_noreuse(vm_page_t m) { vm_page_mvqueue(m, PQ_INACTIVE, PGA_REQUEUE_HEAD); } /* * Put a page in the laundry, or requeue it if it is already there. */ void vm_page_launder(vm_page_t m) { vm_page_mvqueue(m, PQ_LAUNDRY, PGA_REQUEUE); } /* * Put a page in the PQ_UNSWAPPABLE holding queue. */ void vm_page_unswappable(vm_page_t m) { KASSERT(!vm_page_wired(m) && (m->oflags & VPO_UNMANAGED) == 0, ("page %p already unswappable", m)); vm_page_dequeue(m); vm_page_enqueue(m, PQ_UNSWAPPABLE); } /* * Release a page back to the page queues in preparation for unwiring. */ static void vm_page_release_toq(vm_page_t m, uint8_t nqueue, const bool noreuse) { vm_page_astate_t old, new; uint16_t nflag; /* * Use a check of the valid bits to determine whether we should * accelerate reclamation of the page. The object lock might not be * held here, in which case the check is racy. At worst we will either * accelerate reclamation of a valid page and violate LRU, or * unnecessarily defer reclamation of an invalid page. * * If we were asked to not cache the page, place it near the head of the * inactive queue so that is reclaimed sooner. */ if (noreuse || m->valid == 0) { nqueue = PQ_INACTIVE; nflag = PGA_REQUEUE_HEAD; } else { nflag = PGA_REQUEUE; } old = vm_page_astate_load(m); do { new = old; /* * If the page is already in the active queue and we are not * trying to accelerate reclamation, simply mark it as * referenced and avoid any queue operations. */ new.flags &= ~PGA_QUEUE_OP_MASK; if (nflag != PGA_REQUEUE_HEAD && old.queue == PQ_ACTIVE) new.flags |= PGA_REFERENCED; else { new.flags |= nflag; new.queue = nqueue; } } while (!vm_page_pqstate_commit(m, &old, new)); } /* * Unwire a page and either attempt to free it or re-add it to the page queues. */ void vm_page_release(vm_page_t m, int flags) { vm_object_t object; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("vm_page_release: page %p is unmanaged", m)); if ((flags & VPR_TRYFREE) != 0) { for (;;) { - object = (vm_object_t)atomic_load_ptr(&m->object); + object = atomic_load_ptr(&m->object); if (object == NULL) break; /* Depends on type-stability. */ if (vm_page_busied(m) || !VM_OBJECT_TRYWLOCK(object)) break; if (object == m->object) { vm_page_release_locked(m, flags); VM_OBJECT_WUNLOCK(object); return; } VM_OBJECT_WUNLOCK(object); } } vm_page_unwire_managed(m, PQ_INACTIVE, flags != 0); } /* See vm_page_release(). */ void vm_page_release_locked(vm_page_t m, int flags) { VM_OBJECT_ASSERT_WLOCKED(m->object); KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("vm_page_release_locked: page %p is unmanaged", m)); if (vm_page_unwire_noq(m)) { if ((flags & VPR_TRYFREE) != 0 && (m->object->ref_count == 0 || !pmap_page_is_mapped(m)) && m->dirty == 0 && vm_page_tryxbusy(m)) { vm_page_free(m); } else { vm_page_release_toq(m, PQ_INACTIVE, flags != 0); } } } static bool vm_page_try_blocked_op(vm_page_t m, void (*op)(vm_page_t)) { u_int old; KASSERT(m->object != NULL && (m->oflags & VPO_UNMANAGED) == 0, ("vm_page_try_blocked_op: page %p has no object", m)); KASSERT(vm_page_busied(m), ("vm_page_try_blocked_op: page %p is not busy", m)); VM_OBJECT_ASSERT_LOCKED(m->object); old = m->ref_count; do { KASSERT(old != 0, ("vm_page_try_blocked_op: page %p has no references", m)); if (VPRC_WIRE_COUNT(old) != 0) return (false); } while (!atomic_fcmpset_int(&m->ref_count, &old, old | VPRC_BLOCKED)); (op)(m); /* * If the object is read-locked, new wirings may be created via an * object lookup. */ old = vm_page_drop(m, VPRC_BLOCKED); KASSERT(!VM_OBJECT_WOWNED(m->object) || old == (VPRC_BLOCKED | VPRC_OBJREF), ("vm_page_try_blocked_op: unexpected refcount value %u for %p", old, m)); return (true); } /* * Atomically check for wirings and remove all mappings of the page. */ bool vm_page_try_remove_all(vm_page_t m) { return (vm_page_try_blocked_op(m, pmap_remove_all)); } /* * Atomically check for wirings and remove all writeable mappings of the page. */ bool vm_page_try_remove_write(vm_page_t m) { return (vm_page_try_blocked_op(m, pmap_remove_write)); } /* * vm_page_advise * * Apply the specified advice to the given page. * * The object and page must be locked. */ void vm_page_advise(vm_page_t m, int advice) { VM_OBJECT_ASSERT_WLOCKED(m->object); if (advice == MADV_FREE) /* * Mark the page clean. This will allow the page to be freed * without first paging it out. MADV_FREE pages are often * quickly reused by malloc(3), so we do not do anything that * would result in a page fault on a later access. */ vm_page_undirty(m); else if (advice != MADV_DONTNEED) { if (advice == MADV_WILLNEED) vm_page_activate(m); return; } if (advice != MADV_FREE && m->dirty == 0 && pmap_is_modified(m)) vm_page_dirty(m); /* * Clear any references to the page. Otherwise, the page daemon will * immediately reactivate the page. */ vm_page_aflag_clear(m, PGA_REFERENCED); /* * Place clean pages near the head of the inactive queue rather than * the tail, thus defeating the queue's LRU operation and ensuring that * the page will be reused quickly. Dirty pages not already in the * laundry are moved there. */ if (m->dirty == 0) vm_page_deactivate_noreuse(m); else if (!vm_page_in_laundry(m)) vm_page_launder(m); } static inline int vm_page_grab_pflags(int allocflags) { int pflags; KASSERT((allocflags & VM_ALLOC_NOBUSY) == 0 || (allocflags & VM_ALLOC_WIRED) != 0, ("vm_page_grab_pflags: the pages must be busied or wired")); KASSERT((allocflags & VM_ALLOC_SBUSY) == 0 || (allocflags & VM_ALLOC_IGN_SBUSY) != 0, ("vm_page_grab_pflags: VM_ALLOC_SBUSY/VM_ALLOC_IGN_SBUSY " "mismatch")); pflags = allocflags & ~(VM_ALLOC_NOWAIT | VM_ALLOC_WAITOK | VM_ALLOC_WAITFAIL | VM_ALLOC_NOBUSY); if ((allocflags & VM_ALLOC_NOWAIT) == 0) pflags |= VM_ALLOC_WAITFAIL; if ((allocflags & VM_ALLOC_IGN_SBUSY) != 0) pflags |= VM_ALLOC_SBUSY; return (pflags); } /* * Grab a page, waiting until we are waken up due to the page * changing state. We keep on waiting, if the page continues * to be in the object. If the page doesn't exist, first allocate it * and then conditionally zero it. * * This routine may sleep. * * The object must be locked on entry. The lock will, however, be released * and reacquired if the routine sleeps. */ vm_page_t vm_page_grab(vm_object_t object, vm_pindex_t pindex, int allocflags) { vm_page_t m; int pflags; VM_OBJECT_ASSERT_WLOCKED(object); pflags = vm_page_grab_pflags(allocflags); retrylookup: if ((m = vm_page_lookup(object, pindex)) != NULL) { if (!vm_page_acquire_flags(m, allocflags)) { if (vm_page_busy_sleep_flags(object, m, "pgrbwt", allocflags)) goto retrylookup; return (NULL); } goto out; } if ((allocflags & VM_ALLOC_NOCREAT) != 0) return (NULL); m = vm_page_alloc(object, pindex, pflags); if (m == NULL) { if ((allocflags & (VM_ALLOC_NOWAIT | VM_ALLOC_WAITFAIL)) != 0) return (NULL); goto retrylookup; } if (allocflags & VM_ALLOC_ZERO && (m->flags & PG_ZERO) == 0) pmap_zero_page(m); out: if ((allocflags & VM_ALLOC_NOBUSY) != 0) { if ((allocflags & VM_ALLOC_IGN_SBUSY) != 0) vm_page_sunbusy(m); else vm_page_xunbusy(m); } return (m); } /* * Grab a page and make it valid, paging in if necessary. Pages missing from * their pager are zero filled and validated. If a VM_ALLOC_COUNT is supplied * and the page is not valid as many as VM_INITIAL_PAGEIN pages can be brought * in simultaneously. Additional pages will be left on a paging queue but * will neither be wired nor busy regardless of allocflags. */ int vm_page_grab_valid(vm_page_t *mp, vm_object_t object, vm_pindex_t pindex, int allocflags) { vm_page_t m; vm_page_t ma[VM_INITIAL_PAGEIN]; bool sleep, xbusy; int after, i, pflags, rv; KASSERT((allocflags & VM_ALLOC_SBUSY) == 0 || (allocflags & VM_ALLOC_IGN_SBUSY) != 0, ("vm_page_grab_valid: VM_ALLOC_SBUSY/VM_ALLOC_IGN_SBUSY mismatch")); KASSERT((allocflags & (VM_ALLOC_NOWAIT | VM_ALLOC_WAITFAIL | VM_ALLOC_ZERO)) == 0, ("vm_page_grab_valid: Invalid flags 0x%X", allocflags)); VM_OBJECT_ASSERT_WLOCKED(object); pflags = allocflags & ~(VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY); pflags |= VM_ALLOC_WAITFAIL; retrylookup: xbusy = false; if ((m = vm_page_lookup(object, pindex)) != NULL) { /* * If the page is fully valid it can only become invalid * with the object lock held. If it is not valid it can * become valid with the busy lock held. Therefore, we * may unnecessarily lock the exclusive busy here if we * race with I/O completion not using the object lock. * However, we will not end up with an invalid page and a * shared lock. */ if (!vm_page_all_valid(m) || (allocflags & (VM_ALLOC_IGN_SBUSY | VM_ALLOC_SBUSY)) == 0) { sleep = !vm_page_tryxbusy(m); xbusy = true; } else sleep = !vm_page_trysbusy(m); if (sleep) { (void)vm_page_busy_sleep_flags(object, m, "pgrbwt", allocflags); goto retrylookup; } if ((allocflags & VM_ALLOC_NOCREAT) != 0 && !vm_page_all_valid(m)) { if (xbusy) vm_page_xunbusy(m); else vm_page_sunbusy(m); *mp = NULL; return (VM_PAGER_FAIL); } if ((allocflags & VM_ALLOC_WIRED) != 0) vm_page_wire(m); if (vm_page_all_valid(m)) goto out; } else if ((allocflags & VM_ALLOC_NOCREAT) != 0) { *mp = NULL; return (VM_PAGER_FAIL); } else if ((m = vm_page_alloc(object, pindex, pflags)) != NULL) { xbusy = true; } else { goto retrylookup; } vm_page_assert_xbusied(m); MPASS(xbusy); if (vm_pager_has_page(object, pindex, NULL, &after)) { after = MIN(after, VM_INITIAL_PAGEIN); after = MIN(after, allocflags >> VM_ALLOC_COUNT_SHIFT); after = MAX(after, 1); ma[0] = m; for (i = 1; i < after; i++) { if ((ma[i] = vm_page_next(ma[i - 1])) != NULL) { if (ma[i]->valid || !vm_page_tryxbusy(ma[i])) break; } else { ma[i] = vm_page_alloc(object, m->pindex + i, VM_ALLOC_NORMAL); if (ma[i] == NULL) break; } } after = i; vm_object_pip_add(object, after); VM_OBJECT_WUNLOCK(object); rv = vm_pager_get_pages(object, ma, after, NULL, NULL); VM_OBJECT_WLOCK(object); vm_object_pip_wakeupn(object, after); /* Pager may have replaced a page. */ m = ma[0]; if (rv != VM_PAGER_OK) { if ((allocflags & VM_ALLOC_WIRED) != 0) vm_page_unwire_noq(m); for (i = 0; i < after; i++) { if (!vm_page_wired(ma[i])) vm_page_free(ma[i]); else vm_page_xunbusy(ma[i]); } *mp = NULL; return (rv); } for (i = 1; i < after; i++) vm_page_readahead_finish(ma[i]); MPASS(vm_page_all_valid(m)); } else { vm_page_zero_invalid(m, TRUE); } out: if ((allocflags & VM_ALLOC_NOBUSY) != 0) { if (xbusy) vm_page_xunbusy(m); else vm_page_sunbusy(m); } if ((allocflags & VM_ALLOC_SBUSY) != 0 && xbusy) vm_page_busy_downgrade(m); *mp = m; return (VM_PAGER_OK); } /* * Return the specified range of pages from the given object. For each * page offset within the range, if a page already exists within the object * at that offset and it is busy, then wait for it to change state. If, * instead, the page doesn't exist, then allocate it. * * The caller must always specify an allocation class. * * allocation classes: * VM_ALLOC_NORMAL normal process request * VM_ALLOC_SYSTEM system *really* needs the pages * * The caller must always specify that the pages are to be busied and/or * wired. * * optional allocation flags: * VM_ALLOC_IGN_SBUSY do not sleep on soft busy pages * VM_ALLOC_NOBUSY do not exclusive busy the page * VM_ALLOC_NOWAIT do not sleep * VM_ALLOC_SBUSY set page to sbusy state * VM_ALLOC_WIRED wire the pages * VM_ALLOC_ZERO zero and validate any invalid pages * * If VM_ALLOC_NOWAIT is not specified, this routine may sleep. Otherwise, it * may return a partial prefix of the requested range. */ int vm_page_grab_pages(vm_object_t object, vm_pindex_t pindex, int allocflags, vm_page_t *ma, int count) { vm_page_t m, mpred; int pflags; int i; VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(((u_int)allocflags >> VM_ALLOC_COUNT_SHIFT) == 0, ("vm_page_grap_pages: VM_ALLOC_COUNT() is not allowed")); pflags = vm_page_grab_pflags(allocflags); if (count == 0) return (0); i = 0; retrylookup: m = vm_radix_lookup_le(&object->rtree, pindex + i); if (m == NULL || m->pindex != pindex + i) { mpred = m; m = NULL; } else mpred = TAILQ_PREV(m, pglist, listq); for (; i < count; i++) { if (m != NULL) { if (!vm_page_acquire_flags(m, allocflags)) { if (vm_page_busy_sleep_flags(object, m, "grbmaw", allocflags)) goto retrylookup; break; } } else { if ((allocflags & VM_ALLOC_NOCREAT) != 0) break; m = vm_page_alloc_after(object, pindex + i, pflags | VM_ALLOC_COUNT(count - i), mpred); if (m == NULL) { if ((allocflags & (VM_ALLOC_NOWAIT | VM_ALLOC_WAITFAIL)) != 0) break; goto retrylookup; } } if (vm_page_none_valid(m) && (allocflags & VM_ALLOC_ZERO) != 0) { if ((m->flags & PG_ZERO) == 0) pmap_zero_page(m); vm_page_valid(m); } if ((allocflags & VM_ALLOC_NOBUSY) != 0) { if ((allocflags & VM_ALLOC_IGN_SBUSY) != 0) vm_page_sunbusy(m); else vm_page_xunbusy(m); } ma[i] = mpred = m; m = vm_page_next(m); } return (i); } /* * Mapping function for valid or dirty bits in a page. * * Inputs are required to range within a page. */ vm_page_bits_t vm_page_bits(int base, int size) { int first_bit; int last_bit; KASSERT( base + size <= PAGE_SIZE, ("vm_page_bits: illegal base/size %d/%d", base, size) ); if (size == 0) /* handle degenerate case */ return (0); first_bit = base >> DEV_BSHIFT; last_bit = (base + size - 1) >> DEV_BSHIFT; return (((vm_page_bits_t)2 << last_bit) - ((vm_page_bits_t)1 << first_bit)); } void vm_page_bits_set(vm_page_t m, vm_page_bits_t *bits, vm_page_bits_t set) { #if PAGE_SIZE == 32768 atomic_set_64((uint64_t *)bits, set); #elif PAGE_SIZE == 16384 atomic_set_32((uint32_t *)bits, set); #elif (PAGE_SIZE == 8192) && defined(atomic_set_16) atomic_set_16((uint16_t *)bits, set); #elif (PAGE_SIZE == 4096) && defined(atomic_set_8) atomic_set_8((uint8_t *)bits, set); #else /* PAGE_SIZE <= 8192 */ uintptr_t addr; int shift; addr = (uintptr_t)bits; /* * Use a trick to perform a 32-bit atomic on the * containing aligned word, to not depend on the existence * of atomic_{set, clear}_{8, 16}. */ shift = addr & (sizeof(uint32_t) - 1); #if BYTE_ORDER == BIG_ENDIAN shift = (sizeof(uint32_t) - sizeof(vm_page_bits_t) - shift) * NBBY; #else shift *= NBBY; #endif addr &= ~(sizeof(uint32_t) - 1); atomic_set_32((uint32_t *)addr, set << shift); #endif /* PAGE_SIZE */ } static inline void vm_page_bits_clear(vm_page_t m, vm_page_bits_t *bits, vm_page_bits_t clear) { #if PAGE_SIZE == 32768 atomic_clear_64((uint64_t *)bits, clear); #elif PAGE_SIZE == 16384 atomic_clear_32((uint32_t *)bits, clear); #elif (PAGE_SIZE == 8192) && defined(atomic_clear_16) atomic_clear_16((uint16_t *)bits, clear); #elif (PAGE_SIZE == 4096) && defined(atomic_clear_8) atomic_clear_8((uint8_t *)bits, clear); #else /* PAGE_SIZE <= 8192 */ uintptr_t addr; int shift; addr = (uintptr_t)bits; /* * Use a trick to perform a 32-bit atomic on the * containing aligned word, to not depend on the existence * of atomic_{set, clear}_{8, 16}. */ shift = addr & (sizeof(uint32_t) - 1); #if BYTE_ORDER == BIG_ENDIAN shift = (sizeof(uint32_t) - sizeof(vm_page_bits_t) - shift) * NBBY; #else shift *= NBBY; #endif addr &= ~(sizeof(uint32_t) - 1); atomic_clear_32((uint32_t *)addr, clear << shift); #endif /* PAGE_SIZE */ } static inline vm_page_bits_t vm_page_bits_swap(vm_page_t m, vm_page_bits_t *bits, vm_page_bits_t newbits) { #if PAGE_SIZE == 32768 uint64_t old; old = *bits; while (atomic_fcmpset_64(bits, &old, newbits) == 0); return (old); #elif PAGE_SIZE == 16384 uint32_t old; old = *bits; while (atomic_fcmpset_32(bits, &old, newbits) == 0); return (old); #elif (PAGE_SIZE == 8192) && defined(atomic_fcmpset_16) uint16_t old; old = *bits; while (atomic_fcmpset_16(bits, &old, newbits) == 0); return (old); #elif (PAGE_SIZE == 4096) && defined(atomic_fcmpset_8) uint8_t old; old = *bits; while (atomic_fcmpset_8(bits, &old, newbits) == 0); return (old); #else /* PAGE_SIZE <= 4096*/ uintptr_t addr; uint32_t old, new, mask; int shift; addr = (uintptr_t)bits; /* * Use a trick to perform a 32-bit atomic on the * containing aligned word, to not depend on the existence * of atomic_{set, swap, clear}_{8, 16}. */ shift = addr & (sizeof(uint32_t) - 1); #if BYTE_ORDER == BIG_ENDIAN shift = (sizeof(uint32_t) - sizeof(vm_page_bits_t) - shift) * NBBY; #else shift *= NBBY; #endif addr &= ~(sizeof(uint32_t) - 1); mask = VM_PAGE_BITS_ALL << shift; old = *bits; do { new = old & ~mask; new |= newbits << shift; } while (atomic_fcmpset_32((uint32_t *)addr, &old, new) == 0); return (old >> shift); #endif /* PAGE_SIZE */ } /* * vm_page_set_valid_range: * * Sets portions of a page valid. The arguments are expected * to be DEV_BSIZE aligned but if they aren't the bitmap is inclusive * of any partial chunks touched by the range. The invalid portion of * such chunks will be zeroed. * * (base + size) must be less then or equal to PAGE_SIZE. */ void vm_page_set_valid_range(vm_page_t m, int base, int size) { int endoff, frag; vm_page_bits_t pagebits; vm_page_assert_busied(m); if (size == 0) /* handle degenerate case */ return; /* * If the base is not DEV_BSIZE aligned and the valid * bit is clear, we have to zero out a portion of the * first block. */ if ((frag = rounddown2(base, DEV_BSIZE)) != base && (m->valid & (1 << (base >> DEV_BSHIFT))) == 0) pmap_zero_page_area(m, frag, base - frag); /* * If the ending offset is not DEV_BSIZE aligned and the * valid bit is clear, we have to zero out a portion of * the last block. */ endoff = base + size; if ((frag = rounddown2(endoff, DEV_BSIZE)) != endoff && (m->valid & (1 << (endoff >> DEV_BSHIFT))) == 0) pmap_zero_page_area(m, endoff, DEV_BSIZE - (endoff & (DEV_BSIZE - 1))); /* * Assert that no previously invalid block that is now being validated * is already dirty. */ KASSERT((~m->valid & vm_page_bits(base, size) & m->dirty) == 0, ("vm_page_set_valid_range: page %p is dirty", m)); /* * Set valid bits inclusive of any overlap. */ pagebits = vm_page_bits(base, size); if (vm_page_xbusied(m)) m->valid |= pagebits; else vm_page_bits_set(m, &m->valid, pagebits); } /* * Set the page dirty bits and free the invalid swap space if * present. Returns the previous dirty bits. */ vm_page_bits_t vm_page_set_dirty(vm_page_t m) { vm_page_bits_t old; VM_PAGE_OBJECT_BUSY_ASSERT(m); if (vm_page_xbusied(m) && !pmap_page_is_write_mapped(m)) { old = m->dirty; m->dirty = VM_PAGE_BITS_ALL; } else old = vm_page_bits_swap(m, &m->dirty, VM_PAGE_BITS_ALL); if (old == 0 && (m->a.flags & PGA_SWAP_SPACE) != 0) vm_pager_page_unswapped(m); return (old); } /* * Clear the given bits from the specified page's dirty field. */ static __inline void vm_page_clear_dirty_mask(vm_page_t m, vm_page_bits_t pagebits) { vm_page_assert_busied(m); /* * If the page is xbusied and not write mapped we are the * only thread that can modify dirty bits. Otherwise, The pmap * layer can call vm_page_dirty() without holding a distinguished * lock. The combination of page busy and atomic operations * suffice to guarantee consistency of the page dirty field. */ if (vm_page_xbusied(m) && !pmap_page_is_write_mapped(m)) m->dirty &= ~pagebits; else vm_page_bits_clear(m, &m->dirty, pagebits); } /* * vm_page_set_validclean: * * Sets portions of a page valid and clean. The arguments are expected * to be DEV_BSIZE aligned but if they aren't the bitmap is inclusive * of any partial chunks touched by the range. The invalid portion of * such chunks will be zero'd. * * (base + size) must be less then or equal to PAGE_SIZE. */ void vm_page_set_validclean(vm_page_t m, int base, int size) { vm_page_bits_t oldvalid, pagebits; int endoff, frag; vm_page_assert_busied(m); if (size == 0) /* handle degenerate case */ return; /* * If the base is not DEV_BSIZE aligned and the valid * bit is clear, we have to zero out a portion of the * first block. */ if ((frag = rounddown2(base, DEV_BSIZE)) != base && (m->valid & ((vm_page_bits_t)1 << (base >> DEV_BSHIFT))) == 0) pmap_zero_page_area(m, frag, base - frag); /* * If the ending offset is not DEV_BSIZE aligned and the * valid bit is clear, we have to zero out a portion of * the last block. */ endoff = base + size; if ((frag = rounddown2(endoff, DEV_BSIZE)) != endoff && (m->valid & ((vm_page_bits_t)1 << (endoff >> DEV_BSHIFT))) == 0) pmap_zero_page_area(m, endoff, DEV_BSIZE - (endoff & (DEV_BSIZE - 1))); /* * Set valid, clear dirty bits. If validating the entire * page we can safely clear the pmap modify bit. We also * use this opportunity to clear the PGA_NOSYNC flag. If a process * takes a write fault on a MAP_NOSYNC memory area the flag will * be set again. * * We set valid bits inclusive of any overlap, but we can only * clear dirty bits for DEV_BSIZE chunks that are fully within * the range. */ oldvalid = m->valid; pagebits = vm_page_bits(base, size); if (vm_page_xbusied(m)) m->valid |= pagebits; else vm_page_bits_set(m, &m->valid, pagebits); #if 0 /* NOT YET */ if ((frag = base & (DEV_BSIZE - 1)) != 0) { frag = DEV_BSIZE - frag; base += frag; size -= frag; if (size < 0) size = 0; } pagebits = vm_page_bits(base, size & (DEV_BSIZE - 1)); #endif if (base == 0 && size == PAGE_SIZE) { /* * The page can only be modified within the pmap if it is * mapped, and it can only be mapped if it was previously * fully valid. */ if (oldvalid == VM_PAGE_BITS_ALL) /* * Perform the pmap_clear_modify() first. Otherwise, * a concurrent pmap operation, such as * pmap_protect(), could clear a modification in the * pmap and set the dirty field on the page before * pmap_clear_modify() had begun and after the dirty * field was cleared here. */ pmap_clear_modify(m); m->dirty = 0; vm_page_aflag_clear(m, PGA_NOSYNC); } else if (oldvalid != VM_PAGE_BITS_ALL && vm_page_xbusied(m)) m->dirty &= ~pagebits; else vm_page_clear_dirty_mask(m, pagebits); } void vm_page_clear_dirty(vm_page_t m, int base, int size) { vm_page_clear_dirty_mask(m, vm_page_bits(base, size)); } /* * vm_page_set_invalid: * * Invalidates DEV_BSIZE'd chunks within a page. Both the * valid and dirty bits for the effected areas are cleared. */ void vm_page_set_invalid(vm_page_t m, int base, int size) { vm_page_bits_t bits; vm_object_t object; /* * The object lock is required so that pages can't be mapped * read-only while we're in the process of invalidating them. */ object = m->object; VM_OBJECT_ASSERT_WLOCKED(object); vm_page_assert_busied(m); if (object->type == OBJT_VNODE && base == 0 && IDX_TO_OFF(m->pindex) + size >= object->un_pager.vnp.vnp_size) bits = VM_PAGE_BITS_ALL; else bits = vm_page_bits(base, size); if (object->ref_count != 0 && vm_page_all_valid(m) && bits != 0) pmap_remove_all(m); KASSERT((bits == 0 && vm_page_all_valid(m)) || !pmap_page_is_mapped(m), ("vm_page_set_invalid: page %p is mapped", m)); if (vm_page_xbusied(m)) { m->valid &= ~bits; m->dirty &= ~bits; } else { vm_page_bits_clear(m, &m->valid, bits); vm_page_bits_clear(m, &m->dirty, bits); } } /* * vm_page_invalid: * * Invalidates the entire page. The page must be busy, unmapped, and * the enclosing object must be locked. The object locks protects * against concurrent read-only pmap enter which is done without * busy. */ void vm_page_invalid(vm_page_t m) { vm_page_assert_busied(m); VM_OBJECT_ASSERT_LOCKED(m->object); MPASS(!pmap_page_is_mapped(m)); if (vm_page_xbusied(m)) m->valid = 0; else vm_page_bits_clear(m, &m->valid, VM_PAGE_BITS_ALL); } /* * vm_page_zero_invalid() * * The kernel assumes that the invalid portions of a page contain * garbage, but such pages can be mapped into memory by user code. * When this occurs, we must zero out the non-valid portions of the * page so user code sees what it expects. * * Pages are most often semi-valid when the end of a file is mapped * into memory and the file's size is not page aligned. */ void vm_page_zero_invalid(vm_page_t m, boolean_t setvalid) { int b; int i; /* * Scan the valid bits looking for invalid sections that * must be zeroed. Invalid sub-DEV_BSIZE'd areas ( where the * valid bit may be set ) have already been zeroed by * vm_page_set_validclean(). */ for (b = i = 0; i <= PAGE_SIZE / DEV_BSIZE; ++i) { if (i == (PAGE_SIZE / DEV_BSIZE) || (m->valid & ((vm_page_bits_t)1 << i))) { if (i > b) { pmap_zero_page_area(m, b << DEV_BSHIFT, (i - b) << DEV_BSHIFT); } b = i + 1; } } /* * setvalid is TRUE when we can safely set the zero'd areas * as being valid. We can do this if there are no cache consistancy * issues. e.g. it is ok to do with UFS, but not ok to do with NFS. */ if (setvalid) vm_page_valid(m); } /* * vm_page_is_valid: * * Is (partial) page valid? Note that the case where size == 0 * will return FALSE in the degenerate case where the page is * entirely invalid, and TRUE otherwise. * * Some callers envoke this routine without the busy lock held and * handle races via higher level locks. Typical callers should * hold a busy lock to prevent invalidation. */ int vm_page_is_valid(vm_page_t m, int base, int size) { vm_page_bits_t bits; bits = vm_page_bits(base, size); return (m->valid != 0 && (m->valid & bits) == bits); } /* * Returns true if all of the specified predicates are true for the entire * (super)page and false otherwise. */ bool vm_page_ps_test(vm_page_t m, int flags, vm_page_t skip_m) { vm_object_t object; int i, npages; object = m->object; if (skip_m != NULL && skip_m->object != object) return (false); VM_OBJECT_ASSERT_LOCKED(object); npages = atop(pagesizes[m->psind]); /* * The physically contiguous pages that make up a superpage, i.e., a * page with a page size index ("psind") greater than zero, will * occupy adjacent entries in vm_page_array[]. */ for (i = 0; i < npages; i++) { /* Always test object consistency, including "skip_m". */ if (m[i].object != object) return (false); if (&m[i] == skip_m) continue; if ((flags & PS_NONE_BUSY) != 0 && vm_page_busied(&m[i])) return (false); if ((flags & PS_ALL_DIRTY) != 0) { /* * Calling vm_page_test_dirty() or pmap_is_modified() * might stop this case from spuriously returning * "false". However, that would require a write lock * on the object containing "m[i]". */ if (m[i].dirty != VM_PAGE_BITS_ALL) return (false); } if ((flags & PS_ALL_VALID) != 0 && m[i].valid != VM_PAGE_BITS_ALL) return (false); } return (true); } /* * Set the page's dirty bits if the page is modified. */ void vm_page_test_dirty(vm_page_t m) { vm_page_assert_busied(m); if (m->dirty != VM_PAGE_BITS_ALL && pmap_is_modified(m)) vm_page_dirty(m); } void vm_page_valid(vm_page_t m) { vm_page_assert_busied(m); if (vm_page_xbusied(m)) m->valid = VM_PAGE_BITS_ALL; else vm_page_bits_set(m, &m->valid, VM_PAGE_BITS_ALL); } void vm_page_lock_KBI(vm_page_t m, const char *file, int line) { mtx_lock_flags_(vm_page_lockptr(m), 0, file, line); } void vm_page_unlock_KBI(vm_page_t m, const char *file, int line) { mtx_unlock_flags_(vm_page_lockptr(m), 0, file, line); } int vm_page_trylock_KBI(vm_page_t m, const char *file, int line) { return (mtx_trylock_flags_(vm_page_lockptr(m), 0, file, line)); } #if defined(INVARIANTS) || defined(INVARIANT_SUPPORT) void vm_page_assert_locked_KBI(vm_page_t m, const char *file, int line) { vm_page_lock_assert_KBI(m, MA_OWNED, file, line); } void vm_page_lock_assert_KBI(vm_page_t m, int a, const char *file, int line) { mtx_assert_(vm_page_lockptr(m), a, file, line); } #endif #ifdef INVARIANTS void vm_page_object_busy_assert(vm_page_t m) { /* * Certain of the page's fields may only be modified by the * holder of a page or object busy. */ if (m->object != NULL && !vm_page_busied(m)) VM_OBJECT_ASSERT_BUSY(m->object); } void vm_page_assert_pga_writeable(vm_page_t m, uint16_t bits) { if ((bits & PGA_WRITEABLE) == 0) return; /* * The PGA_WRITEABLE flag can only be set if the page is * managed, is exclusively busied or the object is locked. * Currently, this flag is only set by pmap_enter(). */ KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("PGA_WRITEABLE on unmanaged page")); if (!vm_page_xbusied(m)) VM_OBJECT_ASSERT_BUSY(m->object); } #endif #include "opt_ddb.h" #ifdef DDB #include #include DB_SHOW_COMMAND(page, vm_page_print_page_info) { db_printf("vm_cnt.v_free_count: %d\n", vm_free_count()); db_printf("vm_cnt.v_inactive_count: %d\n", vm_inactive_count()); db_printf("vm_cnt.v_active_count: %d\n", vm_active_count()); db_printf("vm_cnt.v_laundry_count: %d\n", vm_laundry_count()); db_printf("vm_cnt.v_wire_count: %d\n", vm_wire_count()); db_printf("vm_cnt.v_free_reserved: %d\n", vm_cnt.v_free_reserved); db_printf("vm_cnt.v_free_min: %d\n", vm_cnt.v_free_min); db_printf("vm_cnt.v_free_target: %d\n", vm_cnt.v_free_target); db_printf("vm_cnt.v_inactive_target: %d\n", vm_cnt.v_inactive_target); } DB_SHOW_COMMAND(pageq, vm_page_print_pageq_info) { int dom; db_printf("pq_free %d\n", vm_free_count()); for (dom = 0; dom < vm_ndomains; dom++) { db_printf( "dom %d page_cnt %d free %d pq_act %d pq_inact %d pq_laund %d pq_unsw %d\n", dom, vm_dom[dom].vmd_page_count, vm_dom[dom].vmd_free_count, vm_dom[dom].vmd_pagequeues[PQ_ACTIVE].pq_cnt, vm_dom[dom].vmd_pagequeues[PQ_INACTIVE].pq_cnt, vm_dom[dom].vmd_pagequeues[PQ_LAUNDRY].pq_cnt, vm_dom[dom].vmd_pagequeues[PQ_UNSWAPPABLE].pq_cnt); } } DB_SHOW_COMMAND(pginfo, vm_page_print_pginfo) { vm_page_t m; boolean_t phys, virt; if (!have_addr) { db_printf("show pginfo addr\n"); return; } phys = strchr(modif, 'p') != NULL; virt = strchr(modif, 'v') != NULL; if (virt) m = PHYS_TO_VM_PAGE(pmap_kextract(addr)); else if (phys) m = PHYS_TO_VM_PAGE(addr); else m = (vm_page_t)addr; db_printf( "page %p obj %p pidx 0x%jx phys 0x%jx q %d ref %u\n" " af 0x%x of 0x%x f 0x%x act %d busy %x valid 0x%x dirty 0x%x\n", m, m->object, (uintmax_t)m->pindex, (uintmax_t)m->phys_addr, m->a.queue, m->ref_count, m->a.flags, m->oflags, m->flags, m->a.act_count, m->busy_lock, m->valid, m->dirty); } #endif /* DDB */ Index: projects/clang1000-import/sys/vm/vm_pageout.c =================================================================== --- projects/clang1000-import/sys/vm/vm_pageout.c (revision 357965) +++ projects/clang1000-import/sys/vm/vm_pageout.c (revision 357966) @@ -1,2243 +1,2243 @@ /*- * SPDX-License-Identifier: (BSD-4-Clause AND MIT-CMU) * * Copyright (c) 1991 Regents of the University of California. * All rights reserved. * Copyright (c) 1994 John S. Dyson * All rights reserved. * Copyright (c) 1994 David Greenman * All rights reserved. * Copyright (c) 2005 Yahoo! Technologies Norway AS * All rights reserved. * * This code is derived from software contributed to Berkeley by * The Mach Operating System project at Carnegie-Mellon University. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by the University of * California, Berkeley and its contributors. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * from: @(#)vm_pageout.c 7.4 (Berkeley) 5/7/91 * * * Copyright (c) 1987, 1990 Carnegie-Mellon University. * All rights reserved. * * Authors: Avadis Tevanian, Jr., Michael Wayne Young * * Permission to use, copy, modify and distribute this software and * its documentation is hereby granted, provided that both the copyright * notice and this permission notice appear in all copies of the * software, derivative works or modified versions, and any portions * thereof, and that both notices appear in supporting documentation. * * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. * * Carnegie Mellon requests users of this software to return to * * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU * School of Computer Science * Carnegie Mellon University * Pittsburgh PA 15213-3890 * * any improvements or extensions that they make and grant Carnegie the * rights to redistribute these changes. */ /* * The proverbial page-out daemon. */ #include __FBSDID("$FreeBSD$"); #include "opt_vm.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* * System initialization */ /* the kernel process "vm_pageout"*/ static void vm_pageout(void); static void vm_pageout_init(void); static int vm_pageout_clean(vm_page_t m, int *numpagedout); static int vm_pageout_cluster(vm_page_t m); static void vm_pageout_mightbe_oom(struct vm_domain *vmd, int page_shortage, int starting_page_shortage); SYSINIT(pagedaemon_init, SI_SUB_KTHREAD_PAGE, SI_ORDER_FIRST, vm_pageout_init, NULL); struct proc *pageproc; static struct kproc_desc page_kp = { "pagedaemon", vm_pageout, &pageproc }; SYSINIT(pagedaemon, SI_SUB_KTHREAD_PAGE, SI_ORDER_SECOND, kproc_start, &page_kp); SDT_PROVIDER_DEFINE(vm); SDT_PROBE_DEFINE(vm, , , vm__lowmem_scan); /* Pagedaemon activity rates, in subdivisions of one second. */ #define VM_LAUNDER_RATE 10 #define VM_INACT_SCAN_RATE 10 static int vm_pageout_oom_seq = 12; static int vm_pageout_update_period; static int disable_swap_pageouts; static int lowmem_period = 10; static int swapdev_enabled; static int vm_panic_on_oom = 0; SYSCTL_INT(_vm, OID_AUTO, panic_on_oom, CTLFLAG_RWTUN, &vm_panic_on_oom, 0, "Panic on the given number of out-of-memory errors instead of killing the largest process"); SYSCTL_INT(_vm, OID_AUTO, pageout_update_period, CTLFLAG_RWTUN, &vm_pageout_update_period, 0, "Maximum active LRU update period"); SYSCTL_INT(_vm, OID_AUTO, lowmem_period, CTLFLAG_RWTUN, &lowmem_period, 0, "Low memory callback period"); SYSCTL_INT(_vm, OID_AUTO, disable_swapspace_pageouts, CTLFLAG_RWTUN, &disable_swap_pageouts, 0, "Disallow swapout of dirty pages"); static int pageout_lock_miss; SYSCTL_INT(_vm, OID_AUTO, pageout_lock_miss, CTLFLAG_RD, &pageout_lock_miss, 0, "vget() lock misses during pageout"); SYSCTL_INT(_vm, OID_AUTO, pageout_oom_seq, CTLFLAG_RWTUN, &vm_pageout_oom_seq, 0, "back-to-back calls to oom detector to start OOM"); static int act_scan_laundry_weight = 3; SYSCTL_INT(_vm, OID_AUTO, act_scan_laundry_weight, CTLFLAG_RWTUN, &act_scan_laundry_weight, 0, "weight given to clean vs. dirty pages in active queue scans"); static u_int vm_background_launder_rate = 4096; SYSCTL_UINT(_vm, OID_AUTO, background_launder_rate, CTLFLAG_RWTUN, &vm_background_launder_rate, 0, "background laundering rate, in kilobytes per second"); static u_int vm_background_launder_max = 20 * 1024; SYSCTL_UINT(_vm, OID_AUTO, background_launder_max, CTLFLAG_RWTUN, &vm_background_launder_max, 0, "background laundering cap, in kilobytes"); int vm_pageout_page_count = 32; u_long vm_page_max_user_wired; SYSCTL_ULONG(_vm, OID_AUTO, max_user_wired, CTLFLAG_RW, &vm_page_max_user_wired, 0, "system-wide limit to user-wired page count"); static u_int isqrt(u_int num); static int vm_pageout_launder(struct vm_domain *vmd, int launder, bool in_shortfall); static void vm_pageout_laundry_worker(void *arg); struct scan_state { struct vm_batchqueue bq; struct vm_pagequeue *pq; vm_page_t marker; int maxscan; int scanned; }; static void vm_pageout_init_scan(struct scan_state *ss, struct vm_pagequeue *pq, vm_page_t marker, vm_page_t after, int maxscan) { vm_pagequeue_assert_locked(pq); KASSERT((marker->a.flags & PGA_ENQUEUED) == 0, ("marker %p already enqueued", marker)); if (after == NULL) TAILQ_INSERT_HEAD(&pq->pq_pl, marker, plinks.q); else TAILQ_INSERT_AFTER(&pq->pq_pl, after, marker, plinks.q); vm_page_aflag_set(marker, PGA_ENQUEUED); vm_batchqueue_init(&ss->bq); ss->pq = pq; ss->marker = marker; ss->maxscan = maxscan; ss->scanned = 0; vm_pagequeue_unlock(pq); } static void vm_pageout_end_scan(struct scan_state *ss) { struct vm_pagequeue *pq; pq = ss->pq; vm_pagequeue_assert_locked(pq); KASSERT((ss->marker->a.flags & PGA_ENQUEUED) != 0, ("marker %p not enqueued", ss->marker)); TAILQ_REMOVE(&pq->pq_pl, ss->marker, plinks.q); vm_page_aflag_clear(ss->marker, PGA_ENQUEUED); pq->pq_pdpages += ss->scanned; } /* * Add a small number of queued pages to a batch queue for later processing * without the corresponding queue lock held. The caller must have enqueued a * marker page at the desired start point for the scan. Pages will be * physically dequeued if the caller so requests. Otherwise, the returned * batch may contain marker pages, and it is up to the caller to handle them. * * When processing the batch queue, vm_page_queue() must be used to * determine whether the page has been logically dequeued by another thread. * Once this check is performed, the page lock guarantees that the page will * not be disassociated from the queue. */ static __always_inline void vm_pageout_collect_batch(struct scan_state *ss, const bool dequeue) { struct vm_pagequeue *pq; vm_page_t m, marker, n; marker = ss->marker; pq = ss->pq; KASSERT((marker->a.flags & PGA_ENQUEUED) != 0, ("marker %p not enqueued", ss->marker)); vm_pagequeue_lock(pq); for (m = TAILQ_NEXT(marker, plinks.q); m != NULL && ss->scanned < ss->maxscan && ss->bq.bq_cnt < VM_BATCHQUEUE_SIZE; m = n, ss->scanned++) { n = TAILQ_NEXT(m, plinks.q); if ((m->flags & PG_MARKER) == 0) { KASSERT((m->a.flags & PGA_ENQUEUED) != 0, ("page %p not enqueued", m)); KASSERT((m->flags & PG_FICTITIOUS) == 0, ("Fictitious page %p cannot be in page queue", m)); KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("Unmanaged page %p cannot be in page queue", m)); } else if (dequeue) continue; (void)vm_batchqueue_insert(&ss->bq, m); if (dequeue) { TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); vm_page_aflag_clear(m, PGA_ENQUEUED); } } TAILQ_REMOVE(&pq->pq_pl, marker, plinks.q); if (__predict_true(m != NULL)) TAILQ_INSERT_BEFORE(m, marker, plinks.q); else TAILQ_INSERT_TAIL(&pq->pq_pl, marker, plinks.q); if (dequeue) vm_pagequeue_cnt_add(pq, -ss->bq.bq_cnt); vm_pagequeue_unlock(pq); } /* * Return the next page to be scanned, or NULL if the scan is complete. */ static __always_inline vm_page_t vm_pageout_next(struct scan_state *ss, const bool dequeue) { if (ss->bq.bq_cnt == 0) vm_pageout_collect_batch(ss, dequeue); return (vm_batchqueue_pop(&ss->bq)); } /* * Determine whether processing of a page should be deferred and ensure that any * outstanding queue operations are processed. */ static __always_inline bool vm_pageout_defer(vm_page_t m, const uint8_t queue, const bool enqueued) { vm_page_astate_t as; as = vm_page_astate_load(m); if (__predict_false(as.queue != queue || ((as.flags & PGA_ENQUEUED) != 0) != enqueued)) return (true); if ((as.flags & PGA_QUEUE_OP_MASK) != 0) { vm_page_pqbatch_submit(m, queue); return (true); } return (false); } /* * Scan for pages at adjacent offsets within the given page's object that are * eligible for laundering, form a cluster of these pages and the given page, * and launder that cluster. */ static int vm_pageout_cluster(vm_page_t m) { vm_object_t object; vm_page_t mc[2 * vm_pageout_page_count], p, pb, ps; vm_pindex_t pindex; int ib, is, page_base, pageout_count; object = m->object; VM_OBJECT_ASSERT_WLOCKED(object); pindex = m->pindex; vm_page_assert_xbusied(m); mc[vm_pageout_page_count] = pb = ps = m; pageout_count = 1; page_base = vm_pageout_page_count; ib = 1; is = 1; /* * We can cluster only if the page is not clean, busy, or held, and * the page is in the laundry queue. * * During heavy mmap/modification loads the pageout * daemon can really fragment the underlying file * due to flushing pages out of order and not trying to * align the clusters (which leaves sporadic out-of-order * holes). To solve this problem we do the reverse scan * first and attempt to align our cluster, then do a * forward scan if room remains. */ more: while (ib != 0 && pageout_count < vm_pageout_page_count) { if (ib > pindex) { ib = 0; break; } if ((p = vm_page_prev(pb)) == NULL || vm_page_tryxbusy(p) == 0) { ib = 0; break; } if (vm_page_wired(p)) { ib = 0; vm_page_xunbusy(p); break; } vm_page_test_dirty(p); if (p->dirty == 0) { ib = 0; vm_page_xunbusy(p); break; } if (!vm_page_in_laundry(p) || !vm_page_try_remove_write(p)) { vm_page_xunbusy(p); ib = 0; break; } mc[--page_base] = pb = p; ++pageout_count; ++ib; /* * We are at an alignment boundary. Stop here, and switch * directions. Do not clear ib. */ if ((pindex - (ib - 1)) % vm_pageout_page_count == 0) break; } while (pageout_count < vm_pageout_page_count && pindex + is < object->size) { if ((p = vm_page_next(ps)) == NULL || vm_page_tryxbusy(p) == 0) break; if (vm_page_wired(p)) { vm_page_xunbusy(p); break; } vm_page_test_dirty(p); if (p->dirty == 0) { vm_page_xunbusy(p); break; } if (!vm_page_in_laundry(p) || !vm_page_try_remove_write(p)) { vm_page_xunbusy(p); break; } mc[page_base + pageout_count] = ps = p; ++pageout_count; ++is; } /* * If we exhausted our forward scan, continue with the reverse scan * when possible, even past an alignment boundary. This catches * boundary conditions. */ if (ib != 0 && pageout_count < vm_pageout_page_count) goto more; return (vm_pageout_flush(&mc[page_base], pageout_count, VM_PAGER_PUT_NOREUSE, 0, NULL, NULL)); } /* * vm_pageout_flush() - launder the given pages * * The given pages are laundered. Note that we setup for the start of * I/O ( i.e. busy the page ), mark it read-only, and bump the object * reference count all in here rather then in the parent. If we want * the parent to do more sophisticated things we may have to change * the ordering. * * Returned runlen is the count of pages between mreq and first * page after mreq with status VM_PAGER_AGAIN. * *eio is set to TRUE if pager returned VM_PAGER_ERROR or VM_PAGER_FAIL * for any page in runlen set. */ int vm_pageout_flush(vm_page_t *mc, int count, int flags, int mreq, int *prunlen, boolean_t *eio) { vm_object_t object = mc[0]->object; int pageout_status[count]; int numpagedout = 0; int i, runlen; VM_OBJECT_ASSERT_WLOCKED(object); /* * Initiate I/O. Mark the pages shared busy and verify that they're * valid and read-only. * * We do not have to fixup the clean/dirty bits here... we can * allow the pager to do it after the I/O completes. * * NOTE! mc[i]->dirty may be partial or fragmented due to an * edge case with file fragments. */ for (i = 0; i < count; i++) { KASSERT(vm_page_all_valid(mc[i]), ("vm_pageout_flush: partially invalid page %p index %d/%d", mc[i], i, count)); KASSERT((mc[i]->a.flags & PGA_WRITEABLE) == 0, ("vm_pageout_flush: writeable page %p", mc[i])); vm_page_busy_downgrade(mc[i]); } vm_object_pip_add(object, count); vm_pager_put_pages(object, mc, count, flags, pageout_status); runlen = count - mreq; if (eio != NULL) *eio = FALSE; for (i = 0; i < count; i++) { vm_page_t mt = mc[i]; KASSERT(pageout_status[i] == VM_PAGER_PEND || !pmap_page_is_write_mapped(mt), ("vm_pageout_flush: page %p is not write protected", mt)); switch (pageout_status[i]) { case VM_PAGER_OK: /* * The page may have moved since laundering started, in * which case it should be left alone. */ if (vm_page_in_laundry(mt)) vm_page_deactivate_noreuse(mt); /* FALLTHROUGH */ case VM_PAGER_PEND: numpagedout++; break; case VM_PAGER_BAD: /* * The page is outside the object's range. We pretend * that the page out worked and clean the page, so the * changes will be lost if the page is reclaimed by * the page daemon. */ vm_page_undirty(mt); if (vm_page_in_laundry(mt)) vm_page_deactivate_noreuse(mt); break; case VM_PAGER_ERROR: case VM_PAGER_FAIL: /* * If the page couldn't be paged out to swap because the * pager wasn't able to find space, place the page in * the PQ_UNSWAPPABLE holding queue. This is an * optimization that prevents the page daemon from * wasting CPU cycles on pages that cannot be reclaimed * becase no swap device is configured. * * Otherwise, reactivate the page so that it doesn't * clog the laundry and inactive queues. (We will try * paging it out again later.) */ if (object->type == OBJT_SWAP && pageout_status[i] == VM_PAGER_FAIL) { vm_page_unswappable(mt); numpagedout++; } else vm_page_activate(mt); if (eio != NULL && i >= mreq && i - mreq < runlen) *eio = TRUE; break; case VM_PAGER_AGAIN: if (i >= mreq && i - mreq < runlen) runlen = i - mreq; break; } /* * If the operation is still going, leave the page busy to * block all other accesses. Also, leave the paging in * progress indicator set so that we don't attempt an object * collapse. */ if (pageout_status[i] != VM_PAGER_PEND) { vm_object_pip_wakeup(object); vm_page_sunbusy(mt); } } if (prunlen != NULL) *prunlen = runlen; return (numpagedout); } static void vm_pageout_swapon(void *arg __unused, struct swdevt *sp __unused) { atomic_store_rel_int(&swapdev_enabled, 1); } static void vm_pageout_swapoff(void *arg __unused, struct swdevt *sp __unused) { if (swap_pager_nswapdev() == 1) atomic_store_rel_int(&swapdev_enabled, 0); } /* * Attempt to acquire all of the necessary locks to launder a page and * then call through the clustering layer to PUTPAGES. Wait a short * time for a vnode lock. * * Requires the page and object lock on entry, releases both before return. * Returns 0 on success and an errno otherwise. */ static int vm_pageout_clean(vm_page_t m, int *numpagedout) { struct vnode *vp; struct mount *mp; vm_object_t object; vm_pindex_t pindex; int error, lockmode; object = m->object; VM_OBJECT_ASSERT_WLOCKED(object); error = 0; vp = NULL; mp = NULL; /* * The object is already known NOT to be dead. It * is possible for the vget() to block the whole * pageout daemon, but the new low-memory handling * code should prevent it. * * We can't wait forever for the vnode lock, we might * deadlock due to a vn_read() getting stuck in * vm_wait while holding this vnode. We skip the * vnode if we can't get it in a reasonable amount * of time. */ if (object->type == OBJT_VNODE) { vm_page_xunbusy(m); vp = object->handle; if (vp->v_type == VREG && vn_start_write(vp, &mp, V_NOWAIT) != 0) { mp = NULL; error = EDEADLK; goto unlock_all; } KASSERT(mp != NULL, ("vp %p with NULL v_mount", vp)); vm_object_reference_locked(object); pindex = m->pindex; VM_OBJECT_WUNLOCK(object); lockmode = MNT_SHARED_WRITES(vp->v_mount) ? LK_SHARED : LK_EXCLUSIVE; if (vget(vp, lockmode | LK_TIMELOCK, curthread)) { vp = NULL; error = EDEADLK; goto unlock_mp; } VM_OBJECT_WLOCK(object); /* * Ensure that the object and vnode were not disassociated * while locks were dropped. */ if (vp->v_object != object) { error = ENOENT; goto unlock_all; } /* * While the object was unlocked, the page may have been: * (1) moved to a different queue, * (2) reallocated to a different object, * (3) reallocated to a different offset, or * (4) cleaned. */ if (!vm_page_in_laundry(m) || m->object != object || m->pindex != pindex || m->dirty == 0) { error = ENXIO; goto unlock_all; } /* * The page may have been busied while the object lock was * released. */ if (vm_page_tryxbusy(m) == 0) { error = EBUSY; goto unlock_all; } } /* * Remove all writeable mappings, failing if the page is wired. */ if (!vm_page_try_remove_write(m)) { vm_page_xunbusy(m); error = EBUSY; goto unlock_all; } /* * If a page is dirty, then it is either being washed * (but not yet cleaned) or it is still in the * laundry. If it is still in the laundry, then we * start the cleaning operation. */ if ((*numpagedout = vm_pageout_cluster(m)) == 0) error = EIO; unlock_all: VM_OBJECT_WUNLOCK(object); unlock_mp: if (mp != NULL) { if (vp != NULL) vput(vp); vm_object_deallocate(object); vn_finished_write(mp); } return (error); } /* * Attempt to launder the specified number of pages. * * Returns the number of pages successfully laundered. */ static int vm_pageout_launder(struct vm_domain *vmd, int launder, bool in_shortfall) { struct scan_state ss; struct vm_pagequeue *pq; vm_object_t object; vm_page_t m, marker; vm_page_astate_t new, old; int act_delta, error, numpagedout, queue, refs, starting_target; int vnodes_skipped; bool pageout_ok; object = NULL; starting_target = launder; vnodes_skipped = 0; /* * Scan the laundry queues for pages eligible to be laundered. We stop * once the target number of dirty pages have been laundered, or once * we've reached the end of the queue. A single iteration of this loop * may cause more than one page to be laundered because of clustering. * * As an optimization, we avoid laundering from PQ_UNSWAPPABLE when no * swap devices are configured. */ if (atomic_load_acq_int(&swapdev_enabled)) queue = PQ_UNSWAPPABLE; else queue = PQ_LAUNDRY; scan: marker = &vmd->vmd_markers[queue]; pq = &vmd->vmd_pagequeues[queue]; vm_pagequeue_lock(pq); vm_pageout_init_scan(&ss, pq, marker, NULL, pq->pq_cnt); while (launder > 0 && (m = vm_pageout_next(&ss, false)) != NULL) { if (__predict_false((m->flags & PG_MARKER) != 0)) continue; /* * Don't touch a page that was removed from the queue after the * page queue lock was released. Otherwise, ensure that any * pending queue operations, such as dequeues for wired pages, * are handled. */ if (vm_pageout_defer(m, queue, true)) continue; /* * Lock the page's object. */ if (object == NULL || object != m->object) { if (object != NULL) VM_OBJECT_WUNLOCK(object); - object = (vm_object_t)atomic_load_ptr(&m->object); + object = atomic_load_ptr(&m->object); if (__predict_false(object == NULL)) /* The page is being freed by another thread. */ continue; /* Depends on type-stability. */ VM_OBJECT_WLOCK(object); if (__predict_false(m->object != object)) { VM_OBJECT_WUNLOCK(object); object = NULL; continue; } } if (vm_page_tryxbusy(m) == 0) continue; /* * Check for wirings now that we hold the object lock and have * exclusively busied the page. If the page is mapped, it may * still be wired by pmap lookups. The call to * vm_page_try_remove_all() below atomically checks for such * wirings and removes mappings. If the page is unmapped, the * wire count is guaranteed not to increase after this check. */ if (__predict_false(vm_page_wired(m))) goto skip_page; /* * Invalid pages can be easily freed. They cannot be * mapped; vm_page_free() asserts this. */ if (vm_page_none_valid(m)) goto free_page; refs = object->ref_count != 0 ? pmap_ts_referenced(m) : 0; for (old = vm_page_astate_load(m);;) { /* * Check to see if the page has been removed from the * queue since the first such check. Leave it alone if * so, discarding any references collected by * pmap_ts_referenced(). */ if (__predict_false(_vm_page_queue(old) == PQ_NONE)) goto skip_page; new = old; act_delta = refs; if ((old.flags & PGA_REFERENCED) != 0) { new.flags &= ~PGA_REFERENCED; act_delta++; } if (act_delta == 0) { ; } else if (object->ref_count != 0) { /* * Increase the activation count if the page was * referenced while in the laundry queue. This * makes it less likely that the page will be * returned prematurely to the laundry queue. */ new.act_count += ACT_ADVANCE + act_delta; if (new.act_count > ACT_MAX) new.act_count = ACT_MAX; new.flags &= ~PGA_QUEUE_OP_MASK; new.flags |= PGA_REQUEUE; new.queue = PQ_ACTIVE; if (!vm_page_pqstate_commit(m, &old, new)) continue; /* * If this was a background laundering, count * activated pages towards our target. The * purpose of background laundering is to ensure * that pages are eventually cycled through the * laundry queue, and an activation is a valid * way out. */ if (!in_shortfall) launder--; VM_CNT_INC(v_reactivated); goto skip_page; } else if ((object->flags & OBJ_DEAD) == 0) { new.flags |= PGA_REQUEUE; if (!vm_page_pqstate_commit(m, &old, new)) continue; goto skip_page; } break; } /* * If the page appears to be clean at the machine-independent * layer, then remove all of its mappings from the pmap in * anticipation of freeing it. If, however, any of the page's * mappings allow write access, then the page may still be * modified until the last of those mappings are removed. */ if (object->ref_count != 0) { vm_page_test_dirty(m); if (m->dirty == 0 && !vm_page_try_remove_all(m)) goto skip_page; } /* * Clean pages are freed, and dirty pages are paged out unless * they belong to a dead object. Requeueing dirty pages from * dead objects is pointless, as they are being paged out and * freed by the thread that destroyed the object. */ if (m->dirty == 0) { free_page: /* * Now we are guaranteed that no other threads are * manipulating the page, check for a last-second * reference. */ if (vm_pageout_defer(m, queue, true)) goto skip_page; vm_page_free(m); VM_CNT_INC(v_dfree); } else if ((object->flags & OBJ_DEAD) == 0) { if (object->type != OBJT_SWAP && object->type != OBJT_DEFAULT) pageout_ok = true; else if (disable_swap_pageouts) pageout_ok = false; else pageout_ok = true; if (!pageout_ok) { vm_page_launder(m); goto skip_page; } /* * Form a cluster with adjacent, dirty pages from the * same object, and page out that entire cluster. * * The adjacent, dirty pages must also be in the * laundry. However, their mappings are not checked * for new references. Consequently, a recently * referenced page may be paged out. However, that * page will not be prematurely reclaimed. After page * out, the page will be placed in the inactive queue, * where any new references will be detected and the * page reactivated. */ error = vm_pageout_clean(m, &numpagedout); if (error == 0) { launder -= numpagedout; ss.scanned += numpagedout; } else if (error == EDEADLK) { pageout_lock_miss++; vnodes_skipped++; } object = NULL; } else { skip_page: vm_page_xunbusy(m); } } if (object != NULL) { VM_OBJECT_WUNLOCK(object); object = NULL; } vm_pagequeue_lock(pq); vm_pageout_end_scan(&ss); vm_pagequeue_unlock(pq); if (launder > 0 && queue == PQ_UNSWAPPABLE) { queue = PQ_LAUNDRY; goto scan; } /* * Wakeup the sync daemon if we skipped a vnode in a writeable object * and we didn't launder enough pages. */ if (vnodes_skipped > 0 && launder > 0) (void)speedup_syncer(); return (starting_target - launder); } /* * Compute the integer square root. */ static u_int isqrt(u_int num) { u_int bit, root, tmp; bit = num != 0 ? (1u << ((fls(num) - 1) & ~1)) : 0; root = 0; while (bit != 0) { tmp = root + bit; root >>= 1; if (num >= tmp) { num -= tmp; root += bit; } bit >>= 2; } return (root); } /* * Perform the work of the laundry thread: periodically wake up and determine * whether any pages need to be laundered. If so, determine the number of pages * that need to be laundered, and launder them. */ static void vm_pageout_laundry_worker(void *arg) { struct vm_domain *vmd; struct vm_pagequeue *pq; uint64_t nclean, ndirty, nfreed; int domain, last_target, launder, shortfall, shortfall_cycle, target; bool in_shortfall; domain = (uintptr_t)arg; vmd = VM_DOMAIN(domain); pq = &vmd->vmd_pagequeues[PQ_LAUNDRY]; KASSERT(vmd->vmd_segs != 0, ("domain without segments")); shortfall = 0; in_shortfall = false; shortfall_cycle = 0; last_target = target = 0; nfreed = 0; /* * Calls to these handlers are serialized by the swap syscall lock. */ (void)EVENTHANDLER_REGISTER(swapon, vm_pageout_swapon, vmd, EVENTHANDLER_PRI_ANY); (void)EVENTHANDLER_REGISTER(swapoff, vm_pageout_swapoff, vmd, EVENTHANDLER_PRI_ANY); /* * The pageout laundry worker is never done, so loop forever. */ for (;;) { KASSERT(target >= 0, ("negative target %d", target)); KASSERT(shortfall_cycle >= 0, ("negative cycle %d", shortfall_cycle)); launder = 0; /* * First determine whether we need to launder pages to meet a * shortage of free pages. */ if (shortfall > 0) { in_shortfall = true; shortfall_cycle = VM_LAUNDER_RATE / VM_INACT_SCAN_RATE; target = shortfall; } else if (!in_shortfall) goto trybackground; else if (shortfall_cycle == 0 || vm_laundry_target(vmd) <= 0) { /* * We recently entered shortfall and began laundering * pages. If we have completed that laundering run * (and we are no longer in shortfall) or we have met * our laundry target through other activity, then we * can stop laundering pages. */ in_shortfall = false; target = 0; goto trybackground; } launder = target / shortfall_cycle--; goto dolaundry; /* * There's no immediate need to launder any pages; see if we * meet the conditions to perform background laundering: * * 1. The ratio of dirty to clean inactive pages exceeds the * background laundering threshold, or * 2. we haven't yet reached the target of the current * background laundering run. * * The background laundering threshold is not a constant. * Instead, it is a slowly growing function of the number of * clean pages freed by the page daemon since the last * background laundering. Thus, as the ratio of dirty to * clean inactive pages grows, the amount of memory pressure * required to trigger laundering decreases. We ensure * that the threshold is non-zero after an inactive queue * scan, even if that scan failed to free a single clean page. */ trybackground: nclean = vmd->vmd_free_count + vmd->vmd_pagequeues[PQ_INACTIVE].pq_cnt; ndirty = vmd->vmd_pagequeues[PQ_LAUNDRY].pq_cnt; if (target == 0 && ndirty * isqrt(howmany(nfreed + 1, vmd->vmd_free_target - vmd->vmd_free_min)) >= nclean) { target = vmd->vmd_background_launder_target; } /* * We have a non-zero background laundering target. If we've * laundered up to our maximum without observing a page daemon * request, just stop. This is a safety belt that ensures we * don't launder an excessive amount if memory pressure is low * and the ratio of dirty to clean pages is large. Otherwise, * proceed at the background laundering rate. */ if (target > 0) { if (nfreed > 0) { nfreed = 0; last_target = target; } else if (last_target - target >= vm_background_launder_max * PAGE_SIZE / 1024) { target = 0; } launder = vm_background_launder_rate * PAGE_SIZE / 1024; launder /= VM_LAUNDER_RATE; if (launder > target) launder = target; } dolaundry: if (launder > 0) { /* * Because of I/O clustering, the number of laundered * pages could exceed "target" by the maximum size of * a cluster minus one. */ target -= min(vm_pageout_launder(vmd, launder, in_shortfall), target); pause("laundp", hz / VM_LAUNDER_RATE); } /* * If we're not currently laundering pages and the page daemon * hasn't posted a new request, sleep until the page daemon * kicks us. */ vm_pagequeue_lock(pq); if (target == 0 && vmd->vmd_laundry_request == VM_LAUNDRY_IDLE) (void)mtx_sleep(&vmd->vmd_laundry_request, vm_pagequeue_lockptr(pq), PVM, "launds", 0); /* * If the pagedaemon has indicated that it's in shortfall, start * a shortfall laundering unless we're already in the middle of * one. This may preempt a background laundering. */ if (vmd->vmd_laundry_request == VM_LAUNDRY_SHORTFALL && (!in_shortfall || shortfall_cycle == 0)) { shortfall = vm_laundry_target(vmd) + vmd->vmd_pageout_deficit; target = 0; } else shortfall = 0; if (target == 0) vmd->vmd_laundry_request = VM_LAUNDRY_IDLE; nfreed += vmd->vmd_clean_pages_freed; vmd->vmd_clean_pages_freed = 0; vm_pagequeue_unlock(pq); } } /* * Compute the number of pages we want to try to move from the * active queue to either the inactive or laundry queue. * * When scanning active pages during a shortage, we make clean pages * count more heavily towards the page shortage than dirty pages. * This is because dirty pages must be laundered before they can be * reused and thus have less utility when attempting to quickly * alleviate a free page shortage. However, this weighting also * causes the scan to deactivate dirty pages more aggressively, * improving the effectiveness of clustering. */ static int vm_pageout_active_target(struct vm_domain *vmd) { int shortage; shortage = vmd->vmd_inactive_target + vm_paging_target(vmd) - (vmd->vmd_pagequeues[PQ_INACTIVE].pq_cnt + vmd->vmd_pagequeues[PQ_LAUNDRY].pq_cnt / act_scan_laundry_weight); shortage *= act_scan_laundry_weight; return (shortage); } /* * Scan the active queue. If there is no shortage of inactive pages, scan a * small portion of the queue in order to maintain quasi-LRU. */ static void vm_pageout_scan_active(struct vm_domain *vmd, int page_shortage) { struct scan_state ss; vm_object_t object; vm_page_t m, marker; struct vm_pagequeue *pq; vm_page_astate_t old, new; long min_scan; int act_delta, max_scan, ps_delta, refs, scan_tick; uint8_t nqueue; marker = &vmd->vmd_markers[PQ_ACTIVE]; pq = &vmd->vmd_pagequeues[PQ_ACTIVE]; vm_pagequeue_lock(pq); /* * If we're just idle polling attempt to visit every * active page within 'update_period' seconds. */ scan_tick = ticks; if (vm_pageout_update_period != 0) { min_scan = pq->pq_cnt; min_scan *= scan_tick - vmd->vmd_last_active_scan; min_scan /= hz * vm_pageout_update_period; } else min_scan = 0; if (min_scan > 0 || (page_shortage > 0 && pq->pq_cnt > 0)) vmd->vmd_last_active_scan = scan_tick; /* * Scan the active queue for pages that can be deactivated. Update * the per-page activity counter and use it to identify deactivation * candidates. Held pages may be deactivated. * * To avoid requeuing each page that remains in the active queue, we * implement the CLOCK algorithm. To keep the implementation of the * enqueue operation consistent for all page queues, we use two hands, * represented by marker pages. Scans begin at the first hand, which * precedes the second hand in the queue. When the two hands meet, * they are moved back to the head and tail of the queue, respectively, * and scanning resumes. */ max_scan = page_shortage > 0 ? pq->pq_cnt : min_scan; act_scan: vm_pageout_init_scan(&ss, pq, marker, &vmd->vmd_clock[0], max_scan); while ((m = vm_pageout_next(&ss, false)) != NULL) { if (__predict_false(m == &vmd->vmd_clock[1])) { vm_pagequeue_lock(pq); TAILQ_REMOVE(&pq->pq_pl, &vmd->vmd_clock[0], plinks.q); TAILQ_REMOVE(&pq->pq_pl, &vmd->vmd_clock[1], plinks.q); TAILQ_INSERT_HEAD(&pq->pq_pl, &vmd->vmd_clock[0], plinks.q); TAILQ_INSERT_TAIL(&pq->pq_pl, &vmd->vmd_clock[1], plinks.q); max_scan -= ss.scanned; vm_pageout_end_scan(&ss); goto act_scan; } if (__predict_false((m->flags & PG_MARKER) != 0)) continue; /* * Don't touch a page that was removed from the queue after the * page queue lock was released. Otherwise, ensure that any * pending queue operations, such as dequeues for wired pages, * are handled. */ if (vm_pageout_defer(m, PQ_ACTIVE, true)) continue; /* * A page's object pointer may be set to NULL before * the object lock is acquired. */ - object = (vm_object_t)atomic_load_ptr(&m->object); + object = atomic_load_ptr(&m->object); if (__predict_false(object == NULL)) /* * The page has been removed from its object. */ continue; /* Deferred free of swap space. */ if ((m->a.flags & PGA_SWAP_FREE) != 0 && VM_OBJECT_TRYWLOCK(object)) { if (m->object == object) vm_pager_page_unswapped(m); VM_OBJECT_WUNLOCK(object); } /* * Check to see "how much" the page has been used. * * Test PGA_REFERENCED after calling pmap_ts_referenced() so * that a reference from a concurrently destroyed mapping is * observed here and now. * * Perform an unsynchronized object ref count check. While * the page lock ensures that the page is not reallocated to * another object, in particular, one with unmanaged mappings * that cannot support pmap_ts_referenced(), two races are, * nonetheless, possible: * 1) The count was transitioning to zero, but we saw a non- * zero value. pmap_ts_referenced() will return zero * because the page is not mapped. * 2) The count was transitioning to one, but we saw zero. * This race delays the detection of a new reference. At * worst, we will deactivate and reactivate the page. */ refs = object->ref_count != 0 ? pmap_ts_referenced(m) : 0; old = vm_page_astate_load(m); do { /* * Check to see if the page has been removed from the * queue since the first such check. Leave it alone if * so, discarding any references collected by * pmap_ts_referenced(). */ if (__predict_false(_vm_page_queue(old) == PQ_NONE)) break; /* * Advance or decay the act_count based on recent usage. */ new = old; act_delta = refs; if ((old.flags & PGA_REFERENCED) != 0) { new.flags &= ~PGA_REFERENCED; act_delta++; } if (act_delta != 0) { new.act_count += ACT_ADVANCE + act_delta; if (new.act_count > ACT_MAX) new.act_count = ACT_MAX; } else { new.act_count -= min(new.act_count, ACT_DECLINE); } if (new.act_count > 0) { /* * Adjust the activation count and keep the page * in the active queue. The count might be left * unchanged if it is saturated. The page may * have been moved to a different queue since we * started the scan, in which case we move it * back. */ ps_delta = 0; if (old.queue != PQ_ACTIVE) { new.flags &= ~PGA_QUEUE_OP_MASK; new.flags |= PGA_REQUEUE; new.queue = PQ_ACTIVE; } } else { /* * When not short for inactive pages, let dirty * pages go through the inactive queue before * moving to the laundry queue. This gives them * some extra time to be reactivated, * potentially avoiding an expensive pageout. * However, during a page shortage, the inactive * queue is necessarily small, and so dirty * pages would only spend a trivial amount of * time in the inactive queue. Therefore, we * might as well place them directly in the * laundry queue to reduce queuing overhead. * * Calling vm_page_test_dirty() here would * require acquisition of the object's write * lock. However, during a page shortage, * directing dirty pages into the laundry queue * is only an optimization and not a * requirement. Therefore, we simply rely on * the opportunistic updates to the page's dirty * field by the pmap. */ if (page_shortage <= 0) { nqueue = PQ_INACTIVE; ps_delta = 0; } else if (m->dirty == 0) { nqueue = PQ_INACTIVE; ps_delta = act_scan_laundry_weight; } else { nqueue = PQ_LAUNDRY; ps_delta = 1; } new.flags &= ~PGA_QUEUE_OP_MASK; new.flags |= PGA_REQUEUE; new.queue = nqueue; } } while (!vm_page_pqstate_commit(m, &old, new)); page_shortage -= ps_delta; } vm_pagequeue_lock(pq); TAILQ_REMOVE(&pq->pq_pl, &vmd->vmd_clock[0], plinks.q); TAILQ_INSERT_AFTER(&pq->pq_pl, marker, &vmd->vmd_clock[0], plinks.q); vm_pageout_end_scan(&ss); vm_pagequeue_unlock(pq); } static int vm_pageout_reinsert_inactive_page(struct vm_pagequeue *pq, vm_page_t marker, vm_page_t m) { vm_page_astate_t as; vm_pagequeue_assert_locked(pq); as = vm_page_astate_load(m); if (as.queue != PQ_INACTIVE || (as.flags & PGA_ENQUEUED) != 0) return (0); vm_page_aflag_set(m, PGA_ENQUEUED); TAILQ_INSERT_BEFORE(marker, m, plinks.q); return (1); } /* * Re-add stuck pages to the inactive queue. We will examine them again * during the next scan. If the queue state of a page has changed since * it was physically removed from the page queue in * vm_pageout_collect_batch(), don't do anything with that page. */ static void vm_pageout_reinsert_inactive(struct scan_state *ss, struct vm_batchqueue *bq, vm_page_t m) { struct vm_pagequeue *pq; vm_page_t marker; int delta; delta = 0; marker = ss->marker; pq = ss->pq; if (m != NULL) { if (vm_batchqueue_insert(bq, m)) return; vm_pagequeue_lock(pq); delta += vm_pageout_reinsert_inactive_page(pq, marker, m); } else vm_pagequeue_lock(pq); while ((m = vm_batchqueue_pop(bq)) != NULL) delta += vm_pageout_reinsert_inactive_page(pq, marker, m); vm_pagequeue_cnt_add(pq, delta); vm_pagequeue_unlock(pq); vm_batchqueue_init(bq); } /* * Attempt to reclaim the requested number of pages from the inactive queue. * Returns true if the shortage was addressed. */ static int vm_pageout_scan_inactive(struct vm_domain *vmd, int shortage, int *addl_shortage) { struct scan_state ss; struct vm_batchqueue rq; vm_page_t m, marker; struct vm_pagequeue *pq; vm_object_t object; vm_page_astate_t old, new; int act_delta, addl_page_shortage, deficit, page_shortage, refs; int starting_page_shortage; /* * The addl_page_shortage is an estimate of the number of temporarily * stuck pages in the inactive queue. In other words, the * number of pages from the inactive count that should be * discounted in setting the target for the active queue scan. */ addl_page_shortage = 0; /* * vmd_pageout_deficit counts the number of pages requested in * allocations that failed because of a free page shortage. We assume * that the allocations will be reattempted and thus include the deficit * in our scan target. */ deficit = atomic_readandclear_int(&vmd->vmd_pageout_deficit); starting_page_shortage = page_shortage = shortage + deficit; object = NULL; vm_batchqueue_init(&rq); /* * Start scanning the inactive queue for pages that we can free. The * scan will stop when we reach the target or we have scanned the * entire queue. (Note that m->a.act_count is not used to make * decisions for the inactive queue, only for the active queue.) */ marker = &vmd->vmd_markers[PQ_INACTIVE]; pq = &vmd->vmd_pagequeues[PQ_INACTIVE]; vm_pagequeue_lock(pq); vm_pageout_init_scan(&ss, pq, marker, NULL, pq->pq_cnt); while (page_shortage > 0 && (m = vm_pageout_next(&ss, true)) != NULL) { KASSERT((m->flags & PG_MARKER) == 0, ("marker page %p was dequeued", m)); /* * Don't touch a page that was removed from the queue after the * page queue lock was released. Otherwise, ensure that any * pending queue operations, such as dequeues for wired pages, * are handled. */ if (vm_pageout_defer(m, PQ_INACTIVE, false)) continue; /* * Lock the page's object. */ if (object == NULL || object != m->object) { if (object != NULL) VM_OBJECT_WUNLOCK(object); - object = (vm_object_t)atomic_load_ptr(&m->object); + object = atomic_load_ptr(&m->object); if (__predict_false(object == NULL)) /* The page is being freed by another thread. */ continue; /* Depends on type-stability. */ VM_OBJECT_WLOCK(object); if (__predict_false(m->object != object)) { VM_OBJECT_WUNLOCK(object); object = NULL; goto reinsert; } } if (vm_page_tryxbusy(m) == 0) { /* * Don't mess with busy pages. Leave them at * the front of the queue. Most likely, they * are being paged out and will leave the * queue shortly after the scan finishes. So, * they ought to be discounted from the * inactive count. */ addl_page_shortage++; goto reinsert; } /* Deferred free of swap space. */ if ((m->a.flags & PGA_SWAP_FREE) != 0) vm_pager_page_unswapped(m); /* * Check for wirings now that we hold the object lock and have * exclusively busied the page. If the page is mapped, it may * still be wired by pmap lookups. The call to * vm_page_try_remove_all() below atomically checks for such * wirings and removes mappings. If the page is unmapped, the * wire count is guaranteed not to increase after this check. */ if (__predict_false(vm_page_wired(m))) goto skip_page; /* * Invalid pages can be easily freed. They cannot be * mapped, vm_page_free() asserts this. */ if (vm_page_none_valid(m)) goto free_page; refs = object->ref_count != 0 ? pmap_ts_referenced(m) : 0; for (old = vm_page_astate_load(m);;) { /* * Check to see if the page has been removed from the * queue since the first such check. Leave it alone if * so, discarding any references collected by * pmap_ts_referenced(). */ if (__predict_false(_vm_page_queue(old) == PQ_NONE)) goto skip_page; new = old; act_delta = refs; if ((old.flags & PGA_REFERENCED) != 0) { new.flags &= ~PGA_REFERENCED; act_delta++; } if (act_delta == 0) { ; } else if (object->ref_count != 0) { /* * Increase the activation count if the * page was referenced while in the * inactive queue. This makes it less * likely that the page will be returned * prematurely to the inactive queue. */ new.act_count += ACT_ADVANCE + act_delta; if (new.act_count > ACT_MAX) new.act_count = ACT_MAX; new.flags &= ~PGA_QUEUE_OP_MASK; new.flags |= PGA_REQUEUE; new.queue = PQ_ACTIVE; if (!vm_page_pqstate_commit(m, &old, new)) continue; VM_CNT_INC(v_reactivated); goto skip_page; } else if ((object->flags & OBJ_DEAD) == 0) { new.queue = PQ_INACTIVE; new.flags |= PGA_REQUEUE; if (!vm_page_pqstate_commit(m, &old, new)) continue; goto skip_page; } break; } /* * If the page appears to be clean at the machine-independent * layer, then remove all of its mappings from the pmap in * anticipation of freeing it. If, however, any of the page's * mappings allow write access, then the page may still be * modified until the last of those mappings are removed. */ if (object->ref_count != 0) { vm_page_test_dirty(m); if (m->dirty == 0 && !vm_page_try_remove_all(m)) goto skip_page; } /* * Clean pages can be freed, but dirty pages must be sent back * to the laundry, unless they belong to a dead object. * Requeueing dirty pages from dead objects is pointless, as * they are being paged out and freed by the thread that * destroyed the object. */ if (m->dirty == 0) { free_page: /* * Now we are guaranteed that no other threads are * manipulating the page, check for a last-second * reference that would save it from doom. */ if (vm_pageout_defer(m, PQ_INACTIVE, false)) goto skip_page; /* * Because we dequeued the page and have already checked * for pending dequeue and enqueue requests, we can * safely disassociate the page from the inactive queue * without holding the queue lock. */ m->a.queue = PQ_NONE; vm_page_free(m); page_shortage--; continue; } if ((object->flags & OBJ_DEAD) == 0) vm_page_launder(m); skip_page: vm_page_xunbusy(m); continue; reinsert: vm_pageout_reinsert_inactive(&ss, &rq, m); } if (object != NULL) VM_OBJECT_WUNLOCK(object); vm_pageout_reinsert_inactive(&ss, &rq, NULL); vm_pageout_reinsert_inactive(&ss, &ss.bq, NULL); vm_pagequeue_lock(pq); vm_pageout_end_scan(&ss); vm_pagequeue_unlock(pq); VM_CNT_ADD(v_dfree, starting_page_shortage - page_shortage); /* * Wake up the laundry thread so that it can perform any needed * laundering. If we didn't meet our target, we're in shortfall and * need to launder more aggressively. If PQ_LAUNDRY is empty and no * swap devices are configured, the laundry thread has no work to do, so * don't bother waking it up. * * The laundry thread uses the number of inactive queue scans elapsed * since the last laundering to determine whether to launder again, so * keep count. */ if (starting_page_shortage > 0) { pq = &vmd->vmd_pagequeues[PQ_LAUNDRY]; vm_pagequeue_lock(pq); if (vmd->vmd_laundry_request == VM_LAUNDRY_IDLE && (pq->pq_cnt > 0 || atomic_load_acq_int(&swapdev_enabled))) { if (page_shortage > 0) { vmd->vmd_laundry_request = VM_LAUNDRY_SHORTFALL; VM_CNT_INC(v_pdshortfalls); } else if (vmd->vmd_laundry_request != VM_LAUNDRY_SHORTFALL) vmd->vmd_laundry_request = VM_LAUNDRY_BACKGROUND; wakeup(&vmd->vmd_laundry_request); } vmd->vmd_clean_pages_freed += starting_page_shortage - page_shortage; vm_pagequeue_unlock(pq); } /* * Wakeup the swapout daemon if we didn't free the targeted number of * pages. */ if (page_shortage > 0) vm_swapout_run(); /* * If the inactive queue scan fails repeatedly to meet its * target, kill the largest process. */ vm_pageout_mightbe_oom(vmd, page_shortage, starting_page_shortage); /* * Reclaim pages by swapping out idle processes, if configured to do so. */ vm_swapout_run_idle(); /* * See the description of addl_page_shortage above. */ *addl_shortage = addl_page_shortage + deficit; return (page_shortage <= 0); } static int vm_pageout_oom_vote; /* * The pagedaemon threads randlomly select one to perform the * OOM. Trying to kill processes before all pagedaemons * failed to reach free target is premature. */ static void vm_pageout_mightbe_oom(struct vm_domain *vmd, int page_shortage, int starting_page_shortage) { int old_vote; if (starting_page_shortage <= 0 || starting_page_shortage != page_shortage) vmd->vmd_oom_seq = 0; else vmd->vmd_oom_seq++; if (vmd->vmd_oom_seq < vm_pageout_oom_seq) { if (vmd->vmd_oom) { vmd->vmd_oom = FALSE; atomic_subtract_int(&vm_pageout_oom_vote, 1); } return; } /* * Do not follow the call sequence until OOM condition is * cleared. */ vmd->vmd_oom_seq = 0; if (vmd->vmd_oom) return; vmd->vmd_oom = TRUE; old_vote = atomic_fetchadd_int(&vm_pageout_oom_vote, 1); if (old_vote != vm_ndomains - 1) return; /* * The current pagedaemon thread is the last in the quorum to * start OOM. Initiate the selection and signaling of the * victim. */ vm_pageout_oom(VM_OOM_MEM); /* * After one round of OOM terror, recall our vote. On the * next pass, current pagedaemon would vote again if the low * memory condition is still there, due to vmd_oom being * false. */ vmd->vmd_oom = FALSE; atomic_subtract_int(&vm_pageout_oom_vote, 1); } /* * The OOM killer is the page daemon's action of last resort when * memory allocation requests have been stalled for a prolonged period * of time because it cannot reclaim memory. This function computes * the approximate number of physical pages that could be reclaimed if * the specified address space is destroyed. * * Private, anonymous memory owned by the address space is the * principal resource that we expect to recover after an OOM kill. * Since the physical pages mapped by the address space's COW entries * are typically shared pages, they are unlikely to be released and so * they are not counted. * * To get to the point where the page daemon runs the OOM killer, its * efforts to write-back vnode-backed pages may have stalled. This * could be caused by a memory allocation deadlock in the write path * that might be resolved by an OOM kill. Therefore, physical pages * belonging to vnode-backed objects are counted, because they might * be freed without being written out first if the address space holds * the last reference to an unlinked vnode. * * Similarly, physical pages belonging to OBJT_PHYS objects are * counted because the address space might hold the last reference to * the object. */ static long vm_pageout_oom_pagecount(struct vmspace *vmspace) { vm_map_t map; vm_map_entry_t entry; vm_object_t obj; long res; map = &vmspace->vm_map; KASSERT(!map->system_map, ("system map")); sx_assert(&map->lock, SA_LOCKED); res = 0; VM_MAP_ENTRY_FOREACH(entry, map) { if ((entry->eflags & MAP_ENTRY_IS_SUB_MAP) != 0) continue; obj = entry->object.vm_object; if (obj == NULL) continue; if ((entry->eflags & MAP_ENTRY_NEEDS_COPY) != 0 && obj->ref_count != 1) continue; switch (obj->type) { case OBJT_DEFAULT: case OBJT_SWAP: case OBJT_PHYS: case OBJT_VNODE: res += obj->resident_page_count; break; } } return (res); } static int vm_oom_ratelim_last; static int vm_oom_pf_secs = 10; SYSCTL_INT(_vm, OID_AUTO, oom_pf_secs, CTLFLAG_RWTUN, &vm_oom_pf_secs, 0, ""); static struct mtx vm_oom_ratelim_mtx; void vm_pageout_oom(int shortage) { struct proc *p, *bigproc; vm_offset_t size, bigsize; struct thread *td; struct vmspace *vm; int now; bool breakout; /* * For OOM requests originating from vm_fault(), there is a high * chance that a single large process faults simultaneously in * several threads. Also, on an active system running many * processes of middle-size, like buildworld, all of them * could fault almost simultaneously as well. * * To avoid killing too many processes, rate-limit OOMs * initiated by vm_fault() time-outs on the waits for free * pages. */ mtx_lock(&vm_oom_ratelim_mtx); now = ticks; if (shortage == VM_OOM_MEM_PF && (u_int)(now - vm_oom_ratelim_last) < hz * vm_oom_pf_secs) { mtx_unlock(&vm_oom_ratelim_mtx); return; } vm_oom_ratelim_last = now; mtx_unlock(&vm_oom_ratelim_mtx); /* * We keep the process bigproc locked once we find it to keep anyone * from messing with it; however, there is a possibility of * deadlock if process B is bigproc and one of its child processes * attempts to propagate a signal to B while we are waiting for A's * lock while walking this list. To avoid this, we don't block on * the process lock but just skip a process if it is already locked. */ bigproc = NULL; bigsize = 0; sx_slock(&allproc_lock); FOREACH_PROC_IN_SYSTEM(p) { PROC_LOCK(p); /* * If this is a system, protected or killed process, skip it. */ if (p->p_state != PRS_NORMAL || (p->p_flag & (P_INEXEC | P_PROTECTED | P_SYSTEM | P_WEXIT)) != 0 || p->p_pid == 1 || P_KILLED(p) || (p->p_pid < 48 && swap_pager_avail != 0)) { PROC_UNLOCK(p); continue; } /* * If the process is in a non-running type state, * don't touch it. Check all the threads individually. */ breakout = false; FOREACH_THREAD_IN_PROC(p, td) { thread_lock(td); if (!TD_ON_RUNQ(td) && !TD_IS_RUNNING(td) && !TD_IS_SLEEPING(td) && !TD_IS_SUSPENDED(td) && !TD_IS_SWAPPED(td)) { thread_unlock(td); breakout = true; break; } thread_unlock(td); } if (breakout) { PROC_UNLOCK(p); continue; } /* * get the process size */ vm = vmspace_acquire_ref(p); if (vm == NULL) { PROC_UNLOCK(p); continue; } _PHOLD_LITE(p); PROC_UNLOCK(p); sx_sunlock(&allproc_lock); if (!vm_map_trylock_read(&vm->vm_map)) { vmspace_free(vm); sx_slock(&allproc_lock); PRELE(p); continue; } size = vmspace_swap_count(vm); if (shortage == VM_OOM_MEM || shortage == VM_OOM_MEM_PF) size += vm_pageout_oom_pagecount(vm); vm_map_unlock_read(&vm->vm_map); vmspace_free(vm); sx_slock(&allproc_lock); /* * If this process is bigger than the biggest one, * remember it. */ if (size > bigsize) { if (bigproc != NULL) PRELE(bigproc); bigproc = p; bigsize = size; } else { PRELE(p); } } sx_sunlock(&allproc_lock); if (bigproc != NULL) { if (vm_panic_on_oom != 0 && --vm_panic_on_oom == 0) panic("out of swap space"); PROC_LOCK(bigproc); killproc(bigproc, "out of swap space"); sched_nice(bigproc, PRIO_MIN); _PRELE(bigproc); PROC_UNLOCK(bigproc); } } /* * Signal a free page shortage to subsystems that have registered an event * handler. Reclaim memory from UMA in the event of a severe shortage. * Return true if the free page count should be re-evaluated. */ static bool vm_pageout_lowmem(void) { static int lowmem_ticks = 0; int last; bool ret; ret = false; last = atomic_load_int(&lowmem_ticks); while ((u_int)(ticks - last) / hz >= lowmem_period) { if (atomic_fcmpset_int(&lowmem_ticks, &last, ticks) == 0) continue; /* * Decrease registered cache sizes. */ SDT_PROBE0(vm, , , vm__lowmem_scan); EVENTHANDLER_INVOKE(vm_lowmem, VM_LOW_PAGES); /* * We do this explicitly after the caches have been * drained above. */ uma_reclaim(UMA_RECLAIM_TRIM); ret = true; break; } /* * Kick off an asynchronous reclaim of cached memory if one of the * page daemons is failing to keep up with demand. Use the "severe" * threshold instead of "min" to ensure that we do not blow away the * caches if a subset of the NUMA domains are depleted by kernel memory * allocations; the domainset iterators automatically skip domains * below the "min" threshold on the first pass. * * UMA reclaim worker has its own rate-limiting mechanism, so don't * worry about kicking it too often. */ if (vm_page_count_severe()) uma_reclaim_wakeup(); return (ret); } static void vm_pageout_worker(void *arg) { struct vm_domain *vmd; u_int ofree; int addl_shortage, domain, shortage; bool target_met; domain = (uintptr_t)arg; vmd = VM_DOMAIN(domain); shortage = 0; target_met = true; /* * XXXKIB It could be useful to bind pageout daemon threads to * the cores belonging to the domain, from which vm_page_array * is allocated. */ KASSERT(vmd->vmd_segs != 0, ("domain without segments")); vmd->vmd_last_active_scan = ticks; /* * The pageout daemon worker is never done, so loop forever. */ while (TRUE) { vm_domain_pageout_lock(vmd); /* * We need to clear wanted before we check the limits. This * prevents races with wakers who will check wanted after they * reach the limit. */ atomic_store_int(&vmd->vmd_pageout_wanted, 0); /* * Might the page daemon need to run again? */ if (vm_paging_needed(vmd, vmd->vmd_free_count)) { /* * Yes. If the scan failed to produce enough free * pages, sleep uninterruptibly for some time in the * hope that the laundry thread will clean some pages. */ vm_domain_pageout_unlock(vmd); if (!target_met) pause("pwait", hz / VM_INACT_SCAN_RATE); } else { /* * No, sleep until the next wakeup or until pages * need to have their reference stats updated. */ if (mtx_sleep(&vmd->vmd_pageout_wanted, vm_domain_pageout_lockptr(vmd), PDROP | PVM, "psleep", hz / VM_INACT_SCAN_RATE) == 0) VM_CNT_INC(v_pdwakeups); } /* Prevent spurious wakeups by ensuring that wanted is set. */ atomic_store_int(&vmd->vmd_pageout_wanted, 1); /* * Use the controller to calculate how many pages to free in * this interval, and scan the inactive queue. If the lowmem * handlers appear to have freed up some pages, subtract the * difference from the inactive queue scan target. */ shortage = pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); if (shortage > 0) { ofree = vmd->vmd_free_count; if (vm_pageout_lowmem() && vmd->vmd_free_count > ofree) shortage -= min(vmd->vmd_free_count - ofree, (u_int)shortage); target_met = vm_pageout_scan_inactive(vmd, shortage, &addl_shortage); } else addl_shortage = 0; /* * Scan the active queue. A positive value for shortage * indicates that we must aggressively deactivate pages to avoid * a shortfall. */ shortage = vm_pageout_active_target(vmd) + addl_shortage; vm_pageout_scan_active(vmd, shortage); } } /* * Initialize basic pageout daemon settings. See the comment above the * definition of vm_domain for some explanation of how these thresholds are * used. */ static void vm_pageout_init_domain(int domain) { struct vm_domain *vmd; struct sysctl_oid *oid; vmd = VM_DOMAIN(domain); vmd->vmd_interrupt_free_min = 2; /* * v_free_reserved needs to include enough for the largest * swap pager structures plus enough for any pv_entry structs * when paging. */ vmd->vmd_pageout_free_min = 2 * MAXBSIZE / PAGE_SIZE + vmd->vmd_interrupt_free_min; vmd->vmd_free_reserved = vm_pageout_page_count + vmd->vmd_pageout_free_min + vmd->vmd_page_count / 768; vmd->vmd_free_min = vmd->vmd_page_count / 200; vmd->vmd_free_severe = vmd->vmd_free_min / 2; vmd->vmd_free_target = 4 * vmd->vmd_free_min + vmd->vmd_free_reserved; vmd->vmd_free_min += vmd->vmd_free_reserved; vmd->vmd_free_severe += vmd->vmd_free_reserved; vmd->vmd_inactive_target = (3 * vmd->vmd_free_target) / 2; if (vmd->vmd_inactive_target > vmd->vmd_free_count / 3) vmd->vmd_inactive_target = vmd->vmd_free_count / 3; /* * Set the default wakeup threshold to be 10% below the paging * target. This keeps the steady state out of shortfall. */ vmd->vmd_pageout_wakeup_thresh = (vmd->vmd_free_target / 10) * 9; /* * Target amount of memory to move out of the laundry queue during a * background laundering. This is proportional to the amount of system * memory. */ vmd->vmd_background_launder_target = (vmd->vmd_free_target - vmd->vmd_free_min) / 10; /* Initialize the pageout daemon pid controller. */ pidctrl_init(&vmd->vmd_pid, hz / VM_INACT_SCAN_RATE, vmd->vmd_free_target, PIDCTRL_BOUND, PIDCTRL_KPD, PIDCTRL_KID, PIDCTRL_KDD); oid = SYSCTL_ADD_NODE(NULL, SYSCTL_CHILDREN(vmd->vmd_oid), OID_AUTO, "pidctrl", CTLFLAG_RD, NULL, ""); pidctrl_init_sysctl(&vmd->vmd_pid, SYSCTL_CHILDREN(oid)); } static void vm_pageout_init(void) { u_int freecount; int i; /* * Initialize some paging parameters. */ if (vm_cnt.v_page_count < 2000) vm_pageout_page_count = 8; freecount = 0; for (i = 0; i < vm_ndomains; i++) { struct vm_domain *vmd; vm_pageout_init_domain(i); vmd = VM_DOMAIN(i); vm_cnt.v_free_reserved += vmd->vmd_free_reserved; vm_cnt.v_free_target += vmd->vmd_free_target; vm_cnt.v_free_min += vmd->vmd_free_min; vm_cnt.v_inactive_target += vmd->vmd_inactive_target; vm_cnt.v_pageout_free_min += vmd->vmd_pageout_free_min; vm_cnt.v_interrupt_free_min += vmd->vmd_interrupt_free_min; vm_cnt.v_free_severe += vmd->vmd_free_severe; freecount += vmd->vmd_free_count; } /* * Set interval in seconds for active scan. We want to visit each * page at least once every ten minutes. This is to prevent worst * case paging behaviors with stale active LRU. */ if (vm_pageout_update_period == 0) vm_pageout_update_period = 600; if (vm_page_max_user_wired == 0) vm_page_max_user_wired = freecount / 3; } /* * vm_pageout is the high level pageout daemon. */ static void vm_pageout(void) { struct proc *p; struct thread *td; int error, first, i; p = curproc; td = curthread; mtx_init(&vm_oom_ratelim_mtx, "vmoomr", NULL, MTX_DEF); swap_pager_swap_init(); for (first = -1, i = 0; i < vm_ndomains; i++) { if (VM_DOMAIN_EMPTY(i)) { if (bootverbose) printf("domain %d empty; skipping pageout\n", i); continue; } if (first == -1) first = i; else { error = kthread_add(vm_pageout_worker, (void *)(uintptr_t)i, p, NULL, 0, 0, "dom%d", i); if (error != 0) panic("starting pageout for domain %d: %d\n", i, error); } error = kthread_add(vm_pageout_laundry_worker, (void *)(uintptr_t)i, p, NULL, 0, 0, "laundry: dom%d", i); if (error != 0) panic("starting laundry for domain %d: %d", i, error); } error = kthread_add(uma_reclaim_worker, NULL, p, NULL, 0, 0, "uma"); if (error != 0) panic("starting uma_reclaim helper, error %d\n", error); snprintf(td->td_name, sizeof(td->td_name), "dom%d", first); vm_pageout_worker((void *)(uintptr_t)first); } /* * Perform an advisory wakeup of the page daemon. */ void pagedaemon_wakeup(int domain) { struct vm_domain *vmd; vmd = VM_DOMAIN(domain); vm_domain_pageout_assert_unlocked(vmd); if (curproc == pageproc) return; if (atomic_fetchadd_int(&vmd->vmd_pageout_wanted, 1) == 0) { vm_domain_pageout_lock(vmd); atomic_store_int(&vmd->vmd_pageout_wanted, 1); wakeup(&vmd->vmd_pageout_wanted); vm_domain_pageout_unlock(vmd); } } Index: projects/clang1000-import/sys/x86/x86/mp_x86.c =================================================================== --- projects/clang1000-import/sys/x86/x86/mp_x86.c (revision 357965) +++ projects/clang1000-import/sys/x86/x86/mp_x86.c (revision 357966) @@ -1,1857 +1,1857 @@ /*- * Copyright (c) 1996, by Steve Passe * Copyright (c) 2003, by Peter Wemm * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. The name of the developer may NOT be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #ifdef __i386__ #include "opt_apic.h" #endif #include "opt_cpu.h" #include "opt_ddb.h" #include "opt_kstack_pages.h" #include "opt_pmap.h" #include "opt_sched.h" #include "opt_smp.h" #include "opt_stack.h" #include #include #include #include /* cngetc() */ #include #include #ifdef GPROF #include #endif #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static MALLOC_DEFINE(M_CPUS, "cpus", "CPU items"); /* lock region used by kernel profiling */ int mcount_lock; int mp_naps; /* # of Applications processors */ int boot_cpu_id = -1; /* designated BSP */ /* AP uses this during bootstrap. Do not staticize. */ char *bootSTK; int bootAP; /* Free these after use */ void *bootstacks[MAXCPU]; void *dpcpu; struct pcb stoppcbs[MAXCPU]; struct susppcb **susppcbs; #ifdef COUNT_IPIS /* Interrupt counts. */ static u_long *ipi_preempt_counts[MAXCPU]; static u_long *ipi_ast_counts[MAXCPU]; u_long *ipi_invltlb_counts[MAXCPU]; u_long *ipi_invlrng_counts[MAXCPU]; u_long *ipi_invlpg_counts[MAXCPU]; u_long *ipi_invlcache_counts[MAXCPU]; u_long *ipi_rendezvous_counts[MAXCPU]; static u_long *ipi_hardclock_counts[MAXCPU]; #endif /* Default cpu_ops implementation. */ struct cpu_ops cpu_ops; /* * Local data and functions. */ static volatile cpuset_t ipi_stop_nmi_pending; volatile cpuset_t resuming_cpus; volatile cpuset_t toresume_cpus; /* used to hold the AP's until we are ready to release them */ struct mtx ap_boot_mtx; /* Set to 1 once we're ready to let the APs out of the pen. */ volatile int aps_ready = 0; /* * Store data from cpu_add() until later in the boot when we actually setup * the APs. */ struct cpu_info *cpu_info; int *apic_cpuids; int cpu_apic_ids[MAXCPU]; _Static_assert(MAXCPU <= MAX_APIC_ID, "MAXCPU cannot be larger that MAX_APIC_ID"); _Static_assert(xAPIC_MAX_APIC_ID <= MAX_APIC_ID, "xAPIC_MAX_APIC_ID cannot be larger that MAX_APIC_ID"); static void release_aps(void *dummy); static void cpustop_handler_post(u_int cpu); static int hyperthreading_allowed = 1; SYSCTL_INT(_machdep, OID_AUTO, hyperthreading_allowed, CTLFLAG_RDTUN, &hyperthreading_allowed, 0, "Use Intel HTT logical CPUs"); static int hyperthreading_intr_allowed = 0; SYSCTL_INT(_machdep, OID_AUTO, hyperthreading_intr_allowed, CTLFLAG_RDTUN, &hyperthreading_intr_allowed, 0, "Allow interrupts on HTT logical CPUs"); static struct topo_node topo_root; static int pkg_id_shift; static int node_id_shift; static int core_id_shift; static int disabled_cpus; struct cache_info { int id_shift; int present; } static caches[MAX_CACHE_LEVELS]; unsigned int boot_address; static bool stop_mwait = false; SYSCTL_BOOL(_machdep, OID_AUTO, stop_mwait, CTLFLAG_RWTUN, &stop_mwait, 0, "Use MONITOR/MWAIT when stopping CPU, if available"); #define MiB(v) (v ## ULL << 20) void mem_range_AP_init(void) { if (mem_range_softc.mr_op && mem_range_softc.mr_op->initAP) mem_range_softc.mr_op->initAP(&mem_range_softc); } /* * Round up to the next power of two, if necessary, and then * take log2. * Returns -1 if argument is zero. */ static __inline int mask_width(u_int x) { return (fls(x << (1 - powerof2(x))) - 1); } /* * Add a cache level to the cache topology description. */ static int add_deterministic_cache(int type, int level, int share_count) { if (type == 0) return (0); if (type > 3) { printf("unexpected cache type %d\n", type); return (1); } if (type == 2) /* ignore instruction cache */ return (1); if (level == 0 || level > MAX_CACHE_LEVELS) { printf("unexpected cache level %d\n", type); return (1); } if (caches[level - 1].present) { printf("WARNING: multiple entries for L%u data cache\n", level); printf("%u => %u\n", caches[level - 1].id_shift, mask_width(share_count)); } caches[level - 1].id_shift = mask_width(share_count); caches[level - 1].present = 1; if (caches[level - 1].id_shift > pkg_id_shift) { printf("WARNING: L%u data cache covers more " "APIC IDs than a package (%u > %u)\n", level, caches[level - 1].id_shift, pkg_id_shift); caches[level - 1].id_shift = pkg_id_shift; } if (caches[level - 1].id_shift < core_id_shift) { printf("WARNING: L%u data cache covers fewer " "APIC IDs than a core (%u < %u)\n", level, caches[level - 1].id_shift, core_id_shift); caches[level - 1].id_shift = core_id_shift; } return (1); } /* * Determine topology of processing units and caches for AMD CPUs. * See: * - AMD CPUID Specification (Publication # 25481) * - BKDG for AMD NPT Family 0Fh Processors (Publication # 32559) * - BKDG For AMD Family 10h Processors (Publication # 31116) * - BKDG For AMD Family 15h Models 00h-0Fh Processors (Publication # 42301) * - BKDG For AMD Family 16h Models 00h-0Fh Processors (Publication # 48751) * - PPR For AMD Family 17h Models 00h-0Fh Processors (Publication # 54945) */ static void topo_probe_amd(void) { u_int p[4]; uint64_t v; int level; int nodes_per_socket; int share_count; int type; int i; /* No multi-core capability. */ if ((amd_feature2 & AMDID2_CMP) == 0) return; /* For families 10h and newer. */ pkg_id_shift = (cpu_procinfo2 & AMDID_COREID_SIZE) >> AMDID_COREID_SIZE_SHIFT; /* For 0Fh family. */ if (pkg_id_shift == 0) pkg_id_shift = mask_width((cpu_procinfo2 & AMDID_CMP_CORES) + 1); /* * Families prior to 16h define the following value as * cores per compute unit and we don't really care about the AMD * compute units at the moment. Perhaps we should treat them as * cores and cores within the compute units as hardware threads, * but that's up for debate. * Later families define the value as threads per compute unit, * so we are following AMD's nomenclature here. */ if ((amd_feature2 & AMDID2_TOPOLOGY) != 0 && CPUID_TO_FAMILY(cpu_id) >= 0x16) { cpuid_count(0x8000001e, 0, p); share_count = ((p[1] >> 8) & 0xff) + 1; core_id_shift = mask_width(share_count); /* * For Zen (17h), gather Nodes per Processor. Each node is a * Zeppelin die; TR and EPYC CPUs will have multiple dies per * package. Communication latency between dies is higher than * within them. */ nodes_per_socket = ((p[2] >> 8) & 0x7) + 1; node_id_shift = pkg_id_shift - mask_width(nodes_per_socket); } if ((amd_feature2 & AMDID2_TOPOLOGY) != 0) { for (i = 0; ; i++) { cpuid_count(0x8000001d, i, p); type = p[0] & 0x1f; level = (p[0] >> 5) & 0x7; share_count = 1 + ((p[0] >> 14) & 0xfff); if (!add_deterministic_cache(type, level, share_count)) break; } } else { if (cpu_exthigh >= 0x80000005) { cpuid_count(0x80000005, 0, p); if (((p[2] >> 24) & 0xff) != 0) { caches[0].id_shift = 0; caches[0].present = 1; } } if (cpu_exthigh >= 0x80000006) { cpuid_count(0x80000006, 0, p); if (((p[2] >> 16) & 0xffff) != 0) { caches[1].id_shift = 0; caches[1].present = 1; } if (((p[3] >> 18) & 0x3fff) != 0) { nodes_per_socket = 1; if ((amd_feature2 & AMDID2_NODE_ID) != 0) { /* * Handle multi-node processors that * have multiple chips, each with its * own L3 cache, on the same die. */ v = rdmsr(0xc001100c); nodes_per_socket = 1 + ((v >> 3) & 0x7); } caches[2].id_shift = pkg_id_shift - mask_width(nodes_per_socket); caches[2].present = 1; } } } } /* * Determine topology of processing units for Intel CPUs * using CPUID Leaf 1 and Leaf 4, if supported. * See: * - Intel 64 Architecture Processor Topology Enumeration * - Intel 64 and IA-32 ArchitecturesSoftware Developer’s Manual, * Volume 3A: System Programming Guide, PROGRAMMING CONSIDERATIONS * FOR HARDWARE MULTI-THREADING CAPABLE PROCESSORS */ static void topo_probe_intel_0x4(void) { u_int p[4]; int max_cores; int max_logical; /* Both zero and one here mean one logical processor per package. */ max_logical = (cpu_feature & CPUID_HTT) != 0 ? (cpu_procinfo & CPUID_HTT_CORES) >> 16 : 1; if (max_logical <= 1) return; if (cpu_high >= 0x4) { cpuid_count(0x04, 0, p); max_cores = ((p[0] >> 26) & 0x3f) + 1; } else max_cores = 1; core_id_shift = mask_width(max_logical/max_cores); KASSERT(core_id_shift >= 0, ("intel topo: max_cores > max_logical\n")); pkg_id_shift = core_id_shift + mask_width(max_cores); } /* * Determine topology of processing units for Intel CPUs * using CPUID Leaf 11, if supported. * See: * - Intel 64 Architecture Processor Topology Enumeration * - Intel 64 and IA-32 ArchitecturesSoftware Developer’s Manual, * Volume 3A: System Programming Guide, PROGRAMMING CONSIDERATIONS * FOR HARDWARE MULTI-THREADING CAPABLE PROCESSORS */ static void topo_probe_intel_0xb(void) { u_int p[4]; int bits; int type; int i; /* Fall back if CPU leaf 11 doesn't really exist. */ cpuid_count(0x0b, 0, p); if (p[1] == 0) { topo_probe_intel_0x4(); return; } /* We only support three levels for now. */ for (i = 0; ; i++) { cpuid_count(0x0b, i, p); bits = p[0] & 0x1f; type = (p[2] >> 8) & 0xff; if (type == 0) break; /* TODO: check for duplicate (re-)assignment */ if (type == CPUID_TYPE_SMT) core_id_shift = bits; else if (type == CPUID_TYPE_CORE) pkg_id_shift = bits; else printf("unknown CPU level type %d\n", type); } if (pkg_id_shift < core_id_shift) { printf("WARNING: core covers more APIC IDs than a package\n"); core_id_shift = pkg_id_shift; } } /* * Determine topology of caches for Intel CPUs. * See: * - Intel 64 Architecture Processor Topology Enumeration * - Intel 64 and IA-32 Architectures Software Developer’s Manual * Volume 2A: Instruction Set Reference, A-M, * CPUID instruction */ static void topo_probe_intel_caches(void) { u_int p[4]; int level; int share_count; int type; int i; if (cpu_high < 0x4) { /* * Available cache level and sizes can be determined * via CPUID leaf 2, but that requires a huge table of hardcoded * values, so for now just assume L1 and L2 caches potentially * shared only by HTT processing units, if HTT is present. */ caches[0].id_shift = pkg_id_shift; caches[0].present = 1; caches[1].id_shift = pkg_id_shift; caches[1].present = 1; return; } for (i = 0; ; i++) { cpuid_count(0x4, i, p); type = p[0] & 0x1f; level = (p[0] >> 5) & 0x7; share_count = 1 + ((p[0] >> 14) & 0xfff); if (!add_deterministic_cache(type, level, share_count)) break; } } /* * Determine topology of processing units and caches for Intel CPUs. * See: * - Intel 64 Architecture Processor Topology Enumeration */ static void topo_probe_intel(void) { /* * Note that 0x1 <= cpu_high < 4 case should be * compatible with topo_probe_intel_0x4() logic when * CPUID.1:EBX[23:16] > 0 (cpu_cores will be 1) * or it should trigger the fallback otherwise. */ if (cpu_high >= 0xb) topo_probe_intel_0xb(); else if (cpu_high >= 0x1) topo_probe_intel_0x4(); topo_probe_intel_caches(); } /* * Topology information is queried only on BSP, on which this * code runs and for which it can query CPUID information. * Then topology is extrapolated on all packages using an * assumption that APIC ID to hardware component ID mapping is * homogenious. * That doesn't necesserily imply that the topology is uniform. */ void topo_probe(void) { static int cpu_topo_probed = 0; struct x86_topo_layer { int type; int subtype; int id_shift; } topo_layers[MAX_CACHE_LEVELS + 4]; struct topo_node *parent; struct topo_node *node; int layer; int nlayers; int node_id; int i; if (cpu_topo_probed) return; CPU_ZERO(&logical_cpus_mask); if (mp_ncpus <= 1) ; /* nothing */ else if (cpu_vendor_id == CPU_VENDOR_AMD || cpu_vendor_id == CPU_VENDOR_HYGON) topo_probe_amd(); else if (cpu_vendor_id == CPU_VENDOR_INTEL) topo_probe_intel(); KASSERT(pkg_id_shift >= core_id_shift, ("bug in APIC topology discovery")); nlayers = 0; bzero(topo_layers, sizeof(topo_layers)); topo_layers[nlayers].type = TOPO_TYPE_PKG; topo_layers[nlayers].id_shift = pkg_id_shift; if (bootverbose) printf("Package ID shift: %u\n", topo_layers[nlayers].id_shift); nlayers++; if (pkg_id_shift > node_id_shift && node_id_shift != 0) { topo_layers[nlayers].type = TOPO_TYPE_GROUP; topo_layers[nlayers].id_shift = node_id_shift; if (bootverbose) printf("Node ID shift: %u\n", topo_layers[nlayers].id_shift); nlayers++; } /* * Consider all caches to be within a package/chip * and "in front" of all sub-components like * cores and hardware threads. */ for (i = MAX_CACHE_LEVELS - 1; i >= 0; --i) { if (caches[i].present) { if (node_id_shift != 0) KASSERT(caches[i].id_shift <= node_id_shift, ("bug in APIC topology discovery")); KASSERT(caches[i].id_shift <= pkg_id_shift, ("bug in APIC topology discovery")); KASSERT(caches[i].id_shift >= core_id_shift, ("bug in APIC topology discovery")); topo_layers[nlayers].type = TOPO_TYPE_CACHE; topo_layers[nlayers].subtype = i + 1; topo_layers[nlayers].id_shift = caches[i].id_shift; if (bootverbose) printf("L%u cache ID shift: %u\n", topo_layers[nlayers].subtype, topo_layers[nlayers].id_shift); nlayers++; } } if (pkg_id_shift > core_id_shift) { topo_layers[nlayers].type = TOPO_TYPE_CORE; topo_layers[nlayers].id_shift = core_id_shift; if (bootverbose) printf("Core ID shift: %u\n", topo_layers[nlayers].id_shift); nlayers++; } topo_layers[nlayers].type = TOPO_TYPE_PU; topo_layers[nlayers].id_shift = 0; nlayers++; topo_init_root(&topo_root); for (i = 0; i <= max_apic_id; ++i) { if (!cpu_info[i].cpu_present) continue; parent = &topo_root; for (layer = 0; layer < nlayers; ++layer) { node_id = i >> topo_layers[layer].id_shift; parent = topo_add_node_by_hwid(parent, node_id, topo_layers[layer].type, topo_layers[layer].subtype); } } parent = &topo_root; for (layer = 0; layer < nlayers; ++layer) { node_id = boot_cpu_id >> topo_layers[layer].id_shift; node = topo_find_node_by_hwid(parent, node_id, topo_layers[layer].type, topo_layers[layer].subtype); topo_promote_child(node); parent = node; } cpu_topo_probed = 1; } /* * Assign logical CPU IDs to local APICs. */ void assign_cpu_ids(void) { struct topo_node *node; u_int smt_mask; int nhyper; smt_mask = (1u << core_id_shift) - 1; /* * Assign CPU IDs to local APIC IDs and disable any CPUs * beyond MAXCPU. CPU 0 is always assigned to the BSP. */ mp_ncpus = 0; nhyper = 0; TOPO_FOREACH(node, &topo_root) { if (node->type != TOPO_TYPE_PU) continue; if ((node->hwid & smt_mask) != (boot_cpu_id & smt_mask)) cpu_info[node->hwid].cpu_hyperthread = 1; if (resource_disabled("lapic", node->hwid)) { if (node->hwid != boot_cpu_id) cpu_info[node->hwid].cpu_disabled = 1; else printf("Cannot disable BSP, APIC ID = %d\n", node->hwid); } if (!hyperthreading_allowed && cpu_info[node->hwid].cpu_hyperthread) cpu_info[node->hwid].cpu_disabled = 1; if (mp_ncpus >= MAXCPU) cpu_info[node->hwid].cpu_disabled = 1; if (cpu_info[node->hwid].cpu_disabled) { disabled_cpus++; continue; } if (cpu_info[node->hwid].cpu_hyperthread) nhyper++; cpu_apic_ids[mp_ncpus] = node->hwid; apic_cpuids[node->hwid] = mp_ncpus; topo_set_pu_id(node, mp_ncpus); mp_ncpus++; } KASSERT(mp_maxid >= mp_ncpus - 1, ("%s: counters out of sync: max %d, count %d", __func__, mp_maxid, mp_ncpus)); mp_ncores = mp_ncpus - nhyper; smp_threads_per_core = mp_ncpus / mp_ncores; } /* * Print various information about the SMP system hardware and setup. */ void cpu_mp_announce(void) { struct topo_node *node; const char *hyperthread; struct topo_analysis topology; printf("FreeBSD/SMP: "); if (topo_analyze(&topo_root, 1, &topology)) { printf("%d package(s)", topology.entities[TOPO_LEVEL_PKG]); if (topology.entities[TOPO_LEVEL_GROUP] > 1) printf(" x %d groups", topology.entities[TOPO_LEVEL_GROUP]); if (topology.entities[TOPO_LEVEL_CACHEGROUP] > 1) printf(" x %d cache groups", topology.entities[TOPO_LEVEL_CACHEGROUP]); if (topology.entities[TOPO_LEVEL_CORE] > 0) printf(" x %d core(s)", topology.entities[TOPO_LEVEL_CORE]); if (topology.entities[TOPO_LEVEL_THREAD] > 1) printf(" x %d hardware threads", topology.entities[TOPO_LEVEL_THREAD]); } else { printf("Non-uniform topology"); } printf("\n"); if (disabled_cpus) { printf("FreeBSD/SMP Online: "); if (topo_analyze(&topo_root, 0, &topology)) { printf("%d package(s)", topology.entities[TOPO_LEVEL_PKG]); if (topology.entities[TOPO_LEVEL_GROUP] > 1) printf(" x %d groups", topology.entities[TOPO_LEVEL_GROUP]); if (topology.entities[TOPO_LEVEL_CACHEGROUP] > 1) printf(" x %d cache groups", topology.entities[TOPO_LEVEL_CACHEGROUP]); if (topology.entities[TOPO_LEVEL_CORE] > 0) printf(" x %d core(s)", topology.entities[TOPO_LEVEL_CORE]); if (topology.entities[TOPO_LEVEL_THREAD] > 1) printf(" x %d hardware threads", topology.entities[TOPO_LEVEL_THREAD]); } else { printf("Non-uniform topology"); } printf("\n"); } if (!bootverbose) return; TOPO_FOREACH(node, &topo_root) { switch (node->type) { case TOPO_TYPE_PKG: printf("Package HW ID = %u\n", node->hwid); break; case TOPO_TYPE_CORE: printf("\tCore HW ID = %u\n", node->hwid); break; case TOPO_TYPE_PU: if (cpu_info[node->hwid].cpu_hyperthread) hyperthread = "/HT"; else hyperthread = ""; if (node->subtype == 0) printf("\t\tCPU (AP%s): APIC ID: %u" "(disabled)\n", hyperthread, node->hwid); else if (node->id == 0) printf("\t\tCPU0 (BSP): APIC ID: %u\n", node->hwid); else printf("\t\tCPU%u (AP%s): APIC ID: %u\n", node->id, hyperthread, node->hwid); break; default: /* ignored */ break; } } } /* * Add a scheduling group, a group of logical processors sharing * a particular cache (and, thus having an affinity), to the scheduling * topology. * This function recursively works on lower level caches. */ static void x86topo_add_sched_group(struct topo_node *root, struct cpu_group *cg_root) { struct topo_node *node; int nchildren; int ncores; int i; KASSERT(root->type == TOPO_TYPE_SYSTEM || root->type == TOPO_TYPE_CACHE || root->type == TOPO_TYPE_GROUP, ("x86topo_add_sched_group: bad type: %u", root->type)); CPU_COPY(&root->cpuset, &cg_root->cg_mask); cg_root->cg_count = root->cpu_count; if (root->type == TOPO_TYPE_SYSTEM) cg_root->cg_level = CG_SHARE_NONE; else cg_root->cg_level = root->subtype; /* * Check how many core nodes we have under the given root node. * If we have multiple logical processors, but not multiple * cores, then those processors must be hardware threads. */ ncores = 0; node = root; while (node != NULL) { if (node->type != TOPO_TYPE_CORE) { node = topo_next_node(root, node); continue; } ncores++; node = topo_next_nonchild_node(root, node); } if (cg_root->cg_level != CG_SHARE_NONE && root->cpu_count > 1 && ncores < 2) cg_root->cg_flags = CG_FLAG_SMT; /* * Find out how many cache nodes we have under the given root node. * We ignore cache nodes that cover all the same processors as the * root node. Also, we do not descend below found cache nodes. * That is, we count top-level "non-redundant" caches under the root * node. */ nchildren = 0; node = root; while (node != NULL) { if ((node->type != TOPO_TYPE_GROUP && node->type != TOPO_TYPE_CACHE) || (root->type != TOPO_TYPE_SYSTEM && CPU_CMP(&node->cpuset, &root->cpuset) == 0)) { node = topo_next_node(root, node); continue; } nchildren++; node = topo_next_nonchild_node(root, node); } cg_root->cg_child = smp_topo_alloc(nchildren); cg_root->cg_children = nchildren; /* * Now find again the same cache nodes as above and recursively * build scheduling topologies for them. */ node = root; i = 0; while (node != NULL) { if ((node->type != TOPO_TYPE_GROUP && node->type != TOPO_TYPE_CACHE) || (root->type != TOPO_TYPE_SYSTEM && CPU_CMP(&node->cpuset, &root->cpuset) == 0)) { node = topo_next_node(root, node); continue; } cg_root->cg_child[i].cg_parent = cg_root; x86topo_add_sched_group(node, &cg_root->cg_child[i]); i++; node = topo_next_nonchild_node(root, node); } } /* * Build the MI scheduling topology from the discovered hardware topology. */ struct cpu_group * cpu_topo(void) { struct cpu_group *cg_root; if (mp_ncpus <= 1) return (smp_topo_none()); cg_root = smp_topo_alloc(1); x86topo_add_sched_group(&topo_root, cg_root); return (cg_root); } static void cpu_alloc(void *dummy __unused) { /* * Dynamically allocate the arrays that depend on the * maximum APIC ID. */ cpu_info = malloc(sizeof(*cpu_info) * (max_apic_id + 1), M_CPUS, M_WAITOK | M_ZERO); apic_cpuids = malloc(sizeof(*apic_cpuids) * (max_apic_id + 1), M_CPUS, M_WAITOK | M_ZERO); } SYSINIT(cpu_alloc, SI_SUB_CPU, SI_ORDER_FIRST, cpu_alloc, NULL); /* * Add a logical CPU to the topology. */ void cpu_add(u_int apic_id, char boot_cpu) { if (apic_id > max_apic_id) { panic("SMP: APIC ID %d too high", apic_id); return; } KASSERT(cpu_info[apic_id].cpu_present == 0, ("CPU %u added twice", apic_id)); cpu_info[apic_id].cpu_present = 1; if (boot_cpu) { KASSERT(boot_cpu_id == -1, ("CPU %u claims to be BSP, but CPU %u already is", apic_id, boot_cpu_id)); boot_cpu_id = apic_id; cpu_info[apic_id].cpu_bsp = 1; } if (bootverbose) printf("SMP: Added CPU %u (%s)\n", apic_id, boot_cpu ? "BSP" : "AP"); } void cpu_mp_setmaxid(void) { /* * mp_ncpus and mp_maxid should be already set by calls to cpu_add(). * If there were no calls to cpu_add() assume this is a UP system. */ if (mp_ncpus == 0) mp_ncpus = 1; } int cpu_mp_probe(void) { /* * Always record BSP in CPU map so that the mbuf init code works * correctly. */ CPU_SETOF(0, &all_cpus); return (mp_ncpus > 1); } /* Allocate memory for the AP trampoline. */ void alloc_ap_trampoline(vm_paddr_t *physmap, unsigned int *physmap_idx) { unsigned int i; bool allocated; allocated = false; for (i = *physmap_idx; i <= *physmap_idx; i -= 2) { /* * Find a memory region big enough and below the 1MB boundary * for the trampoline code. * NB: needs to be page aligned. */ if (physmap[i] >= MiB(1) || (trunc_page(physmap[i + 1]) - round_page(physmap[i])) < round_page(bootMP_size)) continue; allocated = true; /* * Try to steal from the end of the region to mimic previous * behaviour, else fallback to steal from the start. */ if (physmap[i + 1] < MiB(1)) { boot_address = trunc_page(physmap[i + 1]); if ((physmap[i + 1] - boot_address) < bootMP_size) boot_address -= round_page(bootMP_size); physmap[i + 1] = boot_address; } else { boot_address = round_page(physmap[i]); physmap[i] = boot_address + round_page(bootMP_size); } if (physmap[i] == physmap[i + 1] && *physmap_idx != 0) { memmove(&physmap[i], &physmap[i + 2], sizeof(*physmap) * (*physmap_idx - i + 2)); *physmap_idx -= 2; } break; } if (!allocated) { boot_address = basemem * 1024 - bootMP_size; if (bootverbose) printf( "Cannot find enough space for the boot trampoline, placing it at %#x", boot_address); } } /* * AP CPU's call this to initialize themselves. */ void init_secondary_tail(void) { u_int cpuid; pmap_activate_boot(vmspace_pmap(proc0.p_vmspace)); /* * On real hardware, switch to x2apic mode if possible. Do it * after aps_ready was signalled, to avoid manipulating the * mode while BSP might still want to send some IPI to us * (second startup IPI is ignored on modern hardware etc). */ lapic_xapic_mode(); /* Initialize the PAT MSR. */ pmap_init_pat(); /* set up CPU registers and state */ cpu_setregs(); /* set up SSE/NX */ initializecpu(); /* set up FPU state on the AP */ #ifdef __amd64__ fpuinit(); #else npxinit(false); #endif if (cpu_ops.cpu_init) cpu_ops.cpu_init(); /* A quick check from sanity claus */ cpuid = PCPU_GET(cpuid); if (PCPU_GET(apic_id) != lapic_id()) { printf("SMP: cpuid = %d\n", cpuid); printf("SMP: actual apic_id = %d\n", lapic_id()); printf("SMP: correct apic_id = %d\n", PCPU_GET(apic_id)); panic("cpuid mismatch! boom!!"); } /* Initialize curthread. */ KASSERT(PCPU_GET(idlethread) != NULL, ("no idle thread")); PCPU_SET(curthread, PCPU_GET(idlethread)); mtx_lock_spin(&ap_boot_mtx); mca_init(); /* Init local apic for irq's */ lapic_setup(1); /* Set memory range attributes for this CPU to match the BSP */ mem_range_AP_init(); smp_cpus++; CTR1(KTR_SMP, "SMP: AP CPU #%d Launched", cpuid); if (bootverbose) printf("SMP: AP CPU #%d Launched!\n", cpuid); else printf("%s%d%s", smp_cpus == 2 ? "Launching APs: " : "", cpuid, smp_cpus == mp_ncpus ? "\n" : " "); /* Determine if we are a logical CPU. */ if (cpu_info[PCPU_GET(apic_id)].cpu_hyperthread) CPU_SET(cpuid, &logical_cpus_mask); if (bootverbose) lapic_dump("AP"); if (smp_cpus == mp_ncpus) { /* enable IPI's, tlb shootdown, freezes etc */ atomic_store_rel_int(&smp_started, 1); } #ifdef __amd64__ /* * Enable global pages TLB extension * This also implicitly flushes the TLB */ load_cr4(rcr4() | CR4_PGE); if (pmap_pcid_enabled) load_cr4(rcr4() | CR4_PCIDE); load_ds(_udatasel); load_es(_udatasel); load_fs(_ufssel); #endif mtx_unlock_spin(&ap_boot_mtx); /* Wait until all the AP's are up. */ while (atomic_load_acq_int(&smp_started) == 0) ia32_pause(); #ifndef EARLY_AP_STARTUP /* Start per-CPU event timers. */ cpu_initclocks_ap(); #endif kcsan_cpu_init(cpuid); /* * Assert that smp_after_idle_runnable condition is reasonable. */ MPASS(PCPU_GET(curpcb) == NULL); sched_throw(NULL); panic("scheduler returned us to %s", __func__); /* NOTREACHED */ } static void smp_after_idle_runnable(void *arg __unused) { struct pcpu *pc; int cpu; for (cpu = 1; cpu < mp_ncpus; cpu++) { pc = pcpu_find(cpu); - while (atomic_load_ptr(&pc->pc_curpcb) == (uintptr_t)NULL) + while (atomic_load_ptr(&pc->pc_curpcb) == NULL) cpu_spinwait(); kmem_free((vm_offset_t)bootstacks[cpu], kstack_pages * PAGE_SIZE); } } SYSINIT(smp_after_idle_runnable, SI_SUB_SMP, SI_ORDER_ANY, smp_after_idle_runnable, NULL); /* * We tell the I/O APIC code about all the CPUs we want to receive * interrupts. If we don't want certain CPUs to receive IRQs we * can simply not tell the I/O APIC code about them in this function. * We also do not tell it about the BSP since it tells itself about * the BSP internally to work with UP kernels and on UP machines. */ void set_interrupt_apic_ids(void) { u_int i, apic_id; for (i = 0; i < MAXCPU; i++) { apic_id = cpu_apic_ids[i]; if (apic_id == -1) continue; if (cpu_info[apic_id].cpu_bsp) continue; if (cpu_info[apic_id].cpu_disabled) continue; /* Don't let hyperthreads service interrupts. */ if (cpu_info[apic_id].cpu_hyperthread && !hyperthreading_intr_allowed) continue; intr_add_cpu(i); } } #ifdef COUNT_XINVLTLB_HITS u_int xhits_gbl[MAXCPU]; u_int xhits_pg[MAXCPU]; u_int xhits_rng[MAXCPU]; static SYSCTL_NODE(_debug, OID_AUTO, xhits, CTLFLAG_RW, 0, ""); SYSCTL_OPAQUE(_debug_xhits, OID_AUTO, global, CTLFLAG_RW, &xhits_gbl, sizeof(xhits_gbl), "IU", ""); SYSCTL_OPAQUE(_debug_xhits, OID_AUTO, page, CTLFLAG_RW, &xhits_pg, sizeof(xhits_pg), "IU", ""); SYSCTL_OPAQUE(_debug_xhits, OID_AUTO, range, CTLFLAG_RW, &xhits_rng, sizeof(xhits_rng), "IU", ""); u_int ipi_global; u_int ipi_page; u_int ipi_range; u_int ipi_range_size; SYSCTL_INT(_debug_xhits, OID_AUTO, ipi_global, CTLFLAG_RW, &ipi_global, 0, ""); SYSCTL_INT(_debug_xhits, OID_AUTO, ipi_page, CTLFLAG_RW, &ipi_page, 0, ""); SYSCTL_INT(_debug_xhits, OID_AUTO, ipi_range, CTLFLAG_RW, &ipi_range, 0, ""); SYSCTL_INT(_debug_xhits, OID_AUTO, ipi_range_size, CTLFLAG_RW, &ipi_range_size, 0, ""); #endif /* COUNT_XINVLTLB_HITS */ /* * Init and startup IPI. */ void ipi_startup(int apic_id, int vector) { /* * This attempts to follow the algorithm described in the * Intel Multiprocessor Specification v1.4 in section B.4. * For each IPI, we allow the local APIC ~20us to deliver the * IPI. If that times out, we panic. */ /* * first we do an INIT IPI: this INIT IPI might be run, resetting * and running the target CPU. OR this INIT IPI might be latched (P5 * bug), CPU waiting for STARTUP IPI. OR this INIT IPI might be * ignored. */ lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_LEVEL | APIC_LEVEL_ASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_INIT, apic_id); lapic_ipi_wait(100); /* Explicitly deassert the INIT IPI. */ lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_LEVEL | APIC_LEVEL_DEASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_INIT, apic_id); DELAY(10000); /* wait ~10mS */ /* * next we do a STARTUP IPI: the previous INIT IPI might still be * latched, (P5 bug) this 1st STARTUP would then terminate * immediately, and the previously started INIT IPI would continue. OR * the previous INIT IPI has already run. and this STARTUP IPI will * run. OR the previous INIT IPI was ignored. and this STARTUP IPI * will run. */ lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_EDGE | APIC_LEVEL_ASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_STARTUP | vector, apic_id); if (!lapic_ipi_wait(100)) panic("Failed to deliver first STARTUP IPI to APIC %d", apic_id); DELAY(200); /* wait ~200uS */ /* * finally we do a 2nd STARTUP IPI: this 2nd STARTUP IPI should run IF * the previous STARTUP IPI was cancelled by a latched INIT IPI. OR * this STARTUP IPI will be ignored, as only ONE STARTUP IPI is * recognized after hardware RESET or INIT IPI. */ lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_EDGE | APIC_LEVEL_ASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_STARTUP | vector, apic_id); if (!lapic_ipi_wait(100)) panic("Failed to deliver second STARTUP IPI to APIC %d", apic_id); DELAY(200); /* wait ~200uS */ } /* * Send an IPI to specified CPU handling the bitmap logic. */ void ipi_send_cpu(int cpu, u_int ipi) { u_int bitmap, old, new; u_int *cpu_bitmap; KASSERT((u_int)cpu < MAXCPU && cpu_apic_ids[cpu] != -1, ("IPI to non-existent CPU %d", cpu)); if (IPI_IS_BITMAPED(ipi)) { bitmap = 1 << ipi; ipi = IPI_BITMAP_VECTOR; cpu_bitmap = &cpuid_to_pcpu[cpu]->pc_ipi_bitmap; old = *cpu_bitmap; for (;;) { if ((old & bitmap) == bitmap) break; new = old | bitmap; if (atomic_fcmpset_int(cpu_bitmap, &old, new)) break; } if (old) return; } lapic_ipi_vectored(ipi, cpu_apic_ids[cpu]); } void ipi_bitmap_handler(struct trapframe frame) { struct trapframe *oldframe; struct thread *td; int cpu = PCPU_GET(cpuid); u_int ipi_bitmap; td = curthread; ipi_bitmap = atomic_readandclear_int(&cpuid_to_pcpu[cpu]-> pc_ipi_bitmap); /* * sched_preempt() must be called to clear the pending preempt * IPI to enable delivery of further preempts. However, the * critical section will cause extra scheduler lock thrashing * when used unconditionally. Only critical_enter() if * hardclock must also run, which requires the section entry. */ if (ipi_bitmap & (1 << IPI_HARDCLOCK)) critical_enter(); td->td_intr_nesting_level++; oldframe = td->td_intr_frame; td->td_intr_frame = &frame; #if defined(STACK) || defined(DDB) if (ipi_bitmap & (1 << IPI_TRACE)) stack_capture_intr(); #endif if (ipi_bitmap & (1 << IPI_PREEMPT)) { #ifdef COUNT_IPIS (*ipi_preempt_counts[cpu])++; #endif sched_preempt(td); } if (ipi_bitmap & (1 << IPI_AST)) { #ifdef COUNT_IPIS (*ipi_ast_counts[cpu])++; #endif /* Nothing to do for AST */ } if (ipi_bitmap & (1 << IPI_HARDCLOCK)) { #ifdef COUNT_IPIS (*ipi_hardclock_counts[cpu])++; #endif hardclockintr(); } td->td_intr_frame = oldframe; td->td_intr_nesting_level--; if (ipi_bitmap & (1 << IPI_HARDCLOCK)) critical_exit(); } /* * send an IPI to a set of cpus. */ void ipi_selected(cpuset_t cpus, u_int ipi) { int cpu; /* * IPI_STOP_HARD maps to a NMI and the trap handler needs a bit * of help in order to understand what is the source. * Set the mask of receiving CPUs for this purpose. */ if (ipi == IPI_STOP_HARD) CPU_OR_ATOMIC(&ipi_stop_nmi_pending, &cpus); while ((cpu = CPU_FFS(&cpus)) != 0) { cpu--; CPU_CLR(cpu, &cpus); CTR3(KTR_SMP, "%s: cpu: %d ipi: %x", __func__, cpu, ipi); ipi_send_cpu(cpu, ipi); } } /* * send an IPI to a specific CPU. */ void ipi_cpu(int cpu, u_int ipi) { /* * IPI_STOP_HARD maps to a NMI and the trap handler needs a bit * of help in order to understand what is the source. * Set the mask of receiving CPUs for this purpose. */ if (ipi == IPI_STOP_HARD) CPU_SET_ATOMIC(cpu, &ipi_stop_nmi_pending); CTR3(KTR_SMP, "%s: cpu: %d ipi: %x", __func__, cpu, ipi); ipi_send_cpu(cpu, ipi); } /* * send an IPI to all CPUs EXCEPT myself */ void ipi_all_but_self(u_int ipi) { cpuset_t other_cpus; other_cpus = all_cpus; CPU_CLR(PCPU_GET(cpuid), &other_cpus); if (IPI_IS_BITMAPED(ipi)) { ipi_selected(other_cpus, ipi); return; } /* * IPI_STOP_HARD maps to a NMI and the trap handler needs a bit * of help in order to understand what is the source. * Set the mask of receiving CPUs for this purpose. */ if (ipi == IPI_STOP_HARD) CPU_OR_ATOMIC(&ipi_stop_nmi_pending, &other_cpus); CTR2(KTR_SMP, "%s: ipi: %x", __func__, ipi); lapic_ipi_vectored(ipi, APIC_IPI_DEST_OTHERS); } int ipi_nmi_handler(void) { u_int cpuid; /* * As long as there is not a simple way to know about a NMI's * source, if the bitmask for the current CPU is present in * the global pending bitword an IPI_STOP_HARD has been issued * and should be handled. */ cpuid = PCPU_GET(cpuid); if (!CPU_ISSET(cpuid, &ipi_stop_nmi_pending)) return (1); CPU_CLR_ATOMIC(cpuid, &ipi_stop_nmi_pending); cpustop_handler(); return (0); } int nmi_kdb_lock; void nmi_call_kdb_smp(u_int type, struct trapframe *frame) { int cpu; bool call_post; cpu = PCPU_GET(cpuid); if (atomic_cmpset_acq_int(&nmi_kdb_lock, 0, 1)) { nmi_call_kdb(cpu, type, frame); call_post = false; } else { savectx(&stoppcbs[cpu]); CPU_SET_ATOMIC(cpu, &stopped_cpus); while (!atomic_cmpset_acq_int(&nmi_kdb_lock, 0, 1)) ia32_pause(); call_post = true; } atomic_store_rel_int(&nmi_kdb_lock, 0); if (call_post) cpustop_handler_post(cpu); } /* * Handle an IPI_STOP by saving our current context and spinning (or mwaiting, * if available) until we are resumed. */ void cpustop_handler(void) { struct monitorbuf *mb; u_int cpu; bool use_mwait; cpu = PCPU_GET(cpuid); savectx(&stoppcbs[cpu]); use_mwait = (stop_mwait && (cpu_feature2 & CPUID2_MON) != 0 && !mwait_cpustop_broken); if (use_mwait) { mb = PCPU_PTR(monitorbuf); atomic_store_int(&mb->stop_state, MONITOR_STOPSTATE_STOPPED); } /* Indicate that we are stopped */ CPU_SET_ATOMIC(cpu, &stopped_cpus); /* Wait for restart */ while (!CPU_ISSET(cpu, &started_cpus)) { if (use_mwait) { cpu_monitor(mb, 0, 0); if (atomic_load_int(&mb->stop_state) == MONITOR_STOPSTATE_STOPPED) cpu_mwait(0, MWAIT_C1); continue; } ia32_pause(); /* * Halt non-BSP CPUs on panic -- we're never going to need them * again, and might as well save power / release resources * (e.g., overprovisioned VM infrastructure). */ while (__predict_false(!IS_BSP() && KERNEL_PANICKED())) halt(); } cpustop_handler_post(cpu); } static void cpustop_handler_post(u_int cpu) { CPU_CLR_ATOMIC(cpu, &started_cpus); CPU_CLR_ATOMIC(cpu, &stopped_cpus); /* * We don't broadcast TLB invalidations to other CPUs when they are * stopped. Hence, we clear the TLB before resuming. */ invltlb_glob(); #if defined(__amd64__) && defined(DDB) amd64_db_resume_dbreg(); #endif if (cpu == 0 && cpustop_restartfunc != NULL) { cpustop_restartfunc(); cpustop_restartfunc = NULL; } } /* * Handle an IPI_SUSPEND by saving our current context and spinning until we * are resumed. */ void cpususpend_handler(void) { u_int cpu; mtx_assert(&smp_ipi_mtx, MA_NOTOWNED); cpu = PCPU_GET(cpuid); if (savectx(&susppcbs[cpu]->sp_pcb)) { #ifdef __amd64__ fpususpend(susppcbs[cpu]->sp_fpususpend); #else npxsuspend(susppcbs[cpu]->sp_fpususpend); #endif /* * suspended_cpus is cleared shortly after each AP is restarted * by a Startup IPI, so that the BSP can proceed to restarting * the next AP. * * resuming_cpus gets cleared when the AP completes * initialization after having been released by the BSP. * resuming_cpus is probably not the best name for the * variable, because it is actually a set of processors that * haven't resumed yet and haven't necessarily started resuming. * * Note that suspended_cpus is meaningful only for ACPI suspend * as it's not really used for Xen suspend since the APs are * automatically restored to the running state and the correct * context. For the same reason resumectx is never called in * that case. */ CPU_SET_ATOMIC(cpu, &suspended_cpus); CPU_SET_ATOMIC(cpu, &resuming_cpus); /* * Invalidate the cache after setting the global status bits. * The last AP to set its bit may end up being an Owner of the * corresponding cache line in MOESI protocol. The AP may be * stopped before the cache line is written to the main memory. */ wbinvd(); } else { #ifdef __amd64__ fpuresume(susppcbs[cpu]->sp_fpususpend); #else npxresume(susppcbs[cpu]->sp_fpususpend); #endif pmap_init_pat(); initializecpu(); PCPU_SET(switchtime, 0); PCPU_SET(switchticks, ticks); /* Indicate that we have restarted and restored the context. */ CPU_CLR_ATOMIC(cpu, &suspended_cpus); } /* Wait for resume directive */ while (!CPU_ISSET(cpu, &toresume_cpus)) ia32_pause(); /* Re-apply microcode updates. */ ucode_reload(); #ifdef __i386__ /* Finish removing the identity mapping of low memory for this AP. */ invltlb_glob(); #endif if (cpu_ops.cpu_resume) cpu_ops.cpu_resume(); #ifdef __amd64__ if (vmm_resume_p) vmm_resume_p(); #endif /* Resume MCA and local APIC */ lapic_xapic_mode(); mca_resume(); lapic_setup(0); /* Indicate that we are resumed */ CPU_CLR_ATOMIC(cpu, &resuming_cpus); CPU_CLR_ATOMIC(cpu, &suspended_cpus); CPU_CLR_ATOMIC(cpu, &toresume_cpus); } void invlcache_handler(void) { uint32_t generation; #ifdef COUNT_IPIS (*ipi_invlcache_counts[PCPU_GET(cpuid)])++; #endif /* COUNT_IPIS */ /* * Reading the generation here allows greater parallelism * since wbinvd is a serializing instruction. Without the * temporary, we'd wait for wbinvd to complete, then the read * would execute, then the dependent write, which must then * complete before return from interrupt. */ generation = smp_tlb_generation; wbinvd(); PCPU_SET(smp_tlb_done, generation); } /* * This is called once the rest of the system is up and running and we're * ready to let the AP's out of the pen. */ static void release_aps(void *dummy __unused) { if (mp_ncpus == 1) return; atomic_store_rel_int(&aps_ready, 1); while (smp_started == 0) ia32_pause(); } SYSINIT(start_aps, SI_SUB_SMP, SI_ORDER_FIRST, release_aps, NULL); #ifdef COUNT_IPIS /* * Setup interrupt counters for IPI handlers. */ static void mp_ipi_intrcnt(void *dummy) { char buf[64]; int i; CPU_FOREACH(i) { snprintf(buf, sizeof(buf), "cpu%d:invltlb", i); intrcnt_add(buf, &ipi_invltlb_counts[i]); snprintf(buf, sizeof(buf), "cpu%d:invlrng", i); intrcnt_add(buf, &ipi_invlrng_counts[i]); snprintf(buf, sizeof(buf), "cpu%d:invlpg", i); intrcnt_add(buf, &ipi_invlpg_counts[i]); snprintf(buf, sizeof(buf), "cpu%d:invlcache", i); intrcnt_add(buf, &ipi_invlcache_counts[i]); snprintf(buf, sizeof(buf), "cpu%d:preempt", i); intrcnt_add(buf, &ipi_preempt_counts[i]); snprintf(buf, sizeof(buf), "cpu%d:ast", i); intrcnt_add(buf, &ipi_ast_counts[i]); snprintf(buf, sizeof(buf), "cpu%d:rendezvous", i); intrcnt_add(buf, &ipi_rendezvous_counts[i]); snprintf(buf, sizeof(buf), "cpu%d:hardclock", i); intrcnt_add(buf, &ipi_hardclock_counts[i]); } } SYSINIT(mp_ipi_intrcnt, SI_SUB_INTR, SI_ORDER_MIDDLE, mp_ipi_intrcnt, NULL); #endif /* * Flush the TLB on other CPU's */ /* Variables needed for SMP tlb shootdown. */ vm_offset_t smp_tlb_addr1, smp_tlb_addr2; pmap_t smp_tlb_pmap; volatile uint32_t smp_tlb_generation; #ifdef __amd64__ #define read_eflags() read_rflags() #endif static void smp_targeted_tlb_shootdown(cpuset_t mask, u_int vector, pmap_t pmap, vm_offset_t addr1, vm_offset_t addr2) { cpuset_t other_cpus; volatile uint32_t *p_cpudone; uint32_t generation; int cpu; /* It is not necessary to signal other CPUs while in the debugger. */ if (kdb_active || KERNEL_PANICKED()) return; /* * Check for other cpus. Return if none. */ if (CPU_ISFULLSET(&mask)) { if (mp_ncpus <= 1) return; } else { CPU_CLR(PCPU_GET(cpuid), &mask); if (CPU_EMPTY(&mask)) return; } if (!(read_eflags() & PSL_I)) panic("%s: interrupts disabled", __func__); mtx_lock_spin(&smp_ipi_mtx); smp_tlb_addr1 = addr1; smp_tlb_addr2 = addr2; smp_tlb_pmap = pmap; generation = ++smp_tlb_generation; if (CPU_ISFULLSET(&mask)) { ipi_all_but_self(vector); other_cpus = all_cpus; CPU_CLR(PCPU_GET(cpuid), &other_cpus); } else { other_cpus = mask; while ((cpu = CPU_FFS(&mask)) != 0) { cpu--; CPU_CLR(cpu, &mask); CTR3(KTR_SMP, "%s: cpu: %d ipi: %x", __func__, cpu, vector); ipi_send_cpu(cpu, vector); } } while ((cpu = CPU_FFS(&other_cpus)) != 0) { cpu--; CPU_CLR(cpu, &other_cpus); p_cpudone = &cpuid_to_pcpu[cpu]->pc_smp_tlb_done; while (*p_cpudone != generation) ia32_pause(); } mtx_unlock_spin(&smp_ipi_mtx); } void smp_masked_invltlb(cpuset_t mask, pmap_t pmap) { if (smp_started) { smp_targeted_tlb_shootdown(mask, IPI_INVLTLB, pmap, 0, 0); #ifdef COUNT_XINVLTLB_HITS ipi_global++; #endif } } void smp_masked_invlpg(cpuset_t mask, vm_offset_t addr, pmap_t pmap) { if (smp_started) { smp_targeted_tlb_shootdown(mask, IPI_INVLPG, pmap, addr, 0); #ifdef COUNT_XINVLTLB_HITS ipi_page++; #endif } } void smp_masked_invlpg_range(cpuset_t mask, vm_offset_t addr1, vm_offset_t addr2, pmap_t pmap) { if (smp_started) { smp_targeted_tlb_shootdown(mask, IPI_INVLRNG, pmap, addr1, addr2); #ifdef COUNT_XINVLTLB_HITS ipi_range++; ipi_range_size += (addr2 - addr1) / PAGE_SIZE; #endif } } void smp_cache_flush(void) { if (smp_started) { smp_targeted_tlb_shootdown(all_cpus, IPI_INVLCACHE, NULL, 0, 0); } } /* * Handlers for TLB related IPIs */ void invltlb_handler(void) { uint32_t generation; #ifdef COUNT_XINVLTLB_HITS xhits_gbl[PCPU_GET(cpuid)]++; #endif /* COUNT_XINVLTLB_HITS */ #ifdef COUNT_IPIS (*ipi_invltlb_counts[PCPU_GET(cpuid)])++; #endif /* COUNT_IPIS */ /* * Reading the generation here allows greater parallelism * since invalidating the TLB is a serializing operation. */ generation = smp_tlb_generation; if (smp_tlb_pmap == kernel_pmap) invltlb_glob(); #ifdef __amd64__ else invltlb(); #endif PCPU_SET(smp_tlb_done, generation); } void invlpg_handler(void) { uint32_t generation; #ifdef COUNT_XINVLTLB_HITS xhits_pg[PCPU_GET(cpuid)]++; #endif /* COUNT_XINVLTLB_HITS */ #ifdef COUNT_IPIS (*ipi_invlpg_counts[PCPU_GET(cpuid)])++; #endif /* COUNT_IPIS */ generation = smp_tlb_generation; /* Overlap with serialization */ #ifdef __i386__ if (smp_tlb_pmap == kernel_pmap) #endif invlpg(smp_tlb_addr1); PCPU_SET(smp_tlb_done, generation); } void invlrng_handler(void) { vm_offset_t addr, addr2; uint32_t generation; #ifdef COUNT_XINVLTLB_HITS xhits_rng[PCPU_GET(cpuid)]++; #endif /* COUNT_XINVLTLB_HITS */ #ifdef COUNT_IPIS (*ipi_invlrng_counts[PCPU_GET(cpuid)])++; #endif /* COUNT_IPIS */ addr = smp_tlb_addr1; addr2 = smp_tlb_addr2; generation = smp_tlb_generation; /* Overlap with serialization */ #ifdef __i386__ if (smp_tlb_pmap == kernel_pmap) #endif do { invlpg(addr); addr += PAGE_SIZE; } while (addr < addr2); PCPU_SET(smp_tlb_done, generation); } Index: projects/clang1000-import =================================================================== --- projects/clang1000-import (revision 357965) +++ projects/clang1000-import (revision 357966) Property changes on: projects/clang1000-import ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head:r357931-357965