Page MenuHomeFreeBSD
Differential D30936

xen/intr: move interrupt allocation/release to architecture
ClosedPublic
Actions

Authored by ehem_freebsd_m5p.com on Jun 28 2021, 9:55 PM.
Tags
None
Referenced Files
F113756097: D30936.id91464.diff
Thu, Apr 3, 3:21 AM
Unknown Object (File)
Tue, Apr 1, 6:42 AM
Unknown Object (File)
Thu, Mar 20, 10:22 AM
Unknown Object (File)
Fri, Mar 7, 2:51 AM
Unknown Object (File)
Feb 26 2025, 4:04 PM
Unknown Object (File)
Feb 26 2025, 8:34 AM
Unknown Object (File)
Feb 25 2025, 6:24 PM
Unknown Object (File)
Feb 24 2025, 10:25 AM
View All Files
Subscribers
imp

Details

Reviewers
royger
mhorne
julien_xen.org
Commits
rG6699c22c1cb0: xen/intr: move interrupt allocation/release to architecture
Summary

Inspired by the work of Julien Grall <julien@xen.org>,
2015-10-20 09:14:56, but heavily adjusted.

Simply moving the interrupt allocation and release functions into files
which belong to the architecture. Since x86 interrupt handling is quite
distinct from other architectures, this is a crucial necessary step.

Identifying the border between x86 and architecture-independent is
actually quite tricky. Similarly, getting the prototypes for the
border right is also quite tricky.

Since interrupt structure allocation/release is now in architecture
code, the malloc structure needs to be passed. Some architectures also
need the device when allocating interrupts.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 40152
Build 37041: arc lint + arc unit

Event Timeline

ehem_freebsd_m5p.com created this revision.Jun 28 2021, 9:55 PM
Herald added a subscriber: imp. · View Herald TranscriptJun 28 2021, 9:55 PM
ehem_freebsd_m5p.com requested review of this revision.Jun 28 2021, 9:55 PM
Harbormaster completed remote builds in B40152: Diff 91464.Jun 28 2021, 9:55 PM
ehem_freebsd_m5p.com added a comment.Jun 28 2021, 10:24 PM

Lots of action in D30936 as it is near the completion of the architecture-independent API for FreeBSD/Xen. I ended up splitting this off of D30909 due to the shear complexity involved in this portion. Unfortunately this has the effect of requiring xen_intr_pic being global/declared in sys/x86/include/xen/arch-intr.c between D30909 and D30936. Yet isolating this portion of the complexity seems worthwhile.

As you can see, not one, but 3 functions are fairly tightly interlinked and are best to move together. Similarly 3 static variables also need to stick together.

sys/xen/arch-intr.h
63–65

I'm not 100% certain I got these right. The device name is useful for the intr_create_event() call on arm64, though I don't believe that is absolutely required.

The bigger question is whether to have M_XENINTR remain in xen_intr.c, versus moving with xen_arch_intr_alloc() to the architecture-specific file. In earlier versions the call to malloc() had remained in xen_intr.c, but that became untenable when it was clear xen_intr_find_unused_isrc() should move to architecture (since identifying reusable interrupts relies on highly architecture-specific structures).

I think I had also initially been seeing hints part of the memory handling might remain in xen_intr.c, but that may have disappeared with PVHv1.

ehem_freebsd_m5p.com added a parent revision: D30935: xen/intr: move interrupt balancing into xen_intr_alloc_isrc().Jun 29 2021, 4:59 AM
ehem_freebsd_m5p.com added parent revisions: D30726: xen/intr: rework locking, prepare xen_intr_alloc_isrc() for split, D30909: xen/intr: introduce xen_arch_intr.c.Jun 30 2021, 1:10 AM
ehem_freebsd_m5p.com added a child revision: D30950: xen/arm64: enable arm64 Xen configuration.Jun 30 2021, 1:17 AM
ehem_freebsd_m5p.com added a parent revision: D30648: xen/intr: move x86 bits into xen_arch_isrc_t structure.Jul 3 2021, 12:51 AM
ehem_freebsd_m5p.com added inline comments.Jul 3 2021, 1:48 AM
sys/xen/arch-intr.h
63–65

So this really does look like a good question of what these should look like. The minimum arguments are:

struct xenisrc *xen_arch_intr_alloc(enum evtchn_type type, evtchn_port_t port);
void    xen_arch_intr_release(struct xenisrc *isrc);

Testing confirmed passing the name is actually unnecessary. Meanwhile, if the architecture file takes ownership of M_XENINTR then that can be static to architecture and need not be passed around. Checking found the other use of M_XENINTR disappeared from with D30228.

Thoughts? (somehow I'm guessing those are going to be "do it")

ehem_freebsd_m5p.com added a child revision: D31063: xen/intr: move isrc allocation to architecture (aarch64 version).Jul 5 2021, 6:50 PM
ehem_freebsd_m5p.com removed a child revision: D30950: xen/arm64: enable arm64 Xen configuration.Jul 5 2021, 7:14 PM
ehem_freebsd_m5p.com mentioned this in D30599: xen/intr: rework xen_intr_resume() for in-place remapping.Jul 5 2021, 10:23 PM
ehem_freebsd_m5p.com added a comment.Jul 15 2021, 3:43 AM

Another possible prototype might be struct xenisrc *xen_arch_intr_alloc(const struct xenisrc *params);. This has the advantage of xen_intr_bind_isrc() could merge its type and port parameters together. Better yet, if xen_intr_bind_virq() set xi_virq then arm64 could use these in the intr_event_create() call (for virqs I would tend towards "xen-virq-%u" as the format).

Then the architecture functions simply do memcpy(&isrc->xi_arch + 1, &params->xi_arch + 1, sizeof(struct xenisrc) - sizeof(xen_arch_isrc_t));, to copy everything from the passed-in temporary structure to the final structure.

ehem_freebsd_m5p.com mentioned this in D31188: xen/intr: move isrc allocation out of xen_intr_bind_isrc().Jul 15 2021, 5:09 AM
ehem_freebsd_m5p.com updated this revision to Diff 93431.Aug 9 2021, 5:14 AM

Update to reflect adjustment to D30726.

Harbormaster completed remote builds in B40937: Diff 93431.Aug 9 2021, 5:14 AM
ehem_freebsd_m5p.com updated this revision to Diff 94406.Aug 31 2021, 3:10 AM

Updating diff to match with D30726 update.

Harbormaster completed remote builds in B41275: Diff 94406.Aug 31 2021, 3:10 AM
ehem_freebsd_m5p.com updated this revision to Diff 95054.Sep 13 2021, 5:20 PM

Updating diff to match other diffs. Deltas which overlap can be trouble.

Harbormaster completed remote builds in B41485: Diff 95054.Sep 13 2021, 5:20 PM
ehem_freebsd_m5p.com added a parent revision: D31995: xen/intr: add check for intr_register_source() errors.Sep 16 2021, 10:55 PM
ehem_freebsd_m5p.com removed a parent revision: D31995: xen/intr: add check for intr_register_source() errors.Sep 17 2021, 1:00 AM
ehem_freebsd_m5p.com updated this revision to Diff 96851.Oct 14 2021, 2:21 AM

Update reflecting logic fix to D30909. The return of xen_arch_intr_has_handlers() is inverted from what had previously been used.

Harbormaster completed remote builds in B42134: Diff 96851.Oct 14 2021, 2:21 AM
ehem_freebsd_m5p.com mentioned this in D32789: xen/intr: mark several pieces unsigned.Nov 1 2021, 8:26 PM
ehem_freebsd_m5p.com mentioned this in D30726: xen/intr: rework locking, prepare xen_intr_alloc_isrc() for split.Nov 8 2021, 4:39 PM
ehem_freebsd_m5p.com added inline comments.Nov 11 2021, 2:37 AM
sys/x86/xen/xen_arch_intr.c
304–306

Mentioned on D30726, should this segment be dropped? These values must be set during the allocation process, while the architecture code only looks at ->xi_type (== EVTCHN_TYPE_UNBOUND, presently unallocated, other values indicate in use). Though looks like xen_intr_bind_isrc() may need to start clearing ->xi_cookie (which it should have been doing already, D31188 implements this implicitly).

sys/xen/arch-intr.h
63–65

Though by doing less in D30935 it is possible to remove the requirement for passing the port (leaving the xen_intr_assign_cpu() in xen_intr_bind_isrc()). Alternatively D31188 passes the port (and everything else; this in fact was the inspiration for D31188).

ehem_freebsd_m5p.com updated this revision to Diff 98347.Nov 11 2021, 4:51 AM

Updating to present status. Difficult keeping everything up to date when you've got >60 diffs and ~100 commits in the queue.

Harbormaster completed remote builds in B42721: Diff 98347.Nov 11 2021, 4:51 AM
This revision was not accepted when it landed; it landed in state Needs Review.Apr 14 2023, 2:02 PM
Closed by commit rG6699c22c1cb0: xen/intr: move interrupt allocation/release to architecture (authored by ehem_freebsd_m5p.com, committed by royger). · Explain Why
This revision was automatically updated to reflect the committed changes.
royger added a commit: rG6699c22c1cb0: xen/intr: move interrupt allocation/release to architecture.

Revision Contents
Changeset List

PathSize
sys/
x86/
include/
xen/
2 lines
xen/
135 lines
xen/
3 lines
130 lines

Diff 91464

View Options

sys/x86/include/xen/arch-intr.h

Show All 34 Lines
struct xen_arch_isrc { struct xen_arch_isrc {
struct intsrc xai_intsrc; /* @TOP -> *xen_arch_isrc */ struct intsrc xai_intsrc; /* @TOP -> *xen_arch_isrc */
u_int xai_vector; /* Global isrc vector number */ u_int xai_vector; /* Global isrc vector number */
}; };
typedef struct xen_arch_isrc xen_arch_isrc_t; typedef struct xen_arch_isrc xen_arch_isrc_t;
extern struct pic xen_intr_pic;
#endif /* _MACHINE_X86_XEN_ARCH_INTR_H_ */ #endif /* _MACHINE_X86_XEN_ARCH_INTR_H_ */
View Options

sys/x86/xen/xen_arch_intr.c

Show First 20 Lines • Show All 42 Lines • ▼ Show 20 Lines
#include <machine/intr_machdep.h> #include <machine/intr_machdep.h>
#include <x86/apicvar.h> #include <x86/apicvar.h>
#include <xen/xen-os.h> #include <xen/xen-os.h>
#include <xen/xen_intr.h> #include <xen/xen_intr.h>
#include <xen/arch-intr.h> #include <xen/arch-intr.h>
/*
* Lock for x86-related structures. Notably modifying
* xen_intr_auto_vector_count, allocating/releasing interrupts require this
* lock to be held. This means modifying isrc->xi_type requires this lock to
* be held.
*/
static struct mtx xen_intr_x86_lock;
static u_int first_evtchn_irq;
static u_int xen_intr_auto_vector_count;
void
xen_intr_alloc_irqs(void)
{
if (num_io_irqs > UINT_MAX - NR_EVENT_CHANNELS)
panic("IRQ allocation overflow (num_msi_irqs too high?)");
first_evtchn_irq = num_io_irqs;
num_io_irqs += NR_EVENT_CHANNELS;
}
/********************************* EVTCHN PIC ********************************/ /********************************* EVTCHN PIC ********************************/
static void static void
xen_intr_pic_enable_source(struct intsrc *isrc) xen_intr_pic_enable_source(struct intsrc *isrc)
{ {
CTASSERT(offsetof(struct xenisrc, xi_arch.xai_intsrc) == 0); CTASSERT(offsetof(struct xenisrc, xi_arch.xai_intsrc) == 0);
▲ Show 20 Lines • Show All 86 Lines • ▼ Show 20 Linesxen_intr_pic_assign_cpu(struct intsrc *isrc, u_int apic_id)
return (xen_intr_assign_cpu((struct xenisrc *)isrc, return (xen_intr_assign_cpu((struct xenisrc *)isrc,
apic_cpuid(apic_id))); apic_cpuid(apic_id)));
} }
/** /**
* PIC interface for all event channel port types except physical IRQs. * PIC interface for all event channel port types except physical IRQs.
*/ */
struct pic xen_intr_pic = { static struct pic xen_intr_pic = {
.pic_enable_source = xen_intr_pic_enable_source, .pic_enable_source = xen_intr_pic_enable_source,
.pic_disable_source = xen_intr_pic_disable_source, .pic_disable_source = xen_intr_pic_disable_source,
.pic_eoi_source = xen_intr_pic_eoi_source, .pic_eoi_source = xen_intr_pic_eoi_source,
.pic_enable_intr = xen_intr_pic_enable_intr, .pic_enable_intr = xen_intr_pic_enable_intr,
.pic_disable_intr = xen_intr_pic_disable_intr, .pic_disable_intr = xen_intr_pic_disable_intr,
.pic_vector = xen_intr_pic_vector, .pic_vector = xen_intr_pic_vector,
.pic_source_pending = xen_intr_pic_source_pending, .pic_source_pending = xen_intr_pic_source_pending,
.pic_suspend = xen_intr_pic_suspend, .pic_suspend = xen_intr_pic_suspend,
.pic_resume = xen_intr_pic_resume, .pic_resume = xen_intr_pic_resume,
.pic_config_intr = xen_intr_pic_config_intr, .pic_config_intr = xen_intr_pic_config_intr,
.pic_assign_cpu = xen_intr_pic_assign_cpu .pic_assign_cpu = xen_intr_pic_assign_cpu
}; };
/****************************** ARCH wrappers ********************************/ /****************************** ARCH wrappers ********************************/
void void
xen_arch_intr_init(void) xen_arch_intr_init(void)
{ {
mtx_init(&xen_intr_x86_lock, "xen-x86-table-lock", NULL, MTX_DEF);
intr_register_pic(&xen_intr_pic); intr_register_pic(&xen_intr_pic);
}
/**
* Search for an already allocated but currently unused Xen interrupt
* source object.
*
* \param type Restrict the search to interrupt sources of the given
* type.
*
* \return A pointer to a free Xen interrupt source object or NULL.
*/
static struct xenisrc *
xen_intr_find_unused_isrc(enum evtchn_type type)
{
int isrc_idx;
mtx_assert(&xen_intr_x86_lock, MA_OWNED);
for (isrc_idx = 0; isrc_idx < xen_intr_auto_vector_count; isrc_idx ++) {
struct xenisrc *isrc;
u_int vector;
vector = first_evtchn_irq + isrc_idx;
isrc = (struct xenisrc *)intr_lookup_source(vector);
if (isrc != NULL
&& isrc->xi_type == EVTCHN_TYPE_UNBOUND) {
KASSERT(xen_arch_intr_has_handlers(isrc),
("Free evtchn still has handlers"));
isrc->xi_type = type;
return (isrc);
}
}
return (NULL);
}
/**
* Allocate a Xen interrupt source object.
*
* \param type The type of interrupt source to create.
*
* \return A pointer to a newly allocated Xen interrupt source
* object or NULL.
*/
struct xenisrc *
xen_arch_intr_alloc(struct malloc_type *mtype, const char *name __unused,
enum evtchn_type type, evtchn_port_t port)
{
static int warned;
struct xenisrc *isrc;
unsigned int vector;
mtx_lock(&xen_intr_x86_lock);
isrc = xen_intr_find_unused_isrc(type);
if (isrc != NULL) {
mtx_unlock(&xen_intr_x86_lock);
goto out;
}
if (xen_intr_auto_vector_count >= NR_EVENT_CHANNELS) {
if (!warned) {
warned = 1;
printf("%s: Event channels exhausted.\n", __func__);
}
mtx_unlock(&xen_intr_x86_lock);
return (NULL);
}
vector = first_evtchn_irq + xen_intr_auto_vector_count;
xen_intr_auto_vector_count++;
KASSERT((intr_lookup_source(vector) == NULL),
("Trying to use an already allocated vector"));
mtx_unlock(&xen_intr_x86_lock);
isrc = malloc(sizeof(*isrc), mtype, M_WAITOK | M_ZERO);
isrc->xi_arch.xai_intsrc.is_pic = &xen_intr_pic;
isrc->xi_arch.xai_vector = vector;
isrc->xi_type = type;
intr_register_source(&isrc->xi_arch.xai_intsrc);
out:
/* xen_intr_assign_cpu() requires this to be set */
isrc->xi_port = port;
#ifdef SMP
if (type == EVTCHN_TYPE_PORT) {
/*
* By default all interrupts are assigned to vCPU#0
* unless specified otherwise, so shuffle them to balance
* the interrupt load.
*/
xen_intr_assign_cpu(isrc, intr_next_cpu(0));
}
#endif
return (isrc);
}
void
xen_arch_intr_release(struct malloc_type *mtype __unused, struct xenisrc *isrc)
{
isrc->xi_cpu = 0;
isrc->xi_port = ~0;
isrc->xi_cookie = NULL;
mtx_lock(&xen_intr_x86_lock);
isrc->xi_type = EVTCHN_TYPE_UNBOUND;
mtx_unlock(&xen_intr_x86_lock);
ehem_freebsd_m5p.comAuthorUnsubmitted
Done
Inline Actions

Mentioned on D30726, should this segment be dropped? These values must be set during the allocation process, while the architecture code only looks at ->xi_type (== EVTCHN_TYPE_UNBOUND, presently unallocated, other values indicate in use). Though looks like xen_intr_bind_isrc() may need to start clearing ->xi_cookie (which it should have been doing already, D31188 implements this implicitly).

ehem_freebsd_m5p.com: Mentioned on D30726, should this segment be dropped? These values //must// be set during the…
} }
bool bool
xen_arch_intr_has_handlers(struct xenisrc *isrc) xen_arch_intr_has_handlers(struct xenisrc *isrc)
{ {
return (isrc->xi_arch.xai_intsrc.is_handlers != 0); return (isrc->xi_arch.xai_intsrc.is_handlers != 0);
} }
Show All 38 Lines
View Options

sys/xen/arch-intr.h

Show First 20 Lines • Show All 54 Lines • ▼ Show 20 Lines
void xen_intr_eoi_source(struct xenisrc *isrc); void xen_intr_eoi_source(struct xenisrc *isrc);
void xen_intr_enable_intr(struct xenisrc *isrc); void xen_intr_enable_intr(struct xenisrc *isrc);
void xen_intr_disable_intr(struct xenisrc *isrc); void xen_intr_disable_intr(struct xenisrc *isrc);
int xen_intr_assign_cpu(struct xenisrc *isrc, u_int to_cpu); int xen_intr_assign_cpu(struct xenisrc *isrc, u_int to_cpu);
/******************* Functions implemented by each architecture **************/ /******************* Functions implemented by each architecture **************/
void xen_arch_intr_init(void); void xen_arch_intr_init(void);
struct xenisrc *xen_arch_intr_alloc(struct malloc_type *mtype, const char *name,
enum evtchn_type type, evtchn_port_t port);
void xen_arch_intr_release(struct malloc_type *mtype, struct xenisrc *isrc);
ehem_freebsd_m5p.comAuthorUnsubmitted
Done
Inline Actions

I'm not 100% certain I got these right. The device name is useful for the intr_create_event() call on arm64, though I don't believe that is absolutely required.

The bigger question is whether to have M_XENINTR remain in xen_intr.c, versus moving with xen_arch_intr_alloc() to the architecture-specific file. In earlier versions the call to malloc() had remained in xen_intr.c, but that became untenable when it was clear xen_intr_find_unused_isrc() should move to architecture (since identifying reusable interrupts relies on highly architecture-specific structures).

I think I had also initially been seeing hints part of the memory handling might remain in xen_intr.c, but that may have disappeared with PVHv1.

ehem_freebsd_m5p.com: I'm not 100% certain I got these right. The device name is useful for the intr_create_event()…
ehem_freebsd_m5p.comAuthorUnsubmitted
Not Done
Inline Actions

So this really does look like a good question of what these should look like. The minimum arguments are:

struct xenisrc *xen_arch_intr_alloc(enum evtchn_type type, evtchn_port_t port);
void    xen_arch_intr_release(struct xenisrc *isrc);

Testing confirmed passing the name is actually unnecessary. Meanwhile, if the architecture file takes ownership of M_XENINTR then that can be static to architecture and need not be passed around. Checking found the other use of M_XENINTR disappeared from with D30228.

Thoughts? (somehow I'm guessing those are going to be "do it")

ehem_freebsd_m5p.com: So this really does look like a good question of what these should look like. The minimum…
ehem_freebsd_m5p.comAuthorUnsubmitted
Done
Inline Actions

Though by doing less in D30935 it is possible to remove the requirement for passing the port (leaving the xen_intr_assign_cpu() in xen_intr_bind_isrc()). Alternatively D31188 passes the port (and everything else; this in fact was the inspiration for D31188).

ehem_freebsd_m5p.com: Though by doing //less// in D30935 it is possible to remove the requirement for passing the…
bool xen_arch_intr_has_handlers(struct xenisrc *isrc); bool xen_arch_intr_has_handlers(struct xenisrc *isrc);
void xen_arch_intr_execute_handlers(struct xenisrc *isrc, void xen_arch_intr_execute_handlers(struct xenisrc *isrc,
struct trapframe *frame); struct trapframe *frame);
int xen_arch_intr_add_handler(const char *name, driver_filter_t filter, int xen_arch_intr_add_handler(const char *name, driver_filter_t filter,
driver_intr_t handler, void *arg, enum intr_type flags, driver_intr_t handler, void *arg, enum intr_type flags,
struct xenisrc *isrc); struct xenisrc *isrc);
int xen_arch_intr_describe(struct xenisrc *isrc, const char *descr); int xen_arch_intr_describe(struct xenisrc *isrc, const char *descr);
int xen_arch_intr_remove_handler(struct xenisrc *isrc); int xen_arch_intr_remove_handler(struct xenisrc *isrc);
int xen_arch_intr_event_bind(struct xenisrc *isrc, u_int cpu); int xen_arch_intr_event_bind(struct xenisrc *isrc, u_int cpu);
#endif /* _XEN_ARCH_INTR_H_ */ #endif /* _XEN_ARCH_INTR_H_ */
View Options

sys/xen/xen_intr.c

Show First 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
#include <sys/interrupt.h> #include <sys/interrupt.h>
#include <sys/pcpu.h> #include <sys/pcpu.h>
#include <sys/smp.h> #include <sys/smp.h>
#include <sys/refcount.h> #include <sys/refcount.h>
#include <vm/vm.h> #include <vm/vm.h>
#include <vm/pmap.h> #include <vm/pmap.h>
#include <machine/intr_machdep.h>
#include <machine/smp.h> #include <machine/smp.h>
#include <machine/stdarg.h> #include <machine/stdarg.h>
#include <machine/xen/synch_bitops.h> #include <machine/xen/synch_bitops.h>
#include <xen/xen-os.h> #include <xen/xen-os.h>
#include <xen/arch-intr.h> #include <xen/arch-intr.h>
#include <xen/hypervisor.h> #include <xen/hypervisor.h>
#include <xen/xen_intr.h> #include <xen/xen_intr.h>
#include <dev/xen/xenpci/xenpcivar.h> #include <dev/xen/xenpci/xenpcivar.h>
#include <dev/pci/pcivar.h> #include <dev/pci/pcivar.h>
#ifdef DDB #ifdef DDB
#include <ddb/ddb.h> #include <ddb/ddb.h>
#endif #endif
static MALLOC_DEFINE(M_XENINTR, "xen_intr", "Xen Interrupt Services"); static MALLOC_DEFINE(M_XENINTR, "xen_intr", "Xen Interrupt Services");
/*
* Lock for x86-related structures. Notably modifying
* xen_intr_auto_vector_count, allocating/releasing interrupts require this
* lock to be held. This means modifying isrc->xi_type requires this lock to
* be held.
*/
static struct mtx xen_intr_x86_lock;
static u_int first_evtchn_irq;
/** /**
* Per-cpu event channel processing state. * Per-cpu event channel processing state.
*/ */
struct xen_intr_pcpu_data { struct xen_intr_pcpu_data {
/** /**
* The last event channel bitmap section (level one bit) processed. * The last event channel bitmap section (level one bit) processed.
* This is used to ensure we scan all ports before * This is used to ensure we scan all ports before
* servicing an already servied port again. * servicing an already servied port again.
Show All 31 Lines
* Lock for interrupt core data. Notably modifying xen_intr_port_to_isrc[], * Lock for interrupt core data. Notably modifying xen_intr_port_to_isrc[],
* or isrc->xi_port (which implies the former) requires this lock be held. * or isrc->xi_port (which implies the former) requires this lock be held.
* *
* Any time this lock is not held, the condition * Any time this lock is not held, the condition
* `!xen_intr_port_to_isrc[i] || (xen_intr_port_to_isrc[i]->ix_port == i)` * `!xen_intr_port_to_isrc[i] || (xen_intr_port_to_isrc[i]->ix_port == i)`
* MUST be true for all values of i which are valid indicies of the array. * MUST be true for all values of i which are valid indicies of the array.
*/ */
static struct mtx xen_intr_isrc_lock; static struct mtx xen_intr_isrc_lock;
static u_int xen_intr_auto_vector_count;
static struct xenisrc *xen_intr_port_to_isrc[NR_EVENT_CHANNELS]; static struct xenisrc *xen_intr_port_to_isrc[NR_EVENT_CHANNELS];
/*------------------------- Private Functions --------------------------------*/ /*------------------------- Private Functions --------------------------------*/
/** /**
* Retrieve a handle for a Xen interrupt source. * Retrieve a handle for a Xen interrupt source.
* *
* \param isrc A valid Xen interrupt source structure. * \param isrc A valid Xen interrupt source structure.
▲ Show 20 Lines • Show All 83 Lines • ▼ Show 20 Linesxen_intr_intrcnt_add(u_int cpu)
if (pcpu->evtchn_intrcnt != NULL) if (pcpu->evtchn_intrcnt != NULL)
return; return;
snprintf(buf, sizeof(buf), "cpu%d:xen", cpu); snprintf(buf, sizeof(buf), "cpu%d:xen", cpu);
intrcnt_add(buf, &pcpu->evtchn_intrcnt); intrcnt_add(buf, &pcpu->evtchn_intrcnt);
} }
/** /**
* Search for an already allocated but currently unused Xen interrupt
* source object.
*
* \param type Restrict the search to interrupt sources of the given
* type.
*
* \return A pointer to a free Xen interrupt source object or NULL.
*/
static struct xenisrc *
xen_intr_find_unused_isrc(enum evtchn_type type)
{
int isrc_idx;
mtx_assert(&xen_intr_x86_lock, MA_OWNED);
for (isrc_idx = 0; isrc_idx < xen_intr_auto_vector_count; isrc_idx ++) {
struct xenisrc *isrc;
u_int vector;
vector = first_evtchn_irq + isrc_idx;
isrc = (struct xenisrc *)intr_lookup_source(vector);
if (isrc != NULL
&& isrc->xi_type == EVTCHN_TYPE_UNBOUND) {
KASSERT(xen_arch_intr_has_handlers(isrc),
("Free evtchn still has handlers"));
isrc->xi_type = type;
return (isrc);
}
}
return (NULL);
}
/**
* Allocate a Xen interrupt source object.
*
* \param type The type of interrupt source to create.
*
* \return A pointer to a newly allocated Xen interrupt source
* object or NULL.
*/
static struct xenisrc *
xen_intr_alloc_isrc(enum evtchn_type type, evtchn_port_t port)
{
static int warned;
struct xenisrc *isrc;
unsigned int vector;
mtx_lock(&xen_intr_x86_lock);
isrc = xen_intr_find_unused_isrc(type);
if (isrc != NULL) {
mtx_unlock(&xen_intr_x86_lock);
goto out;
}
if (xen_intr_auto_vector_count >= NR_EVENT_CHANNELS) {
if (!warned) {
warned = 1;
printf("%s: Event channels exhausted.\n", __func__);
}
mtx_unlock(&xen_intr_x86_lock);
return (NULL);
}
vector = first_evtchn_irq + xen_intr_auto_vector_count;
xen_intr_auto_vector_count++;
KASSERT((intr_lookup_source(vector) == NULL),
("Trying to use an already allocated vector"));
mtx_unlock(&xen_intr_x86_lock);
isrc = malloc(sizeof(*isrc), M_XENINTR, M_WAITOK | M_ZERO);
isrc->xi_arch.xai_intsrc.is_pic = &xen_intr_pic;
isrc->xi_arch.xai_vector = vector;
isrc->xi_type = type;
intr_register_source(&isrc->xi_arch.xai_intsrc);
out:
/* xen_intr_assign_cpu() requires this to be set */
isrc->xi_port = port;
#ifdef SMP
if (type == EVTCHN_TYPE_PORT) {
/*
* By default all interrupts are assigned to vCPU#0
* unless specified otherwise, so shuffle them to balance
* the interrupt load.
*/
xen_intr_assign_cpu(isrc, intr_next_cpu(0));
}
#endif
return (isrc);
}
/**
* Attempt to free an active Xen interrupt source object. * Attempt to free an active Xen interrupt source object.
* *
* \param isrc The interrupt source object to release. * \param isrc The interrupt source object to release.
* *
* \returns EBUSY if the source is still in use, otherwise 0. * \returns EBUSY if the source is still in use, otherwise 0.
*/ */
static int static int
xen_intr_release_isrc(struct xenisrc *isrc) xen_intr_release_isrc(struct xenisrc *isrc)
Show All 14 Lines if (isrc->xi_close != 0) {
if (HYPERVISOR_event_channel_op(EVTCHNOP_close, &close)) if (HYPERVISOR_event_channel_op(EVTCHNOP_close, &close))
panic("EVTCHNOP_close failed"); panic("EVTCHNOP_close failed");
} }
xen_intr_port_to_isrc[isrc->xi_port] = NULL; xen_intr_port_to_isrc[isrc->xi_port] = NULL;
/* not reachable from xen_intr_port_to_isrc[], therefore unlock */ /* not reachable from xen_intr_port_to_isrc[], therefore unlock */
mtx_unlock(&xen_intr_isrc_lock); mtx_unlock(&xen_intr_isrc_lock);
isrc->xi_cpu = 0; xen_arch_intr_release(M_XENINTR, isrc);
isrc->xi_port = ~0;
isrc->xi_cookie = NULL;
mtx_lock(&xen_intr_x86_lock);
isrc->xi_type = EVTCHN_TYPE_UNBOUND;
mtx_unlock(&xen_intr_x86_lock);
return (0); return (0);
} }
/** /**
* Associate an interrupt handler with an already allocated local Xen * Associate an interrupt handler with an already allocated local Xen
* event channel port. * event channel port.
* *
* \param isrcp The returned Xen interrupt object associated with * \param isrcp The returned Xen interrupt object associated with
Show All 23 Linesxen_intr_bind_isrc(struct xenisrc **isrcp, evtchn_port_t local_port,
int error; int error;
*isrcp = NULL; *isrcp = NULL;
if (port_handlep == NULL) { if (port_handlep == NULL) {
printf("%s: %s: Bad event handle\n", intr_owner, __func__); printf("%s: %s: Bad event handle\n", intr_owner, __func__);
return (EINVAL); return (EINVAL);
} }
isrc = xen_intr_alloc_isrc(type, local_port); isrc = xen_arch_intr_alloc(M_XENINTR, intr_owner, type, local_port);
if (isrc == NULL) if (isrc == NULL)
return (ENOSPC); return (ENOSPC);
mtx_lock(&xen_intr_isrc_lock); mtx_lock(&xen_intr_isrc_lock);
xen_intr_port_to_isrc[isrc->xi_port] = isrc; xen_intr_port_to_isrc[isrc->xi_port] = isrc;
refcount_init(&isrc->xi_refcount, 1); refcount_init(&isrc->xi_refcount, 1);
mtx_unlock(&xen_intr_isrc_lock); mtx_unlock(&xen_intr_isrc_lock);
/* Assign the opaque handler */ /* Assign the opaque handler */
▲ Show 20 Lines • Show All 172 Lines • ▼ Show 20 Linesxen_intr_init(void *dummy __unused)
struct xen_intr_pcpu_data *pcpu; struct xen_intr_pcpu_data *pcpu;
int i; int i;
if (!xen_domain()) if (!xen_domain())
return (0); return (0);
mtx_init(&xen_intr_isrc_lock, "xen-irq-lock", NULL, MTX_DEF); mtx_init(&xen_intr_isrc_lock, "xen-irq-lock", NULL, MTX_DEF);
mtx_init(&xen_intr_x86_lock, "xen-x86-table-lock", NULL, MTX_DEF);
/* /*
* Set the per-cpu mask of CPU#0 to enable all, since by default all * Set the per-cpu mask of CPU#0 to enable all, since by default all
* event channels are bound to CPU#0. * event channels are bound to CPU#0.
*/ */
CPU_FOREACH(i) { CPU_FOREACH(i) {
pcpu = DPCPU_ID_PTR(i, xen_intr_pcpu); pcpu = DPCPU_ID_PTR(i, xen_intr_pcpu);
memset(pcpu->evtchn_enabled, i == 0 ? ~0 : 0, memset(pcpu->evtchn_enabled, i == 0 ? ~0 : 0,
sizeof(pcpu->evtchn_enabled)); sizeof(pcpu->evtchn_enabled));
Show All 22 Linesxen_intrcnt_init(void *dummy __unused)
/* /*
* Register interrupt count manually as we aren't guaranteed to see a * Register interrupt count manually as we aren't guaranteed to see a
* call to xen_intr_assign_cpu() before our first interrupt. * call to xen_intr_assign_cpu() before our first interrupt.
*/ */
CPU_FOREACH(i) CPU_FOREACH(i)
xen_intr_intrcnt_add(i); xen_intr_intrcnt_add(i);
} }
SYSINIT(xen_intrcnt_init, SI_SUB_INTR, SI_ORDER_MIDDLE, xen_intrcnt_init, NULL); SYSINIT(xen_intrcnt_init, SI_SUB_INTR, SI_ORDER_MIDDLE, xen_intrcnt_init, NULL);
void
xen_intr_alloc_irqs(void)
{
if (num_io_irqs > UINT_MAX - NR_EVENT_CHANNELS)
panic("IRQ allocation overflow (num_msi_irqs too high?)");
first_evtchn_irq = num_io_irqs;
num_io_irqs += NR_EVENT_CHANNELS;
}
/*--------------------------- Common PIC Functions ---------------------------*/ /*--------------------------- Common PIC Functions ---------------------------*/
#if defined(__amd64__) || defined(__i386__) #if defined(__amd64__) || defined(__i386__)
/** /**
* Prepare this PIC for system suspension. * Prepare this PIC for system suspension.
*/ */
void void
xen_intr_suspend(void) xen_intr_suspend(void)
▲ Show 20 Lines • Show All 670 Lines • Show Last 20 Lines