HomeFreeBSD

drm: Dereference pointers given to qsort_r()'s cmp callback
rS258549Unpublished

Unpublished Commit ยท Learn More

No further details are available.

Description

drm: Dereference pointers given to qsort_r()'s cmp callback

drm_le_cmp() (qsort_r()'s callback) receives pointers to elements in the
array passed to qsort_r(), not the elements themselves.

Before this fix, the use of qsort_r() shuffled the array, not sorted it,
because the compare callback accessed random memory locations, not the
expected elements.

This bug triggered an infinite loop in KDE/xserver:

  1. KDE has a kded module called "randrmonitor" which queries xserver for current monitors at startup and then listens to RandR notifications from xserver.
  2. xserver handles the query from "randrmonitor" by polling the video device using the "drm_mode_getconnector()" ioctl. This ioctl returns a list of connectors and, for those with a connected monitor, the available modes. Each modes list is sorted by the kernel before returning. When xserver gets the connectors list, it sorts the modes lists again.

    In the case of this bug, when two modes are equal (in xserver's compare function PoV), their order is kept stable (ie. the kernel order is kept for those two modes). And because the list was shuffled by the kernel, the order of two equal modes was frequently changed in the final modes list in xserver.
  3. xserver compares the returned connectors list with the list obtained earlier. In particular, it compares the sorted modes lists for each connector. If a property of a connector changes (eg. modes), xserver sends a "RRNotify_OutputChange" notification.

    Because of the change of order between equal modes, xserver sent a notification after each polling of the connectors.
  4. "randrmonitor" receives a notification, triggered by its query. The notification doesn't contain the new connectors list, therefore, it asks for the new list using the same function: go back to step #2.

MFC after: 3 days

Details

Provenance
dumbbellAuthored on
Parents
rS258548: Enable reset mechanism for rk3188.
Branches
Unknown
Tags
Unknown

Event Timeline