I think it's the wrong way. The initiation of child PICs is the job of the parent PIC and must be done in right order and in coordination with the parent PIC. Please see D43452 for more details.

However, these new IPIs still generate interrupts/exceptions - so some form of more or less abstract PIC is still needed. Of course, if these instructions have evolved into a fixed part of the CPU (that is, the IPI will be generated by the exact instruction and raise an exact exception), we can hardcode it and omit it from INTRNQ...

Thanks fixing my bugs..

Maximum PLL lock time is 20 microsecond. Alternatively, we may use the 'us' to avoid UTF-8 in source files.

I'm afraid that using virtual time outside the hypervisor environment is somewhat problematic, or creates a new (not always fulfilled) dependency on firmware. The virtual timer uses the value stored in CNTVOFF as an offset to generate the timer value. CNTVOFF is stored in a bank with an undefined reset value, so it *MUST* be initialized, otherwise we can end up with a situation where each core will have a different, non-zero value - so each core will have a different time. So the actual code expects someone (firmware, hypervisor) to initialize CNTVOFF on all cores - which is not always true (or better said, it is uncommon in the ARMv7 world and guaranteed in ARM64 ).

This broke (at least) armv7 cross-buildworld on amd64.

I certainly don't want to block your work. I just don't want to introduce new KPIs that are problematic ( at least for me) for further expansion , nothing else.

I see. So if I assume that the full implementation should have multiple roots (per type). My main objection is about using bitfield for interrupt type. Is misleading, not much compatible with multiple root implementation and it allows only single bit value almost everywhere.

The problem is that the FIQ tree may have a different topology than a normal interrupts. So instead of passing flags to the root handler, we should create a new root for the FIQ tree (or for any other type of interrupt), including a new intr_pic_claim_root_<foo>() (or we can convert intr_irq_root_dev and irq_root_* to arrays or so...).

I strongly disagree, it's a clear step backwards. Intrng was designed with strong layer separation and you are trying to break it in a similar way to x86.
The consumer interface for interrupts is in the kern_intr.c file and no one else is allowed to use it. The provider interface is intr_isrc_*. All other functions should (and must) remain private or unpublished outside of intrng.

More than enough, thanks. And sorry for starting this in the wrong review.

I see. So in this case can we enable swp emulation using sysctl, disabled by default? I'm worried that this may cause bad behavior for ports build for arm32 - the port may detect swp during build. A bit paranoid, I know.

Hmm, the SWP/SWPB instruction is obsolete in ARMv7 ISA (in favor of atomic operations). It not supported/implemented on ARMv7 SMP cpus and arm32 kernel doesn't support it at all. IMHO we should be consistent at least between arm32 and compat32 on aarch64.

fyi, original idea was provide (not-preferred) way how to (limited) boot FreeBSD on a board with a problematic u-boot that might be locked or need lots of GPL or Linux specific utilities. It was easy for arm64 (where any u-boot can handle a booti image that is generic and has an easy format), but it's hard for armv7, where bootm needs an external program and the resulting image is not generic - it has a built-in load and boot address.
from my current point of view something like user submake stubs will be better (in simple words - be may include and run user definable makefile fragment) -> so system make can try to run this fragment for postprocessing

I like that a lot, I'd do it anyway, thanks. .

IMHO NEW_PCIB is an essential feature for all arm/arm64 FDT based boards. Or more generally, it is essential for all boards where the bootloader/firmware does not fully initialize all PCI hardware, i.e. all systems without ACPI.

I don't think this feature is appropriate for a generic layer. Practically no physical clock can implement it. A PLL can typically produce trillions of discrete frequencies, which makes it practically impossible to enumerate.

LGTM

Thanks .

Isn't that a bit (I don't want to say too much) excessive? All that's needed is to teach sysctl about skipping deleted (empty) intrnames, and to add topology sx lock to deliver a consistent view into user space (plus an allocator for free intrnames item in isrc->isrc_index). I'm almost certain that this big unnecessary change is uncommittable.

I see a misunderstanding here.
INTR_SOLO have nothing to do with shared interrupts. It's a fast path for cascading interrupts, and instead of removing it, we should extend it to support pass-through interrupts, polish the API for it, and put it in the regular build. Both of these are common in the arm/aarch64 world, and we lose performance significantly in these cases (and the big question is whether this should help us for LPI interrupts as well).

I don't have a single problem with intr_irq_root_dev partitioning. I feel that this implementation (priority based selection) is probably not viable in the long term and it seems to be RISC-V specific (i.e. not suitable for a platform independent framework). But decoupling the IPI controller from the root controller will allow us to do it all in a platform-specific "virtual" layer, so I don't went block you. But I have a slight problem with keeping the fallback IPI functionality in the root interrupt controller. We actually only have a few root controllers in the codebase, so adding a one-line intr_ipi_pic_register() function to all of them is not a big deal, which an reduce future bloating of spaghettis like functionally. What do you think?

Perfect, many thanks for cooperation.

I'm so sorry, but NACK.
This is the wrong way. A thermal sensor is not a thermal zone - > one sensor can be part of multiple zones, and one zone can consist of multiple sensors. The zone is a property of the board/case, while the sensor is (mostly) a property of the SOC. We should not mix them.
Moreover an in this case, since the sensors are located on the SoC, there is no reason why we should not hardcode its name (and number of sensors) directly in the driver.

That's fine with me, thank you.

Only minor nit.
Is necessary to shadow HWPARAMS? Shadowing the HW registry is always a pain and a source of confusion and errors. IMHO it should only be used as a last resort...

Thanks

And why didn't you leave it enabled? IMHO all HWACCEL blocks look the same, so the risk of a possible error is minimal.

How is '#ifdef HWACCEL3' compatible with the generic kernel and why do we need it at all?

It solved my problem and it boots without any problems on all my arm64 boards. Thanks.

ping

In D34031#777724, @mindal_semihalf.com wrote:

Hmm, I just did a simple test by adding a call to kdb_backtrace to ofw_bus_default_get_node and removing ofw_pci.c from the build.
Is this the scenario where you expected ENXIO to be returned from ofw_bus_get_node?
I'm not sure if I understand you correctly here.

In D34031#777672, @mindal_semihalf.com wrote:

This only applies to buses derived from simple/ofwbus, not to others (e.g. pci). For other device objects, the ofw_bus_get_node() function returns ENXIO (which may be a valid node ID). However, calling a class function on an object that is not derived from the given class should be considered an error.

Do you mean the case when kobj_lookup_method fails to find anything and we end up in kobj_error_method?
I don't think this is a possible scenario.
The default implementation - ofw_bus_default_get_node returns -1.
Note that BUS_GET_PROPERTY being resolved to simplebus_get_property implies that the bus was derived from ofw.
Because of that the OFW_BUS_GET_NODE will always have an implementation, possibly the default one.

Yes, I mean we end up in kobj_error_method. And that's a very real scenario. In newbus, the default method is only applied for the given class and its subclasses. *Not for all classes*

Unfortunately, it looks bad.
Up until SDHCI v3.0, SDMA is strictly 32-bit, only ADMA mode (which we don't support yet, it is optionally 64-bit). As you can see at https://cgit.freebsd.org/src/tree/sys/dev/sdhci/sdhci.c#n1977, the driver ends up writing only the 32-part of buffer physical address to the DMA base register.

Just a very minor notice . Wouldn't it be better to name the VM memory attribute by function, not by type? Different architectures may have different attribute mapping requirements for the PCI configuration space, and something like a global VM_MEMATTR_PCI_CONFIG (may be aliased to platform specific VM_MEMATTR_DEVICE_NP on arm64) gives us a much better chance of having cross-platform drivers without #ifdef hell.

Ahh right, thanks.

There is also a tegra pcie driver that is common to arm and arm64 , so we should define VM_MEMATTR_DEVICE_NP for arm as well. But I'm not sure if we have the rest of the necessary infrastructure implemented on arm.

In D34031#776866, @mindal_semihalf.com wrote:

That's exactly how I initially did this, take a look at https://reviews.freebsd.org/D33457?id=100979#inline-208849.

I see and I think that original implementation was more correct.

I don't think that's right. We can't expect that all indirect descendants of simplebus were instantiated by simplebus. Typically all enumerable buses (e.g. pci or multifunction devices represented by single FDT node) are not derived from simplebus. You cannot use ofw_bus_get_node(child) on those.
IMHO we can have a single generic implementation of <foo>_get_property for a given bus , but it should be explicitly defined in the device_methods structure for all appropriate drives.

Yes, I've applied all the patches. The problem is that the HS SD card (i.e. without debugging) also doesn't work and I don't know why.

Due to problems with the D32706 I can't verify this on Honeycomb, plus I've never been able to have a working HW tuning. The current state is that this whole series of patches leaves Honeycomb with a non-functioning SD and eMMC. I currently have almost no free time to hack this problem, so everything is taking forever, sorry :(

First of all, I apologize for being too harsh.
The problem is that this patch breaks both SD and eMMC on Honeycomb (LX2160). the eMMC is not detected at all, abd HSSSD card fires out with tons of CRC errors.
Unfortunately however this patch break SD and eMMC on Honezcomb (LX2160).
After further digging, I thing that fsl_sdhc_fdt_set_clock() is broken from day zero.
fsl_sdhc_fdt_set_clock() was supposed to be paired with fsl_sdhc_fdt_get_clock() to perform a conversion of differences between SDHCI and eSDHCl clock register format.

Unfortunately this does not work at all and should be reverted.
eSDHC implements two versions of the clock divider, which are selected by the ESDHCCTL[CRS] bit.

LGTM

Tried and tested - it fixes my previous problem, thanks.

You know I'm "creative", especially when it comes to booting SBC.
The failing sequence is:

Unfortunately thiss broke tftp kernel loading.
The tftp_open() function will prefetch the first block and leave tftpfile->currblock set to 1, so the first block is never loaded into the cache.
The trivial fix is to reset it back to 0, it works for me but I'm not sure if that's the right solution.

Sorry for the strong language, please don't take it the wrong way.
clknode_init_freq_value() - this function is completely unnecessary, and only complicates an already complicated interface. Moreover, it is only used here as papering over a real problem in sysctl dump code. Ii really don't want that.

LGTM