finish implementation

• added manpage
• added default implementation for other architectures
• kcsan functions was implemented
• fault detection fault for arm64. was rewritten

In D25371#559724, @kib wrote:

Do you need them in asm ?

See below, but what's wrong on assembly code? I'm asking as the one who starts his programmers career on 6502 assembly :)
But I really think, in this case, that asm code is more appropriate (in terms of the stability of the generated code) that hacked C - I just hate all ideas like 'read_once ()' or something.

As I understand the idea, the code is approximately

uint32_t
bus_space_poke_4(...)
{
    curthread->td_pcb->pcb_onfault = <peek_handler>;
    res = bus_space_read_4(...);
    dmb(sy);
    curthread->td_pcb->pcb_onfault = NULL;
}

Right.

And this structure immediately illustrates another question. You currently use fsu_fault for handler, which suppresses all kinds of exceptions. But bus_space_read() needs to dereference some pointers, which might fault due to programmer' errors, not because of the hardware state. It would be nice to distinguish the mistake/hardware error situations and react differently.

Well, in ideal world we should use specialized fault handler fired only by small subset of aborts ( I think that reality is that only synchronous external abort[1] is appropriate) and covering only single register to memory load/store (+ necessary synchronization instruction). In real-world implementation I select to not make trap.c more complicated and i think that handling all aborts over single load/store (that is reason, why is executive code implemented in asm) is acceptable compromise.
What do you think?

In D25371#559709, @andrew wrote:

The kcsan implementation for peek will be something like the following (untested):

Yes, it changes CPU IDs. (For FDT case only. I assume that ACPI must always start on CPU0, even if I didn't find it in standard).
Please take in account that"

• actual FreeBSD must start on CPU0. imho, Nathan tried to fix this two/three years ago without success.
• numbering of CPUs in FDT is creative - is typically sequential, but some DT uses sparse numbering inspired by MPIDR. (see https://svnweb.freebsd.org/base/head/sys/gnu/dts/arm64/freescale/fsl-lx2160a.dtsi?revision=361848&view=markup#l160 )
• renumbering in original code also failed if the boot processor was not the first in the cluster.

It also looks OK for me. I have a few small items that we should rethink or fix, but we can do all of these (including pandaboard support) as follow-up fixes. I can commit this as is over the weekend and then we can start discussing improvements (and I also hope to have a little more free time from next week).
Oscar, manu do you agree?
Problematic (for me) items are:

• The ti_dpll_clk_set_freq routine should respect rounding (dpll) and DRY_RUN.
• I’m not a big friend of bulk clock enable in ti_sysc_clock_enable().
• probably , it will be easier to convert scm_clocks and prcm_clocks instances from FDT data driven to compiled-in table driven clocks implementation.

But all these are minor and should be discussed firstly...

Update to the final version. r359280 removed the need to access the cpu_mpidr table from the assembler. So eliminate it and insert the mpidr field into the pcpu structure as originally requested.

Oskar,
Firstly, it seems to have forgotten to thank you for your efforts. I apologize for my discourtesy.
Also allow me to continue here..

Rebased to head. Fixed cpuid assignment

Ping. Andrew, please, do you want to finish (and commit) by yourself? Alternatively, I can manage it if you prefer.

Sorry for late response, i'm busy by real life these days and you forced me to dig into problem much deeper than I wanted :).

Why you choose to use pci_host_generic as base class? By my best knowledge pci_host_generic is driver for ACPI/ECAM compatible PCI root port and it have nothing with Broadcom iProc PCIe controller. I think that it should be derived from ofw_pci_driver. Please see pci_dw.c for template...
(Don't kill me immediately, this change is not that horrible as it looks on first view).

Thanks!

from IRC:
There is at least 11 variants of QoriQ arm64 based SoCs. This is too much for per-SOC options.
We can always add a SOC_NXP_LS_BLAH and !it if one soc differs

The LS family has too many members and can share 99% of the code. It seems unnecessary for me to have the option for each individual variant. How about 'SOC_NXP_LS' ?

LGTM.
I looking forward when my HoneyComb LX2K arrives, order was placed . I got offer (paid by this card :) ) to port FreeBSD to CEx7 LX2160A module. And because customer wants maximum control of clocks, clocks gating, inactive phys and so, only fully featured FDT base port is possible. So, please, be prepared to do some testing of new code on your board :)

Tested on RK3399 in FDT based environment, no problems detected.
LGTM

Imho, It will be better to wait with this until we find consensus with D24351, right ?

This is much better implementation, thanks.
I see only one majoe issue (for me), fortunately it’s easy to fix.
Please, never use synthetic (generated) names for clock mapping, instead use a clock ID for this job. Names must be unique per system and you cannot guarantee this. As opposite, clock ID is property of given clock domain - so you can guarantee that these are unique within single clock domain.
Please combine type cell value and index cell value into unique number and the use clknode_find_by_name() inside qoriq_clkgen_ofw_mapper(). See [1]

LGTM

With full respect, this is nothing that fake driver.
Faking chip clock structure with set of fixed clock nodes with statically computed frequency is not useful at all.
Please try again, this time so that the driver reflects the real clock tree structure.
And because I'm responsible for not having the documentation for clock framework , I offering maximum possible help.

Most boards don't have a battery connected. What happens in this case? I don't see any validity detection in this code

In D23085#505942, @hselasky wrote:

common/include/xxx.h

#if defined(LINUXKPI_BASEDRM)
Do it this way
#else
Do it that way
#endif

basedrm/include/xxx put the DRM header files here which you need.

Static inline functions is your friend.

Maybe Johannes has some opinions too.

Is it possible to add this driver as part of the existing KMS DRM driver?

I think that this is wrong way to fix you issue. We should not rely on preset values from loader. Voltages should be adjusted by drivers, and if this is not possible, the PMIC driver itself should set right values within its initialization - in exactly same way as we do within clocks initialization.

This is slightly controversial. "regulator-min-microvolt" nor"regulator-max-microvolt" are not required properties for fixed regulators so we can't do this so simply. inmo,, we should monitor the presence of these properties and use this info for generate of the result.

oups, my bug. Sorry for trouble.

tested on arm64

imho, some bits of this should be implemented as part of generic fdt_pinctr interface.
We should have at least
FDT_PINCTRL_IS_GPIO()
FDT_PINCTRL_SWITCH_TO__GPIO()
FDT_PINCTRL_SET_GPIO_FLAGS()
FDT_PINCTRL_GET_GPIO_FLAGS()
Major issue is how to get right reference to appropriate pinctrl instance(within gpoio driver) and how to convert pin numbers from gipo nomenclature to pinctrl nomenclature. This should be covered by "gpio-range = <>" property, but world is not ideal :(

Tested on AIO-3288.

i just realized that this breaks EARLY_UART because SOCDEV_PA mapping is not passed to initarm().
So I think that we should postpone change of kernel ttbt0 until cninit() is processed.

@@ -1164,6 +1164,7 @@ initarm(struct arm64_bootparams *abp)
        valid = bus_probe();

Tested on RockPro64 an Tegra TX1 without problems. Unfortunately, it doesn't affect RK3399 issues in either way...

Move common macros to rk_cru.h

With full respect, I have no idea why you want/need these changes to original D13861.
Adding initrd support to top of D13861 is simple (we can use fragments from this patch).
For me most clean way how to solve this situation is:

1. revert original commit.
2. commit D13861 (without putting LINUX_BOOT_ABI to GENERIC)
3. convert this patch to adding initrd support on top of D13861 ( I offering full help for this)
4. fix generation of booti image (by utility program or by some hex editing, having booti header in locore.s is not a right way)

Add own version of clock ID's header without single copyright

Revert previous diff update, committed to wrong review.

I pushed outdated diff, update it to final version

In D22255#489934, @siddharthtuli_gmail.com wrote:

Hi Mmel,

In D22255#489923, @mmel wrote:

In D22255#489896, @siddharthtuli_gmail.com wrote:

Hi Mmel,

Please find my comments inline.

After more detailed analysis, I think that there are more problems.
Let me explain situation. Givens are:

• Actual locore.S mappings of kernel expects that physical memory is aligned to 1GB boundary. This implies that we don’t support boards with lower that 1 GB memory (I don’t see other real limitation caused by this fact).

In locore.S L1 block page table entry is built on 1GiB aligned physical load addr of kernel. However this is irrespective of using loader or U-Boot to boot the FreeBSD kernel.

• U-Boot booti command expect that kernel, fdt, and initrd blobs are loaded at ${kernel_addr_r}, ${fdt_addr_r} and ${initrd_addr_r} offsets.

Yes.
As part of this patch linux_load_initrd() will do build_l1_block_pagetable() to map initrd start address aligned to 1GiB PA=VA.
(By default, initrd_addr_r will be relocated to highmem by booti in U-Boot code (do_bootm_linux->boot_prep_linux->boot_reloc_ramdisk->boot_ramdisk_high)).

locore.S maps 1GiB algined kernel_addr_r PA=VA. I assumed that fdt_addr_r ( a couple of KiB’s in size) can also be within this 1GiB aligned addr. However, fdt_addr_r can be mapped similar to initrd start addr (in linux_load_initrd()).

Real value of these variables is driven by u-boot and we cannot expect nothing about this value. (order, gaps, etc..)

Sure.

It’s possible that kernel want to load kernel at begining of memory and initrd blob at end. We must support this, it's generic part of ‘distro_boot’ u-booot protocol/framework. And support of distro_boot is one of major goals for having booti supported, imo.

Sure. My understanding is that patch handles this. Physical load address of,
kernel is mapped into KVA starting at KERNBASE in locore.S
dtb blob is mapped into KVA starting at [ KERNBASE+(kernel size) ] in machdep_boot.c
initrd is mapped into KVA starting at [ KERNBASE+(kernel size) + (dtb size) ] in machdep_boot.c

This give me some conclusions:
Because of 1) optimal mapping granularity is 1GB, more fine grained granularity is nothing than waste of TLB entries and build_l2_block_pagetable() is useless. (at least I don’t see single advantage of using fine grained mappings, while disadvantage is clear - more TLB pressure ).

[1]
locore.S does maps kernel physical load addr into KVA starting at [ KERNBASE ] to [ KERNBASE + (kernel size)] using build_l2_block_pagetable. KVA is in L2 table which has 512, 2MiB entries. 1GiB L1 table entry is linked to this L2 table.

The following comments from create_pagetables: in locore.S,

• It relys on:
• We were loaded to an address that is on a 2MiB boundary
• All the memory must not cross a 1GiB boundaty
• x28 contains the physical address we were loaded from *

I am mapping dtb and initrd image into L2 table following the end of kernel. Therefore I created build_l2_block_pagetable() in machdep_boot.c
In fact if this not done dtb or initrd cannot be added into KVA. Due to above comments, I added a print in machdep_boot.c if kernel+dtb+initrd will exceed 1GiB.

This is not exact description. You allocates and maps new memory (by using lastaddr) for fdt and initrd blobs. Then you *copy* these blobs to new location. See lines 122 and 127 for fdt, and lines 270, 273 and 275 for initrd.

Basically, I am creating 2MiB KVA L2 table entries to move the fdt and initrd blobs from PA to KVA.

As you can see, you have mapped both source and destination address for initrd blob memmove(), but only destination address was mapped for fdt blob memmove() (point me to right place if I’m wrong, please).

Sorry I did not understand this clearly. inirtd size (in counts of 2MiB) is calculated using initrd-end - inird-start. For dtb the size is available in its header.

Problem is not about initrd but fdt blob
at line 127 we do
memmove((void *)lastaddr, dtb_ptr, dtb_size);
destination memory is mapped at line 122 by
build_l2_block_pagetable(lastaddr, dtb_size, abp
but where is mapped source memory (pointed by dtb_ptr in memmove())????

In D22255#489896, @siddharthtuli_gmail.com wrote:

Hi Mmel,

Please find my comments inline.

After more detailed analysis, I think that there are more problems.
Let me explain situation. Givens are:

• Actual locore.S mappings of kernel expects that physical memory is aligned to 1GB boundary. This implies that we don’t support boards with lower that 1 GB memory (I don’t see other real limitation caused by this fact).

In locore.S L1 block page table entry is built on 1GiB aligned physical load addr of kernel. However this is irrespective of using loader or U-Boot to boot the FreeBSD kernel.

• U-Boot booti command expect that kernel, fdt, and initrd blobs are loaded at ${kernel_addr_r}, ${fdt_addr_r} and ${initrd_addr_r} offsets.

Yes.
As part of this patch linux_load_initrd() will do build_l1_block_pagetable() to map initrd start address aligned to 1GiB PA=VA.
(By default, initrd_addr_r will be relocated to highmem by booti in U-Boot code (do_bootm_linux->boot_prep_linux->boot_reloc_ramdisk->boot_ramdisk_high)).

locore.S maps 1GiB algined kernel_addr_r PA=VA. I assumed that fdt_addr_r ( a couple of KiB’s in size) can also be within this 1GiB aligned addr. However, fdt_addr_r can be mapped similar to initrd start addr (in linux_load_initrd()).

Real value of these variables is driven by u-boot and we cannot expect nothing about this value. (order, gaps, etc..)

Sure.

It’s possible that kernel want to load kernel at begining of memory and initrd blob at end. We must support this, it's generic part of ‘distro_boot’ u-booot protocol/framework. And support of distro_boot is one of major goals for having booti supported, imo.

Sure. My understanding is that patch handles this. Physical load address of,
kernel is mapped into KVA starting at KERNBASE in locore.S
dtb blob is mapped into KVA starting at [ KERNBASE+(kernel size) ] in machdep_boot.c
initrd is mapped into KVA starting at [ KERNBASE+(kernel size) + (dtb size) ] in machdep_boot.c

This give me some conclusions:
Because of 1) optimal mapping granularity is 1GB, more fine grained granularity is nothing than waste of TLB entries and build_l2_block_pagetable() is useless. (at least I don’t see single advantage of using fine grained mappings, while disadvantage is clear - more TLB pressure ).

[1]
locore.S does maps kernel physical load addr into KVA starting at [ KERNBASE ] to [ KERNBASE + (kernel size)] using build_l2_block_pagetable. KVA is in L2 table which has 512, 2MiB entries. 1GiB L1 table entry is linked to this L2 table.

The following comments from create_pagetables: in locore.S,

• It relys on:
• We were loaded to an address that is on a 2MiB boundary
• All the memory must not cross a 1GiB boundaty
• x28 contains the physical address we were loaded from *

I am mapping dtb and initrd image into L2 table following the end of kernel. Therefore I created build_l2_block_pagetable() in machdep_boot.c
In fact if this not done dtb or initrd cannot be added into KVA. Due to above comments, I added a print in machdep_boot.c if kernel+dtb+initrd will exceed 1GiB.

This is not exact description. You allocates and maps new memory (by using lastaddr) for fdt and initrd blobs. Then you *copy* these blobs to new location. See lines 122 and 127 for fdt, and lines 270, 273 and 275 for initrd. As you can see, you have mapped both source and destination address for initrd blob memmove(), but only destination address was mapped for fdt blob memmove() (point me to right place if I’m wrong, please).

After more detailed analysis, I think that there are more problems.
Let me explain situation. Givens are:

1. Actual locore.S mappings of kernel expects that physical memory is aligned to 1GB boundary. This implies that we don’t support boards with lower that 1 GB memory (I don’t see other real limitation caused by this fact).
2. U-Boot booti command expect that kernel, fdt, and initrd blobs are loaded at ${kernel_addr_r}, ${fdt_addr_r} and ${initrd_addr_r} offsets. Real value of these variables is driven by u-boot and we cannot expect nothing about this value. (order, gaps, etc..) It’s possible that kernel want to load kernel at begining of memory and initrd blob at end. We must support this, it's generic part of ‘distro_boot’ u-booot protocol/framework. And support of distro_boot is one of major goals for having booti supported, imo.

This give me some conclusions:
Because of 1) optimal mapping granularity is 1GB, more fine grained granularity is nothing than waste of TLB entries and build_l2_block_pagetable() is useless. (at least I don’t see single advantage of using fine grained mappings, while disadvantage is clear - more TLB pressure ).
Because of 2) – all 3 booti blobs must be taken separately. I don’t see single reason for postponing mapping of FDT to C code. Original locoer.S works and build_l2_block_pagetable() is unusable. Using lastaddr for reservation of memory used by initrd is clearly nonsense. Initrd blob should be reserved by using (probably) arm_physmem_exclude_regions() and map it later , when pmap is fully bootstrapped.

please, use original locore.S from D13861 I'ts tested on many boards and it works :).

Moreover this doesn't boot on RockPro64, where original code from D13861 works without problems. jhibbits, please revert this, this code needs definitely more love. (Please, be sure that I'm first who wants booti support in kernel, but broken support is worse that no support)

I'm sorry for delayed review, but I am overloaded with my current work on Rockchip ports.
But in any case, if you want to reuse someone else's patch, it would be better to include the original author directly as the reviewer.

LGTM

LGTM

rest is OK for me.

The code itself looks OK for me. But I have problem with using its usage as default regulator init method, because:

• it cannot be used by regulators with its own init
• it expect that regulator is accessible early, at registration time. I'm not sure if this is possible in all cases (like I2C and interrupts ..)

Can you, please, give me your motivation for adding new resource types when it’s clear that these resources cannot be handled by resman/newbus without extreme API breakage?
For OFW/FDT world, major issues are:

• • given resource (request, allocated) cannot be described by single integer, and infrastructure must take it fully opaque (there is nothing like ‘resource start’, ‘resource end’ and interval enumerator)
• hierarchy for other resources(clk, regulator, reset, phy, ...) is not bus related and can have circular dependencies – so it cannot be represented by tree, but by graph. This result to fact that bus enumerator cannot prepare resource list for its child devices. This is something which break basic design expectation of newbus.
• circular resource dependencies (between types – frequent, within single resource type – no that frequent but exist mainly for clocks) also means, that we relay need staggered device_attach(), there is no way to do this by automatic ordering of devices attach, driven by resource availability.

Another current newbus or bus_space limitation is that we should have (general, globalized) memory attributes and be should pass it do all memory mapping function. On arm or arm64, PCI config space should be mapped with different attribute (strongly ordered) that pci (or other devices) mmio register space (posted writes are allowed). Actual behavior (all devices are mapped as strongly ordered) have measurable performance impact.
Don’t take me badly, I’m still newbus fan, I only don’t see reliable way how we can extend it without fatal/significant API breakage.

Yep, thanks.