Hi, any comments to the patch?

Do not add redundant 'dma-coherent', and add the property only for the PL310 node. Unlike linux, we do not have to do it in runtime, hacking the platform code.

• Modify nexus dma tag, following https://reviews.freebsd.org/D11202
• Improve commit log
• Change ddr_attr for A38x SoCs, depending on the compatible string in the ofwbus ("/")

Extract @ian 's changes around nexus and modify the commit log.

Need to take over the revision for the patch update.

Correct comments in the code, pointed by @ian

In D11202#233493, @ian wrote:

In D11202#233463, @mw_semihalf.com wrote:

Hi @ian we cleaned the code a bit, solution seems nice, but... bus_dma_tag_create() won't work in platform_late_init stage, because it uses malloc :/ I have some hacks in mind how to overcome it, e.g. create the default dma tag in nexus code only when nexus_set_dma_tag() is called. The caller would only pass the desired flag and trigger tag creation inside nexus. What do you think?

Oh. Hmm, simple fix, add a "void *dummy" arg to ​mv_busdma_tag_init(), and instead of calling it from platform_late_init(), add:

SYSINIT(​mv_dmatag_init, SI_SUB_DRIVERS, SI_ORDER_ANY, ​mv_busdma_tag_init, NULL);

Hi @ian we cleaned the code a bit, solution seems nice, but... bus_dma_tag_create() won't work in platform_late_init stage, because it uses malloc :/ I have some hacks in mind how to overcome it, e.g. create the default dma tag in nexus code only when nexus_set_dma_tag() is called. The caller would only pass the desired flag and trigger tag creation inside nexus. What do you think?

@ian thank - I'll test and let know.

In D11202#233185, @ian wrote:

In D11202#233184, @mw_semihalf.com wrote:

Hi @ian
...
In linux all 'magic' happens in arch/arm/mach-mvebu/coherency.c, which is:

• set coherent dma ops
• runtime PL310 DT node fixup (add "arm,io-coherent") property, due to CONFIG_SMP limitation (not relevant for us)

We tested and unfortunately it seems there is no way to add new property to the DT in FreeBSD (only modify existing one, but the value must be not bigger than original), hence another way should be made up.

I would really like to be able to use some mechanism (e.g. in platform_late_init). So basing your expertise, do you think it is possible to force settings of parent dma tag (nexus, ofwbus, simple-bus - whatever you prefer), so that all children (devices) can inherit this when creating theirs?

Yes, doing it in platform_late_init is just the sort of thing I was imagining, if it is a property of the entire system. Let me put together a little proposal-patch to see if my vague idea is practical.

In D11202#233180, @ian wrote:

In D11202#233164, @mkm_semihalf.com wrote:

I decided to create dma tag in ofwbus and moved 'dma-coherent' property to '/' node in DTS.
All changes related to dma tag creation are now done only if FDT is defined.

The 'dma-coherent' property is not documented on FreeBSD.
On Linux it is used by arm/arm64 code and can be placed in various device and bus nodes (e.g. amba bus in artpec6.dtsi).

That was kind of my point: it's not documented, period. You just made it up, taking us another step further away from supporting standard DTS sources for marvell on freebsd.

How does the standard DTS for marvell systems handle this? How does linux handle it?

If this is truly a platform-wide attribute, not a per-bus attribute, then nexus is where the tag should be.

If standard DTS and linux don't describe this with a property in the dts source, but rather just let the platform code handle it somehow, then we should do something similar rather than creating even more non-standard dts for ourselves.

Improve checking the "arm,io-coherent" property as pointed by @meloun-miracle-cz

I checked, and indeed r320054 fixes problems with obtaing the variables, so this patch is no longer needed.

Add "arm,io-coherent" to PL310 node (see https://reviews.freebsd.org/D11245)

Do not remove cpufuncs.cf_l2cache_drain_writebuf, due to introducing "arm,io-coherent" property for PL310 in https://reviews.freebsd.org/D11245

Tested on Armada 388 Clearfog

Tested on Armada 38x

In D11203#231935, @meloun-miracle-cz wrote:

I do not think this patch was accepted in mainline Linux -> http://lists.infradead.org/pipermail/linux-arm-kernel/2015-August/363953.html

But this one is -> https://github.com/torvalds/linux/commit/98ea2dba65932ffc456b6d7b11b8a0624e2f7b95
Please note that commit log uses "the outer cache sync operation is useless: and not a " the entire outer cache operations are useless".
Also note that this behavior is driven by "arm,io-coherent" property.

Yes, it's all documented in Marvell errata for the support. Both their private and the mainline kernels had to be modified in a similar way. Just please take a look:
https://patchwork.kernel.org/patch/6993601/

Instead of using weak_reference for get_cpu_clk, provide dummy functions for all other Marvell SoC's (same way as get_tclk is done).

Indeed, but what you propose (remove cpu_class check for ARMv7), was done in the first version of this patch - briefly rejected.

Commited to revision 319706

• correct style for all #define and other style(9) fixes
• remove magics
• enable dynamic coherency settings detection instead of option config
• remove SoC ifdefs in mv_common and use custom setup/dump routines for neta
• enable setting MVNETA_MULTIQUEUE and MVNETA_KTR as a config option
• optimize DELAY usage in busy wait loops in miibus methods
• add missing resources free on error in init
• remove HWCSUM_IPV6 ifdefs
• replace fdt_find_compatible with ofw_bus_find_compatible

Replace magic number with macro, as pointed by @bz

Instead of removing cpu_class check in hwpmc, enable its usage amoung ARMv6/v7 SoCs.

Hi @meloun-miracle-cz and @andrew

GL3224 is a dual LUN, but fails to report them properly - the outcome without quirks are errors and undetected devices, connected to it.

Thanks for a brief response. It happened on various Marvell Armada 38x boards + different AR983x cards. With this small fix all init problems were gone.

Switched to nitems, thanks for the suggestion.

The issue was already fixed on HEAD in r312746

Thanks, I've just prepared a patch rebased against your change - it has slightly improved if statements. Verified with the network traffic.

Thanks for the suggestion. It works, so I'll the patch in a moment.

Would it be possible that you commit change pointed by zbb?
https://github.com/strejda/tegra/commit/3b5138751ee5643992b20fcb21b280fab433bb20

Ok, checked - Armada38x comprises r3p3 cache controller revision. I'm uploading second version of the patch, which adds missing condition checks for PL310_ERRATA_727915.

Here's an explanation from gber, who is the commit author:
If my understanding of code responsible for initial mapping is correct, then all the memory reserved from kernel space by pmap_alloc_specials() function called in pmap_bootstrap() should be mapped initially by initarm(). To create initial mapping initarm() function reserves proper number of l2 page tables. However the number of the l2 page tables does not take into account memory for: pmap_kernel_l2ptp_kva, pmap_kernel_l2dtable_kva, crashdumpmap, etc.

Ok, agree, the ifdef is not nice. Thank you for the hint, I'll try it.

Thanks. I'll confirm this and either abandon this patch or get back with something cleaner.

Does GENERIC kernel support Armada 38x SoC? If not, is this really an issue here?