Booting ARM11MPCore in SMP secondary cores boot failed due to u-boot...

---

cpu/arm11mpcore/start.S

 .text          0x00000000      0x3e0 cpu/arm11mpcore/start.o
                0x00000000                _start

board/cavium/cns3000/lowlevel_init.S

 .text          0x00011b80      0x400 board/cavium/cns3000/libcns3000.a(lowlevel_init.o)
                0x00011b80                secondary_cores
                0x00011bd8                lowlevel_init

if secondary_cores is put at:
0x 00,04,08,0c,10,14 is ok
0x 18,1c failed

add a ".align 5" to align secondary_cores

---

It is found that the alignment of following code marked in red in lowlevel_init caused the boot failed:

ldr r5, =0x35678855
cmp r6, r5
movne pc, r6

This movne is the very point that the secondary core jump from u-boot to Linux kernel.

---

under failed situation, with RVDS on Core1 with no_reset_and_no_stop,
0x10738 WFI
0x10764 movne

but I got an FFF00010 (DataAbort) @ 0x1077c, and the code seems changed, for previous WFI and movne section cannot be found at the same address.

---

kernel_start -> rest_init -> kernel_init -> smp_prepare_cpu_secondary -> poke_milo ->


ARM Linux Boot Sequence
http://gicl.cs.drexel.edu/people/sevy/linux/ARM_Linux_boot_sequence.html

---

Fixed

The secondary boot failure is caused by kernel which decompress image to 0x8000(32kB), while the u-boot takes around 132kB from 0x0, and the function secondary_cores are linked below 32kB, therefore overwritten by decompressed kernel.

Although it is overwritten, the ICache is enabled on secondary core, therefore, in some situation SMP/SOP boot would pass if the required code are cached.

---

http://sca.uwaterloo.ca/coldfire/gcc-doc/docs/as_67.html

.align abs-expr, abs-expr, abs-expr

Pad the location counter (in the current subsection) to a particular storage boundary. The first expression (which must be absolute) is the alignment required, as described below.

---

Booting ARM Linux SMP on MPCore
http://hi.baidu.com/romashell/blog/item/b9b1cad0c9efe683a1ec9cf4.html

Booting Linux SMP on MPCore
http://www.embexperts.com/viewthread.php?tid=19

ARM Linux Booting Process (ARM920T)
http://ezinearticles.com/?ARM-Linux-Booting-Process&id=4774950

arm smp support patch
http://www.spinics.net/lists/arm-kernel/msg103559.html
http://www.spinics.net/lists/arm-kernel/msg103738.html

---

Performance Measurement on ARM
http://www.pengutronix.de/development/kernel/arm-benchmarks-20100729_en.html