Difference between revisions of "Unit BootRPi3"

From Ultibo.org
Jump to: navigation, search
 
(11 intermediate revisions by the same user not shown)
Line 4: Line 4:
 
=== Description ===
 
=== Description ===
 
----
 
----
 +
 +
'''Ultibo Initialization code for Raspberry Pi 3 unit'''
  
 
'''Raspberry Pi 3'''
 
'''Raspberry Pi 3'''
Line 10: Line 12:
 
   
 
   
 
CPU: Cortex A53 (ARMv8) (4 @ 1200MHz)
 
CPU: Cortex A53 (ARMv8) (4 @ 1200MHz)
−
Cache: L1 32KB (Per Core) / L2 512KB (Shared by all Cores)
+
Cache: L1 32KB (Per Core)/L2 512KB (Shared by all Cores)
 
   
 
   
 
FPU: VFPV3
 
FPU: VFPV3
Line 28: Line 30:
 
Bluetooth: Broadcom (BCM43438)
 
Bluetooth: Broadcom (BCM43438)
  
−
Other: GPIO / SPI / I2C / I2S / PL011 (UART) / PWM / SMI / Watchdog (PM) / Random (RNG) / Timer ???
+
Other: GPIO/SPI/I2C/I2S/PL011 (UART)/PWM/SMI/Watchdog (PM)/Random (RNG)/Timer
 
   
 
   
 +
 
'''Boot RPi3'''
 
'''Boot RPi3'''
  
Line 39: Line 42:
  
 
R2 - Address of the ARM Tags structure (Normally 0x0100)
 
R2 - Address of the ARM Tags structure (Normally 0x0100)
 +
  
 
On entry to this code the processor will be in the following state:
 
On entry to this code the processor will be in the following state:
Line 63: Line 67:
 
If the processor is in Hypervisor mode (Firmware behaviour after 2/10/2015) then Ultibo switches it to Supervisor mode during initial boot.
 
If the processor is in Hypervisor mode (Firmware behaviour after 2/10/2015) then Ultibo switches it to Supervisor mode during initial boot.
 
   
 
   
−
If the configuration option RPI3_SECURE_BOOT is set to 1 then Ultibo switches the processor back to Secure world during initial boot (see note below).
+
If the configuration option ARMSecureBoot is set to 1 then Ultibo switches the processor back to Secure world during initial boot (see note below).
 
   
 
   
 
Ultibo then switches the processor to System mode for all operations and remains in either the Secure or Non Secure World as per the option above.
 
Ultibo then switches the processor to System mode for all operations and remains in either the Secure or Non Secure World as per the option above.
Line 70: Line 74:
 
   
 
   
 
Note that this code is currently identical to the Raspberry Pi 2 boot code but has been separated to allow for supporting 64 bit mode on the ARMv8 in future. Ultibo currently runs the RPi3 in 32 bit mode which is almost identical to ARMv7.
 
Note that this code is currently identical to the Raspberry Pi 2 boot code but has been separated to allow for supporting 64 bit mode on the ARMv8 in future. Ultibo currently runs the RPi3 in 32 bit mode which is almost identical to ARMv7.
 +
 +
 +
'''ARM64 support on Raspberry Pi 3'''
 +
 +
Very little is known so far about the implementation of 64 bit mode by the RPi firmware, however some details have started to appear.
 +
 +
There is a config.txt switch (arm_control=0x200) which causes the firmware to boot all 4 cores in 64 bit mode instead of 32 bit mode. It appears that currently the firmware does not supply any loader code when booting in 64 bit mode and the switch kernel_old=1 also needs to be used to load the kernel image at address 0x00000000. This also means that the kernel image must deal with the startup of all 4 cores and holding the secondaries until the primary has completed the system initialization. No ATAG parameters are provided when using kernel_old=1 so command line parameters would not be available.
 +
 +
See this post for known information: https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963
 +
 
 +
It is likely that a 64 bit loader will be included in the firmware at some point in order to support Linux boot in 64 bit mode. Without it device tree support etc would not be available and therefore make Linux use the old style device setup.
 +
 +
 +
'''Returning to Secure World:'''
 +
 +
The Raspberry Pi 3 firmware always switches to Non Secure world in order to:
 +
 +
a) Reset CNTVOFF (Virtual Offset register) to zero
 +
 
 +
b) Switch to Hypervisor mode (Firmware later than 2/10/2015)
 +
 
 +
 +
It is not normally possible (by design) to return from the Non Secure world to the Secure world except by invoking the Secure Monitor which then handles any requests in the Secure world.
 +
 +
However due to the fact that this switch is performed by the boot loader code (loaded between  0x00000000 and 0x000000FF by the firmware) and that the code appears to set the secure, non secure and monitor vector base addresses to 0x00000000 and because all memory is writable from non secure at boot (the MMU is disabled) then the option exists to return to the Secure world as follows:
 +
 +
- Install a new Secure Monitor Call vector at the appropriate point in the vector table
 +
 +
- Clean the data cache by Modified Virtual Address at address 0x00000000
 +
 +
- Perform a data synchronisation barrier
 +
 +
- Invalidate the entire instruction cache
 +
 +
- Perform a data synchronisation barrier
 +
 +
- Perform an instruction synchronisation barrier
 +
 +
- Perform a Secure Monitor Call (SMC)
 +
 
 +
- From within the newly installed Secure Monitor Call read the Secure Configuration Register (SCR) mask off the Non Secure (NS) bit, rewrite the SCR and then return to Supervisor mode.
 +
 +
 +
Secondary cores also switch back to the Secure world using the same technique however they do not need to install the new SMC vector and instead use the one installed by the primary core.
 +
 +
Firmware changes from October 2020 disable the ability to perform the SMC instruction from non secure mode which means this technique will no longer work with the default firmware. To allow continued support for returning to the Secure world a custom ARM boot stub is now included with Ultibo that allows the SMC instruction to be performed.
 +
 +
This custom stub contains a marker that allows the boot process to detect it and disable the attempt to switch back to the Secure world if the default boot stub is used instead.
 +
 +
As the default boot stub continues to evolve in response to demands from the Linux community this technique will allow Ultibo to continue to provide the features we want without having to make frequent changes to combat items that are beyond our control.
 +
 +
Thanks to the Circle project for introducing the custom boot stub technique to work around these changes.
  
 
=== Constants ===
 
=== Constants ===
 
----
 
----
  
−
''To be documented''
+
''None defined''
  
 
=== Type definitions ===
 
=== Type definitions ===
 
----
 
----
  
−
''To be documented''
+
''None defined''
  
 
=== Public variables ===
 
=== Public variables ===
 
----
 
----
  
−
''To be documented''
+
''None defined''
  
 
=== Function declarations ===
 
=== Function declarations ===
 
----
 
----
  
−
''To be documented''
 
  
 +
'''Boot functions'''
 +
 +
<div class="toccolours mw-collapsible mw-collapsed" style="border: 1; font-family: arial; padding-top: 0px; padding-bottom: 15px;">
 +
<pre style="border: 0; padding-bottom:0px;">procedure Startup; assembler; nostackframe; public name '_START';</pre>
 +
<div style="font-size: 14px; padding-left: 12px;">'''Description:''' Entry point of Ultibo on Raspberry Pi 3, this will be the very first byte executed and will be loaded by the GPU at address 0x00008000</div>
 +
<div class="mw-collapsible-content" style="text-align: left; padding-left: 5px;">
 +
{| class="wikitable" style="font-size: 14px; background: white;"
 +
|-
 +
! Note
 +
| None documented
 +
|-
 +
|}
 +
</div></div>
 +
<br />
 +
<div class="toccolours mw-collapsible mw-collapsed" style="border: 1; font-family: arial; padding-top: 0px; padding-bottom: 15px;">
 +
<pre style="border: 0; padding-bottom:0px;">procedure Vectors; assembler; nostackframe;</pre>
 +
<div style="font-size: 14px; padding-left: 12px;">'''Description:''' ARM exception vector table which is copied to the vector base address by the StartupHandler</div>
 +
<div class="mw-collapsible-content" style="text-align: left; padding-left: 5px;">
 +
{| class="wikitable" style="font-size: 14px; background: white;"
 +
|-
 +
! Note
 +
| See A2.6 "Exceptions" of the ARM Architecture Reference Manual
 +
|-
 +
|}
 +
</div></div>
 +
<br />
 +
<div class="toccolours mw-collapsible mw-collapsed" style="border: 1; font-family: arial; padding-top: 0px; padding-bottom: 15px;">
 +
<pre style="border: 0; padding-bottom:0px;">procedure SecureVectors; assembler; nostackframe;</pre>
 +
<div style="font-size: 14px; padding-left: 12px;">'''Description:''' ARM secure vector table which is copied to the sector vector base address by the StartupHandler</div>
 +
<div class="mw-collapsible-content" style="text-align: left; padding-left: 5px;">
 +
{| class="wikitable" style="font-size: 14px; background: white;"
 +
|-
 +
! Note
 +
| See A2.6 "Exceptions" of the ARM Architecture Reference Manual
 +
|-
 +
|}
 +
</div></div>
 +
<br />
 +
<div class="toccolours mw-collapsible mw-collapsed" style="border: 1; font-family: arial; padding-top: 0px; padding-bottom: 15px;">
 +
<pre style="border: 0; padding-bottom:0px;">procedure SecureMonitor; assembler; nostackframe;</pre>
 +
<div style="font-size: 14px; padding-left: 12px;">'''Description:''' Secure monitor mode handler to switch to secure mode</div>
 +
<div class="mw-collapsible-content" style="text-align: left; padding-left: 5px;">
 +
{| class="wikitable" style="font-size: 14px; background: white;"
 +
|-
 +
! Note
 +
| None documented
 +
|-
 +
|}
 +
</div></div>
 +
<br />
 +
<div class="toccolours mw-collapsible mw-collapsed" style="border: 1; font-family: arial; padding-top: 0px; padding-bottom: 15px;">
 +
<pre style="border: 0; padding-bottom:0px;">procedure StartupSwitch; assembler; nostackframe;</pre>
 +
<div style="font-size: 14px; padding-left: 12px;">'''Description:''' Startup handler routine to switch from hypervisor mode</div>
 +
<div class="mw-collapsible-content" style="text-align: left; padding-left: 5px;">
 +
{| class="wikitable" style="font-size: 14px; background: white;"
 +
|-
 +
! Note
 +
| None documented
 +
|-
 +
|}
 +
</div></div>
 +
<br />
 +
<div class="toccolours mw-collapsible mw-collapsed" style="border: 1; font-family: arial; padding-top: 0px; padding-bottom: 15px;">
 +
<pre style="border: 0; padding-bottom:0px;">procedure StartupSecure; assembler; nostackframe;</pre>
 +
<div style="font-size: 14px; padding-left: 12px;">'''Description:''' Startup handler routine to switch to secure mode</div>
 +
<div class="mw-collapsible-content" style="text-align: left; padding-left: 5px;">
 +
{| class="wikitable" style="font-size: 14px; background: white;"
 +
|-
 +
! Note
 +
| None documented
 +
|-
 +
|}
 +
</div></div>
 +
<br />
 +
<div class="toccolours mw-collapsible mw-collapsed" style="border: 1; font-family: arial; padding-top: 0px; padding-bottom: 15px;">
 +
<pre style="border: 0; padding-bottom:0px;">procedure StartupHandler; assembler; nostackframe;</pre>
 +
<div style="font-size: 14px; padding-left: 12px;">'''Description:''' Startup handler routine executed to start the Ultibo kernel</div>
 +
<div class="mw-collapsible-content" style="text-align: left; padding-left: 5px;">
 +
{| class="wikitable" style="font-size: 14px; background: white;"
 +
|-
 +
! Note
 +
| None documented
 +
|-
 +
|}
 +
</div></div>
 +
<br />
  
 
Return to [[Unit_Reference|Unit Reference]]
 
Return to [[Unit_Reference|Unit Reference]]

Latest revision as of 01:17, 22 April 2022

Return to Unit Reference


Description

Ultibo Initialization code for Raspberry Pi 3 unit

Raspberry Pi 3

SoC: Broadcom BCM2837

CPU: Cortex A53 (ARMv8) (4 @ 1200MHz) Cache: L1 32KB (Per Core)/L2 512KB (Shared by all Cores)

FPU: VFPV3

GPU: Broadcom VideoCore IV (VC4)

RAM: 1GB

USB: Synopsys DesignWare Hi-Speed USB 2.0 On-The-Go Controller (DWCOTG)

LAN: SMSC LAN9514 (SMSC95XX)

SD/MMC: Arasan (BCM2710)

WiFi: Broadcom (BCM43438)

Bluetooth: Broadcom (BCM43438)

Other: GPIO/SPI/I2C/I2S/PL011 (UART)/PWM/SMI/Watchdog (PM)/Random (RNG)/Timer


Boot RPi3

The boot loader on the Raspberry Pi 3 will load this code at address 0x00008000 onwards and set the following registers before jumping to this code.

R0 - Zero

R1 - Machine Type (Raspberry Pi 3 or BCM2710 = 0x0C42)

R2 - Address of the ARM Tags structure (Normally 0x0100)


On entry to this code the processor will be in the following state:

World - Non Secure (Firmware causes a switch to Non Secure before jumping to Startup code)

Mode - Supervisor (ARM_MODE_SVC) Note: Firmware later than 2/10/2015 will boot in Hypervisor mode (ARM_MODE_HYP)

MMU - Disabled

FPU - Disabled

L1 Data Cache - Enabled (Firmware enabled prior to Non Secure switch)

L1 Instruction Cache - Enabled (Firmware enabled prior to Non Secure switch)

Branch Predication - Disabled

Unaligned Data Access - Enabled (Always enabled on ARMv8)

SMP Coherence - Enabled (Firmware enabled prior to Non Secure switch)


If the processor is in Hypervisor mode (Firmware behaviour after 2/10/2015) then Ultibo switches it to Supervisor mode during initial boot.

If the configuration option ARMSecureBoot is set to 1 then Ultibo switches the processor back to Secure world during initial boot (see note below).

Ultibo then switches the processor to System mode for all operations and remains in either the Secure or Non Secure World as per the option above.

The initialization process enables the MMU, FPU, L1 Cache and other performance optimizations.

Note that this code is currently identical to the Raspberry Pi 2 boot code but has been separated to allow for supporting 64 bit mode on the ARMv8 in future. Ultibo currently runs the RPi3 in 32 bit mode which is almost identical to ARMv7.


ARM64 support on Raspberry Pi 3

Very little is known so far about the implementation of 64 bit mode by the RPi firmware, however some details have started to appear.

There is a config.txt switch (arm_control=0x200) which causes the firmware to boot all 4 cores in 64 bit mode instead of 32 bit mode. It appears that currently the firmware does not supply any loader code when booting in 64 bit mode and the switch kernel_old=1 also needs to be used to load the kernel image at address 0x00000000. This also means that the kernel image must deal with the startup of all 4 cores and holding the secondaries until the primary has completed the system initialization. No ATAG parameters are provided when using kernel_old=1 so command line parameters would not be available.

See this post for known information: https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963

It is likely that a 64 bit loader will be included in the firmware at some point in order to support Linux boot in 64 bit mode. Without it device tree support etc would not be available and therefore make Linux use the old style device setup.


Returning to Secure World:

The Raspberry Pi 3 firmware always switches to Non Secure world in order to:

a) Reset CNTVOFF (Virtual Offset register) to zero

b) Switch to Hypervisor mode (Firmware later than 2/10/2015)


It is not normally possible (by design) to return from the Non Secure world to the Secure world except by invoking the Secure Monitor which then handles any requests in the Secure world.

However due to the fact that this switch is performed by the boot loader code (loaded between 0x00000000 and 0x000000FF by the firmware) and that the code appears to set the secure, non secure and monitor vector base addresses to 0x00000000 and because all memory is writable from non secure at boot (the MMU is disabled) then the option exists to return to the Secure world as follows:

- Install a new Secure Monitor Call vector at the appropriate point in the vector table

- Clean the data cache by Modified Virtual Address at address 0x00000000

- Perform a data synchronisation barrier

- Invalidate the entire instruction cache

- Perform a data synchronisation barrier

- Perform an instruction synchronisation barrier

- Perform a Secure Monitor Call (SMC)

- From within the newly installed Secure Monitor Call read the Secure Configuration Register (SCR) mask off the Non Secure (NS) bit, rewrite the SCR and then return to Supervisor mode.


Secondary cores also switch back to the Secure world using the same technique however they do not need to install the new SMC vector and instead use the one installed by the primary core.

Firmware changes from October 2020 disable the ability to perform the SMC instruction from non secure mode which means this technique will no longer work with the default firmware. To allow continued support for returning to the Secure world a custom ARM boot stub is now included with Ultibo that allows the SMC instruction to be performed.

This custom stub contains a marker that allows the boot process to detect it and disable the attempt to switch back to the Secure world if the default boot stub is used instead.

As the default boot stub continues to evolve in response to demands from the Linux community this technique will allow Ultibo to continue to provide the features we want without having to make frequent changes to combat items that are beyond our control.

Thanks to the Circle project for introducing the custom boot stub technique to work around these changes.

Constants

None defined

Type definitions

None defined

Public variables

None defined

Function declarations


Boot functions 

procedure Startup; assembler; nostackframe; public name '_START';
Description: Entry point of Ultibo on Raspberry Pi 3, this will be the very first byte executed and will be loaded by the GPU at address 0x00008000
Note None documented 


procedure Vectors; assembler; nostackframe;
Description: ARM exception vector table which is copied to the vector base address by the StartupHandler
Note See A2.6 "Exceptions" of the ARM Architecture Reference Manual 


procedure SecureVectors; assembler; nostackframe;
Description: ARM secure vector table which is copied to the sector vector base address by the StartupHandler
Note See A2.6 "Exceptions" of the ARM Architecture Reference Manual 


procedure SecureMonitor; assembler; nostackframe;
Description: Secure monitor mode handler to switch to secure mode
Note None documented 


procedure StartupSwitch; assembler; nostackframe;
Description: Startup handler routine to switch from hypervisor mode
Note None documented 


procedure StartupSecure; assembler; nostackframe;
Description: Startup handler routine to switch to secure mode
Note None documented 


procedure StartupHandler; assembler; nostackframe;
Description: Startup handler routine executed to start the Ultibo kernel
Note None documented


Return to Unit Reference

Retrieved from "https://ultibo.org/mediawiki_ultibo_org/index.php?title=Unit_BootRPi3&oldid=5317"