Difference between revisions of "Unit BootRPi3"

From Ultibo.org
Jump to: navigation, search
 
(7 intermediate revisions by the same user not shown)
Line 4: Line 4:
 
=== Description ===
 
=== Description ===
 
----
 
----
 +
 +
'''Ultibo Initialization code for Raspberry Pi 3 unit'''
  
 
'''Raspberry Pi 3'''
 
'''Raspberry Pi 3'''
Line 10: Line 12:
 
   
 
   
 
CPU: Cortex A53 (ARMv8) (4 @ 1200MHz)
 
CPU: Cortex A53 (ARMv8) (4 @ 1200MHz)
−
Cache: L1 32KB (Per Core) / L2 512KB (Shared by all Cores)
+
Cache: L1 32KB (Per Core)/L2 512KB (Shared by all Cores)
 
   
 
   
 
FPU: VFPV3
 
FPU: VFPV3
Line 28: Line 30:
 
Bluetooth: Broadcom (BCM43438)
 
Bluetooth: Broadcom (BCM43438)
  
−
Other: GPIO / SPI / I2C / I2S / PL011 (UART) / PWM / SMI / Watchdog (PM) / Random (RNG) / Timer ???
+
Other: GPIO/SPI/I2C/I2S/PL011 (UART)/PWM/SMI/Watchdog (PM)/Random (RNG)/Timer
 
   
 
   
 +
 
'''Boot RPi3'''
 
'''Boot RPi3'''
  
Line 39: Line 42:
  
 
R2 - Address of the ARM Tags structure (Normally 0x0100)
 
R2 - Address of the ARM Tags structure (Normally 0x0100)
 +
  
 
On entry to this code the processor will be in the following state:
 
On entry to this code the processor will be in the following state:
Line 59: Line 63:
  
 
SMP Coherence - Enabled (Firmware enabled prior to Non Secure switch)  
 
SMP Coherence - Enabled (Firmware enabled prior to Non Secure switch)  
 +
 
   
 
   
 
If the processor is in Hypervisor mode (Firmware behaviour after 2/10/2015) then Ultibo switches it to Supervisor mode during initial boot.
 
If the processor is in Hypervisor mode (Firmware behaviour after 2/10/2015) then Ultibo switches it to Supervisor mode during initial boot.
 
   
 
   
−
If the configuration option RPI3_SECURE_BOOT is set to 1 then Ultibo switches the processor back to Secure world during initial boot (see note below).
+
If the configuration option ARMSecureBoot is set to 1 then Ultibo switches the processor back to Secure world during initial boot (see note below).
 
   
 
   
 
Ultibo then switches the processor to System mode for all operations and remains in either the Secure or Non Secure World as per the option above.
 
Ultibo then switches the processor to System mode for all operations and remains in either the Secure or Non Secure World as per the option above.
Line 69: Line 74:
 
   
 
   
 
Note that this code is currently identical to the Raspberry Pi 2 boot code but has been separated to allow for supporting 64 bit mode on the ARMv8 in future. Ultibo currently runs the RPi3 in 32 bit mode which is almost identical to ARMv7.
 
Note that this code is currently identical to the Raspberry Pi 2 boot code but has been separated to allow for supporting 64 bit mode on the ARMv8 in future. Ultibo currently runs the RPi3 in 32 bit mode which is almost identical to ARMv7.
 +
  
 
'''ARM64 support on Raspberry Pi 3'''
 
'''ARM64 support on Raspberry Pi 3'''
Line 80: Line 86:
 
It is likely that a 64 bit loader will be included in the firmware at some point in order to support Linux boot in 64 bit mode. Without it device tree support etc would not be available and therefore make Linux use the old style device setup.
 
It is likely that a 64 bit loader will be included in the firmware at some point in order to support Linux boot in 64 bit mode. Without it device tree support etc would not be available and therefore make Linux use the old style device setup.
 
   
 
   
−
'''Returning to Secure World'''
+
 
 +
'''Returning to Secure World:'''
 
   
 
   
 
The Raspberry Pi 3 firmware always switches to Non Secure world in order to:
 
The Raspberry Pi 3 firmware always switches to Non Secure world in order to:
Line 88: Line 95:
 
b) Switch to Hypervisor mode (Firmware later than 2/10/2015)
 
b) Switch to Hypervisor mode (Firmware later than 2/10/2015)
 
    
 
    
 +
 
It is not normally possible (by design) to return from the Non Secure world to the Secure world except by invoking the Secure Monitor which then handles any requests in the Secure world.
 
It is not normally possible (by design) to return from the Non Secure world to the Secure world except by invoking the Secure Monitor which then handles any requests in the Secure world.
 
   
 
   
Line 106: Line 114:
 
- Perform a Secure Monitor Call (SMC)
 
- Perform a Secure Monitor Call (SMC)
 
    
 
    
−
- From within the newly installed Secure Monitor Call read the Secure Configuration Register (SCR) mask off the Non Secure (NS) bit, rewrite the SCR and then return to Supervisor mode
+
- From within the newly installed Secure Monitor Call read the Secure Configuration Register (SCR) mask off the Non Secure (NS) bit, rewrite the SCR and then return to Supervisor mode.
 
   
 
   
 +
 
Secondary cores also switch back to the Secure world using the same technique however they do not need to install the new SMC vector and instead use the one installed by the primary core.
 
Secondary cores also switch back to the Secure world using the same technique however they do not need to install the new SMC vector and instead use the one installed by the primary core.
 
   
 
   
−
Future changes to the firmware will hopefully not invalidate this technique, however there is also the option to use the config.txt parameter kernel_old=1 which would load Ultibo at 0x00000000 and allow us to setup the boot state independant of the firmware. This would require changes to both Ultibo (this boot module) and to the FPC compiler to change the linker start location.
+
Firmware changes from October 2020 disable the ability to perform the SMC instruction from non secure mode which means this technique will no longer work with the default firmware. To allow continued support for returning to the Secure world a custom ARM boot stub is now included with Ultibo that allows the SMC instruction to be performed.  
 +
 +
This custom stub contains a marker that allows the boot process to detect it and disable the attempt to switch back to the Secure world if the default boot stub is used instead.
 +
 +
As the default boot stub continues to evolve in response to demands from the Linux community this technique will allow Ultibo to continue to provide the features we want without having to make frequent changes to combat items that are beyond our control.
 +
 +
Thanks to the Circle project for introducing the custom boot stub technique to work around these changes.
  
 
=== Constants ===
 
=== Constants ===
 
----
 
----
  
−
''To be documented''
+
''None defined''
  
 
=== Type definitions ===
 
=== Type definitions ===
 
----
 
----
  
−
''To be documented''
+
''None defined''
  
 
=== Public variables ===
 
=== Public variables ===
 
----
 
----
  
−
''To be documented''
+
''None defined''
  
 
=== Function declarations ===
 
=== Function declarations ===
Line 134: Line 149:
  
 
<div class="toccolours mw-collapsible mw-collapsed" style="border: 1; font-family: arial; padding-top: 0px; padding-bottom: 15px;">
 
<div class="toccolours mw-collapsible mw-collapsed" style="border: 1; font-family: arial; padding-top: 0px; padding-bottom: 15px;">
−
<pre style="border: 0; padding-bottom:0px;">procedure Startup; assembler; nostackframe; [public, alias: '_START'];</pre>
+
<pre style="border: 0; padding-bottom:0px;">procedure Startup; assembler; nostackframe; public name '_START';</pre>
 
<div style="font-size: 14px; padding-left: 12px;">'''Description:''' Entry point of Ultibo on Raspberry Pi 3, this will be the very first byte executed and will be loaded by the GPU at address 0x00008000</div>
 
<div style="font-size: 14px; padding-left: 12px;">'''Description:''' Entry point of Ultibo on Raspberry Pi 3, this will be the very first byte executed and will be loaded by the GPU at address 0x00008000</div>
 
<div class="mw-collapsible-content" style="text-align: left; padding-left: 5px;">
 
<div class="mw-collapsible-content" style="text-align: left; padding-left: 5px;">
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
|-
 
|-
−
! '''Note'''
+
! Note
 
| None documented
 
| None documented
 
|-
 
|-
Line 151: Line 166:
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
|-
 
|-
−
! '''Note'''
+
! Note
 
| See A2.6 "Exceptions" of the ARM Architecture Reference Manual
 
| See A2.6 "Exceptions" of the ARM Architecture Reference Manual
 
|-
 
|-
Line 163: Line 178:
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
|-
 
|-
−
! '''Note'''
+
! Note
 
| See A2.6 "Exceptions" of the ARM Architecture Reference Manual
 
| See A2.6 "Exceptions" of the ARM Architecture Reference Manual
 
|-
 
|-
Line 175: Line 190:
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
|-
 
|-
−
! '''Note'''
+
! Note
 
| None documented
 
| None documented
 
|-
 
|-
Line 187: Line 202:
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
|-
 
|-
−
! '''Note'''
+
! Note
 
| None documented
 
| None documented
 
|-
 
|-
Line 199: Line 214:
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
|-
 
|-
−
! '''Note'''
+
! Note
 
| None documented
 
| None documented
 
|-
 
|-
Line 211: Line 226:
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
{| class="wikitable" style="font-size: 14px; background: white;"
 
|-
 
|-
−
! '''Note'''
+
! Note
 
| None documented
 
| None documented
 
|-
 
|-

Latest revision as of 01:17, 22 April 2022

Return to Unit Reference


Description

Ultibo Initialization code for Raspberry Pi 3 unit

Raspberry Pi 3

SoC: Broadcom BCM2837

CPU: Cortex A53 (ARMv8) (4 @ 1200MHz) Cache: L1 32KB (Per Core)/L2 512KB (Shared by all Cores)

FPU: VFPV3

GPU: Broadcom VideoCore IV (VC4)

RAM: 1GB

USB: Synopsys DesignWare Hi-Speed USB 2.0 On-The-Go Controller (DWCOTG)

LAN: SMSC LAN9514 (SMSC95XX)

SD/MMC: Arasan (BCM2710)

WiFi: Broadcom (BCM43438)

Bluetooth: Broadcom (BCM43438)

Other: GPIO/SPI/I2C/I2S/PL011 (UART)/PWM/SMI/Watchdog (PM)/Random (RNG)/Timer


Boot RPi3

The boot loader on the Raspberry Pi 3 will load this code at address 0x00008000 onwards and set the following registers before jumping to this code.

R0 - Zero

R1 - Machine Type (Raspberry Pi 3 or BCM2710 = 0x0C42)

R2 - Address of the ARM Tags structure (Normally 0x0100)


On entry to this code the processor will be in the following state:

World - Non Secure (Firmware causes a switch to Non Secure before jumping to Startup code)

Mode - Supervisor (ARM_MODE_SVC) Note: Firmware later than 2/10/2015 will boot in Hypervisor mode (ARM_MODE_HYP)

MMU - Disabled

FPU - Disabled

L1 Data Cache - Enabled (Firmware enabled prior to Non Secure switch)

L1 Instruction Cache - Enabled (Firmware enabled prior to Non Secure switch)

Branch Predication - Disabled

Unaligned Data Access - Enabled (Always enabled on ARMv8)

SMP Coherence - Enabled (Firmware enabled prior to Non Secure switch)


If the processor is in Hypervisor mode (Firmware behaviour after 2/10/2015) then Ultibo switches it to Supervisor mode during initial boot.

If the configuration option ARMSecureBoot is set to 1 then Ultibo switches the processor back to Secure world during initial boot (see note below).

Ultibo then switches the processor to System mode for all operations and remains in either the Secure or Non Secure World as per the option above.

The initialization process enables the MMU, FPU, L1 Cache and other performance optimizations.

Note that this code is currently identical to the Raspberry Pi 2 boot code but has been separated to allow for supporting 64 bit mode on the ARMv8 in future. Ultibo currently runs the RPi3 in 32 bit mode which is almost identical to ARMv7.


ARM64 support on Raspberry Pi 3

Very little is known so far about the implementation of 64 bit mode by the RPi firmware, however some details have started to appear.

There is a config.txt switch (arm_control=0x200) which causes the firmware to boot all 4 cores in 64 bit mode instead of 32 bit mode. It appears that currently the firmware does not supply any loader code when booting in 64 bit mode and the switch kernel_old=1 also needs to be used to load the kernel image at address 0x00000000. This also means that the kernel image must deal with the startup of all 4 cores and holding the secondaries until the primary has completed the system initialization. No ATAG parameters are provided when using kernel_old=1 so command line parameters would not be available.

See this post for known information: https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963

It is likely that a 64 bit loader will be included in the firmware at some point in order to support Linux boot in 64 bit mode. Without it device tree support etc would not be available and therefore make Linux use the old style device setup.


Returning to Secure World:

The Raspberry Pi 3 firmware always switches to Non Secure world in order to:

a) Reset CNTVOFF (Virtual Offset register) to zero

b) Switch to Hypervisor mode (Firmware later than 2/10/2015)


It is not normally possible (by design) to return from the Non Secure world to the Secure world except by invoking the Secure Monitor which then handles any requests in the Secure world.

However due to the fact that this switch is performed by the boot loader code (loaded between 0x00000000 and 0x000000FF by the firmware) and that the code appears to set the secure, non secure and monitor vector base addresses to 0x00000000 and because all memory is writable from non secure at boot (the MMU is disabled) then the option exists to return to the Secure world as follows:

- Install a new Secure Monitor Call vector at the appropriate point in the vector table

- Clean the data cache by Modified Virtual Address at address 0x00000000

- Perform a data synchronisation barrier

- Invalidate the entire instruction cache

- Perform a data synchronisation barrier

- Perform an instruction synchronisation barrier

- Perform a Secure Monitor Call (SMC)

- From within the newly installed Secure Monitor Call read the Secure Configuration Register (SCR) mask off the Non Secure (NS) bit, rewrite the SCR and then return to Supervisor mode.


Secondary cores also switch back to the Secure world using the same technique however they do not need to install the new SMC vector and instead use the one installed by the primary core.

Firmware changes from October 2020 disable the ability to perform the SMC instruction from non secure mode which means this technique will no longer work with the default firmware. To allow continued support for returning to the Secure world a custom ARM boot stub is now included with Ultibo that allows the SMC instruction to be performed.

This custom stub contains a marker that allows the boot process to detect it and disable the attempt to switch back to the Secure world if the default boot stub is used instead.

As the default boot stub continues to evolve in response to demands from the Linux community this technique will allow Ultibo to continue to provide the features we want without having to make frequent changes to combat items that are beyond our control.

Thanks to the Circle project for introducing the custom boot stub technique to work around these changes.

Constants

None defined

Type definitions

None defined

Public variables

None defined

Function declarations


Boot functions 

procedure Startup; assembler; nostackframe; public name '_START';
Description: Entry point of Ultibo on Raspberry Pi 3, this will be the very first byte executed and will be loaded by the GPU at address 0x00008000
Note None documented 


procedure Vectors; assembler; nostackframe;
Description: ARM exception vector table which is copied to the vector base address by the StartupHandler
Note See A2.6 "Exceptions" of the ARM Architecture Reference Manual 


procedure SecureVectors; assembler; nostackframe;
Description: ARM secure vector table which is copied to the sector vector base address by the StartupHandler
Note See A2.6 "Exceptions" of the ARM Architecture Reference Manual 


procedure SecureMonitor; assembler; nostackframe;
Description: Secure monitor mode handler to switch to secure mode
Note None documented 


procedure StartupSwitch; assembler; nostackframe;
Description: Startup handler routine to switch from hypervisor mode
Note None documented 


procedure StartupSecure; assembler; nostackframe;
Description: Startup handler routine to switch to secure mode
Note None documented 


procedure StartupHandler; assembler; nostackframe;
Description: Startup handler routine executed to start the Ultibo kernel
Note None documented


Return to Unit Reference

Retrieved from "https://ultibo.org/mediawiki_ultibo_org/index.php?title=Unit_BootRPi3&oldid=5317"