Latest commits (Ultibo core 1.3.077)

Releases, updates and announcements from the Ultibo team.
User avatar
Ultibo
Site Admin
Posts: 1549
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Latest commits (Ultibo core 1.3.077)

Postby Ultibo » Wed Dec 28, 2016 5:51 am

The latest commits for Ultibo core are now available on GitHub. Changes include C library support, SQLite and FreeType2 support, enhancements, bug fixes and more.

This commit is accompanied by a new release of the Ultibo core installer, some changes required to support C libraries are not usable without the installer update so if you are wanting to take advantage of them please ensure you also download and install the latest Ultibo core installer.

List of changes:

  • Add Syscalls unit to support inclusion of standard C library
  • Add Ultibo specific versions of libc.a, libm.a and libgcc.a for ARMv6, ARMv7 and ARMv8
  • Enable the SQLite package and add platform support routines for Ultibo core
  • Enable SQLite in the fcl-db package
  • Enable FreeType2 in the fcl-image package
  • Add new ClockDeviceSetRate, ClockDeviceProperties and ClockDeviceWrite64 functions for Clock devices
  • Add Clock device driver for ARM Timer on BCM2835, BCM2836 and BCM2837
  • Fix FramebufferDeviceCopyRect incorrect stride value in overlapped (Left to Right) copy
  • Fix BCM2708/9/10SPI0WriteRead to prevent cleaning or invalidation if DMA source or destination buffers are nil
  • Fix for DMARequestComplete to prevent referencing a request buffer after it has been released
  • Fixes for DMARequestSubmit and DMARequestCancel to match DMARequestComplete changes
  • Fix missing MEMORY_PAGE_SIZE in PlatformQEMUVPB
  • Fix FileSysLoggingStart to not fail if target is empty
  • Adjust KeyboardGet to be consistent with KeyboardPeek
  • Fix decoding of ShareMode parameter in FSCreateFile and TFileSysDriver.CreateFile
  • Change ARMv7 page table attributes to remove use of Write-Through caching
  • Move GENERIC_* file access constants to GlobalConst
  • Move FILE_SHARE_* file sharing constants to GlobalConst
  • Add FSFileGetAttrEx and TFileSysDriver.FileGetAttrEx function to return attributes from a handle
  • Change to using EABIHF application binary interface (ABI) in all cases for compatibility with GCC
  • Fix Read and ReadLn functions not returning on Enter key
  • Change LineEnding constant in System unit to be CRLF instead of LF
  • Fix USBRequestComplete to prevent referencing a request buffer after it has been released
  • Fixes for USBRequestSubmit and USBRequestCancel to match USBRequestComplete changes
  • Add __buildrtl.bat file to Core to allow building with automated tools
  • Fix GetDiskFreeSpaceEx to not fail if lpTotalNumberOfFreeBytes is nil
  • Change GetNumaHighestNodeNumber (Dummy) to comply with MSDN example

C library support

Ultibo core now supports including static libraries compiled with the GCC compiler in your project in order to use their functionality. As part of this support we now provide a specific build of the standard C libraries libc.a and libm.a which contain much of the support code needed by libraries compiled in C.

This release also includes a new Syscalls unit which provides the interface between the standard C library and Ultibo core and handles operations like memory, console i/o, date and time and file handling.

It is important to note that you cannot simply take libraries from another platform such as Linux and include them in your Ultibo project, at the very least they need to be recompiled with the appropriate parameters, depending on the requirements of the library some may also need you to supply support functions that interface the library to Ultibo core. We have provided two libraries with the release to show how the process works, the FreeType2 library requires no additional support code and uses only standard C library functions to manage memory, files and other operations, the SQLite3 library on the other hand needs a complete platform support layer in order to implement the required functions for memory, locking and file access.

The standard C library implementation that is included is based on Newlib which is an open source implementation of the C library supplied by RedHat. The Syscalls unit provides the functions needed to support the standard C library, one of the most critical of these is memory allocation. Most C code uses malloc() or similar functions to allocate memory and the Newlib library provides optimized versions of these functions which are compatible with a wide range of code, these functions are ultimately connected to the Ultibo core GetMem/FreeMem functions via the sbrk() call which requests memory in the form of a linear heap starting at a given point and incrementing until the end of available memory. In order to support this model the Syscalls unit maps all memory allocated to the C library into a virtual memory space which is expanded on demand in fixed size blocks, the behavior, mapping and allocation sizes of this memory management can be adjusted by a number of parameters in the GlobalConfig unit, see the SYSCALLS_HEAP_* variables for more information.

Additional information will be added to the forum and the wiki over time to explain in more detail how to compile a C library for use in your projects, including the parameters needed for correct interaction between the library and Ultibo core.


Change to EABIHF application binary interface (ABI)

In order to obtain compatibility between code compiled with FreePascal and code compiled with GCC it has been necessary to change the application binary interface (ABI) used by Ultibo core to be the ARM Embedded ABI (EABI) standard with hardware floating point support (HF). It might seem obvious that code running on an ARM processor should use the ABI defined by ARM but unless told otherwise FPC uses what it terms the "Default" ABI which while very similar to EABI has subtle differences that make it incompatible for certain data types and function parameters.

The change to EABIHF has been enabled in the FPC compiler as well as all other elements of Ultibo core so you should not need to take any action unless you build your projects or the RTL from the command line, the wiki will be updated shortly to reflect the changes needed when compiling FPC or the RTL and you should update your process to take account of these changes.

This change will not have any impact on your code unless you interface with assembler functions in which case you should review the parameters passed to and from those functions in order to ensure they are compliant with the ARM EABI.


Changes to DMARequestComplete and USBRequestComplete functions

For some months the wiki Current Status page has contained an issue for Very occasional deadlocks during console scroll which we have finally managed to track down. While the description of the bug doesn't make it appear serious the actual cause turns out to be very important because it would affect more and more operations over time in ways that would be difficult to diagnose. The issue turns out to be a situation where the DMARequestComplete() function could reference a DMA request after it had already been released which would result in an exception or a crash, once found it was evident that the USBRequestComplete() function also had the same issue and although not seen in any current issues the problem could occur at any time.


Removal of Write-Through caching in page table configuration

This release contains a small change to the page table configuration for ARMv7 processors to remove the use of Write-Through caching in all circumstances. In response to a forum post and following extensive investigation it was found that the ARMv7 and later processors, including the Cortex A7 in the Raspberry Pi 2B and the Cortex A53 in the Raspberry Pi 3B no longer implement Write-Through cache in order to reduce the complexity of the chip design. Unfortunately instead of removing the option and making it mandatory to use a different cache mode the designers decided to take the very unusual step of making all memory pages marked as Write-Through actually act as uncached instead which can significantly change the performance of certain code sequences.

In order to address this change in the behavior of the chip Ultibo core now uses Write-Back caching in place of Write-Through which removes the performance penalty caused by the design change above without impacting any other aspect of the design and behavior of Ultibo itself.

We will write a brief outline with more details of this issue and the way it impacts performance shortly however you should not assume that this change provides any sort of linear performance gain, as noted above the change has a significant affect on certain code and yet has little or no impact on other code.


For details of how to apply the latest source to your Ultibo core installation and rebuild your run time library see the wiki page Building from Source or watch the Building the RTL video on YouTube.
Ultibo.org | Make something amazing
https://ultibo.org
pik33
Posts: 608
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Latest commits (Ultibo core 1.3.077)

Postby pik33 » Wed Dec 28, 2016 7:06 am

So, as I understand, I have to... wait some time until the new build instruction will appear in wiki.
User avatar
Ultibo
Site Admin
Posts: 1549
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Latest commits (Ultibo core 1.3.077)

Postby Ultibo » Wed Dec 28, 2016 9:18 am

pik33 wrote:So, as I understand, I have to... wait some time until the new build instruction will appear in wiki.

We didn't plan on doing an update for the installer just yet, the changes for the C compatibility made it necessary.

We'll get the wiki documentation for building FPC and the RTL updated as soon as we can.
Ultibo.org | Make something amazing
https://ultibo.org
pik33
Posts: 608
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Latest commits (Ultibo core 1.3.077)

Postby pik33 » Wed Dec 28, 2016 10:53 am

Ultibo wrote:
We'll get the wiki documentation for building FPC and the RTL updated as soon as we can.


That's what I need - as I am working with ultibo under Raspbian, so I have now to rebuild fpc and rtl as I did several times whenever a new version was available, but now, as I can understand, rebuilding process needs some new parameters, so these instructions will not work anymore.
Jyv
Posts: 132
Joined: Mon Feb 08, 2016 1:30 pm

Re: Latest commits (Ultibo core 1.3.077)

Postby Jyv » Wed Dec 28, 2016 1:19 pm

concerning: Fix decoding of ShareMode parameter in FSCreateFile and TFileSysDriver.CreateFile
it looks like the case CREATE_NEW calls directly FileCreate without taking into account ShareMode
FileHandle:=FDriver.OpenFileHandle(FVolume,FDrive,Parent,Current,fmOpenReadWrite or fmShareExclusive,True,FILESYS_LOCK_WRITE);
and so ending with a file handle being ShareExclusive
I am not sure I am using this case, but since reviewing the updates I prefer to write down a little note about it,
or it may conform by design to match some obscure MSDN documentation :-)
User avatar
Ultibo
Site Admin
Posts: 1549
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Latest commits (Ultibo core 1.3.077)

Postby Ultibo » Wed Dec 28, 2016 10:56 pm

Jyv wrote:it looks like the case CREATE_NEW calls directly FileCreate without taking into account ShareMode

Correct, in fact any case that results in a call to FileCreate ends up creating a file handle with fmShareExclusive set.

CreateFile was originally added to implement something similar to the Windows API functionality using the existing functions, what really needs to happen in future is to expand CreateFile down to the TFileSystem class and implement it directly without calling FileOpen or FileCreate, I just checked my list and I already have this down as an item to revisit along with a number of other Win32 compatible API functions.
Ultibo.org | Make something amazing
https://ultibo.org
pik33
Posts: 608
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Latest commits (Ultibo core 1.3.077)

Postby pik33 » Fri Dec 30, 2016 6:59 am

As there is still no new instruction: what have I change in EABI settings in these command line parameters to recompile this new version in Raspbian?
User avatar
Ultibo
Site Admin
Posts: 1549
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Latest commits (Ultibo core 1.3.077)

Postby Ultibo » Sat Dec 31, 2016 3:07 am

The wiki instructions for building Ultibo FPC on Windows as well as Debian and Raspbian Linux have now been updated and tested to include the changes for EABIHF in 1.3.077

Please be sure to check carefully when following the instructions as there have been a number of changes for each platform.
Ultibo.org | Make something amazing
https://ultibo.org
pik33
Posts: 608
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Latest commits (Ultibo core 1.3.077)

Postby pik33 » Sun Jan 01, 2017 1:23 pm

Mission completed. New fpc/rtl installed on RPi.

Ugly hack

Code: Select all

  Entry.Flags:=$3b2; 


removed and the player still works:)

Some remarks:

- RPI3.CFG :)
- now I know in what way .ppu files were found in fpc/bin. The Lazarus needs system.ppu and classes.ppu to be there or it complains at start. Last time I installed the fpc I simply copied all /rtl to /fpc/bin to make the Lazarus shut up. Now I copied ony the files it complained, one by one, until it works and then only these two files was required by it.
- this time I had to unzip the downloaded rtl and fpc with the PC: the RPi cannot open these ZIP files with GUI tool; the Midnight Commander opened them, but then started to unzip VERY slow even on attached HD, so this was not SD problem. There were no such problems when unzipping previous versions - they had to change something in their zipping program at Github? The PC has no problem with unzipping these folders, fast.
User avatar
Ultibo
Site Admin
Posts: 1549
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Latest commits (Ultibo core 1.3.077)

Postby Ultibo » Sun Jan 01, 2017 11:09 pm

pik33 wrote:The Lazarus needs system.ppu and classes.ppu to be there or it complains at start.

On Raspbian these units would normally be found in $HOME/ultibo/core/fpc/lib/fpc/3.1.1/units/arm-linux/rtl so we'll have to work out why Lazarus doesn't look there when we do the instructions for building Lazarus.
Ultibo.org | Make something amazing
https://ultibo.org

Return to “Ultibo”

Who is online

Users browsing this forum: No registered users and 1 guest