Latest commits (Ultibo core 1.3.397)

Releases, updates and announcements from the Ultibo team.
User avatar
Ultibo
Site Admin
Posts: 1549
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Latest commits (Ultibo core 1.3.397)

Postby Ultibo » Sun Jul 23, 2017 7:29 am

The latest commits for Ultibo core are now available on GitHub. Changes include POSIX Threads (pthreads) support, condition variables, new handle functions, bug fixes and more.

While this commit is the first one for a while that doesn't contain any new drivers, there is still a significant amount of new functionality as well as fixes and improvements for existing features.

List of changes:

  • Addition of POSIX Threads (pthreads) support for standard C library
  • New FileSysStartCompleted function to simplify startup detection
  • New TouchSetBuffer function to support changes in Raspberry Pi touchscreen
  • New VirtualGPIOSetBuffer function to support Raspberry Pi firmware changes
  • Add CPUGetGroup function for future cluster ID handling
  • Implementation of condition variables for cross platform compatibility
  • Fix SynchronizerWriterLockEx function not checking WriterOwner and WriterCount on zero timeout
  • Fix minor compiler warning in Winsock2 classes
  • Add new ThreadGetFlags, ThreadSetFlags, ThreadAddFlags, ThreadRemoveFlags functions
  • Define new THREAD_FLAG_* values to support POSIX Threads
  • Add usleep, sleep, nanosleep, clock_gettime, posix_memalign and more to Syscalls
  • Fix data abort flag never enabled for initial thread
  • Add new TWinsock2SocketThreads.FindByID method
  • Remove some locks around naturally atomic operations in network layer
  • Apply local lock to Name properties in Adapter, Transport and Protocol classes
  • Add informational logging to IP configuration handling to notify acquire, release, renew, rebind operations
  • Add additional information to the Thread List page in WebStatus, you can now view the full details of any thread
  • Allow viewing the details of any device from the Devices page in WebStatus (including expanded details for some device types)
  • Change DHCP and BOOTP to only enforce InitDelay once
  • Add support to DHCP and BOOTP for configuring Delay, Retries and Timeout from the command line
  • Add new standard IO redirection functions for StdOut, StdIn and StdErr
  • Fix PL110 framebuffer driver not enforcing max width and height
  • Fix FileCreate function fails if file already exists
  • Fix QuadPart of ULARGE_INTEGER to be ULONGLONG instead of LONGLONG
  • Fix GetDiskFreeSpaceEx to be QWord instead of Int64
  • Fix SetFilePointer to be LongInt instead of LongWord
  • Change FileSeek to use LongInt instead of Integer to match FPC
  • Fix FileSeek cannot seek negative offset
  • Review all use of TULargeInteger/TLargeInteger for correct usage
  • Fix potential nil pointer read/write in ReadFile and WriteFile
  • Fix FileRead and FileWrite to return -1 on error to match FPC
  • Fix ReadFile and WriteFile to return success on zero bytes
  • Fix RenameFile failure on move with same name
  • Fix AddEntry, RemoveEntry, RenameEntry etc on all file systems to eliminate potential duplicate names
  • Add new HandleCreate, HandleDestroy, HandleOpen, HandleClose and HandleEnum functions in Platform
  • Fix off by one in TSCSIStandardInquiryData structure (Caused incorrect Vendor and Product strings to be returned)
  • Add new Handles page to WebStatus
  • Rework WSAGetLastError and WSASetLastError to use a threadvar and prevent overwriting by GetLastError and SetLastError
  • Fix some of the IP Helper API functions (more to be completed)
  • Add additional information to the Network page in WebStatus (IP and MAC addresses, ARP tables etc)

POSIX Threads (pthreads) support

The Syscalls unit now includes the complete set of POSIX Threads functions in order to simplify porting more advanced C libraries to run under Ultibo.

This includes the thread, mutex, condition variable and synchronization functions as well as the separate POSIX Semaphore API. The implementation is complete and fully functional although at present some operations have only received limited testing due to their uncommon usage.

All of the POSIX thread and semaphore functionality is directly mapped onto the internal Ultibo thread and synchronization primitives, the support even allows some operations which would not normally be possibly in POSIX operating systems (such as Linux) due to design limitations.


Condition Variables

As part of adding the POSIX Threads support above it was decided to also provide a condition variables implementation that includes not only the features required for POSIX but allows supporting an internal Ultibo specific API and most of the Windows condition variable API as well.

The new functions are all contained within the Threads unit and include the following:

  • ConditionCreate - Create a new condition variable
  • ConditionDestroy - Destroy an existing condition variable
  • ConditionWaitMutex - Release a mutex and wait on a condition variable (reacquire the mutex on wakeup)
  • ConditionWaitSynchronizer - Release a synchronizer (reader, writer lock) and wait on a condition variable (reacquire the synchronizer on wakeup)
  • ConditionWaitCriticalSection - Release a critical section and wait on a condition variable (reacquire the critical section on wakeup)
  • ConditionWake - Wake one thread waiting on a condition variable
  • ConditionWakeAll - Wake all threads waiting on a condition variable
The Ultibo unit also contains Windows compatible implementations of InitializeConditionVariable, DeleteConditionVariable and WakeConditionVariable etc which internally use the Ultibo API.


DHCP and BOOTP Delay, Retries and Timeout

In response to a forum question we investigated the timing of DHCP and BOOTP configuration during startup, while we found a double delay case which was removed we also noticed that the delay, retry and timeout values used for a physical platform like Raspberry Pi are not necessarily required for a virtual platform like QEMU.

To resolve these opposing requirements we have added support for specifying the DHCP and BOOTP delay, retry and timeout values from the command line. The available parameters are as follows:

Code: Select all

DHCP_CONFIG_DELAY - Initial delay in milliseconds before attempting a DHCP discover (default is 1000)
DHCP_CONFIG_RETRIES - Number of times to retry a DHCP request before returning with a failure (default is 6)
DHCP_CONFIG_TIMEOUT - Timeout in milliseconds for each DHCP request before retrying or failing (default is 8000)

Code: Select all

BOOTP_CONFIG_DELAY - As per DHCP_CONFIG_DELAY but for BOOTP protocol
BOOTP_CONFIG_RETRIES - As per DHCP_CONFIG_RETRIES
BOOTP_CONFIG_TIMEOUT - Ad per DHCP_CONFIG_TIMEOUT

These can be passed on the command line using cmdline.txt on the Raspberry Pi or -append for QEMU.


StdOut, StdIn and StdErr redirection

In order to support a greater range of options when using standard RTL functions such as WriteLn and ReadLn as well as to add enhanced flexibility when dealing with C libraries, we have implemented a new set of redirection functions that allow sending the standard IO input and output to almost anywhere you can imagine.

Previously when the StdOut, StdIn and StdErr text files were opened they were directed to the console input and output only, the default behavior now is to direct them to a set of replaceable handlers that can point to things such as the console, a file, a serial port or even a network socket.

In addition to the changes in the Platform unit, we have provided a number of simple functions to allow redirecting standard IO to the location of your choice. Some of the available functions are:

  • ConsoleWindowRedirectOutput - Send both standard output and standard error to any console window
  • FileSysRedirectInput - Use any previously opened file as the standard input
  • FileSysRedirectOutput - Send standard output and error to any open file
  • SerialDeviceRedirectInput - Read standard input from a serial device
  • SerialDeviceRedirectOutput - Send standard output and error to a serial device
  • Winsock2RedirectInput - Use a Winsock socket (TCP or UDP) as the standard input
  • Winsock2RedirectOutput - Redirect standard output and error to any Winsock socket (TCP or UDP)
  • LoggingDeviceRedirectOutput - Send the standard output and error to the current logging device
You can also implement your own function that redirects either standard input or output to anywhere you choose, the functions above provide a very simple template of how to create your own.


FileRead and FileWrite now return -1 on error

In order to obtain compatibility with the documented behavior of the FileRead and FileWrite functions as shown in the official FPC documentation, we have changed the return value to be -1 on error instead of zero. This means that a read or write of zero bytes is now considered successful even though no data will be read or written.

This change of return value is regarded as a breaking change because anyone with code that interprets a return of zero as a failure will now need to update their code to test for -1 instead. We apologize for any inconvenience this may cause but it was considered important that the functionality be as expected from the FPC documentation.


For details of how to apply the latest source to your Ultibo core installation and rebuild your run time library see the wiki page Building from Source or watch the Building the RTL video on YouTube.
Ultibo.org | Make something amazing
https://ultibo.org
Gavinmc42
Posts: 1053
Joined: Sun Jun 05, 2016 12:38 pm
Location: Brisbane, Australia

Re: Latest commits (Ultibo core 1.3.397)

Postby Gavinmc42 » Sun Jul 23, 2017 8:29 am

POSIX and redirection of STDxx, that's a pretty big step forwards.
You make it sound so easy. Well done, stepping up to the plate in the pro league now.

https://en.wikipedia.org/wiki/POSIX
Yep, long time ago but I can remember back then it was a really big deal.
Do they still have PC based cash registers, 1990's they had to be POSIX?
Painful memories of that transition ;)

Very big implications for library calls.
mark
Posts: 813
Joined: Mon Oct 03, 2016 2:12 am
Location: Indianapolis, US

Re: Latest commits (Ultibo core 1.3.397)

Postby mark » Mon Jul 24, 2017 2:14 pm

Code: Select all

WEBSTATUS_FONT_NAME:String = 'Arial';
The WebStatus unit has a font setting, so for instance you can change it to monospace. Cheers, Mark.
mark
Posts: 813
Joined: Mon Oct 03, 2016 2:12 am
Location: Indianapolis, US

Re: Latest commits (Ultibo core 1.3.397)

Postby mark » Thu Jul 27, 2017 7:20 pm

Ultibo wrote:New VirtualGPIOSetBuffer function to support Raspberry Pi firmware changes
Do these changes lead to any inroads into the vc4? I noticed a lot of mailbox references. Mark.
User avatar
Ultibo
Site Admin
Posts: 1549
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Latest commits (Ultibo core 1.3.397)

Postby Ultibo » Fri Jul 28, 2017 12:04 am

mark wrote:Do these changes lead to any inroads into the vc4? I noticed a lot of mailbox references

The TouchSetBuffer and VirtualGPIOSetBuffer are in response to changes made to the Raspberry Pi firmware, as best as we can determine using a block of memory allocated by the GPU is sometimes problematic for Linux so they have changed to allowing Linux (or whoever is running on the ARM) to specify a suitable block of memory to use.

In making those changes to the firmware the previous model of using VirtualGPIOGetBuffer to allocate memory from the GPU seems to have become slightly unpredictable so we have switched to the new API as well.

Of course you will also find a number of new mailbox constants and type definitions in this commit, those are part of a continuous effort to add support for everything that is available via the mailbox interface. The POSIX threads support is intended to allow larger and more complex C libraries to run on top of Ultibo, one that definitely will require that is the Userland library used to access the VC4 services.
Ultibo.org | Make something amazing
https://ultibo.org
mark
Posts: 813
Joined: Mon Oct 03, 2016 2:12 am
Location: Indianapolis, US

Re: Latest commits (Ultibo core 1.3.397)

Postby mark » Mon Sep 18, 2017 11:07 pm

Ultibo wrote:Implementation of condition variables for cross platform compatibility
Where are these? Thanks, Mark.
User avatar
Ultibo
Site Admin
Posts: 1549
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Latest commits (Ultibo core 1.3.397)

Postby Ultibo » Mon Sep 18, 2017 11:24 pm

mark wrote:Where are these?

As per the announcement, these are in the Threads unit.

Ultibo wrote:Condition Variables

As part of adding the POSIX Threads support above it was decided to also provide a condition variables implementation that includes not only the features required for POSIX but allows supporting an internal Ultibo specific API and most of the Windows condition variable API as well.

The new functions are all contained within the Threads unit and include the following:

  • ConditionCreate - Create a new condition variable
  • ConditionDestroy - Destroy an existing condition variable
    ...

They can also be accessed from the Syscalls unit using the pthread_cond_* functions (although we don't recommend using those for Pascal code) and from the Ultibo unit using the *ConditionVariable functions, all of which tie back to the actual implementation in Threads.
Ultibo.org | Make something amazing
https://ultibo.org
mark
Posts: 813
Joined: Mon Oct 03, 2016 2:12 am
Location: Indianapolis, US

Re: Latest commits (Ultibo core 1.3.397)

Postby mark » Mon Sep 18, 2017 11:38 pm

Ultibo wrote:
mark wrote:Where are these?

As per the announcement, these are in the Threads unit.
Right. I was confused with something else. Thanks.

Return to “Ultibo”

Who is online

Users browsing this forum: No registered users and 1 guest