Latest commits (Ultibo core 1.3.327)

Releases, updates and announcements from the Ultibo team.
User avatar
Ultibo
Site Admin
Posts: 1549
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Latest commits (Ultibo core 1.3.327)

Postby Ultibo » Sat Jun 03, 2017 6:22 am

The latest commits for Ultibo core are now available on GitHub. Changes include SD card drivers for QEMU, TCP enhancements, improved USB handling, bug fixes and more.

List of changes:

  • Add new PL18X SD card driver for QEMU platform
  • Add new GPIO driver for Raspberry Pi GPIO Expander
  • Add new clock driver for 24MHz clock on QEMU platform
  • Change Winsock/Winsock2 select functions to perform event based wait if only one socket is passed
  • Rework USB split transaction scheduling for Full/Low Speed devices to resolve the Keyboard and Mouse on the same hub issue
  • Implement new USBControlTransfer, USBBulkTransfer and USBInterruptTransfer functions to provide simplified synchronous transfers
  • Implement new GraphicsWindowGetProperties function to match console window version
  • Change ClockGetCount (and GetTickCount) for QEMU to rollover every 71 minutes as per Raspberry Pi instead of every 178 seconds
  • Fix BCM2835_PERIPHERALS_SIZE (and BCM2836, BCM2837) declaration which was off by one
  • Ensure all devices have a DeviceDescription property which reflects their function and origin
  • Change DeviceProperties function in some APIs to be DeviceGetProperties for consistency (DeviceProperties remains available as an inline alias)
  • Add new SerialDeviceFlush function to the Serial device API
  • Fix handling of Window border width for Text and Graphics Console windows
  • Rework handling of clock interrupt and clock seconds to eliminate clock drift (especially on QEMU)
  • Fix THTTPClient reuse for repeated requests
  • Fix TLS index flags not being set correctly on newly created threads
  • Change CriticalSection and Synchronizer lock and unlock to allow opportunistic shortcut when safe
  • Fix deadlock during startup caused by synchronous device notifications
  • Implement SystemRestart function for QEMU platform (Note that SystemShutdown is not available)
  • Fix MailboxPropertyCallEx function failing incorrectly on some newer firmware functions
  • Fix potential deadlock in ThreadHalt and ThreadTerminate functions
  • Add new SetLocalTime function to the SysUtils unit
  • Resolve a deadlock with MountImage when CreateImage is called before filesystem startup has completed
  • Implement new USBDeviceGetHub and USBDeviceGetPort functions
  • Implement new USBHubPortDisable, USBHubPortPowerOn and USBHubPortPowerOff functions
  • Complete the implementation of all USB string descriptor functions (all USB devices now report a Manufacturer, Product and Serial number if available)
  • Fix missing Manufacturer, Product and SerialNumber strings in TDiskDevice class
  • Fix QEMU mouse coordinates upside down

Changes to USB split transaction scheduling for Full/Low Speed devices

While many of you may not know what a USB split transaction is or why you would need one you will probably be aware of the existing bug in Ultibo core which causes a failure if you plug more than one low speed USB device into the same hub.

After carefully studying the behavior of the DWCOTG USB host driver during these failures it was determined that the problem was related to the timing of the two parts of the transaction required to communicate with a low or full speed device connected to a high speed hub. This communication is called a split transaction and consists of a start split followed by a complete split operation with a strict timing relationship between the two parts.

The transaction scheduling in Ultibo has now been redesigned for all split transactions (both Full and Low Speed devices) in order to eliminate the failures, in testing we have not observed any further failures using multiple different combinations of 2 or more low speed and full speed USB devices connected either to the internal hub in the Raspberry Pi or to external USB 2.0 and 3.0 hubs. In addition the changes also include a number of fixes that resolve unusual behavior for certain devices and also improve overall keyboard and mouse response in some cases due to less errors occurring which necessitate transaction retries.

Since this issue has been outstanding for some time now and a number of people have asked about it, we would appreciate if anyone who has previously experienced failures when using both a keyboard and mouse at the same time retest with this version to confirm that the problem is resolved.


New QEMU SD card driver

This release includes a new driver for the PL181 MMCI (SD card) device in the QEMU Versatile PB emulation, this driver completes the basic set and when combined with the network driver provided in the last commit allows many of the features of Ultibo core to be used directly under QEMU for the purpose of developing and testing your application.

The QEMU Versatile PB emulation includes two SD card devices and the new driver will automatically support both of them, in order to access files you will need to create a disk image and configure QEMU to attach it as an SD card.

Creating an image

The version of QEMU included with Ultibo core contains a utility for creating and modifying disk image files, to create a simple 100MB raw image file called "MyDisk.img" in the C:\Projects\MyDisk folder do the following from a command prompt:

Code: Select all

cd C:\Ultibo\Core\qemu
qemu-img.exe create -f raw C:\Projects\MyDisk\MyDisk.img 100M

Don't forget that qemu-img.exe will create the file for you but the directory must already exist.

Attaching the image to QEMU

Once you have a disk image you can attach it to QEMU for use with your Ultibo application by adding the correct parameters to the QEMULauncher.ini file in C:\Ultibo\Core\tools. To attach the disk image file we just created add the following to QEMULauncher.ini (if you don't already have a QEMULauncher.ini file simply create a new text document and rename it to QEMULauncher.ini).

Code: Select all

[QEMULauncher]
ExtraParams=-drive file=C:\Projects\MyDisk\MyDisk.img,if=sd,format=raw

If you are already using the network settings from the last commit you can combine them with the disk information like this:

Code: Select all

[QEMULauncher]
ExtraParams=-net user -net nic -drive file=C:\Projects\MyDisk\MyDisk.img,if=sd,format=raw

To attach a second disk named "MySecondDisk.img" add the following extra parameters:

Code: Select all

[QEMULauncher]
ExtraParams=-net user -net nic -drive file=C:\Projects\MyDisk\MyDisk.img,if=sd,format=raw -drive file=C:\Projects\MyDisk\MySecondDisk.img,if=sd,format=raw

Partitioning and formatting

The disk image files created above by qemu-img.exe will be completely blank and contain no files or other information. In order to use them with your Ultibo application they need to be partitioned and formatted which can easily be done using Ultibo core itself.

To partition and format a new disk create an empty QEMU Versatile PB Application in Lazarus and add the ConsoleShell and ShellFilesystem units to the uses clause.

Code: Select all

uses
  QEMUVersatilePB,
  GlobalConfig,
  GlobalConst,
  GlobalTypes,
  Platform,
  Threads,
  SysUtils,
  Classes,
  Ultibo,
  ConsoleShell,
  ShellFilesystem
  { Add additional units here };

Build the new project and then run it by selecting Tools, Run in QEMU.. from the menu. If that was successful you should see a console screen with the shell window on the right hand side prompting with type HELP for a list of available commands.

Type DISK and press enter to check that the SD card devices are visible, if you type DISK SHOW \harddisk0 you should see the details of your new disk image.

To partition and format the new disk with a single FAT32 partition simply type the following in the console shell window:

Code: Select all

PARTITION ADD \harddisk0 /TYPE=FAT32 /MAX /ACTIVE
VOLUME FORMAT \volume1 /TYPE=FAT32

If everything was successful you should be left at the C:\ prompt with a newly formatted blank drive.

Using an existing SD card image

QEMU supports attaching a number of different disk image formats, you can even use a standard SD card image like those supplied by many Raspberry Pi projects or created using tools like Win32 Disk Imager.

For more information about creating and manipulating disk images with QEMU please see the documentation.


New GPIO driver for Raspberry Pi GPIO Expander

With the introduction of the Raspberry Pi 3 a change was made to the design that has frustrated many wanting to experiment with their devices at the lowest level, that change was to relocate certain GPIO functions from the inbuilt GPIO device to a GPIO expander that is attached to the GPU and is not accessible to the ARM CPU.

One of the symptoms of the change is that the simple act of blinking the activity LED on the Raspberry Pi 3 now requires a large amount of extra code in order to communicate with the GPU firmware, controlling the power LED also became completely impossible to achieve.

So it was with great interest that we noticed the content of a recent commit to the Raspbian Linux repository on GitHub that implemented a new GPIO expander driver using mailbox requests to the firmware, because Ultibo core already had support for controlling the activity LED we wondered if it would finally be possible to gain access to the other functions of the expander and after some experiments it turns out you can now have almost full control of these previously inaccessible GPIO pins.

The new unit is called RPiGPIOExpander and simply including it in your application will auto register the new GPIO expander device during startup. The GPIO expander includes 8 pins which can be accessed either via the normal GPIO API functions or by using the VirtualGPIO functions (VirtualGPIOInputGet, VirtualGPIOOutputSet etc).

From testing the following pin assignments are known:

GPIO_PIN_2 = Activity LED
GPIO_PIN_4 = HDMI Detect (Input / Active Low)
GPIO_PIN_7 = Power LED (Defaults to Input / Active Low, change to Output to control)

The unit automatically hooks into the Platform functions for PowerLED and ActivityLED during initialization so simply calling PowerLEDEnable followed by PowerLEDOn/PowerLEDOff will give control of the red LED on the board.

In order to use the standard GPIO functions you must pass the correct device which can be found by calling GPIODeviceFindByDescription with the description "Raspberry Pi Firmware GPIO Expander".

Please note that to use the new GPIO expander you must also use a very recent version of the Raspberry Pi firmware, from about March 2017 or later. Simply download the current firmware from GitHub and extract the bootcode.bin, start.elf and fixup.dat files to your SD card.


For details of how to apply the latest source to your Ultibo core installation and rebuild your run time library see the wiki page Building from Source or watch the Building the RTL video on YouTube.
Ultibo.org | Make something amazing
https://ultibo.org
pik33
Posts: 608
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Latest commits (Ultibo core 1.3.327)

Postby pik33 » Sat Jun 03, 2017 7:53 am

Rework USB split transaction scheduling for Full/Low Speed devices to resolve the Keyboard and Mouse on the same hub issue


:) :) :) :) :) :) :)

This will be tested today - and in more different hardware, in Monday. Now I have to recompile RTL

Edit: in my home environment, the keyboard/mouse now works OK.
develone
Posts: 200
Joined: Wed Dec 28, 2016 7:40 pm
Location: El Paso Tx USA

Re: Latest commits (Ultibo core 1.3.327)

Postby develone » Sat Jun 03, 2017 4:47 pm

Hello All
Updated my verison and built & tested the Ultibo FPC and RaspBian FPC.

see
https://github.com/develone/jpeg-2000-test/blob/master/bare-metal/build_ultibo_RPi/ultibo_update_060317.txt


I tested with build my openjp application.
Ultibo Core (Release: Cucumber Version: 1.3.327 Date 3 June 2017)


Is a Raspbian installer any closer?

My scripts appear to have worked okay but no Ultibo Lazarus.
jpeg-2000-test/bare-metal/build_ultibo_RPi/build_fpc_lazarus.sh
jpeg-2000-test/bare-metal/build_ultibo_RPi/build_ultibo_fpc.sh
jpeg-2000-test/bare-metal/build_ultibo_RPi/build_ultibo_rtl_armv6.sh
jpeg-2000-test/bare-metal/build_ultibo_RPi/build_ultibo_rtl_armv7.sh
cp jpeg-2000-test/bare-metal/build_ultibo_RPi/rpi*.cfg ultibo/core/fpc/bin


Let me know if I can provide any additional information.

Cheers,
develone
Posts: 200
Joined: Wed Dec 28, 2016 7:40 pm
Location: El Paso Tx USA

Re: Latest commits (Ultibo core 1.3.327)

Postby develone » Sat Jun 03, 2017 8:40 pm

mark
Posts: 813
Joined: Mon Oct 03, 2016 2:12 am
Location: Indianapolis, US

Re: Latest commits (Ultibo core 1.3.327)

Postby mark » Sun Jun 04, 2017 1:19 am

FYI I am getting

SD: Unknown CMD23
SD: Unknown CMD23

printed by qemu/Debian/x64 at the start of the minimal suggested program (ConsoleShell and ShellFileSystem.)

I'm working on getting more info. Mark.
Gavinmc42
Posts: 1053
Joined: Sun Jun 05, 2016 12:38 pm
Location: Brisbane, Australia

Re: Latest commits (Ultibo core 1.3.327)

Postby Gavinmc42 » Sun Jun 04, 2017 7:32 am

Time to start messing about with the USB stuff, serious bugs fixed now.
USBhub stuff will be interesting, I might get the Cluster Hat working in Ultibo before I get it working in my version of Linux.

Time to go back to QEMU coding now that SDcard is working :P

GPIO Expander used for anything else? WiFi or SW reg chip?
Pins 2/4/7 are know, wonder what the others do?
Here be dragons for sure.

Simply download the current firmware from GitHub and extract the bootcode.bin, start.elf and fixup.dat files to your SD card.

I have been getting a Zero to work with USB booting just the bootload.bin, start.elf, kernel.bin.
it left me to wonder what is fixup.dat used for?

Only needed if you use more than 16MB for GPU?
http://elinux.org/RPi_Software
So for headless stuff no fixup.dat?
With no VC4 stuff except framebuffer/mailbox working yet do we need more than 16MB for VC4?

Hmm, does VC4 do multiple framebuffer swapping?
pik33
Posts: 608
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Latest commits (Ultibo core 1.3.327)

Postby pik33 » Sun Jun 04, 2017 12:25 pm

Hmm, does VC4 do multiple framebuffer swapping?


Yes, it does. You declare a framebuffer with more lines than you display, then you tell GPU from what line it have to display the picture. In my programs, I declared 1920x2400 framebuffer while the physical size is 1920x1200, so in the reality I have 2 1920x1200 framebuffers and I can switch between them.
Gavinmc42
Posts: 1053
Joined: Sun Jun 05, 2016 12:38 pm
Location: Brisbane, Australia

Re: Latest commits (Ultibo core 1.3.327)

Postby Gavinmc42 » Mon Jun 05, 2017 12:58 am

That would tend to indicate we could have a virtual screen bigger than the actual screen.
We could have a 4K, 5K, 10K screen and use the console screen as a windowed view to a larger screen.
Side scrolling games, one super wide framebuffer?

This would allow Desktop window swapping, now that is very cool.
pik33
Posts: 608
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Latest commits (Ultibo core 1.3.327)

Postby pik33 » Mon Jun 05, 2017 4:22 am

We can have a virtual framebufer with the size you want, if the retromachime is too complex, you have dancing boxes Ultibo example how to do this, but then, until we have GPU accelerated OpenGL there is no need to use GPU RAM for side scrolling or window managing. 2-3 screens for swapping are enough, one is displayed while another is prepared. The rest can be better done in normal RAM. The Mario Bros world or River Raid river are too big to make them all at once: they have to be generaed in real time.
Gavinmc42
Posts: 1053
Joined: Sun Jun 05, 2016 12:38 pm
Location: Brisbane, Australia

Re: Latest commits (Ultibo core 1.3.327)

Postby Gavinmc42 » Mon Jun 05, 2017 4:56 am

The Mario Bros world or River Raid river are too big to make them all at once: they have to be generated in real time.

Side scroller landscapes could be procedurally generated, did this on a Microbee decades ago, back when fractals were all the rage.
Now I should be able to do this with 24bit colour plus Alpha.
With multiple backgrounds scrolling at different speeds the 3D effects kick in.

Now that C drive is working on QEMU I can start game coding, er once I figure out how C drive on QEMU works ;)

Return to “Ultibo”

Who is online

Users browsing this forum: No registered users and 1 guest