Confused about options for graphics

Anything and everything about programming graphics with Ultibo
msx80
Posts: 2
Joined: Thu Aug 02, 2018 8:38 am

Confused about options for graphics

Postby msx80 » Thu Aug 02, 2018 8:56 am

Hi there :) I'd like to tinker a little with RPI and Ultibo, i have some experience with baremetal programming with circle.

Now i'm a little confused about what options i have available to do some graphics. If i understand correctly, there's the framebuffer option which doesn't use any hardware acceleration (and you basically just write to a pointer to memory mapped video), and the "videocore" options that uses the hardware accelerator. Is this correct?
If i want to use the videocore, do i have to talk with it with OpenglES or are there other interfaces ?
Do both options support 8 bit indexed mode?

I'm mostly interested in simple 2d graphics, like sprites, backgrounds, tiles and the like.

I've also read about DispmanX, what is it and how does it fit in all the above?

Thanks :)
User avatar
Ultibo
Site Admin
Posts: 1932
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Confused about options for graphics

Postby Ultibo » Thu Aug 02, 2018 11:34 am

msx80 wrote:Hi there :) I'd like to tinker a little with RPI and Ultibo, i have some experience with baremetal programming with circle.

Hi, welcome to Ultibo.

msx80 wrote:Now i'm a little confused about what options i have available to do some graphics. If i understand correctly, there's the framebuffer option which doesn't use any hardware acceleration (and you basically just write to a pointer to memory mapped video), and the "videocore" options that uses the hardware accelerator. Is this correct?

If i want to use the videocore, do i have to talk with it with OpenglES or are there other interfaces ?

Those are the two options available although there are a number of sub options within each one as well.

When using the framebuffer you can use the built in console and graphics console functionality if you just want to draw simple shapes, text etc. You can also choose to work directly with the framebuffer memory either using an existing library or by creating your own functions for total control of everything.

The hardware acceleration using the Videocore supports several standards based protocols, the main one is OpenGL ES (for 2D and 3D rendering) but there is also OpenVG (for 2D) as well as OpenMAX and the Broadcom specific MMAL for video and multimedia.

Again with the hardware accelerated modes you have the option to work directly with the interfaces documented by Kronos (or elsewhere) or use a library to handle some of the details for you. We have a fully working port available of the Asphyre/PXL graphics library which supports OpenGL ES and provides a huge collection of functionality out of the box.

msx80 wrote:Do both options support 8 bit indexed mode?

The framebuffer option supports 8, 16. 24 and 32 bit modes and allows you to set your own palette in 8 bit mode. See the Bouncing Boxes example for how to use 8 bit indexed mode with the framebuffer.

As for OpenGL ES I'm not actually sure if it does or doesn't support 8 bit, you might need to do some research. If the standard supports it then so should Ultibo as long as the Pi hardware does.

msx80 wrote:I'm mostly interested in simple 2d graphics, like sprites, backgrounds, tiles and the like.

I've also read about DispmanX, what is it and how does it fit in all the above?

DispmanX is sometimes described as a window manager but that probably isn't a great description because it doesn't work well when trying to use it for that purpose. More likely it should be described as a compositor that takes multiple layers and composites them into the final display output.

You can take advantage of what DispmanX does without needing to know a lot about it, each of the display options on the Pi uses one or more DispmanX layers. So the framebuffer is in one layer, OpenGL ES in another, OpenVG in a different layer again and so on for OpenMAX etc.

This means multiple things can be overlaid to form the final output, for example a 2D graphics display with a video (or camera feed) playing in a small area or even a HD video with text overlay etc.

Have a look at the Forum Quick Reference for a few more pointers to graphics information, you will also find others if you browse some of the earlier posts on the forum.

Hope that helps, feel free to ask if you need to know more.


EDIT: Forgot to mention that Andrew Duncan has a repo on Github with some DispmanX examples that show simple animation etc, you might find it interesting.
Ultibo.org | Make something amazing
https://ultibo.org
pik33
Posts: 710
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Confused about options for graphics

Postby pik33 » Thu Aug 02, 2018 11:45 am

You have a lot of options. What I have used was asm+non accelerated framebuffer

The framebuffer supports 8-bit modes, but in my project (https://github.com/pik33/ultibo_retro_gui) I decided to use 32-bit fb with the procedure which refreshes the screen every vblank translating my own 8-bit framebuffer to 24-bit color space. I have also software generated sprites (8 sprites, 32x32 pixels, up to 8x zoom). This was done having a display list("copper list") and raster interrupts simulation in mind, impossible when using native 8-bit fb on RPi where you cannot change the palette in the middle of the screen while my solution allows this.

I also tried hardware "acceleration" stuff. The main problem is: this is limited by memory bandwidth. If you want a 2D/3D object you can rotate, etc, then yes, you can use opengles or dispmanx for this, but if you have to change the screen every frame (as in retrocomputer emulation) using the hardware acceleration makes you have to (1) prepare the texture in RAM, then (2) give it to the dispmanx/opengl which then copies it to its own RAM. While using your own procedures, you can work directly on the frame buffer memory and this means one memory copy less.

You can use dispmanx to create and display sprites. My experiments showed the limit of 26 such object if they are small and about 4-5 objects which are big (covers most of full HD screen). I experimented with dispmanx powered mouse cursor: it worked, but then there was visible delay between mouse move and the cursor, I don't know why, so I returned to the software solution which is vblank synchronized.
msx80
Posts: 2
Joined: Thu Aug 02, 2018 8:38 am

Re: Confused about options for graphics

Postby msx80 » Thu Aug 02, 2018 3:01 pm

Thanks Ultibo for the awesome recap! I'll take a look at everything. Asphyre/PXL looks interesting.

pik33 wrote:I also tried hardware "acceleration" stuff. The main problem is: this is limited by memory bandwidth. If you want a 2D/3D object you can rotate, etc, then yes, you can use opengles or dispmanx for this, but if you have to change the screen every frame (as in retrocomputer emulation) using the hardware acceleration makes you have to (1) prepare the texture in RAM, then (2) give it to the dispmanx/opengl which then copies it to its own RAM. While using your own procedures, you can work directly on the frame buffer memory and this means one memory copy less.


I'm not sure i understand this issue. If i have a bunch of images i want to draw (like spritesheets, backgrounds etc), i could make them all textures, load them at startup and i should have no problem of bandwidth, right? If i understand correctly, you're speaking about the case when you need to generate an image on the fly and paste that on the screen? In this case i understand the problem.

Btw, what kind of program are you running? Is Retromachine some kind of 8 bit like gaming system for rpi? if so, i'm in your same area of interest :)
pik33
Posts: 710
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Confused about options for graphics

Postby pik33 » Thu Aug 02, 2018 7:45 pm

Yes, it was in its basics 32-bit, but 8-bit like system. It still is, but then I created a window manager layer and an audio player (Winamp skinnable) for it.

As an 8-bit-like system it has an 8-bit framebuffer and palette register set. There is also a sprite machine which can display up to 8 (this can be changed) "pseudo-hardware" sprites (you have to poke the sprite data pointer and sprite coordinates into a memory address and the sprite machine does the rest, changing the sprite position in the next vblank)

A display list is a work in progress.
The screen procedures are vblank synchronixed as in these good old machines.
There are 6502 and SID emulators in the project, too

The opengl stuff is usable if you have a bunch of predefined objects, uploaded once to the GPU memory area and then you only manipulate them in 2d/3d area by rotating, scaling, etc. If the screen is calculated at the real time (and in most retrocomputer related cases it is) you have to upload changed textures every vblank, as the emulated computer will generate a screen every vblank while in game... and of course, all these computers should have to know when the vblank is. Here you have access to real vblankk synchronization, you can also set your rpi at 50 Hz refresh rate :)

What I want to emulate first using the code base I already have written, is 8-bit Atari :)
Gavinmc42
Posts: 1367
Joined: Sun Jun 05, 2016 12:38 pm
Location: Brisbane, Australia

Re: Confused about options for graphics

Postby Gavinmc42 » Fri Aug 03, 2018 1:50 am

Thanks Ultibo for the awesome recap! I'll take a look at everything. Asphyre/PXL looks interesting.

Let us know how that goes, it is still on my list to use, I have only run the demos.

Graphics options are many now, the good thing is they are much easier to use in Ultibo than Linux.
OpenGL is still Linux based using Eric Anholt's opensource driver, which is getting better every month.

Who knows but that too will be doable in Ultibo one day.

Deep underneath the VC4 is using control lists for speed, it is bit head bending for me at the moment.
The OpenVG examples get the closest to doing that in away that is understandable.

Return to “Graphics”

Who is online

Users browsing this forum: No registered users and 1 guest