HDMI Audio support

Anything related to hardware, devices, drivers and how to use them with Ultibo.
msx80
Posts: 32
Joined: Thu Aug 02, 2018 8:38 am

HDMI Audio support

Postby msx80 » Fri Nov 30, 2018 7:56 pm

Hi there, i'm now trying to add (hdmi) audio to my thingy. For what i've gathered, there's nothing ready in pascal, but there's working version of hello_audio in C (this one?), called from inside ultibo in Examples. Is this correct?

So i guess it's a matter of porting that example in freepascal, shouldn't be too hard. Has anybody attempted it? Any reason why nobody has ever done it?

Thanks :)
pik33
Posts: 788
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: HDMI Audio support

Postby pik33 » Sat Dec 01, 2018 7:48 am

Yes, it is and it can be done using OMX calls. I didn't tried this yet as I have my audio driver for audio jack.

The OMX is overcomplicated and underdocumented and this seems to be main problem with it.

The HelloAudo example calls the ready made code in C so this cannot be used in another project without translating this C code to Pascal, or make some API out of it.

As I did something for the camera using omx calls, I can also try to make an audio driver: the main problem is free time with working brain: I have something other to do, I have a working audio on audio jack and I haven't a hdmi monitor with sound so the HDMI sound is at low priority on my list (moved from "undoable - knowledge too low" to "low priority" after I run the camera) .
User avatar
Ultibo
Site Admin
Posts: 2079
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: HDMI Audio support

Postby Ultibo » Sat Dec 01, 2018 9:08 am

msx80 wrote:Hi there, i'm now trying to add (hdmi) audio to my thingy. For what i've gathered, there's nothing ready in pascal, but there's working version of hello_audio in C (this one?), called from inside ultibo in Examples. Is this correct?

So i guess it's a matter of porting that example in freepascal, shouldn't be too hard. Has anybody attempted it? Any reason why nobody has ever done it?

Thanks :)

Hi msx80,

Ultibo member and regular contributor @pjde has done a lot of work on porting the OMX headers and examples to FreePascal, you can find his openmax repository here including an audio test example in FPC.

You might be able to piggyback on what he has done or collaborate in expanding it further, either way it would be great to see more of this working.
Ultibo.org | Make something amazing
https://ultibo.org
msx80
Posts: 32
Joined: Thu Aug 02, 2018 8:38 am

Re: HDMI Audio support

Postby msx80 » Sat Dec 01, 2018 2:13 pm

Hi pik33, i've seen your code, but i'm mostly interested in HDMI audio :)

Great, some headers for OpenMAX will surely be useful, and there's an example for audio too!

I've actually started translating the HelloAudio demo to pascal. I translates the play_api_test function succesfully leaving the other functions in C, and it works correctly. With this i can already start experimenting with various settings.

Now the problem is i don't really know much about sound systems :P what i understood from this code is that it instantiates 10 buffers and fill them as soon as they're emptied with data derived from the main sinewave. There's a callback that i think is called when a buffer is emptied so i guess one can use that to maintain a steady stream of data. I'll have to experiment a bit.

If anybody is interested, here's the HelloAudio so far

Code: Select all

program HelloAudio;

{$mode objfpc}{$H+}

{ VideoCore IV example - Hello Audio                                           }
{                                                                              }
{ A very simple example showing audio output over HDMI                         }
{                                                                              }
{ To compile the example select Run, Compile (or Run, Build) from the menu.    }
{                                                                              }
{ Once compiled copy the kernel7.img file to an SD card along with the         }
{ firmware files and use it to boot your Raspberry Pi.                         }
{                                                                              }
{ This version is for Raspberry Pi 2B and will also work on a 3B.              }

uses
  RaspberryPi2, {Include RaspberryPi2 to make sure all standard functions are included}
  GlobalConst,
  GlobalTypes,
  Threads,
  Console,
  ctypes,
  classes,sysutils,
  UltiboUtils,  {Include Ultibo utils for some command line manipulation}
  Syscalls,     {Include the Syscalls unit to provide C library support}
  VC4;          {Include the VC4 unit to enable access to the GPU}

var
 WindowHandle:TWindowHandle;

{Link our C library to include the original example}
{$linklib hello_audio}


const BUFFER_SIZE_SAMPLES = 1024;
const CTTW_SLEEP_TIME = 10;
const MIN_LATENCY_TIME = 20;
const N_WAVE  = 1024;    (* dimension of Sinewave[] *)


 var Sinewave:array[0..N_WAVE-1] of cshort = (
   0,    201,    402,    603,    804,   1005,   1206,   1406,
   1607,   1808,   2009,   2209,   2410,   2610,   2811,   3011,
   3211,   3411,   3611,   3811,   4011,   4210,   4409,   4608,
   4807,   5006,   5205,   5403,   5601,   5799,   5997,   6195,
   6392,   6589,   6786,   6982,   7179,   7375,   7571,   7766,
   7961,   8156,   8351,   8545,   8739,   8932,   9126,   9319,
   9511,   9703,   9895,  10087,  10278,  10469,  10659,  10849,
   11038,  11227,  11416,  11604,  11792,  11980,  12166,  12353,
   12539,  12724,  12909,  13094,  13278,  13462,  13645,  13827,
   14009,  14191,  14372,  14552,  14732,  14911,  15090,  15268,
   15446,  15623,  15799,  15975,  16150,  16325,  16499,  16672,
   16845,  17017,  17189,  17360,  17530,  17699,  17868,  18036,
   18204,  18371,  18537,  18702,  18867,  19031,  19194,  19357,
   19519,  19680,  19840,  20000,  20159,  20317,  20474,  20631,
   20787,  20942,  21096,  21249,  21402,  21554,  21705,  21855,
   22004,  22153,  22301,  22448,  22594,  22739,  22883,  23027,
   23169,  23311,  23452,  23592,  23731,  23869,  24006,  24143,
   24278,  24413,  24546,  24679,  24811,  24942,  25072,  25201,
   25329,  25456,  25582,  25707,  25831,  25954,  26077,  26198,
   26318,  26437,  26556,  26673,  26789,  26905,  27019,  27132,
   27244,  27355,  27466,  27575,  27683,  27790,  27896,  28001,
   28105,  28208,  28309,  28410,  28510,  28608,  28706,  28802,
   28897,  28992,  29085,  29177,  29268,  29358,  29446,  29534,
   29621,  29706,  29790,  29873,  29955,  30036,  30116,  30195,
   30272,  30349,  30424,  30498,  30571,  30643,  30713,  30783,
   30851,  30918,  30984,  31049,
   31113,  31175,  31236,  31297,
   31356,  31413,  31470,  31525,  31580,  31633,  31684,  31735,
   31785,  31833,  31880,  31926,  31970,  32014,  32056,  32097,
   32137,  32176,  32213,  32249,  32284,  32318,  32350,  32382,
   32412,  32441,  32468,  32495,  32520,  32544,  32567,  32588,
   32609,  32628,  32646,  32662,  32678,  32692,  32705,  32717,
   32727,  32736,  32744,  32751,  32757,  32761,  32764,  32766,
   32767,  32766,  32764,  32761,  32757,  32751,  32744,  32736,
   32727,  32717,  32705,  32692,  32678,  32662,  32646,  32628,
   32609,  32588,  32567,  32544,  32520,  32495,  32468,  32441,
   32412,  32382,  32350,  32318,  32284,  32249,  32213,  32176,
   32137,  32097,  32056,  32014,  31970,  31926,  31880,  31833,
   31785,  31735,  31684,  31633,  31580,  31525,  31470,  31413,
   31356,  31297,  31236,  31175,  31113,  31049,  30984,  30918,
   30851,  30783,  30713,  30643,  30571,  30498,  30424,  30349,
   30272,  30195,  30116,  30036,  29955,  29873,  29790,  29706,
   29621,  29534,  29446,  29358,  29268,  29177,  29085,  28992,
   28897,  28802,  28706,  28608,  28510,  28410,  28309,  28208,
   28105,  28001,  27896,  27790,  27683,  27575,  27466,  27355,
   27244,  27132,  27019,  26905,  26789,  26673,  26556,  26437,
   26318,  26198,  26077,  25954,  25831,  25707,  25582,  25456,
   25329,  25201,  25072,  24942,  24811,  24679,  24546,  24413,
   24278,  24143,  24006,  23869,  23731,  23592,  23452,  23311,
   23169,  23027,  22883,  22739,  22594,  22448,  22301,  22153,
   22004,  21855,  21705,  21554,  21402,  21249,  21096,  20942,
   20787,  20631,  20474,  20317,  20159,  20000,  19840,  19680,
   19519,  19357,  19194,  19031,  18867,  18702,  18537,  18371,
   18204,  18036,  17868,  17699,  17530,  17360,  17189,  17017,
   16845,  16672,  16499,  16325,  16150,  15975,  15799,  15623,
   15446,  15268,  15090,  14911,  14732,  14552,  14372,  14191,
   14009,  13827,  13645,  13462,  13278,  13094,  12909,  12724,
   12539,  12353,  12166,  11980,  11792,  11604,  11416,  11227,
   11038,  10849,  10659,  10469,  10278,  10087,   9895,   9703,
   9511,   9319,   9126,   8932,   8739,   8545,   8351,   8156,
   7961,   7766,   7571,   7375,   7179,   6982,   6786,   6589,
   6392,   6195,   5997,   5799,   5601,   5403,   5205,   5006,
   4807,   4608,   4409,   4210,   4011,   3811,   3611,   3411,
   3211,   3011,   2811,   2610,   2410,   2209,   2009,   1808,
   1607,   1406,   1206,   1005,    804,    603,    402,    201,
   0,   -201,   -402,   -603,   -804,  -1005,  -1206,  -1406,
   -1607,  -1808,  -2009,  -2209,  -2410,  -2610,  -2811,  -3011,
   -3211,  -3411,  -3611,  -3811,  -4011,  -4210,  -4409,  -4608,
   -4807,  -5006,  -5205,  -5403,  -5601,  -5799,  -5997,  -6195,
   -6392,  -6589,  -6786,  -6982,  -7179,  -7375,  -7571,  -7766,
   -7961,  -8156,  -8351,  -8545,  -8739,  -8932,  -9126,  -9319,
   -9511,  -9703,  -9895, -10087, -10278, -10469, -10659, -10849,
   -11038, -11227, -11416, -11604, -11792, -11980, -12166, -12353,
   -12539, -12724, -12909, -13094, -13278, -13462, -13645, -13827,
   -14009, -14191, -14372, -14552, -14732, -14911, -15090, -15268,
   -15446, -15623, -15799, -15975, -16150, -16325, -16499, -16672,
   -16845, -17017, -17189, -17360, -17530, -17699, -17868, -18036,
   -18204, -18371, -18537, -18702, -18867, -19031, -19194, -19357,
   -19519, -19680, -19840, -20000, -20159, -20317, -20474, -20631,
   -20787, -20942, -21096, -21249, -21402, -21554, -21705, -21855,
   -22004, -22153, -22301, -22448, -22594, -22739, -22883, -23027,
   -23169, -23311, -23452, -23592, -23731, -23869, -24006, -24143,
   -24278, -24413, -24546, -24679, -24811, -24942, -25072, -25201,
   -25329, -25456, -25582, -25707, -25831, -25954, -26077, -26198,
   -26318, -26437, -26556, -26673, -26789, -26905, -27019, -27132,
   -27244, -27355, -27466, -27575, -27683, -27790, -27896, -28001,
   -28105, -28208, -28309, -28410, -28510, -28608, -28706, -28802,
   -28897, -28992, -29085, -29177, -29268, -29358, -29446, -29534,
   -29621, -29706, -29790, -29873, -29955, -30036, -30116, -30195,
   -30272, -30349, -30424, -30498, -30571, -30643, -30713, -30783,
   -30851, -30918, -30984, -31049, -31113, -31175, -31236, -31297,
   -31356, -31413, -31470, -31525, -31580, -31633, -31684, -31735,
   -31785, -31833, -31880, -31926, -31970, -32014, -32056, -32097,
   -32137, -32176, -32213, -32249, -32284, -32318, -32350, -32382,
   -32412, -32441, -32468, -32495, -32520, -32544, -32567, -32588,
   -32609, -32628, -32646, -32662, -32678, -32692, -32705, -32717,
   -32727, -32736, -32744, -32751, -32757, -32761, -32764, -32766,
   -32767, -32766, -32764, -32761, -32757, -32751, -32744, -32736,
   -32727, -32717, -32705, -32692, -32678, -32662, -32646, -32628,
   -32609, -32588, -32567, -32544, -32520, -32495, -32468, -32441,
   -32412, -32382, -32350, -32318, -32284, -32249, -32213, -32176,
   -32137, -32097, -32056, -32014, -31970, -31926, -31880, -31833,
   -31785, -31735, -31684, -31633, -31580, -31525, -31470, -31413,
   -31356, -31297, -31236, -31175, -31113, -31049, -30984, -30918,
   -30851, -30783, -30713, -30643, -30571, -30498, -30424, -30349,
   -30272, -30195, -30116, -30036, -29955, -29873, -29790, -29706,
   -29621, -29534, -29446, -29358, -29268, -29177, -29085, -28992,
   -28897, -28802, -28706, -28608, -28510, -28410, -28309, -28208,
   -28105, -28001, -27896, -27790, -27683, -27575, -27466, -27355,
   -27244, -27132, -27019, -26905, -26789, -26673, -26556, -26437,
   -26318, -26198, -26077, -25954, -25831, -25707, -25582, -25456,
   -25329, -25201, -25072, -24942, -24811, -24679, -24546, -24413,
   -24278, -24143, -24006, -23869, -23731, -23592, -23452, -23311,
   -23169, -23027, -22883, -22739, -22594, -22448, -22301, -22153,
   -22004, -21855, -21705, -21554, -21402, -21249, -21096, -20942,
   -20787, -20631, -20474, -20317, -20159, -20000, -19840, -19680,
   -19519, -19357, -19194, -19031, -18867, -18702, -18537, -18371,
   -18204, -18036, -17868, -17699, -17530, -17360, -17189, -17017,
   -16845, -16672, -16499, -16325, -16150, -15975, -15799, -15623,
   -15446, -15268, -15090, -14911, -14732, -14552, -14372, -14191,
   -14009, -13827, -13645, -13462, -13278, -13094, -12909, -12724,
   -12539, -12353, -12166, -11980, -11792, -11604, -11416, -11227,
   -11038, -10849, -10659, -10469, -10278, -10087,  -9895,  -9703,
   -9511,  -9319,  -9126,  -8932,  -8739,  -8545,  -8351,  -8156,
   -7961,  -7766,  -7571,  -7375,  -7179,  -6982,  -6786,  -6589,
   -6392,  -6195,  -5997,  -5799,  -5601,  -5403,  -5205,  -5006,
   -4807,  -4608,  -4409,  -4210,  -4011,  -3811,  -3611,  -3411,
   -3211,  -3011,  -2811,  -2610,  -2410,  -2209,  -2009,  -1808,
   -1607,  -1406,  -1206,  -1005,   -804,   -603,   -402,   -201
);


type AUDIOPLAY_STATE_T = record
  // not really needed for now
end;
  PAUDIOPLAY_STATE_T = ^ AUDIOPLAY_STATE_T;
  PPAUDIOPLAY_STATE_T = ^ PAUDIOPLAY_STATE_T;


//  #define OUT_CHANNELS(num_channels) ((num_channels) > 4 ? 8: (num_channels) > 2 ? 4: (num_channels))
function OUT_CHANNELS(num_channels:integer):integer;
begin
 if (num_channels) > 4 then result := 8
 else if (num_channels) > 2 then result := 4
 else result := num_channels;

end;

function xSIN(x:cint):cint;
begin
 result := Sinewave[((x) shr 6) and (N_WAVE-1)];
end;

// procedure play_api_test(samplerate, bitdepth, nchannels, dest:cint); cdecl; external 'hello_audio' name 'play_api_test';
function  audioplay_create(handle:PPAUDIOPLAY_STATE_T; sample_rate, num_channels, bit_depth, num_buffers, buffer_size:cuint32):int32_t; cdecl; external 'hello_audio' name 'audioplay_create';
function audioplay_set_dest(st: PAUDIOPLAY_STATE_T; const name:pchar):cint32; cdecl; external 'hello_audio';
function audioplay_get_buffer(st: PAUDIOPLAY_STATE_T):puint8; cdecl; external 'hello_audio';
function audioplay_get_latency(st: PAUDIOPLAY_STATE_T):uint32; cdecl; external 'hello_audio';
function audioplay_play_buffer(st: PAUDIOPLAY_STATE_T; buffer:puint8; length:uint32):cint32; cdecl; external 'hello_audio';
function audioplay_delete(st: PAUDIOPLAY_STATE_T):uint32; cdecl; external 'hello_audio';

procedure play_api_test(samplerate, bitdepth, nchannels, dest:cint);
var st:pAUDIOPLAY_STATE_T;
   ret:cint32;
   i,j,n:cuint;
   phase:cint = 0;
   inc2:cint = 256 shl 16;
   dinc:cint = 0;
   buffer_size:cint;


   buf :puint8;
   p:pint16;
   latency:cuint32 ;
    val:int16;
begin
   buffer_size:= (BUFFER_SIZE_SAMPLES * bitdepth * OUT_CHANNELS(nchannels)) shr 3;
   ret := audioplay_create(@st, samplerate, nchannels, bitdepth, 10, buffer_size);

   ConsoleWindowWriteLn(WindowHandle,'audioplay_create ret '+inttostr(ret));

   ret := audioplay_set_dest(st, 'hdmi');

   ConsoleWindowWriteLn(WindowHandle,'audioplay_set_dest ret '+inttostr(ret));

   // iterate for 5 seconds worth of packets
   for n:=0 to ((samplerate * 1000) div BUFFER_SIZE_SAMPLES)-1 do
   begin

      buf := audioplay_get_buffer(st);
      while buf = nil do
         begin
            ConsoleWindowWriteLn(WindowHandle,'audioplay_get_buffer is null, waiting');
                usleep(10*1000);
                buf := audioplay_get_buffer(st);
         end;

      p := pcint16(buf);

      // fill the buffer
      for i:=0 to BUFFER_SIZE_SAMPLES-1 do
      begin

         val := xSIN(phase);
         inc(phase, inc2 shr 16);
         inc2 += dinc;
         if (inc2 shr 16) < 512 then
            inc(dinc)
         else
            dec(dinc);

         for j:=0 to OUT_CHANNELS(nchannels)-1 do
         begin
            if bitdepth = 32 then
             begin
               // *p++ = 0;
               p^:=0;
               inc(p);
             end;
            // *p++ = val;
            p^:=val;
            inc(p);
         end;
      end;

      // try and wait for a minimum latency time (in ms) before
      // sending the next packet
      latency :=audioplay_get_latency(st);
      while latency  > (samplerate * (MIN_LATENCY_TIME + CTTW_SLEEP_TIME) div 1000) do
         begin
         usleep(CTTW_SLEEP_TIME*1000);
         latency :=audioplay_get_latency(st);
         end;

      ret := audioplay_play_buffer(st, buf, buffer_size);
      ConsoleWindowWriteLn(WindowHandle,'audioplay_play_buffer ret '+inttostr(ret));
   end;

   audioplay_delete(st);
end;


begin
 {Create a console window as usual}
 WindowHandle:=ConsoleWindowCreate(ConsoleDeviceGetDefault,CONSOLE_POSITION_FULL,True);

 ConsoleWindowWriteLn(WindowHandle,'Starting Hello Audio');
 BCMHostInit;

 ConsoleWindowWriteLn(WindowHandle,'Play api '+inttostr(cardinal(@play_api_test)));
 play_api_test(48000, 16, 2, 1);
 
 ConsoleWindowWriteLn(WindowHandle,'Completed Hello Audio');
 
 {Halt the main thread here}
 ThreadHalt(0);
end.
pjde
Posts: 425
Joined: Tue Feb 09, 2016 6:21 am
Location: Sydney, Australia

Re: HDMI Audio support

Postby pjde » Sat Dec 01, 2018 8:02 pm

Check out https://ultibo.org/forum/viewtopic.php?f=12&t=497&p=7430&hilit=audio#p7430

There is an OMX example to play wave files via HDMI.

Regards

Paul
Gavinmc42
Posts: 1420
Joined: Sun Jun 05, 2016 12:38 pm
Location: Brisbane, Australia

Re: HDMI Audio support

Postby Gavinmc42 » Thu Dec 06, 2018 3:20 am

Been looking at this recently,
Found a mention that audio on HDMI takes frames from SPDIF and inserts them into the HDMI stream.
https://www.kernel.org/doc/html/v4.16/gpu/vc4.html

How the SPDIF is generated I still don't know.
Most addon audio hats use a i2s to SPDIFchip.
So is there an undocumented audio/SPDIF to HDMI hardware or is this done in userland or start.elf?

The standard hello_audio example defaults to analog output?
Pauls's code does the same sort of thing, tells OMX the stream and the destination(local or HDMI) and OMX handles the magic conversion somehow.

Looking inside start.elf and start_cd.elf it does look like the audio is handled here.

Eric Anholt's driver has more clues

https://github.com/torvalds/linux/blob/ ... vc4_hdmi.c
* DOC: VC4 Falcon HDMI module
*
* The HDMI core has a state machine and a PHY. On BCM2835, most of
* the unit operates off of the HSM clock from CPRMAN. It also
* internally uses the PLLH_PIX clock for the PHY.
*
* HDMI infoframes are kept within a small packet ram, where each
* packet can be individually enabled for including in a frame.
*
* HDMI audio is implemented entirely within the HDMI IP block. A
* register in the HDMI encoder takes SPDIF frames from the DMA engine
* and transfers them over an internal MAI (multi-channel audio
* interconnect) bus to the encoder side for insertion into the video
* blank regions.
*
* The driver's HDMI encoder does not yet support power management.
* The HDMI encoder's power domain and the HSM/pixel clocks are kept
* continuously running, and only the HDMI logic and packet ram are
* powered off/on at disable/enable time.
*
* The driver does not yet support CEC control, though the HDMI
* encoder block has CEC support.
*/


HDMI IP block - propriety IP?
vc4_regs.h HDMI audio registers ;)

Code: Select all

/* If set, then multichannel, otherwise 2 channel. */
Multichannel audio? Pi's can do 5.1 audio?

I had to google this :o
Q. Does HDMI support Dolby 5.1 audio and high-resolution audio formats?

Yes. From the start, HDMI was defined to carry 8-channels, of 192kHz, 24-bit uncompressed audio, which exceeds all current consumer media formats. In addition, HDMI can carry any flavor of compressed audio format such as Dolby or DTS. (Such compressed formats are the only multi-channel or high-resolution audio formats that can be carried across the older S/PDIF or AES/EBU interfaces.) Additionally, most existing HDMI sources can output any compressed stream, and the newer sources can output uncompressed 6-channel, 96kHz audio from a DVD-Audio disk. There are A/V receivers on the market that can accept and process the 6- or 8-channel audio from HDMI.


Wow, can a Pi become a 8 channel audio generator?
Compressed audio has lower data rate?
7.1 channels of Procedurally generated audio? Whoops better not mention what that's for :lol:

Never thought about this before but are Pi's being used in audio effects setups?
Ethernet to 7.1 speaker systems? 7.1 museum effects audio?
Gavinmc42
Posts: 1420
Joined: Sun Jun 05, 2016 12:38 pm
Location: Brisbane, Australia

Re: HDMI Audio support

Postby Gavinmc42 » Thu Dec 06, 2018 3:23 am

https://www.raspberrypi.org/forums/view ... p?p=796689
Anyone tested this can do 8 channels?

Return to “General”

Who is online

Users browsing this forum: No registered users and 1 guest