Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Releases, updates and announcements from the Ultibo team.
User avatar
Ultibo
Site Admin
Posts: 1930
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby Ultibo » Fri Mar 30, 2018 5:15 am

In response to a report of failures when calling SerialDeviceRead() at a very high rate with the SERIAL_READ_NON_BLOCK flag set a deadlock condition was discovered in the internal functions UARTSerialDeviceReceive() and UARTSerialDeviceTransmit() due to the locking behavior of a secondary call to another function.

A fix has now been developed which addresses the issue and after extensive testing no further occurrences of the deadlock condition have been observed. If you are using the serial device API in your projects we recommend updating to the latest version of the RTL in order to apply this fix.

For details of how to apply the latest source to your Ultibo core installation and rebuild your run time library see the wiki page Building from Source or watch the Building the RTL video on YouTube.
Ultibo.org | Make something amazing
https://ultibo.org
develone
Posts: 271
Joined: Wed Dec 28, 2016 7:40 pm
Location: El Paso Tx USA

Re: Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby develone » Sat Mar 31, 2018 12:39 am

One of my 12 https://github.com/develone/Ultibo_Proj ... i2/gps.lpi

projects in https://github.com/develone/Ultibo_Projects
was working
commit 2fc363461c20af392c41ea6be6b4de06f32aa911
Author: Ultibo <admin@ultibo.org>
Date: Mon Mar 19 14:24:48 2018 +1100

I rebuilt the other projects and they all worked okay
with the eception of my gps project.
using latest version of ultibo.

commit 75c2ec246626fa5f2cf652da1af52da72d8ca7e5
Author: Ultibo <admin@ultibo.org>
Date: Fri Mar 30 16:06:07 2018 +1100

Any ideas what caused this issue?
Do I have to add additional code with chgs.

-rwxr-xr-x 1 pi pi 2835896 Mar 30 22:43 /home/pi/Ultibo_Projects/gps/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 3156940 Mar 30 22:42 /home/pi/Ultibo_Projects/jpeg2000/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 3230476 Mar 30 22:49 /home/pi/Ultibo_Projects/HelloPi/HelloVideo_TFTP/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 3267440 Mar 30 22:48 /home/pi/Ultibo_Projects/HelloTeapot/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 2837712 Mar 30 22:50 /home/pi/Ultibo_Projects/Demo/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 2604184 Mar 30 22:44 /home/pi/Ultibo_Projects/RemoteLed/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 3021564 Mar 30 22:47 /home/pi/Ultibo_Projects/HelloVideo/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 2784440 Mar 30 22:44 /home/pi/Ultibo_Projects/Ultibo_PWM_TFTP/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 3013992 Mar 30 22:46 /home/pi/Ultibo_Projects/OpenGLES/HelloGLES/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 2792648 Mar 30 22:39 /home/pi/Ultibo_Projects/wall/RPi2/kernel7.img
-rwxr-xr-x 1 pi pi 2784440 Mar 30 22:44 /home/pi/Ultibo_Projects/Ultibo_PWM_TFTP/RPi2/kernel7.img
I use the scropt ./run_demos.sh 185 or 200 xxx
where xxx
Now 12 projects
demo
gps
playvideo
video
jpeg
servo
RemoteLed
OpenGLES
HelloTeapot
HelloVideo
wall
HelloAudio

Thanks.
User avatar
Ultibo
Site Admin
Posts: 1930
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby Ultibo » Sat Mar 31, 2018 9:45 am

develone wrote:I rebuilt the other projects and they all worked okay
with the eception of my gps project.
using latest version of ultibo.

...

Any ideas what caused this issue?

Hi develone,

Can you describe what symptoms or failures you are seeing with the GPS project since applying the changes?
Ultibo.org | Make something amazing
https://ultibo.org
develone
Posts: 271
Joined: Wed Dec 28, 2016 7:40 pm
Location: El Paso Tx USA

Re: Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby develone » Sat Mar 31, 2018 9:39 pm

With the version of Core on 19 Mar. I never saw the hanging after booting.
When it works I see lots of messages in both the main & logging window.
After Sending the kernel7.img

./run_demos.sh 185 gps
tftping gps kernel7.img
to ultibo system 192.168.1.185
tftp> tftp> Sent 2835896 bytes in 10.3 seconds
Sometimes it works okay and starts receiving the serial data from the gps only 9600 baud.
Let me know if I can provide additional information.
Attachments
gpshangs1a.png
gpshangs1a.png (365.76 KiB) Viewed 339 times
User avatar
Ultibo
Site Admin
Posts: 1930
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby Ultibo » Sun Apr 01, 2018 10:58 am

develone wrote:With the version of Core on 19 Mar. I never saw the hanging after booting.
When it works I see lots of messages in both the main & logging window.
...
Sometimes it works okay and starts receiving the serial data from the gps only 9600 baud.
Let me know if I can provide additional information.

Without having a GPS to test with we connected your project up to another Pi that was setup to send continuous messages over the UART.

After many reboots, power cycles and other tests we cannot reproduce anything that seems like what you describe.

Are you able to provide a sample of what the gps data looks like?

Can you narrow it down any further, for example leave out the call to the test() function, to come up with something we can reproduce?

Can you revert your UART unit to the version prior to this fix and retest if the problem occurs or not?
Ultibo.org | Make something amazing
https://ultibo.org
mark
Posts: 1137
Joined: Mon Oct 03, 2016 2:12 am
Location: Indianapolis, US

Re: Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby mark » Mon Apr 02, 2018 4:41 am

Ultibo wrote:In response to a report of failures when calling SerialDeviceRead() at a very high rate with the SERIAL_READ_NON_BLOCK flag set a deadlock condition was discovered in the internal functions UARTSerialDeviceReceive() and UARTSerialDeviceTransmit() due to the locking behavior of a secondary call to another function.

A fix has now been developed which addresses the issue and after extensive testing no further occurrences of the deadlock condition have been observed. If you are using the serial device API in your projects we recommend updating to the latest version of the RTL in order to apply this fix.

For details of how to apply the latest source to your Ultibo core installation and rebuild your run time library see the wiki page Building from Source or watch the Building the RTL video on YouTube.

As usual industrial-strength support. I just submitted $299 for April 2018 forum support which is the usual 99 plus 100 for this repair and another 100 in honor of Paul's basic work in getting the rpi3b/zerow bluetooth started. I have enough in place to add ble peripheral mode to the users' demo this week. Regards, Mark.
Ultibo - The Internet of Things that Just Work
pjde
Posts: 378
Joined: Tue Feb 09, 2016 6:21 am
Location: Sydney, Australia

Re: Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby pjde » Tue Apr 03, 2018 7:02 am

For what its worth.

Tried develone's GPS program with latest firmware with my Microstack GPS.

Kept getting log messages of "BCM2789: Framing error on receive character"

Regards

Paul
User avatar
Ultibo
Site Admin
Posts: 1930
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby Ultibo » Tue Apr 03, 2018 10:22 am

pjde wrote:Kept getting log messages of "BCM2789: Framing error on receive character"

Thanks Paul,

Do you get this error continuously or do you just get it in batches?

Does the same error always happen or does the program work sometimes?

How frequently does the GPS send data?

@develone, Do you get these errors as well?
Ultibo.org | Make something amazing
https://ultibo.org
develone
Posts: 271
Joined: Wed Dec 28, 2016 7:40 pm
Location: El Paso Tx USA

Re: Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby develone » Tue Apr 03, 2018 1:21 pm

I think that this may have been the problem. The program that computes the checksum also writes to the file gps.dat.
I just tftp the file
Received 17059799 bytes in 59.7 seconds
Since I have ofp = fopen("gps.dat","a+");
I deleted the file and it started right away.

Since the file was so big, the writes were causing it to take a log time.and hang.
I have not seen the framing errors
Let me if this makes sense, that the long write time was hanging it up?.
pjde
Posts: 378
Joined: Tue Feb 09, 2016 6:21 am
Location: Sydney, Australia

Re: Fix for deadlock in UARTSerialDeviceReceive and UARTSerialDeviceTransmit

Postby pjde » Tue Apr 03, 2018 10:37 pm

Do you get this error continuously or do you just get it in batches?
Does the same error always happen or does the program work sometimes?
How frequently does the GPS send data?


GPS transmits approx every second which is same rate as the framing errors occur, Occasionally (say every 20 secs) get a break error.
Commented out call to test in develone's program with no affect.

Can't guarantee my GPS is not the culprit.

Regards

Paul

Return to “Ultibo”

Who is online

Users browsing this forum: No registered users and 1 guest