Bit-banged 8-port RS-232 communication library

Discussion and questions about programming with Ultibo.
ZeroHobby
Posts: 7
Joined: Fri Aug 09, 2019 2:11 pm

Bit-banged 8-port RS-232 communication library

Postby ZeroHobby » Sat Aug 10, 2019 3:29 pm

Hello everyone! I stumbled across Ultibo from a Raspberry Pi forum, and it looks like I hit the jackpot of where I need to be! (One second boot time? You've got to be kidding me :D )

First, a little background: I've done a LOT of programming for embedded systems...in 100% assembler (i.e. modified Z80, Microchip PIC10,12,16,18, dsPIC30, ARM7TDMI), as well as some programming in higher level languages such as Basic, C, and Spin (Parallax Propeller; I understand it's based a lot on Pascal). Perhaps my biggest programming project was to code an entire BASIC IDE in 100% pure ARM7TDMI assembler (using both ARM and THUMB) on a Nintendo Gameboy Advance. That took me several months, resulting in 741KB of source code (with a bazillion comments, mind you). I actually quite enjoyed programming in pure ARM7, as each instruction is SO powerful...compared to a Microchip PIC, anyway.
All that to say that I should be able to figure a lot of things out with Ultibo/Raspberry Pi (and that I'm not afraid of Pascal), but I'm here to ask for a little guidance/maybe some help, etc. I've always been daunted by the thought of a processor running at 1GHz (and running Linux to boot!)--that sounds a lot harder to work with than an 8-bit Microchip PIC that tops out at 20MHz. But after seeing some Ultibo examples, and finding that it's not another operating system that you can watch YouTube on...I think this will work.

My project: I have a LOT of things to monitor and control; mostly PICs measuring various sensors, and converting that to an RS-232 connection for transmission. As of right now, I'm using a Parallax Propeller; it is a 32-bit CPU with 8 cores, running at 80MHz. It's shortfall is the ginormous amount of 32K RAM shared between program code and data RAM. (That's not even enough for Bill Gates.) It has no problem running a 4-port RS-232 driver (at multiple different speeds, including FIFO buffers on each port) from a single core--and being a non-pipelined processor, that works out at 20MIPS.

I currently have a Raspberry Pi Zero v1.3; bought it at MicroCenter a few years back, and finally pulled it out of the package this year when I found Ultibo and decided that there was a chance that it could do better than the $12 Propeller (that's the price I paid for it, anyway!) I'd like to do a lot of graphing, data storage, analyzing, etc.--and the Propeller is a little small for that sort of work. I could probably shoehorn it under the hood, but that'd be a little difficult.

SO: I am wondering if an 8-port (bit-banged) 9600baud RS-232 driver could successfully be written in Ultibo for the Raspberry Pi Zero? (Anything can be written, the question is whether it'll actually work :twisted: )
From what I've gleaned from the Ultibo forum over the past few days, I'm guessing that I will need to configure a timer or similar interrupt to fire at ~4x the baud rate (to reduce errors, as RS-232 is asynchronous), so assume (9600 * 4) = 76.8KHz. This high priority interrupt would call a short and sweet routine written in Assembler:
--- read the I/O port
--- (loop through all eight ports):
------> Is the send iteration count nonzero?
---------> Yes, so decrement iteration count. Are the 2 LSBs "00"? (using the two LSBs to divide by 4, so we send at the proper rate)
------------> Yes, shift sending byte over, mask and apply output bit to proper I/O bit
---------> No, check for a new byte in the FIFO buffer?
------------> There is a byte. Move it to the buffer for sending. Assert I/O pin for RS-232 start bit. Set iteration count to (10 * 4) [need to send START and STOP bits, thus the 10 count]
------> Is the receive iteration count nonzero?
---------> Yes, so decrement iteration count. Are the two LSBs "00"? (ditto; this is why we need the x4 resolution, NOT for sending!)
------------> Yes, shift receiving byte over, copy the I/O pin bit to the register [Have to write the byte to the FIFO when done]
---------> No, check the I/O bit for a START bit?
------------> Yes, set iteration count to (9 * 4) + 2 [try to sample in the middle of the signal, not on the ends. This is the reason for the 4x rate, not to mention the ease of dividing by 4.]
--- write the I/O port back

(note on receive the bitcount of 9, not 10: the "stop" bit can basically be ignored, as it's the same line level as "nothing.")
(on the iteration counts: note that they will get set in whichever of the four stages the "START" bit is detected, or something is put on the FIFO, etc.)

I am not certain whether there is much of a "market" on Ultibo for something like this, so I'm not sure how much effort would be good to putting into making it work on all PIs. (For that matter, I am not very good at writing cross-platform code.)
Regardless, the assembler routine will need the following:
- direct access to the I/O port register (shouldn't be a problem on the single-core CPU of the RPI Zero v1.3)
- access to a memory location accessible from Pascal (FIFO, etc.)
- static variables for the iteration counts, send/receive work registers. (IF the interrupt uses a different CPU mode [that Ultibo doesn't use], it would be possible to have a separate stack location. However, LDR/STM are the fastest memory access instructions, so it probably isn't much of a problem.)

Thoughts? Suggestions? Pointers? Help? Once I figure out how Ultibo functions, it should be a no-brainer; I just don't want to have three weeks of baby steps if I don't have to ;-).
pik33
Posts: 891
Joined: Fri Sep 30, 2016 6:30 pm
Location: Poland
Contact:

Re: Bit-banged 8-port RS-232 communication library

Postby pik33 » Sun Aug 11, 2019 6:46 am

The problem is: Ultibo is a multitasking preemptive environment. There are a lot of things running in the background. As there is no 'impossible" things here, I think it is still doable, but you have to take care of these threads/scheduler/interrupts things which may be not an easy task.

Having a 4-core RPi (2,3, in the future: 4) can make things simpler as you can make a "dedicated cpu" free of interrupts and scheduler and then use it for bitbanging, but this approach is not doable in one-core RPi0

The alternative may be Parallax Propeller: doing 8x UART is easy there
User avatar
Ultibo
Site Admin
Posts: 2303
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Bit-banged 8-port RS-232 communication library

Postby Ultibo » Sun Aug 11, 2019 10:40 am

Hi ZeroHobby, welcome to Ultibo!

ZeroHobby wrote:SO: I am wondering if an 8-port (bit-banged) 9600baud RS-232 driver could successfully be written in Ultibo for the Raspberry Pi Zero? (Anything can be written, the question is whether it'll actually work :twisted: )

While you are correct that anything can be written, in this case I think your requirements sound quite reasonable as long as you are willing to put in some effort to make it happen.

I would ask first up if you have considered using something like I2C instead of RS232 since that can support multiple devices on a single bus (3 wires) and can run at speeds way beyond 9600baud with the advantage that you can let the hardware handle all the timing details. Of course if your sensors and other devices don't support I2C (or require a lot of work to support it) then that probably wipes out most of the gains.

ZeroHobby wrote: - direct access to the I/O port register (shouldn't be a problem on the single-core CPU of the RPI Zero v1.3)

I'm assuming by I/O port register you mean the GPIO device in the Pi? I'll also make the assumption that you want 8 separate RS232 connections and are not using some sort of multiplexing scheme to share a single connection. If that is the case you'll need at least 16 GPIO pins in order to support RX and TX for 8 connections, unless you want to do flow control as well which will need more.

The register organisation of the Pi means you could read all available pins (54 of them) in 3 32bit reads, if you select the pins that you use carefully you might be able to read all 8 inputs in one or two reads. A similar situation applies to writing to the GPIO outputs.

ZeroHobby wrote: - access to a memory location accessible from Pascal (FIFO, etc.)

Any interrupt handler will execute in FIQ or SVC mode, will have access to all memory and devices and can access variables and structures used in Pascal.

ZeroHobby wrote: - static variables for the iteration counts, send/receive work registers. (IF the interrupt uses a different CPU mode [that Ultibo doesn't use], it would be possible to have a separate stack location. However, LDR/STM are the fastest memory access instructions, so it probably isn't much of a problem.)

All interrupt handlers run on their own stack, that's part of the Ultibo design. The interrupt dispatch mechanism also allows passing a pointer to a block of data when registering your handler, then you receive that pointer when your handler is called and you don't need to worry about static or global variables etc.

ZeroHobby wrote:Thoughts? Suggestions? Pointers? Help?

There's a lot of ground to cover but you seem to have a pretty clear idea of the logic you would need to make this work, the best idea would be to start with some experiments to work out the nuts and bolts of what you need and ask questions as you run into things that aren't clear or you can't find elsewhere.

We've covered using the ARM timer with an FIQ interrupt in other posts and we have an example here as a starting point.

Mixing assembler in line with FreePascal has also been discussed multiple times, a couple of posts that you could start from are this one and this one but there are more if you search within the forum.

Ultibo member Ron did some work early on with software I2C and SPI and has some code available on his website that might be helpful as well.

Overall the 76.8KHz interrupt rate should be fairly easy to achieve, provided the handler can complete all the needed operations in the timeframe before the next interrupt is due.

Hope that helps, feel free to ask any questions as you think of them.


pik33 wrote:The problem is: Ultibo is a multitasking preemptive environment. There are a lot of things running in the background. As there is no 'impossible" things here, I think it is still doable, but you have to take care of these threads/scheduler/interrupts things which may be not an easy task.

I think this is unlikely to be an issue if the timer interrupt is setup to use FIQ (fast interrupt) which will preempt everything else in the system. Of course you do need to make sure that anything else you want happening like disk or network I/O, graphics display etc has time to execute otherwise they will run very slowly, overall 9600baud is quite a low speed so it shouldn't require more than a small percentage of the CPU time.
Ultibo.org | Make something amazing
https://ultibo.org
ZeroHobby
Posts: 7
Joined: Fri Aug 09, 2019 2:11 pm

Re: Bit-banged 8-port RS-232 communication library

Postby ZeroHobby » Sun Aug 11, 2019 5:55 pm

Ultibo wrote:Hi ZeroHobby, welcome to Ultibo!

ZeroHobby wrote:SO: I am wondering if an 8-port (bit-banged) 9600baud RS-232 driver could successfully be written in Ultibo for the Raspberry Pi Zero? (Anything can be written, the question is whether it'll actually work :twisted: )

While you are correct that anything can be written, in this case I think your requirements sound quite reasonable as long as you are willing to put in some effort to make it happen.

I would ask first up if you have considered using something like I2C instead of RS232 since that can support multiple devices on a single bus (3 wires) and can run at speeds way beyond 9600baud with the advantage that you can let the hardware handle all the timing details. Of course if your sensors and other devices don't support I2C (or require a lot of work to support it) then that probably wipes out most of the gains.

I2C is not a viable option, as it was not designed for long-range communication. Think well over 100 feet of communications lines, with a good bit of EMI/RF interference thrown in for good measure. On top of that, say one sensor gets damaged, shorting the lines out--and the ENTIRE system goes down. Thirdly, it's really easy to debug RS-232 devices with a laptop: plug in a USB to serial converter, pull up PuTTY (or GtkTerm) and debug away.
I am using a 7-inch touchscreen LCD from BuyDisplay.com for the "main screen", which the Pi Zero would directly run from the I2C interface (I'll have to code my own driver to use the Ultibo Console object with it...shouldn't be TOO hard!) I have that display configured to use I2C (RA8875 driver controller + FT5206 touchscreen controller on the same two wires), and I had a terrible time getting that to work over six feet of lines without interference garbling up the display--not just what was being written to it, but the display configuration as well. (I ended up having to put a P82B715 I2C bus extender chip at each end of the wire, which helped a lot...but I still have garbled communications from time to time.)

All that to say...RS-232 was designed as a long-range communications protocol, and at least in my case, it's what I need to use.

And yes, I am quite willing to put in a lot of effort to make it happen. I didn't come to Ultibo forums expecting an "all you can eat" buffet of programmers throwing perfectly working code at me :lol: . I'll just likely need a hand or two along the way to get to the finish line.

Ultibo wrote:I'm assuming by I/O port register you mean the GPIO device in the Pi? I'll also make the assumption that you want 8 separate RS232 connections and are not using some sort of multiplexing scheme to share a single connection. If that is the case you'll need at least 16 GPIO pins in order to support RX and TX for 8 connections, unless you want to do flow control as well which will need more.

The register organisation of the Pi means you could read all available pins (54 of them) in 3 32bit reads, if you select the pins that you use carefully you might be able to read all 8 inputs in one or two reads. A similar situation applies to writing to the GPIO outputs.

Yes, I am referring to the Pi's GPIO register(s). (Didn't know there was more than one...guess that shows that I haven't really done all THAT much research!) And that would be very nice to be able to read all 8 input ports in one read, and write the output ports in one write. Can you share a link to the Pi Zero's register map?
Yes, I am planning eight separate RS232 connections (16 I/O ports total), no multiplexing. Simultaneous transmissions/receptions could occur at any time. No flow control necessary or planned--just RX/TX lines.

Ultibo wrote:
ZeroHobby wrote: - access to a memory location accessible from Pascal (FIFO, etc.)

Any interrupt handler will execute in FIQ or SVC mode, will have access to all memory and devices and can access variables and structures used in Pascal.

ZeroHobby wrote: - static variables for the iteration counts, send/receive work registers. (IF the interrupt uses a different CPU mode [that Ultibo doesn't use], it would be possible to have a separate stack location. However, LDR/STM are the fastest memory access instructions, so it probably isn't much of a problem.)

All interrupt handlers run on their own stack, that's part of the Ultibo design. The interrupt dispatch mechanism also allows passing a pointer to a block of data when registering your handler, then you receive that pointer when your handler is called and you don't need to worry about static or global variables etc.

Perfect, perfect, and perfect again. Passing a pointer to a block of data...that's perfect.
One slight issue that might come up is if the FIQ happens to land in the middle of an R-M-W cycle in Ultibo core...? Conceivably, that could cause Ultibo to overwrite the values just written via the FIQ RS232 service routine. True? False? Thoughts? (Saving context during interrupts was one of those things I learned early on with PICs...because I can guarantee you that if an interrupt potentially COULD mess something up, it eventually WILL.)

Thank you for the links; I'd seen the FIQ discussion before, and concluded that with a very short, sweet and simple assembler routine, I should have a relatively low CPU load with a 76.8KHz interrupt rate. It'll be the basis of my starting point. That is, after I get the I2C communication working to my touchscreen display.

Ultibo wrote:
pik33 wrote:The problem is: Ultibo is a multitasking preemptive environment. There are a lot of things running in the background. As there is no 'impossible" things here, I think it is still doable, but you have to take care of these threads/scheduler/interrupts things which may be not an easy task.

I think this is unlikely to be an issue if the timer interrupt is setup to use FIQ (fast interrupt) which will preempt everything else in the system. Of course you do need to make sure that anything else you want happening like disk or network I/O, graphics display etc has time to execute otherwise they will run very slowly, overall 9600baud is quite a low speed so it shouldn't require more than a small percentage of the CPU time.

If FIQ is the highest interrupt level in the system, I should be just fine; any system tasks will occur at far slower rates. (The 8 serial ports are the most demanding part of the project.) After looking at Raspbian, I was beginning to doubt that the RPi Zero was capable of outsmarting a 20MIPs processor, but thanks to Ultibo, it looks quite attainable.

@pik33: I'm coming from the Parallax Propeller. Doing 8 serial ports is a piece of cake for that processor (probably because I'm using some genius's 4-port library that only needs one core...and the Propeller has 8 cores). However, it falls severely short in the memory department--which is where the RPi is light years ahead.

@Ultibo: Thanks a lot for your response. I'll start my baby steps, but at least I have the assurance that it should be possible. I should also be able to add an I2C RA8875 driver to the Ultibo Console interface library. (That could be a little tricky, as I don't have an HDMI display to debug things with ;-) )
User avatar
Ultibo
Site Admin
Posts: 2303
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Bit-banged 8-port RS-232 communication library

Postby Ultibo » Sun Aug 11, 2019 11:56 pm

ZeroHobby wrote:All that to say...RS-232 was designed as a long-range communications protocol, and at least in my case, it's what I need to use.

Fair enough, just asking the question to be sure you have considered all options.

ZeroHobby wrote:Can you share a link to the Pi Zero's register map?

The official register documentation is in the BCM2835 ARM Peripherals pdf and the majority of the values and structures have been defined in the BCM2835 unit of Ultibo.

There are also some other useful links on the Resources page of the Ultibo wiki.

Ultibo wrote:The register organisation of the Pi means you could read all available pins (54 of them) in 3 32bit reads, if you select the pins that you use carefully you might be able to read all 8 inputs in one or two reads. A similar situation applies to writing to the GPIO outputs.

I'll adjust my earlier statement, you can read or write all available pins on the Pi in 2 32bit reads or writes because the third GPIO bank is never implemented on the Pi. So with careful selection you might be able to use a single read and write to query and update all 8 of your connections.

ZeroHobby wrote:One slight issue that might come up is if the FIQ happens to land in the middle of an R-M-W cycle in Ultibo core...? Conceivably, that could cause Ultibo to overwrite the values just written via the FIQ RS232 service routine. True? False? Thoughts? (Saving context during interrupts was one of those things I learned early on with PICs...because I can guarantee you that if an interrupt potentially COULD mess something up, it eventually WILL.)

Ultibo is a preemptive environment so the core interrupt dispatching code always saves and restores the current context.

However because it is a preemptive environment you must ensure that you take steps to protect access to data when doing read/modify/write operations so that data consistency is maintained.

There are two available options, you can use a spinlock around the read/modify/write in order to prevent the interrupt from occurring until the operation is completed or you can use atomic operations to ensure that changes are made in a guaranteed sequence without risk of accidental overwrite.

The spinlock is the simplest method but in some cases atomic operations (while more complex) can give higher performance.

For standard spinlocks look at the SpinLockIRQFIQ / SpinUnlockIRQFIQ functions in the Threads unit.

For atomic operations look at the InterlockedIncrement / InterlockedDecrement / InterlockedExchange / InterlockedAddExchange / InterlockedCompareExchange functions in the Platform unit or see the FreePascal documentation for the equivalent functions in the FPC RTL.

ZeroHobby wrote:I should also be able to add an I2C RA8875 driver to the Ultibo Console interface library. (That could be a little tricky, as I don't have an HDMI display to debug things with ;-) )

We recommend using the on board UART to output debug information when working in a headless environment, it is initialized early in the boot so it can output messages long before they would appear on screen anyway.
Ultibo.org | Make something amazing
https://ultibo.org
stevenpostma
Posts: 11
Joined: Thu Jan 18, 2018 5:44 pm

Re: Bit-banged 8-port RS-232 communication library

Postby stevenpostma » Thu Aug 15, 2019 4:38 pm

Ultibo supports USB devices quite well,
so I would recommend using two USB to Quad UARTs chips like
FTDI FT4232H , EXAR XR21B1424 or MOSCHIP MCS7840 .

----

By the way, I made triple software serial ports in MPASM assembly for Microchip PIC16F1xxx devices.
It uses the PortChange interrupt to detect start bits for the receiver part, and upon detected start bit,
edge detect interrupt is disabled, and an 8-bit timer is started with initial delay of 1.5 bit times.
From then on, the timer interrupt is used to clock in next databit.
After receiving the stop bit, the edge detect interrupt is enabled again to wait for next start bit.
So the hardware used is 3x 8-bit timers for receiving data, and 1x 8-bit imer for transmitting of all three ports.
A PIC on 32 MHz can handle 3x software UART @ 9600 Baud + 2x hardware UART @ 115200 Baud full duplex on all channels.
ZeroHobby
Posts: 7
Joined: Fri Aug 09, 2019 2:11 pm

Re: Bit-banged 8-port RS-232 communication library

Postby ZeroHobby » Thu Aug 15, 2019 4:50 pm

That is a very tempting suggestion, though as the Pi Zero only has one USB port (...maybe someone else will set me straight and say it has two...), I'd need a USB hub. ('Course, then I really wouldn't need Ultibo, as Raspbian could handle all the serial ports.) Found some commercial USB to 8-port serial hubs, though they're a little pricey for my purposes. Thanks for the suggestion, though.

I haven't actually gotten to delving into the Ultibo portion of the project--other things have come up, but I think that I'll experiment with bit-banging first, to see whether it's actually feasible. After all, I already have the RS-232 ports, and the SP3232 level-shifting drivers.
stevenpostma
Posts: 11
Joined: Thu Jan 18, 2018 5:44 pm

Re: Bit-banged 8-port RS-232 communication library

Postby stevenpostma » Thu Aug 15, 2019 5:17 pm

stevenpostma
Posts: 11
Joined: Thu Jan 18, 2018 5:44 pm

Re: Bit-banged 8-port RS-232 communication library

Postby stevenpostma » Thu Aug 15, 2019 5:22 pm

not that it will help you much; but I found another USB to quad UART chip:
Silicon Labs CP2018.
stevenpostma
Posts: 11
Joined: Thu Jan 18, 2018 5:44 pm

Re: Bit-banged 8-port RS-232 communication library

Postby stevenpostma » Thu Aug 15, 2019 5:38 pm

the cheapest USB to 8x RS232 box I can find is $99:
https://www.usbgear.com/GM-U28RS232.html

----

There is still another way to add 4 serial ports to a rPi; i.e. via SPI:
https://www.maxlinear.com/product/inter ... s/xr28v384

You can add multiple devices to one SPI bus if you have a separate chip select line
for each SPI slave device, so adding xr28v384 chips will cost you 5 pins:
3 pins for SPI (SCK, SDI, SDO), and 2 pins for chip select (CS1, CS2), see
https://www.analog.com/en/analog-dialog ... rface.html

With greetings to you,
Steven

Return to “General”

Who is online

Users browsing this forum: No registered users and 34 guests