Problem using RTS/CTS with USB serial

Discussion and questions about programming with Ultibo.
sbk58
Posts: 25
Joined: Fri Jul 21, 2017 7:48 pm
Location: Canada

Problem using RTS/CTS with USB serial

Postby sbk58 » Sat Mar 02, 2019 1:02 am

I have been unable to get the RTS/CTS handshake signals to work on both FTDI and PL2303 serial drivers.

On the PL2303 the Ultibo driver correctly reads its CTS signal and limits data transmission correctly. However the RTS signal does not handshake correctly to limit the data flow on the receive side.

The FTDI Ultibo driver does not respond to its CTS signal and will continue to transmit data when the connected device deactivates the CTS line. The RTS signal stays inactive preventing the connected device from sending data to the Ultibo device.

Has anyone else come across this issue?
User avatar
Ultibo
Site Admin
Posts: 2208
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Problem using RTS/CTS with USB serial

Postby Ultibo » Sat Mar 02, 2019 10:47 pm

sbk58 wrote:I have been unable to get the RTS/CTS handshake signals to work on both FTDI and PL2303 serial drivers.

Both of our FTDI and PL2303 devices only have RX and TX lines exposed so we cannot test RTS/CTS but if you have hardware that supports it then it should be easy enough to resolve whatever is stopping it from working.

Do you have a simple test case available or can we create one based off something like the Ultibo serial example?

Our first priority will be to check that the devices respond correctly when setting the various options and that no errors are reported. After that we can get you to try some experiments with the different options to compare results and see if we can work out what is missing.
Ultibo.org | Make something amazing
https://ultibo.org
sbk58
Posts: 25
Joined: Fri Jul 21, 2017 7:48 pm
Location: Canada

Re: Problem using RTS/CTS with USB serial

Postby sbk58 » Sat Mar 02, 2019 11:33 pm

Hi thanks for quick response.

I am capable of testing any changes you want to try to either the FTDI an PL2303 USB drivers. I have a meter and oscilloscope so electrical measurements are not a problem.

I can easily reproduce the problem with my current project so software testing is also not a problem.

Let me know what you would like to try...
User avatar
Ultibo
Site Admin
Posts: 2208
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Problem using RTS/CTS with USB serial

Postby Ultibo » Mon Mar 04, 2019 10:41 pm

sbk58 wrote:Let me know what you would like to try...

Just reviewing both drivers to see if there is anything specific about the handling of RTS/CTS etc, there are a lot of supported devices for both PL2303 and FTDI some of which exhibit "unusual" behavior so if you could tell us the device ID (vendorid : productid pair) for your devices that will help to narrow which cases we are looking at.

You can normally obtain these easily from lsusb in Linux but there are also other options if you don't have a Linux installation available.

Thanks.
Ultibo.org | Make something amazing
https://ultibo.org
sbk58
Posts: 25
Joined: Fri Jul 21, 2017 7:48 pm
Location: Canada

Re: Problem using RTS/CTS with USB serial

Postby sbk58 » Tue Mar 05, 2019 3:10 pm

I have 3 PL2303 cables and 5 FTDI devices I have attempted to use with Ultibo.

All of my PL2303 cables:
Vendor: 067b
Product: 2303

Various FTDI devices:
Vendor: 0403
Product: 6001

I used lsusb under Ubuntu 16.04 x64 system to read vendor and product Ids.
sbk58
Posts: 25
Joined: Fri Jul 21, 2017 7:48 pm
Location: Canada

Re: Problem using RTS/CTS with USB serial

Postby sbk58 » Mon Apr 01, 2019 12:32 am

Finally got a chance this weekend to get back to this issue.

I decided to start with the FTDI devices as I have a number of peripherals (gps, xbee, propeller boards...) that use the FTDI devices.

I used wireshark to capture the USB trafic from an FTDI FT232R device on my Ubuntu system. When opening the FTDI serial device the Ubuntu driver always cleared the hardware flow before setting it. It also set the RTS/DTR modem control signals to high before activating hardware flow control. After I changed the Ultibo FTDI driver to match the Ubuntu driver the RTS pin now goes low when RTS/CTS handshaking is enabled and Ultibo respects the CTS input when sending data.

Unfortunetly this did not fix the problem of the FTDI properly signaling the state of its receive control line (RTS) to control the received data rate. Now both the FTDI and PL2303 USB Ultibo drivers have the same issue causing received data to be lost. I used a scope to monitor the RTS signal on the FTDI and it is randomly being set high but not at a rate nearly as fast as you would expect to see. Also at times the RTS line is held high for 30 to 60 milli-seconds.

When I monitor the RTS signal of a Pi running Raspbarian with my Ubuntu system sending high speed data I see continuous short (10..40 micro second) signaling of the RTS line to the inactive state on the scope.

When I use the raspi internal UART on Ultibo setting RTC/CTS flow control works and I see continuous short (7..20 micro second) signaling of the RTS line on my scope. I can set my baud rate to 460800 and not lose a byte in either direction. At this point I do not believe that my code controling the serial device under Ultibo or that the Ultibo serial device wrapper is the issue.

Under Ultibo both the FTDI and PL2303 drivers have the same receive data over run issue. As there is some activity on the RTS line I think that both USB devices control the RTS line in handshake flow mode. If that is the case I am at a loss to explain what would cause the data receive over runs.

If anyone has any ideas please chime in. I have included my patch for the FTDI driver which should be incorperated (in some form) into the FTDI driver.


This code replaces code in function FTDISerialDeviceOpen code starting at line 2510 to line 2533:

Code: Select all

  {ready flow control params}
  Flow:=0;
  {Check Force RTS-CTS}
  if PFTDISerialDevice(Serial).ForceRTSCTS then begin
    FlowControl:=SERIAL_FLOW_RTS_CTS;
  end;
  {Flow Control}
  case FlowControl of
    SERIAL_FLOW_NONE:Flow:=Flow or FTDISERIAL_SET_FLOW_CTRL_NONE;
    SERIAL_FLOW_RTS_CTS:Flow:=Flow or FTDISERIAL_SET_FLOW_CTRL_RTS_CTS;
    SERIAL_FLOW_DSR_DTR:Flow:=Flow or FTDISERIAL_SET_FLOW_CTRL_DTR_DSR;
  end;

  {configure hardware flow control}
  {first clear flow settings}
  IoStatus:= FTDISerialSetFlowControl(PFTDISerialDevice(Serial), FTDISERIAL_SET_FLOW_CTRL_NONE);
  if SERIAL_LOG_ENABLED and (IoStatus <> ERROR_SUCCESS) then
    SerialLogError(Serial,'FTDI Serial: Set flow control off failed');
  if IoStatus = ERROR_SUCCESS then begin
    {Set modem control to RTS and DTR high}
    IoStatus:= FTDISerialSetModemControl(PFTDISerialDevice(Serial), FTDISERIAL_SET_MODEM_CTRL_RTS_HIGH or FTDISERIAL_SET_MODEM_CTRL_DTR_HIGH);
    if SERIAL_LOG_ENABLED and (IoStatus <> ERROR_SUCCESS) then
      SerialLogError(Serial,'FTDI Serial: Set modem control RTS/DTR failed');
    if (IoStatus = ERROR_SUCCESS) and (Flow <> FTDISERIAL_SET_FLOW_CTRL_NONE) then begin
      {Enable Flow Control if required}
      IoStatus:= FTDISerialSetFlowControl(PFTDISerialDevice(Serial),Flow);
      if SERIAL_LOG_ENABLED and (IoStatus <> ERROR_SUCCESS) then
         SerialLogError(Serial,'FTDI Serial: Set flow control failed');
    end;
  end;
  if IoStatus <> ERROR_SUCCESS then begin
    Result:= ERROR_OPERATION_FAILED;
    Exit;
  end;
 
User avatar
Ultibo
Site Admin
Posts: 2208
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Problem using RTS/CTS with USB serial

Postby Ultibo » Mon Apr 01, 2019 11:32 pm

sbk58 wrote:After I changed the Ultibo FTDI driver to match the Ubuntu driver the RTS pin now goes low when RTS/CTS handshaking is enabled and Ultibo respects the CTS input when sending data.

This is excellent work, we've been struggling to get a clear picture of what the Linux driver is doing with the RTS/CTS handshaking and the fact that no official datasheet of the registers seems to be available isn't helping.

sbk58 wrote:Under Ultibo both the FTDI and PL2303 drivers have the same receive data over run issue. As there is some activity on the RTS line I think that both USB devices control the RTS line in handshake flow mode. If that is the case I am at a loss to explain what would cause the data receive over runs.

As best we can figure there are two possibilities, either we need to assert and deassert the RTS line as our internal receive buffer fills and empties or we need to stop and start the submission of receive requests to the chip and let it assert and deassert the RTS line based on its own buffer state.

We need to do a little more investigation but we'll also apply your changes and so that the behavior of both devices is consistent.
Ultibo.org | Make something amazing
https://ultibo.org
sbk58
Posts: 25
Joined: Fri Jul 21, 2017 7:48 pm
Location: Canada

Re: Problem using RTS/CTS with USB serial

Postby sbk58 » Wed Apr 03, 2019 1:13 pm

The only way the driver can prevent the RTS signal from going to the inactive state is by reading the USB serial device data at a fast enough rate. I took a look at the FTDI driver code in specific the receive worker thread. The worker thread does in fact toss any USB request data that will not fit in to the Ultibo receive buffer when the packet is received. Instead it must wait for space in the receive buffer and copy the entire USB request packet data to the receive buffer before requesting a new read request from the USB serial device. I am not sure exactly how to handle this. I see three main issues;

1) The modem and device status flags are sent as the first two bytes of the received data from the FTDI serial device. If the Ultibo receive buffer becomes full and the application does not read data from the serial device you would no longer receive modem and device status.

2) There would need to be a mechanism to shutdown the worker thread in the event the serial device is closed or ejected while waiting for the receive buffer to empty enough to copy any remaining data in the USB request packet.

3) How to wait for space in the Ultibo receive buffer without blocking the application from accessing the receive buffer (ie. not holding the lock or acquiring it to often).
User avatar
Ultibo
Site Admin
Posts: 2208
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Problem using RTS/CTS with USB serial

Postby Ultibo » Wed Apr 03, 2019 11:33 pm

sbk58 wrote:The only way the driver can prevent the RTS signal from going to the inactive state is by reading the USB serial device data at a fast enough rate. I took a look at the FTDI driver code in specific the receive worker thread. The worker thread does in fact toss any USB request data that will not fit in to the Ultibo receive buffer when the packet is received. Instead it must wait for space in the receive buffer and copy the entire USB request packet data to the receive buffer before requesting a new read request from the USB serial device.

That's exactly what I see, the default behavior of the driver is to discard data when the buffer is full.

On the other hand the UART driver for the Pi stops reading data from the FIFO when the Ultibo buffer is full, which causes the UART to block receiving from the remote end if CTS/RTS is enabled (or to discard incoming data if it is not).

sbk58 wrote:1) The modem and device status flags are sent as the first two bytes of the received data from the FTDI serial device. If the Ultibo receive buffer becomes full and the application does not read data from the serial device you would no longer receive modem and device status.

This is probably not a major issue, if an application isn't bothering to read the received data from the serial device then it probably isn't that interested in the device status either.

sbk58 wrote:2) There would need to be a mechanism to shutdown the worker thread in the event the serial device is closed or ejected while waiting for the receive buffer to empty enough to copy any remaining data in the USB request packet.

No, the worker thread wouldn't do waiting. It's a general principle of the design that the worker threads aren't used for indefinite waiting operations, there are better ways to do that.

Instead the worker thread processing PL2303ReceiveWorker or FTDISerialReceiveWorker would need to determine if there is sufficient buffer available to receive another USB request from the device, if there is it would submit the request again, if not it would leave the request idle and return.

I should note, out of interest, that signaling another thread to shutdown (or stop doing something) is quite easily achieved in Ultibo. Unlike the Linux design of locks and semaphores where destroying the object while threads are waiting for it is considered unsafe (and the results undefined) in Ultibo you can destroy a lock, semaphore or event while threads are waiting and they will be released gracefully and return with an error. As long as the thread checks for the error before proceeding then nothing will go wrong.

sbk58 wrote:3) How to wait for space in the Ultibo receive buffer without blocking the application from accessing the receive buffer (ie. not holding the lock or acquiring it to often).

It is the caller (ie the application) that will do the waiting.

Once the buffer reaches the point where it cannot accommodate another USB request then the receiving will stop (and the device should deassert RTS), at that point nothing changes until the caller reads from the buffer and this action is what will trigger receiving to start again.

We have a very similar model in place for the transmit buffer of the PL2303 and FTDI so something like that for the receive should be all that is needed.

It actually is possibly simpler than it originally seemed, we'll have a look at doing a prototype to see how it works.
User avatar
Ultibo
Site Admin
Posts: 2208
Joined: Sat Dec 19, 2015 3:49 am
Location: Australia

Re: Problem using RTS/CTS with USB serial

Postby Ultibo » Wed Apr 10, 2019 11:57 pm

Ultibo wrote:It actually is possibly simpler than it originally seemed, we'll have a look at doing a prototype to see how it works.

Okay so we've created a prototype of a model that stops and starts the reception based on the availability of space in the Ultibo buffer, this has been applied to the PL2303 driver (because it was convenient to test that device) but it should be applicable to all of the USB to serial drivers with only minor adjustments.

If we post an updated version of the PL2303 driver containing these changes are you able to undertake some testing or do you need it to be the FTDI driver instead?
Ultibo.org | Make something amazing
https://ultibo.org

Return to “General”

Who is online

Users browsing this forum: No registered users and 1 guest