Regarding rsslReadEx return code

Siril

Needed some help with using the ETA library to connect to servers. We were facing issues with connection reset by peer and as per lseg we were not consuming faster enough. I believe I found the issue in the code were we are not consuming from the library properly. However with the change am not seeing the behavior as explained in the ETA specification. We are currently using rsslReadEx instead of rsslRead (however I believe other than the exception of having two extra parameter they both are the same. Please correct me if wrong).

As per below snippet from document, I see we are always getting retCode as 1 and so our process is now stuck at processing the updates from rsslReadEx continuously (we never get this RSSl_RET_SUCCESS).

From our logs as you can see below rsslRet (=retcode above) is always 1 which doesn’t explain what rsslCode it is from rsslRetCodes.h file (I see enums for all except 1). And as you can see the bytesRead and bytesDecompressed are both 0s. So I believe this buffer length (returned from rsslReadEx) is the data from the rssl library buffers? So is it expected to continuously receive this value of 1 for rsslRet code? Appreciate your help with this. Thanks a lot.

Jirapongse

@Siril

Thank you for reaching out to us.

The document indicates that if the return code is more than 0, this indicates that the rsslReadEx call was successful and there are remaining bytes in the input buffer. The I/O notification mechanism will not notify the user of these bytes. The function should be called again to ensure that the remaining bytes are processed.
be updated.

Yes, the buffer length (returned from rsslReadEx) is the data from the RSSL library buffers. The contents are only valid until the next call to rsslReadEx.

I think the buffers returned in the blue box are from the 5027 decompressed bytes. According to the output, this decompressed bytes (5027) contains 21 messages and the rsslReadEx returns a buffer that contains each message for each call.

You can refer to the Consumer example that demonstrates how to use this function.

Jirapongse

Are you decoding and processing the data on the same thread that calls the rsslReadEx? You may need to remove the decoding or processing logic to other threads.

You may count the number of update messages that the application can process per second.

Calling the rsslReadEx method from multiple threads can result in data being processed or returned out of order.

Siril

https://community.developers.refinitiv.com/discussion/comment/126068#Comment_126068

Thanks a lot for the response. Is there a way to find if we are still lagging in consuming updates from the rsslReadEx and if the input buffers are not getting consumed faster? Also is it possible for multiple threads to call rsslReadEx in parallel so we don't end up having the buffers fill up and causing disconnection. Appreciate your help. Thanks a lot.

Siril

https://community.developers.refinitiv.com/discussion/comment/126139#Comment_126139

Thanks again @Jirapongse . We are running all the rsslDecode* of the received data from rsslReadEx on the same thread. So should we move this rssl decoding to a different thread? However our specific logic for this decoded data runs on a different thread (binded to a different CPU). So currently I was able to improve some performance issues and not seeing this queuing of messages anymore. However it looks like we are running the rsslReadEx on a blocking mode (RsslConnectOptions::blocking set to RSSL_TRUE) and so now the rsslReadEx takes some time (<1ms) to return the data. And I see the CPU core assigned to this processing thread goes into sleep state most of the time (I believe rsslReadEx is waiting for the kernel to give the packet from network). Two questions.
1. Is there a way to run rsslReadEx in a busy wait loop so it doesn't go into a sleep (interruptible sleep state). Its already running on an isolated CPU core and can consume the core to its full extent. Currently I see the core is mostly idle (although we aren't queuing any data on TCP queues and consuming it immediately). Only worried if this is adding to any latency on the received data.
2. We use onload library to bypass kernel and get the packets quicker. Is there a way to enable onload on ETA sockets?

Jirapongse

The library can run with blocking and non-blocking mode. You can refer to the \Cpp-C\Eta\Applications\PerfTools\ConsPerf sample that demonstrates best practice and coding for performance.

You can also refer to the OPEN SOURCE PERFORMANCE TOOLS GUIDE in the package.

To process the raw packet, you need to know our RSSL specification which is our proprietary standard. Therefore, we don't support this kind of usage.