question

Upvotes
Accepted
kwunyu.wu avatar image
3 ●0 ●0 ●3

Getting out of buffer exception due to slow consumer (EMA C++ OMM API)

I'm building a interactive provider (EMA C++ OMM API) for streaming the market data to different downstream applications, which are Refinitiv Consumer, through TCP socket.

I'm testing the scenario that how my provider behaves when I have very slow consumer connect to it.

I found that the provider will throw an out of buffer exception when I submit the updateMsg.

I'm wondering if there are configurations or control throw that can allow the provider to detect and drop the slow consumer connection such that to avoid the out of buffer exception.


The following is the error message I found in the core dump:

"Internal error: rsslReactorSubmitMsg() failed in OmmIProviderImpl::submit( const UpdateMsg& ).
Error: 1009 ipcDataBuffer() failed, out of output buffers. The output buffer may need to be flushed.


#technologyema-apierrorc++commconsumerbufferinteractive-provider
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

2 Answers

· Write an Answer
Upvotes
Accepted
Jirapongse avatar image
79.8k ●257 ●52 ●74

@kwunyu.wu

Thanks for reaching out to us.

The provider applications are unable to detect that the connected consumer applications are slow consumers because the provider applications don't know how fast the consumer applications process the data. Moreover, typically, the consumer applications themselves don't know that they are slow consumers until the connections have been disconnected.

The application submits data to the API's buffers then the API frees the buffers by flushing data to the network. The out-of-buffer exception indicates that the API was unable to flush data to the network. The cause of the exception could be a network issue or a slow consumer.

I don't think that the EMA API has configurations or other methods to detect slow consumers. RTDS (ADS) also uses the buffer overflow condition to cut channels.

I hope that this information is of help.



icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

kwunyu.wu avatar image kwunyu.wu

Thanks for your comment.


I have one more question, assuming I have multiple consumers, one of them is being a slow consumer, if the buffer overflow exception raised, does this exception correspond to the slow consumer only? As you mentioned cutting the channels, do you mean catch the exception and calling `closeChannel()` method manually to close that particular connection?


Upvotes
Jirapongse avatar image
79.8k ●257 ●52 ●74

@kwunyu.wu

The out-of-output buffers exception just indicates that it was unable to get output buffers from the channel so it was able to encode the message to the buffer and send the data. One cause of this issue could be slow consumers but it could be from other network issues.

Typically, the server handles the buffer overflow condition by closing the channels that have this issue. In EMA C++, you can close the connection by using the following method.

refinitiv::ema::access::OmmProvider::closeChannel(UInt64 clientHandle)    
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.