Getting out of buffer exception due to slow consumer (EMA C++ OMM API)

kwunyu.wu
kwunyu.wu Newcomer
in EMA

I'm building a interactive provider (EMA C++ OMM API) for streaming the market data to different downstream applications, which are Refinitiv Consumer, through TCP socket.

I'm testing the scenario that how my provider behaves when I have very slow consumer connect to it.

I found that the provider will throw an out of buffer exception when I submit the updateMsg.

I'm wondering if there are configurations or control throw that can allow the provider to detect and drop the slow consumer connection such that to avoid the out of buffer exception.


The following is the error message I found in the core dump:

"Internal error: rsslReactorSubmitMsg() failed in OmmIProviderImpl::submit( const UpdateMsg& ).
Error: 1009 ipcDataBuffer() failed, out of output buffers. The output buffer may need to be flushed.


Tagged:

Best Answer

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    Answer ✓

    @kwunyu.wu

    Thanks for reaching out to us.

    The provider applications are unable to detect that the connected consumer applications are slow consumers because the provider applications don't know how fast the consumer applications process the data. Moreover, typically, the consumer applications themselves don't know that they are slow consumers until the connections have been disconnected.

    The application submits data to the API's buffers then the API frees the buffers by flushing data to the network. The out-of-buffer exception indicates that the API was unable to flush data to the network. The cause of the exception could be a network issue or a slow consumer.

    I don't think that the EMA API has configurations or other methods to detect slow consumers. RTDS (ADS) also uses the buffer overflow condition to cut channels.

    I hope that this information is of help.



Answers

  • kwunyu.wu
    kwunyu.wu Newcomer
    https://community.developers.refinitiv.com/discussion/comment/104606#Comment_104606

    Thanks for your comment.


    I have one more question, assuming I have multiple consumers, one of them is being a slow consumer, if the buffer overflow exception raised, does this exception correspond to the slow consumer only? As you mentioned cutting the channels, do you mean catch the exception and calling `closeChannel()` method manually to close that particular connection?


  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @kwunyu.wu

    The out-of-output buffers exception just indicates that it was unable to get output buffers from the channel so it was able to encode the message to the buffer and send the data. One cause of this issue could be slow consumers but it could be from other network issues.

    Typically, the server handles the buffer overflow condition by closing the channels that have this issue. In EMA C++, you can close the connection by using the following method.

    refinitiv::ema::access::OmmProvider::closeChannel(UInt64 clientHandle)    

Categories