Timeout in the RTO large batch subscription

Frederic

Sometimes we met the timeout in the RTO large batch subscription. Any suggestion / experience on subscriptions of large batches?

e.g., subscribe 2000 ~ 3000 RICs in a batch, continuously for 100K RICs.

The aim is to subscribe all those RICs without timeout while do the whole subscription procedure as fast as possible.

umer.nalla

Hi @Frederic

The usual recommendations are to pace the request i.e. limit how many requests you make per second / minute etc Either the RTO server you are connected to cannot cope with the pace of your request or your application cannot process the response quickly enough and the RTO server is disconnecting your application as the buffer it maintains for your connection overflows.

Also, if you are doing some processing in the OnRefreshMsg/onUpdateMsg callbacks, try and spin that off into a separate thread - these callbacks are executed in the main API thread context and if you spend too much time in the callback, the API cannot read the data from the RTO server in a timely manner. If you search this forum for 'slow consumer' you may find other useful information.

You can also try horizontal scaling i.e. create more than one OmmConsumer instance - where each instance subscribes to a subset of the overall watchlist.

Frederic

https://community.developers.refinitiv.com/discussion/comment/92362#Comment_92362

umer.nalla

Apr 14, 2022

Hi @Frederic

The usual recommendations are to pace the request i.e. limit how many requests you make per second / minute etc Either the RTO server you are connected to cannot cope with the pace of your request or your application cannot process the response quickly enough and the RTO server is disconnecting your application as the buffer it maintains for your connection overflows.

Also, if you are doing some processing in the OnRefreshMsg/onUpdateMsg callbacks, try and spin that off into a separate thread - these callbacks are executed in the main API thread context and if you spend too much time in the callback, the API cannot read the data from the RTO server in a timely manner. If you search this forum for 'slow consumer' you may find other useful information.

You can also try horizontal scaling i.e. create more than one OmmConsumer instance - where each instance subscribes to a subset of the overall watchlist.

@umer.nalla For RTO's horizontal scaling, if we call createOmmConsumer multiple times(each time with user name and password), will that cause a problem of multiple logins for the same machine id?

umer.nalla

Hi @Frederic

Sorry - but I have been out of office for a while.

Token sharing is feature of ETA Reactor – tokens can be shared if authentication mechanism and credentials match within a Reactor instance.

With EMA, each OMMConsumer is using its own Reactor. Therefore, is is not possible to share the token between OMMConsumers.

So, at present the customer would need to use unique MachineIDs for each OmmConsumer instance.

Frederic

https://community.developers.refinitiv.com/discussion/comment/93951#Comment_93951

Thank you Umer.