'Login stream was closed.' EMA error message during batch request from RTO

Our client is subscribing to 12k items from Refinitiv Real-Time Optimized (RTO) datafeed using EMA C++. They are running into the following issue:

We saw the following status messages for all the subscribed rics on 06/14 around 04:16 NYT and after these messages we did not get any more update/refresh messages till very later in the day.

2021-06-15T08:16:01.145Z INFO OmmClient::onStatusMsg grs_ommclient.cpp:110 Item Name : USDDKKSWFIXMP=WM Item State : Closed / Suspect / None / 'Login stream was closed.'
2021-06-15T08:16:01.145Z INFO OmmClient::onStatusMsg grs_ommclient.cpp:110 Item Name : USDDKKSWFIXP=WM Item State : Closed / Suspect / None / 'Login stream was closed.'
2021-06-15T08:16:01.145Z INFO OmmClient::onStatusMsg grs_ommclient.cpp:110 Item Name : USDDKKTNFIXMP=WM Item State : Closed / Suspect / None / 'Login stream was closed.'
2021-06-15T08:16:01.145Z INFO OmmClient::onStatusMsg grs_ommclient.cpp:110 Item Name : USDDKKTNFIXP=WM Item State : Closed / Suspect / None / 'Login stream was closed.'
2021-06-15T08:16:01.145Z INFO OmmClient::onStatusMsg grs_ommclient.cpp:110 Item Name : USDEEK1MFIXMP=WM Item State : Closed / Suspect / None / 'Login stream was closed.'

We had to restart our application to start processing messages from Refinitiv again.

Questions

What might be the reason/scenario for these status messages ?
What is the best way to recover in case the application gets into such a state ? Should the application try to relogin ?
Is there a particular time during the day when the refinitiv service is unavailable ?

Find more posts tagged with

elektron

elektron-sdk

refinitiv-realtime

rrt

ema-api

batch-request

Accepted answers

wasin.w

Hello @psrivastav18

The given log and status messages are mostly identical to this thread which indicates that the server-side disconnected your application.

Regarding the "we are bringing two services simultaneously that connect to refinitiv using the same credentials. ", do you mean you are using one Machine-ID credential for two sessions (or two instances)? If so, that might be the reason for the "TREP Authentication token has expired." error which causes the OMM Login stream to disconnected.

I suggest you use a unique Machine-ID for each instance (please contact your Refinitiv Representative/Account Manager on this).

If the same disconnection issue is still occurring (with one Machine-ID/one session or two Machine-IDs for two sessions), you may need to contact the RTO Support to investigate on the server-side by submitting a ticket via https://my.refinitiv.com/content/mytr/en/productsupport.html website, and select "Product" Refinitiv Real-Time - Optimized (please give the Machine-ID and time detail in the description too).

All comments

wasin.w

Hello @dumitru.arama

The "Login Stream was Closed" error message with "Closed / Suspect" status means the login steam is closed, so this item stream is closed too and the API will not recover the subscription for the application.

The issue can occur for various reasons such as the disconnection between API and server, the slow consumer, etc. Are there any error messages about the disconnection or login stream?

umer.nalla

Hi @dumitru.arama

Since the application was logged out from the server, so all open item streams were also closed off.

Is the application using

the session management feature to allow the API to select an endpoint in their region (example 113_MP_SessionMgmt),
or specific RTO endpoint(s) in their config
or using the Service Discovery option to select an endpoint (demonstrated in example 450_MP_QueryServiceDiscovery)

Are they using ChannelSets to specify multiple endpoints or locations in their config?

See section 'Using ChannelSets with Refinitiv Real-Time Optimized Cloud-based Services' in the following article Enterprise Message API (EMA) - Configuration Overview | Refinitiv Developers

psrivastav18

Thank you for looking into this issue, We saw this issue again yesterday evening 15:57 NYT.
Do you know why was the application logged off from the server ? Would be great if. you can put some scenarios in which the application can be logged off from the server, and how can the application best recover from such a state ? Also does the API automatically reconnect back in case of a TCP DISCONNECTIONs ?

@wasin.w
I did not see any disconnect logs ,I saw the following log

@umer.nalla

Please find below the answer to your questions( answers start with a ->)

the session management feature to allow the API to select an endpoint in their region (example 113_MP_SessionMgmt),

-> We are closely following example 113 from the EMA examples, and use "Consumer_4" from the EMAConfig.xml as consumer type. My understanding based on EMA C++ guide section 6.7 is setting the consumerType to "Consumer_4" in OmmConsumerConfig enables the session management use case. I expect that only by setting consumer type in OmmConsmerConfig object to "Consumer_4" the API should take care of the following by default

• Obtains a token (according to the details in Section 6.3.2)
• Queries service discovery (according to the details in Section 6.4)
• Consumes market data (according to the details in Section 6.5)
• Manages login reissues when needed on a cyclical basis (according to the details in Section 6.3.3
Reconnects back on Disconnections

or specific RTO endpoint(s) in their config
-> We don't connect to any specific end point, but use "Channel_4" configuration from EMAConfig.xml

or using the Service Discovery option to select an endpoint (demonstrated in example 450_MP_QueryServiceDiscovery)
-> My understanding is setting OmmConsumerConfig.channelType to "Channel_4" enables session management which takes care of service discovery.

Please let us know if you need anything more from our side for carrying out your investigations.
Thanks again for helping out.

wasin.w

Hello @psrivastav18

Thank you for the information. Based on the given error messages, the "TREP authentication token has expired." means the RDP authentication between the client and RTO Real-Time streaming server was expired, so the Login stream was closed.

Basically, the EMA automatically takes care of refreshing the Access Token with RDP Auth Service (via HTTP REST) before the session expired.

What is the version of RTSDK/EMA C+++ that you are using? I suggest you test with the latest version of the SDK. The latest version of the SDK is 2.0.1.L2 (EMA 6.3.1L2) You can download the SDK from the following links:

The other way is to enable the REST log messages between the API and RDP Auth service to verify the API behavior and RDP session. You can configure it via the following parameters in EmaConfig.xml file. Please remove your credentials before sharing the log file.

psrivastav18

Thankyou @wasin.w for the response,
Yes we are using the latest version of the API for linux ( which is Real-Time-SDK-2.0.1.L2.linux)
I have enabled the rest logs and would let you know when it happens again. Please note it has happened 3-4 times in the past week.

Question,
The OmmConsumer stopping to retry to connect to tokenservice after 2 timeouts is something concerning for us. We would want to recover from this state automatically if possible.

Trying to understand what are the options available for the client to recover automatically from this state.

I see the following three options, Please comment if they make sense, or if there is a better way of handling this.

-> Would increasing the RestRequestTimeout from 90 to a higher number help ?
-> Can the client resubscribe for the securities for which the stream was closed ? To be more specific invoke call OmmConsumer::registerClient() for expired securities on the existing OmmConsumer instance ?
-> Should the client create a new instance of OmmConsumer and resubscribe for all the securities ?

wasin.w

Hello @psrivastav18

You should try to increase the value of RestRequestTimeout first. Please note that there is no "golden number", so you may need to adjust the value, test, and then repeat the adjust-test steps to find the number that matches your environment.

The other parameter that may help you is the TokenReissueRation.

Regarding the subscription, you need to re-initialize the connection with the EmaFactory.createOmmConsumer() statement before re-subscribe items with registerClient() function.

psrivastav18

Thank you @wasin.w I will try to tweak those parameters.

We again encountered some errors and this time we saw a different message ' Recoverable / Suspect / None / 'Service for this item was lost.' This looks like a different issue to me, and was resulted from a network issue.

Below is the log

Wanted to bring to your notice that we are bringing two services simultaneously that connect to refinitiv using the same credentials. Do you think that might be the reason for this behaviour ?

Also how are the disconnects handled in general by the API ?

wasin.w

Hello @psrivastav18

The given log and status messages are mostly identical to this thread which indicates that the server-side disconnected your application.

I suggest you use a unique Machine-ID for each instance (please contact your Refinitiv Representative/Account Manager on this).

EXPLORE OUR SITES