Posting the resolution message from case 13386108:

Our hosting team did not find any suspecting log on APP-ECR server logs. We see only login accept during the given time stamp. Attached the reference logs.
Advised to check with your network team to check their end, if there's any maintenance or network service restart.

Hello @mktdata

Question 1: We want to understand why this is happening as connection to TREP never went down.

Answer: Basically, the error message with "F<number> : <text>" pattern is generated from the ADH server, not the API or ADS server. It is possible that one of upstream server/component has gone down at the time and sent this message down the line. (e.g. one if the RTDS, 2 levels from application was down, Application > RTDS1> RTDS2> RTDS3(FAILED) > … > Exchange.)

I strongly suggest you contact your market data team or the team that manages your infra.

Question 2: Also In what all conditions API will force close the active subscription on valid symbols that was streaming the data ?

Answer: If the application receives the Status message with StreamState = Closed, it means the item stream is already closed and the API does not recovery item subscription for the application. The application must re-request that item manually.

Question 3: Is application guaranteed to receive { DataState = Suspect; StreamState = Open } on every active subscription when connection with TREP goes down or when service goes down ?

Answer: If the application receives a status message with DataState = Suspect; StreamState = Open state, the API automatically recovers the item subscription for the client.

Please note that this behavior is also depending on the SingleOpen and AllowSuspectData attributes from the ADS's Login Refresh message too. Please find more detail on section "3.4.8 Single Open and Allow Suspect Data" of the RDM Usage Guide document (available in the RFA C++ package).

Question 4: What happens when new subscriptions are made to TREP when connection with TREP is down? Are they accepted? Does application get Suspect DataState on those subscriptions?

Answer: If the connection between the RFA API and RTDS is down, the RFA will hold the request message until the RTDS become available. The application will receive Open/Suspect/None with text “Source unavailable... will recover when source is up.” as a response.

Hi @wasin.w When subscription is force closed by API for valid symbol that was previously streaming data, can application immediately try to resubscribe?

Hello @mktdata

According to on the RFA C++ Development Guide section 7.4.2.1 "Stream and Data States: Consumer Point of View", it indicates that the SteamState: Closed with DataState OK or Suspect means "Data is not available on this service and connection and is not likely to become available."

On section 7.4.2.3 "State Code Definitions" of the same document, it describes the StatusCode : NotFound meaning as "Indicates that requested information was not found, though it might become available later or by changing some of the request’s parameters."

Normally, it is advisable to check the StatusText attribute to verify what action an application should do next such as:

• log the message to end user
• report the message to the Market Data team
• or even re-subscription that RIC manually after waiting for a while

However, there is no status text in your response Closed Status Message, so I suggest you not re-subscribe the item immediately, and check with your Market Data team to find the rootcause or issue on the backend side.

We have some additional questions -->

1. In what all situations TREP will close the subscriptions ? We know below events[actions] that can lead to TREP closing the subscription but are there any other events that can also lead to TREP[or api] closing the subscription? Also, we assume that only in case of event <6> below, consumer application should recover[resubscribe] ? is that correct assumption?

   1. User requested closing of subscription

   2. Subscription on invalid RIC

   3. Snapshot request [NOT Streaming request]

   4. Server user not entitled for subscribed symbol.

   5. RIC change while there is active subscription on that RIC. RIC change will cause api to close the subscription. We should not recover such subscriptions but check with @Satees Pasupulati

   6. Upstream source going down. At the moment this is the only event for which we should recover the subscription.

2. When upstream source goes down api gives stale data alert but then why it closes the subscription after some time

Hello @mktdata

Question 1: In what all situations TREP will close the subscriptions ? We know below events[actions] that can lead to TREP closing the subscription but are there any other events that can also lead to TREP[or api] closing the subscription? Also, we assume that only in case of event <6> below, consumer application should recover[resubscribe] ? is that correct assumption?

Answer: The other situation that leads the API to close the subscription is the request timeout scenario.

To get the full detail of scenarios/events that lead the RTDS or API close the subscription, I strongly suggest you contact the RTDS and Real-Time APIs (RDC) support team via https://my.refinitiv.com/content/mytr/en/productsupport.html website.

Please note that an application should rely on the Status Message DataState, StreamState, StatusCode, and StatusText to consider which action should do next.

Question 2: When upstream source goes down api gives stale data alert but then why it closes the subscription after some time.

Answer: I strongly suggest you contact the RTDS support team or Real-Time APIs support team to verify this behavior in detail.

@wasin.w One last thing , will consumer client resubscribing immediately after subscription is closed due to "request timeout scenario" work? or consumer client is not expected to recover this subscription.