question

Upvotes
Accepted
Paolo Parlapiano avatar image
95 ●7 ●12 ●25

IProvider - Managing temporary unavailability of data

Hi,

please help me with this case.

Scenario: Interactive Provider providing data to downstream consumers.

For reasons external to the application, the Interactive Provider may not have yet the items requested by the consumers (e.g. in startup scenarios).

If in this timeframe requests are received, they are replied with a "Closed" status.

Note: the provider is aware of the structure of an item (i.e. the fields composing the RIC) only when this item is eventually known to it.


In this specific scenario, the subscribers are not able to recover from the "Closed" status, e.g. by re-trying the subscription at a later time.

As per my understanding, the user cannot leverage the "notInCacheData" feature of the ADS to mitigate the issue, as it applies to NonInteractive Providers only.


Is there any alternative / recommendation (including possibly changing the application logic) that could help in this scenario?


Thanks

Regards,


Paolo

ema-apirequestinteractive-provider
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

2 Answers

· Write an Answer
Upvotes
Accepted
Gurpreet avatar image
22.1k ●59 ●14 ●21

Hi @Paolo Parlapiano,

I think you already know the issue here - since its an interactive provider, the infrastructure (ADS) really doesn't play much role in terms of state management. A subscriber might directly connect to the provider, and it will respond with "closed" status since the item doesn't exist in its cache yet.

One way to mitigate this issue is to not start listening for connections (keep iprovider down) until the upstream is ready to provide data and the items universe is known.


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Paolo Parlapiano avatar image Paolo Parlapiano
Thanks, @Gurpreet,

but unfortunately in this case there's no event or condition signaling that the upstream data has been completely received, I'll need to find another way to handle the issue.

Regards,

Paolo

wasin.w avatar image wasin.w Paolo Parlapiano

Hello @Paolo Parlapiano

Is it possible to send the Open/Suspect status message like "Open / Suspect / None / 'Service not available" to the consumer while the upstream data is not ready?

Paolo Parlapiano avatar image Paolo Parlapiano wasin.w

Hi @wasin.w,

are you suggesting replying with a status message to the request?

Is it fine?

I may be wrong, but I remember to have read somewhere that the recommendation is always to reply with a refreshMessage to a request?

In this case, I would not know the set of fields of the item, so either I implement something like the "notInCacheData" ADS feature, or I provide an empty fieldSet? Does it make sense?

Thanks

Regards,

Paolo

Upvotes
wasin.w avatar image
24.7k ●54 ●17 ●14

Hello @Paolo Parlapiano

About replying with the status message while the upstream data is not ready, I got the idea from the ADS behavior that returns the "Open / Suspect / None / 'Service not available" status message to the consumer if the service is not available (down or not exist). Once the service is up, the API on the consumer side automatic recovery subscription for the consumer application.

Example with IProvider example 100 --> ADS --> EMA Consumer scenario:

< -- IProvide is not start -->
StatusMsg
    streamId="5"
    domain="MarketPrice Domain"
    state="Open / Suspect / None / 'Service not available'"
    name="USDSWFSR="
    serviceName="DIRECT_FEED"
StatusMsgEnd
< -- IProvide start -->
RefreshMsg
    streamId="5"
    domain="MarketPrice Domain"
    solicited
    RefreshComplete
    state="Open / Ok / None / '*Refresh Completed'"
    itemGroup="00 01"
    name="USDSWFSR="
    serviceId="10004"
    serviceName="DIRECT_FEED"
    Payload dataType="FieldList"
        FieldList FieldListNum="0" DictionaryId="0"
            FieldEntry fid="22" name="BID" dataType="Real" value="39.90"
            ....

I am not sure if it matches your scenario.

However, according to the ETA Java Development guide document, the Provider should send a StatusMsg to indicate the reason and close the stream if it cannot satisfy the request.

eta-provide-devguide.png

At last, I think it is based on your IProvider and Consumer requirements/agreements about how to handle each message scenario.



eta-provide-devguide.png (163.2 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Paolo Parlapiano avatar image Paolo Parlapiano

Hello @wasin.w,

thanks for the reply, but If I understood correctly, I'm afraid that this approach would not work on my side: in my scenario, some of the RICs may be available, and some others may not be, so replying with a status like "Service not available" will probably be incorrect/misleading here, especially if the requests come from the same consumer. I mean, If you imply changing the status of my service (so to leverage the auto-recovery mechanism later), I think I should not do it in this case..

In the meantime, I'm exploring the option of sending blank images, in a way similar to the Source Distributor, so by sending replies like "DSPLY_NAME, Not In Cache", since the consumers of my scenario work fine with this functionality.

A first PoC on this appears promising, and working as usual after the target item becomes eventually available..Anyway I have a question about this option...once the data is available, the provider sends a solicited refresh message to all the active clients, with the actual data: should I set the "clearCache" flag of the refresh message to true so that the consumers can remove any cached data of the blank image ("DSPLY_NAME, Not In Cache")?

Thanks

Best Regards,

Paolo

wasin.w avatar image wasin.w Paolo Parlapiano

Hello @Paolo Parlapiano

Thank you for the clarification.

According to the EMA API RDM Guide document, the application should clear the cache when performing a refresh as follows:

clearcache.png

However, I think this unsolicited refresh message is suitable for this "missing data is now available" scenario because it is a refresh message that updates data of the previous solicited refresh message (contains "not in cache").

refreshmsg-type.png

clearcache.png (116.1 KiB)
refreshmsg-type.png (110.9 KiB)
wasin.w avatar image wasin.w Paolo Parlapiano

Hi @Paolo Parlapiano

The first picture above comes from the EMA J RDM Usage Guide document.

The second picture comes from the Transport API J Developers Guide document.

If you are using RTSDK C/C++, the documents are available here.

Paolo Parlapiano avatar image Paolo Parlapiano wasin.w

Hi @wasin.w,

many thanks for the details.

I was thinking to send the refreshMessage with actual data with clearCache=true because, if the actual data does not carry any value for DSPLY_NAME, the final image on the consumer could still contain DSPLY_NAME="Not In Cache" after receiving this message, does it make sense as a reasoning?

Show more comments

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.