How to use and manage service id to service name associations in ETA API.

I'm using Elektron ETA API in Unix.

I
would like to keep a map of EMS service names to serviceId retrieved from source
directory response (sourceDirectoryResponseInfo_[i].ServiceId) to be used in
making request. This will be done every time the client connects/reconnects to
the server. Is that a correct approach ?

I
see the serviceId available from the following fields:

msg.msgBase.msgKey.serviceId

msg.refreshMsg.msgBase.msgKey.serviceId

msg.updateMsg.msgBase.msgKey.serviceId

Based
on the value of msg.msgBase.msgClass (RSSL_MC_REFRESH and RSSL_MC_UPDATE), do I
need to use the serviceId from msg.refreshMsg.msgBase.msgKey.serviceId or msg.updateMsg.msgBase.msgKey.serviceId
? Can I use the service Id from msg.msgBase.msgKey.serviceId in the response
for both RSSL_MC_REFRESH and RSSL_MC_UPDATE messages ?

Find more posts tagged with

elektron

elektron-sdk

refinitiv-realtime

elektron-transport-api

eta-api

rrt

service-id

service-name

Accepted answers

Jirapongse

In my environment, after setting RSSL_RQMF_MSG_KEY_IN_UPDATES in the request, the update messages of MSFT.O contain the service ID in the message keys.

<requestMsg domainType="RSSL_DMT_MARKET_PRICE" streamId="6" containerType="RSSL_DT_NO_DATA" flags="0x4E (RSSL_RQMF_HAS_PRIORITY|RSSL_RQMF_STREAMING|RSSL_RQMF_MSG_KEY_IN_UPDATES|RSSL_RQMF_HAS_QOS)" qosDynamic="0" qosRate="1" qosTimeliness="1" priorityClass="1" priorityCount="1" dataSize="0">
    <key  flags="0x7 (RSSL_MKF_HAS_SERVICE_ID|RSSL_MKF_HAS_NAME|RSSL_MKF_HAS_NAME_TYPE)" serviceId="2115" name="MSFT.O" nameType="1"/>
    <dataBody>
    </dataBody>
</requestMsg>

<updateMsg domainType="RSSL_DMT_MARKET_PRICE" streamId="6" containerType="RSSL_DT_FIELD_LI
ST" flags="0x18 (RSSL_UPMF_HAS_MSG_KEY|RSSL_UPMF_HAS_SEQ_NUM)" updateType="1 (RDM_UPD_EVENT_TYPE_QUOTE)" seqNum="63744" dataSize="117">
    <key  flags="0x7 (RSSL_MKF_HAS_SERVICE_ID|RSSL_MKF_HAS_NAME|RSSL_MKF_HAS_NAME_TYPE)" serviceId="2115" name="MSFT.O" nameType="1"/>
    <dataBody>
        <fieldList flags="0x8 (RSSL_FLF_HAS_STANDARD_DATA)">
            <fieldEntry fieldId="22" data="0C22 6E"/>
            <fieldEntry fieldId="25" data="0C22 B6"/>
            <fieldEntry fieldId="30" data="0E05"/>

You can enable ETA tracing to verify the request and update message.

if (xmlTrace) 
		{
			int debugFlags = 0x2C0;
			RsslTraceOptions traceOptions;


			rsslClearTraceOptions(&traceOptions);
			traceOptions.traceMsgFileName = traceOutputFile;
			traceOptions.traceFlags |= RSSL_TRACE_TO_FILE_ENABLE | RSSL_TRACE_TO_MULTIPLE_FILES | RSSL_TRACE_TO_STDOUT | RSSL_TRACE_READ | RSSL_TRACE_WRITE;
			traceOptions.traceMsgMaxFileSize = 100000000;
			rsslIoctl(chnl, (RsslIoctlCodes)RSSL_TRACE, (void *)&traceOptions, error);
		}

Otherwise, you can run Consumer example in ETA package with -x option to verify the problem.

If the request messages of US RIC's contain RSSL_RQMF_MSG_KEY_IN_UPDATES but the update messages don't have the message keys, you can contact TREP support team to verify the problem.

All comments

nick.zincone

Hi @RAJ,

Yes, the correct approach for your use case is to keep a map of the names and IDs for the services retrieved.

Regarding whether you should use the details from the REFRESH or UPDATE, you will likely need to monitor both the REFRESH and UPDATE events. The REFRESH message contains the complete collection of services available for your connection and other important details such as capability and the state of each service at that point in time.

The UPDATE is strictly used when changes occur, most likely resulting from a service going down or coming back up. In such a case, the service ID is strictly used for you as a reference to which service was affected. However, the UPDATE is also used to indicate the addition or removal of services. In practice it's not too common for short-lived applications to encounter such events. In many cases, applications will never encounter these updates - this depends on the volatility of your environment you are connecting into. However, for long running applications, it is quite possible that a service may be added or removed. You will have to account for this by monitoring the updates.

RAJ

Will the service name to service Id association be different for connections to different ADS IP's and data centers ?

art.brzezinski1

RAJ,

service name to service id mappings are not guaranteed to be identical across ADS instances and/or data centers. You should map them for each individual connection when you receive source directory message.

RAJ

I expected the service id to come with every single data update. But, I see it only in the initial record. When I examined the following members in an update record, I see only "0".

msg.msgBase.msgKey.serviceId

msg.requestMsg.msgBase.msgKey.serviceId

msg.updateMsg.msgBase.msgKey.serviceId

How can I retrieve the service id from the update ?

If I can't, how do I know the service for which the update is received if I have subscription open for multiple services ?

nick.zincone

Hi RAJ,

Do you have XML Tracing turned on so we can see what output you are receiving for these Directory Updates?

Jirapongse

Refer to RDM usage guide, typically a message key in the update message
is optional. It is required if the application specifies the RSSL_RQMF_MSG_KEY_IN_UPDATES in the item request.

The following code is setting the RSSL_RQMF_MSG_KEY_IN_UPDATES flag in the request message.

msg.flags = RSSL_RQMF_HAS_QOS | RSSL_RQMF_STREAMING | RSSL_RQMF_HAS_PRIORITY | RSSL_RQMF_MSG_KEY_IN_UPDATES;

Moreover, before accessing the optional data, such as a message key, the application should check its presence by using flags. For example:

Checking the presence of a message key in the update message by using the RSSL_UPMF_HAS_MSG_KEY flag

  if (msg->updateMsg.flags & RSSL_UPMF_HAS_MSG_KEY){
              key = (RsslMsgKey *)rsslGetMsgKey(msg);
   }

Checking the presence of a service id in the message key by using the RSSL_MKF_HAS_SERVICE_ID flag

if (key && (key->flags & RSSL_MKF_HAS_SERVICE_ID))
{                
    printf("\nService Id: %d\n", key->serviceId);
}

If the application doesn't check the presence of optional data before accessing the data, the application can retrieve the invalid value and then behave unexpectedly.

For more information, please refer to the RDM Usage guide.

RAJ

Hi @Jirapongse

I made this change and can see serviceId in the update for the international RIC's, but not for the US RIC's like MSFT.O and IBM.N. For the US RIC, I see the serviceId only in Initial record, but not in the update. I see the same behavior with delayed feed as well.

If I need to forward this question to the TR team who had set up the environment for my consumption, please let me know.

----

General question regarding the reliability getting serviceId fields as part of the RsslMsg:

The need to check the availability of serviceId using RSSL_MKF_HAS_SERVICE_ID tells me that I can't use this field reliably to organize the data based on serviceId in order to process the update correctly. If that is the case, what would you recommend to get the serviceId reliably in the update ?

Do I need to cache the map of streamId to serviceId (received in initial record), to associate the streamId from the update to the previously provided serviceId in the initial record ?

art.brzezinski1

Hi RAJ,

As far as I understand, you should parse source directory payload to get the mapping of service id to service name. The source directory refresh message should be all you need as I don't believe you would ever see source directory update with different service id to service name associations, at least I'm not aware of any way we could do it without re-starting ADS and bringing down your connection.

Nick asked earlier if you can share some traces from your application where it receives source directory refresh and updates so that we can see what your application is seeing and make some recommendations. Is this something you can share?

As far as enabling service id on all market data updates, I'm not sure I understand why you'd want to or need to do it. When you make a request, you specify service id yourself so the refresh and updates to this request would all be from the service mapped to service id that you requested.

RAJ

Hi @artb

My question is not related to source directory update or refresh record.

My question is specific to getting serviceId with every single market date update. Earlier this week, @Jirapongse responded by saying that by setting a flag in the request, I can enable the server to send serviceId with every single market data update. As I organize my data cache by serviceId, if I receive the serviceId in the data update, I would know the service for which the update was received. So, if a particular service goes down on the ADS, all the RIC's from the downed service can be easily moved to another ADS.

If getting the serviceId from the update and using it to make the decision internally in the application is not the recommended (or reliable) approach, then I have to internally maintain the streamId to serviceId association at the time of making the request to ADS and use this information to figure out the serviceId based on the streamId if a service should go down.

art.brzezinski1

Hi @RAJ,

My apologies, I was going by the subject of this post but with your most recent update, I better understand what you're trying to achieve now. I just run a quick test right and I do see serviceId on updates for MSFT.O but let me run some more tests tomorrow after US market open and get some more stats.

Also note that in case service goes down, you should receive corresponding source directory update which may help you identify the service that went down. But I'm not sure how well this would work with the logic you want your application to have.

Jirapongse

In my environment, after setting RSSL_RQMF_MSG_KEY_IN_UPDATES in the request, the update messages of MSFT.O contain the service ID in the message keys.

<requestMsg domainType="RSSL_DMT_MARKET_PRICE" streamId="6" containerType="RSSL_DT_NO_DATA" flags="0x4E (RSSL_RQMF_HAS_PRIORITY|RSSL_RQMF_STREAMING|RSSL_RQMF_MSG_KEY_IN_UPDATES|RSSL_RQMF_HAS_QOS)" qosDynamic="0" qosRate="1" qosTimeliness="1" priorityClass="1" priorityCount="1" dataSize="0">
    <key  flags="0x7 (RSSL_MKF_HAS_SERVICE_ID|RSSL_MKF_HAS_NAME|RSSL_MKF_HAS_NAME_TYPE)" serviceId="2115" name="MSFT.O" nameType="1"/>
    <dataBody>
    </dataBody>
</requestMsg>

<updateMsg domainType="RSSL_DMT_MARKET_PRICE" streamId="6" containerType="RSSL_DT_FIELD_LI
ST" flags="0x18 (RSSL_UPMF_HAS_MSG_KEY|RSSL_UPMF_HAS_SEQ_NUM)" updateType="1 (RDM_UPD_EVENT_TYPE_QUOTE)" seqNum="63744" dataSize="117">
    <key  flags="0x7 (RSSL_MKF_HAS_SERVICE_ID|RSSL_MKF_HAS_NAME|RSSL_MKF_HAS_NAME_TYPE)" serviceId="2115" name="MSFT.O" nameType="1"/>
    <dataBody>
        <fieldList flags="0x8 (RSSL_FLF_HAS_STANDARD_DATA)">
            <fieldEntry fieldId="22" data="0C22 6E"/>
            <fieldEntry fieldId="25" data="0C22 B6"/>
            <fieldEntry fieldId="30" data="0E05"/>

You can enable ETA tracing to verify the request and update message.

if (xmlTrace) 
		{
			int debugFlags = 0x2C0;
			RsslTraceOptions traceOptions;


			rsslClearTraceOptions(&traceOptions);
			traceOptions.traceMsgFileName = traceOutputFile;
			traceOptions.traceFlags |= RSSL_TRACE_TO_FILE_ENABLE | RSSL_TRACE_TO_MULTIPLE_FILES | RSSL_TRACE_TO_STDOUT | RSSL_TRACE_READ | RSSL_TRACE_WRITE;
			traceOptions.traceMsgMaxFileSize = 100000000;
			rsslIoctl(chnl, (RsslIoctlCodes)RSSL_TRACE, (void *)&traceOptions, error);
		}

Otherwise, you can run Consumer example in ETA package with -x option to verify the problem.

If the request messages of US RIC's contain RSSL_RQMF_MSG_KEY_IN_UPDATES but the update messages don't have the message keys, you can contact TREP support team to verify the problem.

RAJ

I found there was an issue in the way I made the US equity request. I had a code path where the RSSL_RQMF_MSG_KEY_IN_UPDATES flag was not set. Now I'm able to get serviceId in the update for MSFT.O

Just to follow up on the remaining unanswered question in this thread.

1. The need to check the availability of serviceId using RSSL_MKF_HAS_SERVICE_ID tells me that I can't use this field reliably to organize the data based on serviceId in order to process the update correctly. Can I expect to receive serviceId reliably in the update if the flag RSSL_RQMF_MSG_KEY_IN_UPDATES is set.

2. Otherwise, would you recommend the client application to cache the map of streamId to serviceId (received in initial record), to associate the streamId from the update to the previously provided serviceId in the initial record ?

Jirapongse

TREP will follow standards defined in the RDM guide. Therefore, if the application connects to TREP, you can expect to receive serviceId reliably in the update if the flag RSSL_RQMF_MSG_KEY_IN_UPDATES is set.

However, the third party application may not follow this standard. Therefore, you can't expect the serviceId in the update when directly connecting to the third party application.

Yes, instead of using the flag RSSL_RQMF_MSG_KEY_IN_UPDATES, you can cache the map of streamId to serviceId.

EXPLORE OUR SITES