question

Upvotes
Accepted
24 7 9 14

How to use and manage service id to service name associations in ETA API.

I'm using Elektron ETA API in Unix.

I would like to keep a map of EMS service names to serviceId retrieved from source directory response (sourceDirectoryResponseInfo_[i].ServiceId) to be used in making request. This will be done every time the client connects/reconnects to the server. Is that a correct approach ?

I see the serviceId available from the following fields:

msg.msgBase.msgKey.serviceId

msg.refreshMsg.msgBase.msgKey.serviceId

msg.updateMsg.msgBase.msgKey.serviceId

Based on the value of msg.msgBase.msgClass (RSSL_MC_REFRESH and RSSL_MC_UPDATE), do I need to use the serviceId from msg.refreshMsg.msgBase.msgKey.serviceId or msg.updateMsg.msgBase.msgKey.serviceId ? Can I use the service Id from msg.msgBase.msgKey.serviceId in the response for both RSSL_MC_REFRESH and RSSL_MC_UPDATE messages ?

elektronrefinitiv-realtimeelektron-sdkrrteta-apielektron-transport-apiservice-idservice-name
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
76.7k 239 52 72

In my environment, after setting RSSL_RQMF_MSG_KEY_IN_UPDATES in the request, the update messages of MSFT.O contain the service ID in the message keys.

<requestMsg domainType="RSSL_DMT_MARKET_PRICE" streamId="6" containerType="RSSL_DT_NO_DATA" flags="0x4E (RSSL_RQMF_HAS_PRIORITY|RSSL_RQMF_STREAMING|RSSL_RQMF_MSG_KEY_IN_UPDATES|RSSL_RQMF_HAS_QOS)" qosDynamic="0" qosRate="1" qosTimeliness="1" priorityClass="1" priorityCount="1" dataSize="0">
    <key  flags="0x7 (RSSL_MKF_HAS_SERVICE_ID|RSSL_MKF_HAS_NAME|RSSL_MKF_HAS_NAME_TYPE)" serviceId="2115" name="MSFT.O" nameType="1"/>
    <dataBody>
    </dataBody>
</requestMsg>
<updateMsg domainType="RSSL_DMT_MARKET_PRICE" streamId="6" containerType="RSSL_DT_FIELD_LI
ST" flags="0x18 (RSSL_UPMF_HAS_MSG_KEY|RSSL_UPMF_HAS_SEQ_NUM)" updateType="1 (RDM_UPD_EVENT_TYPE_QUOTE)" seqNum="63744" dataSize="117">
    <key  flags="0x7 (RSSL_MKF_HAS_SERVICE_ID|RSSL_MKF_HAS_NAME|RSSL_MKF_HAS_NAME_TYPE)" serviceId="2115" name="MSFT.O" nameType="1"/>
    <dataBody>
        <fieldList flags="0x8 (RSSL_FLF_HAS_STANDARD_DATA)">
            <fieldEntry fieldId="22" data="0C22 6E"/>
            <fieldEntry fieldId="25" data="0C22 B6"/>
            <fieldEntry fieldId="30" data="0E05"/>

You can enable ETA tracing to verify the request and update message.

if (xmlTrace) 
		{
			int debugFlags = 0x2C0;
			RsslTraceOptions traceOptions;


			rsslClearTraceOptions(&traceOptions);
			traceOptions.traceMsgFileName = traceOutputFile;
			traceOptions.traceFlags |= RSSL_TRACE_TO_FILE_ENABLE | RSSL_TRACE_TO_MULTIPLE_FILES | RSSL_TRACE_TO_STDOUT | RSSL_TRACE_READ | RSSL_TRACE_WRITE;
			traceOptions.traceMsgMaxFileSize = 100000000;
			rsslIoctl(chnl, (RsslIoctlCodes)RSSL_TRACE, (void *)&traceOptions, error);
		}

Otherwise, you can run Consumer example in ETA package with -x option to verify the problem.

If the request messages of US RIC's contain RSSL_RQMF_MSG_KEY_IN_UPDATES but the update messages don't have the message keys, you can contact TREP support team to verify the problem.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16.6k 79 39 63

Hi @RAJ,

Yes, the correct approach for your use case is to keep a map of the names and IDs for the services retrieved.

Regarding whether you should use the details from the REFRESH or UPDATE, you will likely need to monitor both the REFRESH and UPDATE events. The REFRESH message contains the complete collection of services available for your connection and other important details such as capability and the state of each service at that point in time.

The UPDATE is strictly used when changes occur, most likely resulting from a service going down or coming back up. In such a case, the service ID is strictly used for you as a reference to which service was affected. However, the UPDATE is also used to indicate the addition or removal of services. In practice it's not too common for short-lived applications to encounter such events. In many cases, applications will never encounter these updates - this depends on the volatility of your environment you are connecting into. However, for long running applications, it is quite possible that a service may be added or removed. You will have to account for this by monitoring the updates.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
24 7 9 14

Will the service name to service Id association be different for connections to different ADS IP's and data centers ?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

RAJ,

service name to service id mappings are not guaranteed to be identical across ADS instances and/or data centers. You should map them for each individual connection when you receive source directory message.

Upvotes
24 7 9 14

I expected the service id to come with every single data update. But, I see it only in the initial record. When I examined the following members in an update record, I see only "0".

msg.msgBase.msgKey.serviceId

msg.requestMsg.msgBase.msgKey.serviceId

msg.updateMsg.msgBase.msgKey.serviceId

How can I retrieve the service id from the update ?

If I can't, how do I know the service for which the update is received if I have subscription open for multiple services ?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hi RAJ,

Do you have XML Tracing turned on so we can see what output you are receiving for these Directory Updates?

Upvotes
76.7k 239 52 72

Refer to RDM usage guide, typically a message key in the update message is optional. It is required if the application specifies the RSSL_RQMF_MSG_KEY_IN_UPDATES in the item request.

The following code is setting the RSSL_RQMF_MSG_KEY_IN_UPDATES flag in the request message.

msg.flags = RSSL_RQMF_HAS_QOS | RSSL_RQMF_STREAMING | RSSL_RQMF_HAS_PRIORITY | RSSL_RQMF_MSG_KEY_IN_UPDATES;

Moreover, before accessing the optional data, such as a message key, the application should check its presence by using flags. For example:

  • Checking the presence of a message key in the update message by using the RSSL_UPMF_HAS_MSG_KEY flag
  if (msg->updateMsg.flags & RSSL_UPMF_HAS_MSG_KEY){
              key = (RsslMsgKey *)rsslGetMsgKey(msg);
   }
  • Checking the presence of a service id in the message key by using the RSSL_MKF_HAS_SERVICE_ID flag
if (key && (key->flags & RSSL_MKF_HAS_SERVICE_ID))
{                
    printf("\nService Id: %d\n", key->serviceId);
}

If the application doesn't check the presence of optional data before accessing the data, the application can retrieve the invalid value and then behave unexpectedly.

For more information, please refer to the RDM Usage guide.


msgkey.png (52.1 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
24 7 9 14

Hi @jirapongse.phuriphanvichai

I made this change and can see serviceId in the update for the international RIC's, but not for the US RIC's like MSFT.O and IBM.N. For the US RIC, I see the serviceId only in Initial record, but not in the update. I see the same behavior with delayed feed as well.

If I need to forward this question to the TR team who had set up the environment for my consumption, please let me know.

----

General question regarding the reliability getting serviceId fields as part of the RsslMsg:

The need to check the availability of serviceId using RSSL_MKF_HAS_SERVICE_ID tells me that I can't use this field reliably to organize the data based on serviceId in order to process the update correctly. If that is the case, what would you recommend to get the serviceId reliably in the update ?

Do I need to cache the map of streamId to serviceId (received in initial record), to associate the streamId from the update to the previously provided serviceId in the initial record ?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
20 0 1 2

Hi RAJ,

As far as I understand, you should parse source directory payload to get the mapping of service id to service name. The source directory refresh message should be all you need as I don't believe you would ever see source directory update with different service id to service name associations, at least I'm not aware of any way we could do it without re-starting ADS and bringing down your connection.

Nick asked earlier if you can share some traces from your application where it receives source directory refresh and updates so that we can see what your application is seeing and make some recommendations. Is this something you can share?

As far as enabling service id on all market data updates, I'm not sure I understand why you'd want to or need to do it. When you make a request, you specify service id yourself so the refresh and updates to this request would all be from the service mapped to service id that you requested.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
24 7 9 14

Hi @art.brzezinski

My question is not related to source directory update or refresh record.

My question is specific to getting serviceId with every single market date update. Earlier this week, @jirapongse.phuriphanvichai responded by saying that by setting a flag in the request, I can enable the server to send serviceId with every single market data update. As I organize my data cache by serviceId, if I receive the serviceId in the data update, I would know the service for which the update was received. So, if a particular service goes down on the ADS, all the RIC's from the downed service can be easily moved to another ADS.

If getting the serviceId from the update and using it to make the decision internally in the application is not the recommended (or reliable) approach, then I have to internally maintain the streamId to serviceId association at the time of making the request to ADS and use this information to figure out the serviceId based on the streamId if a service should go down.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
20 0 1 2

Hi @RAJ,

My apologies, I was going by the subject of this post but with your most recent update, I better understand what you're trying to achieve now. I just run a quick test right and I do see serviceId on updates for MSFT.O but let me run some more tests tomorrow after US market open and get some more stats.

Also note that in case service goes down, you should receive corresponding source directory update which may help you identify the service that went down. But I'm not sure how well this would work with the logic you want your application to have.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
24 7 9 14

I found there was an issue in the way I made the US equity request. I had a code path where the RSSL_RQMF_MSG_KEY_IN_UPDATES flag was not set. Now I'm able to get serviceId in the update for MSFT.O

Just to follow up on the remaining unanswered question in this thread.

1. The need to check the availability of serviceId using RSSL_MKF_HAS_SERVICE_ID tells me that I can't use this field reliably to organize the data based on serviceId in order to process the update correctly. Can I expect to receive serviceId reliably in the update if the flag RSSL_RQMF_MSG_KEY_IN_UPDATES is set.

2. Otherwise, would you recommend the client application to cache the map of streamId to serviceId (received in initial record), to associate the streamId from the update to the previously provided serviceId in the initial record ?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

TREP will follow standards defined in the RDM guide. Therefore, if the application connects to TREP, you can expect to receive serviceId reliably in the update if the flag RSSL_RQMF_MSG_KEY_IN_UPDATES is set.

However, the third party application may not follow this standard. Therefore, you can't expect the serviceId in the update when directly connecting to the third party application.

Yes, instead of using the flag RSSL_RQMF_MSG_KEY_IN_UPDATES, you can cache the map of streamId to serviceId.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.