ETA group format

andy.sciascia

In the message below the "Group" "data" is reported as "4". When the message is received by my VA consumer application it's not a string but a raw data buffer of length 2 with 0 in the 1st byte and 4 in the second.

In this instance the message arrived from the rmdstestserver.

My question is - is there any convention associated with the layout of this buffer ? Is the buffer always 2 bytes for example ?

Also, in the example code for dealing with groups a maximum number of them is specified to be 10. I assume this is just for the example.

Is there any real limit to the number of groups ?

<!-- Incoming Message (Channel IPC descriptor = 485012) -->
<!-- Time: 11:24:32:392 -->
<!-- rwfMajorVer="14" rwfMinorVer="1" -->
<updateMsg domainType="RSSL_DMT_SOURCE" streamId="0" containerType="RSSL_DT_MAP" flags="0xC8 (RSSL_UPMF_HAS_MSG_KEY|RSSL_ UPMF_DO_NOT_CACHE|RSSL_UPMF_ DO_NOT_CONFLATE)" updateType="0 (RDM_UPD_EVENT_TYPE_ UNSPECIFIED)" dataSize="2981">
<key flags="0x8 (RSSL_MKF_HAS_FILTER)" filter="4"/>
<dataBody>
<map flags="0x8 (RSSL_MPF_HAS_TOTAL_COUNT_ HINT)" countHint="1" keyPrimitiveType="RSSL_DT_ UINT" containerType="RSSL_DT_FILTER_ LIST" >
<mapEntry flags="0x0" action="RSSL_MPEA_UPDATE_ ENTRY" key="333" >
<filterList containerType="RSSL_DT_ ELEMENT_LIST" countHint="2" flags="0x2 (RSSL_FTF_HAS_TOTAL_COUNT_ HINT)">
<filterEntry id="3" action="RSSL_FTEA_SET_ENTRY" flags="0x0" containerType="RSSL_DT_ ELEMENT_LIST">
<elementList flags="0x8 (RSSL_ELF_HAS_STANDARD_DATA)">
<elementEntry name="Group" dataType="RSSL_DT_BUFFER" data="4"/>
<elementEntry name="Status" dataType="RSSL_DT_STATE" dataState="RSSL_DATA_SUSPECT" streamState="RSSL_STREAM_OPEN" code="RSSL_SC_NONE" text="Group Stale" />
</elementList>
</filterEntry>
</filterList>
</mapEntry>
</map>
</dataBody>
</updateMsg>

zoya faberov

Hello @andy.sciascia

Thanks, your explanation helps. You are designing a consumer app, you are looking to capture service status changes as reported by Elektron Edge Device.

I would approach requirement from ETA Consumer example that came with Elektron SDK. It subscribes to Source Directory, and does complete parsing of Source Directory updates. You can use the parsing from the example, to lay out and understand the information you receive from EED. The example prints any errors or irregularities encountered on parsing, if something is off, to report the issues, otherwise, it parses source directory (RSSL_DMT_SOURCE) silently. To better understand the structure and the parsing, I would add the output on the valid directory as well. In

RsslRet decodeSourceDirectoryResponse(RsslSourceDirectoryResponseInfo* srcDirRespInfo,

I would introduce debug prints, for example, but not limited to:

case RDM_DIRECTORY_SERVICE_INFO_ID:
      if ((ret = decodeServiceGeneralInfo(&srcDirRespInfo[serviceCount].ServiceGeneralInfo, dIter, maxCapabilities, maxQOS, maxDictionaries)) != RSSL_RET_SUCCESS)
      {
       printf("decodeServiceGeneralInfo() failed with return code: %d\n", ret);
       return ret;
      }
      printf("***decodeServiceGeneralInfo ServiceName=%s\n", srcDirRespInfo[serviceCount].ServiceGeneralInfo.ServiceName);
      break;
     case RDM_DIRECTORY_SERVICE_STATE_ID:
      if ((ret = decodeServiceStateInfo(&srcDirRespInfo[serviceCount].ServiceStateInfo, dIter)) != RSSL_RET_SUCCESS)
      {
       printf("decodeServiceStateInfo() failed with return code: %d\n", ret);
       return ret;
      }
      printf("***decodeRsslServiceStateInfo ServiceState=%i AcceptingRequests=%i\n", srcDirRespInfo[serviceCount].ServiceStateInfo.ServiceState, srcDirRespInfo[serviceCount].ServiceStateInfo.AcceptingRequests);
      break;
     case RDM_DIRECTORY_SERVICE_GROUP_ID:
      if ((ret = decodeServiceGroupInfo(&srcDirRespInfo[serviceCount].ServiceGroupInfo, dIter)) != RSSL_RET_SUCCESS)
      {
       printf("decodeServiceGroupInfo() failed with return code: %d\n", ret);
       return ret;
      }
      printf("***decodeServiceGroupInfo: Group %i Status=%s", srcDirRespInfo[serviceCount].ServiceGroupInfo.Group, srcDirRespInfo[serviceCount].ServiceGroupInfo.Status.text);
     
      break;
 

From my quick test on our testbed EED, I do not see RDM_DIRECTORY_SERVICE_GROUP_ID updated, in source directory updates. The service state is provided via RDM_DIRECTORY_SERVICE_STATE_ID info within the directory update. I am seeing, from my introduced prints:

***decodeServiceGeneralInfo ServiceName=ELEKTRON_DD
***decodeRsslServiceStateInfo ServiceState=1 AcceptingRequests=1
***decodeServiceGeneralInfo ServiceName=ELEKTRON_AD
***decodeRsslServiceStateInfo ServiceState=1 AcceptingRequests=1

This may also be configuration dependent, and available on your local EED. So I would put in additional prints on source directory within example, and examine your local output.

The reference documentation can be a good auxiliary tool to understand the output.

Jirapongse

@andy.sciascia

The GroupID has been explained in the ETA Developers Guide at 13.4.1 Item Group Buffer Contents.

The consuming application should treat data (which may be of varying length) contained in the groupId buffer as opaque. A simple memory comparison operation can determine whether two groups are equivalent. The actual data contained in the groupId buffer is a collection of one or more unsigned two-byte, unsigned integer values, where each two-byte value is appended to the end of the current groupId RsslBuffer.

The size of GroupID is mentioned in the 2.2.2.1 RSSL Refresh Message Structure Members.

groupId:   
An RsslBuffer containing information about the item group to which this stream belongs. The groupId RsslBuffer has a maximum allowed length of 255 bytes.

andy.sciascia

Hi. Thanks for the response.

Re the number of groups - I meant what is the maximum ( if any ) number of individual groups that a service can contain ?

i.e. - when I get a RsslRDMDirectoryRefresh service message I look at the groupStateCount in the RsslRDMService part of that message to find out the number of RsslRDMServiceGroupState messages. Does that count have a maximum value ?

The example code for groups suggests that it should be limited in the application but I'm not sure how to decide that limit.

Related to that - I assume that a service can contain N groups but that a RsslRDMDirectoryRefresh or RsslRDMDirectoryUpdate message may contain anything from 0-N RsslRDMServiceGroupState messages depending on what groups have changed state.

Is that correct ?

zoya faberov

Hello @andy.sciascia

As mentioned by @Jirapongse on the technical level, the maximum number of groups is defined and limited by groupId length.

However, in terms of practical usage, a service that provides a couple of groups has a mirror expectation on the downstream consumers of the service, they understand the logical division of the instruments into groups within the service and closely track the status updates per those groups. So the intended usage is to have several, well-defined, groups of instruments per service.

The corresponding RsslRDMServiceGroupState message should contain the updates to any updated groups, and is not limited technically (you have an option of reviewing the implementation of the ETA C/C++ library, it is open source on Elektron-SDK CPP/C on GitHub) but typically, status messages on the consumer side imply action on every status, and commonly are logged, filtered and then reported, for example, via email, so large status from many groups would be unexpected and imply a lot of downstream processing.

zoya faberov

https://community.developers.refinitiv.com/discussion/comment/64472#Comment_64472

Hello @andy.sciascia

If you can discuss the use case you are working through, if we can understand the requirements better, we can be of more help?

andy.sciascia

Hi,

The consumer application we've implemented has never used groups before.

The issue we have is that the application is connected to trep via an Edge server. This means that our normal way of detecting issues with service or channel has been removed because the Edge server never tells us if the channel or service is down.

When the service or channel goes down the Edge server sends us a RSSL_DMT_SOURCE message telling us all the groups have gone down ( this is what we've been told ).

We don't need to fan out the groups status for this because in this case we're designed to disconnect from our downstream peer.

We're only interested in group status from the point of view of figuring out if the service is gone.

So one of the things I need to do is keep track of the groups in a service so I can figure out if they are all down.

I may also get RSSL_DMT_SOURCE messages where individual groups go up or down and I need to be able to figure out when I get to the state when all groups are down.

What I don't understand from your comment above is how groupId length has anything to do with group state count. I thought we get an array of RsslRDMServiceGroupState group state information of length groupStateCount and in each RsslRDMServiceGroupState there is a group id buffer which has length and data as described by the 1st answer above i.e. a number of 2 byte identifiers.

andy.sciascia

I'm on holidays for 2 weeks from today but I'll check in when I get back. Thanks in advance.

andy.sciascia

Thanks. Our code follows the VA consumer example pretty faithfully but in the use case we have the service state will not change - only the group states.

In the case where we connect directly to trep01 we get the information you mention above and we log it and act on it. That's not the issue.

The issue is that with an edge server between our app and trep we're not getting that information. The service state is being reported as up. We've been told this by Refinitiv. Edge is actively stopping the service state change and channel state change getting to our app and only sending group status. So we need to update our app to detect "all groups down" as being equivalent to "service down".

zoya faberov

https://community.developers.refinitiv.com/discussion/comment/64501#Comment_64501

Hello @andy.sciascia,

Understood.

ETA VA Consumer example covers both service state and group state changes processing, as Consumer example.

Are you able to run VA Consumer example while printing the group info out? Do you see it, in source directory refresh? in the update (do you see any updates)? Something like pService->groupStateCount(), what do you see in the refresh?

andy.sciascia

Hi. I'm on holidays for 2 weeks so I'll check back on this when I get back.

I've already updated my application based on the VA example code for groups but using it for our use case. The example code use case is for fanning out group status to items.

Our use case is - we want to disconnect our peer if we detect channel or service down ( specifically if the priority service is down i.e. ELEKTRON_DD ).

When we're connected via an Edge server this use case now becomes - we want to disconnect our peer if all groups in the service are down ( because we'll never be told the channel or service is down by the Edge server ).

Our local installation of trep01 always has a groupStateCount of 0. Also, when I connect the app directly to our Edge server the group state count is 0.

The original issue happened at an installation which I currently don't have access to but when the incident happened we didn't switch because the app didn't have group handling code. I've been told by Refinitiv that the Edge server at that installation had the issue it sent a message with all the groups down.

I have the rmdstestserver so I can poke my application with that and see my new group handling code reacting as the groups states change.

But as far as I know we don't always get the group information for all groups in all RSSL_DMT_SOURCE messages. That is if a service comprises of N groups we might often get RSSL_DMT_SOURCE message where the group state count is < N.

So how can I figure out if all the groups are down if I don't know how many groups are in the service ?

Do we get the status of all groups in the initial refresh message from an Edge server ? If that was the case I'm OK because I can cache the group states and work out when all of them are down. I've assumed that is the case in the code I've implemented.

BTW: I've been told that I don't need to consider group merges so I'm ignoring that.

Jirapongse

@andy.sciascia

It could be a problem at the Edge server. With the API, the provider application (Edge server) can publish all available groupIDs in the Source Directory refresh message. I assume that, currently, the edge server doesn't provide this information.

Moreover, the Edge server should send a source directory update message with service down to indicate the service now status. You may need to directly contact the server team to verify it.