Stuck instruments when posting

I run the below program which publishes to a service SERVICE on TREP using the latest RFA.net api.

It publishes 2 instruments in /exactly/ the same way but for the instrument anderstest4 the published change does not stick, regardless of the order in which I publish the instruments (or whether I publish both or not).

Furthermore, both publications receive positive acknowledgment from TREP.

There is no issue doing exactly the same using the EMA api, going through the same endpoint.

Why does TREP discard the publications for certain instruments when they come from the RFA.net api but not from EMA?

SERVICE is an internal service with a persistent cache that we update with post messages.

static void Main(string[] args)
{
Random rng = new Random();

stalltest("anderstest4", rng.NextDouble().ToString());
stalltest("anderstest42", rng.NextDouble().ToString());

}

static void stalltest(string instrument, string testvalue)
{
Realtime.TREP.Request request = Realtime.TREP.Request.As(Environment.UserName, "UAT");
var sf = request.MarketAccessTo("SERVICE");
using (var form = sf.CreatePublishingFormStream())
{
form.FillOut(instrument, "QUERY", testvalue);
form.Submit();
System.Threading.Thread.Sleep(100);
}

using (var image = sf.CreateSnapshotSubscription(new[] { instrument }, new[] { "QUERY" }))
{
string test = image.TakeSnapshot()[instrument, "QUERY"].FieldValue;
if (test != testvalue)
{
Console.WriteLine(instrument + " stalled!");
}
else
{
Console.WriteLine(instrument + " works as expected.");
}
}
}

it prints out

anderstest4 stalled!
anderstest42 works as expected.

The PublishingFormStream posts PostMsg objects off-stream with a FieldList as payload.

The FieldList contains the FID for the QUERY field and a RMTES string encoding the random double as a string.

The anderstest4 instrument was created with exactly the same code and has worked fine for some time, until it suddenly got stuck. This happens daily for other instrument that we create and publish to. We currently can't use the RFA.net api for publishing our updates because of this.

We're using ADH version adh3.2.0.L1.linux.tis.rrg 64-bit .

Sincerely

Anders A. Søndergaard

Find more posts tagged with

trep

rfa

elektron

ADH

posting

elektron-sdk

refinitiv-realtime

Accepted answers

umer.nalla

Hi @ansn

Thanks for confirming that you are indeed populating the databuffer correctly - just needed to be eliminate that possibility - as this is a common mistake we have seen being made by other developers.

Also thanks for sharing the the 2nd trace file where you published 'umer' to the same fid for the two different instruments. I can see that on the outgoing post the identical data is being sent:

PostMsg anderstest42 <fieldEntry fieldId="32650" data="1B25 3075 6D65 72"/>

PostMsg anderstest4 <fieldEntry fieldId="32650" data="1B25 3075 6D65 72"/>

This confirms that the API and your application is working and encoding the outgoing payload correctly.

However, when you subscribe to the items we get back different values.

Refresh anderstest42 <fieldEntry fieldId="32650" data="1B25 3075 6D65 72"/>

Refresh anderstest4 <fieldEntry fieldId="32650" data="7465 7374 3437"/>

This indicates that the problem lies elsewhere i.e. the TREP infrastructure components.

As per my initial post, I recommend you speak to your Market Data team so they can check if there is anything reported in their TREP logs.
If they confirm that they do not see any issues reported, then I recommend they raise a support ticket with our TREP support team.
If your Market Data team passes on the Trace files you have generated, they will provide evidence that your application is behaving correctly and publishing the right data and that the problem lies somewhere in TREP.

One final thing you could try - just for elimination purpose is to remove the '1B 25 30' bytes from the RMTES String payload -

e.g. something like

dataBuffer.SetFromString(value, DataBuffer.DataBufferEnum.StringRMTES)

- and run the test again with ASCII only string value as per your above tests.

As you say, if it works once it should work a 2nd time, but I just want to eliminate the possibility that TREP is perhaps occasionally not coping well this switching function.

I tried recreating your scenario here and published two RICs in sequence with identical payload including FID 32650 and I could not get it to fail (with or without the above 3 bytes).

All comments

ansn

Just to add that we see the updates in TREP explorer for anderstest42 but not for anderstest4.

umer.nalla

Hi @ansn

The above code does not resemble RFA.NET code - I can only assume you are using some internal wrapper? Without sight of the actual underlying RFA.net code it would not be possible to confirm if there are any issues with said code.

There a couple of things to try:

Check with your Market Data team if there any issues reported in the TREP (and ATS if you are posting to one) log files.
Enable the low level trace for RFA.net so we can see the message flow between our app and the server

If you have access to the RFA config files, you can enable the low level trace using the following parameters:

\Connections\<Connection_RSSL>\traceMsgToFile = True 
\Connections\<Connection_RSSL>\traceMsgFileName = "<your trace file location and name>"

where <Connection_RSSL> should be replaced with the actual connection being used by whichever Session you acquire from the config.

ansn

Trace attached rfatrace_5984.txt

ansn

The point I want to get across is that the code is working in general, except for particular instruments. That means it's unlikely to be an issue with the code.

The code that calls the RFA looks like this

this.Msg.Clear();
this.Attrib.Clear();

Attrib.ServiceName = RFAserviceName; // i.e. new RFA_String("SERVICE")
Attrib.NameType = RDM.INSTRUMENT_NAME_TYPES.INSTRUMENT_NAME_RIC;
Attrib.Name = new RFA_String(instrument);
Msg.AttribInfo = Attrib;

Msg.MsgModelType = RDM.MESSAGE_MODEL_TYPES.MMT_MARKET_PRICE;
Msg.IndicationMask = PostMsg.IndicationMaskFlag.MessageInit | PostMsg.IndicationMaskFlag.MessageComplete;
Msg.IndicationMask |= PostMsg.IndicationMaskFlag.WantAck;
Msg.SeqNum = PostID; // don't need the fine granularity provided by seqnum. Just set it to postId.
Msg.PostID = PostID;
if ((Msg.HintMask & PostMsg.HintMaskFlag.Seq) == 0) throw new System.Exception("Flag not set");
pendingAcks.Add(PostID, new AckInfo { InstrumentName = instrument });
++PostID;

{
FieldList container = new FieldList();
container.Clear();
container.SetInfo(this.RDMdict.DictId, (short)Dictionary.DICTIONARY_TYPES.DICTIONARY_RECORD_TEMPLATES);
flItr.Start(container);
foreach (var field in instrumentForm.Fields)
{
msgEntry.Clear();
msgEntry.FieldID = field.Key;
msgEntry.Data = field.Value;
flItr.Bind(msgEntry);
}
flItr.Complete();

Msg.Payload = container;
}

OMMHandleItemCmd cmd = new OMMHandleItemCmd();
cmd.Handle = this.loginHandler.LoginStreamHandle; // Off-stream posting, using our loginHandlers stream (and its event handler for ack messages)
cmd.Msg = Msg;
consumer.Submit(cmd);

umer.nalla

Hi @ansn

As mentioned, without your underlying code, it is difficult to investigate what is going wrong.

However, based on the trace I can see that data being sent to server for two instruments is different in length.

For anderstest42 your application is sending the following value:

<fieldEntry fieldId="32650" data="1B25 3030 2E39 3132 3632 3734 3138 3031 3931 37"/>

For anderstest4 your application is sending the following value:

<fieldEntry fieldId="32650" data="1B25 3030 2E35 3438 3830 3038 3235 3836 3235 3837"/>

When you then subscribe to those items, you are receiving back the following payload:

for anderstest42:

<fieldEntry fieldId="32650" data="1B25 3030 2E39 3132 3632 3734 3138 3031 3931 37"/>

and for anderstest4:

<fieldEntry fieldId="32650" data="7465 7374 3437"/>

incidentally - the above hex data converts to 'test47' - does this mean anything to you?

What are you contributing the data to - what does service 'SERVICE' represent? Is it some contributions engine, which perhaps has some rules defined for what length/values it can accept for the fid 32650 and is therefore being rejected? You may well get an ACK from the ADS because it considers the post a valid post msg format, but it is then being discarded by the upstream server.

Also, I should point out that your current wrapper implementation is incredibly inefficient - just in that single trace file you are performing connect, login, source directory download, logout, disconnect four times - which is a considerable overhead given that you are only posting and subscribing two items.

As a test - please post the identical value to the field for both anderstest4 and anderstest42 and see what happens.

ansn

The "test47" string is what the field value is from before my test, set by another client using the EMA api.

I've already told you everything I know about SERVICE. It's an internal persistent cache that's only updated with Post message, configured in the ADH. No interactive providers are attached to it.

The wrapper code is deliberately inefficient during upstart. It is however a lot more efficient than the raw RFA api in terms of developer time spent coding stuff, and it does not sacrifice performance once a subscription is established.

New trace attached where I send the string "umer". rfatrace_332.txt

umer.nalla

Hi @ansn

Thanks for sharing the code. I cannot see how instrumentForm.Fields and its constituent Value are populated, so cannot be sure if the FieldEntry.Data is being populated with the correct DataBuffer content.

Please see attached file for an example of how FieldEntry.Data is populated for different field types e.g. for a string type field (such as 32650):

field.FieldID = 296 // Ask MMID 1 - Alphanumeric/RMTEST_STRING field 
RFA_String AskStr = new RFA_String("Ask MMID data");                            
dataBuffer.SetFromString(AskStr, DataBuffer.DataBufferEnum.StringRMTES); 
field.Data = dataBuffer;                             
fieldListWIt.Bind(field);

The attached code is extracted from the Encoder.cs file which can be found in the Examples\Common\ folder of the RFA.NET SDK.

ansn

Logically, and as you can verify from the trace, instrumentForm.Fields and its constituent Value must be populated properly since it works for one instrument but not the other, having run exactly the same code with only the instrument name differing.

Anyway here is the code if you want to prove me wrong

public void FillOut(string instrument, string field, object value)

{

RDMFidDef def = GetFidDef(field);

Open(instrument).Fields[def.FieldId] = Encoder.Encode(def, value);

}

public Data Encode(RDMFidDef fidDef, object value)
{
byte type_indicator = RFATypes.OMMType2DataBufferEnum(fidDef);
return Encode(type_indicator, value);
}

switch (fidDef.OMMType)
{
case OMMTypeEnum.RWF_TYPE_XML:
return DataBuffer.DataBufferEnum.XML;
case OMMTypeEnum.RWF_TYPE_OPAQUE:
return DataBuffer.DataBufferEnum.Opaque;
case OMMTypeEnum.RWF_TYPE_RMTES_STRING:
return DataBuffer.DataBufferEnum.StringRMTES;
case OMMTypeEnum.RWF_TYPE_UTF8_STRING:
return DataBuffer.DataBufferEnum.StringUTF8;
case OMMTypeEnum.RWF_TYPE_ASCII_STRING:
return DataBuffer.DataBufferEnum.StringAscii;
case OMMTypeEnum.RWF_TYPE_DATE_TIME:
return DataBuffer.DataBufferEnum.DateTime;
case OMMTypeEnum.RWF_TYPE_TIME:
return DataBuffer.DataBufferEnum.Time;
case OMMTypeEnum.RWF_TYPE_DATE:
return DataBuffer.DataBufferEnum.Date;
case OMMTypeEnum.RWF_TYPE_REAL:
return DataBuffer.DataBufferEnum.Real;
case OMMTypeEnum.RWF_TYPE_DOUBLE:
return DataBuffer.DataBufferEnum.Double;
case OMMTypeEnum.RWF_TYPE_FLOAT:
return DataBuffer.DataBufferEnum.Float;

etc etc etc

private Data Encode(byte type_indicator, object value, ErrMsgr default_prepend) {
DataBuffer result = new DataBuffer();
if (value == null)
{
result.SetBlankData(type_indicator);
return result;
}
switch (type_indicator) {

case DataBufferEnum.StringRMTES: // A nice string type with a private encoding scheme and no public encoder for it.
byte[] utf8_part = Encoding.UTF8.GetBytes(dynamic_cast<string>(value, default_prepend)); // The RMTESConverter is one-way only.
byte[] rmtes = new byte[3 + utf8_part.LongLength]; // Tell the RMTES string that we contain utf8.
new byte[] { 0x1B, 0x25, 0x30 }.CopyTo(rmtes, 0); // see https://developers.refinitiv.com/article/encoding-and-decoding-non-ascii-text-using-ema-and-rfa-cnet
utf8_part.CopyTo(rmtes, 3);
result.SetBuffer(new ThomsonReuters.RFA.Common.Buffer(rmtes), type_indicator);
break;
case DataBufferEnum.StringUTF8:
case DataBufferEnum.XML:
result.SetBuffer(new ThomsonReuters.RFA.Common.Buffer(Encoding.UTF8.GetBytes(dynamic_cast<string>(value, default_prepend))), type_indicator);
break;
case DataBufferEnum.StringAscii:
result.SetFromString(new RFA_String(dynamic_cast<string>(value, default_prepend)), type_indicator);
break;

case DataBufferEnum.Double:
result.SetDouble(dynamic_cast<double>(value, default_prepend),type_indicator);
break
case DataBufferEnum.Real: // Radix 10 floating point number implementation (!?)
result.SetFromString(new RFA_String(dynamic_cast<double>(value, default_prepend).ToString()), type_indicator);
break;

etc etc etc

static T dynamic_cast<T>(object value, ErrMsgr prepend_err = null)
{
try
{
return (T)(dynamic)value;
}
catch ....

Anders