We are running an application with a single RFA Client Consumer in .NET Framework, that both request and posts data in one session with a single eventqueue.
Post: Sends updates messages for our ATS using off-stream.
Consume: Retrives data using snapshots only, from ATS and hEDD.
Consumer handles requests from multiple threads. Each requesting between 20 and 500 rics pr. batch. Some threads request every 30 sec, others every 5 min during the entire day. And offen times multiple requests are executed at the same time, while data is posted via the same client.
In the OMMItemEvent we receive, the item name matches the requested aswell as the closure item, but the payload does not match expected. The problem started after we updated from 7.5 to 8.2.x and increased the load on the RFAConsumer requesting more data in parallel. It seems that the memory for the payload is being rewritten with new data before we are able to decode it. How do we protect against this issue.
What we expirence from time to time is that the payload we decode belongs to another RIC and we been able to backtrack what RIC they originated from, and then returns to normal on the next update. I Included a few loglines below. Fields are renamed in log to internal names. Bid ask remains the same.
***log snippet from production ***
Line 37971: 27-12-2023 08:50:32 <RIC>DKK=</RIC><Bid>6.7527</Bid><Ask>6.7537</Ask>
Line 38092: 27-12-2023 08:51:02 <RIC>DKK=</RIC><Bid>-985</Bid><Ask>-964.6</Ask>
Line 38614: 27-12-2023 08:51:31 <RIC>DKK=</RIC><Bid>6.7527</Bid><Ask>6.7537</Ask>
Line 253635: 27-12-2023 12:25:01 <RIC>EUR20Y2Y6EATM=ICAP</RIC><FWD_STR_PRC>1.913</FWD_STR_PRC><Bid>45.1</Bid><Date>27 DEC 2023</Date><Last>73.658</Last><Premium>511</Premium><Time>11:18:54</Time><Black76Shift>19.4</Black76Shift><Black76ShiftSz>2</Black76ShiftSz>
Line 258636: 27-12-2023 12:30:03 <RIC>EUR20Y2Y6EATM=ICAP</RIC><Bid></Bid><Date></Date><Last>85.75</Last><Premium>86.15</Premium><Time></Time><Black76ShiftSz></Black76ShiftSz>
Line 262261: 27-12-2023 12:35:03 <RIC>EUR20Y2Y6EATM=ICAP</RIC><FWD_STR_PRC>1.910</FWD_STR_PRC><Bid>45.2</Bid><Date>27 DEC 2023</Date><Last>73.658</Last><Premium>511</Premium><Time>11:41:46</Time><Black76Shift>19.4</Black76Shift><Black76ShiftSz>2</Black76ShiftSz>
We decode on the callback thread, but do have a delegated raised in the process before decoding. I removed all code not relating to payload and decoding from code below.
public class RFAConsumer : Client
{
//callback method to process marketpriceresponse
private void ProcessMarketPriceResponse(OMMItemEvent evnt, RespMsg respMsg)
{
Data payload = null;
if ((respMsg.HintMask & RespMsg.HintMaskFlag.Payload) != 0)
{
payload = respMsg.Payload;
}
if (string.IsNullOrEmpty(itemName))
itemName = respMsg.AttribInfo.Name.ToString();
var marketPriceMessage = new RFADataMessage{
RicName = itemName,
Payload = payload,
};
RaiseOnDataEvent(marketPriceMessage);
}
private void RaiseOnDataEvent(RFADataMessage ricElement)
{
if (OnDataEvent != null)
OnDataEvent(this, new ReutersDataEventArgs(ricElement));
}
}
//Event handler for OnDataEvent in handler
private void Consumer_OnDataEvent(object sender, ReutersDataEventArgs args)
{
//Some replys comes at multipart. First msg. is a status msg. 2nd contains the payload
if (args.Message.Payload == null) return;
Decoder.DecodeData(args.Message.Payload, out fieldDataList);
var response = new Response(){
Ric = args.Message.RicName,
FieldValues = fieldDataList;
};
WriteResponse(response);
}
