For futures ExtractRaw CSVs, what does this DELETE message mean?

Jwan622

I'm trying to use the ExtractRaw endpoint in datascope for the RawMarketByPrice view. I see messages like this for 0#TU (the price is around $103 currently).

TUM25,Market By Price,2025-05-02T09:30:01.148170445Z,-5,Raw,UPDATE,UNSPECIFIED,,,,3153,,1056,,0
,,,,Summary,,,,,,,,,,4
,,,,FID,1021,,SEQNUM,74937256,
,,,,FID,17,,ACTIV_DATE,2025-05-02,
,,,,FID,4148,,TIMACT_MS,34201142,
,,,,FID,14269,,TIMACT_NS,09:30:01.142384087,
,,,,MapEntry,,DELETE,,,,,,30:26604/2,,0
,,,,MapEntry,,ADD,,,,,,30:26614/2,,8
,,,,FID,3430,,NO_ORD,71,
,,,,FID,8591,,ORD_TONE,1,
,,,,FID,4356,,ACC_SIZE,707,
,,,,FID,14268,,LV_TIM_NS,09:30:01.142384087,
,,,,FID,6527,,LV_TIM_MS,34201142,
,,,,FID,6529,,LV_DATE,2025-05-02,
,,,,FID,3427,,ORDER_PRC,103.9609375,
,,,,FID,3428,,ORDER_SIDE,2,ASK

When I encounter a message like this… what do I do? What order book price-size pair do I delete? Why is the 26604 nowhere near the 103.9609375 price? Where do I find documentation on this? What do I add?

Jirapongse

@Jwan622

Thank you for reaching out to us.

Each map entry has an associated map key. Therefore, 30:26604/2 and 30:26614/2 are map keys.

,,,,MapEntry,,DELETE,,,,,,30:26604/2,,0

,,,,MapEntry,,ADD,,,,,,30:26614/2,,8

This Market By Price data was captured from real-time feed. You can refer to this How to Sort & Process Level 2 Orderbook Data using EMA C++ API article and RDM Usage Guide in the Market By Price Domain section.

Jwan622

Sorry can you explain that again? I don't understand.

Jirapongse

https://community.developers.refinitiv.com/discussion/comment/125602#Comment_125602

First, please read this How to Sort & Process Level 2 Orderbook Data using EMA C++ API article.

Jwan622

Okay I see this in this document: https://developers.lseg.com/en/article-catalog/article/how-sort-process-level-2-orderbook-data-using-ema-c-api

UpdateMsg

• An UpdateMsg will contain a Map with one or more Map entries with Actions of Add, Update or Delete – where each entry includes the Map entry Key as well any other relevant content
• If a new row is added to the book then you will receive a Map entry with Action of Add; the entry payload will be a FieldList containing the new row details
• If an existing MarketByOrder entry changes, then you will receive a Map entry with action of Update, the Key of the existing entry to update and a FieldList containing the revised row details
• Since the MarketByPrice Map entry Keys are based on the price, the behaviour is slightly different
• For a non-price change, you will receive a Map Entry with Action of *Update*, the existing Key and a FieldList containing the revised row details. As mentioned earlier, the FieldList will contain the existing price – however other fields such as Activity Time, Order size will reflect any changes to the row
• When the price of a MarketByPrice row changes, this affects the associated Map entry’s Key itself. Since there is no mechanism for updating a Map entry’s Key, the entry needs to be deleted and replaced with one with the new Key. Therefore, you will first receive a Map Entry with *Delete* action and Key of the entry to delete. This will be followed by another Map Entry with *Add* action for the new (revised) entry along with the new Key and corresponding FieldList.
• When an row is removed from the Order book e.g. if a Pending Order was filled, you will receive a Map entry with an Action of Delete and the Map Entry Key of the entry to erase


 

That should cover the main relevant points you need to be aware of when dealing with Order books.


Just to recap:

• Order Books are implemented using the MarketByOrder and MarketByPrice Reuters domain Models.
• The payload consists of Summary Data (which is non-order specific information) and a Map where each entry contains a FieldList representing an Order / Price point.
• The Map entries include an Action which instructs whether to Add the entry as a new row in the Order book, to Update an existing row or Delete an existing row.
• For Add or Update entries if any of the Mandatory fields are missing the entry should be ignored.
• Since Order books can be quite large, they are usually delivered as multi-part Refresh messages and the final Refresh message is indicated by the Complete flag being set

I am not going to cover much of the code required to request and process an Order book. This is because we already have Level 2 tutorials on our developer portal for most of our Real-Time APIs (see links at the end of the article). As the tutorials are basic console output applications, they don’t tend to address the multi-part Refresh mechanism, so I will cover that as part of the next section of this article.

The other aspect that the existing tutorials do not demonstrate, is how to implement the above sorting / ranking mechanism for display purposes.

My questions:

1. So in a message like this:

TUM25,Market By Price,2025-05-02T09:30:01.148170445Z,-5,Raw,UPDATE,UNSPECIFIED,,,,3153,,1072,,0
,,,,Summary,,,,,,,,,,4
,,,,FID,1021,,SEQNUM,74937257,
,,,,FID,17,,ACTIV_DATE,2025-05-02,
,,,,FID,4148,,TIMACT_MS,34201142,
,,,,FID,14269,,TIMACT_NS,09:30:01.142386815,
,,,,MapEntry,,DELETE,,,,,,30:26594/1,,0
,,,,MapEntry,,ADD,,,,,,30:26604/1,,8
,,,,FID,3430,,NO_ORD,1,
,,,,FID,8591,,ORD_TONE,0,
,,,,FID,4356,,ACC_SIZE,9,
,,,,FID,14268,,LV_TIM_NS,09:30:01.142386815,
,,,,FID,6527,,LV_TIM_MS,34201142,
,,,,FID,6529,,LV_DATE,2025-05-02,
,,,,FID,3427,,ORDER_PRC,103.921875,
,,,,FID,3428,,ORDER_SIDE,1,BID

is the whole 30:26594 and 30:26604 the key? the part after teh / is the side right? For a delete, we're supposed to remove the price associated with that key right? Is the price 103.921875? So we need to maintain a dict of keys and prices and use that when an ADD or DELETE is in this format correct?

What's odd is not all RawMarketByPrice messages are like this. Some directly just reference a price. Is there a way to just get that format from the API? Is there a way to tell ahead of time which format we're getting? Or should we just look for Key/Msg Sequences that are non-null and if it's populated with a value like 30:<number>, we maintain a dictionary of keys to prices?

Jirapongse

@Jwan622

The whole 30:26594/1 and 30:26604/1 are keys.

The key format may depend on RICs or Markets. You need to contact the content support team directly via MyAccount for content questions.

Yes, you need to maintain a dictionary of keys and values and manage it according to the ADD, UPDATE, or DELETE action.