Parsing OMM Payload on different thread

We are building Level2 market maker data processing application using RFA and OMM consumer. We want to do the parsing of different handles(Ric symbols) in parallel so we want to pass the RFA OMM payload to a different thread and not parse it in the consumer thread. Is it safe to pass the payload to different processing thread.
I know in EMA is it not recommended to do so because the field entries and field lists are reused.
Thanks
Anil Kalra
Best Answer
-
Hi @anil.kalra I would recommend that you try out a barebone EMA sample like example210__MarketByOrder__Streaming for multiple instruments, and suppress printing to console and compare if there is still extraneous GC going on.
You can see the EMA source code at: https://github.com/Refinitiv/Elektron-SDK/tree/master/Java
0
Answers
-
Hi @anil.kalra For any new application, I would strongly discourage using RFA API for any development. It is feature complete, and you will want to use an API which is supported for a long time to come - which is EMA API.
In both EMA and RFA, the API owns the OMM payload message which is delivered to the application layer and will get reused by the API. For any parallel processing, you can register multiple clients and event queue can be dispatched by multiple threads. All this requires careful execution, since many times, thread contention slows down rather than speed things up.
You will also have to account for race conditions - for e.g. a L2 delete order message with no payload, might get processed before a previous update order message with payload is processed.
0 -
Gurpreet. for the reply. So we cannot handover the payload to a different thread unless we do a deep copy. As far as update and delete race condition is concerned, we were thinking to send the payload over a queue and same handle will always go to same queue so that they don't get process out of order.
Our idea was to not parse the message inline on a processEvent method on the consumer thread and not block it but if the payload is reused then we cannot achieve this.
Thanks
Anil
0 -
Gurpreet. Also for EMA we tried it generated lot of garbage collection which increased our micro collection and pauses in the application. If there is any workaround for this we will be happy to change to EMA.
0 -
@anil.kalra There is no simple answer here. There can be dozens of way an application might be architected and almost always interfaces with inhouse services. We have clients who used EMA for building their Level2 shadow books for whole venue, so it is definately doable.
0 -
Thanks @Gurpreet I agree there are many ways to implement. We have a working model of Level2 using EMA and RFA. But our initial testing shows that EMA generates way lot more garbage compared to RFA and that was the only reason we decided to go with RFA.
Any suggestion on EMA garbage generation.
Thanks for your reply and suggestions.
Anil
0
Categories
- All Categories
- 3 Polls
- 6 AHS
- 36 Alpha
- 166 App Studio
- 6 Block Chain
- 4 Bot Platform
- 18 Connected Risk APIs
- 47 Data Fusion
- 34 Data Model Discovery
- 687 Datastream
- 1.4K DSS
- 623 Eikon COM
- 5.2K Eikon Data APIs
- 11 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- 3 Trading API
- 2.9K Elektron
- 1.4K EMA
- 255 ETA
- 557 WebSocket API
- 38 FX Venues
- 14 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 23 Messenger Bot
- 3 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 60 Open Calais
- 276 Open PermID
- 44 Entity Search
- 2 Org ID
- 1 PAM
- PAM - Logging
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 22 RDMS
- 1.9K Refinitiv Data Platform
- 682 Refinitiv Data Platform Libraries
- 4 LSEG Due Diligence
- LSEG Due Diligence Portal API
- 4 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.2K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 12 World-Check Customer Risk Screener
- 1K World-Check One
- 46 World-Check One Zero Footprint
- 45 Side by Side Integration API
- 2 Test Space
- 3 Thomson One Smart
- 10 TR Knowledge Graph
- 151 Transactions
- 143 REDI API
- 1.8K TREP APIs
- 4 CAT
- 27 DACS Station
- 121 Open DACS
- 1.1K RFA
- 105 UPA
- 194 TREP Infrastructure
- 229 TRKD
- 918 TRTH
- 5 Velocity Analytics
- 9 Wealth Management Web Services
- 91 Workspace SDK
- 11 Element Framework
- 5 Grid
- 18 World-Check Data File
- 1 Yield Book Analytics
- 48 中文论坛