RFA .NET excessive memory usage when subscribing to thousands of RIC codes
We are currently using RFA.NET 8.1.0 and we are seeing a drastic increase in our memory usage while subscribing to around thousands RICs. We've attempted to reduce the memory usage by making use of Dynamic Views to reduce the number of Field Entries we get back and we've moved our processing logic out of the ProcessEvent() method. We now write the events to an internal collection and read from that on a separate thread.
Our issue seems to be with the volume of RICs we're subscribing to. Is there a way we can improve our RAM usage and more efficiently handle all of the incoming market events for all of these RICs?
Hi
Some alternatives here:
- if acceptable use a conflated service so it updates only, for instance, once per second
- if acceptable request snaps in a loop, use a view field list to request only the fields needed, you might need to check the request period as it may take time to iterate over all instruments
- compute update frequency for all itens, the ones that update too fast go to a `request snap list`, for these you get snaps as described in the second item. The other itens continue as normal subscriptions.
- some applications try to update a GUI for every update then the GUI thread becomes the bottleneck. (same idea when application try to save market data to a Database).
Could you please reveal how many items are you trying to subscribe?
I have faced similar problems here with a project that tries to subscribe more than 10k itens.
I am trying to implement the mix approach snap+realtime combination as there is no conflated service available.
Another point is when you expand or not chains in your code as one chain ric can result in hundreds others being subscribed to.