Memory usage of instrument metadata cache (InstrumentMeta object)

cory.schmidt.1

Our thick client desktop application provides a tool similar to Workspace Monitor, where our users can open a list of instruments with specific fields to monitor. Sometimes that list can be fairly large, up to 500 instruments. In profiling the memory usage in our desktop application we noticed that a large portion of the memory was being used by Refinitiv.Data.Content.HistoricalPricing.TSI.Metadata.InstrumentMeta objects which are around 1.75 MB each. Is there anything that could be done to reduce the memory footprint of the metadata cache? Here is a snapshot of our memory profile after opening 100 instruments...

Find more posts tagged with

memory

historical

refinitiv-data-platform-libraries

streaming-data

#product

Accepted answers

cristian.baciuwahl

Hello @cory.schmidt.1

Good news !

I implemented a solution to reduce memory usage of Instrument meta data by about 90%.

Have sent you an email with the link on BAMS - newest package version is 1.0.5

Bellow is a snapshot of pre and post fix with 1 instrument and then post fix scaled to 100 instruments from which about 90 opened. Because I do not have Resharper tools, I am using the Profiler inside Visual Studio.

Instrument Meta memory fix.png

All comments

cristian.baciuwahl

Hello @cory.schmidt.1

The instrument metadata is cached in an attempt to reduce the number of requests to the backend when data is requested for large instruments lists. I've created a ticket in our backlog to investigate and try to optimize memory usage.

We will keep you posted when a new update is available.

Thank you for your patience,

Cristian

cristian.baciuwahl

Hello @cory.schmidt.1

Good news !

I implemented a solution to reduce memory usage of Instrument meta data by about 90%.

Have sent you an email with the link on BAMS - newest package version is 1.0.5

Instrument Meta memory fix.png

cory.schmidt.1

Thanks Cristian. The memory footprint looks awesome now with 1.0.5. However, I've seen a crash twice now when requesting two instruments in MetaStock at the same time, 1YMc1 and ESc1. I have a screen shot from VS but I'm not sure what else would be helpful.

cristian.baciuwahl

Hello @cory.schmidt.1

I will check those instruments today as soon as possible and provide a fix ASAP.

Thanks,

Cristian

cristian.baciuwahl

The issue that the screenshot presents is a null _timezones field.

This happens if the entire GlobalRules is null, or if the TimeZone token object from the Global Rules could not be retrieved.

I tried testing on my local machine with these 2 instruments, but was unable to reproduce this problem. Here is a snapshot of results:

instrument tests.png

What I can do for now is harden the code in that area, so it won't crash anymore when time zones are not available, and maybe improve the logging as well.

If you could share the code used for this case and any logs available, they might help me recreate the problem.

After looking into it, I found out that Global Rules is not instrument specific, so the problem might be something not instrument related...

Thanks,

Cristian

cristian.baciuwahl

Newest version (1.0.6) with null check and additional logging is available on BAMS.

Thanks,

Cristian

EXPLORE OUR SITES