question

Upvotes
Accepted
cory.schmidt.1 avatar image
25 ●2 ●2 ●8

Memory usage of instrument metadata cache (InstrumentMeta object)

Our thick client desktop application provides a tool similar to Workspace Monitor, where our users can open a list of instruments with specific fields to monitor. Sometimes that list can be fairly large, up to 500 instruments. In profiling the memory usage in our desktop application we noticed that a large portion of the memory was being used by Refinitiv.Data.Content.HistoricalPricing.TSI.Metadata.InstrumentMeta objects which are around 1.75 MB each. Is there anything that could be done to reduce the memory footprint of the metadata cache? Here is a snapshot of our memory profile after opening 100 instruments...

1708367316869.png

#producthistoricalrefinitiv-data-platform-librariesstreaming-datamemory
1708367316869.png (198.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

raksina.samasiri avatar image   raksina.samasiri

Hi @cory.schmidt.1 ,

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query?
If so please can you click the 'Accept' text next to the appropriate reply? This will guide all community members who have a similar question.

Thanks,
AHS

2 Answers

· Write an Answer
Upvotes
Accepted
cristian.baciuwahl avatar image
142 ●2 ●1 ●3

Hello @cory.schmidt.1

Good news !

I implemented a solution to reduce memory usage of Instrument meta data by about 90%.

Have sent you an email with the link on BAMS - newest package version is 1.0.5

Bellow is a snapshot of pre and post fix with 1 instrument and then post fix scaled to 100 instruments from which about 90 opened. Because I do not have Resharper tools, I am using the Profiler inside Visual Studio.

Instrument Meta memory fix.png


instrument-meta-memory-fix.png (54.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

cory.schmidt.1 avatar image cory.schmidt.1

Thanks Cristian. The memory footprint looks awesome now with 1.0.5. However, I've seen a crash twice now when requesting two instruments in MetaStock at the same time, 1YMc1 and ESc1. I have a screen shot from VS but I'm not sure what else would be helpful.

1708983746463.png

1708983746463.png (413.2 KiB)
cristian.baciuwahl avatar image cristian.baciuwahl cory.schmidt.1
Hello @cory.schmidt.1

I will check those instruments today as soon as possible and provide a fix ASAP.

Thanks,

Cristian

cristian.baciuwahl avatar image cristian.baciuwahl cory.schmidt.1

The issue that the screenshot presents is a null _timezones field.

This happens if the entire GlobalRules is null, or if the TimeZone token object from the Global Rules could not be retrieved.

I tried testing on my local machine with these 2 instruments, but was unable to reproduce this problem. Here is a snapshot of results:

instrument tests.png

What I can do for now is harden the code in that area, so it won't crash anymore when time zones are not available, and maybe improve the logging as well.

If you could share the code used for this case and any logs available, they might help me recreate the problem.

After looking into it, I found out that Global Rules is not instrument specific, so the problem might be something not instrument related...

Thanks,

Cristian

instrument-tests.png (111.4 KiB)
cristian.baciuwahl avatar image cristian.baciuwahl cory.schmidt.1

Newest version (1.0.6) with null check and additional logging is available on BAMS.

Thanks,

Cristian

cory.schmidt.1 avatar image cory.schmidt.1 cristian.baciuwahl
Is there something I need to do to enabled logging and where are the log files written out?
Show more comments
Upvote
cristian.baciuwahl avatar image
142 ●2 ●1 ●3
Hello @cory.schmidt.1

The instrument metadata is cached in an attempt to reduce the number of requests to the backend when data is requested for large instruments lists. I've created a ticket in our backlog to investigate and try to optimize memory usage.

We will keep you posted when a new update is available.

Thank you for your patience,

Cristian

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.