question

Upvotes
arek.majka avatar image
1 ●0 ●0 ●1

Data API C# - Streams stop working

Hello,

I am having issues where the streams are freezing (no more data comes in) and then my program has excessive memory usage. When I take a heap shot, all of the memory are used by json objects which look to be refinitiv data objects. It's definitely not my application part holding onto memory. I paused a debug session and nothing more comes to my OnUpdate() or OnInsert() type functions from a stream.


When I finally stop my application and try to restart, it can't connect to workspace api. It just says: "State: Pending. DesktopSession is Pending"


Here are the RDP logs in the example of trying to reconnect:


2024-10-21 10:16:43.6641|Info|Refinitiv.Data.Log|1|Registering log system
2024-10-21 10:16:43.6641|Info|Refinitiv.Data.Initializer|1|Library version: 1.0.0-beta5
2024-10-21 10:16:43.6641|Info|Refinitiv.Data.Initializer|1|.Net runtime: 8.0.8
2024-10-21 10:16:43.6641|Info|Refinitiv.Data.Initializer|1|.Net version: 8.0.8+08338fcaa5c9b9a8190abb99222fed12aaba956c
2024-10-21 10:16:43.6923|Info|Refinitiv.Data.Core.DataLibraryConfig|1|Successfully loaded the Stream:[internal] configuration database.
2024-10-21 10:16:43.7084|Info|Refinitiv.Data.Core.DesktopSessionCore|1|No named session defined.  Successfully loaded desktop default configuration.
2024-10-21 10:16:43.7084|Info|Refinitiv.Data.Core.DesktopSessionCore|1|DesktopSession Session State transitioned from Closed to Pending
2024-10-21 10:16:43.7243|Info|Refinitiv.Data.Core.DesktopSessionCore|1|Locating proxy port within: C:\Users\Administrator\AppData\Roaming/Refinitiv/Data API Proxy/.portInUse
2024-10-21 10:16:43.7243|Info|Refinitiv.Data.Core.DesktopSessionCore|1|Found port 9000 in file: C:\Users\Administrator\AppData\Roaming/Refinitiv/Data API Proxy/.portInUse


Now of course I can just completely stop workspace.. but something is happening that completely kills my streams and then I can't even reconnect to the API. Workspace was still fully functional on the front-end.


The only useful log I can give you related to the streams stopping is this portion

System.NullReferenceException: Object reference not set to an instance of an object.
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSTable..ctor(TSTable dataTable)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSBars.SummarizePartialBar(DateTime datetime, IDerivationRules derivatives)
   at Refinitiv.Data.Content.HistoricalPricing.IntradaySummarizer.SummarizeRecords(IList`1 records)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.ProcessMessageAsync(Boolean isRefresh)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.<StartMessageProcessing>b__20_0()
2024-10-21 03:35:09.7106|Error|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|205|Problems Summarizing record
System.NullReferenceException: Object reference not set to an instance of an object.
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSTable..ctor(TSTable dataTable)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSBars.SummarizePartialBar(DateTime datetime, IDerivationRules derivatives)
   at Refinitiv.Data.Content.HistoricalPricing.IntradaySummarizer.SummarizeRecords(IList`1 records)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.ProcessMessageAsync(Boolean isRefresh)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.<StartMessageProcessing>b__20_0()
2024-10-21 03:35:09.7106|Error|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|205|Problems Summarizing record
System.NullReferenceException: Object reference not set to an instance of an object.
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSTable..ctor(TSTable dataTable)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSBars.SummarizePartialBar(DateTime datetime, IDerivationRules derivatives)
   at Refinitiv.Data.Content.HistoricalPricing.IntradaySummarizer.SummarizeRecords(IList`1 records)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.ProcessMessageAsync(Boolean isRefresh)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.<StartMessageProcessing>b__20_0()
2024-10-21 03:35:09.7106|Error|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|205|Problems Summarizing record
System.NullReferenceException: Object reference not set to an instance of an object.
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSTable..ctor(TSTable dataTable)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSBars.SummarizePartialBar(DateTime datetime, IDerivationRules derivatives)
   at Refinitiv.Data.Content.HistoricalPricing.IntradaySummarizer.SummarizeRecords(IList`1 records)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.ProcessMessageAsync(Boolean isRefresh)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.<StartMessageProcessing>b__20_0()
2024-10-21 03:35:09.7106|Error|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|205|Problems Summarizing record
System.NullReferenceException: Object reference not set to an instance of an object.
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSTable..ctor(TSTable dataTable)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSBars.SummarizePartialBar(DateTime datetime, IDerivationRules derivatives)
   at Refinitiv.Data.Content.HistoricalPricing.IntradaySummarizer.SummarizeRecords(IList`1 records)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.ProcessMessageAsync(Boolean isRefresh)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.<StartMessageProcessing>b__20_0()
2024-10-21 03:35:09.7928|Error|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|90|Unexpected error processing TSI message
System.ArgumentException: Can not add property _SEQUENCE_NUMBER_ to Newtonsoft.Json.Linq.JObject. Property with the same name already exists on object.
   at Newtonsoft.Json.Linq.JContainer.InsertItem(Int32 index, JToken item, Boolean skipParentCheck, Boolean copyAnnotations)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.<StartMessageProcessing>b__20_0()
2024-10-21 03:35:09.7928|Info|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|90|Stopped TSI Message Processing...
2024-10-21 03:35:09.7996|Error|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|119|Unexpected error processing TSI message
System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at Newtonsoft.Json.Linq.JObject.GetEnumerator()+MoveNext()
   at Newtonsoft.Json.Linq.JObject.MergeItem(Object content, JsonMergeSettings settings)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.<StartMessageProcessing>b__20_0()
2024-10-21 03:35:09.7996|Info|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|119|Stopped TSI Message Processing...
2024-10-21 03:35:11.1755|Error|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|115|Problems Summarizing record
System.NullReferenceException: Object reference not set to an instance of an object.
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSTable..ctor(TSTable dataTable)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.TSBars.SummarizePartialBar(DateTime datetime, IDerivationRules derivatives)
   at Refinitiv.Data.Content.HistoricalPricing.IntradaySummarizer.SummarizeRecords(IList`1 records)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.ProcessMessageAsync(Boolean isRefresh)
   at Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer.<StartMessageProcessing>b__20_0()
2024-10-21 03:35:11.1755|Error|Refinitiv.Data.Content.HistoricalPricing.TSI.Summarizer|158|Problems Summarizing record


workspace-data-api#technology#productc#
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

3 Answers

· Write an Answer
Upvotes
nick.zincone avatar image
19.1k ●86 ●39 ●63

Hi @arek.majka

Thanks for the feedback. We are very close to releasing a new version of the library - it will be rebranded to LSEG, along with the namespace changes. There has been work done on the Historical streaming side related to memory and performance which may help with your issue.

It will be helpful if you provide some more details such as:

  • how many historical streaming instruments are you running simultaneously when this occurred?
  • is this behavior easily repeatable?
  • does it only occur at specific times in the trading day? Market open and/or market close?

Also, can you clarify that you left Workspace running, stopped and started your .Net application and it failed to start a session? It would be helpful if you can enable trace logs to better understand what may be happening and why.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
arek.majka avatar image
1 ●0 ●0 ●1

Hi Nick,


Thanks for the update.


I was streaming a few hundred markets. Half of them daily, half of them one minute bars.

Yes it is repeatable, but it doesn't happen right away so it's still fairly random in that regard. I started the session Sunday before any markets were open, and it lasted until about the middle of the night / morning when trading activity increased. As you can guess, just hard to test when something works for 12 hours then stops.


>you left Workspace running, stopped and started your .Net application and it failed to start a session

Yes exactly, nothing would reconnect to the desktop session until I restarted Workspace.


>It would be helpful if you can enable trace logs to better understand what may be happening and why.

How would you like me to do this?



icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

nick.zincone avatar image nick.zincone

Hi @arek.majka

Thanks for the clarification. I've asked the development team to review your findings as it does appear to be related to market activity and given you have many active instruments running simultaneously, this was something that was discovered and potentially addressed.

Regarding your issues connecting, I noticed you used the word "reconnect", which I believe implies that you did not stop the application but actually disconnected and failed to reconnect. Do I have that right? The 1st log output you have above is the startup, not the reconnect - I just want to make sure we're consistent with terminology here. Reconnect is when it loses a connect and tries to recover/reconnect. I see no logs above showing this.

If you want to turn on log tracing, you can refer to either example here: Example.DataLibrary.DotNet/src/4. Misc/4.0-Log at main · LSEG-API-Samples/Example.DataLibrary.DotNet (github.com)

Controlling logging can be done via NLog configuration or with code. Within code is easiest as it is a quick test and really 1 line of code. Doing it via configuration is more flexible and useful when you can't change source code.

Upvotes
arek.majka avatar image
1 ●0 ●0 ●1

When I said reconnect.. I mean I fully stopped all of my applications and restarted them, and could not connect to Workspace via the API at all. I had to fully close Workspace and restart it before I could connect again to the API via my applications.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

nick.zincone avatar image nick.zincone
Hi @arek.majka

Thanks for the clarification. In this specific case, it is not related to your application that is using the .Net library, but rather the state of Workspace. The only logical explanation I have is that Workspace will log the user out on the weekends which requires a login. If you're convinced Workspace was logged in and operating perfectly but when you restarted your application it would not connect, then there may have been some other issue with Workspace - we can further troubleshoot, but if you can enable logs and try to recreate this scenario, we could provide some more clues.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.