This is in reference to support case 07985604. I believe @alex.putkov has looked at some of our logs before in relation to this issue.
In short, while streaming data for a large number of US equities (roughly top 1000 liquid) using the 32-bit Eikon .NET API during the US trading day, we sometimes get intermittent pauses with 0 updates, each of which lasts up to 30 seconds. When this occurs, it tends to recur repeatedly until we either restart Eikon or are remotely disconnected by the server for a bad connection. Usually (but not always) restarting the Eikon application fixes the issue. Restarting an individual Eikon API process does not fix the issue.
We have reviewed with the ADS team and they found the following messages on their servers:
RSSL disconnect from "SL1-5XCN6JG" at position "0.0.0.0/SL1-5XCN6JG" on host "CC-NY-RT01" using application "56" on channel 2978. Reason: rsslWrite() returned flush failed and channel is not active: <Impl/ripcsrvr.c:9552> Error: 1002 ripcWrite() failed. System errno: (32)
RSSL disconnect from "SL1-5XCN6JG" at position "0.0.0.0/SL1-5XCN6JG" on host "CC-NY-RT01" using application "56" on channel 2696. Reason: User SL1-5XCN6JG at position 0.0.0.0/SL1-5XCN6JG on host CC-NY-RT01 using application 56 has been disconnected due to max update delay time being exceeded.
ADS indicated that this is consistent with a slow consumer, though it seems anything breaking the connection between the client and server would be consistent with the above messages. However, we have monitored the network connections out of the machine and everything is bypassing all middleboxes, and we are not seeing any packet drops.
We have no RAM issues on this machine; on average, system memory is less than 30% utilized. CPU utilization also does not spike during such quote interruptions. However, after doing a further review of our system logs, during periods we have this problem there is a very high number of soft (demand zero and transition) page faults, relative to when the issue is not occurring. What we're wondering: is it possible these pauses have something to do with .NET garbage collection, either in our app, in the DLL our APIs call, or in the upstream Eikon system? If so, is it possible we would gain a benefit from converting from 32-bit Eikon API to the 64-bit API? Is there anything else you are aware of that would cause an unusually large number of soft page faults?
