We have a client : tried to download tick data from datascope for below RICs:
We are using the C# datascope select SDK call for the download. The problem we encounter is: when we run more than one download concurrently, e.g., download 2 days of data at the same time, randomly data of some of the days are missing in the downloaded files. I attach a log file Concurrent_download.log Concurrent_download.txt to explain this in details.
The log shows that we wanted to download 4 days of above data, we run the C# downloader to open 4 connections in parallel each for a day. All of them were downloaded, the notes show correct tick count and everything looks fine. But in fact out of these 4 days, only one day's downloaded file contains full data. The rest 3 contains some data but miss most of it. Our log file shows 3 exceptions were thrown during writing the file to disk: One or more errors occurred. (Error while copying content to a stream.) This sounds like an exception thrown from HttpClient of .NET.
When we download these 4 days of data individually with single connection, all data can be downloaded successfully. It iindicates to us that we have problem with concurrent connection with datascope select. However this only happen with relatively bigger tick data, we use same C# program to download small daily data with 4 concurrent connections, we don't observe similar problem.