Slow processing of on-demand PriceHistory requests (up to 7 hours)
Hi Forum!
The forum looks very active, looking forward to learn from you guys! :)
We are loading terms and conditions as well as price time series with 6 fields for around 5000 instruments with the on-demand TermsAndConditions and PriceHistory requests. We try to adhere to the limit of 50 concurrent PriceHistory requests, as laid out in the manual. However, often our PriceHistory requests end up being stuck in the queue at Datascope, taking several hours to complete (up to 7 hours). In the notes we can see the actual processing times of the PriceHistory requests, which are in the range of seconds.
Does anybody have an idea why our requests get stuck in the queue at Datascope, and can help us in answering our questions:
- Why do our requests get stuck in the queue?
- How does the scheduling algorithm of Datascope work?
- How can we optimize our requests such that they won't get stuck in the queue?
Hello @quantE,
These are excellent questions.
I am afraid there is no all-encompassing answers that I can think of.
I also understand that this happens for some, few, of the requests that you issue.
Please note that if you submit full 50 allowed Price History requests at a time, and some may be large, only 2 are taken up for processing, while the rest 48 remain in the queue till the first 2 complete processing, please see more info in DSS Best Practices section Extraction Limits for more details on processing. I.e. if you have one or two request that are very large, they may hold up the remaining 48 on queue till completed. So I think there is a chance your requests may be queued long, rather then processed long.
Please try to analyze your submission timeline and see if you can better understand the pattern to those requests that take long?
If not, are you able to test the same PriceHistory requests, submitting 2 at a time, till completed and see if this takes care of the timing discrepancies?
If not, then for the specific request that has taken 7 hours, I would suggest, as a customer, to investigate this specific request. When this happens, to collect full info on when it was submitted, when the status changed to completed, on your side, the jobId, extraction notes if available, and open a support case with with DSS product support with Refinitivi Helpdesk Online to investigate the specific case.
Please let us know if this information helps you proceed.