Is there any data size limitation on responded data? If I request 10 minute bar with about 200 symbols but long date range like 20 years. What's the best way to do? If I break my request either with symbol or date range, can I use the same token.
2
●5 ●4 ●9
TRTH intraday summary
Hi,
Thank you for your participation in the forum.
Are any of the replies below satisfactory in resolving your query?
If yes please click the 'Accept' text next to the most appropriate reply. This will guide all community members who have a similar question.
Otherwise please post again offering further insight into your question.
Thanks,
AHS
Hi @jim.yu
Thank you for your participation in the forum.
Are any of the replies below satisfactory in resolving your query?
If yes please click the 'Accept' text next to the most appropriate reply. This will guide all community members who have a similar question.
Otherwise please post again offering further insight into your question.
Thanks,
AHS
13.7k
●26 ●8 ●12
@jim.yu,
There is no data size limit with the REST API, you can request 10 minute bars for 200 RICs for 20 years. You can also break that down into several requests, if you prefer, but you will get better performance making a few large requests. Making many small requests is not a good practice, they have more overhead.
Beware of instrument expansion: if you use ISINs, CUSIPs or SEDOLs in an intraday summary request, the result will contain data for all RICs that match those identifier codes (an ISIN can correspond to many RICs, the same applies to CUSIPs and SEDOLs), you could end up with several thousand instruments. That is why we recommend converting ISINs to RICs before making the data requests.
If you make large requests, it is better to save the returned (compressed) data to disk first, and then treat it, instead of decompressing the data on the fly. More information on this topic in this advisory.
Large requests benefit from AWS (Amazon Web Services) downloads, for more information see this article and this advisory.
As to the authentication token, we recommend you reuse it, which is more efficient than requesting a new one for each request.
The best practices document will give you more detail on these topics.
4.4k
●10 ●6 ●9
There is no data size limit, but there are extraction limits. See BEST PRACTICES AND LIMITS FOR TICK HISTORY.
You can extract up to 5,000,000 RIC-days in requests that are running at any one time.
RIC-days is the total number of days multiplied by total number of RICs. For example, 200 RICs with 20 years date range would equal to 200*(365*20) = 1,460,000 RIC-days.
You don't have to request a new token for each request. An authentication token remains valid for 24 hours. You can reuse a token as often as you like within that 24-hour period.
79.2k
●251 ●52 ●74
If you perform the extraction of 200 symbols for 20 years, the file size may be very huge.
You may have a problem while downloading the result file. Although you can download the file via AWS, it may take several hours to complete the download. Moreover, the connection may be disconnected during the download. Therefore, if the application doesn't implement Range request, it means that the application needs to start downloading the whole file again. In addition, the AWS link can also be expired so you may need to re-generate the new AWS link.
My suggestion is using 200 items with yearly or quarterly period instead. Therefore, you will not have a problem while downloading the result files. However, if your internet is very fast, you can extract all required data at once.
