How to download 'up to the current minute' historical data?
At the moment I use the Datascope rest api to retrieve historical data from which I can build models. I have the elektron websocket api set up to receive realtime data and use a combination of these to make market predictions. I have the realtime stream running through the night on most weeknights and the issue I have is that sometimes there are problems at my end, ranging from poor connectivity to poor coding on my part, that mean the machine breaks down.
I would like to be able to just do a bulk download to catch up the data that I may have missed for that day whilst I've been asleep but my historic downloads are about 5 hours behind realtime and I don't think it's possible to use the realtime api to go back in time. I'm assuming that there must be some way of using Datascope to do this but I haven't been able to find it.
Any help on this would be great.
Thank you!
The API that meets your requirement is Refinitiv Tick History (RTH) - REST API.
You can use it to extract the raw data which is quite similar to the data (field list) retrieved from the WebSocket API. Please refer to the REST API Tutorial 8: On Demand raw data extraction.
The raw data looks like the following:
#RIC,Domain,Date-Time,GMT Offset,Type,MsgClass/FID number,UpdateType/Action,FID Name,FID Value,FID Enum String,PE Code,Template Number,Key/Msg Sequence Number,Alias Underlying RIC,Number of FIDs CARR.PA,Market Price,2016-09-29T10:00:00.258628004Z,+2,Raw,UPDATE,QUOTE,,,,8132,,45808,,14 ,,,,FID,25,,ASK,23.31, ,,,,FID,6580,,ASK_COND_N,0, ,,,,FID,31,,ASKSIZE,1828, ,,,,FID,4147,,ASK_TIM_MS,36000210, ,,,,FID,346,,ASK_TONE," ", ,,,,FID,3867,,AUC_ASK,, ,,,,FID,3864,,AUC_ASKSIZ,, ,,,,FID,6516,,BOOK_STATE,1,N ,,,,FID,134,,MID_PRICE,23.3075, ,,,,FID,292,,NO_ASKMMKR,6, ,,,,FID,6554,,NO_ASKORD1,6, ,,,,FID,1025,,QUOTIM,10:00:00.000000000, ,,,,FID,3855,,QUOTIM_MS,36000210, ,,,,FID,5,,TIMACT,10:00:00.000000000,
Typically, today's data will be available two hours after the market close.
I don't think that this data is available in the DSS report templates. You can contact the DataScope support team via MyRefinitiv for confirmation.
Thanks for your reply. I'm already using the RTH - REST API I think. I checked the documentation you referred to and this is exactly the code I'm using which gives me historic data but not up to my current minute, about 5 hours late.
If I am to go to work in the morning and find that there has been some sort of issue that means my Websocket API has been unable to download the last couple of hours of data, is there some way I can get those last couple of hours worth of data in a single historic download up to my current minute. The RTH - REST API doesn't seem to be able to give me the data up to the current minute, it's about 5 hours behind.
Thanks for your time,
and your patience!
It was mentioned in the Tick History User Guide that Tick History data is made available in a near real-time framework with data for most instruments available for download a few hours after a market has closed. Most data is available two hours post venue close time. Therefore, you need to wait for two hours after the market before extracting the missing real-time data.
If you need to get historical data in the raw format (field list), as I know, only Refinitiv Tick History can provide it.
You may need to contact your Refinitiv account representative for more information.