Requesting details on Datascope market and trade data extracts
We have a scheduled jobs that extracts the trade and market information from Reuter. We are using data scope api to fetch trade/market (tickhistory/endofday )data based on the instrument list created.
We are looking on data consistency checks and ways in which we can confirm the data consumed in REST responses are in their totality which will help us in identifying any miss during the extraction process. Could you please advise us on below two points.
Is there any way we can get the total record counts we are extracting with the response? We have looked into the “Notes” received in REST response but not able to find total record count.
Is there any kind of checksum field with the extraction request which can be used to verify the file downloaded ?
This will help us in validating the extracted file against the Reuter sources. Please let us know if required any inputs from our side.
There is no checksum functionality, but there is one for the total records count.
When using scheduled jobs, you can add a RowCount line to the header (or trailer) of the returned data file. This will add a line in the returned data, containing a number corresponding to the number of rows of data you should receive. This option is defined in the extraction template. Below is a screenshot showing how it is done in the GUI:
Thank You Christiaan Meihsl,
Is this possible for TickHistory extracts also? I am struggling to find that.
I just checked and am sorry to say that for the 4 TRTH templates this feature is not available.
That said, the only time I saw data missing from the downloaded results was when the issue described in this advisory occurred, so if the code is written taking the precautions outlined in the advisory, the download should be complete. Our code samples illustrate how this is done.
Hope this helps.