We have a scheduled jobs that extracts the trade and market information from Reuter. We are using data scope api to fetch trade/market (tickhistory/endofday )data based on the instrument list created.
We are looking on data consistency checks and ways in which we can confirm the data consumed in REST responses are in their totality which will help us in identifying any miss during the extraction process. Could you please advise us on below two points.
Is there any way we can get the total record counts we are extracting with the response? We have looked into the “Notes” received in REST response but not able to find total record count.
Is there any kind of checksum field with the extraction request which can be used to verify the file downloaded ?
This will help us in validating the extracted file against the Reuter sources. Please let us know if required any inputs from our side.
There is no checksum functionality, but there is one for the total records count.
When using scheduled jobs, you can add a RowCount line to the header (or trailer) of the returned data file. This will add a line in the returned data, containing a number corresponding to the number of rows of data you should receive. This option is defined in the extraction template. Below is a screenshot showing how it is done in the GUI: