Datascope, ReportExtractions response containing min values

amulligan

Hi,

We are using the DSS Reuters Rest API to download rates file. We have seen that 'LastWriteTimeUtc' in the result of a call to 'report extractions' has come back with the min value of (01/01/0001 00:00:00 +00:00) for DatetimeOffset for 'LastWriteTimeUtc' for both the .notes and .xml files.

We have seen from the extracted files notes that it took over 4 minutes for processing to be completed.

In a case like this, where files take a long time to be processed, do Reuters populate fields like 'LastWriteTimeUtc' with default values?

-------------------------------------

Edit:

We are making the following calls to download the content of a rates xml file:

1. Authentication:

POST https://hosted.datascopeapi.reuters.com/RestApi/v1/Authentication/RequestToken

We pass in credentials and receive a token back which we pass in in all subsequent calls.

2. LastExtraction:

GET https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/Schedules('0x111111111111')/LastExtraction

We pass in appropriate schedule id for the given file and we receive a response, from which we take the report extraction id to use in the next call.

3. Files:

GET https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ReportExtractions('111111')/Files

We pass in the extraction id from previous call. We receive the files data associated with last extraction. The files are: the rates file and notes. We spool through the file data and get the xml file id for the full rates xml file.capture1.png

4. Value:

GET https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractedFiles('V111111111=')/$value

We pass in the extracted file id from the previous call. We download the content of the file.

We have noticed on a few occasions that the response we received from the call number 3 (Files) contained fields with default values. One of them was LastWriteTimeUtc, which was set to "01/01/0001 00:00:00 +00:00". We were wondering what can be the cause.

Unfortunately we can't provide any notes as they were overridden by the next extraction and we don't store them. Please let me know if you need any other information.

Rick Weyrauch

@amulligan

Is there any chance I can get some captures of the requests you are making for this case?

I want to make sure I understand the use case and timing you are involved in.

Can you provide the notes for the extraction?

amulligan

Rick Weyrauch

Rick Weyrauch

https://community.developers.refinitiv.com/discussion/comment/9522#Comment_9522

You can always email me directly at rick.weyrauch@thomsonreuters.com with specific request data (so you do not have to share confidential data here on the forums).

Rick Weyrauch

I see that this file info data is a bit of a "feature in flight" where there is a legacy model and a new model where by we can always provide this data on each generated file. The problem is that most of the file results being returned today use the legacy model which cannot always fully populate this data (why we needed a new model).

For the time being, this data may be of limited use and ignored. I will have to check with Product Management about when we can be expected to be fully converted to the new model and always have this data provided.

Sridevi P

Hi, Rick asked the product team to review this request. We do not plan to make the change to the converted model anytime soon in 2016. We will try to prioritize the change in our backlog for 2017. As Rick mentioned, this data may be of limited use and can be ignored for now.

amulligan

@Rick Weyrauch

The LastWriteTimeUtc and ReceivedDateUTC are of no great importance to us so these can be ignored as you have said but the size field is. We have seen that when size is '0' we do not receive any file in the final (call 4 see original question).

@Rick Weyrauch you state in your reply that "The problem is that most of the file results being returned today use the legacy model which cannot always fully populate this data (why we needed a new model)." Can you confirm that we are receiving both models and on any given day we can be returned a legacy model which could cause no file to be returned?

We are using 3 different schedules which all point to the creation of the same xml rates file. Yesterday when using call 3(see original question) for these 3 schedules we saw that one of the scheduleIds was returning a legacy model object and no associated rates file while the other 2 are returning the correct objects with an associated rates file. Is this expected behaviour?

Rick Weyrauch

You stated "Unfortunately we can't provide any notes as they were overridden by the next extraction and we don't store them." but I need to stress the importance of the notes. The data contained in the notes are crucial to identifying the proper extraction to review when there are questions about the extraction run of the data it returned. Notes are "your receipt" and so are recommended to be save for at least a few days, or as long as it take your system to raise all concerns.

Alternately, you can review the diagnostic header section of the help for other identifiers that your system can record for these inquires.

Would it be possible for you to add some logging and then provide specific identifiers or notes for each "good" and "bad" request you are commenting on?

In the meantime, I am looking into why you would ever get back a file reference where the .ContentsExist property would be true while at the same time the .Size is 0 and there are no contents.

amulligan

https://community.developers.refinitiv.com/discussion/comment/9569#Comment_9569

Hi @Rick Weyrauch,

Has there been any update on this question?

Rick Weyrauch

This was sent to the poster via email...

Ok, so while we can consider
that we have a bug here, it’s not really a REST API exactly. It is an artifact
of our old FTP roots. This would not have ended well for an pure FTP client
either.

You have 3 schedules that all
use the same output file name – without any %D %T macro’s to make each of them
a unique filename.

So, as these run, they stomp on
each other so to speak as they each take turns updating the one file in your
Reports directory. In the GUI this manifests itself as “you only get 1 file no
matter which schedule you are using.” That is, all three have file contents,
but it is only correct for one of them. The REST API tries to solve that by
only associating the file that does exist with the extraction that it goes
with. It’s not a perfect algorithm, but as long as we do not have this
configuration it works pretty well.

That is, in a way, the answer
you got back was perfectly correct, the file data for [some extractions] is
no longer available even though the record of it happening is available.

If the client sets up each schedule to use unique output filenames, this
artifact should go away and they always get the file contents they are
expecting.