For a deeper look into our DataScope Select REST API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

question

Upvote
amulligan avatar image
31 ●1 ●1 ●5

Datascope, ReportExtractions response containing min values

Hi,

We are using the DSS Reuters Rest API to download rates file. We have seen that 'LastWriteTimeUtc' in the result of a call to 'report extractions' has come back with the min value of (01/01/0001 00:00:00 +00:00) for DatetimeOffset for 'LastWriteTimeUtc' for both the .notes and .xml files.

We have seen from the extracted files notes that it took over 4 minutes for processing to be completed.

In a case like this, where files take a long time to be processed, do Reuters populate fields like 'LastWriteTimeUtc' with default values?

-------------------------------------

Edit:

We are making the following calls to download the content of a rates xml file:

1. Authentication:

POST https://hosted.datascopeapi.reuters.com/RestApi/v1/Authentication/RequestToken

We pass in credentials and receive a token back which we pass in in all subsequent calls.

2. LastExtraction:

GET https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/Schedules('0x111111111111')/LastExtraction

We pass in appropriate schedule id for the given file and we receive a response, from which we take the report extraction id to use in the next call.

3. Files:

GET https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ReportExtractions('111111')/Files

We pass in the extraction id from previous call. We receive the files data associated with last extraction. The files are: the rates file and notes. We spool through the file data and get the xml file id for the full rates xml file.capture1.png

4. Value:

GET https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractedFiles('V111111111=')/$value

We pass in the extracted file id from the previous call. We download the content of the file.

We have noticed on a few occasions that the response we received from the call number 3 (Files) contained fields with default values. One of them was LastWriteTimeUtc, which was set to "01/01/0001 00:00:00 +00:00". We were wondering what can be the cause.

Unfortunately we can't provide any notes as they were overridden by the next extraction and we don't store them. Please let me know if you need any other information.

dss-rest-apidatascope-selectdssrest-api
capture1.png (20.3 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

veerapath.rungruengrayubkul avatar image   veerapath.rungruengrayubkul

Hi @amulligan,

Could you please give the information requested by Rick? It is required to answer your question.

Thanks & Regards,

AHS

sunil.mahale avatar image   sunil.mahale

Hi @amulligan

Thank you for your participation in the forum.

Are any of the replies below satisfactory in resolving your query?

If yes please click the 'Accept' text next to the most appropriate reply. This will guide all community members who have a similar question.

Otherwise please post again offering further insight into your question.

Thanks,
AHS

Christiaan Meihsl avatar image   Christiaan Meihsl

This discussion was continued outside of the forum, via emails and calls.

7 Answers

· Write an Answer
Upvotes
Rick Weyrauch avatar image
1.1k ●5 ●3 ●5

@amulligan

Is there any chance I can get some captures of the requests you are making for this case?

I want to make sure I understand the use case and timing you are involved in.

Can you provide the notes for the extraction?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvote
amulligan avatar image
31 ●1 ●1 ●5
Edited question @ Rick Weyrauch
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Rick Weyrauch avatar image Rick Weyrauch

You can always email me directly at rick.weyrauch@thomsonreuters.com with specific request data (so you do not have to share confidential data here on the forums).

Upvote
Rick Weyrauch avatar image
1.1k ●5 ●3 ●5

I see that this file info data is a bit of a "feature in flight" where there is a legacy model and a new model where by we can always provide this data on each generated file. The problem is that most of the file results being returned today use the legacy model which cannot always fully populate this data (why we needed a new model).

For the time being, this data may be of limited use and ignored. I will have to check with Product Management about when we can be expected to be fully converted to the new model and always have this data provided.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Sridevi P avatar image
251 ●1 ●2 ●2

Hi, Rick asked the product team to review this request. We do not plan to make the change to the converted model anytime soon in 2016. We will try to prioritize the change in our backlog for 2017. As Rick mentioned, this data may be of limited use and can be ignored for now.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
amulligan avatar image
31 ●1 ●1 ●5

Hi @Rick Weyrauch @Sridevi P

The LastWriteTimeUtc and ReceivedDateUTC are of no great importance to us so these can be ignored as you have said but the size field is. We have seen that when size is '0' we do not receive any file in the final (call 4 see original question).

@Rick Weyrauch you state in your reply that "The problem is that most of the file results being returned today use the legacy model which cannot always fully populate this data (why we needed a new model)." Can you confirm that we are receiving both models and on any given day we can be returned a legacy model which could cause no file to be returned?

We are using 3 different schedules which all point to the creation of the same xml rates file. Yesterday when using call 3(see original question) for these 3 schedules we saw that one of the scheduleIds was returning a legacy model object and no associated rates file while the other 2 are returning the correct objects with an associated rates file. Is this expected behaviour?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Rick Weyrauch avatar image
1.1k ●5 ●3 ●5

You stated "Unfortunately we can't provide any notes as they were overridden by the next extraction and we don't store them." but I need to stress the importance of the notes. The data contained in the notes are crucial to identifying the proper extraction to review when there are questions about the extraction run of the data it returned. Notes are "your receipt" and so are recommended to be save for at least a few days, or as long as it take your system to raise all concerns.

Alternately, you can review the diagnostic header section of the help for other identifiers that your system can record for these inquires.

Would it be possible for you to add some logging and then provide specific identifiers or notes for each "good" and "bad" request you are commenting on?

In the meantime, I am looking into why you would ever get back a file reference where the .ContentsExist property would be true while at the same time the .Size is 0 and there are no contents.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

amulligan avatar image amulligan

Hi @Rick Weyrauch,

Has there been any update on this question?

Upvotes
Rick Weyrauch avatar image
1.1k ●5 ●3 ●5

This was sent to the poster via email...

Ok, so while we can consider that we have a bug here, it’s not really a REST API exactly. It is an artifact of our old FTP roots. This would not have ended well for an pure FTP client either.

You have 3 schedules that all use the same output file name – without any %D %T macro’s to make each of them a unique filename.

So, as these run, they stomp on each other so to speak as they each take turns updating the one file in your Reports directory. In the GUI this manifests itself as “you only get 1 file no matter which schedule you are using.” That is, all three have file contents, but it is only correct for one of them. The REST API tries to solve that by only associating the file that does exist with the extraction that it goes with. It’s not a perfect algorithm, but as long as we do not have this configuration it works pretty well.

That is, in a way, the answer you got back was perfectly correct, the file data for [some extractions] is no longer available even though the record of it happening is available.

If the client sets up each schedule to use unique output filenames, this artifact should go away and they always get the file contents they are expecting.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.