Intra day pricing Timeouts

Apologies for the delay, the investigation took some time. I also asked the product team to double check the conclusions outlined below. They recommend using On Demand extractions, as I describe below. They do not recommend using an immediate schedule.

Intraday On Demand request (recommended)

With an On Demand request it is not possible to retrieve partial deliveries (those that are not embargoed) before those that are embargoed.

Here is the way to do it using On Demand requests:

Make a first request, setting OnlyNonEmbargoedData to true, to receive the non embargoed data.
Compare the list of received instruments to those in the original request, and then make a second query for the missing instruments (without setting OnlyNonEmbargoedData), this will retrieve the embargoed ones.

Caveat: if your instrument list contains instruments that have different embargo delays, step 2 will wait for the longest embargo delay before delivering all embargoed data.

Another solution would be to sort instruments in advance, to directly make separate requests (one for non-embargoed data, and other calls (one per embargo delay) for embargoed data). As you know your product permissions, you could create a mapping table using the ISIN's first 2 letters to identify the venue (if you were using RICs you could use the RIC extension). You can also easily find the embargo delays per venue in the extraction notes file, so once you have run an extraction for all venues you should have all the required information to sort instruments to make your requests.

Immediate schedule (not recommended)

I found a way to do it with an immediate schedule, and created a new Java sample to illustrate it (DSS2ImmediateScheduleIntraDay). You can find its description here, and download it in the latest set of Java samples from the DSS downloads tab.

Here is the workflow:

0. Create an instrument list and report template.

1. Create an immediate schedule for Intra Day data

2. Request the Next Extraction (/Extractions/Schedules('reportScheduleId')/NextExtraction).

3. If step 2 returns HTTP status 204 there is no next extraction, i.e. the extraction is complete (no embargo). In that case:

3.1. Request the Last extraction (/Extractions/Schedules('reportScheduleId')/LastExtraction), to get the ReportExtractionId.

3.2. Use that Id to retrieve all the files (/Extractions/ReportExtractions('reportExtractionId')/Files).

3.3 Task is complete, exit here.

4. If step 2 returns an HTTP status 200 then it is still processing (embargo or extraction taking more than 30 sec). In that case:

4.1. extract the ReportExtractionId and Status from the response.

4.2. Enter a while loop to:

4.2.1. List files that might be available (typically: notes, non-embargoed data).

4.2.2. Download those that were not already downloaded (select them by checking the names)

4.2.3. Wait a minute or more. Important: do not wait less than 1 minute, it would make no sense as embargoes last several minutes, and you don't want to put unnecessary load on the servers.

4.2.4. Request the Next Extraction

4.2.5. Exit the loop if Next Extraction returns a 204 (which means the extraction has completed).

4.3. List available files (this is the final complete list).

4.4. Download those that were not already downloaded (select them by checking the names) (typically: embargoed data)

4.5 Task is complete, exit here.

Final note: the instrument list, report template and schedule can be destroyed after all files were retrieved.

All comments

If you are getting a timeout on an intraday request, it could be due to an embargo. Have you looked at the DSS REST Tutorial on Intraday extractions ? It details the workflow and calls, and contains a link to the DSS .Net Tutorial 4 which gives details on embargoes and how they are handled. In a nutshell, if you are not entitled to real-time data for 1 or more of the instruments you are requesting, then you will have to wait for a (venue dependent) embargo period before the data can be retrieved. This could be anywhere from 15 minutes to several hours.

That said, your log file intrigues me, because it seems to indicate that the URL used is "https://hosted.datascopeapi.reuters.com.Extract:Request 1", which is an invalid URL. But maybe my interpretation of the log file is erroneous, it is difficult to say.

I guess you are using Java. We have a working Java sample for intraday requests, called DSS2OnDemandIntraday. It can be downloaded here. You could compare your code to that one, which might help find the issue.

If the information above does not suffice, then for us to be able to help you more we'd need you to give us details on your request, i.e. post the following information:

URL
Method (POST or GET)
Headers
Body (if POST)

What would be even better would be to post your entire code as an attachment, but please remove your username and password from the code before posting it.

Hi Christiaan,

Thank you for your response. I referred to the links and now understand the concept of embargoes.

Also I tried to test the sample java code from DSS2OnDemandIntraday.java. I noticed that the process just hangs after the second check after a status of 202 Accepted in case of embargoed data.
So I tried using the OnlyNonEmbargoedData condition and it returned me the data but the notes did not mention where I can look for the remaining identifiers. So basically via the REST API how do I get an intermediate data pull(nonEmbargoed) and then read the notes so that I can again access the 'Location URI' for the rest of the embargoed data.

Please let me know if this can be done. Thank you. I tried searching for it in the API Tree but could not find this functionality.

This is selected on the DSS GUI

Trying to get all data, this never returns:

With .put( "OnlyNonEmbargoedData", true)

Session Token (expires in 24 hours):

xxxxx

Intraday extraction JSON request content:

@{"ExtractionRequest":{"@odata.type":"#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.IntradayPricingExtractionRequest","ContentFieldNames":["Last Price","User Defined Identifier","Volume"],"IdentifierList":{"@odata.type":"#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList","InstrumentIdentifiers":[{"Identifier":"AT0000609607","IdentifierType":"Isin"},{"Identifier":"AT0000618251","IdentifierType":"Isin"},{"Identifier":"AT0000730007","IdentifierType":"Isin"},{"Identifier":"AT0000641352","IdentifierType":"Isin"},{"Identifier":"AT0000818802","IdentifierType":"Isin"},{"Identifier":"AT0000624705","IdentifierType":"Isin"},{"Identifier":"AT0000624739","IdentifierType":"Isin"},{"Identifier":"US0038301067","IdentifierType":"Isin"},{"Identifier":"AT0000644505","IdentifierType":"Isin"},{"Identifier":"BE0003764785","IdentifierType":"Isin"},{"Identifier":"AT0000723606","IdentifierType":"Isin"},{"Identifier":"AT0000728209","IdentifierType":"Isin"},{"Identifier":"AT0000938204","IdentifierType":"Isin"},{"Identifier":"CH0012488190","IdentifierType":"Isin"},{"Identifier":"AT0000625132","IdentifierType":"Isin"},{"Identifier":"CH0008837566","IdentifierType":"Isin"},{"Identifier":"CH0014420829","IdentifierType":"Isin"},{"Identifier":"US0378331005","IdentifierType":"Isin"}],"ValidationOptions":null,"UseUserPreferencesForValidationOptions":false},"Condition":{"ScalableCurrency":true}}}

We submit the request, a response should arrive in approximately 30 seconds ...

HTTP status: 202 - We must wait, and poll the location URL.

Location URL: https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractWithNotesResult(ExtractionId='0x06c454955770e802')

Waiting 30 seconds ..

HTTP status: 202

Waiting 30 seconds ..

with .put( "OnlyNonEmbargoedData", true). This returns but I would want to know how I can retrieve the embargoed data after the embargo expires. The notes do not mention this.

Session Token (expires in 24 hours):

xxxx

Intraday extraction JSON request content:

We submit the request, a response should arrive in approximately 30 seconds ...

HTTP status: 200 - A response is available now!

Result received:

@{"@odata.context":"https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.ExtractionResult","Contents":[{"IdentifierType":"Isin","Identifier":"AT0000609607","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000618251","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000730007","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000641352","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000818802","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000624705","Last Price":15.8,"Volume":600},{"IdentifierType":"Isin","Identifier":"AT0000624739","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"US0038301067","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000644505","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"BE0003764785","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000723606","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000728209","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000938204","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"AT0000625132","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"CH0008837566","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"CH0014420829","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"US0378331005","Last Price":null,"Volume":null},{"IdentifierType":"Isin","Identifier":"CH0012488190","Error":"Not found"}],"Notes":["Extraction Services Version 13.1.40889 (ec84d57d2aa3), Built Aug 9 2019 18:16:31\r\nProcessing started at 08/27/2019 14:45:30.\r\nUser ID: xxx\r\nExtraction ID: 381847685\r\nSchedule: _OnD_0x06c473d374a0e81c (ID = 0x06c473d37620e81c)\r\nInput List (17 items): _OnD_0x06c473d374a0e81c (ID = 06c473d37550e81c) Created: 08/27/2019 14:45:28 Last Modified: 08/27/2019 14:45:28\r\nSchedule Time: 08/27/2019 14:45:29\r\nReport Template (8 fields): _OnD_0x06c473d374a0e81c (ID = 0x06c473d374c0e81c) Created: 08/27/2019 14:45:28 Last Modified: 08/27/2019 14:45:28\r\nWARNING: No Pricing for instrument (ISN,AT0000618251,VIE,AEMV.VI^B17), segment 'E' due to instrument not being traded.\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 08/27/2019 14:45:30, it was scheduled to snap at 08/27/2019 14:45:29.\r\nContent delivery suppressed due to lack of permission for NAS1 (NASDAQ UTP) for quotes from NAS, NSM\r\nEmbargo delay of 15 minutes required by [ EN1 (EURONEXT CONTINENTAL CASH(CONSOLIDATED PACK) L 1), ENS (EURONEXT CONTINENTAL CASH(CONSOL PACK) LEVEL 2), ENPI (EURONEXT PRIVATE INVESTOR PACKAGE PREMIUM) ] for quotes from BRU\r\nEmbargo delay of 15 minutes required by [ VIE (WIENER BORSE CASH MKT STRUCTURED PROD L1), VI2 (WBAG CASH MKT STRUCTURED PROD L1 L2) ] for quotes from VIE\r\nEmbargo delay of 15 minutes required by [ VTX (SIX SWISS EXCHANGE LEVEL 1), VT2 (SIX SWISS EXCHANGE L1 L2), VTP (SIX SWISS EXCHANGE LEVEL 2 PLUS) ] for quotes from SWX\r\nEmbargo delays caused suppression of 10 instruments.\r\nProcessing completed successfully at 08/27/2019 14:45:30, taking 0.541 Secs.\r\nExtraction finished at 08/27/2019 13:45:30 UTC, with servers: x09i08, QSHC18 (0.0 secs), QSSHA1 (0.1 secs)\r\nUsage Summary for User xxx, Client 103424, Template Type Intraday Pricing\r\nBase Usage\r\n Instrument Instrument Terms Price\r\n Count Type Subtype Source Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n 1 Equities N/A N/A\r\n-------\r\n 1 Total instrument charged.\r\n 16 Instruments with no reported data.\r\n=======\r\n 17 Instruments in the input list.\r\nNo TRPS complex usage to report -- 17 Instruments in the input list had no reported data.\r\n"]}

JSON response:

@{"Contents":[{"Last Price":null,"Identifier":"AT0000609607","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000618251","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000730007","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000641352","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000818802","Volume":null,"IdentifierType":"Isin"},{"Last Price":15.8,"Identifier":"AT0000624705","Volume":600,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000624739","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"US0038301067","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000644505","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"BE0003764785","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000723606","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000728209","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000938204","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000625132","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"CH0008837566","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"CH0014420829","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"US0378331005","Volume":null,"IdentifierType":"Isin"},{"Identifier":"CH0012488190","Error":"Not found","IdentifierType":"Isin"}],"@odata.context":"https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.ExtractionResult","Notes":["Extraction Services Version 13.1.40889 (ec84d57d2aa3), Built Aug 9 2019 18:16:31\r\nProcessing started at 08/27/2019 14:45:30.\r\nUser ID: xxx\r\nExtraction ID: 381847685\r\nSchedule: _OnD_0x06c473d374a0e81c (ID = 0x06c473d37620e81c)\r\nInput List (17 items): _OnD_0x06c473d374a0e81c (ID = 06c473d37550e81c) Created: 08/27/2019 14:45:28 Last Modified: 08/27/2019 14:45:28\r\nSchedule Time: 08/27/2019 14:45:29\r\nReport Template (8 fields): _OnD_0x06c473d374a0e81c (ID = 0x06c473d374c0e81c) Created: 08/27/2019 14:45:28 Last Modified: 08/27/2019 14:45:28\r\nWARNING: No Pricing for instrument (ISN,AT0000618251,VIE,AEMV.VI^B17), segment 'E' due to instrument not being traded.\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 08/27/2019 14:45:30, it was scheduled to snap at 08/27/2019 14:45:29.\r\nContent delivery suppressed due to lack of permission for NAS1 (NASDAQ UTP) for quotes from NAS, NSM\r\nEmbargo delay of 15 minutes required by [ EN1 (EURONEXT CONTINENTAL CASH(CONSOLIDATED PACK) L 1), ENS (EURONEXT CONTINENTAL CASH(CONSOL PACK) LEVEL 2), ENPI (EURONEXT PRIVATE INVESTOR PACKAGE PREMIUM) ] for quotes from BRU\r\nEmbargo delay of 15 minutes required by [ VIE (WIENER BORSE CASH MKT STRUCTURED PROD L1), VI2 (WBAG CASH MKT STRUCTURED PROD L1 L2) ] for quotes from VIE\r\nEmbargo delay of 15 minutes required by [ VTX (SIX SWISS EXCHANGE LEVEL 1), VT2 (SIX SWISS EXCHANGE L1 L2), VTP (SIX SWISS EXCHANGE LEVEL 2 PLUS) ] for quotes from SWX\r\nEmbargo delays caused suppression of 10 instruments.\r\nProcessing completed successfully at 08/27/2019 14:45:30, taking 0.541 Secs.\r\nExtraction finished at 08/27/2019 13:45:30 UTC, with servers: x09i08, QSHC18 (0.0 secs), QSSHA1 (0.1 secs)\r\nUsage Summary for User xxx, Client 103424, Template Type Intraday Pricing\r\nBase Usage\r\n Instrument Instrument Terms Price\r\n Count Type Subtype Source Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n 1 Equities N/A N/A\r\n-------\r\n 1 Total instrument charged.\r\n 16 Instruments with no reported data.\r\n=======\r\n 17 Instruments in the input list.\r\nNo TRPS complex usage to report -- 17 Instruments in the input list had no reported data.\r\n"]}

Notes extracted from JSON response:

["Extraction Services Version 13.1.40889 (ec84d57d2aa3), Built Aug 9 2019 18:16:31\r\nProcessing started at 08/27/2019 14:45:30.\r\nUser ID: xxx\r\nExtraction ID: 381847685\r\nSchedule: _OnD_0x06c473d374a0e81c (ID = 0x06c473d37620e81c)\r\nInput List (17 items): _OnD_0x06c473d374a0e81c (ID = 06c473d37550e81c) Created: 08/27/2019 14:45:28 Last Modified: 08/27/2019 14:45:28\r\nSchedule Time: 08/27/2019 14:45:29\r\nReport Template (8 fields): _OnD_0x06c473d374a0e81c (ID = 0x06c473d374c0e81c) Created: 08/27/2019 14:45:28 Last Modified: 08/27/2019 14:45:28\r\nWARNING: No Pricing for instrument (ISN,AT0000618251,VIE,AEMV.VI^B17), segment 'E' due to instrument not being traded.\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 08/27/2019 14:45:30, it was scheduled to snap at 08/27/2019 14:45:29.\r\nContent delivery suppressed due to lack of permission for NAS1 (NASDAQ UTP) for quotes from NAS, NSM\r\nEmbargo delay of 15 minutes required by [ EN1 (EURONEXT CONTINENTAL CASH(CONSOLIDATED PACK) L 1), ENS (EURONEXT CONTINENTAL CASH(CONSOL PACK) LEVEL 2), ENPI (EURONEXT PRIVATE INVESTOR PACKAGE PREMIUM) ] for quotes from BRU\r\nEmbargo delay of 15 minutes required by [ VIE (WIENER BORSE CASH MKT STRUCTURED PROD L1), VI2 (WBAG CASH MKT STRUCTURED PROD L1 L2) ] for quotes from VIE\r\nEmbargo delay of 15 minutes required by [ VTX (SIX SWISS EXCHANGE LEVEL 1), VT2 (SIX SWISS EXCHANGE L1 L2), VTP (SIX SWISS EXCHANGE LEVEL 2 PLUS) ] for quotes from SWX\r\nEmbargo delays caused suppression of 10 instruments.\r\nProcessing completed successfully at 08/27/2019 14:45:30, taking 0.541 Secs.\r\nExtraction finished at 08/27/2019 13:45:30 UTC, with servers: x09i08, QSHC18 (0.0 secs), QSSHA1 (0.1 secs)\r\nUsage Summary for User xxx, Client 103424, Template Type Intraday Pricing\r\nBase Usage\r\n Instrument Instrument Terms Price\r\n Count Type Subtype Source Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n 1 Equities N/A N/A\r\n-------\r\n 1 Total instrument charged.\r\n 16 Instruments with no reported data.\r\n=======\r\n 17 Instruments in the input list.\r\nNo TRPS complex usage to report -- 17 Instruments in the input list had no reported data.\r\n"]

SUCCESS: Processing completed successfully !

Content extracted from JSON response:

[{"Last Price":null,"Identifier":"AT0000609607","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000618251","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000730007","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000641352","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000818802","Volume":null,"IdentifierType":"Isin"},{"Last Price":15.8,"Identifier":"AT0000624705","Volume":600,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000624739","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"US0038301067","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000644505","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"BE0003764785","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000723606","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000728209","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000938204","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000625132","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"CH0008837566","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"CH0014420829","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"US0378331005","Volume":null,"IdentifierType":"Isin"},{"Identifier":"CH0012488190","Error":"Not found","IdentifierType":"Isin"}]

if you set "OnlyNonEmbargoedData",true, then, as the setting's name implies, you will only receive the non embargoed data.

The embargoed data will not be delivered.

That is the whole point of this parameter setting: to only deliver non embargoed data.

It actually says so in so many words in the extraction notes:

Embargo delays caused suppression of 10 instruments

Christiaan Meihsl Yes, I agree on the embargoed instruments point and understood that part. But I do not understand why I do not get a 'Partial Delivery' even though it has been enabled in the 'User Preferences'. I am attaching the screenshot and the logs again.

2019/08/28 14:15:29 : Session Token (expires in 24 hours):
xxxx
2019/08/28 14:15:29 : Intraday extraction JSON request content:
@{"ExtractionRequest":{"@odata.type":"#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.IntradayPricingExtractionRequest","ContentFieldNames":["Last Price","User Defined Identifier","Volume"],"IdentifierList":{"@odata.type":"#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList","InstrumentIdentifiers":[{"Identifier":"AT0000609607","IdentifierType":"Isin"},{"Identifier":"AT0000618251","IdentifierType":"Isin"},{"Identifier":"AT0000730007","IdentifierType":"Isin"},{"Identifier":"AT0000641352","IdentifierType":"Isin"},{"Identifier":"AT0000818802","IdentifierType":"Isin"},{"Identifier":"AT0000624705","IdentifierType":"Isin"},{"Identifier":"AT0000624739","IdentifierType":"Isin"},{"Identifier":"US0038301067","IdentifierType":"Isin"},{"Identifier":"AT0000644505","IdentifierType":"Isin"},{"Identifier":"BE0003764785","IdentifierType":"Isin"},{"Identifier":"AT0000723606","IdentifierType":"Isin"},{"Identifier":"AT0000728209","IdentifierType":"Isin"},{"Identifier":"AT0000938204","IdentifierType":"Isin"},{"Identifier":"CH0012488190","IdentifierType":"Isin"},{"Identifier":"AT0000625132","IdentifierType":"Isin"},{"Identifier":"CH0008837566","IdentifierType":"Isin"},{"Identifier":"CH0014420829","IdentifierType":"Isin"},{"Identifier":"US0378331005","IdentifierType":"Isin"}],"ValidationOptions":null,"UseUserPreferencesForValidationOptions":true},"Condition":{"ScalableCurrency":true}}}

2019/08/28 14:15:29 : We submit the request, a response should arrive in approximately 30 seconds ...

2019/08/28 14:16:00 : HTTP status: 202 - We must wait, and poll the location URL.
2019/08/28 14:16:00 : Location URL: https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractWithNotesResult(ExtractionId='0x06c4a702b540e85a')
2019/08/28 14:16:00 : Waiting 30 seconds ..

2019/08/28 14:17:00 : HTTP status: 202
2019/08/28 14:17:00 : Waiting 30 seconds ..

2019/08/28 14:17:30 : Executing..

2019/08/28 14:18:00 : HTTP status: 202
2019/08/28 14:18:00 : Waiting 30 seconds ..

2019/08/28 14:18:30 : Executing..

2019/08/28 14:19:01 : HTTP status: 202
2019/08/28 14:19:01 : Waiting 30 seconds ..

2019/08/28 14:19:31 : Executing..

2019/08/28 14:20:01 : HTTP status: 202
2019/08/28 14:20:01 : Waiting 30 seconds ..

2019/08/28 14:20:31 : Executing..

2019/08/28 14:21:01 : HTTP status: 202
2019/08/28 14:21:01 : Waiting 30 seconds ..

2019/08/28 14:21:31 : Executing..

2019/08/28 14:22:01 : HTTP status: 202
2019/08/28 14:22:01 : Waiting 30 seconds ..

2019/08/28 14:22:31 : Executing..

2019/08/28 14:23:01 : HTTP status: 202
2019/08/28 14:23:01 : Waiting 30 seconds ..

2019/08/28 14:23:31 : Executing..

2019/08/28 14:24:01 : HTTP status: 202
2019/08/28 14:24:01 : Waiting 30 seconds ..

2019/08/28 14:24:31 : Executing..

2019/08/28 14:25:01 : HTTP status: 202
2019/08/28 14:25:01 : Waiting 30 seconds ..

2019/08/28 14:25:31 : Executing..

2019/08/28 14:26:02 : HTTP status: 202
2019/08/28 14:26:02 : Waiting 30 seconds ..

2019/08/28 14:26:32 : Executing..

2019/08/28 14:27:02 : HTTP status: 202
2019/08/28 14:27:02 : Waiting 30 seconds ..

2019/08/28 14:27:32 : Executing..

2019/08/28 14:28:02 : HTTP status: 202
2019/08/28 14:28:02 : Waiting 30 seconds ..

2019/08/28 14:28:32 : Executing..

2019/08/28 14:29:02 : HTTP status: 202
2019/08/28 14:29:02 : Waiting 30 seconds ..

2019/08/28 14:29:32 : Executing..

2019/08/28 14:30:02 : HTTP status: 202
2019/08/28 14:30:02 : Waiting 30 seconds ..

2019/08/28 14:30:32 : Executing..

2019/08/28 14:30:33 : HTTP status: 200
2019/08/28 14:30:33 : Result received

2019/08/28 14:30:33 : Notes extracted from JSON response:
["Extraction Services Version 13.1.40889 (ec84d57d2aa3), Built Aug 9 2019 18:16:31\r\nProcessing started at 08/28/2019 13:15:32.\r\nUser ID: xxx\r\nExtraction ID: 382058191\r\nSchedule: _OnD_0x06c4a702b540e85a (ID = 0x06c4a702b6a0e85a)\r\nInput List (17 items): _OnD_0x06c4a702b540e85a (ID = 06c4a702b5e0e85a) Created: 08/28/2019 13:15:31 Last Modified: 08/28/2019 13:15:31\r\nSchedule Time: 08/28/2019 13:15:31\r\nReport Template (8 fields): _OnD_0x06c4a702b540e85a (ID = 0x06c4a702b560e85a) Created: 08/28/2019 13:15:31 Last Modified: 08/28/2019 13:15:31\r\nWARNING: No Pricing for instrument (ISN,AT0000618251,VIE,AEMV.VI^B17), segment 'E' due to instrument not being traded.\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 08/28/2019 13:15:32, it was scheduled to snap at 08/28/2019 13:15:31.\r\nEmbargo delay of 15 minutes required by [ EN1 (EURONEXT CONTINENTAL CASH(CONSOLIDATED PACK) L 1), ENS (EURONEXT CONTINENTAL CASH(CONSOL PACK) LEVEL 2), ENPI (EURONEXT PRIVATE INVESTOR PACKAGE PREMIUM) ] for quotes from BRU\r\nEmbargo delay of 15 minutes required by [ VIE (WIENER BORSE CASH MKT STRUCTURED PROD L1), VI2 (WBAG CASH MKT STRUCTURED PROD L1 L2) ] for quotes from VIE\r\nEmbargo delay of 15 minutes required by [ VTX (SIX SWISS EXCHANGE LEVEL 1), VT2 (SIX SWISS EXCHANGE L1 L2), VTP (SIX SWISS EXCHANGE LEVEL 2 PLUS) ] for quotes from SWX\r\nThe last report will be embargoed until 08/28/2019 13:30:32 (15 minutes) due to quote: ISN,AT0000624705,VIE,KAER.VI - Last Update Time: 08/28/2019 13:15:32.\r\nThe file _OnD_0x06c4a702b540e85a.0min.csv will be available immediately.\r\nThe file _OnD_0x06c4a702b540e85a.15min.csv will be embargoed until 08/28/2019 13:30:32.\r\nProcessing completed successfully at 08/28/2019 13:15:32, taking 0.712 Secs.\r\nExtraction finished at 08/28/2019 12:15:32 UTC, with servers: x09I16, QSHC12 (0.1 secs), QSSHA1 (0.0 secs)\r\nUsage Summary for User xxx, Client 103424, Template Type Intraday Pricing\r\nBase Usage\r\n
Instrument Instrument Terms Price\r\n Count Type Subtype Source Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n 12 Equities N/A N/A\r\n-------\r\n 12 Total instruments charged.\r\n 5 Instruments with no reported data.\r\n=======\r\n 17 Instruments in the input list.\r\nNo TRPS complex usage to report -- 17 Instruments in the input list had no reported data.\r\n"]
2019/08/28 14:30:33 : SUCCESS: Processing completed successfully !

2019/08/28 14:30:33 : Content extracted from JSON response:
[{"Last Price":null,"Identifier":"AT0000618251","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000624739","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"US0038301067","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"AT0000723606","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"CH0014420829","Volume":null,"IdentifierType":"Isin"},{"Last Price":null,"Identifier":"US0378331005","Volume":21572,"IdentifierType":"Isin"},{"Last Price":19.2,"Identifier":"AT0000609607","Volume":18284,"IdentifierType":"Isin"},{"Last Price":31.76,"Identifier":"AT0000730007","Volume":101682,"IdentifierType":"Isin"},{"Last Price":31.15,"Identifier":"AT0000641352","Volume":26048,"IdentifierType":"Isin"},{"Last Price":84.7,"Identifier":"AT0000818802","Volume":4195,"IdentifierType":"Isin"},{"Last Price":15.8,"Identifier":"AT0000624705","Volume":600,"IdentifierType":"Isin"},{"Last Price":85.05,"Identifier":"AT0000644505","Volume":6751,"IdentifierType":"Isin"},{"Last Price":128,"Identifier":"BE0003764785","Volume":8506,"IdentifierType":"Isin"},{"Last Price":99,"Identifier":"AT0000728209","Volume":49,"IdentifierType":"Isin"},{"Last Price":106.4,"Identifier":"AT0000938204","Volume":13763,"IdentifierType":"Isin"},{"Last Price":93,"Identifier":"AT0000625132","Volume":216,"IdentifierType":"Isin"},{"Last Price":185.8,"Identifier":"CH0008837566","Volume":15519,"IdentifierType":"Isin"}]

I would not like to ping the TR servers continuously and keep the thread alive until an embargo expires, could you help me understand why I do not get a Partial Report? As you see here the result was returned after the embargo was lifted.
This is the configuration on the ‘User Preferences’
I have attached the output for your reference, as you see it pings the servers until a ‘200 OK’ is received even though Partial delivery option is enabled.

Thank you.

Are you sure you set "OnlyNonEmbargoedData",true ? I do not see that in the request above, so I believe it was not set.

I tried your request, it did not hang. As I am permissioned for real-time for all the venues you are using, I tried a different instrument where I know I am not permissioned (ALVG.DE, on Xetra, which has a 15 minute embargo). It displayed the "HTTP status: 202" and "Waiting 30 seconds ..." messages many times, and after approximately 15 minutes I received the data. Snippets from the extraction notes:

Real-time data was snapped at 28082019 14:49:05
Embargo delay of 15 minutes required
The last report will be embargoed until 28082019 15:04:03 (15 minutes)

If an embargo applies, then you must wait until the end of the embargo period(s). In your case (just like mine) the embargoes are 15 minutes (you can see that in the extraction notes in your second answer).
Did you wait enough for the embargo to expire ?
Note that the extraction also takes time, so between the embargo and the extraction, you might have to wait a bit longer.

PS: I have the same settings as you set in the GUI General preferences screen.

DSS2OnDemandIntraday Java code sample: no partial delivery

This sample is only a PoC (Proof of Concept), with limited functionality. It does not handle intermediate deliveries, i.e. it does not retrieve data as it is made available (file ending in 0min.csv); instead it waits for the entire embargo period before retrieving the data. I have updated the Java code examples description document, and later I will update the comments in the code and the readme file to make this point more obvious.

Note that all your instruments except the US ones are subjected to a 15 minute embargo, and this query was run before US market open (hence no embargo for US, but some data is missing due to lack of permission for NAS1).

@Christiaan Meihsl Agreed that it is a proof of concept but would you be so kind to point me in the correct direction on any condition which I can use so that the response works intermediately since we have already enabled these features on the 'User Preferences'. I have already gone through the documentation which you had suggested.

@Anuj, I am looking into this, it is not as easy as I thought. Retrieving data as it becomes available might require to use an immediate schedule instead of using an On Demand request. If all goes well I should be able to publish an updated sample tomorrow.

Intraday On Demand request (recommended)

With an On Demand request it is not possible to retrieve partial deliveries (those that are not embargoed) before those that are embargoed.

Here is the way to do it using On Demand requests:

Make a first request, setting OnlyNonEmbargoedData to true, to receive the non embargoed data.
Compare the list of received instruments to those in the original request, and then make a second query for the missing instruments (without setting OnlyNonEmbargoedData), this will retrieve the embargoed ones.

Caveat: if your instrument list contains instruments that have different embargo delays, step 2 will wait for the longest embargo delay before delivering all embargoed data.

Immediate schedule (not recommended)

Here is the workflow:

0. Create an instrument list and report template.

1. Create an immediate schedule for Intra Day data

2. Request the Next Extraction (/Extractions/Schedules('reportScheduleId')/NextExtraction).

3. If step 2 returns HTTP status 204 there is no next extraction, i.e. the extraction is complete (no embargo). In that case:

3.1. Request the Last extraction (/Extractions/Schedules('reportScheduleId')/LastExtraction), to get the ReportExtractionId.

3.2. Use that Id to retrieve all the files (/Extractions/ReportExtractions('reportExtractionId')/Files).

3.3 Task is complete, exit here.

4. If step 2 returns an HTTP status 200 then it is still processing (embargo or extraction taking more than 30 sec). In that case:

4.1. extract the ReportExtractionId and Status from the response.

4.2. Enter a while loop to:

4.2.1. List files that might be available (typically: notes, non-embargoed data).

4.2.2. Download those that were not already downloaded (select them by checking the names)

4.2.3. Wait a minute or more. Important: do not wait less than 1 minute, it would make no sense as embargoes last several minutes, and you don't want to put unnecessary load on the servers.

4.2.4. Request the Next Extraction

4.2.5. Exit the loop if Next Extraction returns a 204 (which means the extraction has completed).

4.3. List available files (this is the final complete list).

4.4. Download those that were not already downloaded (select them by checking the names) (typically: embargoed data)

4.5 Task is complete, exit here.

Final note: the instrument list, report template and schedule can be destroyed after all files were retrieved.

@Anuj, today I updated the description here, and the Java Intraday On Demand code (and comments in both Java Intraday samples) that you can download from the DSS downloads tab, to clarify some points following feedback from the product team.