DSS API ScheduleExtract IntradayPricing

Zuka1

I requested IntradayPricing using ScheduleExtraction because required extra decimal information. small number of symbols.

('JP3193400003', 'Isin'), ('JP3216700009', 'Isin'), ('JP3524600008', 'Isin'), ('JP3783700002', 'Isin'), ('JP3886000003', 'Isin')

small number of Fields (Bid Price, Last Price, Previous Close Price, Previoous Close Date, RIC, ISIN)

The response of FileType keeps 'Partial', never changed to 'Full'. the response was following.. Is this the normal behavior? I requested using ISIN but the response don't include ISIN information..

Bid Price,Last Price,Previous Close Price,Previous Close Date,RIC,ISIN ,,,,7485.NG, ,,3960,10/02/2020,5979.NG,JP3216700009 ,,,,5461.NG, ,,,,2058.FU, ,,,,9664.NG,

zoya faberov

Hello @Zuka1,

I just tested what seems to be your request via Postman tool, as on demand request, is this the correct request? (the result should be the same for on demand and for a scheduled extraction)

{{protocol}}{{host}}{{api}}Extractions/ExtractWithNotes

{
  "ExtractionRequest": {
    "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.IntradayPricingExtractionRequest",
    "ContentFieldNames": [
      "Bid Price", 
      "Last Price", 
      "Previous Close Price", 
      "Previous Close Date", 
      "RIC", 
      "ISIN"
    ],
    "IdentifierList": {
      "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  
      "InstrumentIdentifiers": [
          {"Identifier": "JP3193400003",  "IdentifierType":"Isin"},
          {"Identifier": "JP3216700009",  "IdentifierType":"Isin"},
          {"Identifier":"JP3524600008",  "IdentifierType":"Isin"}, 
          {"Identifier":"JP3783700002",  "IdentifierType":"Isin"}, 
          {"Identifier":"JP3886000003", "IdentifierType": "Isin"}
      ]
    },
    "Condition": { "ScalableCurrency": true }
  }
}

Get 5 rows back as expected.

For example, my notes look like this:

 
"Notes": [         "Extraction Services Version 14.2.41795 (d0fe3ba7b1b1), Built Aug 18 2020 17:24:47\r\nProcessing started at 10/05/2020 15:37:54.\r\nUser ID: 19945\r\nExtraction ID: 480402147\r\nSchedule: _OnD_0x0746ade74316e69f (ID = 0x0746ade74896e69f)\r\nInput List (5 items): _OnD_0x0746ade74316e69f (ID = 0746ade74756e69f) Created: 10/05/2020 15:37:51 Last Modified: 10/05/2020 15:37:51\r\nSchedule Time: 10/05/2020 15:37:51\r\nReport Template (12 fields): _OnD_0x0746ade74316e69f (ID = 0x0746ade745a6e69f) Created: 10/05/2020 15:37:51 Last Modified: 10/05/2020 15:37:51\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 10/05/2020 15:37:55, it was scheduled to snap at 10/05/2020 15:37:51.\r\nNo embargo required for this report.\r\nProcessing completed successfully at 10/05/2020 15:37:55, taking 0.661 Secs.\r\nExtraction finished at 10/05/2020 15:37:55 UTC, with servers: x09I16, QSHC20 (0.1 secs), QSSHA1 (0.0 secs)\r\nUsage Summary for User 19945, Client 8116, Template Type Intraday Pricing\r\nBase Usage\r\n        Instrument                          Instrument                   Terms          Price\r\n  Count Type                                Subtype                      Source         Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n      5 Equities                                                         N/A            N/A\r\n-------\r\n      5 Total instruments charged.\r\n      0 Instruments with no reported data.\r\n=======\r\n      5 Instruments in the input list.\r\nNo TRPS complex usage to report -- 5 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",         "Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"     ]

As I do not seem to reproduce any partial data, some info that would help:

Is the result of 5 instruments what you would expect to see?

Do you consistently get the issue that you describe, not all instruments in your result, or you have only seen it once?

What do you see in extraction notes that come back with your result?

Do you only see this in scheduled, are you able to run on-demand request do you see the same partial result?

Gurpreet

@Zuka1 Please post your request query so that we can help analyze it. Thanks.

Zuka1

Thank you very much for your investigation.

Let me break down to a few questions.

1. FileType

According to the sample code of tutorial (dataAvailabilityExtraction.py), it checks FileType then process it when it is 'Full' . My case is getting 'Partial', how should I manage this?

== dataAvailabilityExtraction.py ==

if FileType == 'Full':

# Only fetch contents of file that has FileType = Full

urlFile = urlGetExtractedFile + '(\'' + ExtractedFileId + '\')/$value'

resp = requests.get(urlFile, headers=header)

==

== response from Schedule Extraction ==

@{'@odata.context': 'https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ExtractedFiles', 'value': [{'ExtractedFileId': 'VjF8fDcxNTQ0MjE4Mg', 'ReportExtractionId': '480502371', 'ScheduleId': '0x0746afb362a6e6a5', 'FileType': 'RicMaintenanceNote', 'ExtractedFileName': '9011414.dataAvailabilitySchedule.20201006.093439.480502371.x13I10.ric.csv', 'LastWriteTimeUtc': '2020-10-06T00:34:41.231Z', 'ContentsExists': True, 'Size': 90, 'ReceivedDateUtc': '2020-10-06T00:34:41.231Z'}, {'ExtractedFileId': 'VjF8fDcxNTQ0MjE4MQ', 'ReportExtractionId': '480502371', 'ScheduleId': '0x0746afb362a6e6a5', 'FileType': 'Partial', 'ExtractedFileName': '9011414.dataAvailabilitySchedule.20201006.093439.480502371.x13I10.0min.csv', 'LastWriteTimeUtc': '2020-10-06T00:34:41.228Z', 'ContentsExists': True, 'Size': 168, 'ReceivedDateUtc': '2020-10-06T00:34:41.228Z'}, {'ExtractedFileId': 'VjF8fDcxNTQ0MjE4MA', 'ReportExtractionId': '480502371', 'ScheduleId': '0x0746afb362a6e6a5', 'FileType': 'Note', 'ExtractedFileName': '9011414.dataAvailabilitySchedule.20201006.093439.480502371.x13I10.csv.notes.txt', 'LastWriteTimeUtc': '2020-10-06T00:34:41.233Z', 'ContentsExists': True, 'Size': 2033, 'ReceivedDateUtc': '2020-10-06T00:34:41.233Z'}]} PARTIAL: 0 Bid Price,Last Price,Previous Close Price,Previous Close Date,RIC,ISIN ,,,,7485.NG, ,,3960,10/02/2020,5979.NG,JP3216700009 ,,,,5461.NG, ,,,,2058.FU, ,,,,9664.NG,

==

2. ISIN data field.

It seems only published ISIN value when other field(s) were updated. WHY?

3. Previous Close (Price/Date)

7485,NG had trade on 2020-10-06, why I don't get the information?

4. The data was gone when I request the same Items (11:03AM)

veerapath.rungruengrayubkul

https://community.developers.refinitiv.com/discussion/comment/67041#Comment_67041

Hi @Zuka1,

For the FileType issue, please verify if the "Enable Early Partial Delivery of Embargoed Reports" option is unchecked on the DSS GUI User Preference. The option is to receive an early partial report instead of full.

veerapath.rungruengrayubkul

https://community.developers.refinitiv.com/discussion/comment/67041#Comment_67041

Hi again,

I understand that you refer to the Data Availability Trigger for EOD Pricing Extraction article. The purpose of the article is to retrieve only "today" end of day data. The code provided use data availability trigger in schedule extraction to receive today data only when the data is available. If you want previous day data, you need to change the message used for creating schedule.

Zuka1

Could you advice me what should I need to change the code for creating schedule which supports previous day's data?

current code is:

schData = OrderedDict([('ListId',listIdToUse),
                       ('Name',"dataAvailabilitySchedule"),
                       ('ReportTemplateId',reportId),
                       ('Recurrence',OrderedDict([
                           ('@odata.type','#ThomsonReuters.Dss.Api.Extractions.Schedules.SingleRecurrence'),
                           ('IsImmediate',True)])),
                       ('Trigger',OrderedDict([('@odata.type','#ThomsonReuters.Dss.Api.Extractions.Schedules.ImmediateTrigger'),('LimitReportToTodaysData',True)]))])
createDataAvailabilitySchedData = json.dumps(schData, sort_keys=False)
currDT = datetime.datetime.strftime(datetime.datetime.now(), "%Y-%m-%d-%H:%M:%S")

resp = requests.post(urlCreateSchedule, data=createDataAvailabilitySchedData, headers=header)

zoya faberov

Hello @Zuka1,

In my understanding, you would not make changes in code. Please refer to the article referenced by @veerapath.rungruengrayubkul -> Prices availability in End of Day Pricing report template.

In Schedule, you would select "Last Updated" in stead of "Today Only" and select the time of day when prices have already been made available for the exchange that you require.

veerapath.rungruengrayubkul

https://community.developers.refinitiv.com/discussion/comment/67074#Comment_67074

Hi @Zuka1,

Your code looks correct, except the 'LimitReportToTodaysData':True which causes the extraction to be generated for today data only. To get latest updated data, you need to change the parameter to False.