Accessing TRTH Intraday summaries in python

jack.coppinger1

Hi, I am trying to use python to retrieve intraday summary data. The first step is fine and I get the token without issue. I then make my request and get the 202 response as expected.

When I then poll the response, it initially says it's not yet complete, and will try again. However after that it states an error has occurred and the job seems to time out.

I have attached screenshots, can you clarify if I'm doing something wrong? Note this exact same process has worked previously about a month ago so not sure if the product has been updated since. Thanks

veerapath.rungruengrayubkul

Hi @jack.coppinger1,

Thanks for the code. I have tried it but cannot replicate the issue. Could you modify the code to use token variable instead of copied value for the requestHeaders? This is to ensure that the token value is valid when the requests are sent. Also, once you try the code, please run the token request step first to get new valid token.

Below is the sample.

#Step 2: send an on demand extraction request using the received token 

requestUrl='https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRaw';

requestHeaders={
    "Prefer":"respond-async",
    "Content-Type":"application/json",
    "Authorization": "token " + token
}

#If status is 202, display the location url we received, and will use to poll the status of the extraction request:
if status_code == 202 :
    requestUrl = r2.headers["location"]
    print ('Extraction is not complete, we shall poll the location URL:')
    print (str(requestUrl))
    
    requestHeaders={
        "Prefer":"respond-async",
        "Content-Type":"application/json",
        "Authorization":"token " + token
    }

Jirapongse

@jack.coppinger1

From the code, the token is hard code.

Are you using an expired token? The token remains valid for 24 hours.

I got the same 401 error if I used an expired token. You can print the body in the response message to verify the problem. In my case, I got:

{
    "error": {
        "message": "Authentication request failed because of invalid token"
    }
}

Please make sure that you are using the current token retrieved when calling https://hosted.datascopeapi.reuters.com/RestApi/v1/Authentication/RequestToken.

Otherwise, you can use the POST method with

https://hosted.datascopeapi.reuters.com/RestApi/v1/Authentication/ValidateToken(Token='<token>') to verify the token.

jack.coppinger1

Hi, thank you for response. I don't believe this is the issue. I generate the token in the first response:

I then paste this same token into the initial request:

Also when I run the extraction I get a 202 response, which indicates it's been accepted by the server and is in progress?

It's the next step when polling to see if the job is complete that the issue seems to be. It polls once, then again after 30 seconds, if the job isn't done it gives me an error rather than repeating

veerapath.rungruengrayubkul

Hi @jack.coppinger1,

The issue might be in the "Authorization" header created for the polling request. Can you add "+" as follows. If there still be an issue, please provide your code in text format so I can try to replicate the issue.

"Authorization":"token " + "<your token>"

jack.coppinger1

hi @veerapath.rungruengrayubkul, when I added the + I got a timeout error

My request to get the token is:

filePath = "/Users/u6069419/Downloads/" #Location to save downloaded files

fileNameRoot = "Python_Test" #Root of the name for the downloaded files

myUsername = "myUsername"

myPassword = "myPassword"

useAws = True

#Set the last parameter above to:

# False to download from TRTH servers

# True to download from Amazon Web Services cloud (recommended, it is faster)

#Imports:

import requests

import json

import shutil

import time

import urllib3

import gzip

#====================================================================================

#Step 1: token request

requestUrl = "https://hosted.datascopeapi.reuters.com/RestApi/v1/Authentication/RequestToken&quot;

requestHeaders={

"Prefer":"respond-async",

"Content-Type":"application/json"

}

requestBody={

"Credentials": {

"Username": 9021083,

"Password": "xxxxxxx"

}

}

r1 = requests.post(requestUrl, json=requestBody,headers=requestHeaders)

if r1.status_code == 200 :

jsonResponse = json.loads(r1.text.encode('ascii', 'ignore'))

token = jsonResponse["value"]

print ('Authentication token (valid 24 hours):')

print (token)

else:

print ('Replace myUserName and myPassword with valid credentials, then repeat the request')

Then the next request is:

requestHeaders={

"Prefer":"respond-async",

"Content-Type":"application/json",

"Authorization": "token " "_XDMENamXOk0ULRojIsNRXQpvDKZgntjVzjxP1PsdBKP1CC136gxdntVx7JnfHAHSeCcwYnFLxxU6vEo1CHZMaQL7qZVDa7qJgJ90VcI_lwpMzx7AbME5DFIC2X8hK0lpZWJZkFpI9SzYujTHtuT3b7b2S7gLt2BSCIjCktJpWmCTV7v6F962WgPpn2UYym6Wf_w5VfhzYyWXKYazSrzhPdoYZWTHufIFr4bK-bcw8TN4jY1VJiRXdNatvairq9q_jLVZcIcjLLJymiIMgeaZTUXn5JQqw0UtlkOixiLI2Gc"

}

requestBody={

"ExtractionRequest": {

"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TickHistoryIntradaySummariesExtractionRequest",

"ContentFieldNames": [

"Close Ask",

"Close Bid",

"High",

"High Ask",

"High Bid",

"Last",

"Low",

"Low Ask",

"Low Bid",

"No. Asks",

"No. Bids",

"No. Trades",

"Open",

"Open Ask",

"Open Bid",

"Volume"

],

"IdentifierList": {

"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",

"InstrumentIdentifiers": [{

"Identifier": "FB.O",

"IdentifierType": "Ric"

}],

"UseUserPreferencesForValidationOptions":"false"

},

"Condition": {

"MessageTimeStampIn": "GmtUtc",

"ReportDateRangeType": "Range",

"QueryStartDate": "2020-05-14T00:00:00.000Z",

"QueryEndDate": "2020-05-16T00:00:00.000Z",

"SummaryInterval": "OneHour",

"TimebarPersistence":"true",

"DisplaySourceRIC":"true"

}

}

}

r2 = requests.post(requestUrl, json=requestBody,headers=requestHeaders)

#Display the HTTP status of the response

#Initial response status (after approximately 30 seconds wait) is usually 202

status_code = r2.status_code

print ("HTTP status of the response: " + str(status_code))

jack.coppinger1

https://community.developers.refinitiv.com/discussion/comment/62333#Comment_62333

Thanks @veerapath.rungruengrayubkul, that solution worked perfectly and am now able to retrieve the data. Thanks for your help on this