For a deeper look into our DataScope Select REST API, look into:
Overview | Quickstart | Documentation | Downloads | Tutorials
I am trying to fetch Composite extraction request data from Data Scope Select using Python and trying to generate gzip csv file but i am only getting Authentication back as result and not getting the result data for Instument. I am trying to test with one ISIN currently.
Currently i am getting error as :
HTTP status of the response: 400
NameError: name 'jobId' is not defined
I have also attached my Python example just to know if i am providing something wrong in the request.
Do we already have any working Python example for Composite extraction request ?
Below is how i am providing Composite extraction request in Python :
requestUrl = reqStart + '/Extractions/ExtractRaw' requestHeaders = { "Prefer": "respond-async", "Content-Type": "application/json", "Authorization": "token " + token } requestBody = { "ExtractionRequest": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest", "ContentFieldNames": [ "ISIN", "Announcement Date" ], "IdentifierList": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList", "InstrumentIdentifiers": [{ "Identifier": "DE000NLB1KJ5", "IdentifierType": "Isin" }] }, "Condition": { } } } r2 = requests.post(requestUrl, json=requestBody, headers=requestHeaders) r3 = r2 # Display the HTTP status of the response # Initial response status (after approximately 30 seconds wait) is usually 202 status_code = r2.status_code log("HTTP status of the response: " + str(status_code)) # If status is 202, display the location url we received, and will use to poll the status of the extraction request: if status_code == 202: requestUrl = r2.headers["location"] log('Extraction is not complete, we shall poll the location URL:') log(str(requestUrl)) requestHeaders = { "Prefer": "respond-async", "Content-Type": "application/json", "Authorization": "token " + token } # As long as the status of the request is 202, the extraction is not finished; # we must wait, and poll the status until it is no longer 202: while (status_code == 202): log('As we received a 202, we wait 30 seconds, then poll again (until we receive a 200)') time.sleep(30) r3 = requests.get(requestUrl, headers=requestHeaders) status_code = r3.status_code log('HTTP status of the response: ' + str(status_code)) # When the status of the request is 200 the extraction is complete; # we retrieve and display the jobId and the extraction notes (it is recommended to analyse their content)): if status_code == 200: r3Json = json.loads(r3.text.encode('ascii', 'ignore')) notes = r3Json["Notes"] log('Extraction notes:\n' + notes[0]) # If instead of a status 200 we receive a different status, there was an error: if status_code != 200: log('An error occured. Try to run this cell again. If it fails, re-run the previous cell.\n') requestUrl = requestUrl = reqStart + "/Extractions/RawExtractionResults" # AWS requires an additional header: X-Direct-Download if useAws: requestHeaders = { "Prefer": "respond-async", "Content-Type": "text/plain", "Accept-Encoding": "gzip", "X-Direct-Download": "true", "Authorization": "token " + token } else: requestHeaders = { "Prefer": "respond-async", "Content-Type": "text/plain", "Accept-Encoding": "gzip", "Authorization": "token " + token }
It returned 400 (Bad Request) with the following error.
HTTP status of the response: 400 { "error": { "message": "Malformed request payload: For the property name \"MessageTimeStampIn\" in the JSON request the value could not be parsed successfully. Please check the casing or spelling of the property." } }
The condition of CompositeExtractionRequest doesn't have the MessageTimeStampIn property.
Referring to the API Reference Tree, the condition of CompositeExtractionRequest contains one property which is the ScalableCurrency parameter.
@Jirapongse Now i have edited the condition and just use below parameter:
"Condition": { "ScalableCurrency": "true" }
But getting error as :
HTTP status of the response: 403
NameError: name 'jobId' is not defined
I think you can also test my python code as it is just replacing the username and password. Did the other parameter are correct in condition ?
Have you applied the changes as mentioned by @Jirapongse !!
requestUrl = requestUrl = reqStart + "/Extractions/RawExtractionResults('"+jobId+"')/$value"
First, you need to get the JobId from the JSON request when the status code is 200.
{ "@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#RawExtractionResults/$entity", "JobId": "0x07de743d5d1d7dc7", "Notes": [ "Extraction Servi
# When the status of the request is 200 the extraction is complete; # we retrieve and display the jobId and the extraction notes (it is recommended to analyse their content)): if status_code == 200: r3Json = json.loads(r3.text.encode('ascii', 'ignore')) notes = r3Json["Notes"] jobId = r3Json["JobId"] log('Extraction notes:\n' + notes[0])
Then, use JobId to create a request URL (/Extractions/RawExtractionResults) to download a file.
# Advisory: avoid incomplete output - decompress then download requestUrl = requestUrl = reqStart + "/Extractions/RawExtractionResults('"+jobId+"')/$value" # AWS requires an additional header: X-Direct-Download if useAws: requestHeaders = {
@Jirapongse i already tried with the JobID but i am getting error as
NameError: name 'jobId' is not defined
I have attached the code : composite_example.txt
When i just replace request:
@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest
with:
@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryTimeAndSalesExtractionRequest
it generates the JobID and always the code works fine with th result data for Instrument...