Composite extraction request example not working in Python

Question

question

rahul.deshmukh

23 ●9 ●14 ●23

Composite extraction request example not working in Python

I am trying to fetch Composite extraction request data from Data Scope Select using Python and trying to generate gzip csv file but i am only getting Authentication back as result and not getting the result data for Instument. I am trying to test with one ISIN currently.

Currently i am getting error as :

HTTP status of the response: 400

NameError: name 'jobId' is not defined

I have also attached my Python example just to know if i am providing something wrong in the request.

composite_example.txt

Do we already have any working Python example for Composite extraction request ?

Below is how i am providing Composite extraction request in Python :

requestUrl = reqStart + '/Extractions/ExtractRaw'

requestHeaders = {

    "Prefer": "respond-async",

    "Content-Type": "application/json",

    "Authorization": "token " + token

}

requestBody = {

    "ExtractionRequest": {

        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",

        "ContentFieldNames": [

            "ISIN", "Announcement Date"

        ],

        "IdentifierList": {

            "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",

            "InstrumentIdentifiers": [{

                "Identifier": "DE000NLB1KJ5",

                "IdentifierType": "Isin"

            }]

        },

        "Condition": {

        }

    }

}


r2 = requests.post(requestUrl, json=requestBody, headers=requestHeaders)

r3 = r2

# Display the HTTP status of the response

# Initial response status (after approximately 30 seconds wait) is usually 202

status_code = r2.status_code

log("HTTP status of the response: " + str(status_code))


# If status is 202, display the location url we received, and will use to poll the status of the extraction request:

if status_code == 202:

    requestUrl = r2.headers["location"]

    log('Extraction is not complete, we shall poll the location URL:')

    log(str(requestUrl))



    requestHeaders = {

        "Prefer": "respond-async",

        "Content-Type": "application/json",

        "Authorization": "token " + token

    }

# As long as the status of the request is 202, the extraction is not finished;

# we must wait, and poll the status until it is no longer 202:

while (status_code == 202):

    log('As we received a 202, we wait 30 seconds, then poll again (until we receive a 200)')

    time.sleep(30)

    r3 = requests.get(requestUrl, headers=requestHeaders)

    status_code = r3.status_code

    log('HTTP status of the response: ' + str(status_code))



# When the status of the request is 200 the extraction is complete;

# we retrieve and display the jobId and the extraction notes (it is recommended to analyse their content)):

if status_code == 200:

    r3Json = json.loads(r3.text.encode('ascii', 'ignore'))

    notes = r3Json["Notes"]

    log('Extraction notes:\n' + notes[0])



# If instead of a status 200 we receive a different status, there was an error:

if status_code != 200:

    log('An error occured. Try to run this cell again. If it fails, re-run the previous cell.\n')



requestUrl = requestUrl = reqStart + "/Extractions/RawExtractionResults"



# AWS requires an additional header: X-Direct-Download

if useAws:

    requestHeaders = {

        "Prefer": "respond-async",

        "Content-Type": "text/plain",

        "Accept-Encoding": "gzip",

        "X-Direct-Download": "true",

        "Authorization": "token " + token

    }

else:

    requestHeaders = {

        "Prefer": "respond-async",

        "Content-Type": "text/plain",

        "Accept-Encoding": "gzip",

        "Authorization": "token " + token

    }

dss-rest-api datascope-select dss

composite-example.txt (10.1 KiB)

Jan 20, 2022 at 10:29 AM

10 |1500

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 1 · 2022-01-20T14:02:31Z

Jirapongse

78.8k ●250 ●52 ●74

@rahul.deshmukh

It returned 400 (Bad Request) with the following error.

HTTP status of the response: 400
{    "error": {        "message": "Malformed request payload: For the property name \"MessageTimeStampIn\" in the JSON request the value could not be parsed successfully. Please check the casing or spelling of the property."    } }

The condition of CompositeExtractionRequest doesn't have the MessageTimeStampIn property.

Referring to the API Reference Tree, the condition of CompositeExtractionRequest contains one property which is the ScalableCurrency parameter.

1642686038714.png (20.7 KiB)

Jan 20, 2022 at 02:02 PM

10 |1500

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

rahul.deshmukh Jan 20, 2022 at 02:13 PM

@Jirapongse Now i have edited the condition and just use below parameter:

"Condition": {            
    "ScalableCurrency": "true"
}

But getting error as :

HTTP status of the response: 403

NameError: name 'jobId' is not defined

I think you can also test my python code as it is just replacing the username and password. Did the other parameter are correct in condition ?

rahul.deshmukh Jan 20, 2022 at 02:44 PM

@zoya faberov @Gurpreet Do you know what is wrong with the request ?

Gurpreet rahul.deshmukh Jan 20, 2022 at 03:06 PM

Have you applied the changes as mentioned by @Jirapongse !!

requestUrl = requestUrl = reqStart + "/Extractions/RawExtractionResults('"+jobId+"')/$value"

rahul.deshmukh Gurpreet Jan 20, 2022 at 03:34 PM

@Gurpreet yes and getting below error:

HTTP status of the response: 403

Show more comments

Answer 2 · 2022-01-20T11:14:37Z

Jirapongse

78.8k ●250 ●52 ●74

@rahul.deshmukh

First, you need to get the JobId from the JSON request when the status code is 200.

{
    "@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#RawExtractionResults/$entity",
    "JobId": "0x07de743d5d1d7dc7",
    "Notes": [
        "Extraction Servi

# When the status of the request is 200 the extraction is complete;
# we retrieve and display the jobId and the extraction notes (it is recommended to analyse their content)):
if status_code == 200:
    r3Json = json.loads(r3.text.encode('ascii', 'ignore'))
    notes = r3Json["Notes"]
    jobId = r3Json["JobId"]
    log('Extraction notes:\n' + notes[0])

Then, use JobId to create a request URL (/Extractions/RawExtractionResults) to download a file.

#   Advisory: avoid incomplete output - decompress then download


requestUrl = requestUrl = reqStart + "/Extractions/RawExtractionResults('"+jobId+"')/$value"


# AWS requires an additional header: X-Direct-Download
if useAws:
    requestHeaders = {

Jan 20, 2022 at 11:14 AM

10 |1500

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

rahul.deshmukh Jan 20, 2022 at 12:06 PM

@Jirapongse i already tried with the JobID but i am getting error as

NameError: name 'jobId' is not defined

I have attached the code : composite_example.txt

When i just replace request:

@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest

with:

@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryTimeAndSalesExtractionRequest

it generates the JobID and always the code works fine with th result data for Instrument...

composite-example.txt (10.5 KiB)

Q&A Forum

question

Composite extraction request example not working in Python

2 Answers

Write an Answer

question

Composite extraction request example not working in Python

2 Answers

Write an Answer

Related Questions