Discover Refinitiv
MyRefinitiv Refinitiv Perspectives Careers
Created with Sketch.
All APIs Questions & Answers  Register |  Login
Ask a question
  • Questions
  • Tags
  • Badges
  • Unanswered
Search:
  • Home /
  • DSS /

For a deeper look into our DataScope Select REST API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

avatar image
Question by rahul.deshmukh · Jan 20 at 10:29 AM · dss-rest-apidatascope-selectdss

Composite extraction request example not working in Python

I am trying to fetch Composite extraction request data from Data Scope Select using Python and trying to generate gzip csv file but i am only getting Authentication back as result and not getting the result data for Instument. I am trying to test with one ISIN currently.

Currently i am getting error as :

HTTP status of the response: 400

NameError: name 'jobId' is not defined


I have also attached my Python example just to know if i am providing something wrong in the request.

composite_example.txt

Do we already have any working Python example for Composite extraction request ?

Below is how i am providing Composite extraction request in Python :

requestUrl = reqStart + '/Extractions/ExtractRaw'

requestHeaders = {

    "Prefer": "respond-async",

    "Content-Type": "application/json",

    "Authorization": "token " + token

}

requestBody = {

    "ExtractionRequest": {

        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",

        "ContentFieldNames": [

            "ISIN", "Announcement Date"

        ],

        "IdentifierList": {

            "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",

            "InstrumentIdentifiers": [{

                "Identifier": "DE000NLB1KJ5",

                "IdentifierType": "Isin"

            }]

        },

        "Condition": {

        }

    }

}


r2 = requests.post(requestUrl, json=requestBody, headers=requestHeaders)

r3 = r2

# Display the HTTP status of the response

# Initial response status (after approximately 30 seconds wait) is usually 202

status_code = r2.status_code

log("HTTP status of the response: " + str(status_code))


# If status is 202, display the location url we received, and will use to poll the status of the extraction request:

if status_code == 202:

    requestUrl = r2.headers["location"]

    log('Extraction is not complete, we shall poll the location URL:')

    log(str(requestUrl))



    requestHeaders = {

        "Prefer": "respond-async",

        "Content-Type": "application/json",

        "Authorization": "token " + token

    }

# As long as the status of the request is 202, the extraction is not finished;

# we must wait, and poll the status until it is no longer 202:

while (status_code == 202):

    log('As we received a 202, we wait 30 seconds, then poll again (until we receive a 200)')

    time.sleep(30)

    r3 = requests.get(requestUrl, headers=requestHeaders)

    status_code = r3.status_code

    log('HTTP status of the response: ' + str(status_code))



# When the status of the request is 200 the extraction is complete;

# we retrieve and display the jobId and the extraction notes (it is recommended to analyse their content)):

if status_code == 200:

    r3Json = json.loads(r3.text.encode('ascii', 'ignore'))

    notes = r3Json["Notes"]

    log('Extraction notes:\n' + notes[0])



# If instead of a status 200 we receive a different status, there was an error:

if status_code != 200:

    log('An error occured. Try to run this cell again. If it fails, re-run the previous cell.\n')



requestUrl = requestUrl = reqStart + "/Extractions/RawExtractionResults"



# AWS requires an additional header: X-Direct-Download

if useAws:

    requestHeaders = {

        "Prefer": "respond-async",

        "Content-Type": "text/plain",

        "Accept-Encoding": "gzip",

        "X-Direct-Download": "true",

        "Authorization": "token " + token

    }

else:

    requestHeaders = {

        "Prefer": "respond-async",

        "Content-Type": "text/plain",

        "Accept-Encoding": "gzip",

        "Authorization": "token " + token

    }



composite-example.txt (10.1 KiB)

People who like this

0 Show 0
Comment
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

2 Replies

  • Sort: 
avatar image
REFINITIV
Best Answer
Answer by Jirapongse · Jan 20 at 02:02 PM

@rahul.deshmukh

It returned 400 (Bad Request) with the following error.

HTTP status of the response: 400
{    "error": {        "message": "Malformed request payload: For the property name \"MessageTimeStampIn\" in the JSON request the value could not be parsed successfully. Please check the casing or spelling of the property."    } }

The condition of CompositeExtractionRequest doesn't have the MessageTimeStampIn property.

Referring to the API Reference Tree, the condition of CompositeExtractionRequest contains one property which is the ScalableCurrency parameter.

1642686038714.png


1642686038714.png (20.7 KiB)
Comment

People who like this

0 Show 9 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
rahul.deshmukh · Jan 20 at 02:13 PM 0
Share

@Jirapongse Now i have edited the condition and just use below parameter:

"Condition": {            
    "ScalableCurrency": "true"
} 

But getting error as :

HTTP status of the response: 403

NameError: name 'jobId' is not defined

I think you can also test my python code as it is just replacing the username and password. Did the other parameter are correct in condition ?

avatar image
rahul.deshmukh · Jan 20 at 02:44 PM 0
Share
@zoya faberov @Gurpreet Do you know what is wrong with the request ?
avatar image
REFINITIV
Gurpreet ♦♦ rahul.deshmukh · Jan 20 at 03:06 PM 0
Share

Have you applied the changes as mentioned by @Jirapongse !!

requestUrl = requestUrl = reqStart + "/Extractions/RawExtractionResults('"+jobId+"')/$value"
avatar image
rahul.deshmukh Gurpreet ♦♦ · Jan 20 at 03:34 PM 0
Share

@Gurpreet yes and getting below error:

HTTP status of the response: 403


avatar image
REFINITIV
Answer by Jirapongse · Jan 20 at 11:14 AM

@rahul.deshmukh

First, you need to get the JobId from the JSON request when the status code is 200.

{
    "@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#RawExtractionResults/$entity",
    "JobId": "0x07de743d5d1d7dc7",
    "Notes": [
        "Extraction Servi
# When the status of the request is 200 the extraction is complete;
# we retrieve and display the jobId and the extraction notes (it is recommended to analyse their content)):
if status_code == 200:
    r3Json = json.loads(r3.text.encode('ascii', 'ignore'))
    notes = r3Json["Notes"]
    jobId = r3Json["JobId"]
    log('Extraction notes:\n' + notes[0])

Then, use JobId to create a request URL (/Extractions/RawExtractionResults) to download a file.

#   Advisory: avoid incomplete output - decompress then download


requestUrl = requestUrl = reqStart + "/Extractions/RawExtractionResults('"+jobId+"')/$value"


# AWS requires an additional header: X-Direct-Download
if useAws:
    requestHeaders = {
                
Comment

People who like this

0 Show 1 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
rahul.deshmukh · Jan 20 at 12:06 PM 0
Share

@Jirapongse i already tried with the JobID but i am getting error as

NameError: name 'jobId' is not defined

I have attached the code : composite_example.txt


When i just replace request:

@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest

with:

@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryTimeAndSalesExtractionRequest 

it generates the JobID and always the code works fine with th result data for Instrument...

composite-example.txt (10.5 KiB)

Watch this question

Add to watch list
Add to your watch list to receive emailed updates for this question. Too many emails? Change your settings >
14 People are following this question.

Related Questions

How to schedule DSS Legal Entity Hierarchy Detail extraction

How does MessageTimeStampIn in ExtractionRequest work?

DSS TRTH inflight status

Need help developing a tool to query historical prices from Datascope Select (DSS) and import them into our SQL Server database

Is there an API endpoint for Embargo information at Price Source-level or at RIC-validation level?

  • Copyright
  • Cookie Policy
  • Privacy Statement
  • Terms of Use
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Alpha
  • App Studio
  • Block Chain
  • Bot Platform
  • Connected Risk APIs
  • DSS
  • Data Fusion
  • Data Model Discovery
  • Datastream
  • Eikon COM
  • Eikon Data APIs
  • Electronic Trading
    • Generic FIX
    • Local Bank Node API
    • Trading API
  • Elektron
    • EMA
    • ETA
    • WebSocket API
  • Intelligent Tagging
  • Legal One
  • Messenger Bot
  • Messenger Side by Side
  • ONESOURCE
    • Indirect Tax
  • Open Calais
  • Open PermID
    • Entity Search
  • Org ID
  • PAM
    • PAM - Logging
  • ProView
  • ProView Internal
  • Product Insight
  • Project Tracking
  • RDMS
  • Refinitiv Data Platform
    • Refinitiv Data Platform Libraries
  • Rose's Space
  • Screening
    • Qual-ID API
    • Screening Deployed
    • Screening Online
    • World-Check One
    • World-Check One Zero Footprint
  • Side by Side Integration API
  • TR Knowledge Graph
  • TREP APIs
    • CAT
    • DACS Station
    • Open DACS
    • RFA
    • UPA
  • TREP Infrastructure
  • TRKD
  • TRTH
  • Thomson One Smart
  • Transactions
    • REDI API
  • Velocity Analytics
  • Wealth Management Web Services
  • Workspace SDK
    • Element Framework
    • Grid
  • World-Check Data File
  • 中文论坛
  • Explore
  • Tags
  • Questions
  • Badges