For a deeper look into our DataScope Select REST API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

question

Upvotes
Accepted
23 8 12 22

Composite extraction response incomplete in Data Scope Select

Hallo All,

I have attached Python code where i am using Composite extraction request to fetch the response for Instruments. I have at the moment 1400 instruments and i am passing 70 instrument at one extraction request.

But i am just getting response for the first 9 instruments everytime and then the code stopped. I am using the same code from Tutorial example from Refinitiv.

composite_example.txt


Did i miss any condition in the extraction request ?

requestUrl = reqStart + '/Extractions/ExtractRaw'

requestHeaders = {
    "Prefer": "respond-async",
    "Content-Type": "application/json",
    "Authorization": "token " + token
}

requestBody = {
    "ExtractionRequest": {
        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",
        "ContentFieldNames": [
            "ISIN","Average Volume - 30 Days","Average Volume - 90 Days","Close on Close Volatility - 90 Days","Dollar Daily Value Average - 30 Days","CESR Free Float","CRA Free Float","Outstanding Shares - Issue Shares Amount","Outstanding Shares - Other Shares Amount"
        ],
        "IdentifierList": {
            "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
            "InstrumentIdentifiers": [
{"Identifier":"FR0000120321","IdentifierType": "Isin"},{"Identifier":"FR0000121014","IdentifierType": "Isin"},{"Identifier":"FR0014000MR3","IdentifierType": "Isin"},{"Identifier":"GB0005630420","IdentifierType": "Isin"},{"Identifier":"GB00BH4HKS39","IdentifierType": "Isin"},{"Identifier":"IE00B1FZS467","IdentifierType": "Isin"},{"Identifier":"IE00B1FZS467","IdentifierType": "Isin"},{"Identifier":"IE00B1XNHC34","IdentifierType": "Isin"},{"Identifier":"IE00B3VWMM18","IdentifierType": "Isin"},{"Identifier":"IE00B4BNMY34","IdentifierType": "Isin"},{"Identifier":"IE00B6R52036","IdentifierType": "Isin"},{"Identifier":"IE00B6R52036","IdentifierType": "Isin"},{"Identifier":"IE00BD2B9488","IdentifierType": "Isin"},{"Identifier":"IE00BH480S68","IdentifierType": "Isin"},{"Identifier":"IE00BKM4GZ66","IdentifierType": "Isin"},{"Identifier":"IE00BKTLJC87","IdentifierType": "Isin"},{"Identifier":"IE00BMYDM794","IdentifierType": "Isin"},{"Identifier":"IE00BQ1YBK98","IdentifierType": "Isin"},{"Identifier":"IE00BQQP9H09","IdentifierType": "Isin"},{"Identifier":"IE00BZ12WP82","IdentifierType": "Isin"},{"Identifier":"LU0133360163","IdentifierType": "Isin"},{"Identifier":"LU0294851513","IdentifierType": "Isin"},{"Identifier":"LU0514695690","IdentifierType": "Isin"},{"Identifier":"LU0971623524","IdentifierType": "Isin"},{"Identifier":"LU1127970256","IdentifierType": "Isin"},{"Identifier":"LU1376267727","IdentifierType": "Isin"},{"Identifier":"LU1659686460","IdentifierType": "Isin"},{"Identifier":"NL0000009538","IdentifierType": "Isin"},{"Identifier":"NL0000395903","IdentifierType": "Isin"},{"Identifier":"NL0010273215","IdentifierType": "Isin"},{"Identifier":"SE0015961909","IdentifierType": "Isin"},{"Identifier":"SE0015988019","IdentifierType": "Isin"},{"Identifier":"US00724F1012","IdentifierType": "Isin"},{"Identifier":"US01609W1027","IdentifierType": "Isin"},{"Identifier":"US02079K3059","IdentifierType": "Isin"},{"Identifier":"US0231351067","IdentifierType": "Isin"},{"Identifier":"US0378331005","IdentifierType": "Isin"},{"Identifier":"US0383361039","IdentifierType": "Isin"},{"Identifier":"US0758871091","IdentifierType": "Isin"},{"Identifier":"US09075V1026","IdentifierType": "Isin"},{"Identifier":"US1713401024","IdentifierType": "Isin"},{"Identifier":"US2358511028","IdentifierType": "Isin"},{"Identifier":"US2546871060","IdentifierType": "Isin"},{"Identifier":"US2788651006","IdentifierType": "Isin"},{"Identifier":"US28176E1082","IdentifierType": "Isin"},{"Identifier":"US29414B1044","IdentifierType": "Isin"},{"Identifier":"US4612021034","IdentifierType": "Isin"},{"Identifier":"US5797802064","IdentifierType": "Isin"},{"Identifier":"US5949181045","IdentifierType": "Isin"},{"Identifier":"US7170811035","IdentifierType": "Isin"},{"Identifier":"US7427181091","IdentifierType": "Isin"}
            ]
        },
        "Condition": {
            "ScalableCurrency": "true"
        }
    }
}



dss-rest-apidss
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@Gurpreet @Jirapongse @zoya faberov i am facing the issue with composite request. Can you please help here ? Thanks!
Upvotes
Accepted
78.2k 246 52 72

@rahul.deshmukh

It could be a permission issue.

I found the following log in the Extraction notes in the 220122refinitiv-liquidity-log.txt file.

15:41:19 - Extraction notes:
Extraction Services Version 15.2.42998 (497fbb60729e), Built Nov 19 2021 21:30:22


Holiday Rollover of Universal Close Price waived.


Processing started at 01/22/2022 15:41:16.


User ID: 9030101


Extraction ID: 568020353

...


Cells suppressed for 19 row(s) for lack of 'CUSIP & CUSIP based ISINs' permission.


Processing completed successfully at 01/22/2022 15:41:17, taking 0.776 Secs.

You may contact the Refinitiv DataScope Select support team via MyRefinitiv to verify the permission.

I ran the same code and could get the data properly.

1642997072450.png



1642997072450.png (31.1 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@Gurpreet Thanks for the feedback. I will check with my Refinitiv Account manager and also contact Refinitiv DataScope Select support team

Upvotes
21.8k 57 14 21

Hi @rahul.deshmukh,

There is some issue with your code. I tried your request and was able to get response for all instruments. Some of the instruments didn't have any data (blanks), but the response message was complete.

My test sample attached.


Test4.txt


test4.txt (6.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@Gurpreet i fixed the codee now..actually i am using the same code from Refinitiv tutorial...but for soe ISIN data it didnt show the ISIN at all but the other field values:

1642778460292.png

Do you also got the same output ? I dont understand why it didnt show the ISIN. For example if we request for below ISIN , in the output it didnt show the ISIN but just the other field values...

US01609W1027

Can you please paste your output result ?

1642778460292.png (26.9 KiB)
Upvotes
21.8k 57 14 21

Yes, I get data for

US01609W1027

If you run the sample I attached, you should be able to see.

US01609W1027,3472104,3473775,49.556830723,425871504.2076191,,,,


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@Gurpreet so you can see all the ISIN ? Can you please tell where and how do you provide authentication in your sample ? I am getting below error:

Data available on server, fetching...

401

{"error":{"message":"Authentication request failed because of invalid token"}}


Do you also have an sample example where you download the data in csv or may be in other file ?


When i run the code i got the below output where some ISIN missing from the start:

test.txt

test.txt (2.6 KiB)

I just generated the report from DSS gui for ISIN

US01609W1027

and in the report also the ISIN was not included..

Average Volume - 30 Days,Average Volume - 90 Days,Close on Close Volatility - 90 Days,Dollar Daily Value Average - 30 Days,ISIN,CESR Free Float,CRA Free Float,Outstanding Shares - Issue Shares Amount,Outstanding Shares - Other Shares Amount
"261,733","193,856",46.323335443,"34,743,402.10466522",,,,"2,718,179,248",
"533,598","287,618",47.148852294,"80,382,679.92849566",,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
130,141,50.111650411,"15,168.767633737",,,,"2,718,179,248",
"1,226","1,167",48.304198972,"145,980.941572050",,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
"2,065","1,848",51.073629501,"266,107.130975591",,,,"2,718,179,248",
212,217,46.686814353,"25,350.666163428",,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
269,273,51.991102539,"34,382.562918954",,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
Upvotes
21.8k 57 14 21

Why didn't you include ISIN in the content field names like I show in the sample. The data for US01609W1027 is there in the output CSV file you attached.

,3472104,3473775,49.556830723,425871504.2076191,,,2718179248,


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@Gurpreet I already included ISIN in the content field names :

1642789243293.png

Yes the data for US01609W1027 is there in the output CSV file but you can see the ISIN value is missing...What i dont understand is for some values ISIN is missing and for some value ISIN is there...

Here is my attached code which i am testing:composite_example.txt


You can see below the output of my code. After row 33 the ISIN are missing but the other values are there which is bit strange:

1642789645274.png

1642789645274.png (76.9 KiB)
There is some bug in your copy/paste. Type this and attach the output file here -
python Test4.py > Test4Output.txt

Show more comments

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.