Composite extraction response incomplete in Data Scope Select

Hallo All,

I have attached Python code where i am using Composite extraction request to fetch the response for Instruments. I have at the moment 1400 instruments and i am passing 70 instrument at one extraction request.

But i am just getting response for the first 9 instruments everytime and then the code stopped. I am using the same code from Tutorial example from Refinitiv.

composite_example.txt

Did i miss any condition in the extraction request ?

requestUrl = reqStart + '/Extractions/ExtractRaw'

requestHeaders = {
    "Prefer": "respond-async",
    "Content-Type": "application/json",
    "Authorization": "token " + token
}

requestBody = {
    "ExtractionRequest": {
        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",
        "ContentFieldNames": [
            "ISIN","Average Volume - 30 Days","Average Volume - 90 Days","Close on Close Volatility - 90 Days","Dollar Daily Value Average - 30 Days","CESR Free Float","CRA Free Float","Outstanding Shares - Issue Shares Amount","Outstanding Shares - Other Shares Amount"
        ],
        "IdentifierList": {
            "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
            "InstrumentIdentifiers": [
{"Identifier":"FR0000120321","IdentifierType": "Isin"},{"Identifier":"FR0000121014","IdentifierType": "Isin"},{"Identifier":"FR0014000MR3","IdentifierType": "Isin"},{"Identifier":"GB0005630420","IdentifierType": "Isin"},{"Identifier":"GB00BH4HKS39","IdentifierType": "Isin"},{"Identifier":"IE00B1FZS467","IdentifierType": "Isin"},{"Identifier":"IE00B1FZS467","IdentifierType": "Isin"},{"Identifier":"IE00B1XNHC34","IdentifierType": "Isin"},{"Identifier":"IE00B3VWMM18","IdentifierType": "Isin"},{"Identifier":"IE00B4BNMY34","IdentifierType": "Isin"},{"Identifier":"IE00B6R52036","IdentifierType": "Isin"},{"Identifier":"IE00B6R52036","IdentifierType": "Isin"},{"Identifier":"IE00BD2B9488","IdentifierType": "Isin"},{"Identifier":"IE00BH480S68","IdentifierType": "Isin"},{"Identifier":"IE00BKM4GZ66","IdentifierType": "Isin"},{"Identifier":"IE00BKTLJC87","IdentifierType": "Isin"},{"Identifier":"IE00BMYDM794","IdentifierType": "Isin"},{"Identifier":"IE00BQ1YBK98","IdentifierType": "Isin"},{"Identifier":"IE00BQQP9H09","IdentifierType": "Isin"},{"Identifier":"IE00BZ12WP82","IdentifierType": "Isin"},{"Identifier":"LU0133360163","IdentifierType": "Isin"},{"Identifier":"LU0294851513","IdentifierType": "Isin"},{"Identifier":"LU0514695690","IdentifierType": "Isin"},{"Identifier":"LU0971623524","IdentifierType": "Isin"},{"Identifier":"LU1127970256","IdentifierType": "Isin"},{"Identifier":"LU1376267727","IdentifierType": "Isin"},{"Identifier":"LU1659686460","IdentifierType": "Isin"},{"Identifier":"NL0000009538","IdentifierType": "Isin"},{"Identifier":"NL0000395903","IdentifierType": "Isin"},{"Identifier":"NL0010273215","IdentifierType": "Isin"},{"Identifier":"SE0015961909","IdentifierType": "Isin"},{"Identifier":"SE0015988019","IdentifierType": "Isin"},{"Identifier":"US00724F1012","IdentifierType": "Isin"},{"Identifier":"US01609W1027","IdentifierType": "Isin"},{"Identifier":"US02079K3059","IdentifierType": "Isin"},{"Identifier":"US0231351067","IdentifierType": "Isin"},{"Identifier":"US0378331005","IdentifierType": "Isin"},{"Identifier":"US0383361039","IdentifierType": "Isin"},{"Identifier":"US0758871091","IdentifierType": "Isin"},{"Identifier":"US09075V1026","IdentifierType": "Isin"},{"Identifier":"US1713401024","IdentifierType": "Isin"},{"Identifier":"US2358511028","IdentifierType": "Isin"},{"Identifier":"US2546871060","IdentifierType": "Isin"},{"Identifier":"US2788651006","IdentifierType": "Isin"},{"Identifier":"US28176E1082","IdentifierType": "Isin"},{"Identifier":"US29414B1044","IdentifierType": "Isin"},{"Identifier":"US4612021034","IdentifierType": "Isin"},{"Identifier":"US5797802064","IdentifierType": "Isin"},{"Identifier":"US5949181045","IdentifierType": "Isin"},{"Identifier":"US7170811035","IdentifierType": "Isin"},{"Identifier":"US7427181091","IdentifierType": "Isin"}
            ]
        },
        "Condition": {
            "ScalableCurrency": "true"
        }
    }
}

Find more posts tagged with

dss

dss-rest-api

Accepted answers

Jirapongse

@rahul.deshmukh

It could be a permission issue.

I found the following log in the Extraction notes in the 220122refinitiv-liquidity-log.txt file.

15:41:19 - Extraction notes:
Extraction Services Version 15.2.42998 (497fbb60729e), Built Nov 19 2021 21:30:22


Holiday Rollover of Universal Close Price waived.


Processing started at 01/22/2022 15:41:16.


User ID: 9030101


Extraction ID: 568020353

...


Cells suppressed for 19 row(s) for lack of 'CUSIP & CUSIP based ISINs' permission.


Processing completed successfully at 01/22/2022 15:41:17, taking 0.776 Secs.

You may contact the Refinitiv DataScope Select support team via MyRefinitiv to verify the permission.

I ran the same code and could get the data properly.

All comments

rahul.deshmukh

@Gurpreet @Jirapongse @zoya faberov i am facing the issue with composite request. Can you please help here ? Thanks!

Gurpreet

Hi @rahul.deshmukh,

There is some issue with your code. I tried your request and was able to get response for all instruments. Some of the instruments didn't have any data (blanks), but the response message was complete.

My test sample attached.

Test4.txt

rahul.deshmukh

@Gurpreet i fixed the codee now..actually i am using the same code from Refinitiv tutorial...but for soe ISIN data it didnt show the ISIN at all but the other field values:

Do you also got the same output ? I dont understand why it didnt show the ISIN. For example if we request for below ISIN , in the output it didnt show the ISIN but just the other field values...

US01609W1027

Can you please paste your output result ?

Gurpreet

Yes, I get data for

US01609W1027

If you run the sample I attached, you should be able to see.

US01609W1027,3472104,3473775,49.556830723,425871504.2076191,,,,

rahul.deshmukh

@Gurpreet so you can see all the ISIN ? Can you please tell where and how do you provide authentication in your sample ? I am getting below error:

Data available on server, fetching...

401

{"error":{"message":"Authentication request failed because of invalid token"}}

Do you also have an sample example where you download the data in csv or may be in other file ?

When i run the code i got the below output where some ISIN missing from the start:

test.txt

rahul.deshmukh

I just generated the report from DSS gui for ISIN

US01609W1027

and in the report also the ISIN was not included..

Average Volume - 30 Days,Average Volume - 90 Days,Close on Close Volatility - 90 Days,Dollar Daily Value Average - 30 Days,ISIN,CESR Free Float,CRA Free Float,Outstanding Shares - Issue Shares Amount,Outstanding Shares - Other Shares Amount
"261,733","193,856",46.323335443,"34,743,402.10466522",,,,"2,718,179,248",
"533,598","287,618",47.148852294,"80,382,679.92849566",,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
130,141,50.111650411,"15,168.767633737",,,,"2,718,179,248",
"1,226","1,167",48.304198972,"145,980.941572050",,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
"2,065","1,848",51.073629501,"266,107.130975591",,,,"2,718,179,248",
212,217,46.686814353,"25,350.666163428",,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
269,273,51.991102539,"34,382.562918954",,,,"2,718,179,248",
,,,,,,,"2,718,179,248",
,,,,,,,"2,718,179,248",

Gurpreet

Why didn't you include ISIN in the content field names like I show in the sample. The data for US01609W1027 is there in the output CSV file you attached.

,3472104,3473775,49.556830723,425871504.2076191,,,2718179248,

rahul.deshmukh

@Gurpreet I already included ISIN in the content field names :

Yes the data for US01609W1027 is there in the output CSV file but you can see the ISIN value is missing...What i dont understand is for some values ISIN is missing and for some value ISIN is there...

Here is my attached code which i am testing:composite_example.txt

rahul.deshmukh

You can see below the output of my code. After row 33 the ISIN are missing but the other values are there which is bit strange:

Gurpreet

There is some bug in your copy/paste. Type this and attach the output file here -

python Test4.py > Test4Output.txt

Jirapongse

@rahul.deshmukh

It could be a permission issue.

I found the following log in the Extraction notes in the 220122refinitiv-liquidity-log.txt file.

15:41:19 - Extraction notes:
Extraction Services Version 15.2.42998 (497fbb60729e), Built Nov 19 2021 21:30:22


Holiday Rollover of Universal Close Price waived.


Processing started at 01/22/2022 15:41:16.


User ID: 9030101


Extraction ID: 568020353

...


Cells suppressed for 19 row(s) for lack of 'CUSIP & CUSIP based ISINs' permission.


Processing completed successfully at 01/22/2022 15:41:17, taking 0.776 Secs.

You may contact the Refinitiv DataScope Select support team via MyRefinitiv to verify the permission.

I ran the same code and could get the data properly.

rahul.deshmukh

@Gurpreet Thanks for the feedback. I will check with my Refinitiv Account manager and also contact Refinitiv DataScope Select support team

EXPLORE OUR SITES