For a deeper look into our DataScope Select REST API, look into:
Overview | Quickstart | Documentation | Downloads | Tutorials
Composite extraction response incomplete in Data Scope Select
Hallo All,
I have attached Python code where i am using Composite extraction request to fetch the response for Instruments. I have at the moment 1400 instruments and i am passing 70 instrument at one extraction request.
But i am just getting response for the first 9 instruments everytime and then the code stopped. I am using the same code from Tutorial example from Refinitiv.
Did i miss any condition in the extraction request ?
requestUrl = reqStart + '/Extractions/ExtractRaw'
requestHeaders = {
"Prefer": "respond-async",
"Content-Type": "application/json",
"Authorization": "token " + token
}
requestBody = {
"ExtractionRequest": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",
"ContentFieldNames": [
"ISIN","Average Volume - 30 Days","Average Volume - 90 Days","Close on Close Volatility - 90 Days","Dollar Daily Value Average - 30 Days","CESR Free Float","CRA Free Float","Outstanding Shares - Issue Shares Amount","Outstanding Shares - Other Shares Amount"
],
"IdentifierList": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [
{"Identifier":"FR0000120321","IdentifierType": "Isin"},{"Identifier":"FR0000121014","IdentifierType": "Isin"},{"Identifier":"FR0014000MR3","IdentifierType": "Isin"},{"Identifier":"GB0005630420","IdentifierType": "Isin"},{"Identifier":"GB00BH4HKS39","IdentifierType": "Isin"},{"Identifier":"IE00B1FZS467","IdentifierType": "Isin"},{"Identifier":"IE00B1FZS467","IdentifierType": "Isin"},{"Identifier":"IE00B1XNHC34","IdentifierType": "Isin"},{"Identifier":"IE00B3VWMM18","IdentifierType": "Isin"},{"Identifier":"IE00B4BNMY34","IdentifierType": "Isin"},{"Identifier":"IE00B6R52036","IdentifierType": "Isin"},{"Identifier":"IE00B6R52036","IdentifierType": "Isin"},{"Identifier":"IE00BD2B9488","IdentifierType": "Isin"},{"Identifier":"IE00BH480S68","IdentifierType": "Isin"},{"Identifier":"IE00BKM4GZ66","IdentifierType": "Isin"},{"Identifier":"IE00BKTLJC87","IdentifierType": "Isin"},{"Identifier":"IE00BMYDM794","IdentifierType": "Isin"},{"Identifier":"IE00BQ1YBK98","IdentifierType": "Isin"},{"Identifier":"IE00BQQP9H09","IdentifierType": "Isin"},{"Identifier":"IE00BZ12WP82","IdentifierType": "Isin"},{"Identifier":"LU0133360163","IdentifierType": "Isin"},{"Identifier":"LU0294851513","IdentifierType": "Isin"},{"Identifier":"LU0514695690","IdentifierType": "Isin"},{"Identifier":"LU0971623524","IdentifierType": "Isin"},{"Identifier":"LU1127970256","IdentifierType": "Isin"},{"Identifier":"LU1376267727","IdentifierType": "Isin"},{"Identifier":"LU1659686460","IdentifierType": "Isin"},{"Identifier":"NL0000009538","IdentifierType": "Isin"},{"Identifier":"NL0000395903","IdentifierType": "Isin"},{"Identifier":"NL0010273215","IdentifierType": "Isin"},{"Identifier":"SE0015961909","IdentifierType": "Isin"},{"Identifier":"SE0015988019","IdentifierType": "Isin"},{"Identifier":"US00724F1012","IdentifierType": "Isin"},{"Identifier":"US01609W1027","IdentifierType": "Isin"},{"Identifier":"US02079K3059","IdentifierType": "Isin"},{"Identifier":"US0231351067","IdentifierType": "Isin"},{"Identifier":"US0378331005","IdentifierType": "Isin"},{"Identifier":"US0383361039","IdentifierType": "Isin"},{"Identifier":"US0758871091","IdentifierType": "Isin"},{"Identifier":"US09075V1026","IdentifierType": "Isin"},{"Identifier":"US1713401024","IdentifierType": "Isin"},{"Identifier":"US2358511028","IdentifierType": "Isin"},{"Identifier":"US2546871060","IdentifierType": "Isin"},{"Identifier":"US2788651006","IdentifierType": "Isin"},{"Identifier":"US28176E1082","IdentifierType": "Isin"},{"Identifier":"US29414B1044","IdentifierType": "Isin"},{"Identifier":"US4612021034","IdentifierType": "Isin"},{"Identifier":"US5797802064","IdentifierType": "Isin"},{"Identifier":"US5949181045","IdentifierType": "Isin"},{"Identifier":"US7170811035","IdentifierType": "Isin"},{"Identifier":"US7427181091","IdentifierType": "Isin"}
]
},
"Condition": {
"ScalableCurrency": "true"
}
}
}
It could be a permission issue.
I found the following log in the Extraction notes in the 220122refinitiv-liquidity-log.txt file.
15:41:19 - Extraction notes: Extraction Services Version 15.2.42998 (497fbb60729e), Built Nov 19 2021 21:30:22 Holiday Rollover of Universal Close Price waived. Processing started at 01/22/2022 15:41:16. User ID: 9030101 Extraction ID: 568020353 ... Cells suppressed for 19 row(s) for lack of 'CUSIP & CUSIP based ISINs' permission. Processing completed successfully at 01/22/2022 15:41:17, taking 0.776 Secs.
You may contact the Refinitiv DataScope Select support team via MyRefinitiv to verify the permission.
I ran the same code and could get the data properly.
@Gurpreet Thanks for the feedback. I will check with my Refinitiv Account manager and also contact Refinitiv DataScope Select support team
Hi @rahul.deshmukh,
There is some issue with your code. I tried your request and was able to get response for all instruments. Some of the instruments didn't have any data (blanks), but the response message was complete.
My test sample attached.
@Gurpreet i fixed the codee now..actually i am using the same code from Refinitiv tutorial...but for soe ISIN data it didnt show the ISIN at all but the other field values:
Do you also got the same output ? I dont understand why it didnt show the ISIN. For example if we request for below ISIN , in the output it didnt show the ISIN but just the other field values...
US01609W1027
Can you please paste your output result ?
Yes, I get data for
US01609W1027
If you run the sample I attached, you should be able to see.
US01609W1027,3472104,3473775,49.556830723,425871504.2076191,,,,
@Gurpreet so you can see all the ISIN ? Can you please tell where and how do you provide authentication in your sample ? I am getting below error:
Data available on server, fetching...
401
{"error":{"message":"Authentication request failed because of invalid token"}}
Do you also have an sample example where you download the data in csv or may be in other file ?
When i run the code i got the below output where some ISIN missing from the start:
I just generated the report from DSS gui for ISIN
US01609W1027
and in the report also the ISIN was not included..
Average Volume - 30 Days,Average Volume - 90 Days,Close on Close Volatility - 90 Days,Dollar Daily Value Average - 30 Days,ISIN,CESR Free Float,CRA Free Float,Outstanding Shares - Issue Shares Amount,Outstanding Shares - Other Shares Amount "261,733","193,856",46.323335443,"34,743,402.10466522",,,,"2,718,179,248", "533,598","287,618",47.148852294,"80,382,679.92849566",,,,"2,718,179,248", ,,,,,,,"2,718,179,248", ,,,,,,,"2,718,179,248", ,,,,,,,"2,718,179,248", 130,141,50.111650411,"15,168.767633737",,,,"2,718,179,248", "1,226","1,167",48.304198972,"145,980.941572050",,,,"2,718,179,248", ,,,,,,,"2,718,179,248", "2,065","1,848",51.073629501,"266,107.130975591",,,,"2,718,179,248", 212,217,46.686814353,"25,350.666163428",,,,"2,718,179,248", ,,,,,,,"2,718,179,248", ,,,,,,,"2,718,179,248", 269,273,51.991102539,"34,382.562918954",,,,"2,718,179,248", ,,,,,,,"2,718,179,248", ,,,,,,,"2,718,179,248",
Why didn't you include ISIN in the content field names like I show in the sample. The data for US01609W1027 is there in the output CSV file you attached.
,3472104,3473775,49.556830723,425871504.2076191,,,2718179248,
@Gurpreet I already included ISIN in the content field names :
Yes the data for US01609W1027 is there in the output CSV file but you can see the ISIN value is missing...What i dont understand is for some values ISIN is missing and for some value ISIN is there...
Here is my attached code which i am testing:composite_example.txt
