For a deeper look into our DataScope Select REST API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

question

Upvotes
Accepted
gareth.teage avatar image
260 ●2 ●2 ●4

Removing Duplicate Identifiers using an On Demand ExtractWithNotes

Hi,

Please could we be advised on the below.

The questions is to being able to use duplicates in the request, but have them removed so for example only a single line is returned for the first example as they are the same security (with same Asset ID)

------------------------------------------------------------------------------------------------------------------

{

"ExtractionRequest": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",

"ContentFieldNames": [

"CUSIP", "ISIN", "SEDOL","Ticker","Issue PermID", "Issuer PermID", "Issuer OrgID","Entity LEI", "Currency Code", "MIC", "CFI Code", "Maturity Date", "Underlying CUSIP", "Underlying ISIN"

],

"IdentifierList": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",

"InstrumentIdentifiers": [

{ "Identifier": "912810TH1", "IdentifierType": "Cusip" },

{ "Identifier": "US912810TH14", "IdentifierType": "Isin" }

]

}

}

}

Same thing for

{

"ExtractionRequest": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TermsAndConditionsExtractionRequest",

"ContentFieldNames": [

"CUSIP", "ISIN", "SEDOL","Ticker","Issue PermID", "Issuer PermID", "Issuer OrgID", "Currency Code", "MIC", "CFI Code", "Maturity Date"//, "Underlying CUSIP", "Underlying ISIN", "Entity LEI",

],

"IdentifierList": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",

"InstrumentIdentifiers": [

{ "Identifier": "00209tab1", "IdentifierType": "Cusip" },

{ "Identifier": "IBM.N", "IdentifierType": "Ric" },

{ "Identifier": "US4592001014", "IdentifierType": "Isin" },

{ "Identifier": "B1YW440", "IdentifierType": "Sedol" }

]

}

}

}

------------------------------------------------------------------------------------------------------------------

We have discovered a separate request to check if identifier have duplicates : POST .../Extractions/InstrumentListValidateIdentifiers, however if there is a way to submit the Extraction request with the duplicates in the list item array, but only return unique records this would be the preferred method.


Best regards,

Gareth

dss-rest-api#product
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

4 Answers

· Write an Answer
Upvotes
Accepted
Jirapongse avatar image
79.4k ●253 ●52 ●74

@rishabh.nahata

I assume that setting is for DSS Web GUI. For the DSS REST API, you can use the KeepDuplicates property to append instruments to an instrument list.

1687833318151.png

Please contact the DSS support team via MyRefinitiv to confirm this.

I propose two solutions.

1. Use the API to create an instrument list. Then, append instruments to the created instrument list with the KeepDuplicates property set to false so the duplicated instruments will be removed. Finally, use the created instrument list in on-demand extractions. 

{
    "ExtractionRequest": {
        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",
        "ContentFieldNames": [
            "CUSIP",
            "ISIN",
            "SEDOL",
            "Ticker",
            "Issue PermID",
            "Issuer PermID",
            "Issuer OrgID",
            "Entity LEI",
            "Currency Code",
            "MIC",
            "CFI Code",
            "Maturity Date",
            "Underlying CUSIP",
            "Underlying ISIN"
        ],
        "IdentifierList": {
            "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentListIdentifierList",
            "InstrumentListId": "<List Id>"
        }
    }
}

2. Validate instruments with the /Extractions/InstrumentListValidateIdentifiers endpoints before using them in on-demand extractions. 

{
    "InputsForValidation": [
        {
            "Identifier": "912810TH1",
            "IdentifierType": "Cusip"
        },
        {
            "Identifier": "US912810TH14",
            "IdentifierType": "Isin"
        }
    ],
    "KeepDuplicates": false
}

1687834016026.png



1687833318151.png (94.5 KiB)
1687834016026.png (75.9 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Jirapongse avatar image
79.4k ●253 ●52 ●74

@gareth.teage

Thank you for reaching out to us.

Sorry. I am unable to find any property that can be used to remove the duplicated entries in the response.

Please kindly contact the DSS support team directly via MyRefinitiv to confirm it.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

gareth.teage avatar image gareth.teage
@Jirapongse Hi, thanks for the feedback, I have raised Support Ticket via Helpdesk to get this reviewed further.
Upvotes
rishabh.nahata avatar image
1 ●0 ●0 ●1

@Jirapongse

I tried changing my DSS GUI preference "Allow Import of Duplicate Items into Instrument Lists" from True to False and then tried importing the instruments

CUSIP - 912810TH1 and ISIN - US912810TH14 and GUI correctly identifies them as same instrument and allows import of only 1 instrument. However, when tried copying my preferences on API using the below query, I still get 2 outputs with the same result. Shouldn't API show only 1 output?

{

"ExtractionRequest": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",

"ContentFieldNames": [

"CUSIP", "ISIN", "SEDOL","Ticker","Issue PermID", "Issuer PermID", "Issuer OrgID","Entity LEI", "Currency Code", "MIC", "CFI Code", "Maturity Date", "Underlying CUSIP", "Underlying ISIN"

],

"IdentifierList": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",

"InstrumentIdentifiers": [

{ "Identifier": "912810TH1", "IdentifierType": "Cusip" },

{ "Identifier": "US912810TH14", "IdentifierType": "Isin" }

],

"ValidationOptions": null,

"UseUserPreferencesForValidationOptions": false

}

}}


I also tried independent validation option from the API Reference Tree by setting - "ImportOfDuplicateInstrumentsAllowed" to false under validation option. Still the output has 2 results (duplicate). This validation option is not working as well.


Please suggest a workaround on API as there is no content issue and GUI is working as expected.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Gurpreet avatar image Gurpreet

You have specified UseUserPreferencesForValidationOptions as false. Set that to true.

rishabh.nahata avatar image rishabh.nahata Gurpreet

Hello @Gurpreet I tried that as well.

{

"ExtractionRequest": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",

"ContentFieldNames": [

"CUSIP", "ISIN", "SEDOL","Ticker","Issue PermID", "Issuer PermID", "Issuer OrgID","Entity LEI", "Currency Code", "MIC", "CFI Code", "Maturity Date", "Underlying CUSIP", "Underlying ISIN"

],

"IdentifierList": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",

"InstrumentIdentifiers": [

{ "Identifier": "912810TH1", "IdentifierType": "Cusip" },

{ "Identifier": "US912810TH14", "IdentifierType": "Isin" }

],

"ValidationOptions": null,

"UseUserPreferencesForValidationOptions": true

}

}}


I still get 2 outputs (duplicates)




Upvotes
rishabh.nahata avatar image
1 ●0 ●0 ●1

{

"@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#DataScope.Select.Api.Extractions.ExtractionRequests.ExtractionResult",

"Contents": [

{

"IdentifierType": "Cusip",

"Identifier": "912810TH1",

"CUSIP": "912810TH1",

"ISIN": "US912810TH14",

"SEDOL": "BNLYVN1",

"Ticker": "UST",

"Issue PermID": "232792772567",

"Issuer PermID": "8589934522",

"Issuer OrgID": "95823",

"Entity LEI": "254900HROIFWPRGM1V77",

"Currency Code": "USD",

"MIC": "XBER",

"CFI Code": "DBFTFR",

"Maturity Date": "2042-05-15",

"Underlying CUSIP": null,

"Underlying ISIN": null

},

{

"IdentifierType": "Isin",

"Identifier": "US912810TH14",

"CUSIP": "912810TH1",

"ISIN": "US912810TH14",

"SEDOL": "BNLYVN1",

"Ticker": "UST",

"Issue PermID": "232792772567",

"Issuer PermID": "8589934522",

"Issuer OrgID": "95823",

"Entity LEI": "254900HROIFWPRGM1V77",

"Currency Code": "USD",

"MIC": "XBER",

"CFI Code": "DBFTFR",

"Maturity Date": "2042-05-15",

"Underlying CUSIP": null,

"Underlying ISIN": null

}

],

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.