"/Extractions/Extract" vs. "/Extractions/ExtractRaw"

jacktan · November 2022

Hello. I am new to DSS REST API.

What is the difference between "/Extractions/Extract" and "/Extractions/ExtractRaw"?

I made an ExtractionRequest using both of them, without the "Prefer: respond-async" arg in the header, and with the following payload:

{"ExtractionRequest": {
        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TermsAndConditionsExtractionRequest",
        "ContentFieldNames": [
            "RIC", "Security Description", "Last Trading Day", "Expiration Date"],
        "IdentifierList": {
            "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
            "InstrumentIdentifiers": [
                {
                    "Identifier": "0#CGB:",
                    "IdentifierType": "ChainRIC"
                }
            ]
        }
    }
}

For the former, I got my results back directly. For the latter, I got back a JobId that I needed to retrieve the results from (even though I did not want "respond-async").

Is it safe to say that "/Extractions/Extract" will always return with the results? For context, I am running a cron job with only a few requests per day, so it's easier for me to not implement polling logic etc.

Thanks!

Jack

Jirapongse · November 2022

@jacktan

Sorry about the issue that you are facing.

/Extractions/Extract is the old endpoint that has been deprecated, as mentioned in this webinar.

Therefore, you need to use /Extractions/ExtractWithNotes instead.

/Extractions/ExtractWithNotes returns data and Notes in the HTTP response. The data is in JSON format.

{
    "@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#DataScope.Select.Api.Extractions.ExtractionRequests.ExtractionResult",
    "Contents": [
        {
            "IdentifierType": "Cusip",
            "Identifier": "438516AC0",
            "Ask Price": null,
            "Asset Category": null,
            "Asset Category Description": null,
            "Asset ID": "0x0000190000004b52",

        },
        {
            "IdentifierType": "Ric",
            "Identifier": "IBM.N",
            "Ask Price": 140.17,
            "Asset Category": "ORD",
            "Asset Category Description": "Ordinary",
            "Asset ID": "0x0003dd001379d460",

        }
    ],
    "Notes": [
        "Extraction Services Version 16.2.44415 (c4ac5f5e8fb7), Built Oct 12 2022 22:20:12\r\nHoliday Rollover of Universal Close Price waived.\r\nUser has overridden estimates broker entitlements.\r\nProcessing started at 11/09/2022 08:11:36.\r\nUser ID: 9008895\r\nExtraction ID: 625411691\r\nCorrelation ID: CiD/9008895/AAAAAA.083cbb530e634e9d/RA/EXT.625411691\r\nSchedule: _OnD_0x083cbb530e734e9d (ID = 0x083cbb5314534e9d)\r\nInput List (2 items): _OnD_0x083cbb530e734e9d (ID = 083cbb530f134e9d) Created: 11/09/2022 08:11:33 Last Modified: 11/09/2022 08:11:34\r\nSchedule Time: 11/09/2022 08:11:34\r\nReport Template (73 fields): _OnD_0x083cbb530e734e9d (ID = 0x083cbb530e834e9d) Created: 11/09/2022 08:11:33 Last Modified: 11/09/2022 08:11:33\r\n(CSP,438516AC0,EJV) is inactive.\r\nColumn 'CIN Code' suppressed for lack of 'CIN Code' permission.\r\nProcessing completed successfully at 11/09/2022 08:11:38, taking 1.87 Secs.\r\nExtraction finished at 11/09/2022 01:11:38 UTC, with servers: x08i03, QSDHA1 (0.0 secs), QSHC16 (0.4 secs)\r\nUsage Summary for User 9008895, Client 65507, Template Type EOD Pricing\r\nBase Usage\r\n        Instrument                          Instrument                   Terms          Price\r\n  Count Type                                Subtype                      Source         Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n      1 Corporate                                                        N/A            N/A\r\n      1 Equities                                                         N/A            N/A\r\n-------\r\n      2 Total instruments charged.\r\n      0 Instruments with no reported data.\r\n=======\r\n      2 Instruments in the input list.\r\nNo Evaluated Pricing Service complex usage to report -- 2 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",
        "Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"
    ]
}

/Extractions/ExtractRaw returns JobID and Notes.

{
    "@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#RawExtractionResults/$entity",
    "JobId": "0x083c8b5204434e72",
    "Notes": [
        "Extraction Services Version 16.2.44415 (c4ac5f5e8fb7), Built Oct 12 2022 22:20:12\r\nHoliday Rollover of Universal Close Price waived.\r\nUser has overridden estimates broker entitlements.\r\nProcessing started at 11/09/2022 08:09:24.\r\nUser ID: 9008895\r\nExtraction ID: 625411128\r\nCorrelation ID: CiD/9008895/AAAAAA.083c8b5204334e72/RA/EXT.625411128\r\nSchedule: _OnD_0x083c8b5204434e72 (ID = 0x083c8b5209c34e72)\r\nInput List (2 items): _OnD_0x083c8b5204434e72 (ID = 083c8b5204e34e72) Created: 11/09/2022 08:09:22 Last Modified: 11/09/2022 08:09:23\r\nSchedule Time: 11/09/2022 08:09:23\r\nReport Template (67 fields): _OnD_0x083c8b5204434e72 (ID = 0x083c8b5204534e72) Created: 11/09/2022 08:09:22 Last Modified: 11/09/2022 08:09:22\r\n(CSP,438516AC0,EJV) is inactive.\r\nColumn 'CIN Code' suppressed for lack of 'CIN Code' permission.\r\nProcessing completed successfully at 11/09/2022 08:09:25, taking 1.12 Secs.\r\nExtraction finished at 11/09/2022 01:09:25 UTC, with servers: x07t02, QSDHA1 (0.0 secs), QSHC11 (0.5 secs)\r\nUsage Summary for User 9008895, Client 65507, Template Type EOD Pricing\r\nBase Usage\r\n        Instrument                          Instrument                   Terms          Price\r\n  Count Type                                Subtype                      Source         Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n      1 Corporate                                                        N/A            N/A\r\n      1 Equities                                                         N/A            N/A\r\n-------\r\n      2 Total instruments charged.\r\n      0 Instruments with no reported data.\r\n=======\r\n      2 Instruments in the input list.\r\nNo Evaluated Pricing Service complex usage to report -- 2 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",
        "Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"
    ]
}

The application needs to send an additional request to get a data file, such as /Extractions/RawExtractionResults('0x083c8b5204434e72')/$value. The data file is in CSV format.

Both endpoints are asynchronous and can return HTTP 202 Accepted with the polling URL in the Location header.

The application always needs to check the HTTP status code. If it is 202 Accepted, the application needs to get the Location header and then use that URL to poll for the data.

I hope that this information is of help

zoya faberov · November 2022

Hello @jacktan ,

According to the current On Demand Extraction spec in: DSS REST API Reference tree- the supported endpoints for on demand request are /Extractions/ExtractRaw and /Extractions/ExtractWithNotes .

It was interesting to learn that /Extractions/Extract, that is not documented, also works at this point of time.

I would not use it for any non-transient implementation, being undocumented, it is likely only available temporary, may be used exclusively for product testing, it may possibly have issues that preclude production usage, so may become unavailable or unreliable at any point of time and without any client notice.

I would suggest using /Extractions/ExtractWithNotes when available, and inspecting notes for extraction-specific issues, and using /Extractions/ExtractRaw otherwise.

As you mention that you are new to DSS REST API, I would like to bring to your attention REST API Tutorial 7: On Demand T&C extraction

Hope that you find this information to be of help

jacktan · November 2022

Thanks @zoya faberov and @Jirapongse for the detailed responses! And yes, I am taking over an old project from when "Extractions/Extract" used to be a thing I guess.

Really appreciate the detailed walkthrough with images!

"/Extractions/Extract" vs. "/Extractions/ExtractRaw"

Best Answer

Answers

Categories

EXPLORE OUR SITES

"/Extractions/Extract" vs. "/Extractions/ExtractRaw"

Best Answer

Answers

Categories