For a deeper look into our DataScope Select REST API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

question

Upvotes
Accepted
opsp avatar image
1 ●3 ●2 ●10

why is a simple retrieval taking so long?

i have a script that I use for other chains (like vix futures) and it takes a couple of minutes to run. I've now been waiting 2 hours for this... why does it take so long?

I just want minute bars for SPX futures for a few months...

{
    "ExtractionRequest": {
        "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TickHistoryIntradaySummariesExtractionRequest",
        "ContentFieldNames": [
            "Open",# "Open Ask", "Open Bid",
            "High",# "High Ask", "High Bid",
            "Low",# "Low Ask", "Low Bid",
            "Last", "Close Ask", "Close Bid",
            "Volume"
        ],
        "IdentifierList": {
            "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
            "InstrumentIdentifiers": [
                #{"Identifier": ".SPX", "IdentifierType": "Ric"},
                {"Identifier": "0#ES:", "IdentifierType": "ChainRIC"}
                #{"Identifier": "0#VX:", "IdentifierType": "ChainRIC"}
            ],
            "ValidationOptions": {"AllowHistoricalInstruments": "true"},
            "UseUserPreferencesForValidationOptions": "false"
        },
        "Condition": {
            "MessageTimeStampIn": "GmtUtc",
            "ReportDateRangeType": "Range",
            "QueryStartDate": "2021-02-01T00:00:00.000Z",
            "QueryEndDate": "2021-06-01T23:59:00.000Z",
            "SummaryInterval": "OneMinute",    # "FifteenMinutes" # OneSecond # OneMinute
            "DisplaySourceRIC": "true"
        }
    }
}
dss-rest-apidatascope-selectdsstick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

4 Answers

Upvotes
Accepted
zoya faberov avatar image
32.2k ●40 ●11 ●19

Hello @opsp,

Absolutely, glad to share my thoughts. RTH HTTP REST on demand, custom report requesting is primarily intended as a light way of requesting RTH content.

As you are looking to request 100s of chains and a decade or more of data, I would consider design and product options:

  • The first approach I would consider, is Venue By Day or VBD. This will allow you to request a a complete exchange day's worth of data, compacted by zip and next select what you require from it, for details see:

Data Dictionary - Refinitiv Tick History Venue By Day Message Reporting and REST API Tutorial 2: Retrieving VBD files.

VBD is an additional product.

Hope this information helps


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
opsp avatar image
1 ●3 ●2 ●10

it's still running....should I kill it? how long should it take? (approx)

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
zoya faberov avatar image
32.2k ●40 ●11 ●19

Hello @opsp,

From what I see, this is a very large request. So would expect it to take long.

Have you tried requesting OneMinute bars, on the same chain RIC, for 1 day interval of time?

In a general scenario, if a request is taking unreasonably long time all of a sudden, especially if the same request takes a short time otherwise, I would suggest verifying with My Refinitiv -> Service Alerts section, to know of any potential outage that could be impacting your request at the time.

Hope this information helps

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
opsp avatar image
1 ●3 ●2 ●10

it's 3MB of sensibly encoded data... how can this be classed as a big request? I can almost fit it on a floppy disk....

it's just minute bars for SPX futures for just a few months... this was the small one.

I'm planning to download a decade or more of it and for 100s more root stems. why does it take such a long time for such a small amount of data?

should I expect it to really take weeks to download all that data? can you help me find optimizations or performance improvements?


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.