why is a simple retrieval taking so long?

opsp
opsp Explorer
in DSS

i have a script that I use for other chains (like vix futures) and it takes a couple of minutes to run. I've now been waiting 2 hours for this... why does it take so long?

I just want minute bars for SPX futures for a few months...

{
    "ExtractionRequest": {
        "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TickHistoryIntradaySummariesExtractionRequest",
        "ContentFieldNames": [
            "Open",# "Open Ask", "Open Bid",
            "High",# "High Ask", "High Bid",
            "Low",# "Low Ask", "Low Bid",
            "Last", "Close Ask", "Close Bid",
            "Volume"
        ],
        "IdentifierList": {
            "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
            "InstrumentIdentifiers": [
                #{"Identifier": ".SPX", "IdentifierType": "Ric"},
                {"Identifier": "0#ES:", "IdentifierType": "ChainRIC"}
                #{"Identifier": "0#VX:", "IdentifierType": "ChainRIC"}
            ],
            "ValidationOptions": {"AllowHistoricalInstruments": "true"},
            "UseUserPreferencesForValidationOptions": "false"
        },
        "Condition": {
            "MessageTimeStampIn": "GmtUtc",
            "ReportDateRangeType": "Range",
            "QueryStartDate": "2021-02-01T00:00:00.000Z",
            "QueryEndDate": "2021-06-01T23:59:00.000Z",
            "SummaryInterval": "OneMinute",    # "FifteenMinutes" # OneSecond # OneMinute
            "DisplaySourceRIC": "true"
        }
    }
}
Tagged:

Best Answer

  • zoya faberov
    zoya faberov ✭✭✭✭✭
    Answer ✓

    Hello @opsp,

    Absolutely, glad to share my thoughts. RTH HTTP REST on demand, custom report requesting is primarily intended as a light way of requesting RTH content.

    As you are looking to request 100s of chains and a decade or more of data, I would consider design and product options:

    • The first approach I would consider, is Venue By Day or VBD. This will allow you to request a a complete exchange day's worth of data, compacted by zip and next select what you require from it, for details see:

    Data Dictionary - Refinitiv Tick History Venue By Day Message Reporting and REST API Tutorial 2: Retrieving VBD files.

    VBD is an additional product.

    Hope this information helps


Answers

  • opsp
    opsp Explorer

    it's still running....should I kill it? how long should it take? (approx)

  • zoya faberov
    zoya faberov ✭✭✭✭✭

    Hello @opsp,

    From what I see, this is a very large request. So would expect it to take long.

    Have you tried requesting OneMinute bars, on the same chain RIC, for 1 day interval of time?

    In a general scenario, if a request is taking unreasonably long time all of a sudden, especially if the same request takes a short time otherwise, I would suggest verifying with My Refinitiv -> Service Alerts section, to know of any potential outage that could be impacting your request at the time.

    Hope this information helps

  • opsp
    opsp Explorer

    it's 3MB of sensibly encoded data... how can this be classed as a big request? I can almost fit it on a floppy disk....

    it's just minute bars for SPX futures for just a few months... this was the small one.

    I'm planning to download a decade or more of it and for 100s more root stems. why does it take such a long time for such a small amount of data?

    should I expect it to really take weeks to download all that data? can you help me find optimizations or performance improvements?


Categories