Performance issues when fetching large amounts of data

I'm using the Dex2 COM API for VBA to get fundamental and historical performance data (20 TR. fields) for about 10k instruments. It takes me from 20min up to more than 1h, depending of the time of the day, which I think it's a very long time!
Are there best-practices and concrete peak-times which we should avoid? For example, around 8am seems to be much faster than 5pm (European Time).
Second question about efficiency: i'm sending requests in batches of 500 instruments for 20 TR. fields - is that optimal or should I rather ask for batches of 2000 instruments but only 5 fields? Any experience with the best-performing combination?
Third question about efficiency: how slow is the AVAIL function? Does dividing the assets in categories (Bonds, Equities, Funds) make sense in term of efficiency, so that I can tailor the query with the assetcategory-specific fields instead of using AVAIL?
Many thanks!
Best Answer
-
@nicola.pestalozzi
Eikon is an end user product. The amount of data you retrieve is borderline where you may want to consider using one of Thomson Reuters enterprise products specifically designed and engineered for bulk data downloads.
As for your questions, the answer to them can only be determined empirically. At Thomson Reuters we don't do any benchmarking of Eikon for bulk data download requests. In other words data retrieval times for bulk data download requests is not a key product performance indicator that we routinely monitor for Eikon.0
Categories
- All Categories
- 3 Polls
- 6 AHS
- 36 Alpha
- 166 App Studio
- 6 Block Chain
- 4 Bot Platform
- 18 Connected Risk APIs
- 47 Data Fusion
- 34 Data Model Discovery
- 684 Datastream
- 1.4K DSS
- 615 Eikon COM
- 5.2K Eikon Data APIs
- 10 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- 3 Trading API
- 2.9K Elektron
- 1.4K EMA
- 250 ETA
- 554 WebSocket API
- 37 FX Venues
- 14 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 23 Messenger Bot
- 3 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 60 Open Calais
- 275 Open PermID
- 44 Entity Search
- 2 Org ID
- 1 PAM
- PAM - Logging
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 22 RDMS
- 1.9K Refinitiv Data Platform
- 643 Refinitiv Data Platform Libraries
- 4 LSEG Due Diligence
- LSEG Due Diligence Portal API
- 4 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.2K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 12 World-Check Customer Risk Screener
- 1K World-Check One
- 46 World-Check One Zero Footprint
- 45 Side by Side Integration API
- 2 Test Space
- 3 Thomson One Smart
- 10 TR Knowledge Graph
- 151 Transactions
- 143 REDI API
- 1.8K TREP APIs
- 4 CAT
- 26 DACS Station
- 121 Open DACS
- 1.1K RFA
- 104 UPA
- 193 TREP Infrastructure
- 228 TRKD
- 915 TRTH
- 5 Velocity Analytics
- 9 Wealth Management Web Services
- 90 Workspace SDK
- 11 Element Framework
- 5 Grid
- 18 World-Check Data File
- 1 Yield Book Analytics
- 46 中文论坛