Discover Refinitiv
MyRefinitiv Refinitiv Perspectives Careers
Created with Sketch.
All APIs Questions & Answers  Register |  Login
Ask a question
  • Questions
  • Tags
  • Badges
  • Unanswered
Search:
  • Home /
  • TRTH /
avatar image
Question by s61670 · Apr 07 at 05:51 AM · TRTHtrthv2

file size information

help

How do request file size on a Refinitiv server via REST API?

At the forum gave advice, to look through the DataScope. What if I download a historical data on individual days and such files thousands? For example, I want to download RIC for 9 years. But download not 1 file, download for 1 day (StartDate_EndDat=1 day)

Creating thousands of reports per DataScope for thousands of days to view the size of each file is not effective.

How do I request file size information through the REST API?

I want to automate the entire process in order to know for sure that when downloading information is not lost.

For example, I downloaded RIC, it weighs 23 Gb, but I know that other RIC by this symbol, the average file size is 50 Gb.

Every time look at DataScope? Not an effective waste of time ((Must be some way through REST API

download speed is not stable.

I have 10 Gb\s Internet.

The actual speed download can be 90 mb\s and then up to 0.01 mb\s


The forum wrote that historical data is stored on AWS

If the Internet speed is high, then you can download very quickly.

Practice shows that the information is not true.

During working days, download speed decreases to 0.01 mb\s - 2 mb\s for many hours, even if 10 gb\s Internet.

It is not possible to download historical data efficiently and quickly.

546.png (40.8 KiB)
223.png (41.3 KiB)

People who like this

0 Show 0
Comment
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

2 Replies

  • Sort: 
avatar image
REFINITIV
Answer by zoya.farberov · Apr 07 at 07:42 PM

Hello @s61670,

I believe that in order to create a well-designed, fully automated solution, you would have to have a rough idea how large is each of the data sets that you require, at the time of design of the solution, and also to check for and handle any errors during run, that can prevent the download from completing.

The speed of the download, as you observe, you are absolutely correct, can differ. And as an example for a proactive design strategy, for exceptionally large requirements, it sometimes may make sense to run those well outside of the relevant market's opening and closing hours, to proactively decrease the chance of heavy load on the service coinciding with your job requests.

---

Let me step back at this point, and try to find out more about your overall Tick History data requirements, in order to try to help?

  • What content set(s) are your looking to download?
  • Does your requirement include a a substantial amount of historical data, to be pulled only once, and then to be updated with the significantly smaller daily changes/deltas?
  • Please include any other info, detail or limitation, that are part of the requirements that you work to meet, that can be relevant to the design and the solution?
  • I am going to also alert your Refinitiv account team, so that they can be of most help as well, to help with working out the best solution for your requirements.
Comment

People who like this

0 Show 3 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
s61670 · Apr 07 at 08:16 PM 0
Share

We are interested in chain RICs: ES and URO (old RIC ESCC), Market Price and LegacyMarketDepth

Does your requirement include a a substantial amount of historical data, to be pulled only once, and then to be updated with the significantly smaller daily changes/deltas?

Yeah, you're right.



avatar image
s61670 · Apr 07 at 08:44 PM 0
Share

For example, ESH1, we plan to download every single day (Market_Price + RawLegacyMarketDepth).

The option to download ESH1 one file (the first day _ day_Expiry) does not interest us.

1. Now the situation is such that the server has an embargo time when we cannot send requests for downloads.

2. We also don't understand how we can control file size information.

3. Download speed is unstable.

avatar image
s61670 · Apr 07 at 08:46 PM 0
Share

single day = 1file

ESH1 = a few hundred days, we want a few hundred files.

avatar image
REFINITIV
Answer by zoya.farberov · Apr 07 at 09:17 PM

Hello @s61670,

So in my understanding you are looking to download TickHistoryRaw content, for instrument ESH1, of Domain MarketPrice and RawLegacyMarketDepth. This data is every tick that market has issued, so it is indeed large.

Let me suggest, and you can tune/improve the strategy.

If you request one day, 24 hours worth of TickHistoryRaw content for MarketPrice, for 3 consecutive business days (for example, for 2021.03.29, for 2021, 03.30 and for 2021.03.31), and once you have the 3 results back, average the size of them, you should have a rough idea on the expected size of this content for 1 day, same goes for RawLegacyMarketPrice.

Next, you can look to split your required request into increments, define how many days per request makes the most sense.

You can further decide if you like to design concurrency into your download strategy, let's say two threads, running concurrently, see Tick History REST API User Guide -> Concurrent Requests to learn more.

If you would like to learn more of the embargo rules for this instrument, which looks to be important for your design, one option is to contact RTH content experts via Refinitiv Helpdesk Online -> Content -> RTH with your specific questions, to get the addressed.

Request processing time and download speed depend on many factors, and you should aim to design your solution as asynchronous, issue a request and keep checking the status till the response is ready, without reliance on the result being available at a specific time.

Hope this information is helpful to you.

Comment

People who like this

0 Show 0 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Watch this question

Add to watch list
Add to your watch list to receive emailed updates for this question. Too many emails? Change your settings >
10 People are following this question.

Related Questions

TRTHv1 Qualifier in TimeAndSales/Trade for NSQ and NYQ

How can i get previous day close for each dy and symbol when making a trth ElektronTimeseries request

​List of files in VBD

Is there a way to distinguish options with multiple expiry date?

Missing TRTH HistoricalChainResolution Constituents for 0#.AORD

  • Feedback
  • Copyright
  • Cookie Policy
  • Privacy Statement
  • Terms of Use
  • Careers
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Alpha
  • App Studio
  • Block Chain
  • Bot Platform
  • Calais
  • Connected Risk APIs
  • DSS
  • Data Fusion
  • Data Model Discovery
  • Datastream
  • Eikon COM
  • Eikon Data APIs
  • Elektron
    • EMA
    • ETA
    • WebSocket API
  • Legal One
  • Messenger Bot
  • Messenger Side by Side
  • ONESOURCE
    • Indirect Tax
  • Open PermID
    • Entity Search
  • Org ID
  • PAM
    • PAM - Logging
  • ProView
  • ProView Internal
  • Product Insight
  • Project Tracking
  • Refinitiv Data Platform
    • Refinitiv Data Platform Libraries
  • Rose's Space
  • Screening
    • Qual-ID API
    • Screening Deployed
    • Screening Online
    • World-Check One
    • World-Check One Zero Footprint
  • Side by Side Integration API
  • TR Knowledge Graph
  • TREP APIs
    • CAT
    • DACS Station
    • Open DACS
    • RFA
    • UPA
  • TREP Infrastructure
  • TRIT
  • TRKD
  • TRTH
  • Thomson One Smart
  • Transactions
    • REDI API
  • Velocity Analytics
  • Wealth Management Web Services
  • World-Check Data File
  • Explore
  • Tags
  • Questions
  • Badges