Discover Refinitiv
MyRefinitiv Refinitiv Perspectives Careers
Created with Sketch.
All APIs Questions & Answers  Register |  Login
Ask a question
  • Questions
  • Tags
  • Badges
  • Unanswered
Search:
  • Home /
  • TRTH /
avatar image
Question by chinelo.okoli · May 07, 2017 at 08:56 PM · tick-history-rest-api

Inconsistent result from DSS TRTH v2

Hi there,@manigandan.r

When i request the extraction result from a Time and Sales report schedule - i get different results using exactly the same request.

I see the file size is exactly the same but my call to response.getEntity().getContent() appears to return inconsistent results. E.g. the last 2 calls i made to the API, T&S returned 69937 vs 101031 lines.

This is how i set up my client:

private CloseableHttpClient httpclient = HttpClientBuilder.create().build();

.....

String urlGet = this.url + "/Extractions/ReportExtractions('"+reportExtractionId+"')/Files";

HttpGet request = new HttpGet(urlGet);

request.addHeader("Authorization", "Token "+this.sessionToken);

HttpResponse response = this.httpclient.execute(request);

BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));

StringBuffer result = new StringBuffer();

String line = "";

while ((line = rd.readLine()) != null) { result.append(line); }

JSONObject jsonGetResponse = new JSONObject(result.toString());

JSONArray valueJArray = jsonGetResponse.getJSONArray("value");

String ExtractedFileId = ""; String ExtractedFileName = ""; String FileType = ""; boolean success = false;

for (int i = 0; i < valueJArray.length(); i++)

{

ExtractedFileId = valueJArray.getJSONObject(i).getString("ExtractedFileId");

......

//FileType="Full"

urlGet = this.url + "/Extractions/ExtractedFiles('"+ExtractedFileId+"')/$value";

request = new HttpGet(urlGet);

request.addHeader("Authorization", "Token "+this.sessionToken);

response = this.httpclient.execute(request);

HttpEntity entity = response.getEntity();

rd = new BufferedReader(new InputStreamReader(entity.getContent()));

......

while ((line = rd.readLine()) != null) {...}

......

}

The same code works fine for Depth extraction which also has a gzipped file output.

Am i missing something here please?

Thanks

People who like this

0 Show 0
Comment
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

3 Replies

  • Sort: 
avatar image
REFINITIV
Best Answer
Answer by Christiaan Meihsl · May 31, 2017 at 12:29 PM

@chinelo.okoli

Sorry this took some time, but it was a complex nut to crack. The support team found a solution, kudos go to them.

The key problem is the merged GZip files cause issues for the default decompression methods. For Java, the Apache Commons Compress https://commons.apache.org/proper/commons-compress/index.html library seems like a safe option. However, it needs to be implemented correctly.

GZIPStream from the java.util.zip package will work if it is done on the local machine, but may fail if used while reading directly from the HTTP Stream.

Key factors for Apache Commons Compress implementation:

Disable Content Compression on the HTTP client

private CloseableHttpClienthttpclient = HttpClients.create().disableContentCompression().build();

Set decompressConcatenated = true when initializing the stream

GzipCompressorInputStream gis =new GzipCompressorInputStream(
myURLConnection.getInputStream(), true);

If these are set, you can use In-Stream Decompression and process the stream while downloading the file.

In a nutshell, here are the code details:

    Add the Apache Commons Compress library:

    import
    org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;

    Use this setting when declaring the httpclient:

    private CloseableHttpClient httpclient = HttpClientBuilder.create().disableContentCompression().build();

    Full code of a method to extract and display on the fly is attached.solutioncode.txt.

    I will update all our Java samples, in the next version they will use this method.


    solutioncode.txt (1.6 KiB)
    Comment

    People who like this

    0 Show 1 · Share
    10 |1500 characters needed characters left characters exceeded
    ▼
    • Viewable by all users
    • Viewable by moderators
    • Viewable by moderators and the original poster
    • Advanced visibility
    Viewable by all users

    Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

    avatar image
    REFINITIV
    Christiaan Meihsl ♦♦ · Jun 02, 2017 at 11:12 AM 0
    Share

    All the TRTH REST API Java samples have been checked, and updated where required to implement the correction. The new package, which also includes new samples and enhanced functionality, is available for download.

    avatar image
    REFINITIV
    Answer by Christiaan Meihsl · May 08, 2017 at 03:55 AM

    @chinelo.okoli, the code snippets you posted look ok.

    Could you share your entire code so I could test it ? Could you also share your report template and instrument list ? If you do not want to post these on the forum you can also send them to me directly.

    Added notes:

    Our sample code DSS2ImmediateScheduleTicksTRTHClient2 (available under the downloads tab of TRTH REST) was tested for extractions up to 14MB gziped (99MB unzipped) without issue. Could you try that one with your template and instrument list to see if it exhibits the same issue ?

    If you are using a pre-defined instrument list and report template on the server, I can send you a variant of DSS2ImmediateScheduleTicksTRTHClient2 that would use them which you could use for testing.

    When you get 2 files of different size, what does the last line of data in the file look like ? Is it truncated ?

    Comment

    People who like this

    0 Show 0 · Share
    10 |1500 characters needed characters left characters exceeded
    ▼
    • Viewable by all users
    • Viewable by moderators
    • Viewable by moderators and the original poster
    • Advanced visibility
    Viewable by all users

    Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

    avatar image
    Answer by chinelo.okoli · Jun 02, 2017 at 12:34 PM

    Thanks a lot Christiaan

    Comment

    People who like this

    0 Show 0 · Share
    10 |1500 characters needed characters left characters exceeded
    ▼
    • Viewable by all users
    • Viewable by moderators
    • Viewable by moderators and the original poster
    • Advanced visibility
    Viewable by all users

    Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

    Watch this question

    Add to watch list
    Add to your watch list to receive emailed updates for this question. Too many emails? Change your settings >
    10 People are following this question.

    Related Questions

    Cannot retreive RIC list

    Working R examples for Eikon with Lipper data

    TRTH REST API - Could you please provide python API code to retrieve extraction ID and also Python code to download extracted file.

    Is it possible to use ReportDataRangeType PerIdentifier for a TickHistoryIntradaySummariesExtractionRequest?

    EOD Implied volatility of equity stock options

    • Copyright
    • Cookie Policy
    • Privacy Statement
    • Terms of Use
    • Anonymous
    • Sign in
    • Create
    • Ask a question
    • Spaces
    • Alpha
    • App Studio
    • Block Chain
    • Bot Platform
    • Connected Risk APIs
    • DSS
    • Data Fusion
    • Data Model Discovery
    • Datastream
    • Eikon COM
    • Eikon Data APIs
    • Electronic Trading
      • Generic FIX
      • Local Bank Node API
      • Trading API
    • Elektron
      • EMA
      • ETA
      • WebSocket API
    • Intelligent Tagging
    • Legal One
    • Messenger Bot
    • Messenger Side by Side
    • ONESOURCE
      • Indirect Tax
    • Open Calais
    • Open PermID
      • Entity Search
    • Org ID
    • PAM
      • PAM - Logging
    • ProView
    • ProView Internal
    • Product Insight
    • Project Tracking
    • RDMS
    • Refinitiv Data Platform
      • Refinitiv Data Platform Libraries
    • Rose's Space
    • Screening
      • Qual-ID API
      • Screening Deployed
      • Screening Online
      • World-Check One
      • World-Check One Zero Footprint
    • Side by Side Integration API
    • TR Knowledge Graph
    • TREP APIs
      • CAT
      • DACS Station
      • Open DACS
      • RFA
      • UPA
    • TREP Infrastructure
    • TRKD
    • TRTH
    • Thomson One Smart
    • Transactions
      • REDI API
    • Velocity Analytics
    • Wealth Management Web Services
    • Workspace SDK
      • Element Framework
      • Grid
    • World-Check Data File
    • 中文论坛
    • Explore
    • Tags
    • Questions
    • Badges