question

Upvotes
Accepted
3 1 1 3

How to download venue by detail files in .gz format for TRTH v2 API?

Hi,

I am able to use the UserPackageDeliveries/$value endpoint to get a response from the API from the .gz venue by day files but this response is in form of a csv object. This is huge in terms of size and hence the download time is also pretty high. Is there a way to directly download the .gz files for a subscription ID or package deliveryID directly?

I am using Powershell for this.

Thanks,

Ashish

tick-history-rest-apiapi
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

Upvote
Accepted
11.3k 25 8 14

@ashish.singh2

Normally the data will be delivered in gzip encoding stream.

Are you using the Poweshell's Invoke-WebRequest orInvoke-RestMethod? As far as I know, the commands will return decompressed response data. It seems that the commands automatically decompress response. If you want to retreive the gzip data directly, you can use other command.

Below is the sample code. I use the System.Net.WebRequest to download data and then save the data to file.

$GetResultUrl = "https://hosted.datascopeapi.reuters.com/RestApi/v1/StandardExtractions/UserPackageDeliveries('" + $PackageDeliveryId + ''')/$value'

# Creating a new HttpWebRequest object.
[System.Net.HttpWebRequest]$oHttpWebRequest = [System.Net.WebRequest]::Create($GetResultUrl)
# This may be superflous if your HTTP server doesn't use compression.
$oHttpWebRequest.AutomaticDecompression = [System.Net.DecompressionMethods]::None
$oHttpWebRequest.Headers.Add("Authorization",'Token'+$token)
$oHttpWebRequest.Headers.Add("X-Direct-Download",'true')

$sr = $oHttpWebRequest.GetResponse().GetResponseStream()

$Wrt = [System.IO.File]::Create("D:\output.gz")
$Buffer = New-Object Byte[] 1024

Do {
	$BytesRead = $sr.Read($Buffer, 0, $Buffer.Length)
	$Wrt.Write($Buffer, 0, $BytesRead)
} While ($BytesRead -gt 0)

$sr.close()
$sr.Dispose()

$Wrt.Flush()
$Wrt.Close()
$Wrt.Dispose()

Moreover, new DataScope Select 11.1 REST API now supports downloading Venue by Day files faster by retrieving them directly from the Amazon cloud in which they are hosted. Below is the information from the Thomson Reuters Tick History 11.1 REST API User Guide / Version 3.0.


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hey,

Thanks for the response, I shall try this asap. Also, I did try the X-Direct-Download:True but for some reason it showed up an error "Forbidden". Do you have any idea on this error? Do I need some sort of access on AWS in order to download files from there directly?

Thanks,

Ashish

@ashish.singh2 I have found the forbidden error, once I use the Invoke-WebRequest with the X-Direct-Download:true header. However, the header works fine with my sample code. Could you try the code?

ashish.singh2 avatar image ashish.singh2 veerapath.rungruengrayubkul

My trial has expired for now, I will try your code as soon as I get it renewed. Thanks a lot for all the help :)

Show more comments