question

Upvotes
Accepted
Samadhan avatar image
1 ●1 ●1 ●3

AWS download - X-Direct-Download set to true returns 302 code and doesn't download file.

My Java code:

con = (HttpURLConnection) obj.openConnection();

con.setInstanceFollowRedirects(false);

con.setRequestProperty("X-Direct-Download", "true");

con.setRequestMethod("GET");

con.setRequestProperty("Accept-Language", "en-US,en;q=0.5");

-------------------------------OUTPUT LOG-----------------------------------------------

Downloading File: I:\2018_Work\Out\ATH-2019-03-18-NORMALIZEDMP-Data-1-of-1.csv.gz

Trying to download first time->i=0

realURL :https://s3.amazonaws.com/tickhistory.query.production.hdc-ebd/normalised/ATH/2019-03-18/data/merged/ATH-2019-03-18-NORMALIZEDMP-Data-1-of-1.csv.gz?AWSAccessKeyId=XXXXXXX&Expires=1553209438&response-content-disposition=attachment%3B%20filename%3DATH-2019-03-18-NORMALIZEDMP-Data-1-of-1.csv.gz&x-amz-request-payer=requester&Signature=XXXXX%3D

Output File is Creating Now

responseCode:302

---------------------------------------------

Somehow getting 302 response code but its not downloading the file with streams. Here is my code which downloads file. (Link printed in logs is downloadable though)

DataInputStream readerIS = new DataInputStream(con.getInputStream());

Files.copy (readerIS, Paths.get (fileName), StandardCopyOption.REPLACE_EXISTING);

Please suggest what else is needed here.

tick-history-rest-apiDownloadaws
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

Upvotes
Accepted
Jirapongse avatar image
47.3k ●111 ●44 ●60

There are Java examples available in the Downloads section of TRTH - REST API.

The examples demonstrate how to download extraction result from AWS.

You may also refer to this ADVISORY: DIRECTLY DOWNLOADING FROM AMAZON GENERATES ERROR.

In short, the steps are:

1. Configure your HTTP client to not redirect a request when it receives a 302 status code

2. When you receive the Tick History download response with the 302 status code, read the response’s Location header field and send the original download request to that URL. Be sure to omit the Authorization header field and the X-Direct-Download header field


java.jpg (38.0 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.