question

Upvotes
Accepted
Samadhan avatar image
1 ●1 ●1 ●3

AWS download - X-Direct-Download set to true returns 302 code and doesn't download file.

My Java code:

con = (HttpURLConnection) obj.openConnection();

con.setInstanceFollowRedirects(false);

con.setRequestProperty("X-Direct-Download", "true");

con.setRequestMethod("GET");

con.setRequestProperty("Accept-Language", "en-US,en;q=0.5");

-------------------------------OUTPUT LOG-----------------------------------------------

Downloading File: I:\2018_Work\Out\ATH-2019-03-18-NORMALIZEDMP-Data-1-of-1.csv.gz

Trying to download first time->i=0

realURL :https://s3.amazonaws.com/tickhistory.query.production.hdc-ebd/normalised/ATH/2019-03-18/data/merged/ATH-2019-03-18-NORMALIZEDMP-Data-1-of-1.csv.gz?AWSAccessKeyId=XXXXXXX&Expires=1553209438&response-content-disposition=attachment%3B%20filename%3DATH-2019-03-18-NORMALIZEDMP-Data-1-of-1.csv.gz&x-amz-request-payer=requester&Signature=XXXXX%3D

Output File is Creating Now

responseCode:302

---------------------------------------------

Somehow getting 302 response code but its not downloading the file with streams. Here is my code which downloads file. (Link printed in logs is downloadable though)

DataInputStream readerIS = new DataInputStream(con.getInputStream());

Files.copy (readerIS, Paths.get (fileName), StandardCopyOption.REPLACE_EXISTING);

Please suggest what else is needed here.

tick-history-rest-apiDownloadaws
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Jirapongse avatar image   Jirapongse

@Samadhan

Hi,

Thank you for your participation in the forum.

Is the reply below satisfactory in answering your question?

If yes please click the 'Accept' text next to the most appropriate reply. This will guide all community members who have a similar question.

Otherwise please post again offering further insight into your question.

Thanks,

AHS

Alex Putkov.1 avatar image   Alex Putkov.1

@Samadhan
Please be informed that a reply has been verified as correct in answering the question, and has been marked as such.
Thanks,
-AHS

1 Answer

· Write an Answer
Upvotes
Accepted
Jirapongse avatar image
79.2k ●251 ●52 ●74

There are Java examples available in the Downloads section of TRTH - REST API.

The examples demonstrate how to download extraction result from AWS.

You may also refer to this ADVISORY: DIRECTLY DOWNLOADING FROM AMAZON GENERATES ERROR.

In short, the steps are:

1. Configure your HTTP client to not redirect a request when it receives a 302 status code

2. When you receive the Tick History download response with the 302 status code, read the response’s Location header field and send the original download request to that URL. Be sure to omit the Authorization header field and the X-Direct-Download header field


java.jpg (38.0 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.