AWS download - X-Direct-Download set to true returns 302 code and doesn't download file.

Samadhan
Samadhan Newcomer
in TRTH

My Java code:

con = (HttpURLConnection) obj.openConnection();

con.setInstanceFollowRedirects(false);

con.setRequestProperty("X-Direct-Download", "true");

con.setRequestMethod("GET");

con.setRequestProperty("Accept-Language", "en-US,en;q=0.5");

-------------------------------OUTPUT LOG-----------------------------------------------

Downloading File: I:\2018_Work\Out\ATH-2019-03-18-NORMALIZEDMP-Data-1-of-1.csv.gz

Trying to download first time->i=0

realURL :https://s3.amazonaws.com/tickhistory.query.production.hdc-ebd/normalised/ATH/2019-03-18/data/merged/ATH-2019-03-18-NORMALIZEDMP-Data-1-of-1.csv.gz?AWSAccessKeyId=XXXXXXX&Expires=1553209438&response-content-disposition=attachment; filename=ATH-2019-03-18-NORMALIZEDMP-Data-1-of-1.csv.gz&x-amz-request-payer=requester&Signature=XXXXX=

Output File is Creating Now

responseCode:302

---------------------------------------------

Somehow getting 302 response code but its not downloading the file with streams. Here is my code which downloads file. (Link printed in logs is downloadable though)

DataInputStream readerIS = new DataInputStream(con.getInputStream());

Files.copy (readerIS, Paths.get (fileName), StandardCopyOption.REPLACE_EXISTING);

Please suggest what else is needed here.

Tagged:

Best Answer

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    Answer ✓

    There are Java examples available in the Downloads section of TRTH - REST API.

    image

    The examples demonstrate how to download extraction result from AWS.

    You may also refer to this ADVISORY: DIRECTLY DOWNLOADING FROM AMAZON GENERATES ERROR.

    In short, the steps are:

    1. Configure your HTTP client to not redirect a request when it receives a 302 status code

    2. When you receive the Tick History download response with the 302 status code, read the
    response’s Location header field and send the original download request to that URL. Be sure to
    omit the Authorization header field and the X-Direct-Download header field

Categories