AWS server return different data type when using REST API with python2 instead of python3

Hi,


I am trying to use REST API in python to extract tick history. I was able to follow the example provided in the download section and extract tick data successfully (in .csv.gzip format) in my development environment, which is in python3. However, when I tested exactly the same code in my production environment (python2) I got an error saying the return data is not in gzip format.

I checked the `Content-Type` field in HTTP response, I found AWS return a xml data, instead of gzip, in my production environment. Message as below:

`Content response headers (AWS server): type: application/xml`

The only difference, as far as I can tell for now, is that I am running with python2. Would this be the problem?

Thanks so much for helping

Best Answer

  • Hi @timothy.liu.1010,

    I have tried the TRTH_OnDemand_IntradayBars.py example which is modified to use TickHistoryTimeAndSales extraction with Python 2.7.9 version, but cannot find the issue. Could you provide the request message you are using? Is there any modification on the example except the request?

    Are you able replicate the issue on demand? Could you 1) enable debug logging, 2) add the X-Client-Session-Id header, and 3) log the response header of the RawExtractionResults endpoint?

    Please provide the application's console log.

    Below is the sample of modified code on Step 5.

    #Step 5: get the extraction results, using the received jobId.
    ...
    #   Advisory: avoid incomplete output - decompress then download
    # 1) enable debug logging
    logging.basicConfig(level=logging.DEBUG)

    requestUrl = "https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/RawExtractionResults" + "('" + jobId + "')" + "/$value"

    #AWS requires an additional header: X-Direct-Download
    if useAws:
        requestHeaders={
            "Prefer":"respond-async",
            "Content-Type":"text/plain",
            "Accept-Encoding":"gzip",
            "X-Direct-Download":"true",
            "Authorization": "token " + token,
    # 2) add the X-Client-Session-Id header
            "X-Client-Session-Id": "9FBAA1E1-DFF3-435C-A0CA-1C9A6C46C266"
        }
    else:
        requestHeaders={
            "Prefer":"respond-async",
            "Content-Type":"text/plain",
            "Accept-Encoding":"gzip",
            "Authorization": "token " + token
        }

    r5 = requests.get(requestUrl,headers=requestHeaders,stream=True)
    #Ensure we do not automatically decompress the data on the fly:
    r5.raw.decode_content = False
    if useAws:
    # 3) log the response header
        for red in r5.history:
            print (red.headers)
        
        print ('Content response headers (AWS server): type: ' + r5.headers["Content-Type"] + '\n')
        #AWS does not set header Content-Encoding="gzip".

Answers