Time and sales too huge and unable to download in compressed format

lim.yongyik

Hello. I'm using Stored & Scheduled Workflow to download Tick History Time And Sales from DSS.

When i download the file using Python, i noticed that the format is raw and too huge.

Below is a sample i tried.

https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractedFiles('VjF8MHgwOGQzYTE2MjA1Nzk5OTYyfA')/$value

The output is below in which i need to write into a file.

what can i do to download directly from the link below (GUI) ? in the Download link, the file size is already in gz format and relatively smaller. How can i get this using Python?

Jirapongse

@lim.yongyik

Thank you for reaching out to us.

We have an article and Python sample regarding this workflow.

The article is DSS REST API: Downloading Completed Extraction Files by a Schedule Name. The Python example is on GitHub.

The download code is in the DSSClient.py.

    def download_file(self, extracted_file, aws):
         decode = True
         headers={}
         headers["Prefer"] = "respond-async"
         headers["Authorization"] = "Token "+self.__token__
         if aws == True:
             headers["X-Direct-Download"] = "True"
         if extracted_file["ExtractedFileName"].endswith("gz") == True:
             decode = False


         requestUrl = self.__dss_uri__+self.__download_file_endpoint__.format(extracted_file["ExtractedFileId"])
         r = requests.get(requestUrl,headers=headers,stream=True)
         chunk_size = 1024 
         with open(extracted_file["ExtractedFileName"], 'wb') as fd:
            for data in r.raw.stream(chunk_size,decode_content=decode):
                fd.write(data)

The code checks the file's surname before opening the stream. Please check if this code works.

lim.yongyik

Thanks @Jirapongse. The key point here is decode = False to ensure the file is downloaded in .gz. However i noticed, although i use AWS to download, the speed is still slow. Any recommendation for this?