For a deeper look into our DataScope Select REST API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

question

Upvotes
Accepted
9 1 0 1

Time and sales too huge and unable to download in compressed format

Hello. I'm using Stored & Scheduled Workflow to download Tick History Time And Sales from DSS.

When i download the file using Python, i noticed that the format is raw and too huge.

Below is a sample i tried.

https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractedFiles('VjF8MHgwOGQzYTE2MjA1Nzk5OTYyfA')/$value

The output is below in which i need to write into a file.

1708439541083.png

what can i do to download directly from the link below (GUI) ? in the Download link, the file size is already in gz format and relatively smaller. How can i get this using Python?

1708439816073.png

#technologytick-history-rest-apicompression
1708439541083.png (229.3 KiB)
1708439816073.png (32.6 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
79.2k 251 52 74

@lim.yongyik

Thank you for reaching out to us.

We have an article and Python sample regarding this workflow.

The article is DSS REST API: Downloading Completed Extraction Files by a Schedule Name. The Python example is on GitHub.

The download code is in the DSSClient.py.

    def download_file(self, extracted_file, aws):
         decode = True
         headers={}
         headers["Prefer"] = "respond-async"
         headers["Authorization"] = "Token "+self.__token__
         if aws == True:
             headers["X-Direct-Download"] = "True"
         if extracted_file["ExtractedFileName"].endswith("gz") == True:
             decode = False


         requestUrl = self.__dss_uri__+self.__download_file_endpoint__.format(extracted_file["ExtractedFileId"])
         r = requests.get(requestUrl,headers=headers,stream=True)
         chunk_size = 1024 
         with open(extracted_file["ExtractedFileName"], 'wb') as fd:
            for data in r.raw.stream(chunk_size,decode_content=decode):
                fd.write(data)

The code checks the file's surname before opening the stream. Please check if this code works.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
9 1 0 1

Thanks @Jirapongse. The key point here is decode = False to ensure the file is downloaded in .gz. However i noticed, although i use AWS to download, the speed is still slow. Any recommendation for this?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.