TRTH: Python Extract market depth data

Unknown · June 2017

Hi,

I'm in trouble to extract data from http response object.

I can't figure out how to extract data using job ID and $value endpoint. Use case reference says users can download the report by folowing final step, but can't find any gzip file in my directory.

How can I retrieve or find data from report_gzip or anyother types of objjects?

Thanks,

report_gzip= requests.get( "https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/RawExtractionResults('0x05c5d18a57bb2f96')/$value",
                            headers = header2, stream=True)

veerapath.rungruengrayubkul · June 2017

@YK_deprecated_1

The raw extraction result will be streamed in gzip format, and then stored in the Response object. You can use this method of the Python Requests module to save raw gzip streamed response to file.

Below is the sample code. You can also see this question.

filename = "result.csv.gz"

with open(filename,'wb') as f:
   for data in report_gzip.raw.stream(decode_content=False):
       f.write(data)

Unknown · June 2017

Thank you, @veerapath.rungruengrayubkul

It works perfect on my environment. I was just wondering how to extract gzip file from response object, but it's now crystal clear.

Thanks again!

veerapath.rungruengrayubkul · June 2017

https://community.developers.refinitiv.com/discussion/comment/15272#Comment_15272

@YK_deprecated_1

I have just realized that the sample code I provided might lead to the issue that data will not be completely decompressed if the extracted result is huge.

I have edited the sample code in my answer. Please use the new sample code instead.

veerapath.rungruengrayubkul · June 2017

https://community.developers.refinitiv.com/discussion/comment/15272#Comment_15272

@YK_deprecated_1

If you want to process the extracted result instead of writing to a gzip file, you need to use gzip to decompress the data. Below is the sample code.

import io
import gzip

fd = io.BytesIO()
for data in report_gzip.raw.stream(decode_content=False):
    fd.write(data)


#decompress data
decompress_data = gzip.decompress(fd.getbuffer());
data = io.StringIO(decompress_data.decode('utf-8'))
    for line in data:
        print(line+'\n')

Hope this help.

Unknown · June 2017

@veerapath.rungruengrayubkul

ok will use that sample code if I need to process the result, thank you!

TRTH: Python Extract market depth data

Best Answer

Answers

Categories

EXPLORE OUR SITES

TRTH: Python Extract market depth data

Best Answer

Answers

Categories