For a deeper look into our DataScope Select REST API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

question

Upvotes
Accepted
Md.Hisham.TR avatar image
35 ●0 ●1 ●5

Downloading compressed file on TRTH

I run into out of memory exception when I add new lines to response body by "using (var gzip = new GZipInputStream(streamResponse.Stream))" after "var streamResponse = ExtractionsContext.GetReadStream(extractionResult);".

I tried to get L1 data for one day for two chain ric "0#HSI:", "0#HCEI:" and got the above issue

is it possible that I just download the compressed file only? How can I do that? I will decompress the file in later stages.

Or is there a better option regarding my above issue?


It does work if I submit a request with a smaller number of identifiers. But this is not the ideal solution.

Though I will not run into “out of memory” exception, the needed time is still way too long.

dss-rest-apidatascope-selectdsstick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

pimchaya.wongrukun01 avatar image   pimchaya.wongrukun01

Hello @Mohamed.Hisham

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query?

If yes, please click the 'Accept' text next to the reply. This will guide all community members who have a similar question. Otherwise please post again offering further insight into your question.

Thanks,

AHS

veerapath.rungruengrayubkul avatar image   veerapath.rungruengrayubkul

Please be informed that a reply has been verified as correct in answering the question, and has been marked as such.

Thanks,

AHS

1 Answer

· Write an Answer
Upvotes
Accepted
Jirapongse avatar image
78.8k ●250 ●52 ●74

@Mohamed.Hisham

Please refer to the .Net SDK Tutorial 5: On Demand extraction, file IO tutorial.

It demonstrates how to set AutomaticDecompression to false and save the data in compressed format.

//Direct download from AWS ?
if (awsDownload) { extractionsContext.DefaultRequestHeaders.Add("x-direct-download", "true"); };

DssStreamResponse streamResponse = extractionsContext.GetReadStream(extractionResult);
using (FileStream fileStream = File.Create(dataOutputFile))
    streamResponse.Stream.CopyTo(fileStream);

//Reset header after direct download from AWS ?
if (awsDownload) { extractionsContext.DefaultRequestHeaders.Remove("x-direct-download"); };

Console.WriteLine("Saved the compressed data file to disk:\n" + gzipDataOutputFile);
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.