Extracting TRTH Time and Sales Data to a File

Hello, When I try to extract the Time and Sales Data to a file I only get partial data. I've attached a snippet of my code for the extraction request which writes to a file on my local machine. Am I missing a step like aggregating files?

  StreamWriter clear = new StreamWriter(dataOutputFile, false);
clear.Close();
Console.WriteLine("Cleared data output file " + dataOutputFile + "\n");


TickHistoryTimeAndSalesExtractionRequest extractionRequest = new TickHistoryTimeAndSalesExtractionRequest {
IdentifierList = InstrumentIdentifierList.Create(instrumentIdentifiers),
Condition = reportTemplate.Condition,
ContentFieldNames = { "Trade - Price", "Trade - Volume", "Quote - Bid Price", "Quote - Bid Size", "Quote - Ask Price", "Quote - Ask Size" }
};


extractionsContext.Options.AutomaticDecompression = true; //Decompress gzip to plain text
RawExtractionResult extractionResult = extractionsContext.ExtractRaw(extractionRequest);


Console.Write("{0:T} Extraction complete ... ", DateTime.Now);
DebugPrintAndWaitForEnter("");


sw = new StreamWriter(dataOutputFile, true);
Console.WriteLine("==================================== DATA =====================================");
DssStreamResponse streamResponse = extractionsContext.GetReadStream(extractionResult);
using (StreamReader reader = new StreamReader(streamResponse.Stream)) {
string line = reader.ReadLine();
if (string.IsNullOrEmpty(line)) {
Console.WriteLine("WARNING: no data returned. Check your request dates.");
sw.WriteLine("WARNING: no data returned. Check your request dates.");
} else {
//The first line is the list of field names:
sw.WriteLine(line);
Console.WriteLine(line);
//The remaining lines are the data:
//Variant 1: write all lines individually to file and console (and set a limit on number of lines)
for (int lineCount = 0; lineCount < maxDataLines && !reader.EndOfStream; lineCount++) {
line = reader.ReadLine();
sw.WriteLine(line);
Console.WriteLine(line);
}
//Variant 2: write all lines to either file or console (not both !):
//sw.WriteLine(line = reader.ReadToEnd());
//Console.WriteLine(line = reader.ReadToEnd());
}
}
sw.Close();
DebugPrintAndWaitForEnter("===============================================================================");
}

Best Answer

  • @helen.ristov

    -What is maxDataLines you set in your test application?

    if it's the same value as our tutorial which is 1000000, it may not enought to print unpacked result. Some report can generate more than 1000000 lines of data in csv format. You have to increase the value.

    -Can you provide more details about reportTemplate.condition you are testing? I would like to test the instrument with the same date range.

    -Please you change ExtractionsContext.Options.AutomaticDecompression to false and then change the codes that your write the result to file to

    //Download the result
    using (var response = ExtractionsContext.RawExtractionResultOperations.GetReadStream(result))
    using (var fileStream = File.Create("savedExtraction.gzip"))
    response.Stream.CopyTo(fileStream);

    Then you can unpack .gzip file and open savedExtraction.csv to compare number of lines from the unpacked data.

Answers