Multiple Rics On demand extraction returns only one Ric

sd
sd Newcomer

Hello,

When running a multiple Rics On demand extraction, it returns only the first Ric.

The code I am using is the below code inspired from the DSS api example

What am I doing wrong?

Thank you for your help on this.

ExtractionsContext.Options.AutomaticDecompression = true; //Decompress gzip to plain text

// UTC start and endDate

var requestDate = new DateTime(2017, 7, 27);

var utcStartDate = requestDate.AddHours(8);

var utcEndDate = requestDate.AddHours(16);

var startDate = new DateTimeOffset(utcStartDate);

var endDate = new DateTimeOffset(utcEndDate);

var condition = new TickHistoryTimeAndSalesCondition
{
ReportDateRangeType = ReportDateRangeType.Range,

QueryStartDate = startDate,

QueryEndDate = endDate,

ApplyCorrectionsAndCancellations = false,

ExtractBy = TickHistoryExtractByMode.Ric,

MessageTimeStampIn = TickHistoryTimeOptions.GmtUtc,

SortBy = TickHistorySort.SingleByRic,
DisplaySourceRIC = true
};

var rics = new string[] { "AAL.L", "MMM.N" };

var instrumentIdentifiers = rics.Select(x => InstrumentIdentifier.Create(IdentifierType.Ric, x)).ToArray();

var indentifierList = new InstrumentIdentifierList { InstrumentIdentifiers = instrumentIdentifiers };

var request = new TickHistoryTimeAndSalesExtractionRequest
{
Condition = condition,
ContentFieldNames = FIELDS,
IdentifierList = indentifierList
};

Task<RawExtractionResult> resultTask;

RawExtractionResult result;

try

{

resultTask = ExtractionsContext.ExtractRawAsync(request);

result = await resultTask;

}

catch (Exception ex)
{


return;
}

Welcome!

It looks like you're new here. Sign in or register to get started.

Best Answer

  • Christiaan Meihsl
    Answer ✓

    @sd, I just tried modifying our latest .Net sample 5 (it was updated on 28 June, along with the related tutorial), available from the downloads tab, to make the same request you make. It delivers data for several instruments (I tried with 5 RICs: ALVG.DE, AAL.L, IBM.N, MMM.N,0001.HK).

    I attached the modified Program.cs file that requests the same data as you programcs.txt (modified the extension). Note that it reads the RICs from a file.

    To install and run the code refer to .Net tutorial 1.

    Hope this helps.

Answers

  • zoya faberov
    zoya faberov ✭✭✭✭✭

    Hello @sd,

    Please try with automaticDecompression = false,

    save on disk and unzip with 7zip?

    Please refer to this question for the complete answer:

    Multi file save on disk

    Let us know if this helped?

    Thanks,

    -AHS

  • sd
    sd Newcomer

    Thank you very much. It seems to work with this set up. I will keep you posted in case of issue

  • https://community.developers.refinitiv.com/discussion/comment/16751#Comment_16751

    Ok, glad it works for you now.

  • Hello,

    In my case, I tried with the RICs "FEXDZ8" and "FEXDZ9", on August 14 for example. When I submit a Time and Sales request (trades and quotes) with both RICs, I get all the data for the first instrument (FEXDZ8) and just a few lines from the second one (FEXDZ9). On the other hand, if I submit the same request with just FEXDZ9, I get the full response. Have you ever experienced this issue? I tried both of the proposed solutions and none of them seems to work.

    Thank you for your help.

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    https://community.developers.refinitiv.com/discussion/comment/17456#Comment_17456

    You should try to retrieve the file as a raw gzip file instead of decompressed data (csv).

    If you are using C#, please change ExtractionsContext.Options.AutomaticDecompression to false and then change the codes that your write the result to file. For more information, please refer to this question.

  • https://community.developers.refinitiv.com/discussion/comment/17501#Comment_17501

    Hi,

    Thank you for your reply. I already tried to follow this method, but the generated ".gzip" file is apparently not an archive (see the attached screenshot). Am I missing something?

    image

  • https://community.developers.refinitiv.com/discussion/comment/17501#Comment_17501

    Actually, it seems that the file I download is a CSV file regardless of the AutomaticDecompression settings.

    using (var response = extractionsContext.RawExtractionResultOperations.GetReadStream(result))
    using (var fileStream = File.Create("savedExtraction.gzip"))
    response.Stream.CopyTo(fileStream);

    In my case, the code above just writes the CSV data in a file names "savedExtraction.gzip". If I rename it as "savedExtraction.csv" then I am able to open it as a normal CSV file.

  • Hi,

    Thank you for replying. I confirm that I already perform this check.

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    https://community.developers.refinitiv.com/discussion/comment/17532#Comment_17532

    Please verify that AutomaticDecompression is set to false at the beginning before calling ExtractRawAsync.

    ExtractionsContext.Options.AutomaticDecompression = false; 

    If the application is still get the csv file, please share the full code. Therefore, we can test the code.

  • https://community.developers.refinitiv.com/discussion/comment/17542#Comment_17542

    Hi,

    The code I used is the one from the C# example in the "Downloads" section (Dss.Api.Examples). You can test with the example named "Request Raw Transaction Data - Save to File" in the "Tick History" section (see the attached picture). Of course, this refers to raw extraction data, not a Time and Sales request. The issue, however, is the same.

    image

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    https://community.developers.refinitiv.com/discussion/comment/17538#Comment_17538

    Please try the latest C# example application. I have commented all Status.Notify codes in RequestRawTransactionsDataAndSaveToGzipFile() function.

    image

    Then, the output savedExtraction.gzip is created properly.

  • https://community.developers.refinitiv.com/discussion/comment/17538#Comment_17538

    I made the same changes to the example application, but the result is the same: a csv file disguised as a gzip one.

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    https://community.developers.refinitiv.com/discussion/comment/17554#Comment_17554

    What are the versions of ThomsonReuters.Dss.Core.RestApi.common.dll and ThomsonReuters.Dss.RestApi.Client.dll used by the example?

    I am using 11.1.527.

  • I am using 11.1.527 as well.

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    Are you using proxy?

    Please uncomment Status.Notify codes in RequestRawTransactionsDataAndSaveToGzipFile() function then set AutomaticDecompression to false.

    ExtractionsContext.Options.AutomaticDecompression = false; 

    Run the Tick History: Request Raw Transactions Data – Save to File example. After receiving the response, please verify the HTTP Response from STEP# 2. You should see the binary data with 1F 8B 08 xxx.

    image

  • I am not using a proxy. The response that I get is different from yours:

    image

  • I have also the same problem: csv file disguised as a gzip file. It's annoying because I need to store zipped files on my server.

    image

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    https://community.developers.refinitiv.com/discussion/comment/17560#Comment_17560

    In the response, I found Via header.

    Via: veg14pxmzdd7

    From this page:

    The Via general header is added by proxies, both forward and reverse proxies, and can appear in the request headers and the response headers. It is used for tracking message forwards, avoiding request loops, and identifying the protocol capabilities of senders along the request/response chain

    I assume that this proxy may decompress the data and send the csv back to the application. Can you verify this with your network team?

  • https://community.developers.refinitiv.com/discussion/comment/17584#Comment_17584

    Hi,

    Sorry, I checked quickly through my browser and did not identify any proxy, but it seems that you are right about the fact that I am behind a proxy. This could be in fact the reason for which the gzipped data is decompressed. I checked during the weekend on another network and I got the same reply of yours.

  • https://community.developers.refinitiv.com/discussion/comment/17584#Comment_17584

    Hi,

    Given that the problem is most likely related to the proxy, as you mentioned, are you aware of any other workaround for being able to get all the requested data? For example, if I have a large request to do (e.g. several securities on several days) then with the version 1 of the SOAP API I was able to split my request into smaller jobs containing one securities per single day. I tried the same method via the REST API but it seems quite slow to perform a single request.

    Alternatively, in version 1 there was the option to split the request by RIC and by day. Is it possible in version 2 too?

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @mantonello

    The workaround should be setting "Range: bytes=0-" in the request's header. Then the application will get 206 PartialContent with binary data. I assume that the proxy will not decompress this content.

    The code looks like:

    ExtractionsContext.SetRequestHeaders(() => { return new System.Collections.Generic.Dictionary<string, string> { { "Range", "bytes=0-" }}; });
    using (var response = ExtractionsContext.RawExtractionResultOperations.GetReadStream(result, 0))

    The code sets the "Range" header before calling GetReadStream.

    If you verify the HTTP, you will see:

    image

    However, the output file may be corrupted. The workaround is removing Status.Notify and Status.EndNotify from the code to get the complete file, as mentioned in the above comment.

  • https://community.developers.refinitiv.com/discussion/comment/17760#Comment_17760

    Hi,

    Thank you for your reply. Unfortunately, it doesn't work as expected: the file is still decompressed in a csv one.

Welcome!

It looks like you're new here. Sign in or register to get started.

Welcome!

It looks like you're new here. Sign in or register to get started.