Invalid response issue

sd
sd Newcomer

Hello,

When using the below code, I have in some occasions a bug in :

resultTask = ExtractionsContext.ExtractRawAsync(request);

result = await resultTask;

with an exception thrown with ex.message=""Invalid response, could not parse the following response: \r\n\r\n\r\n<html>\r\n\r\n <head id=\"Head1\">, ...." the message being to too long to be shown.

It seems that the server is not responding in a proper way anymore. Is there a maximum number of concurent ExtractRawAsync that could cause the issue, or anything else that could explain this?

Thank you for your help,

SD

The code:

var condition = new TickHistoryTimeAndSalesCondition

{

ReportDateRangeType = ReportDateRangeType.Range,

QueryStartDate = startDate,

QueryEndDate = endDate,

ApplyCorrectionsAndCancellations = false,

ExtractBy = TickHistoryExtractByMode.Ric,

MessageTimeStampIn = TickHistoryTimeOptions.GmtUtc,

SortBy = TickHistorySort.SingleByRic,

DisplaySourceRIC = true

};

var rics = InstrumentManager.InstrumentList.Select(x => x.RDTHRic).ToArray();

var instrumentIdentifiers = rics.Select(x => InstrumentIdentifier.Create(IdentifierType.Ric, x)).ToArray();

var indentifierList = new InstrumentIdentifierList { InstrumentIdentifiers = instrumentIdentifiers };

var request = new TickHistoryTimeAndSalesExtractionRequest

{


Condition = condition,

ContentFieldNames = FIELDS,

IdentifierList = indentifierList

};

Task<RawExtractionResult> resultTask;

RawExtractionResult result;

try

{

resultTask = ExtractionsContext.ExtractRawAsync(request);

result = await resultTask;

}

catch (Exception ex)


{

var message = ex.Message.Replace("\n", "").Replace("\r", "").Substring(0, 25);
InstrumentManager.EventsShowMsg("Error in task: " + message, false);

return;


}

Best Answer

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    Answer ✓

    I think that the Location field in the header in HTTP response has been changed.

    Location: http://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x05d1ac85e20b3016')

    The protocol is http, not https.

    If I use this URL to check the status of the request, it will return:

    <html>
    <head id="Head1">
    <link rel="alternate" type="text/xml" href="/DatascopeApi/v1/ExtractionService.asmx?disco" />
    <style type="text/css">

    BODY { color: #000000; background-color: white; font-family: Verdana; margin-left: 0px; margin-top: 0px; }
    #content { margin-left: 30px; font-size: .70em; padding-bottom: 2em; }
    A:link { color: #336699; font-weight: bold; text-decoration: underline; }
    A:visited { color: #6699cc; font-weight: bold; text-decoration: underline; }
    A:active { color: #336699; font-weight: bold; text-decoration: underline; }
    A:hover { color: cc3300; font-weight: bold; text-decoration: underline; }
    ...

    I will contact TRTH support to verify the problem.

Answers

  • To make it very clear: the protocol should be HTTPS, not HTTP. This issue has occured in the past. A workaround is to check the URL, and replace HTTP with HTTPS when required.

    See this query on this topic.

  • sd
    sd Newcomer

    Thank you Christian for your answer.

    In the C# SDK that I am using, how could I check the request URL, and then possibly change the URL and replace HTTP by HTTPS?

    So far, I see no url property in the request of anwhere else.

    Thanks!

    SD

  • I am seeing the same behavior 50% of the time. Also using a TickHistoryTimeAndSalesExtractionRequest. I can get valid data back sometimes. Other times I get the same html back. I'm using the C# SDK as well. I'd assume the HTTP vs HTTPS issue would be handled within the SDK.

  • The servers have an intermittent issue with the location URLs returned in the 202
    response. It is sometimes returned as HTTP instead of HTTPS. Using a location with HTTP will fail.

    Note: the issue does not depend on the type of data request you make.

    The current workaround is that you monitor
    the returned URLs and “add the s” to get consistent performance. A
    solution for the underlying problem is expected in the 11.2 release in
    September.

    Please accept our apologies for the inconvenience.

  • sd
    sd Newcomer

    Thanks Christian for the answer.

    How do I monitor the returned URLs and “add the s” in the C# SDK?

    I do not see any URL property.

    Stephane.

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @sd

    I found the code in Async Key Mechanisms in Managing Active/In-Flight Jobs section.

    var job = extractionsContext.ExtractWithNotesStart(extractionRequest);
    var handle = job.ToString();

    job = AsyncJob<ExtractionResult>.Parse(handle);
    job = extractionsContext.MonitorJob(job);

    From this code, we can change the job to string, modify the string, and then parse it back to the job.

    Therefore, I have created a function to change HTTP to HTTPS.

    static IAsyncJob<RawExtractionResult> FixProtocol(ref IAsyncJob<RawExtractionResult> result)
    {
    var handle = result.ToString();
    Console.WriteLine("handle: {0}", handle);
    if (handle.Contains("http://"))
    {
    handle = handle.Replace("http://", "https://");
    Console.WriteLine("new handle: {0}", handle);
    }
    return AsyncJob<RawExtractionResult>.Parse(handle);
    }

    I have applied it to the following code:

    //AsyncJob is in this namespace
    using ThomsonReuters.Dss.Api.Core.HttpOData;
    ...
    ...
    RawExtractionResult extractionResult = null;
    IAsyncJob<RawExtractionResult> resultTask = null;

    resultTask = extractionsContext.ExtractRawStart(extractionRequest);
    do
    {
    resultTask = extractionsContext.MonitorJob<RawExtractionResult>(FixProtocol(ref resultTask));
    if (resultTask.Status == ThomsonReuters.Dss.Api.Jobs.JobStatus.InProgress)
    {
    System.Threading.Thread.Sleep(2000);
    }
    } while (resultTask.Status != ThomsonReuters.Dss.Api.Jobs.JobStatus.Completed);

    extractionResult = resultTask.Result;

    DssStreamResponse streamResponse = extractionsContext.GetReadStream(extractionResult);

  • sd
    sd Newcomer

    Thank you very much! I will test this extensively on Monday but at first sight it seems to answer my issue.

    Have a great week end.

  • Following the reply given by jirapongse.phuriphanvichai here's the code that I had to add to fix the issue. Hope this helps any one

    IEnumerable<ExtractionRow> Extract(ExtractionRequestBase extractionRequest)
    {
    ExtractionResult extractionResult = null;
    IAsyncJob<ExtractionResult> resultTask = null;


    resultTask = extractionsContext.ExtractWithNotesStart(extractionRequest);


    do
    {
    if (resultTask.MonitorUri.Scheme .Equals("http", StringComparison.InvariantCultureIgnoreCase))
    resultTask = FixProtocol(resultTask);

    resultTask = extractionsContext.MonitorJob(resultTask);


    if (resultTask.Status == Dss.Api.Jobs.JobStatus.InProgress)
    {
    System.Threading.Thread.Sleep(5000);
    }


    } while (resultTask.Status != Dss.Api.Jobs.JobStatus.Completed);


    extractionResult = resultTask.Result;


    return extractionResult.Contents;
    }


    static IAsyncJob<ExtractionResult> FixProtocol(IAsyncJob<ExtractionResult> result)
    {
    var handle = result.ToString();


    handle = handle.Replace("http://", "https://");


    return AsyncJob<ExtractionResult>.Parse(handle);
    }
  • I would refrain from mutating a job handle and simply create a new AsyncJob:

    public static IAsyncJob<TResult> ChangeMonitorUrlToUseHttps<TResult>(this IAsyncJob<TResult> job)
    {
    var url = job.MonitorUri;
    if (url.Scheme == "https")
    return job;

    var urlBuilder = new UriBuilder(url) {Scheme = "https"};
    var newJob = new AsyncJob<TResult>(urlBuilder.Uri, job.Status, job.Progress, job.Result);
    return newJob;
    }