question

Upvote
Accepted
16 1 2 5

Invalid response issue

Hello,

When using the below code, I have in some occasions a bug in :

resultTask = ExtractionsContext.ExtractRawAsync(request);

result = await resultTask;

with an exception thrown with ex.message=""Invalid response, could not parse the following response: \r\n\r\n\r\n<html>\r\n\r\n <head id=\"Head1\">, ...." the message being to too long to be shown.

It seems that the server is not responding in a proper way anymore. Is there a maximum number of concurent ExtractRawAsync that could cause the issue, or anything else that could explain this?

Thank you for your help,

SD

The code:

var condition = new TickHistoryTimeAndSalesCondition

{

ReportDateRangeType = ReportDateRangeType.Range,

QueryStartDate = startDate,

QueryEndDate = endDate,

ApplyCorrectionsAndCancellations = false,

ExtractBy = TickHistoryExtractByMode.Ric,

MessageTimeStampIn = TickHistoryTimeOptions.GmtUtc,

SortBy = TickHistorySort.SingleByRic,

DisplaySourceRIC = true

};

var rics = InstrumentManager.InstrumentList.Select(x => x.RDTHRic).ToArray();

var instrumentIdentifiers = rics.Select(x => InstrumentIdentifier.Create(IdentifierType.Ric, x)).ToArray();

var indentifierList = new InstrumentIdentifierList { InstrumentIdentifiers = instrumentIdentifiers };

var request = new TickHistoryTimeAndSalesExtractionRequest

{

Condition = condition,

ContentFieldNames = FIELDS,

IdentifierList = indentifierList

};

Task<RawExtractionResult> resultTask;

RawExtractionResult result;

try

{

resultTask = ExtractionsContext.ExtractRawAsync(request);

result = await resultTask;

}

catch (Exception ex)

{

var message = ex.Message.Replace("\n", "").Replace("\r", "").Substring(0, 25); InstrumentManager.EventsShowMsg("Error in task: " + message, false);

return;

}

tick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
55.8k 144 45 65

I think that the Location field in the header in HTTP response has been changed.

Location: http://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x05d1ac85e20b3016')

The protocol is http, not https.

If I use this URL to check the status of the request, it will return:

<html>
    <head id="Head1">
        <link rel="alternate" type="text/xml" href="/DatascopeApi/v1/ExtractionService.asmx?disco" />
        <style type="text/css">
    
		BODY { color: #000000; background-color: white; font-family: Verdana; margin-left: 0px; margin-top: 0px; }
		#content { margin-left: 30px; font-size: .70em; padding-bottom: 2em; }
		A:link { color: #336699; font-weight: bold; text-decoration: underline; }
		A:visited { color: #6699cc; font-weight: bold; text-decoration: underline; }
		A:active { color: #336699; font-weight: bold; text-decoration: underline; }
		A:hover { color: cc3300; font-weight: bold; text-decoration: underline; }
...

I will contact TRTH support to verify the problem.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

To make it very clear: the protocol should be HTTPS, not HTTP. This issue has occured in the past. A workaround is to check the URL, and replace HTTP with HTTPS when required.

See this query on this topic.

Upvotes
16 1 2 5

Thank you Christian for your answer.

In the C# SDK that I am using, how could I check the request URL, and then possibly change the URL and replace HTTP by HTTPS?

So far, I see no url property in the request of anwhere else.

Thanks!

SD

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
5 3 0 9

I am seeing the same behavior 50% of the time. Also using a TickHistoryTimeAndSalesExtractionRequest. I can get valid data back sometimes. Other times I get the same html back. I'm using the C# SDK as well. I'd assume the HTTP vs HTTPS issue would be handled within the SDK.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
13.7k 26 8 12

The servers have an intermittent issue with the location URLs returned in the 202 response. It is sometimes returned as HTTP instead of HTTPS. Using a location with HTTP will fail.

Note: the issue does not depend on the type of data request you make.

The current workaround is that you monitor the returned URLs and “add the s” to get consistent performance. A solution for the underlying problem is expected in the 11.2 release in September.

Please accept our apologies for the inconvenience.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16 1 2 5

Thanks Christian for the answer.

How do I monitor the returned URLs and “add the s” in the C# SDK?

I do not see any URL property.

Stephane.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
55.8k 144 45 65

@sd

I found the code in Async Key Mechanisms in Managing Active/In-Flight Jobs section.

var job = extractionsContext.ExtractWithNotesStart(extractionRequest);
var handle = job.ToString();

job = AsyncJob<ExtractionResult>.Parse(handle);
job = extractionsContext.MonitorJob(job);

From this code, we can change the job to string, modify the string, and then parse it back to the job.

Therefore, I have created a function to change HTTP to HTTPS.

static IAsyncJob<RawExtractionResult> FixProtocol(ref IAsyncJob<RawExtractionResult> result)
{           
    var handle = result.ToString();
    Console.WriteLine("handle: {0}", handle);
    if (handle.Contains("http://"))
    {
        handle = handle.Replace("http://", "https://");
        Console.WriteLine("new handle: {0}", handle);
    }
  return AsyncJob<RawExtractionResult>.Parse(handle);          
}

I have applied it to the following code:

//AsyncJob is in this namespace
using ThomsonReuters.Dss.Api.Core.HttpOData;
...
...
RawExtractionResult extractionResult = null;
IAsyncJob<RawExtractionResult> resultTask = null;

resultTask = extractionsContext.ExtractRawStart(extractionRequest);
do
{
    resultTask = extractionsContext.MonitorJob<RawExtractionResult>(FixProtocol(ref resultTask)); 
    if (resultTask.Status == ThomsonReuters.Dss.Api.Jobs.JobStatus.InProgress)
    {
        System.Threading.Thread.Sleep(2000);
    }
} while (resultTask.Status != ThomsonReuters.Dss.Api.Jobs.JobStatus.Completed);

extractionResult = resultTask.Result;

DssStreamResponse streamResponse = extractionsContext.GetReadStream(extractionResult);

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16 1 2 5

Thank you very much! I will test this extensively on Monday but at first sight it seems to answer my issue.

Have a great week end.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
31 0 0 1
Following the reply given by jirapongse.phuriphanvichai here's the code that I had to add to fix the issue. Hope this helps any one

 IEnumerable<ExtractionRow> Extract(ExtractionRequestBase extractionRequest)
        {
            ExtractionResult extractionResult = null;
            IAsyncJob<ExtractionResult> resultTask = null;


            resultTask = extractionsContext.ExtractWithNotesStart(extractionRequest);


            do
            {
                if (resultTask.MonitorUri.Scheme .Equals("http", StringComparison.InvariantCultureIgnoreCase))
                    resultTask = FixProtocol(resultTask);
                
                resultTask = extractionsContext.MonitorJob(resultTask);


                if (resultTask.Status == Dss.Api.Jobs.JobStatus.InProgress)
                {
                    System.Threading.Thread.Sleep(5000);
                }


            } while (resultTask.Status != Dss.Api.Jobs.JobStatus.Completed);


            extractionResult = resultTask.Result;


            return extractionResult.Contents;
        }


        static IAsyncJob<ExtractionResult> FixProtocol(IAsyncJob<ExtractionResult> result)
        {
            var handle = result.ToString();


            handle = handle.Replace("http://", "https://");


            return AsyncJob<ExtractionResult>.Parse(handle);
        }
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Thank you for sharing :-)

Upvotes
31 0 0 1

I would refrain from mutating a job handle and simply create a new AsyncJob:

public static IAsyncJob<TResult> ChangeMonitorUrlToUseHttps<TResult>(this IAsyncJob<TResult> job)
{
    var url = job.MonitorUri;
    if (url.Scheme == "https")
        return job;

    var urlBuilder = new UriBuilder(url) {Scheme = "https"};
    var newJob = new AsyncJob<TResult>(urlBuilder.Uri, job.Status, job.Progress, job.Result);
    return newJob;
}
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.