question

Upvotes
Accepted
6 1 2 4

Job deserialization bug?

Hi,

I run TickHistoryTimeAndSales on demand extractions using the .net API. My pattern is as follows:

  1. Validate instruments using context.InstrumentListOperations.ValidateIdentifiers()
  2. Send request using var job = context.ExtractRawStart(request)
  3. Persist the job to a database using job.ToString()
  4. In a separate process re-hydrate the job using var job = (IAsyncJob<RawExtractionResult>)AsyncJob<RawExtractionResult>.Parse(jobString);
  5. Monitor the job status using job = context.MonitorJob(job);
  6. Download the results

The problem I am facing is that sometimes identifiers pass validation and then may not return results due to permissions or some other reason. In this case at step 2, the job is returned with a Completed status. In this case the MonitorUri property is not set and the serialized job looks something like this:

ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.RawExtractionResult, ThomsonReuters.Dss.RestApi.Client, Version=12.1.509.0, Culture=neutral, PublicKeyToken=c1b530b4d66eacdd||Completed|100

The problem occurs in step 4 when i try to parse the job object from this string it fails with a FormatException complaining about the MonitorUri property.

The following job string deserializes fine:

ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.RawExtractionResult, ThomsonReuters.Dss.RestApi.Client, Version=12.1.509.0, Culture=neutral, PublicKeyToken=c1b530b4d66eacdd|https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x066eca4c0a102260')|InProgress|0

dss-rest-apitick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
106 1 2 1

It is definitely a bug – so I have added DSS-30446 to cover the fact and get it fixed in our next release coming in January.

For the time being, is there any local value to persist a Job if it has failed out-of-hand on the submission? I am certainly going to fix it either way, I just offer that as the reason it was overlooked and not found to date by any other users.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
21.9k 58 14 21

@atappis: Does it work when you don't persist the job in the database? The first one shows that the job is already complete and does not have a monitor URL property.

An isolated code demonstrating this issue will help us identify the issue.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
13.7k 26 8 12

@atappis,

To add to Gurpreet's answer:

The fact that an identifier passes validation only means that the instrument identifier is valid, it does not imply that data will be returned by a call for this instrument, nor does it mean that your account is entitled to request specific data for it. In other words, instrument validation does not mean that a completed data request will return results, the set could be empty. In that case the request will finish with status complete, but the monitor URI will not be set.

This eventuality must be taken into account by the application code.

And yes, a code sample demonstrating the issue you face would help us identify the problem.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
6 1 2 4

Sorry if I wasn't clear. I have no problem with the validation behaviour. My problem is that your object serialization job.ToString() creates something that fails your deserialization. The code is easy to derive from my example above.

// can be created by calling job.ToString();
var jobString = "ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.RawExtractionResult, ThomsonReuters.Dss.RestApi.Client, Version=12.1.509.0, Culture=neutral, PublicKeyToken=c1b530b4d66eacdd||Completed|100";
var job = (IAsyncJob<RawExtractionResult>)AsyncJob<RawExtractionResult>.Parse(jobString);

This code fails with a FormatException because MonitorUri is blank. My point is your deserialization routine should handle this i.e. your serialization generates something that cannot be deserialized.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Thank you, now I get it. I escalated this to the dev team.

Upvotes
6 1 2 4

Merely in separating processes/concerns whereby one component is responsible submitting a bunch of requests and persisting the job handles onto a queue and another (multi-threaded/multi-process) component is responsible for monitoring the jobs and downloading results. We'd like the business logic and error handling of the Completed status to be in one place i.e. the latter of the 2 components.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.