Job deserialization bug?
Hi,
I run TickHistoryTimeAndSales on demand extractions using the .net API. My pattern is as follows:
- Validate instruments using context.InstrumentListOperations.ValidateIdentifiers()
- Send request using var job = context.ExtractRawStart(request)
- Persist the job to a database using job.ToString()
- In a separate process re-hydrate the job using var job = (IAsyncJob<RawExtractionResult>)AsyncJob<RawExtractionResult>.Parse(jobString);
- Monitor the job status using job = context.MonitorJob(job);
- Download the results
The problem I am facing is that sometimes identifiers pass validation and then may not return results due to permissions or some other reason. In this case at step 2, the job is returned with a Completed status. In this case the MonitorUri property is not set and the serialized job looks something like this:
ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.RawExtractionResult, ThomsonReuters.Dss.RestApi.Client, Version=12.1.509.0, Culture=neutral, PublicKeyToken=c1b530b4d66eacdd||Completed|100
The problem occurs in step 4 when i try to parse the job object from this string it fails with a FormatException complaining about the MonitorUri property.
The following job string deserializes fine:
ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.RawExtractionResult, ThomsonReuters.Dss.RestApi.Client, Version=12.1.509.0, Culture=neutral, PublicKeyToken=c1b530b4d66eacdd|https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x066eca4c0a102260')|InProgress|0
It is definitely a bug – so I have added DSS-30446 to cover the fact and get it fixed in our next release coming in January.
For the time being, is there any local value to persist a Job if it has failed out-of-hand on the submission? I am certainly going to fix it either way, I just offer that as the reason it was overlooked and not found to date by any other users.
To add to Gurpreet's answer:
The fact that an identifier passes validation only means that the instrument identifier is valid, it does not imply that data will be returned by a call for this instrument, nor does it mean that your account is entitled to request specific data for it. In other words, instrument validation does not mean that a completed data request will return results, the set could be empty. In that case the request will finish with status complete, but the monitor URI will not be set.
This eventuality must be taken into account by the application code.
And yes, a code sample demonstrating the issue you face would help us identify the problem.
Sorry if I wasn't clear. I have no problem with the validation behaviour. My problem is that your object serialization job.ToString() creates something that fails your deserialization. The code is easy to derive from my example above.
// can be created by calling job.ToString(); var jobString = "ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.RawExtractionResult, ThomsonReuters.Dss.RestApi.Client, Version=12.1.509.0, Culture=neutral, PublicKeyToken=c1b530b4d66eacdd||Completed|100"; var job = (IAsyncJob<RawExtractionResult>)AsyncJob<RawExtractionResult>.Parse(jobString);
This code fails with a FormatException because MonitorUri is blank. My point is your deserialization routine should handle this i.e. your serialization generates something that cannot be deserialized.
Merely in separating processes/concerns whereby one component is responsible submitting a bunch of requests and persisting the job handles onto a queue and another (multi-threaded/multi-process) component is responsible for monitoring the jobs and downloading results. We'd like the business logic and error handling of the Completed status to be in one place i.e. the latter of the 2 components.