question

Upvotes
Accepted
atappis avatar image
6 ●1 ●2 ●4

Instrument validation fails for long RICS and truncates RIC to 40 characters

Hi,

I am attempting to validate a set of instruments prior to executing a TickHIstoryTimeAndSalesRequest. I've ran into an issue whereby in one of my tests which includes a long invalid RIC, the validation appears not to be working.

The list of RICs I am testing with is: ["BARC.L", "$3SK\"&", "ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGDFSFGDFSJKTYURETAERT", "VOD.L"] where I expect only BT.L and VOD.L to be deemed as valid.

I use the following code:

var context = new ExtractionsContext(new Uri(uri), sessionToken);
DataTable instrumentsTable = GetInstrumentListFromDatabase();
var identifierList = new List<InstrumentIdentifier>();
foreach (DataRow row in instrumentsTable.Rows)
{
    identifierList.Add(new InstrumentIdentifier()
    {
        Identifier = (string)row["Identifier"],
        IdentifierType = IdentifierType.Ric
    });
}
var validatedIdentifiers = context.InstrumentListOperations.ValidateIdentifiers(identifierList, false);
var validIds = validatedIdentifiers.ValidatedInstruments.Select(i => i.Identifier).ToList();
var jsonValidInstruments = JsonConvert.SerializeObject(validIds);
LogInfo("Valid Identifiers found", $"Number of valid instruments: {validIds.Count}, List of valid Instruments {jsonValidInstruments}");

The log message i see is:

Number of valid instruments: 3, List of valid Instruments ["BARC.L","ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGD","VOD.L"]

So the long RIC to be returned as VALID from the ValidateIdentifiers() API call and it has also been trimmed to 40 characters.

I then continue with this code:

var ids = identifierList.Select(i => i.Identifier).ToList();
var invalidIds = ids.Except(validIds).ToList();
if (invalidIds.Any())
{
    var jsonInalidInstruments = JsonConvert.SerializeObject(invalidIds);
    LogInfo("Invalid Identifiers found", $"Number of invalid identifiers: {invalidIds.Count()}, List of invalid instruments {jsonInalidInstruments}");
}

Which results in the following log entry:

Number of invalid identifiers: 2, List of invalid identifiers: ["$3SK\"&","ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGDFSFGDFSJKTYURETAERT"]

The problem here is because the long RIC before and after is different, the Except() operation fails to match on RIC.

So in summary 2 issues:

  • The clearly invalid RIC is being returned as VALID
  • The RIC returned does not match the RIC that was sent i.e. appears to be truncated to 40 chars
tick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

veerapath.rungruengrayubkul avatar image   veerapath.rungruengrayubkul

@atappis

Hi,

Are any of the replies below satisfactory in resolving your query?

If yes please click the 'Accept' text next to the most appropriate reply. This will guide all community members who have a similar question.

Otherwise please post again offering further insight into your question.

Thanks,

AHS

4 Answers

· Write an Answer
Upvotes
Accepted
veerapath.rungruengrayubkul avatar image
11.3k ●25 ●9 ●14

Hi @atappis,

I have found that the "Allow Import of Unsupported Instruments into Instrument Lists" in the User Preferences is related to this issue.

If this preference is selected, the ValidateIdentifiers returns 3 validated instruments with Status": "NotFound" for "ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGD".

Please verify that this preference is not selected to get the expected 2 ValidateIdentifiers.

Below is the JSON response.

 {
    "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ThomsonReuters.Dss.Api.Extractions.SubjectLists.InstrumentsValidateIdentifiersResult",
    "ValidatedInstruments": [
        {
            "Identifier": "BARC.L",
            "IdentifierType": "Ric",
            "Source": "LSE",
            "Key": "VjF8MHgwMDAzZGQwMDEzNzUyYzk0fDB4MDAwM2RjMDAzNDdjOGExZHxMU0V8RVFRVXxFUVRZfE9EU0h8RXx8QkFSQy5MfDA2OTE",
            "Description": "BARCLAYS ORD",
            "InstrumentType": "EquityQuote",
            "Status": "Valid"
        },
        {
            "Identifier": "ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGD",
            "IdentifierType": "Ric",
            "Source": "",
            "Key": "VjF8MHgwMDAwMDAwMDAwMDAwMDAwfDB4MDAwMDAwMDAwMDAwMDAwMHx8fHx8fHx8",
            "Description": "",
            "InstrumentType": "MortAggregate",
            "Status": "NotFound"
        },
        {
            "Identifier": "VOD.L",
            "IdentifierType": "Ric",
            "Source": "LSE",
            "Key": "VjF8MHgwMDAzZGQwMDE0OGU3NDMwfDB4MDAwM2RjMDAzNDdjOTEwNnxMU0V8RVFRVXxFUVRZfE9EU0h8RXx8Vk9ELkx8MDY5MQ",
            "Description": "VODAFONE GROUP ORD",
            "InstrumentType": "EquityQuote",
            "Status": "Valid"
        }
    ],
    "ValidationResult": {
        "ValidInstrumentCount": 3,
        "OpenAccessSegments": [],
        "StandardSegments": [
            {
                "Code": "E",
                "Description": "Equity",
                "Count": 2
            }
        ],
        "ValidationDuplicates": [],
        "Messages": [
            {
                "Severity": "Warning",
                "Message": "Invalid characters found in Identifier field of instrument 2 (identifier='$3SK\"&' type='Ric').  Instrument skipped."
            },
            {
                "Severity": "Info",
                "Message": "RIC, ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGD (not found)"
            }
        ]
    }
}

unsupportedinstrument.png (15.9 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
atappis avatar image
6 ●1 ●2 ●4

Hi,

Thanks for your reply and for pointing me in the right direction. I've changed my call to validate to use a version that accepts validation options as an input as follows:

var options = new InstrumentListValidationOptions()
{
    AllowDuplicateInstruments = false,
    AllowInactiveInstruments = false,
    AllowUnsupportedInstruments = false
};
var validatedIdentifiers = context.InstrumentListOperations.ValidateIdentifiersWithOptions(identifierList, options);

I am now receiving the expected results

This still does not explain why the validated results are being returned with the RIC truncated to 40 characters rather than returning to me exactly what I had submitted which I believe is either a bug or a limitation.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
veerapath.rungruengrayubkul avatar image
11.3k ●25 ●9 ●14

Hi @atappis,

I have been confirmed that the REST API allows a length of 64 characters for identifier values but the other parts of DSS limit the identifier values at 40 characters, so an identifier having length between 40 and 64 characters will be truncated. This discrepancy issue should be resolved. For tracking purpose, please submit a new query to TRTH support via Get Support, so that you will get updates.

As a work around you should limit the instrument id values to 40 characters.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
atappis avatar image
6 ●1 ●2 ●4

Yes thanks. Marked the answer.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.