Instrument validation fails for long RICS and truncates RIC to 40 characters

atappis

Hi,

I am attempting to validate a set of instruments prior to executing a TickHIstoryTimeAndSalesRequest. I've ran into an issue whereby in one of my tests which includes a long invalid RIC, the validation appears not to be working.

The list of RICs I am testing with is: ["BARC.L", "$3SK\"&", "ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGDFSFGDFSJKTYURETAERT", "VOD.L"] where I expect only BT.L and VOD.L to be deemed as valid.

I use the following code:

var context = new ExtractionsContext(new Uri(uri), sessionToken);
DataTable instrumentsTable = GetInstrumentListFromDatabase();
var identifierList = new List<InstrumentIdentifier>();
foreach (DataRow row in instrumentsTable.Rows)
{
    identifierList.Add(new InstrumentIdentifier()
    {
        Identifier = (string)row["Identifier"],
        IdentifierType = IdentifierType.Ric
    });
}
var validatedIdentifiers = context.InstrumentListOperations.ValidateIdentifiers(identifierList, false);
var validIds = validatedIdentifiers.ValidatedInstruments.Select(i => i.Identifier).ToList();
var jsonValidInstruments = JsonConvert.SerializeObject(validIds);
LogInfo("Valid Identifiers found", $"Number of valid instruments: {validIds.Count}, List of valid Instruments {jsonValidInstruments}");

The log message i see is:

Number of valid instruments: 3, List of valid Instruments ["BARC.L","ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGD","VOD.L"]

So the long RIC to be returned as VALID from the ValidateIdentifiers() API call and it has also been trimmed to 40 characters.

I then continue with this code:

var ids = identifierList.Select(i => i.Identifier).ToList();
var invalidIds = ids.Except(validIds).ToList();
if (invalidIds.Any())
{
    var jsonInalidInstruments = JsonConvert.SerializeObject(invalidIds);
    LogInfo("Invalid Identifiers found", $"Number of invalid identifiers: {invalidIds.Count()}, List of invalid instruments {jsonInalidInstruments}");
}

Which results in the following log entry:

Number of invalid identifiers: 2, List of invalid identifiers: ["$3SK\"&","ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGDFSFGDFSJKTYURETAERT"]

The problem here is because the long RIC before and after is different, the Except() operation fails to match on RIC.

So in summary 2 issues:

• The clearly invalid RIC is being returned as VALID
• The RIC returned does not match the RIC that was sent i.e. appears to be truncated to 40 chars

veerapath.rungruengrayubkul

Hi @atappis,

I have found that the "Allow Import of Unsupported Instruments into Instrument Lists" in the User Preferences is related to this issue.

If this preference is selected, the ValidateIdentifiers returns 3 validated instruments with Status": "NotFound" for "ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGD".

Please verify that this preference is not selected to get the expected 2 ValidateIdentifiers.

Below is the JSON response.

 {
    "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ThomsonReuters.Dss.Api.Extractions.SubjectLists.InstrumentsValidateIdentifiersResult",
    "ValidatedInstruments": [
        {
            "Identifier": "BARC.L",
            "IdentifierType": "Ric",
            "Source": "LSE",
            "Key": "VjF8MHgwMDAzZGQwMDEzNzUyYzk0fDB4MDAwM2RjMDAzNDdjOGExZHxMU0V8RVFRVXxFUVRZfE9EU0h8RXx8QkFSQy5MfDA2OTE",
            "Description": "BARCLAYS ORD",
            "InstrumentType": "EquityQuote",
            "Status": "Valid"
        },
        {
            "Identifier": "ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGD",
            "IdentifierType": "Ric",
            "Source": "",
            "Key": "VjF8MHgwMDAwMDAwMDAwMDAwMDAwfDB4MDAwMDAwMDAwMDAwMDAwMHx8fHx8fHx8",
            "Description": "",
            "InstrumentType": "MortAggregate",
            "Status": "NotFound"
        },
        {
            "Identifier": "VOD.L",
            "IdentifierType": "Ric",
            "Source": "LSE",
            "Key": "VjF8MHgwMDAzZGQwMDE0OGU3NDMwfDB4MDAwM2RjMDAzNDdjOTEwNnxMU0V8RVFRVXxFUVRZfE9EU0h8RXx8Vk9ELkx8MDY5MQ",
            "Description": "VODAFONE GROUP ORD",
            "InstrumentType": "EquityQuote",
            "Status": "Valid"
        }
    ],
    "ValidationResult": {
        "ValidInstrumentCount": 3,
        "OpenAccessSegments": [],
        "StandardSegments": [
            {
                "Code": "E",
                "Description": "Equity",
                "Count": 2
            }
        ],
        "ValidationDuplicates": [],
        "Messages": [
            {
                "Severity": "Warning",
                "Message": "Invalid characters found in Identifier field of instrument 2 (identifier='$3SK\"&' type='Ric').  Instrument skipped."
            },
            {
                "Severity": "Info",
                "Message": "RIC, ASFASDVFAECTGEDGFDFGHRTSY5RETGDFBVGCDVGD (not found)"
            }
        ]
    }
}

atappis

Hi,

Thanks for your reply and for pointing me in the right direction. I've changed my call to validate to use a version that accepts validation options as an input as follows:

var options = new InstrumentListValidationOptions()
{
    AllowDuplicateInstruments = false,
    AllowInactiveInstruments = false,
    AllowUnsupportedInstruments = false
};
var validatedIdentifiers = context.InstrumentListOperations.ValidateIdentifiersWithOptions(identifierList, options);

I am now receiving the expected results

This still does not explain why the validated results are being returned with the RIC truncated to 40 characters rather than returning to me exactly what I had submitted which I believe is either a bug or a limitation.

veerapath.rungruengrayubkul

Hi @atappis,

I have been confirmed that the REST API allows a length of 64 characters for identifier values but the other parts of DSS limit the identifier values at 40 characters, so an identifier having length between 40 and 64 characters will be truncated. This discrepancy issue should be resolved. For tracking purpose, please submit a new query to TRTH support via Get Support, so that you will get updates.

As a work around you should limit the instrument id values to 40 characters.

atappis

Yes thanks. Marked the answer.