Incorrect prices in Update events after closing and opening streams of different intervals

cory.schmidt.1

Using the latest release of the Refinitiv Data Library, we're seeing an issue where the Update values for a 1-minute stream are being populated with the latest Insert values from the Daily stream.

Screenshot from MetaStock in the comments.

The steps are to open a chart of one instrument (AAPL.O) with a Daily interval, change the interval to 1-minute (under the hood the Daily stream is closed and a new 1-minute stream is opened), then open a new chart with a different instrument with Daily interval. After the second chart is opened the updates to the first chart will contain high/low/volume from the last Daily Insert record of the first instrument.

The following code illustrates the problem:

    static void Main()
    {
        try
        {
            ISession session = DesktopSession.Definition().AppKey("xxxxxxxxxxxxxxxxxxx")
                .GetSession().OnState((state, msg, s) => Console.WriteLine($"{DateTime.Now}: State: {state}. {msg}"))
                .OnEvent((eventCode, msg, s) => Console.WriteLine($"{DateTime.Now}: Event: {eventCode}. {msg}"));
            session.Open();

            // Open a stream with an interval of 1 day
            var stream1 = Summaries
                .Definition("AAPL.O")
                .Fields("OPEN_PRC", "HIGH_1", "LOW_1", "TRDPRC_1", "ACVOL_UNS")
                .Count(10)
                .Interval(Summaries.Interval.P1D)
                .GetStream(session);

            bool isSecondInsert = false;
            bool waitForFirstUpdate = true;
            (double Open, double High, double Low, double Close) lastDailyInsert = (double.NaN, double.NaN, double.NaN, double.NaN);
            stream1
                .OnInsert((data, t) =>
                {
                    if (!isSecondInsert)
                    {
                        isSecondInsert = true;
                    }
                    else
                    {
                        var row = data.Table.Select().FirstOrDefault();

                        lastDailyInsert.Open = GetDataRowValueAsDouble(row, "OPEN_PRC");
                        lastDailyInsert.High = GetDataRowValueAsDouble(row, "HIGH_1");
                        lastDailyInsert.Low = GetDataRowValueAsDouble(row, "LOW_1");
                        lastDailyInsert.Close = GetDataRowValueAsDouble(row, "TRDPRC_1");

                        Display.FormatTable(data.Table, 10, 10);
                    }
                })
                .OnUpdate((data, t) =>
                {
                    waitForFirstUpdate = false;
                })
                .OnError((data, t) => { })
                .Open();

            // I'm not sure if we need an actual update to come in or if we just need to wait some time, but this is required to duplicate the issue.
            while (waitForFirstUpdate) { }

            // Close the stream with an interval of 1 day
            stream1.Close();

            // Open a stream with an interval of 1 min
            var stream2 = Summaries
                .Definition("AAPL.O")
                .Fields("OPEN_PRC", "HIGH_1", "LOW_1", "TRDPRC_1", "ACVOL_UNS")
                .Count(10)
                .Interval(Summaries.Interval.PT1M)
                .GetStream(session);

            stream2
                .OnInsert((data, t) => { })
                .OnUpdate((data, t) =>
                {
                    var row = data.Table.Select().FirstOrDefault();

                    var open = GetDataRowValueAsDouble(row, "OPEN_PRC");
                    var high = GetDataRowValueAsDouble(row, "HIGH_1");
                    var low = GetDataRowValueAsDouble(row, "LOW_1");
                    var close = GetDataRowValueAsDouble(row, "TRDPRC_1");

                    // Only write out the table if the high and low values from the update match the last insert from stream1
                    if (high == lastDailyInsert.High && low == lastDailyInsert.Low)
                    {
                        Display.FormatTable(data.Table, 10, 10);
                    }
                })
                .OnError((data, t) => { })
                .Open();

            // Open a stream with an interval of 1 day with a different ric than stream1 and stream2
            // This is required to reproduce this issue.
            var stream3 = Summaries
                .Definition("IBM.N")
                .Fields("OPEN_PRC", "HIGH_1", "LOW_1", "TRDPRC_1", "ACVOL_UNS")
                .Count(10)
                .Interval(Summaries.Interval.P1D)
                .GetStream(session);

            stream3
                .OnInsert((data, t) => { })
                .OnUpdate((data, t) => { })
                .OnError((data, t) => { })
                .Open();

            Console.ReadLine();
        }
        catch (Exception e)
        {
            Console.WriteLine($"\n**************\nFailed to execute: {e.Message}\n{e.InnerException}\n***************");
        }

The key is the OnUpdate handler for stream2. You'll notice that it only outputs to the console if the high and low of the 1-min update record match the high and low of the last Daily insert record, which is the crux of the issue.

Find more posts tagged with

refinitiv-data-platform-libraries

streaming-data

#content

Accepted answers

All comments

cory.schmidt.1

Screenshot from MetaStock...

nick.zincone

Hi @cory.schmidt.1

Can you explain the logic within stream2 update? When an update occurs, specifically a trade, you will typically get the price of the trade. If that price is a new high or a new low, you will see either one occur. However, it appears you are anticipating that the trade that occurs within the update may cause both the high and low to be sent. Can you confirm?

cory.schmidt.1

That's correct. We anticipate that we would be retrieving the high/low/volume for the current 1-minute bar in stream 2. If that is an incorrect assumption then we're happy to be corrected. :-)

nick.zincone

Hi @cory.schmidt.1

Thanks for the clarification. When a trade occurs, you should never really receive a high and low in the same update. Its either one or the other but not both. I mean it may be possible if there have been no trades reported for a given bar and when the first trade comes in, that trade would update the low and high. However, the state of a bar is based on the historical events and the historical bar for this data would likely have both a high and low recorded already and would likely be different.

I'm not a data person so it might be better if you want to understand the conditions for such a thing by contacting the helpdesk - they can bring in a content expert to explain.

Hope this helps :-)

cory.schmidt.1

That makes sense. We just need to look at the columns returned in the table in the Update event and only pull values from those.

Thanks for the assistance!

cory.schmidt.1

The plot thickens...

We modified the code to iterate through the columns that were returned from the table instead of pulling values from the hard-coded column names. It turns out that in the specific scenario we provided, requesting AAPL.O Daily, then closing the stream, then requesting AAPL.O 1-min, leaving the stream open, then requesting IBM.N Daily, we get one Update event on the AAPL.O 1-min stream with High, Low, Close, and Volume, all with values from the Daily table...

So the original code was misleading, but the problem isn't that we're pulling price values from the table that we aren't meant to, it's that we're given incorrect values in the table in the Update event.

cory.schmidt.1

Here is the updated code...

stream2
    .OnInsert((data, t) => { })
    .OnUpdate((data, t) =>
    {
        var open = double.NaN;
        var high = double.NaN;
        var low = double.NaN;
        var close = double.NaN;

        var row = data.Table.Select().FirstOrDefault();
        foreach (DataColumn column in data.Table.Columns)
        {
            var value = GetDataRowValueAsDouble(row, column.ColumnName);
            switch (column.ColumnName)
            {
                case "OPEN_PRC":
                    open = value;
                    break;
                case "HIGH_1":
                    high = value;
                    break;
                case "LOW_1":
                    low = value;
                    break;
                case "TRDPRC_1":
                    close = value;
                    break;
            }
        }

        if (high == lastDailyInsert.High && low == lastDailyInsert.Low)
        {
            Display.FormatTable(data.Table, 10, 10);
        }
    })
    .OnError((data, t) => { })
    .Open();

nick.zincone

Hi @cory.schmidt.1

Thanks for the clarification. I just tried a quick test and also observed suspicious data:
The above event doesn't look right to me. I then waited for time to pass and capture the recorded historical bar at this point and this is what it should be:

The high and low are clearly incorrect in the first screenshot.

This appears to only occur upon startup - likely a result of the incorrect rules not being filtered out after the data blend. At first glance, this likely would happen without the need to go through the open a daily stream, close, open a 2nd and a 3rd. I did try a quick test by simply opening a 1-minute stream only - however, I didn't see any odd events. The market is closed so I can try when there is more activity. Just curious, did you only see this result with the conditions you tried? Did you happen to try just by opening a single 1-minute stream?

cory.schmidt.1

When we go through the steps I mentioned it happens every time. We had seen it at various times while testing so I don't know if there were different ways to cause it the other times or if we had done the same steps and hadn't noted them.

cory.schmidt.1

This issue seems to have been addressed in build 1.0.4 of Refinitiv Data Library nuget packages.

EXPLORE OUR SITES